Could scale-up of parenting programmes improve child disruptive behaviour and reduce social inequalities? Using individual participant data meta-analysis to establish for whom programmes are effective and cost-effective

Moderators: understanding for whom interventions are effective and their ‘equity’ implications

Given a strong body of evidence showing beneficial main effects of parenting interventions on child outcomes in various trial populations, it is important to understand any effect of heterogeneity between and within study populations. Investigations that aim to establish whether or not there are differential effects for different subgroups in the population are referred to as moderator, effect modifier or subgroup analyses. Importantly, we distinguish moderator from ‘predictor’ analyses, which we define as those that do not examine interaction effects, but instead analyse predictors of outcome in a treated group only, making no comparison with change in the control group. This is a significant distinction, as without this comparison it is not possible to know whether or not any differential effects are related to the intervention per se, or if they merely reflect naturally occurring subgroup differences in prognosis. Understanding treatment effect heterogeneity is vital for a number of reasons, including (1) assessing equity effects of interventions, and whether or not they work for those at greatest risk of poor outcomes, (2) ensuring that interventions are targeted appropriately, (3) understanding for whom intervention strategies may need to be improved or altered and (4) exploring possible differential intervention mechanisms.

Social and socioeconomic disadvantage

The wide dissemination of parenting interventions in the UK makes it urgent to determine the effectiveness of these interventions across a range of social groups. Early poor parenting is patterned by social and socioeconomic disadvantage and in the absence of intervention is predictive of child behaviour problems and diminished educational and health outcomes, which suggests that it is a key mechanism for perpetuating health and social inequalities across generations (e.g. UK birth cohort data, Ermisch8 and Kelly et al. 38). Prospective studies39,40 of high-risk samples support the hypothesis that poor parenting mediates intergenerational transmission of adverse child outcomes. Furthermore, UK cohort analyses suggest that social inequalities in both child and parent mental health appear to be widening over time. 41,42

These observational studies suggest that parenting may be an important mechanism in promoting inequalities and that, with current policies, inequalities in mental health do not appear to be lessening over time. Instead, intervention studies are needed to examine whether improving parenting will have an adverse or beneficial effect on inequalities in child outcomes. Despite a large number of high-quality trials and systematic reviews on the topic, conclusions from the literature are mixed on the question of whether or not there are differential benefits of parenting interventions by family social and socioeconomic disadvantage. A number of trials and systematic reviews have found weaker effects of parenting interventions for more disadvantaged families, including two of the largest meta-analyses of predictor effects. 43,44 On the other hand, other reviews draw more uncertain conclusions. 12,45 Some of the few individual trials testing moderator effects46–48 find that these parenting interventions are equally or sometimes more effective for the most disadvantaged families, which, without intervention, tend to do worse, suggesting the potential for reversing some of the poorer child outcomes associated with family poverty. It is worth noting that most trials have not used their data to ask these questions. However, this limitation aside, there are a number of methodological reasons for these conflicting results, and we argue (in Methodological limitations of current moderation literature) that such moderation questions cannot adequately be answered from individual trials or aggregate-level meta-analyses alone.

The qualitative literature on parents’ experiences can also help identify putative moderators. A systematic review by Kane et al. 49 found five qualitative studies of the views of (mainly disadvantaged) parents who had participated in parenting interventions; our searches found several more recent studies of the Incredible Years^® (IY) programme, two of which were embedded within a trial. 50,51 Barriers to uptake and success in intervention included stresses related to time pressure, financial pressure, the influence of antisocial neighbourhoods, reluctance to share problems with others and lack of support from family members. These findings show congruence between parents’ views of barriers and those factors drawn from the literature on moderators and risk factors for child disruptive behaviour. 52

Ethnicity

Few trials in the UK or elsewhere have been able to examine effects of parenting interventions on families from different ethnic backgrounds. This is vital in order to assess whether such services are likely to reduce or widen inequalities by ethnicity in child and maternal outcomes. 53 This is becoming increasingly important in the UK, and, for example, recent population data in London suggest that in some of the more disadvantaged boroughs one-third of children or more (over half in Tower Hamlets) belong to an ethnic minority. When there is evidence from other countries, mainly from the USA, the picture is quite mixed about whether or not there are differential effects of parenting interventions. Measuring a very wide range of parent and child outcomes, as well as parent engagement and satisfaction, Reid et al. 54 found surprisingly little evidence of differential effects of the IY parenting intervention by ethnicity in a predictor analysis; when there were differences by ethnicity, they tended towards greater engagement and uptake by some minority groups. On the other hand, much theoretical and prevention literature from the USA focuses on the need for interventions to be specially adapted for different ethnic groups. 55,56 Even leaving aside the controversial question of which is most effective,57 approaches involving substantial adaptation by culture would imply the need to run parenting groups that are separated by ethnicity, and this raises critical questions about what would be appropriate service delivery patterns for multiethnic UK inner cities. 58 There have been very few studies of outcome differences in parenting trials by ethnicity in the UK or other European countries, or qualitative studies alongside trials. One exception is the trial by Scott et al. ,59 conducted in a highly deprived London borough. They found considerable baseline differences in parenting practices by ethnicity, but, intriguingly, no ethnic differences in attendance, or in intervention effects on parenting skills. This trial therefore suggested that despite large initial differences in parenting style by ethnicity, parenting programmes apparently based on ‘Western’ family values are equally effective with ethnic minority parents, when sensitively delivered, using a programme with an underlying philosophy that is collaborative and parent centred. 60 A Manchester study of parents’ views of a similar programme, Triple P,61 suggested that Asian and African parents are most inclined to take up a parenting intervention, but that white and African Caribbean parents are somewhat less likely to do so.

Child characteristics: behavioural and emotional problems – age and gender

Relatively little is known about how child characteristics, such as age, gender and initial severity and comorbidity of problems, and parent characteristics, such as depression and parenting style, influence the effects of parenting interventions.

Parents’ clinical and parenting characteristics

Contextual factors

Relevant contextual factors that the literature suggests are likely to affect implementation and effectiveness of parenting interventions can be coded at the level of the trial, because they are the same for all families within the same trial. 75,76 These include type of service provider organisation, level of professional training of the staff delivering the intervention, level of attention to fidelity of implementation, university efficacy compared with ‘real-world’ service effectiveness setting, and geographic factors (e.g. UK vs. other countries, city vs. rural). It is largely unknown whether any reduction in effectiveness is better accounted for by family-level variables (e.g. income and lone parenthood) or, even after taking these into account, whether there are still contextual factor effects. As contextual factors are measured at the trial level, power will be limited for these analyses. In addition, variation between trials on these factors may be limited. For example, the IY parenting intervention was mainly delivered in towns, with few groups in rural areas.

Wider health benefits

Negative (i.e. harsh and inconsistent) parenting, and a lack of positive parenting (i.e. positive reinforcement and monitoring) are of particular importance for child well-being and quality of life; in prevention trials, in which many of the children show quite low levels of behaviour problems, these interventions impact public health by reducing levels of harsh or abusive parenting and family stress. This has been found in universal and selective prevention trials,32,87 and in studies of parents at high risk for abusive parenting. 88,89 The Triple P trial87 showed that widespread implementation of a similar parenting programme reduced admissions to hospital for abuse, measured by county-level indicators. Recent reviews confirm that even mildly harsh parenting is associated with harmful biological effects on children, including, for example, dysfunctional cortisol secretion patterns and raised C-reactive protein, which in turn are associated with increased cardiovascular disease and mortality. 90

Parental well-being (i.e. depression or stress) has been shown in some trials to be improved by parenting interventions, and this will be an important public health benefit to document. Parents with young children spend many or most of their waking hours caring for them, and qualitative studies suggest that failure to succeed in controlling child behaviour is a major source of lack of confidence and depressive cognitions. 51 Improving parent–child dynamics, including more positivity and improved communication, may contribute to parents’ sense of well-being and fulfilment in the parenting role. Some previous work suggests that parenting interventions designed to improve the parent–child relationship and children’s conduct problems also reduce parental symptoms of depression. 12

Although parenting interventions primarily aim to reduce children’s conduct problems, there is evidence to suggest that they may have a wider impact on children’s mental health, including children’s ADHD symptoms, ADHD being the most prevalent comorbidity of conduct problems. 91 Recent findings about the extent to which parenting interventions reduce ADHD symptoms in children are inconsistent, and may in part depend on the type of instrument used (e.g. Daley et al. ,92 Jones et al. 68 and Sonuga-Barke et al. 93). The extent to which parenting interventions designed for reducing conduct problems may benefit children’s emotional well-being, however, remains understudied. Symptoms of anxiety and depression are common in children with conduct problems. 94 Some studies evaluating the IY parenting intervention indicate effects of the intervention on reduced emotional problems in children (e.g. Herman et al. 95). Others, however, have failed to replicate these findings (e.g. Leijten et al. 96) or emotional problems are not measured. These inconsistent findings suggest that more thorough investigation is needed of the extent to which parenting interventions for conduct problems also impact children’s emotional problems.

Potential harmful effects of parenting interventions

As well as benefits, it is important to consider potential harms, especially as they are rarely studied in parenting intervention trials. When harms have been studied in psychosocial trials, they have mainly involved youth-focused interventions and have been defined as main effects in the unintended direction. 97,98 Recent systematic reviews of parenting interventions have not found evidence of harmful effects defined in this way. 12 Parents rarely report potentially harmful outcomes in qualitative studies; Morch et al. 51 mention none, despite interviewing many parents for whom intervention was not successful. A few parents in the study by Furlong and McGilloway50 were concerned about increased conflict with partners related to trying new parenting techniques and the lack of privacy in group interventions that discuss family problems. The Cochrane review by Furlong et al. 12 planned to examine two potential adverse effects, namely the burden on families in attending (e.g. child-care issues) and increased family conflict, but found no studies reporting these outcomes. Given this weak state of evidence on harms from parenting interventions, cautious, exploratory (hypothesis generating, rather than hypothesis driven) analyses are needed to see if there is evidence of main effects in the adverse direction.

Parent satisfaction data

Data on parents’ views are rarely presented in detail in trial reports or reviews12 and the rate of missing data is often high. These are available only for parents in the intervention group, and not the control group, but if numbers are sufficient, and instruments comparable, they could potentially be synthesised.

Design variables: trial level

Design variables: individual level

Baseline measures: individual family level

Baseline measures: trial level

Primary outcome: individual family level

Secondary outcomes and baseline moderators: individual family level

Parent satisfaction data

See Appendix 4 for full details. There were many missing data, at both trial and family levels. Five trials used comparable instruments, but with a high percentage of missing data (45%); hence these were not further analysed or discussed.

Specification of outcomes and (interaction) effect measures (PRISMA-IPD #13)

The primary objective of the moderation analyses was the evaluation of putative moderation effects. We use a single outcome measure: disruptive behaviour as measured by the ECBI-I scale (harmonised from the PACS data in those trials for which the PACS scale was used in place of ECBI-I). This is available at up to three time points:

• Baseline: all trials measured child conduct disorder pretreatment and this information is available in both the treated and control arms, although there are some missing data.

• T1: the first follow-up point for most of the trials. The time window for this measure is between 0 and 2 months post treatment, which is defined as the end of the intervention.

• T2: The second follow-up time point. The time window for this measure was between 4 and 7 months after the end of the intervention. In NL-BS and LON-HCA data are available for both treated and control participants, as the whole sample was followed up twice. In other trials, when there is a second follow-up, data are available only for the active arm. T2 data were used only to improve imputation in the multiple imputation (MI) analyses, and not for evaluating outcomes, because the randomised comparison group was at that point no longer retained.

Notably, the child outcome is missing at all time points for trial 8 because these data were not collected in the trial. Therefore, this trial had to be excluded from all moderator analyses, although it was included in the wider benefits analysis. In particular, our goal was to understand whether the IY intervention is less or more effective for individuals who vary on one of the following putative moderator variables:

Individual-level moderators at baseline (pre randomisation):

1. SES indicators:

   • Low income: whether or not the family has or is at risk for low income.

   • Lone parent: primary parent is a lone parent.

   • Teenage parent: primary parent was aged < 20 years at birth of target child.

   • SES unemployed: there is no employed individual in the household.

   • Primary parent education level: highest education level attained by the primary parent. Dichotomised as lower secondary or lower and upper secondary and higher.

2. Ethnic minority: whether or not the primary parent is white.

3. Child clinical and demographic variables:

   • Child age: age in months of target child at baseline.

   • Child gender: gender of the target child.

   • Baseline severity: child conduct problems as measured by the ECBI-I or harmonised ECBI-I scale at baseline.

   • ADHD comorbidity: child’s ADHD symptoms as measured by the SDQ subscale of hyperactivity/inattentiveness.

   • Emotional problems comorbidity: level of child’s emotional problems as measured by the SDQ subscale of emotional problems.

4. Parent clinical and demographic variables:

   • Parental depression: level of depression of the primary parent at baseline.

   • Negative parenting measures:

     • corporal punishment

     • threatening behaviour

     • laxness

     • shouting.

   • Positive parenting measures:

     • praise

     • tangible rewards

     • monitoring.

Trial-level moderators at baseline (pre randomisation):

1. Contextual variables:

   • UK versus non-UK: whether or not the trial was conducted in the UK. There are seven UK trials from England and Wales within the pooled data set and six from Ireland and other European countries.

   • Urban versus rural: whether the trial was carried out in a mostly urban or mostly rural setting.

   • Service provider: variable denoting the type of service provider organisation.

   • ‘Efficacy setting’: level of control within the trial of the efficacy versus effectiveness.

2. % certified: variable denoting the percentage of individuals delivering the therapy who are professionally certified.

3. % clinically trained: variable denoting the percentage of therapists delivering the intervention who have been clinically trained.

   • Average number of sessions offered by trial design.

   • % staff certified: percentage of staff delivering the IY intervention at a trial level that were IY certified.

   • % Staff clinically trained: percentage of staff at a trial level that were clinically trained (i.e. social worker, psychologist, nurse or psychiatrist).

   • Trial mentor: whether or not an official IY mentor was part of the staff of the trial.

   • Type of trial: binary coded for treatment or prevention trial.

Initially each of these putative moderator variables will be considered individually as a moderator effect, that is, as a two-way interaction with the treatment effect in the analysis model. We then proceeded to condition the analyses of significant treatment effect moderators on moderation effects of variables that were associated with target moderator to shed some light on possible confounding effects.

Descriptive analyses to inform moderation assessment (PRISMA-IPD #13)

Summarising treatment effects on child outcome (Eyberg Child Behavior Inventory)

Cohen’s d was computed for each trial as an unadjusted measure of treatment effect size. Cohen’s d is a standardised measure of the difference in means between two groups. It is computed by taking the difference between the means in each trial arm and dividing by the pooled SD, for which the pooled SD is estimated from the SD within each group under the assumption that the SD is the same across the population. A Cohen’s d of 0.2 is typically considered a ‘small’ effect size, 0.5 is considered a ‘medium’ effect size and ≥ 0.8 is considered a ‘large’ effect size. For the child outcome, the change between baseline and the first follow-up was first calculated and then the Cohen’s d between treatment and control conditions was computed for the change scores. The baseline measure occurs in all trials before treatment has occurred and the first follow-up in all trials is the measure taken at T1, which takes place between 0 and 2 months after the end of the intervention [see Specification of outcomes and (interaction) effect measures (PRISMA-IPD #13)]. The Cohen’s d for each trial and in the pooled sample will be displayed graphically (Figure 2) along with a 95% confidence interval (CI).

FIGURE 2.

Standardised effect sizes for each trial on disruptive behaviour (ECBI-I). Cohen’s d’s provide standardised treatment effects in terms of ECBI-I. Error bars represent 95% CIs calculated from change scores. The horizontal lines represent no (d = 0), a moderate (d = 0.5) or a large IY pooling benefit (d = 0.8), respectively. Trials references: NOR,116 SWED,117 PORT,104,105,118 IRE,49 NL-BS,119 NL-SES,98 WL-SS,86 BIRM,101,103 LON-SPO,120 LON-PAL,60 LON-HCA,121 OXF122 and VTST. 123

Formal inferences

The goal of the inferential analyses was to test each hypothesised moderation effect, adjust for confounding when this was considered a possibility, describe the nature of the moderation effect and quantify the size of the effect.

Synthesis methods (PRISMA-IPD #14)

We used a one-stage approach in which analyses were conducted in a pooled data set of harmonised data from each trial. Clustering of patients within studies was accounted for by analysing data in a hierarchical structure. The pooled data set has a hierarchical structure with families (level 1 units) nested within therapy groups (level 2 units) within the intervention arm and therapy groups nested within trials (level 3 units).

In addition, our one-stage approach allowed us to reflect features of the trial designs in the analysis models: a number of the trials used stratified randomisation and two trials (trials 11 and 14) used a cluster randomised design. In the analysis, these stratification variables were conditioned on, so that trial arm effects are estimated within subpopulations defined by stratifiers. Cluster variables were also available and included as random intercepts to acknowledge the possible correlation between outcome values from individuals of the same randomisation cluster. Finally, in a number of the trials, the randomisation ratio varied over the duration of the trial, which could lead to confounding of the treatment effect on outcome by the time at which the family was randomised. To avoid such bias, dummy variables that code the randomisation batch, when available, will be conditioned on in the analysis model.

Exploration of variation in effects (PRISMA-IPD #14)

The assessment of variation in effects of the intervention by participant characteristics was the main goal of this project. These putative interactions between treatment effects and covariates were all prespecified and discussed in Specification of outcomes and effect measures (PRISMA-IPD #13).

Additional analyses (PRISMA-IPD #16)

Moderation analyses were carried out by CC analysis as well as after MI. By contrasting the results from these two alternative approaches we can assess the sensitivity of findings to changing the assumptions regarding the missing data generating process (from missing completely at random to MAR) with a change in the size of the moderation effect implying some bias removal by the MI analyses.

Specification of outcomes and effect measures (PRISMA-IPD #13)

To assess potential harms and benefits of the IY intervention we consider the main effect of the intervention (IY vs. control) on each secondary outcome within T1 (post treatment). As an initial step in a series of independent, univariate models (i.e. multilevel analyses of covariances), we tested main effects of the intervention on 12 prespecified outcomes: children’s ADHD symptoms, children’s emotional problems, self-reported positive parenting behaviour (use of praise, rewards and monitoring), self-reported negative parenting behaviour (corporal punishment, threatening, laxness and shouting), parental depression, parenting stress and parental self-efficacy. In the future, we plan to carry out a multivariate analysis that simultaneously assesses the effects on the 12 variables.

For positive variables (i.e. those for which a higher value is a positive outcome) a statistically significant positive effect of treatment can be seen as a benefit, whereas a negative coefficient may be interpreted as a harmful effect. For negative variables (i.e. those for which a higher value represents a more negative outcome), the regression coefficients are in the opposite direction for harms and benefits. We condition on baseline values of the outcome of interest because these are known to influence post-treatment outcomes. In addition, random effects are added to the analysis models to account for between-trial heterogeneity both in the intercept and in the coefficient of treatment. Additional random effects for IY group are added only in the IY arm, to allow for the fact that individuals within the same therapy group may be more correlated than individuals in different therapy groups. As for the moderator modelling, fixed effects are added to account for trial design features, including variables used for stratification and randomisation batch when the randomisation ratio was varied within a trial.

Fixed effects included in the analysis model for harms and benefits are:

• baseline values of the outcome of interest

• a dummy variable coding for treatment condition (IY vs. control)

• child age and child gender (used for stratification in some trials)

• dummy variables coding for relevant trial design features (i.e. other stratification variables and randomisation ratio batches).

Random effects included in the model are:

1. Cluster structure of the pooled data:

   1. Random intercept varying at the level of trial (up to 14 levels) to account for predictive effects of trial characteristics (e.g. differences in trial target populations or general service organisation contexts affecting control groups) on outcome under the control condition.

   2. Random intercept varying at the level of treatment cluster when cluster randomisation was used (trial specific).

   3. Random intercept varying at the level of IY training group within the IY arm of a trial to account only for predictive effects of the training group/therapist environment within the active treatment arm.

2. Random coefficients representing effect heterogeneity:

   1. The regression coefficients representing treatment effects (of trial arm dummy variables) are allowed to vary with trial to model treatment–effect heterogeneity (e.g. because of differences in treatment implementation or target population) not already captured by fixed baseline × trial arm interaction terms.

To estimate a standardised effect size, we ran the analysis for each variable using a standardised outcome. The outcome was standardised by dividing it by the SD of the variable at baseline. This expresses the treatment effect size in units of baseline SDs, meaning that the size of this effect can be compared across outcomes. In addition, we analysed ECBI-I in the same way, so as to compare the treatment effect on the secondary outcomes with the effect size on the primary outcome.

Most of the secondary outcomes are available within only a subset of the trials and so the sample size is limited for this analysis.

We addressed these issues by using multilevel modelling (random-effects modelling) to capture the hierarchical structure of each variable, accommodated design features by trial-specific fixed effects (e.g. conditioned the model for trial 10 on school year strata) and fitted resulting models by maximum likelihood which is valid under a MAR assumption regarding the process that generates the missing data. In our context MAR implies that variables that are included in the analysis model can drive missingness without this leading to bias. We also intended to analyse the outcome variables in a single multivariate model to further relax missingness assumptions, by allowing missingness of the values of one variable to be driven by other observed outcome values. However, it was not technically feasible to simultaneously model the hierarchical structure as well as fitting a covariance matrix for the multiple outcome measures per participant. We thus ran two sets of analyses, each having to make some further assumptions to become feasible: (1) a set of univariate analyses that fully capture the hierarchical structure but require the more restrictive MAR assumption and (2) a simplified multivariate analysis that deals with all the outcome variables in one model but does not account for all the trial features (in particular IY group and cluster effects have to be assumed absent).

The single multivariate model (2) includes dummy variables for each outcome and interactions between these dummy variable and each covariate. We estimated the model with an unstructured covariance structure, which allows correlations to vary between each pair of outcomes. We also included our measure of disruptive child behaviour (i.e. the ECBI-I) to the model, the primary outcome of the trials, to allow for further relaxation of the missing data assumptions.

Synthesis methods (PRISMA-IPD #14)

Similar to analytic plan part 1, we used a one-stage approach (this is PRISMA-IPD terminology). This means that analyses were conducted in a pooled data set of harmonised data from each trial, rather than analyses being conducted within the individual trials and aggregate trial results combined afterwards. Clustering of participants within studies was accounted for by analysing data in a hierarchical structure. The pooled data set has a hierarchical structure with families (level 1 units) nested within therapy groups (level 2 units) within the intervention arm and therapy groups nested within trials (level 3 units).

Exploration of variation in effects (PRISMA-IPD #14)

Exploring variation in effects of the intervention by participant characteristics was not part of this research question.

Additional analyses (PRISMA-IPD #16)

No additional analyses were conducted.

Description of included trials

Descriptives for the pooled data set

Overall effect of the parenting intervention

We know from the published trial literature that the IY parenting intervention significantly reduced parent-reported disruptive child behaviour. The majority of studies indicated that the intervention was beneficial in terms of ECBI-I. The overall unadjusted Cohen’s d effect size for our pooled sample was 0.46. Effect sizes varied between trials, ranging from 0.01 to 1.25 (see Figure 2). Trials with larger effect sizes tended to be trials that were indicated prevention or treatment trials (e.g. trials 2 and 7), as opposed to selective prevention trials (i.e. trials 5, 6, 8 and 11). As mentioned earlier, these four trials also study selected subpopulations that differed on various baseline variables. We therefore controlled for the type of prevention trial in all formal analyses to ensure that any trial-level moderator results were not a result of confounding by this selection variable.

Moderators of parenting intervention effectiveness (PRISMA-IPD #21 cont’d)

In the analyses that follow, we report findings on within-trial moderators, based on MI of missing data, as described in the analysis plan, Analytic plan part 1: moderation analyses. Full results of all moderator analyses can be found in Appendix 5, Tables 29 and 30 including both the MI and, for comparison, the CC analysis findings for individual-level moderators. A summary of significant moderator effects, using MI, is shown in Table 14. There were generally few differences in results between these two approaches for handling missing data. For individual-level moderators we therefore present the less bias-prone MI results. Between-trial moderators results are also provided, using CC analysis, in Trial-level moderators.

We start by presenting the unadjusted moderation results by different types of putative moderators. As a preliminary model-building step, and before investigating any interaction effects, we tested whether or not we needed to distinguish between four treatment arms in our basic model using CC analyses. We found no evidence that this was necessary (ECBI-I did not differ significantly between waiting list/no care and minimal intervention control groups; p = 0.11; or between IY pooling arms with and without the addition of a reading intervention; p = 0.46) and thus we always treated all control arms as equivalent in our analyses. Furthermore, as part of our preliminary model-building we report the specific model selected for each putative moderator variable based on CC analyses: for all variables we report whether or not there was any evidence for differential between- and within-trial moderation effects and describe our moderation findings accordingly. In addition, for continuous variables we include a test of whether or not non-linear effects needed to be allowed for, in order to capture the relationship between the variable and ECBI-I. We quantify the strength of any interaction effects by moderation indices. We tested for the necessity of adding random effects to account for trial design features, such as training group in the IY arm or cluster for those trials that used cluster randomisation. The intraclass correlation was < 0.01 for training group in all trials with the exception of trials 3 and 7, and so random intercepts for the IY group were included in the final analysis for only those trials. The intraclass correlation for cluster in trial 14 was < 0.01 and so this random effect was also excluded.

Family-level moderators

Socioeconomic and social disadvantage as moderators

We considered five variables as capturing social and socioeconomic disadvantage: low income, low education, unemployment status, lone parent or teenage parent.

Low income was the SES moderator with the fewest missing values (CC sample size: n = 1614). Figure 3 illustrates the moderation effect of this variable graphically and indicates very similar IY intervention effects in the low- and high-income groups. Formally, we found no evidence that any IY effect moderation by low income varied between the trial and individual level (p = 0.286). We therefore modelled a single interaction effect for low income. This effect could not be shown to be statistically significant (effect modification index 1.9 points on ECBI-I, 95% CI –4.8 to 8.6 points; p = 0.58). There was therefore no evidence to suggest that low or high income status affected the benefit of the IY intervention.

FIGURE 3.

Box plots illustrating low income (SES) as a moderator of the effects of the IY programme on child conduct problems.

Low or high education status was recorded for a slightly smaller subsample (CC sample size: n = 1573). Figure 4 illustrates this moderation effect, and also suggests similar-sized IY intervention benefits in the two education groups. We found no evidence that any IY effect moderation by low education varied between the trial and individual level (p = 0.27). We therefore again modelled a single interaction effect for low education. Albeit larger in size, this effect could not be shown to be statistically significant in the formal analysis (modification index 4.4 points, 95% CI –2.2 to 10.9 points; p = 0.49). There was therefore no evidence to suggest that low or high education status affected the benefit of the IY intervention.

FIGURE 4.

Box plots illustrating low education (SES) as a moderator of the effects of the IY programme on child conduct problems.

Unemployment was recorded for a smaller subsample (CC sample size: n = 1303). Figure 5 illustrates this moderation effect, and also suggests similar-sized IY intervention benefits in the two employment groups. We found no evidence that any IY effect moderation by unemployment varied between the trial and individual level (p = 0.67). We therefore again modelled a single interaction effect for unemployment. Albeit larger in size, this effect could not be shown to be statistically significant in the formal analysis (modification index 4.88 points, 95% CI –2.7 to 12.4 points; p = 0.21). There was therefore no evidence to suggest that unemployment status moderated the intervention effect.

FIGURE 5.

Box plots illustrating unemployment status (SES) as a moderator of the effects of the IY programme on child conduct problems.

Lone parent status was recorded for a subsample of size n = 1606. Figure 6 illustrates this moderation effect, again suggesting similar sizes of IY intervention benefits in the two groups. We found no evidence that any IY effect moderation by lone parent status varied between the trial and individual level (p = 0.840). We therefore again modelled a single interaction effect for lone parent. This moderation effect was small in size and not statistically significant in the formal analysis (modification index 0.5 points, 95% CI –6.1 to 7.1 points; p = 0.88). There was therefore no evidence to suggest that lone parent status moderated the benefit of the IY intervention.

FIGURE 6.

Box plot illustrating lone parent (SES) as a moderator of the effects of the IY programme on child conduct problems.

Teenage parent was recorded for a subsample of size n = 1550. Figure 7 illustrates this moderation effect, and also suggests similar sizes of IY intervention benefits in the two age groups. We found some evidence that any IY effect moderation by teenage parent status varied between the trial and individual level (p = 0.051). We therefore modelled separate interaction effects for teenage parent at the between- and within-trial level. At the trial level there was a trend for a moderation effect, suggesting that children in trials with a higher proportion of teenage parents benefited more from the intervention (modification index –76.1 points, 95% CI –166.1 to 14.0 points; p = 0.10), whereas at the individual (within-trial) level there was no evidence for a moderation effect and the effect direction was reversed (modification index 7.3 points, 95% CI –2.2 to 16.9 points; p = 0.13). Given that the modification index at the trial level represents the effect of all trial participants shifting from not being teenage parents to all being teenage parents and that between-trial effects are liable to hidden confounding, we treat this result as insufficient evidence for effect modification by teenage parent status.

FIGURE 7.

Box plots illustrating teenage parent (SES) as a moderator of the effects of the IY programme on child conduct problems.

Ethnicity as moderator

Most of the data are drawn from six trials (trials 5, 6 and 9–12), which included 97.7% of the ethnic minority families in the pooled sample. We found some evidence that any IY effect moderation by ethnicity varied between the trial and individual level (p = 0.042). We therefore modelled separate interaction effects for ethnicity at the between- and within-trial levels. At the trial level there was a significant effect (modification index 19.5 points, 95% CI 1.0 to 38.1 points; p = 0.04), suggesting that children in trials with a higher proportion of ethnic minorities benefit less from the intervention. However, no effect modification could be detected within trials and the effect was reversed (modification index –1.4 points, 95% CI –9.8 to 7.1 points; p = 0.75). Given that the between-trial effects are susceptible to confounding we considered that there was no evidence to suggest that the effectiveness of the IY parenting intervention to reduce disruptive child behaviour was influenced by the parent’s ethnic background (Figure 8).

FIGURE 8.

Box plots illustrating ethnic minority as a moderator of the effects of the IY programme on child conduct problems.

Child characteristics as moderators

Figure 9 illustrates the observed moderation by child age. We found no evidence that any IY effect moderation by age varied between the trial and individual level (p = 0.45) or that the functional relationship between ECBI-I and age was not linear (p = 0.89). We therefore again modelled a single interaction effect for age. This moderation effect was small in size and not statistically significant in the formal analysis (modification index 0.04 points, 95% CI –0.1 to 0.2 points; p = 0.65). There was therefore no evidence to suggest that child age moderated the benefit of the IY intervention.

FIGURE 9.

Scattergraph illustrating child age as a moderator of the effects of the IY programme on child conduct problems.

Figure 10 illustrates the observed effect moderation by child gender. We found no evidence that any IY effect moderation by gender varied between the trial and individual level (p = 0.21). We therefore again modelled a single interaction effect for gender. This moderation effect was large in size and statistically significant in the formal analysis (modification index –6.6 points, 95% CI –13.0 to –0.3 points; p = 0.04). The direction of this effect was such that boys benefited more from the IY programme than girls.

FIGURE 10.

Box plots illustrating child gender as a moderator of the effects of the IY programme on child conduct problems.

Child disruptive behaviour

Figure 11 illustrates the observed moderation by baseline ECBI-I. We found evidence that any IY effect moderation by baseline ECBI-I varied between the trial and individual level (p = 0.004) but no evidence that the functional relationship between pre- and post-ECBI-I was not linear (p = 0.09). We therefore assessed the moderation effect separately at the trial and individual levels. At the trial level there was a large-sized and statistically significant moderation effect (modification index –18.3 points, 95% CI –24.6 to –12.0 points; p < 0.001). At the individual level the moderation effect remained statistically significant and in the same direction. However, its size was reduced (modification index –4.3 points, 95% CI –7.9 to –0.7 points; p = 0.02). We therefore concluded that the effectiveness of the IY parenting intervention to reduce disruptive child behaviour was moderated by child’s level of disruptive behaviour at baseline, in the direction that children who had more severe behavioural problems at baseline benefited more from the intervention.

FIGURE 11.

Scattergraph illustrating baseline ECBI-I as a moderator of the effects of the IY programme on child conduct problems.

The solid line describes the trend over increasing baseline ECBI-I in the IY arm and the dashed line that in the control arm.

Figure 11 shows the slope of change in disruptive behaviour by level of baseline disruptive child behaviour, for control and intervention groups.

Child attention deficit hyperactivity disorder

Figure 12 illustrates the observed moderation by child ADHD. We did not find any evidence that any IY effect moderation by ADHD varied between the trial and individual level (p = 0.58), but there was a suggestion that the functional relationship was not linear and a quadratic term needed to be included in the model (p = 0.02). The test for effect moderation did not reach statistical significance at the 5% level (p = 0.07). We conclude that there is insufficient evidence that the effectiveness of the IY parenting intervention to reduce disruptive child behaviour was moderated by child’s level of ADHD behaviours at baseline.

FIGURE 12.

Scattergraph illustrating child ADHD as a moderator of the effects of the IY programme on child conduct problems.

Figure 12 shows the change in disruptive behaviour by level of baseline child ADHD, for control and intervention groups.

Child emotional problems

Figure 13 illustrates the observed moderation by child emotional problems. We found no evidence that any IY effect moderation by these problems varied between the trial and individual level (p = 0.28) or that the functional relationship was not linear (p = 0.38). We therefore again modelled a single interaction effect for child emotional problems. This moderation effect was not statistically significant in the formal analysis (modification index –2.3 points, 95% CI –6.7 to 0.9 points; p = 0.13). Therefore, there was no evidence that the effectiveness of the IY parenting intervention to reduce disruptive child behaviour was moderated by child’s level of emotional problems at baseline. Thus, having a high or low level of emotional problems at baseline did not significantly predict any greater or lesser benefit from IY.

FIGURE 13.

Scattergraph illustrating child emotional problems as a moderator of the effects of the IY programme on child conduct problems.

Parent characteristics as moderators

Parent depression

Figure 14 illustrates the observed moderation by parent depression. We found no evidence that any IY effect moderation by depression varied between the trial and individual levels (p = 0.30) or that the functional relationship was not linear (p = 0.31). We therefore again modelled a single interaction effect for depression. This moderation effect was of larger size and statistically significant in the formal analysis (modification index –4.8 points, 95% CI –8.4 to –1.1 points; p = 0.01). Thus, the effectiveness of the IY parenting intervention to reduce disruptive child behaviour was moderated by the parent’s level of depression, in the direction that children whose parent was more depressed benefited more from the intervention. Figure 14 shows the slope of change in disruptive behaviour by level of depression, for control and intervention groups.

FIGURE 14.

Scattergraph illustrating parent depression as a moderator of the effects of the IY programme on child conduct problems.

Parenting behaviour

All baseline, parenting variables, that is, use of monitoring, rewards, praise, corporal punishment, threatening, laxness or shouting, were tested as moderators. The results are shown in Table 14. None showed a significant moderation effect, suggesting that there is no differential benefit of the intervention for children whose parent has higher or lower levels of positive or negative parenting behaviour at baseline.

Risk of bias across studies (PRISMA-IPD #22)

Risk of bias was low with regard to availability of studies, as all but one eligible trial supplied data to the pooled study. The study for which data were no longer available took place in a general practice setting in Oxfordshire. 102 There were 116 children allocated to IY or to no intervention, in a block randomised design, with eligibility based on a screening survey of all parents of 2- to 8-year-olds registered in three practices. Children scoring in the top 50% of disruptive behaviour were invited into the trial. This is a lower criterion of severity than the other indicated prevention studies in the pool. In Patterson’s trial,102 34% were in the clinical range, whereas in many other trials 100% were in this range. Parents came from a range of social backgrounds, although generally the sample was less disadvantaged than other samples in the pool, with 14% of the families headed by a lone parent, 75% in non-manual employment and 9% from an ethnic minority.

The primary outcome was the same: disruptive child behaviour (ECBI) and the intervention yielded significant effects on this variable at 6 months’ follow-up, but not post test. Effect sizes were not reported, and subgroup analyses were not planned, owing to low power. However, it was noted that there appeared to be more change in the children with high levels of behaviour problems at baseline than in those with closer to average levels. In terms of wider benefits, this study found no significant effect on parent depression, child emotional problems or ADHD symptoms. Although there are clearly some differences between this trial and the others in the pool, there is no particular reason to think that our findings would be affected by inclusion of this trial. We note that at trial level, the findings of Patterson et al. 102 echo the broad pattern found in our analyses, whereby samples with lower levels of conduct problems showed weaker effect sizes.

Risk of bias was low with respect to the intervention evaluated, as all trials evaluated the same intervention. Moreover, the IY parenting intervention is strictly protocolled, including training and supervision of the staff. The intervention as evaluated is therefore expected to be similar, broadly speaking, across trials.

Risk of bias may be higher with regard to the data available, and how data for some of the constructs were synthesised. Not all trials included the same measures for the same constructs. This means that for some constructs (e.g. self-reported parenting practices) data came from up to four different instruments. Although these measures individually showed good individual psychometric properties and explicitly aimed to measure the targeted construct, we were not always able to test the extent to which scores across measures correlated with each other. A potential bias of this procedure is that the instruments may not have measured exactly the same construct. Please note, however, that in cases for which we were able to examine correlations between measures targeting the same construct, these were reassuring. For example, correlation between the two measures used for the primary outcome was around. 70

Relatedly, it was inevitable to make several assumptions in the data harmonisation process. For example, we used the same norm scores across trials from different countries, because using different norm scores for different samples would have differentially changed the distribution of scores within the samples. Moreover, norm scores were often not available for every measure from every country (e.g. Portugal and Ireland). We therefore chose to use one set of norm scores (i.e. the instruments original and validated norm scores) for all samples. A potential bias of this procedure is that the original norm scores may not reflect the actual norm scores in another country.

Some relevant variables (e.g. SES and ethnicity) were not evenly distributed across trials. We controlled for trial-level variance in all the analyses, which was essential because several variables were confounded on the trial level (e.g. trials with more ethnic minority families were more often secondary prevention trials that include families with less severe disruptive behaviour problems, rather than indicative prevention or treatment trials that include families with more severe disruptive behaviour problems.

Social and socioeconomic disadvantage

The primary question for this individual participant meta-analysis was whether or not there are differential effects of the IY parenting programme on child disruptive behaviour for families with higher levels of social and socioeconomic disadvantage. The findings were quite clear: in our primary analyses, based on pooled individual-level data (i.e. within-trial analyses), there were no moderator effects, meaning that families who were socioeconomically disadvantaged were just as likely to benefit from the intervention as those who were not. This finding applied across a range of indices of social and economic disadvantage, including low income, low educational level, being a lone parent or a teenage parent and having no employed person in the household. We can be reasonably confident in these findings in several respects. First, for three of the moderator variables, there were near-complete data from all 13 trials (5–7% missing); data for teenage parenthood were unavailable for one trial, and for unemployment for three trials. Second, the primary analyses used MI to account for missing data but closely similar results were obtained from CC analysis (see Appendix 5, Table 30). On the other hand, moderator effects were not particularly consistent across trials; this is likely to be a result in large part of the problem of small sample size in each trial overall, with this being especially marked for some moderator variables. Thus, here and in other analyses, we do not attempt to interpret between-trial variability in size of the moderator indices.

For each moderator variable, we examined whether or not there was any difference between the moderator effects at the between- and within-trial levels. When a difference was found, we then modelled both effects. This was the case for one variable: teenage parenthood. There was no within-trial moderator effect for teenage parent, but a trend towards a between-trial effect (p = 0.1) in the direction that trials with more teenage parents tended to have higher effect sizes. However, this direction of effect was not seen in the within-trial findings, which trended non-significantly in the opposite direction. It should be noted that between-trial effects are unlikely to be important: they are based on a very small sample size, representing the number of trials included in the analysis, which in this case was 12. The small sample size also means that they are likely to be subject to hidden confounding, related to particular characteristics of trials that happen to covary with percentage of teenage parents.

These pooled data findings help to clarify a previously very mixed picture in the literature, in which highly cited reviews have concluded that children from socially disadvantaged families benefit less from parenting interventions to reduce disruptive behaviour. 43,44 On the other hand, one recent narrative review45 and some meta-analyses12,149 concluded that there were few differential effects of social and socioeconomic disadvantage. However, these studies suffer from important drawbacks, which are overcome by our study. First, many studies are limited by examining predictors rather than moderators of outcome. 43,44 Second, most reviews index moderator variables only at the aggregate trial level. 12,43,149 Finally, many studies, although testing moderation at individual level, are instead limited by analysing or synthesising data from small trials, which lack power to detect interaction effects. 45,46 Even when trials are much larger, findings can be hard to interpret or replicate; for example, one US trial found no moderator effects of low income or teenage parenthood on outcome but found stronger effects for low educated parents along with diminished effects for lone parents. 47 Given a prominent body of literature concluding that socially disadvantaged families fare less well in parenting interventions, but which is based on less than ideal methods, our findings are of particular significance. Our study overcomes these three major limitations and points to a clear and more optimistic conclusion, namely that families disadvantaged by low income and low educational level, or by lone parenthood, are just as likely to benefit. Furthermore, there are clear implications for equity effects of parenting interventions: if taken to scale, these interventions are unlikely to result in further widening of existing social inequalities in child disruptive behaviour.

It is worth noting that many of the qualitative studies in the field49 point to parents feeling that family social and economic stresses contribute to low levels of engagement and success in a parenting intervention. Although not invalidating individual experience, which of course is hugely variable, our data suggest that at a group average level, the stresses associated with low income and unemployment do not systematically affect the outcome of these interventions, at least once families have been recruited in to the intervention. Generally, the families in our focus groups (see Appendix 6) were also unsurprised about the results; they were of the view that factors such as educational level of the parent should not make a difference to outcome, as the IY materials, they felt, would be accessible to a wide range of parents. However, in one qualitative study of low-income families attending IY in Ireland,50 parents perceived different social barriers as more salient, including antisocial behaviour in the neighbourhood, and disagreement with their partner about implementing the programme. This may illuminate a possible reason why lone parenthood did not function as a moderator. Although lack of a partner may lead to feeling unsupported in a new parenting role, having a partner who is unsupportive may have a similar effect. Rather than perceiving life stressors as straightforward barriers, many parents stressed their commitment to implementing parenting skills in the face of difficulties, helping them to maintain positive child outcomes in the longer term. 150

Children’s mental health (attention deficit hyperactivity disorder symptoms and emotional problems)

The intervention had wider health benefits on children’s ADHD symptoms (β = –0.30, 95% CI –0.44 to –0.17; see Table 19). Children whose parents had participated in the parenting intervention showed fewer ADHD symptoms than children whose parents had not participated in the parenting intervention. In families that received the intervention, the percentage of children who scored above the borderline threshold on ADHD symptoms fell from 54% to 42%, which is more than a 10% reduction. In families that did not receive the intervention, the percentage of children who scored above the borderline threshold on ADHD symptoms fell from 52% to 50%, a reduction of only 2%.

Earlier findings about the extent to which parenting interventions designed to reduce conduct problems can reduce children’s symptoms of ADHD are inconsistent. Some review studies have suggested that effects may depend on the type of instrument or informant used (e.g. parents vs. teachers93). Even within the same informant, however, results are strikingly inconsistent. Some individual trials find that the IY parenting intervention can successfully reduce ADHD symptoms,68,69 whereas others found no effects on children’s ADHD symptoms (e.g. Leijten et al. 96). Combining individual family-level data from > 1200 families from 11 trials led us to conclude that one of the wider health benefits of the IY parenting intervention is that it does reduce parent-reported ADHD symptoms in children.

The intervention did not have wider health benefits on children’s emotional symptoms (β = –0.06, 95% CI –0.18 to 0.06; p = 0.303). There seemed to be a reduction in children’s emotional problems, regardless of intervention status. In families that received the intervention, the percentage of children who scored above the borderline threshold on emotional problems fell from 40% to 32%, which was an 8% reduction. In families who did not receive the intervention, the percentage of children scoring above the borderline threshold on emotional problems also fell, from 40% to 32%.

Evidence of the extent to which parenting interventions designed to reduce conduct problems reduce children’s emotional problems is limited. Some trials have reported a measure of children’s emotional problems as a secondary outcome, but most did not find any effects of the intervention on this measure (e.g. Leijten et al. 96). However, some trials have specifically focused on the extent to which there are wider health benefits in relation to children’s emotional problems and these do show the hypothesised effects (e.g. Herman et al. 95). Our finding that the intervention does reduce ADHD symptoms, and not emotional problems, may not be surprising. ADHD symptoms more often co-occur with conduct problems than emotional problems. 91 When the intervention successfully reduced conduct problems, ADHD symptoms may therefore have been more easily reduced than the more unrelated emotional problems.

These findings have important implications for policy-makers and practitioners. They suggest that an intervention for one type of externalising behaviour (i.e. conduct problems) may have wider benefits on other types of externalising behaviour (in this case ADHD symptoms). They also indicate, however, that the intervention should not be expected to have wider benefits for children’s internalising behaviour (i.e. emotional problems).

Service use data

Central to the estimation of costs and cost-effectiveness are the records of service use for each person in the trial, including use made of the IY programme by the intervention groups. Of the 14 studies, five were undertaken in countries other than the UK or Ireland, for which the service array and public sector financing systems are very different. Two UK trials did not collect any service use information, and one collected it in a way that was not suitable for use in economic evaluation. Another UK trial was excluded because no baseline service use information was collected. Five trials therefore met the criteria for inclusion.

All service use data were merged and cleaned by the economics team, ensuring that categories of service use and measures of intensity are comparable across all studies. A detailed description of our data harmonisation approach can be found in Appendix 7.

Service use

Information on self-reported service use was obtained from resource use questionnaires completed by trial participants at baseline and follow-up. The baseline period covered 6 months for three trials and 12 months for two trials. Follow-up periods covered 6 months for three trials, 3 months for one trial and 12 months for one trial.

Intervention cost

For the cost-effectiveness analyses, unit costs for six sites that implemented IY interventions were estimated using standard methods informed by economic theory. Data were requested from all collaborating centres using a standardised Service Information Schedule. To ensure comparability with other unit costs used in these analyses, we have excluded set-up costs, such as initial training or amendment of the standard IY programme to meet the needs of the study. These are identified in our paper exploring the costs of the IY intervention and their relationship with fidelity characteristics (J Beecham, 2016, personal communication). Staff time forms the major part of the costs; we have used nationally applicable (all England, i.e. London and the rest of England) salaries and on-costs taken from the Unit Costs of Health and Social Care176 or estimated using a commensurate method. Staff costs are estimated according to their professional background; average salaries from group leaders have been estimated to reflect the staff mix employed in each site. Although we have included the costs of supervisors’ time over the course of the programme, we have excluded their travel and subsistence costs, as the two sites that purchased supervision directly from IY were subject to high (and variable) travel costs, relative to the other four sites. We also excluded the costs of reimbursing participants’ travel costs, as insufficient information was available. Venue costs have been estimated using data from the Unit Costs of Health and Social Care. 176 We also include costs accruing for project management, administrative assistance, materials used in the sessions, snacks and provision of a crèche facility.

Service costs

For each service that participants reported using on a service use questionnaire (Client Service Receipt Inventory; CSRI),177 an appropriate unit cost was obtained from publicly available sources176,178 or calculated using an equivalent approach. 179 Unit costs used in this study are detailed in Appendix 8, Table 31. The total costs associated with service use, as well as subtotals for costs for community health services (including primary care), hospital services, specialist mental health services, social care, accommodation away from home (e.g. foster care) and services provided by the voluntary sector, were calculated for each participant by multiplying the reported number of service contacts by the corresponding unit cost. When the number of service contacts was missing but it was indicated that a participant used the service, the number of contacts was imputed using the mean of service contacts for the specific trial.

All costs are presented in 2014 prices. Given that costs cover a 6-month period, no discount rate is applied.

This re-estimation of unit costs and per participant costs improves the comparability of costs across the trials by ensuring that any cost differences are not a result of the individual approaches to estimating unit costs or total costs, without deviating from the original data collection.

Statistical analyses

Analyses were performed using CCs. Participants were included in the cost analysis if they had completed a CSRI177 at baseline and follow-up, and in the cost-effectiveness analysis if they also had completed the ECBI-I assessments.

Unless stated otherwise, all statistical models account for (1) the skewed distribution typical of cost data and (2) the hierarchical structure of the data and potential clustering by drawing bootstrap samples180 (10,000 replications) by cluster (family and trial) and strata (treatment group, per cent of sites in rural locations, whether or not the trial was conducted in England and whether or not the CSRI data collection period differed from 6 months).

Cost-effectiveness analyses

We tested whether or not IY is likely to be considered cost-effective compared with the control condition. Cost-effectiveness analysis was conducted on the pooled data. The cost-effectiveness analysis takes a public sector perspective, focusing on the children’s use of services and support. The primary analysis uses change in ECBI-I as the outcome measure. A secondary analysis uses a binary variable indicating whether or not a child is moved below the ECBI-I clinical cut-off point (score ≥ 131 points) following the intervention.

Using seemingly unrelated regression, separate regression models were fitted for (1) costs at follow-up, based on our analysis of cost variations and (2) each outcome measure. The outcome measures of interest were (1) change in ECBI-I (linear regression) and (2) whether or not child is below the clinical cut-off point on the ECBI-I (logistic regression).

For each combination of costs and outcome, 10,000 bootstrap replications of the treatment effect were generated. The probability that the intervention is cost-effective was assessed by calculating the proportion of incremental cost-effectiveness ratios that indicated the intervention would be judged cost-effective, given a range of values that a funder or wider society may place on an improvement in outcome. This value is commonly known as ‘willingness to pay’ (WTP). The probability that the intervention would be considered cost-effective was then plotted against its corresponding value of WTP, resulting in a cost-effectiveness acceptability curve (CEAC). 181

Economic modelling

To estimate potential longer-term savings from the IY intervention, we updated our economic model of the long-term costs associated with persistent conduct disorder. 20 The original model included parameters for intervention take-up and dropout. No data for take-up are available in our data set, and, as results for the present study were obtained on an intention-to-treat basis, dropout is already accounted for in the overall effectiveness figure.

As in our original model, we estimate a trajectory of remission in the absence of intervention to use as a simulated control group. Based on the literature, we assume that about 60% of children with behavioural problems at age 3 years still show problems at age 8 years, and that 50% of children with behaviour problems develop adult antisocial personality disorder. 182 Without intervention, we therefore estimate that the chance that behaviour problems at age 5 years persisting beyond age 16 years is approximately 60%.

In this analysis, the estimated intervention effect is calculated in relation to the observed control group trajectory. We used the information from the pooled sample to construct a sensitivity analysis, varying the presumed control group trajectory.

Longer-term cost estimates were updated to reflect more recent research. In addition to uprating the existing cost parameters to 2014 prices, costs were taken from two recent studies of UK cohorts. 164,166 This allowed us to create a ‘high-cost’ and a ‘lower-cost’ scenario for each trajectory.

The following cost categories are considered in the model: NHS, social services departments, Department for Education, voluntary sector, criminal justice system, health impacts of crime and benefits payments. Note that all costs are calculated as additional costs due to conduct disorder, over and above what would be spent on a person who did not experience behaviour problems at age 5 years. Costs are calculated from age 5 years to age 30 years, and discounted to present value using a discount rate of 3.5%.

The intervention effect is based on the results of our cost-effectiveness analysis, in which we calculate the odds ratio of a child falling into the non-clinical range of the ECBI-I (post treatment) for the intervention compared with the control group. In this analysis, we assume that the intervention is delivered at age 5 years, which is approximately the average age of our study sample.

Sample for economic analysis

Data from five trials were merged to conduct the economic analyses. They were trial 4, IRE; trial 7, WL-SS; trial 9, BIRM; trial 10, LON-SPO; and trial 12, LON-HCA. Table 20 shows baseline demographic data for the 608 participants included in the analysis (control group, n = 236; intervention group, n = 372).

There are some notable differences between the full analysis sample and the sample for economic analysis. Children in the latter tend to be younger, on average, and there are differences in the sociodemographic composition of the samples. Indicators of SES tend to be less favourable in the economic sample, whereas the proportion of participants identifying as members of an ethnic minority is smaller.

Trial-level characteristics for the trials included in the economic sample and in the full analysis sample are shown in Table 21. Although the small number of trials makes a comparison difficult, there do not appear to be striking differences between the full set of trials and the sample for economic analysis, with the exception of geographic location.

Service use

Service use at baseline and follow-up is shown by treatment group in Table 22 (a fully expanded service use table can be found in Appendix 9, Table 32). At baseline, > 70% in each treatment group had seen their general practitioner (GP). In the control group, 23% reported contacts with a health visitor, whereas in the intervention group, this percentage was higher, at 32%; this was the only service for which there was a difference between groups that exceeded 5%. Outpatient hospital services were used by 21% in each treatment group, whereas 17% in the control and 19% in the intervention group utilised the accident and emergency department (A&E). Five per cent of participants reported inpatient stays at baseline.

At follow-up, there were no differences between treatment groups in the proportion of participants reporting service contacts of 5% or more for any service, and there were no notable increases in the proportion reporting contacts from baseline. Contact with GPs reduced to around 50%, whereas the proportion in contact with health visitors decreased to 10%. Use of A&E and outpatient services also declined in both groups.

Few participants reported use of services beyond primary care and hospital services. Notably, use of mental health services was low. Placements for children away from their family are worth noting as they tend to be costly but these were not frequent occurrences in our sample.

Costs and cost variations

Table 23 shows costs at baseline and follow up by intervention group.

Cost-effectiveness analyses

Cost-effectiveness acceptability curves

Figure 15 shows the CEAC for improvement in disruptive behaviour (ECBI-I) in the intervention group compared with the control group, for WTP from £0 to £250. The probability that the IY intervention will be considered cost-effective exceeds 50% at a WTP of £109 per 1-point improvement on the ECBI-I. This is equivalent to a coin toss. An 80% chance of cost-effectiveness is exceeded at a WTP of £121, whereas 95% is reached at £134 and 99% at £145.

FIGURE 15.

Cost-effectiveness acceptability curve for the IY intervention: ECBI-I score.

Subgroup analysis

The main analysis was repeated for subgroups by gender (male vs. female), baseline ECBI-I score (< 131 vs. ≥ 131), child age (< 5 years vs. ≥ 5 years) and parental depression at baseline (parental BDI score of ≥ 20 points vs. < 20 points). The corresponding CEACs are shown in Figure 16, together with the curve for all children. Note that these curves were estimated in separate models.

FIGURE 16.

Cost-effectiveness acceptability curves (subgroup analyses): ECBI-I score.

Based on our analysis, the IY intervention is less likely to be considered cost-effective for children with an ECBI-I score below the clinical cut-off point at baseline and who are < 5 years old. It is more likely to be considered cost-effective for boys than for girls, and for children whose parent’s BDI-II score at baseline indicates at least a moderate level of depression.

When the outcome is increasing by 1% the probability of scoring below the clinical threshold on the ECBI-I scale as a result of the intervention (Figure 17), there is a 50% chance that the IY intervention will be considered cost-effective at a WTP between £45 and £50. An 80% chance is reached at a WTP of £70, whereas 95% is reached at £95 and 99% reached at a WTP of £370.

FIGURE 17.

Cost-effectiveness acceptability curve for the IY intervention: ECBI-I score below clinical threshold post intervention.

Longer-term savings from the Incredible Years intervention: economic modelling

Figures 18 and 19 show the probability that a child still experiences behaviour problems over time, with year 0 being the year of the intervention and year 1 the time when the intervention effect manifests. The ‘no intervention’ trajectory in Figure 18 follows the data from the literature, with a 60% chance that a child with behaviour problems at age 5 years will still experience problems after the age of 16 years (11 years post intervention). In the intervention scenario, this probability is 54%.

FIGURE 18.

Trajectory of probability of behaviour problems persisting from age 5 years: scenario 1.

FIGURE 19.

Trajectory of probability of behaviour problems persisting from age 5 years: scenario 2.

Figure 19 shows scenario 2 and the effect of the intervention compared with a trajectory in the absence of intervention that is modelled on the change observed in the pooled IY sample for the control group, with a 12% decrease in the probability of scoring above the ECBI-I cut-off point in year 1. On this trajectory, the probability of behaviour problems persisting past age 16 years is again 60%, but with a steeper decrease in year 1. As the intervention effect is estimated as a multiple of the reduction in probability in the control group, in this scenario, the probability that conduct problems persist beyond age 16 years in the intervention group is reduced to 52%.

Table 25 shows the present value (i.e. discounted to ‘today’s money’ using a 3.5% discount rate) of savings from the intervention based on the assumptions above.

TABLE 25 - Present value of savings from providing the IY intervention at age 5 years, per child and by agency

Agency/budget	Savings ‘scenario 1’		Savings ‘scenario 2’
	Low (£)	High (£)	Low (£)	High (£)
NHS	106	2445	129	3039
Social services departments	72	182	88	221
Department for Education	613	949	746	1148
Voluntary sector	0	26	0	32
Criminal justice system	232	3583	290	4493
Benefits payments	0	381	0	475
Totals	1023	7565	1254	9408
Net of intervention cost	–1391	5151	–1160	6994

In scenario 1, savings range from £1023 to £7565, whereas in scenario 2, they range from £1254 to £9408. In the low-cost scenarios, no savings are estimated to be achieved once the intervention cost is subtracted. Savings are largest in the areas of highest potential costs, the criminal justice system and the Department for Education.

Use of services and service costs

The findings regarding service use are broadly in line with other findings from studies of children with mental health problems. Most support comes from primary care and hospital services, with little involvement from mental health services. Although our data do not allow us to determine whether this is because of a lack of help seeking or a gap in provision, the fact that only one in four children and young people with mental health problems is in contact with mental health services has long been recognised as a significant problem, and increasing access is a priority. 183

Although we observed significant differences in total costs between treatment groups, this was driven by the cost of the IY intervention. The cost of the intervention was estimated for each trial, using the same principles and cost categories. Notably, the average cost of the intervention for this sample was £2414 (2014 prices), and therefore more than twice the previous ‘best estimate’ of £952 (2009 prices);167 it should be noted, however, that this is a median, whereas we report the mean. However, our average cost is similar to the estimate by Edwards et al. 169 of £1934 for IY delivered through Sure Start Centres once we account for inflation.

Mirroring the findings from the analysis of differential treatment effects presented in Chapter 3, we do not find any cost variations associated with social disadvantage, ethnicity, ADHD or child emotional problems at baseline. However, unlike the main analysis, we do not see cost variations associated with baseline levels of disruptive behaviour or parental depression. However, we did find significant variations in costs at follow-up for child gender and age, suggesting that there is both a differential effect and a differential impact on costs from the IY intervention for these characteristics. Girls in our study incurred lower costs on average but they also did not benefit from the intervention as much as boys. The evidence on the relationship between gender and costs is mixed. Although Romeo et al. 163 found costs to be lower for girls than boys aged 3–8 years referred to specialist mental health services, Knapp et al. 165 found higher mental health service costs for girls. We saw (modestly) decreasing costs with age, in line with previous findings,165,184 even though mental health problems generally increase with age. 185 We did not find an association between costs and ethnicity, and again the previous evidence is mixed, with one study finding higher mental health-related special education costs,165 although others have found no link. 184

Cost-effectiveness and longer-term savings

The CEACs presented in this study may appear unfamiliar to some readers. Although cost-effectiveness analyses used to be presented as incremental cost-effectiveness ratio, summarising information on cost and effect differences in a single figure is problematic: a negative ratio can indicate higher costs coupled with worse outcomes, as well as lower costs and better outcomes. Similarly, a positive ratio can represent both higher costs and better outcomes in the experimental group, and lower costs and less favourable outcomes than the control group. Therefore, the incremental cost-effectiveness ratio on its own does not provide the information needed to judge the cost-effectiveness of an intervention. For this reason, CEACs are the current standard method of presenting cost-effectiveness information. For a discussion of the statistical issues associated with the incremental cost-effectiveness ratio and the derivation of the CEAC, see van Hout et al. 181

Our cost-effectiveness analyses show that at a WTP of £109 per point improvement on the ECBI-I is 50%, increasing to 99% at a WTP of £145. Our subgroup analysis showed that the intervention is less likely to be considered cost-effective for children who scored below the clinical threshold on the ECBI-I at baseline, and for children under the age of 5 years. At the same time, it would be more likely to be considered cost-effective for boys than for girls and for children whose parents had a BDI-II score at baseline indicating at least a mild level of depression. It should be noted that this analysis was limited to cases with full data, and results should therefore be interpreted with caution.

The results of cost-effectiveness analyses are often difficult to compare because of different methods used and different coverage of cost categories. In the study by Edwards et al. 169 a 50% probability of cost-effectiveness was reached at a WTP of approximately £75 (2004 costs; note that the data in the Edwards et al. study are not presented in such a way that this comparison can be made easily), whereas a 99% chance is reached around £150. Overall, our findings appear reasonably similar.

Willingness to pay is hard to interpret when there is no established societal threshold for the outcome in question (in this case, the ECBI-I). The only outcome with a well-established WTP threshold is quality of life as measured by quality-adjusted life-years. No quality-of-life measure was included in any of the trials that form part of our analysis here and, arguably, would not be appropriate for these young children with mental health problems. 186 Our findings suggest that a 1% increase in the chance that a child scores below the clinical threshold following intervention can be achieved with reasonable certainty (80%) at a WTP of £70. This figure needs to be evaluated in the context of the likely long-term societal costs associated with persistent conduct disorder.

It should also be noted that the treatment effects for three of the trials included in the economic analysis were below the mean for the pooled sample, and therefore our findings may underestimate the potential for cost-effectiveness for the IY sample as a whole (assuming costs in the other trials would have been similar to our economic sample).

Longer-term savings

There is a lack of empirical evidence on the long-term savings from parenting programmes, especially in a European context. In this study, we adapted our previous model estimating the longer-term savings from parenting programmes intended to prevent persistence of behaviour problems. 20 We drew on additional literature and created a ‘low-cost’ and a ‘high-cost’ scenario. In addition, we varied the assumptions around the natural course of conduct disorder. These differences in the underlying model mean that our results are not directly comparable. Instead, they should be viewed as the next iteration of the model.

The previous study assumed an intervention effect of 34%, that is, the proportion of children who, following the intervention, were below the clinical cut-off point, was 34% higher in the intervention group than in the control group. Note that this study included both group-based parenting programmes and those delivered to individual families. This is a different approach to estimating effectiveness, and our present estimates result in a much more conservative assumption about the intervention effect. Combined with a higher intervention cost (£2414 in 2013–14 prices vs. £1177 in 2008–9 prices).

Our results indicate that if the costs associated with persistent conduct disorder are in fact lower than previously thought (see the ‘low-cost’ scenarios), at the current price and effectiveness, the IY intervention would not result in longer-term savings. However, in the ‘high-cost’ scenario, the return on investment is substantial – threefold in scenario 1 and nearly fourfold in scenario 2. They also indicate that any model-derived findings are very sensitive to even slight changes to the assumptions about the control group trajectory, here a change in the assumed chance of behaviour problems persisting, from 8% to 12%.

Strengths

The study has many strengths that increase our confidence in the findings. Our unique pooled data set means that we have a much larger sample than other studies in this field, and findings that are potentially generalisable across countries, service settings, families and level of child problems. The use of pooled individual-level data is vital for testing moderators and has many advantages compared with traditional meta-analysis, in which data are analysed at a trial level by producing average treatment effects weighted by sample size. A potential problem with this traditional approach is that effects are more likely to be influenced by common causes that occur at a trial level. In this study we analyse data at an individual level rather than a trial level. Individual-level meta-analysis allows the separation of between- and within-trial effects,80 meaning that individual-level effects can be distinguished from trial-level effects that are more likely to be confounded. Individual-level meta-analysis also allows for the analysis of a greater sample size (1799 individuals compared with 14 trials, if this had been a traditional meta-analysis), meaning that there is greater statistical power to detect effects.

There are a number of statistical challenges in pooling data from different trials at an individual level, which our methods were able to address. First, the analysis model must account for the fact that there are trial-level differences in both outcome and treatment effect. Second, different trials used different designs, such as clustered randomisation and different therapy groups in a trial for participants in the intervention arm, and these needed to be accounted for in the analysis model. A multilevel model,188 which accounts for differences between trials, is used to address these issues. Finally, we account for missing data in the moderator analysis using MI,143 an approach that requires less restrictive assumptions than using only cases with no missing values.

Furthermore, plausibility of the findings is enhanced in other ways by our analytical strategies. Randomisation appeared adequate, as we found no evidence for baseline imbalances across the pooled data set. We controlled for the possibility of confounding by the type of sample selection into the trial, that is, whether or not it is a selective prevention compared with an indicated prevention or treatment trial, as this can make a considerable difference to the initial severity of child disruptive behaviour. Sensitivity analyses found no differences according to the type of control group used, such as treatment-as-usual compared with no intervention. We were able to check for non-linear effects, to take account of the possibility that there may be more complex relationships between moderator and outcome. For example, it could be that mild to moderate levels of parent depression enhance the effects of IY on child outcomes, compared with low depression, but, that at very high levels, parents are unable to effectively engage with the programme once they have signed up for it. We found no evidence for this kind of non-linear effect in the depression moderator data.

Plausibility of the findings is also enhanced by reanalysis of complete trial data sets, using a preplanned data-analytic strategy, applied consistently across all trials. This approach is highly likely to reduce the risk of reporting and publication bias, compared with conducting moderator analyses in small trials, which are designed and powered to test main effects but are generally underpowered for moderator analyses. Low power combined with absence of prespecified protocols for moderator analyses in the existing literature would tend to lead to a high risk of publication and reporting bias.

Our data also allowed us to examine within- vs. between-trial moderation effects; these make for an interesting comparison, as they show us how the moderator findings would have looked if we had conducted a conventional metaregression analysis, compared with a meta-analysis with IPD. In some cases, there was evidence of moderator effects at an individual level and none at the trial level (e.g. for parent depression), meaning that a conventional metaregression would have led us to miss significant moderator effects. In other cases, there was evidence of a trial-level moderator effect, which in the individual-level analysis was non-significant and, in the case of ethnicity, was reversed in direction. Thus, with conventional metaregression, we would have concluded that families from ethnic minorities did less well, whereas the IPD analysis suggested that they in fact tended to do slightly (non-significantly) better. We attributed this discrepancy to confounding at trial level by severity of child disruptive behaviour, such that trials with higher proportion of ethnic minority families in them were more likely to be selective prevention trials, and to have lower levels of disruptive behaviour. Interestingly, in one case, trial- and individual-level analyses concurred in showing significant moderation, by baseline level of child disruptive behaviour, although again the magnitude of this modification effect appeared larger in the trial-level analysis.

Our data also allow us to compare the conclusions that might be drawn from predictor compared with moderator analyses. Examining the charts for binary moderator findings, it appears as if the same conclusions would have been drawn from analysing predictors in the intervention group only, without examining the interaction with treatment status. However, we did not plan to make this comparison formally and, in any case, we could not have known this without doing full moderator/interaction analysis. It would not be recommended to conduct predictor analyses in order to understand equity effects, as in some studies, predictors and moderators do show different patterns of results. 46

Limitations

Alongside these substantial strengths of the study, there are a number of limitations. In particular, assumptions and compromises inevitably had to be made in data harmonisation. First, there is the problem of variation in instruments used. Although many trials used similar instruments to assess the primary constructs in this study, we had to harmonise data across instruments for every construct. Whenever data were available to cross-validate our harmonisation procedure, for example by computing correlations between the different instruments, results were reassuring that the instruments indeed seemed to tap into the same constructs. Nevertheless, we had to make several assumptions in combining data across instruments, for example that the different instruments measured the same construct with the same measurement error. Combining data across instruments assumes measurement of the same construct in each instrument and the same measurement error. We were, to a limited extent, able to check whether or not the different instruments indeed measured the same construct. For our primary outcome measure, for example, some trials included data on both instruments used and correlation analysis revealed satisfactory association between the instruments of r = 0.70. Because for some constructs (e.g. parental depression) no overlap in instruments within trials was available, these checks were unfortunately not possible for all constructs for which we harmonised data. Similar problems arose from the need to combine economic data collected using different instruments. These are discussed in Appendix 7.

Second, assumptions had to be made in using norm scores to harmonise data across trials on child disruptive behaviour and parent depression. For example, we used the same norm scores across trials from different countries, because using different norm scores for different samples would have differentially changed the distribution of scores within the samples. Moreover, norm scores were often not available for every measure from every country. We therefore chose to use one set of norm scores, namely the instruments’ original and validated norm scores, for all samples. A potential bias of this procedure is that the original norm scores may not reflect the actual norm scores in another country.

Third, social status and SES were the primary moderators of interest in this study. Availability of different indicators of social and socioeconomic disadvantage (e.g. income, educational level) across trials allowed us to approach the putative moderator role of SES in multiple ways, a recommended approach for studying SES. The individual indicators, however, were all binary variables. Although most individual trials provided data for some variables at an ordinal level, harmonisation often required us to simplify these data into binary variables. From an applied perspective, it may be especially interesting to understand the extent to which interventions benefit the most disadvantaged families. From a statistical view, however, binary variables ignore much of the meaningful variation between families who range on a dimension of SES.

Fourth, there was a great deal of variation between trials in the measures of self-reported parenting used, presenting greater challenges in harmonising these variables than most others. In order to find items that were common across trials, we often had to rely on very small numbers of items for some parenting constructs, and some items were measured in only a small number of trials, leading to considerable reduction in the sample size for some of the analyses, as can be seen in Tables 3 and 4 (see Chapter 2).

Fifth, we were unable to meaningfully synthesise the parent satisfaction data, owing to high levels of missing data. Five trials used the same instrument, the Parent Satisfaction Questionnaire (PSQ), which is administered only to parents in the intervention group. There were 379 families in the intervention groups in these trials; however, with a response rate of 55%, this yielded data on only 210 families, representing only 19% of all families in the intervention condition (the total sample size in the intervention condition is 1116).

Sixth, our study, and the underlying trials, were limited by many of the key outcomes being measured by parent report. It should be noted that most of the trials in our pool also included systematic direct observational measures. Observational measures, although not without technical and other challenges, have many advantages;189,190 in particular, they allow for coding of data by researchers who are blinded to the intervention status of the families. In many cases observational data showed positive effects on child behaviour and on parenting skill outcomes (e.g. Gardner et al. 103 and Hutchings et al. 85), increasing our confidence in the validity of the main findings, based on parent self-report. However, our ability to incorporate observational data in the pooled set depended upon the possibilities for harmonisation of measures in the individual trials. Data on observed child and parenting behaviour were too diverse in terms of content (e.g. compliance vs. negative affect in children) and structure (e.g. Likert scales vs. frequency counts) to harmonise across trials, within the scope of this project. Teacher-reported child behaviour would potentially provide a more independent source; however, these measures were available only in a few trials.

Seventh, although there were many important questions for policy that we would like to have asked about the effects of trial-level moderators, these analyses were of relatively limited value, owing to low power (n = 13) and much potential for confounding at trial level. Furthermore, some trial-level (or site-level) variables showed little variation, such as rural compared with urban intervention site, with most interventions being delivered in urban settings.

Eighth, our moderator findings can assess factors predicting success only once parents have signed on to the programme; we cannot tell if different factors might affect equity of initial access to the programme.

Ninth, our study was unable to inform us about the longer-term effects of IY, because most of the studies used a waiting list design, with families in the control group being offered the intervention ≥ 6 months later. However, it should be noted that a recent systematic review of longer-term effects of largely similar social learning theory-based parenting programmes found that the effects of parenting programmes on disruptive child behaviour are generally highly stable. This review found that the mean effect size of change between post test and follow-up was 0.01, indicating strong maintenance of change compared with control groups, at least up to 3 years after the end of the programme. 191

Finally, it is possible that our findings may not be generalisable to other parenting interventions. On the one hand, the content and principles of the IY programme are very similar to other social learning theory-based parenting programmes. On the other hand, it is possible that IY differs in process more than content; the marked and explicit emphasis in IY on a collaborative and flexible approach, in which parents design goals and strategies to suit their own family situation, may make it more likely to be beneficial to a wide range of families. Partly this may be a function of how carefully these skills are trained and supervised in the IY programme compared with other programmes. At the same time, some group leaders may acquire these kinds of strong clinical skills from their generic training or from training in other parenting programmes rather than from the IY programme. Further IPD meta-analyses would be needed to establish if a similar pattern of moderator effects is found in other programmes.

List of permitted users – January 2014

Professor Stephen Scott (psychiatrist, coinvestigator, King’s College London)

Professor Frances Gardner (psychologist, coinvestigator, University of Oxford)

Dr Sabine Landau (statistician, coinvestigator, King’s College London)

Professor Andrew Pickles (statistician, coinvestigator, King’s College London)

Professor Jennifer Beecham (economist, coinvestigator, London School of Economics and the University of Kent)

Professor Judy Hutchings (psychologist, Bangor University)

Ms Angela Zhang (Doctor of Philosophy student working on the London data sets with Scott and Landau)

Dr Patty Leijten (research officer, for this data pooling study, University of Oxford)

Dr Joanna Mann (research assistant, for this data pooling study, University of Oxford)

Statistician working with Dr Landau, to be arranged after October 2014

Researcher working with Professor Beecham, to be arranged after October 2014

Design features of the trials

Table 26 shows a list of the trials included in the pooled data set and their design features. Across the trials some offered the IY intervention only in the treated group, whereas others offered a literacy intervention alongside IY. Similarly, the control condition differs across trials, with some offering waiting list or treatment as usual and others offering minimal treatment. As this study is focused upon the IY training programme, participants randomised to a literacy-only intervention were excluded from the pooled data set, as well as participants randomised to the IY parent and IY child programme. Table 1 shows the country where the trial was carried out. The column labelled n gives the total sample size from the trial that was used in the IY pooling data set. The column control group details the type of control condition and the column arms used details the arms included in the pooled sample. The trial may have included arms that have not been used in the pooled sample (e.g. reading intervention only or IY child programme), but these are excluded from Table 1 if they were not included in the pooling study. In the control type, care as usual/no care refers to the fact that no support or services were provided in the control arm other than those normally accessible to the patient during their daily life (in particular in NL-BS, in which mothers were recently released from incarceration; they may not have had access to the services that would normally be available). Minimal intervention means that some non-intensive intervention was provided to parents in the control arm, for example a telephone helpline. Most trials used a waiting list design, in which parents in the waiting list control condition were crossed over to the intervention after 6 months but these families were no longer observed after crossover. A full list of the trials included in the pooled sample and corresponding references is given in Chapter 5, Methods.

Trials included are:

1. NOR: a clinic-based treatment trial in Norway, with children with oppositional defiant and conduct problems (4–8 years). 113

2. SWED: a clinic-based treatment trial in Sweden, with children with oppositional defiant and conduct problems (4–8 years). 114

3. PORT: a clinic-based treatment trial, in Portugal, in a university-based clinic, with middle-class families. This trial took place with children with conduct problems (3–6 years). 115,116

4. IRE: a community-based trial in Ireland, with disadvantaged communities and children with conduct problems (2–7 years). 48

5. NL-BS: a community-based trial in the Netherlands, involving mothers who were recently released from incarceration. Children were not screened for conduct problems (aged 2–10 years). 118

6. NL-SES: a community-based trial in the Netherlands, with families of mainly low SES and immigrant families. Children had conduct problems or relational problems (3–8 years). 96

7. WL-SS: a community-based trial in Wales, in Sure Start areas, with disadvantaged families. Children were showing early signs of conduct problems (3–4 years old). 85

8. WL-FS: a community-based trial of the IY toddler parenting programme in Wales, in Flying Start areas, with disadvantaged families. Children were not screened for conduct problems (12–36 months). 104

9. BIRM: a community-based trial in England. Children were showing early signs of conduct problems (3–5 years). 105,119

10. LON-SPO: a community-based trial in England, investigating IY and a literacy intervention in low-income primary schools, with children with conduct problems (1–6 years). 100

11. LON-PAL: a community-based trial in England, investigating IY and a literacy intervention, in low-income primary schools, with children from disadvantaged families. Children were at high risk of social exclusion (2–7 years). 59

12. LON-HCA: a community-based trial in England, investigating IY and a literacy intervention alone, and IY plus literacy intervention, in low-income primary schools, with children with conduct problems (2–6 years). 101

13. OXF: a community NGO-based trial in England, with children referred with conduct problems (2–9 years). 103

14. LON-NHS: a clinic-based trial in England, with children referred for antisocial behaviour (3–8 years). 120

Items from the Parenting Practices Inventory (trials 1, 3, 6 and 10)

Praise (two items: 6b and 8a)

• In general, how often do you praise or compliment your child when your child behaves well or does a good job?

• Within the last 2 days how many times did you praise or compliment your child for anything he/she did well?

Tangible rewards (four items: 6d–f and 8b)

• How often do you buy something for him/her (such as special food, a small toy) or give him/her money for good behaviour when your child behaves well or does a good job?

• How often do you give points or stars on a chart when your child behaves well or does a good job (excluded in trial 10)?

• How often do you give him/her an extra privilege (such as cake, go to the movies, special activity for good behaviour) when your child behaves well or does a good job?

• Within the last 2 days how many times did you give him or her something extra, like a small gift, privileges, or a special activity with you, for something he/she did well?

Monitoring (five items: 12, 13, 14a–c; excluded in trial 10)

• About how many hours in the last 24 hours did your child spend at home without adult supervision?

• Within the last 2 days, about how many total hours was your child involved in activities outside your home without adult supervision?

• What percentage of the time do you know where your child is when he/she is away from your direct supervision?

• What percentage of the time do you know exactly what your child is doing when he/she is away from you?

• What percentage of your child’s friends do you know well?

Items from the Alabama Parenting Questionnaire (trials 5 and 12)

Praise (four items: 2, 13, 16 and 27)

• You let your child know when he/she is doing a good job with something.

• You compliment your child when he/she does something well.

• You praise your child if he/she behaves well.

• You let your child know that you appreciate the child’s help with chores (excluded in trial 12).

Tangible rewards (one item: 5)

• You reward or give something extra to your child for obeying you or behaving well (excluded in trial 12).

Monitoring (10 items: 6, 10, 17, 19, 21, 24, 28–30 and 32)

• Your child fails to leave a note, or let you know where he/she is going.

• Your child stays out in the evening past the time he/she is supposed to be home.

• Your child is out with friends you don’t know.

• Your child goes out without a set time to be home.

• Your child is out after dark without an adult with him/her.

• You get so busy that you forget where your child is and what he/she is doing.

• You don’t check that your child comes home at the time she/he was supposed to.

• You don’t tell your child where you are going.

• Your child comes home from school more than an hour past the time you expect him/her.

• Your child is at home without adult supervision.

Item from the Parenting Scale (trials 3, 7, 9 and 13)

Praise (none)

Tangible rewards (none)

Monitoring (one item: 13)

• When my child is out of sight, I often don’t know what my child is doing.

Items from parenting interview (trials 10 and 11)

Praise (one item)

• How many times did you praise your child for doing something you asked them or doing something well?

Tangible reward (one item)

• In the last week, how many times did you give a reward to your child for doing what you asked, such as sweets or crisps, extra television, more time playing football or a game he/she likes or a sticker?

Monitoring (none)

Items from parenting interview (trial 12)

Praise (one item)

• How many times per day do you praise (child)?

Tangible reward (one item)

• In the last week, how many times did you give a reward to your child for doing what you asked, such as sweets or crisps, extra television, more time playing football or a game he/she likes or a sticker?

Monitoring (none)

Items from parenting interview (trial 14)

Praise (none)

Tangible rewards (none)

Monitoring

Combined indoor and outdoor supervision. Score (ranging 0–3, recoded into 1–7) based on several items. Examples: How long would it be before you checked where he/she was, if he/she was playing outside? Do you always know where he/she is and with whom he/she is playing, by name?

Item from the Parenting Scale (trials 3, 7, 9 and 13)

Corporal punishment (one item: 18)

• I spank, grab, slap or hit my child most of the time.

Threatening (two items: 7 and 20r)

• I threaten to do things that I know I won’t actually do.

• When I give a fair threat or warning, I always do what I said (reversed: I often don’t carry it out).

Laxness (five items: 16, 19r, 24, 26r and 30r)

• When my child does something I don’t like, I often let it go.

• When my child won’t do what I ask, I take some other action (reversed: I often let it go or end up doing it myself).

• If my child misbehaves and then acts sorry, I let it go that time.

• When I say my child can’t do something, I stick to what I said (reversed: I let my child do it anyway).

• If my child gets upset, I stick to what I said (reversed: I back down and give in).

Shouting (five items: 10r, 17r, 22, 25 and 28)

• I speak to my child calmly (reversed: I raise my voice or yell).

• Things build up and I do things I don’t mean to (reversed: things don’t get out of hand).

• I get so frustrated or angry that my child can see I’m upset.

• I almost always use bad language or curse.

• I insult my child, say mean things or call my child names most of the time.

Items from the Alabama Parenting Questionnaire (trials 5 and 12)

Corporal punishment (three items: 31, 33 and 36)

• You spank your child with your hand when he/she has done something wrong (in trial 12: you smack your child with your hand when he/she has done something wrong).

• You slap your child when he/she has done something wrong.

• You hit your child with a belt, switch or other object when he/she has done something wrong (in trial 12: you hit your child with a belt when he/she has done something wrong).

Threatening (one item: 3)

• You threaten to punish your child and then do not actually punish him/her.

Laxness (three items: 7, 21 and 24)

• Your child talks you out of being punished after he/she has done something wrong.

• You let your child out of a punishment early (such as lift restrictions earlier than you originally said).

• Your child is not punished when he/she has done something wrong (excluded in trial 12).

Shouting (two items: 37 and 38r; excluded in trial 12)

• You yell or scream at your child when he/she has done something wrong.

• You calmly explain to your child why his/her behaviour was wrong when he/she misbehaves (reversed).

Items from the Parenting Practices Inventory (trials 1, 3, 6 and 10)

Corporal punishment (six items: 1h, 1i, 2h, 2i, 3h and 3i)

• How often do you do each of the following things when your child misbehaves?

  Give your child a spanking (in trial 10: give your child a smack).

  Slap or hit your child (but not spanking) (in trial 10: slap or hit your child).

• If your child hit another child, how likely is it that you would discipline your child in the following ways?

  Give your child a spanking (in trial 10 give your child a smack).

  Slap or hit your child (but not spanking) (in trial 10: slap or hit your child).

• If your child refused to do what you wanted him/her to do, how likely is it that you would use each of the following discipline techniques?

  Give your child a spanking (in trial 10: give your child a smack).

  Slap or hit your child (but not spanking) (in trial 10: slap or hit your child).

Threatening (three items: 1d, 2d and 3d)

• How often do you do each of the following things when your child misbehaves?

  Threaten to punish him/her (but not really punish him/her).

• If your child hit another child, how likely is it that you would discipline your child in the following ways?

  Threaten to punish him/her (but not really punish him/her).

• If your child refused to do what you wanted him/her to do, how likely is it that you would use each of the following discipline techniques?

  Threaten to punish him/her (but not really punish him/her).

Laxness (three items: 5a, 5br and 5c)

• If you ask your child to do something and he/she doesn’t do it, how often do you give up trying to get him/her to do it?

• If you warn your child that you will discipline him/her if he/she doesn’t stop, how often do you actually discipline him/her if he/she keeps misbehaving (reversed)?

• How often does your child get away with things that you feel he/she should have been disciplined for?

Shouting (five items: 1b, 2b, 3b, 5e and 5f)

• How often do you do each of the following things when your child misbehaves?

  Raise your voice (scold or yell).

• If your child hit another child, how likely is it that you would discipline your child in the following ways?

  Raise your voice (scold or yell).

• If your child refused to do what you wanted him/her to do, how likely is it that you would use each of the following discipline techniques?

  Raise your voice (scold or yell).

• How often do you show anger when you discipline your child?

• How often do arguments with your child build up and you do or say things you don’t mean to?

Items from parenting interview (trials 10–12 and 14)

Corporal punishment (one item)

• Thinking about last week, how many times did you give your child a tap or smack if he/she misbehaved?

Threatening (none)

Laxness (none)

Shouting (two items: excluded in trials 10 and 14)

• How many days a week do you find yourself raising your voice or shouting at your child or getting angry or cross at him/her?

• On the days that it does happen, how many times do you actually find yourself being critical or shouting at your child?

Differences in Client Service Receipt Inventory coverage

All five studies used a variant of the CSRI177 to record information on to service use alongside the trial. Differences in the level and variation in costs can arise from ‘true’ differences in service utilisation, but also from structural differences in the data collection instrument. The CSRIs for the included studies were scrutinised, and emerging differences thought to potentially impact costs guided the harmonisation effort. Note that CSRI data were available for only four trials, and not necessarily for both baseline and follow-up. Although all studies collected information on the number of service contacts, only two recorded the estimated average duration of contacts. Duration data are often a source of missingness in service use data, and this holds true in the current trials that attempted to collect these.

Although all trials explicitly asked for contacts with key primary care staff (GPs, general practice nurses), hospitals (inpatient, outpatient, A&E) and social workers, there was some variability in the services listed. Although not inherently problematic in a single trial, this may have influenced response patterns between trials, as participants were exposed to different prompts. Additional education support was covered in two trials. Note that this is in part related to the age of the study population.

Four trials provide information on the impact of the child’s behaviour on parental employment, three have some data on parental use of services and two have additional data on the impact on parental time spent caring for the child.

Although all studies provided space for participants to record ‘other’ services beyond those listed on the CSRI, few data sets actually included information on the type of service, with most reporting only the number of contacts. Again, this question was a notable source of missing data when this variable was included.

The setting of service provision has implications for the unit cost. For example, overheads for a hospital-based health professional are higher than if that same contact had taken place in a community setting. Three trials recorded service contacts including their setting at baseline, whereas another provided data on whether or not contacts were ‘usually’ at home. At follow-up, this information was available for all but one trial and this was provided at baseline.

The period prior to randomisation covered was 6 months for three trials, and 12 months for two trials. The time between the baseline assessment and the follow-up data collection point was 6 months for three trials, 3 months for one trial and 12 months for one trial.

Another potential source of difference is whether questions on the CSRI were phrased to include all service contacts or only those relating to a particular problem (‘marginal’ perspective).

Assumptions made at point of data entry and missing data strategies

In some cases, the ‘raw’ data provided by the research teams allowed us insight into the assumptions made when interpreting information provided by participants on the CSRI at the point of data entry. This often related to the number of contacts that were entered. A common assumption was that one service contact took place when no contact number was recorded. Another issue encountered at the point of data entry was to translate a number given for a specific time period (e.g. ‘once per week’) into a number that matched the CSRI time frame (e.g. ‘contacts per term’).

In addition to these ‘implicit’ missing data strategies, for several trials it was noted that missing data appeared to have been dealt with more ‘explicitly’ and comprehensively. In at least one trial it is assumed that all missing values were replaced with zeros.

It is worth noting that it may not have been possible to recognise and identify all such strategies that may have been applied to data, and it is impossible to know whether or not these have been applied consistently within a particular trial, especially given that often more than one person is responsible for data entry.