Systematic review of the effects of schools and school environment interventions on health: evidence mapping and synthesis

Research question 1

What theories and conceptual frameworks are most commonly used to inform school environment interventions or explain school-level influences on health? What testable hypotheses do these suggest?

Research question 2

What are the effects of school environment interventions (interventions aiming to promote health by modifying the school's physical, social or cultural environment through actions focused on school policies and practices relating to education, pastoral care, sport, extracurricular activities, catering, travel to and from school and other aspects of school life) evaluated using experimental and quasi-experimental designs, compared with standard school practices, on health (physical and emotional/mental health and well-being; intermediate health measures such as health behaviours, BMI and teenage pregnancy; and health promotion outcomes such as health-related knowledge and attitudes) and health inequalities among school staff and students aged 4–18 years? What are their direct and indirect costs?

Research question 3

How feasible and acceptable are school environment interventions? How does context affect this?

Research question 4

What are the effects of other school-level factors on health and health inequalities among school staff and students aged 4–18 years, examined using multilevel and ecological (school) designs?

Research question 5

Through what processes might these school-level influences occur?

Database searching

Inclusion and exclusion criteria

We included quantitative, qualitative and theoretical literature in the evidence map that theorised or empirically examined the effect of the school social and/or physical environment, interventions to address this and/or processes underlying these effects or interventions (not including the provision of health education or health-related goods or services) on the health or well-being outcomes of students (age 4–18 years) or staff. All references from the searches were uploaded into ER4 and duplicate references were removed (those scoring ≥ 0.85 on ER4's similarity score). Exclusion criteria were developed (Table 1) to remove irrelevant references and thereby identify relevant references. References were screened on title and abstract. A round of pilot screening was conducted by two reviewers on a sample of 200 abstracts to test and refine the criteria. The remaining references were divided between six reviewers (CB, HW, AH, CV, MP and FJ) and screened independently. After each reviewer had screened 2000 references, a random sample of 10% (n = 200) were double screened by another reviewer to ensure consistency in applying the criteria. A threshold of < 20 disagreements per 2000 references on whether to include/exclude was established.

It should be noted that, although these criteria were applied to most of the references, because of time constraints and the large number of references, those that were obviously to be excluded were marked ‘exclude only’ and not assigned an exclusion code.

Evidence map: coding references

Included references were descriptively coded based on title and abstract. Descriptive coding involved identifying the following characteristics of each study:

• relevance to RQs 1, 2, 3, 4 and/or 5

• type of research

• country where research was undertaken

• research design

• target population

• health topic examined

• level of school (e.g. high school, elementary school).

A round of pilot coding was conducted by four reviewers (CB, AH, HW and FJ) on a random sample of 40 references to ensure that the list of characteristics captured was comprehensive and relevant. Two reviewers (FJ and HW) double-coded the remaining references to ensure consistency in coding. Discrepancies were resolved by discussion.

As a result of the large number of references included at this stage, references were coded for the evidence map on the basis of title and abstract only. When screening references for inclusion in the evidence map we erred on the side of inclusion and so there were inevitably errors of overinclusion.

Theory map

A map of theories and conceptual frameworks used to inform school environment interventions or explain school-level influences on health was developed alongside the evidence map. We looked for theories while coding the first half of the references for the evidence map to obtain a broadly representative sample of theories. Those theories that were ‘named’ (e.g. ‘social learning theory’) and which were referenced in multiple references were identified. Summaries of the included theories were obtained through a Google search or were extracted from the original texts where they were first published.

Consultation with stakeholders

To refine the RQs and focus the review, we consulted with key stakeholders regarding the review topic and evidence and theory map.

Flow of literature: from database searching to evidence map

A total of 82,775 references were retrieved from the database searching. Of these, 20,446 were identified as duplicates: either ‘exact’ matches (n = 19,132) or very close matches (n = 1314). The remaining 62,329 references were screened on title and abstract and 61,185 (98.2%) were excluded. In total, 1144 references were included in the evidence map (Figure 1).

FIGURE 1.

Flow of literature stage 1: evidence and theory map.

Evidence map

The 1144 references were descriptively coded based on title and abstract to identify relevant characteristics of references. Because the references were coded for inclusion on title and abstract only, there are inevitably errors of overinclusion. Nonetheless, the evidence map provides a useful overview of the available evidence to inform our decisions about what references to prioritise for in-depth review in stage 2.

Relevant research question(s) of references included in the evidence map

Figure 2 indicates to which RQs references might be relevant. The total number of references, displayed in this figure, does not equal the total number of included references as categories were not mutually exclusive.

FIGURE 2.

Potential RQ(s) addressed by references in the evidence and theory map. RQ1: literature describing/explaining the theories and conceptual frameworks that were used to inform school environment interventions or explain school-level influences on health. RQ2: evaluation references reporting on school environment intervention effects on health, as well as cost, economic and econometric references examining the costs of school environment interventions. RQ3: process evaluations of school environment interventions. RQ4: multilevel or ecological (school) references examining school-level influences on health. RQ5: qualitative references exploring the process by which school-level factors might influence health.

Country where research was undertaken

Figure 3 provides the distribution of research for the top 10 countries where research was conducted. Some references did not report the country of research in the title or abstract (n = 433). The total number of references, displayed in this figure, does not equal the total number of included references as categories were not mutually exclusive.

FIGURE 3.

Countries of primary research of studies included in the evidence map.

Target population and health topics examined

Students were the target population studied for nearly all of the references included in the evidence map (n = 1093), with only 52 titles/abstracts mentioning staff as the target population.

Most of the health topics identified were about violence, bullying and/or harassment (n = 278), eating/drinking (non-alcoholic; n = 270) or physical activity (n = 257). Figure 4 provides the distribution of the different health topics examined among included references. The total number of references displayed in the figure does not equal the total number of included references as categories were not mutually exclusive.

FIGURE 4.

Health topics of the references included in the evidence map.

School level/grade level reported

Most titles/abstracts did not report the school/grade level studied (n = 519). Of those that did, the majority of the research was conducted at elementary/primary schools (n = 167/125) or high/secondary schools (n = 126/117). Figure 5 provides the distribution of the different school/grade levels examined. The total number of references, displayed in the figure, does not equal the total number of included references as categories were not mutually exclusive.

FIGURE 5.

School/grade level of the references included in the evidence map.

Aspect of the school examined

Figure 6 provides the distribution of the different aspects of schools examined among included references. Most reports focused on school management or polices (n = 273), catering or vending (n = 216) and sport or active transport (n = 182). We were unable to determine the aspect of the school examined for about one-quarter of the references based on the title/abstract alone (not clear n = 204; not reported n = 157). The total number of references, displayed in the figure, does not equal the total number of included references as categories were not mutually exclusive.

FIGURE 6.

Aspect of the school examined in the references included in the evidence map.

Theory map

A total of 12 theories/models were identified from the theory map. These include anomie theory, attachment theory, differential association theory, ecological systems theory, educational transmission of class theory, the health belief model, social cognitive theory, social control theory, the social development model, the social learning model, strain theory and the theory of reasoned action.

The theories suggest the potential importance of school-level determinants concerning or interventions addressing:

• how schools structure norms (anomie theory)

• relationships between staff and students (attachment theory, social learning theory)

• the roles and opportunities that schools give to or withhold from students (social development model, strain theory)

• teaching and learning (educational transmission of class theory)

• school rules (social control theory)

• health education (health belief model)

• a combination of these (ecological systems theory).

Stakeholder consultation

The stakeholder consultation with policy-makers, teachers and academics suggested that we needed to define the school environment more clearly because otherwise it might be assumed to be the physical environment only. The consultation group suggested that, based on the presentation of the evidence map, we narrow our focus to:

• policy and management (policies, systems)

• social relationships (including staff–student, student–student and staff–staff relationships)

• student culture (sense of connection, engagement and aspiration)

• staff culture (values, vision, priorities, ethos, leadership)

• physical environment (school grounds).

These stakeholders suggested that they would most value a synthesis of evidence on the effects of schools' ‘core business’ on student health in terms of (1) learning and teaching, (2) pastoral care and (3) discipline policies/practices. The mental health of teachers was also considered important by the stakeholders with whom we consulted.

Young people told us that being healthy and well meant feeling safe and secure, having personal confidence, feeling self-assured and having the support of friends and family. Young people suggested that schools affect their health and well-being in various ways and emphasised the importance of:

• class size (e.g. large classes may mean less personal support, although some young people also thought small classes could be stifling)

• staff attitudes (e.g. having to spend time with teachers in a ‘bad mood’ was unhealthy)

• choice and empowerment at school (e.g. having a say in the running of schools)

• social class composition (e.g. students from poorer backgrounds may feel or be made to feel out of place)

• socialising (e.g. making friends, meeting people from different backgrounds)

• teaching and learning (e.g. ‘making you smarter’, ‘opening your mind’)

• hygiene (e.g. ‘disgusting toilets’)

• school meal options and prices (e.g. healthy food is often more expensive).

Young people thought the following were the most important things that a school could do to improve student well-being:

• reduce class sizes

• foster a positive attitude in teachers and good relationships between staff and students

• focus less on ‘league tables’ and more on ‘learning for learning's sake’

• increase opportunities for students to focus on what they are interested in or good at

• provide more sources of social support for students.

The young people we consulted with cited the following as important outcomes: good relationships (especially with teachers), anxiety, self-image and ‘overachieving’. This informed our decision to focus on aspects of schools' ‘core business’ of teaching, pastoral care and discipline. Based on these consultations we decided not to focus on school activities such as extracurricular activities, catering and vending of food and drinks, physical education (PE) and active transport to and from school.

Research questions for stage 2: in-depth review

Additional searches

Additional searches were conducted by screening the reference lists of all reports from the evidence map that were included in the in-depth review; contacting authors of included references for additional references; and asking Cochrane review collaborators for additional references. References published before June 2011 from the additional searches were considered for inclusion in stage 2.

Inclusion and exclusion criteria

Thirteen stand-alone theory references were identified from the evidence and theory map as relevant to the theory synthesis. We included literature in the in-depth synthesis that theorised how the school social or physical environment (defined in terms of how schools are organised and managed, how they provide teaching, pastoral care or discipline, and schools' physical environment) affects students' health or well-being. An additional two references56,57 were sourced through reference checking. The full-text reports of these references were retrieved and the following exclusion criteria were applied independently by two reviewers (there were no discrepancies to be resolved):

• exclude reports that do not propose an abstracted, generalisable way in which core features of schools and school environment interventions are causally related as (1) a stand-alone theory, (2) a general theory of school health or (3) a theory addressing school influences on health

• exclude reports that are not written in English.

Quality assessment

The descriptions of the theories were extracted from included reports. We then obtained the original source of the theory and used this as a focus for quality assessing theories. The criteria for quality assessing theories are as follows:

• whether or not the constructs are well specified

• whether or not clear causal pathways are specified between constructs

• whether or not it was a simple theory/model

• whether or not it suggested which specific aspects of the school institution might influence health

• whether or not it is applicable to multiple health domains

• what the theory/model assumptions are

• whether these assumptions are implicit or explicit.

We developed these ourselves having searched for but not found existing criteria to assess the quality of theories. Our criteria were intended to determine which theories to use to inform the development of overall hypotheses for the review; these focus on the internal logic of each theory (well-specified constructs; clear pathways; simple; explicit assumptions) and its applicability to understanding school effects on health (which specific aspects of an institution influence health; applicable to multiple health domains). Some of these criteria were necessarily subjective, calling for researcher judgements, for example about whether or not a theory was simple (i.e. parsimonious). The quality assessment criteria were piloted on a random sample of two theories by two reviewers (CB and HW) before being applied by one reviewer (CB) and checked by another (HW), with any differences being settled by discussion.

The quality criteria were used to categorise theories as either primary or secondary theories, which in turn inform our primary and secondary review hypotheses. Theories were not excluded based on quality scores. We used these quality criteria to form a judgement about which theories to draw on to define our primary and secondary review hypotheses. We did not simply require that a theory meet every criterion to be deemed ‘primary’ because our judgements were necessarily more subtle than this.

Data extraction

We extracted data related to the name of the theory, its originator, the year of origin, what constructs and pathways it involved, its disciplinary origins and whether it is linked to any higher-order or lower-order theories. The data extraction tool was piloted by two reviewers (CB and HW) on a random sample of two theories before being applied by one reviewer (CB) and checked by another (HW), with any differences being settled by discussion.

Synthesis

We summarised the primary and secondary theories in tables and used these to inform the development of hypotheses for our review, which we assess against the empirical evidence reviewed and report in Chapter 11.

Flow of literature

Of the 13 references identified from the evidence and theory map, only one62 was included. The other 12 references were excluded because they did not report a stand-alone theory (n = 9), did not report a general theory of school health (n = 2) or did not address school influences on health (n = 1). Two reports identified through citation chasing were included. 56,57 All empirical reports included in RQ2–5 were screened for reference to any theories. A total of 35 reports were identified through this search and were included in the in-depth synthesis. Thus, 38 reports, which reported on 24 theories, were included in the in-depth review (Figure 7).

FIGURE 7.

Flow of literature: theory synthesis (stage 2: in-depth synthesis).

Quality assessment

We judged that three of the 24 theories (ecological systems theory, ecological model of co-ordinated school health programmes and theory of structuration) did not have clear or narrowly specified constructs in the sense that these might be operationalised in empirical research, but that the rest (n = 21) did. We judged that nearly all of the theories used a reasonably small number of components and a simple framework of inter-relations between them to understand potential school effects on health (n = 20). Four were categorised as more complex (ecological systems theory, ecological model of co-ordinated school health programmes, human functioning and school organisation and the theory of triadic influence). We judged causal relations between constructs as being clearly specified in all but three of the theories (contagion theory, ecological systems theory and ecological model of co-ordinated school health programmes). Fourteen of the 24 theories were judged not applicable to multiple health domains: 11 applied only to delinquency outcomes, two to public behaviour only and one to violence only. Moral authority theory, social control theory, deterrence theory, the integrated perspective on delinquent behaviour and strain theory were all judged as relevant to understanding health outcomes that are associated with antisocial behaviour (ASB) only. These theories are not relevant to understanding other health outcomes such as healthy eating and physical activity.

We judged that three theories fully met the criterion of whether or not the theory suggests which specific aspects of the school might influence health. Another 10 partially met it.

Study characteristics

Twenty-four theories were identified in the in-depth theory synthesis. These were cited in a total of 38 reports53,56,57,62–96. The theories most commonly cited in empirical reports were ecological systems theory68,75,77,78,80,83,85,90,96 (n = 10), social control theory71,73,80,82,97 (n = 6), social disorganisation theory70,79,88,89,97 (n = 5), social learning theory71,76,81,94 (n = 4), the theory of human functioning and school organisation53,64–66,95 (n = 5) and social cognitive theory63,76,77,83 (n = 4).

Primary theories that explain the mechanisms by which schools determine health

We identified three theories to be most useful, informing the primary hypotheses to use in interpreting findings from the review's empirical studies. These met all of our quality criteria, with the exception of the theory of human functioning and school organisation, which did not meet our criterion of simplicity. However, we judged that this theory was very strong regarding our other criteria and so included it.

Secondary theories that explain the mechanisms by which schools determine health

We identified a further 10 theories that could be used to suggest our secondary review hypotheses. Several of these theories did not meet our quality criterion of addressing a range of health outcomes but we judged them to be sufficiently useful in understanding school effects on ASB-related outcomes, hence we included them. The theory of triadic influence was included despite not meeting our quality criterion of simplicity because of its clarity and comprehensiveness, while the ecological model of co-ordinated school health programmes was included despite not meeting our quality criteria of simplicity, operationalisable constructs and clear pathways because, unlike all other theories, it attended to certain aspects of the school environment, such as school safety and opportunities for physical activity.

Flay's56 triadic theory of health behaviours suggests that health behaviours are influenced by factors from three domains: intrapersonal factors (social competence and sense of self), socioenvironmental factors (behaviours of others and bonding to others) and the broader cultural environment (information and opportunities about behaviours and culture/religion). Each of these streams has distal and proximal elements moving from the social–personal nexus to expectancies and evaluations and to cognitions and affect, with dynamic inter-relationships between these. Attitudes, socially normative beliefs and self-efficacy determine decisions/intentions and behaviour. Although his theory offers a comprehensive framework for understanding how the influences on health behaviours inter-relate, it does not aim to offer a specific theory of how the school institution structures these factors. Nonetheless, it does suggest the hypothesis that schools may foster healthy behaviours by providing access to knowledge through health education, but also by reducing students' opportunities for engaging in risk and setting rules against risky behaviours, by providing opportunities for bonding with staff and other students, and enabling the development of social skills through health and general education.

Eight theories outline why certain young people may engage in antisocial or risk-taking behaviours. Social control theory107 suggests that individuals with a stake in a particular community will avoid committing acts considered deviant within that community. This might suggest the hypothesis that schools can reduce ASB by giving students some stake in their school community, perhaps by increasing their participation in decisions. The integrated perspective on delinquent behaviour108 suggests that delinquency will be greater among individuals who have experienced a failure to participate in conventional social settings. This would suggest the hypothesis that schools can reduce ASB by ensuring that all students experience success in school activities. Problem behaviour theory109 suggests that young people engage in behaviours such as drug use or risky sexual behaviour to cope with problems dealing with their wider system of conventional behaviour such as educational failure and low self-esteem. This would inform a hypothesis that schools could reduce ASB by ensuring that students' educational and social problems are addressed. Strain theory110 suggests that individuals may engage in ASB when they experience a strain between achieving what they regard as socially legitimate goals and their ability to achieve these through socially legitimate means. Thus, we might hypothesise that schools with lower rates of ASB are better at ensuring that students can achieve their broader goals through school activities.

None of the above theories considers the specific means by which schools may affect these mechanisms. Other theories go a little further towards suggesting what particular aspects of an institution might determine behaviour. For example, deterrence theory111 would suggest the hypothesis that individuals will be deterred from behaviours if these are met with certain, severe and rapid punishments. Similarly, although the theories of reasoned action112 and the theory of planned behaviour113 do not consider how the school environment is likely to influence health, they do suggest that behaviours that are the subject of clear sanctions within schools might be inhibited by encouraging students' acceptance of institutional norms and motivation to conform. This would suggest the hypothesis that schools with strict and strongly enforced codes against activities such as smoking, drinking and violence have lower rates of these outcomes. In contrast, moral authority theory114 argues that a prime aim of schools is to inculcate respect for the specific rules of the school as well as broader rules of social behaviour. However, this would not need to occur through strict enforcement and severe punishments, because acceptance of the rules can be internalised without recourse to such formal processes. This theory would suggest the hypothesis that schools with lower rates of ASB have a positive ethos and do not necessarily have strict rules, although the theory does not offer suggestions as to what system of organisation would be required to foster this positive ethos.

The ecological model of co-ordinated school health programmes,57 although not offering a very deep understanding of how institutions affect health, does direct attention to particular aspects of schools that might promote health across multiple domains. This model would suggest a hypothesis that schools can foster health by promoting a supportive psychosocial environment and safe facilities, as well as opportunities for physical activity within the school.

Summary of key findings

Twenty-four theories were identified in the in-depth theory synthesis. The theories most commonly cited in empirical reports were ecological systems theory, social control theory, social disorganisation theory, social learning theory, the theory of human functioning and school organisation and social cognitive theory. We considered several criteria to decide which theories to use to inform our primary and secondary review hypotheses.

Table 3 indicates the testable hypotheses that the primary and secondary theories we identified might suggest.

Strengths and limitations

Our initial summary of theoretical literature in stage 1 was relatively unsystematic: we noted theories that recurred in the first half of our coding for the evidence map but not in the second. The preliminary summary identified only five of the theories that were identified in the in-depth synthesis. Nonetheless, this provided us with some insights into the range of theories informing the empirical studies. Along with the evidence map it enabled us to have a lively discussion with stakeholders about which types of evidence it would be most interesting and useful to review in depth in stage 2. We cannot rule out the possibility, however, that a more comprehensive summary of theory at this stage would have led to different priorities.

Our summary and assessment of theories in stage 2 was systematic, using a tool of our own devising. The judgements we made were to some extent subjective, for example in determining whether or not a theory was simple and had constructs that could be operationalised in empirical research. We used these multiple criteria to form a judgement about which theories to draw on to define our primary and secondary review hypotheses. We did not simply require that a theory meet every criterion in order to be considered primary because our judgements were necessarily more subtle than this. For example, the theory of human functioning and school organisation did not meet our criterion of simplicity; however, we judged that this theory was very strong regarding our other criteria and so included it. As a further example, several of our theories did not meet our quality criterion of addressing a range of health outcomes but we judged them to be sufficiently useful in understanding school effects on ASB-related outcomes and so we opted to use them to inform secondary review hypotheses. We think this balance between using clear criteria and making overall judgements is acceptable and appropriate given that these concerned the development rather than the testing of hypotheses.

Despite its subjectivity, this process was useful in determining which theories could most usefully provide hypotheses to assess against the empirical reviews. These theories enable us to develop hypotheses about how school environment interventions and school-level exposures might affect health, but did not enable us to focus on specific prehypothesised outcomes.

The theories themselves were biased towards those focusing on ASB, with six of our secondary theories but no primary theories having this focus. However, this reflects the theories that were used in empirical studies of the health effects of schools and school environment interventions and is an interesting finding of our review.

Inclusion and exclusion criteria

A total of 524 references were identified in the evidence and theory map as relevant to RQ2. We included experimental and quasi-experimental evaluations of the effects on health or well-being outcomes in students (aged 4–18 years) of school environment interventions that addressed how schools were led and managed, how they teach, support or discipline students, or schools' physical environment. Two reviewers (CB and HW) independently double-sifted these references based on title and abstract only and on full reports where necessary using the exclusion criteria in Table 4. It should be noted that all references coded as outcome evaluations were screened for inclusion because of the potential limitations of the accuracy of the coding in the stage 1 evidence map. Screening was not hierarchical or mutually exclusive. In other words, references may have been excluded based on multiple criteria.

Quality assessment

All included reports were quality assessed using the following criteria:

• whether or not the impact of the intervention was reported for all outcomes

• whether or not allocation to the intervention and comparison conditions was random

• whether allocation was non-randomised but involved matching or adjustment for potential confounders

• whether or not there were equivalent study groups at baseline

• whether or not attrition overall is ≤ 30%

• whether or not attrition rates differed by ≤ 10% between groups

• whether or not analysis accounted for cluster.

These criteria used for assessing methodological quality were adapted from those used in EPPI-Centre health promotion reviews. 115 These allowed us to assess which studies were most likely to provide us with estimates of the effects of school environment interventions minimally biased by confounding, selection and information bias and random error. Based on the results of these criteria, reviewers judged the study overall as well conducted or less well conducted.

These quality assessment criteria were piloted on a random sample of two reports by two reviewers (CB and HW) before being applied by one reviewer (HW) and checked by another (CB), with any differences being settled by discussion without recourse to a third reviewer. These criteria were not used to include or exclude reports from the in-depth review in stage 2, but they were used to determine what weight we gave study findings in our narrative synthesis.

Data extraction

For the included RCT and quasi-experimental study reports we extracted data related to intervention (description, theory, length, providers internal/external to school), comparison group, study RQs/hypotheses, study site and population, sampling, schools and individuals per arm, data collection methods, analysis methods, results and authors' conclusions. We also extracted data on adjustment for clustering, confounders and methods to control these, waves of follow-up and attrition rates overall and by study arm, outcome measures, effect size estimates (overall and by subgroup) and measures of confidence/significance, intervention costs and indirect resource use and economic measures of cost-effectiveness. Data extraction tools were piloted on a random sample of two reports by two reviewers (CB and HW) before being applied by one reviewer (HW) and checked by another (CB), with any differences being settled by discussion.

Synthesis

We assessed the potential for statistical meta-analysis of intervention outcome evaluations by noting which reports focused on similar combinations of school-level interventions and outcomes. We set a threshold of a minimum of three such reports being required to consider meta-analysis because meta-analysis of only two studies is unlikely to produce stable effect estimates. Valentine et al. 116 suggest that meta-analysis of two studies will be informative only when studies are direct replications. With very few studies, parameter estimation such as point estimates will likely be poor, rendering conclusions uncertain. We found no homogeneity of interventions and outcomes whatsoever for our intervention studies. Therefore, we subjected our quantitative study reports of intervention effects to narrative synthesis. In conducting our narrative synthesis of intervention outcome reports we included all reports in the narrative but made clear where studies were subject to major methodological limitations.

There were three aspects to our narrative synthesis of evidence from intervention outcome evaluations. First, we prepared a narrative summary of the findings from each study. We summarised reported effects in the context of study quality, setting and population. This is reported in the results section of this chapter. Second, we developed a narrative overview of findings from across our outcome evaluations. We noted where studies of similar interventions or influences reported convergent or divergent findings and considered the reasons for this in terms of design and context, rather than merely weighing the overall frequency of differing findings. This is reported in the discussion section of this chapter. Finally, we examined the implications of the findings from outcome evaluations alongside those from other forms of evidence reviewed for our review hypotheses. Because our review hypotheses were worded generally rather than specifically in terms of precise constructs, and because empirical studies had not been designed to test our review hypotheses, we did not test hypotheses in a statistical sense but in a more general narrative sense, to the extent to which these findings supported or undermined the review hypotheses. This is reported in Chapter 11.

Flow of literature

Of the 524 references identified in the evidence map, 514 were excluded: 83 were not an outcome evaluation, 47 were process evaluations only, 251 did not meet our criterion of a school environment intervention addressing school organisation or management, teaching, pastoral care or discipline, or physical environment (e.g. they included health education curricula, or addressed school catering, PE, etc.), 134 were not a cluster RCT or non-randomised prospective study, 85 were related to curriculum or parents and nine were excluded for other reasons (one could not be located, one was a duplicate, two had no relevant health outcomes, one was about teacher health and four were not related to our topic) (all of the study reports were written in English) (Figure 8).

Two RCT reports relating to the same study were suggested by our Cochrane colleagues,54,55 one quasi-experimental study was suggested by CB59 and three references were located by reference sifting. 58,117,118 In total, 10 studies (16 reports) from the evidence map were included in the in-depth synthesis (of which six studies were RCTs and four were non-randomised outcome evaluations).

FIGURE 8.

Flow of literature: outcome evaluation synthesis (stage 2: in-depth synthesis).

Quality assessment

The methodological quality of the studies was judged against the seven quality criteria. All 10 studies44,54,55,58,63,119–122 were deemed suitable to include in the narrative synthesis, although the quality of the studies varied considerably. Six studies employed a RCT design, although this was compromised in Flannery et al. 119 by the replacement of a dropout school with a non-randomised substitute. Four studies involved a quasi-experimental prospective comparison group design. Those reported by Sallis et al.,122 Dzewaltowski et al. 63 and Flay et al. 44 were the strongest studies in terms of design, sample size and adjusting for clustering in the analysis.

All of the RCT studies except for Flay et al. 44 involved arms that compared school environment interventions with normal practice. Fonagy et al. 120 also involved a third arm that was not relevant to our RQs. Flay et al. 44 included three arms: school environment intervention plus curriculum, curriculum only and normal practice. We included this trial because comparison of the first two arms enabled us to examine the effects of school environment over and above the effects of the curriculum element and so this was analogous to comparing school environment only with normal practice.

Study characteristics

Ten RCT reports44,54,55,58,63,119–123 examining six interventions and six reports60,117,118,124–126 from quasi-experimental outcome evaluations examining four interventions were included in the in-depth review. They were published between 1988 and 2010 and evaluated multicomponent interventions delivered in elementary and middle schools in the USA and elementary (primary) and secondary schools in England. The interventions fell into three categories: five aimed to encourage staff and students to develop school climates characterised by a stronger sense of community and/or better interpersonal relations to reduce aggression and other risk behaviours;44,54,55,58,60,119–121,123 two encouraged staff and students to advocate for school environments promoting healthier eating and physical activities;63,122 and three involved improving school playgrounds. 117,118,124–126 All of the interventions were universal in addressing all students. None of these studies reported any data on cost-effectiveness.

Narrative summary of findings from randomised controlled trials

Interventions that encouraged staff and students to develop school climates characterised by a stronger sense of community and better relationships appeared to bring significant benefits for some but not all health outcomes. However, most of these evaluations had important limitations, and the strongest evaluation in this category reported the fewest significant benefits.

Battistich et al. 55 report that the CDP intervention had positive effects on three of the four measures of cognitive social problem-solving in conflict situations among grade four US elementary school students when followed up at grade four. There was an interaction between arm and grade indicating that the intervention effects increased between kindergarten and grade four, suggesting that benefits were cumulative. Battistich et al. 54 report no programme effects for self-esteem at grade four but benefits for loneliness/social dissatisfaction as well as social anxiety at grade six. Solomon et al. 58 also report a positive effect for teacher-reported students' solving of minor interpersonal problems, drawing on cumulative data from kindergarten through to fourth grade. Battistich123 reports intervention effects reducing student self-reported loneliness/social dissatisfaction and social anxiety at grade six but not self-esteem at grade four. However, this evaluation had a number of weaknesses. Although the evaluation involved a RCT design, baseline data were not collected; it is therefore not possible to assess whether or not study groups were equivalent at baseline and so it is impossible to assess the risk of confounding. Furthermore, attrition was high overall and differed markedly between groups so that selection bias is a risk. Finally, no account is taken in the analysis of the clustering of data within schools. Therefore, the results of this study should be interpreted with caution.

Flay et al. ,44 reporting on the AAYP in Chicago elementary and middle schools, compared the arm of their RCT involving a school environment component plus curriculum with the curriculum-only arm. Among boys there were non-significant trends in the direction of benefit for all of the health outcomes (violence, recent sexual intercourse, condom use) at the 3-year follow-up in grade eight. However, only for two non-health outcomes (school delinquency and an overall aggregate measure combining the health outcomes with school delinquency and provoking behaviours) were the differences statistically significant. Among girls there was a significant increase in self-reported condom use (p = 0.03), but no effects for other measures. This was a well-conducted RCT with little risk of confounding or bias.

Fonagy et al. 120 report that, compared with no intervention, the CAPSLE intervention reduced the developmental trend of increasing peer-reported victimisation (p < 0.01), peer-reported aggression (p < 0.05), self-reported aggression (p < 0.05) and peer-reported aggressive bystanding (p < 0.05) and increased peer-reported helpful bystanding (p < 0.001) among third to fifth graders in US elementary schools at the 2-year follow-up. CAPSLE also reduced a decline in self-reported empathy (p < 0.01). These benefits were not maintained at the 3-year follow-up other than for peer-reported helpful bystanding (p < 0.05). Results for self-reported victimisation and self-reported beliefs in the legitimacy of aggression did not suggest intervention benefits at 2 or 3 years' follow-up. This evaluation involved a RCT design but with high attrition that differed between groups so that selection bias is a risk. No account is taken in the analysis of the clustering of data within schools. Therefore, the results of this study should be interpreted with some caution.

Regarding the PeaceBuilders intervention delivered in US elementary schools, Flannery et al. 119 report that, at the 1-year follow-up, compared with students in control schools, teachers rated third to fifth grade but not K–2 grade students in intervention schools as significantly lower in log-aggression, and there were no intervention effects on child self-reported aggression. The intervention effect for teacher-rated aggression was larger for students with higher aggression scores at baseline. However, subgroup effects by socioeconomic status (SES), ethnicity or sex are not reported and a test for interaction with baseline aggression was not performed. For the same intervention, Krug et al. 121 report that, overall, injury-related and fighting-related as well as non-fighting-related rates of visits to the school nurse decreased significantly more in intervention than in comparison schools at the 1-year follow-up while visits from injuries possibly related to fighting did not differ. This evaluation involved a RCT design. One comparison school dropped out at baseline and was replaced with another non-randomised school suggesting that, along with high attrition, which differed between groups, there is some risk of selection bias. The results of this study should therefore be interpreted with some caution.

Both interventions that aimed to address physical activity and healthy eating appeared to affect the former but not the latter. Dzewaltowski et al. 63 report that, immediately after the HYP intervention, US middle schools did not change their fruit and vegetable consumption but did significantly change their levels of physical activity compared with control schools. From sixth to eighth grade the intervention was associated with increases in some but not all measures of vigorous physical activity [vigorous physical activity (VPA), p = 0.03] and moderate/vigorous physical activity [moderate/vigorous physical activity (MVPA), p = 0.005]. There were significant increases in students' efficacy to ensure that schools were creating supportive environments for physical activity as well as their self-efficacy to be physically active, but not in their efficacy with regard to parent or peers creating supportive environments for physical activity. The intervention did not have effects on any outcomes relating to healthy eating other than their own self-efficacy to eat five to seven portions of fruit and vegetables per day and group norms supportive of fruit and vegetable consumption. Sex, race or SES did not interact with the intervention effects over time. Inclusion of a measure of student proxy efficacy in the model for the effect of the intervention on MVPA and VPA suggested that this was a mediator for intervention effects (p = 0.05 and 0.03 for MVPA and VPA respectively). This evaluation involved a RCT design but with high attrition that differed between groups so that selection bias is a risk. The results of this study should therefore be interpreted with some caution.

In US middle schools (student grades not reported), Sallis et al. 122 report that there was a significant intervention effect of the M-SPAN intervention at the 1-year follow-up for observed physical activity for the total group (p < 0.009) and for boys (p < 0.001), but not for girls (p < 0.40). The intervention was not effective for consumption of total fat (p < 0.91) or saturated fat (p < 0.79) at 2 years' follow-up. The intervention did not have a significant effect on self-reported physical activity or sedentary behaviours. Similarly, there was no intervention effect on fatty foods consumed or parental fat avoidance. There was a significant reduction in BMI among intervention boys compared with control boys (p = 0.044), but there was no effect for girls. This was a well-conducted RCT; however, the reported differences in effect were not subject to a test for interaction and so should be interpreted cautiously.

Narrative summary of findings from quasi-experimental studies

In English secondary schools, Bonell et al. 60 report that at the 1-year follow-up there were significantly higher reported rates of feeling safe at school among year 7 students in intervention schools [adjusted odds ratio (OR) 2.89; 95% confidence interval (CI) 1.63 to 5.15; p < 0.001] as well as differences that were of borderline significance (adjusted p < 0.1) in self-reported measures of teasing of others, hurting of others and been in a fight. There were no significant differences regarding self-reported measures of being teased or threatened weekly/hurt ever in this school, belief that will try illegal drugs or smoke a cigarette in the future or belief that will get drunk before age 16 years. This was a small pilot study involving a non-random design and so its findings should be interpreted with caution.

For their playground physical structure intervention delivered in English elementary schools, Ridgers et al. 125 report that, at the 6-week post-intervention follow-up, there were no significant differences between students in intervention and comparison schools in heart rate telemetry-derived and accelerometer-derived measures of physical activity. However, another study by Ridgers et al. 124 reports that at the 6-month follow-up the rates of four outcomes were significantly higher among students at intervention schools after adjustment for possible confounders: MVPA by heart rate (beta coefficient = 4.03; p = 0.042); VPA by heart rate (beta coefficient = 2.43; p = 0.045); MVPA by accelerometry (beta coefficient = 4.53; p = 0.025); and VPA by accelerometry (beta coefficient = 2.32; p = 0.005). At the 12-month follow-up, after adjustment, there were significant differences for MVPA by accelerometry (risk difference = 1.2; 95% CI 2.8 to 5.2) and VPA by heart rate (risk difference = 0.9; 95% CI 1.6 to 3.5). There were no significant differences for MVPA by heart rate, vigorous activity by heart rate, VPA by accelerometry measured at morning recess and MVPA by heart rate and VPA by heart rate measured at lunch recess. Ridgers et al. 125 report that, at the 6-week post-intervention follow-up for MVPA, there was a significant interaction between the intervention and age (p = 0.01), with intervention effects stronger for younger children, and an interaction of borderline significance for recess duration (p = 0.07), with effects stronger for longer recess period, but no interaction for baseline MVPA, sex or BMI. For VPA, there was a significant interaction between the intervention and age (p = 0.09), with intervention effects stronger for younger children, but no interaction for recess duration, baseline VPA, sex or BMI. Ridgers et al. 124 report that a positive interaction occurred between the intervention and recess duration for both MVPA and VPA (p < 0.05). Inverse interaction terms occurred between the intervention and baseline heart rate-measured MVPA and VPA (p < 0.05 and p < 0.1 respectively), indicating that the impact was stronger for children who were less active at baseline. An inverse interaction between the intervention and age occurred for MVPA (p < 0.05), suggesting that the impact was stronger for younger children. A positive interaction occurred between the intervention and time for VPA (p < 0.05), suggesting that the impact strengthened across time. Ridgers et al. 126 report significant interactions between the intervention and time for MVPA measured by heart rate and VPA measured by heart rate at morning recess, and MVPA measured by heart rate, VPA measured by heart rate and MVPA measured by accelerometry at lunch recess. These indicate that physical activity levels were lower at the 12-month follow-up than at the 6-month follow-up, with the greatest intervention effects observable at 6 months. Findings from this study should be interpreted with caution because of its non-random design and unclear reporting of participant follow-up rates.

Stratton117 reports no significant effects on VPA, MVPA or mean heart rate in the intervention school students compared with the comparison school students immediately after the intervention to provide playground surface markings. Comparing students in the four schools that received new playground markings with matched students in the four schools that did not receive this intervention, Stratton and Mullan118 report significant effects immediately after the intervention on VPA (F_1,204 = 4.05; p < 0.03) and MVPA (F_1,204 = 13.7; p < 0.01). Findings from both of these evaluations should be interpreted with caution because of their non-random designs, variable rates of attrition between arms and failure to adjust for school clustering of data.

Narrative overview of findings

Six RCTs (10 reports) and four quasi-experimental evaluation studies (six reports) were included, although the quality of these evaluations was generally quite poor, particularly in terms of attrition and adjustment for clustering in the analysis and small sample sizes. These intervention reports addressed most aspects of our second RQ, for example the AAYP and HSE projects modified the way that schools were managed and addressed pastoral care; CDP changed the way that classroom teaching occurred; CAPSLE and PeaceBuilders modified school discipline; and the playground interventions modified school physical environments. The study reports examined measures of health (such as injuries and social anxiety), health behaviours (such as aggression and condom use) and ‘health promotion’ outcomes (such as attitudes to aggression); however, they provided few data relevant to health inequalities or costs.

Nine papers reporting on five evaluations (all but one report/study involving a RCT) examined interventions that encouraged staff and students to build a stronger sense of community and/or better interpersonal relations. 44,55,63,119,120 Such studies have been conducted in a range of school settings (elementary, middle and secondary/high schools), although all except the quasi-experimental evaluation of the HSE intervention (UK) were conducted in the USA. Studies reported a range of significant benefits for some but not all outcomes. Among US elementary school students, by grade four the CDP intervention was reported as having positive effects on three of the four measures of cognitive social problem-solving in conflict situations, as well as teacher-reported students' solving of minor interpersonal problems, but no effects for self-esteem. By grade 6 it had benefits for loneliness/social dissatisfaction and social anxiety, and reduced student self-reported loneliness/social dissatisfaction and social anxiety. 54,55,58 The AAYP intervention was delivered in Chicago elementary and middle schools. 44 Compared with schools in the curriculum-only arm, schools receiving the school environment component plus curriculum experienced non-significant beneficial trends for violence, recent sexual intercourse and condom use among boys at 3 years' follow-up in grade eight. Only for two non-health outcomes (school delinquency and an overall aggregate measure combining the health outcomes with school delinquency and provoking behaviours) were differences statistically significant. Among girls there was a significant increase in self-reported condom use, but no other effects. The CAPSLE intervention reduced peer-reported victimisation and aggression, self-reported aggression and empathy, and peer-reported aggressive bystanding, and increased peer-reported helpful bystanding among third to fifth graders in US elementary schools at 2 years' follow-up. 120 CAPSLE also reduced a decline in self-reported empathy. These benefits were not maintained at the 3-year follow-up other than for peer-reported helpful bystanding. There were no effects for self-reported victimisation and self-reported beliefs in the legitimacy of aggression. The PeaceBuilders intervention delivered in US elementary schools had significant effects on teacher-rated aggression among third to fifth graders but not grade K–2 students, but there were no intervention effects on child self-reported aggression. 119,121 Effects for teacher-rated aggression were larger for students with higher baseline aggression although no test for interaction was performed. At the 1-year follow-up there were intervention effects for injury-related and fighting-related visits to the school nurse. The HSE intervention was associated with significantly higher reported rates of feeling safe at school post intervention among year 7 students, and there were differences of borderline significance in self-reported teasing of others, hurting of others or been in a fight. There were no significant differences regarding self-reported measures of being teased or threatened weekly/hurt ever in this school, belief that will try illegal drugs or smoke a cigarette in the future or belief that will get drunk before age 16 years. 60

The strongest evaluation in this category, that by Flay et al. 44 of the AAYP intervention, reports the least promising results of school environment intervention. However, although not a focus of this review, compared with the no-intervention comparison for boys, both the school environment plus curriculum arm and the curriculum-only arm provided significant benefits. The AAYP investigators themselves concluded that the environment component may have been critical to the overall impact of the combined environment/curriculum intervention based on the substantially larger effect size of the combined intervention than of the curriculum-only intervention, each compared with the no-intervention comparison, for example with regard to violent behaviour (47% compared with 35% reduction), drug use (34% compared with 32%) and recent sexual intercourse (65% compared with 44%).

Taken together these studies suggest the potential of school environment interventions to bring about benefits, particularly regarding measures concerned with violence and aggression. There was no evidence of any interventions causing harms. However, most of these studies were subject to important methodological limitations, such as not adjusting for clustering and high attrition. The strongest study in this category, that on the AAYP intervention, reported the most equivocal results, although the authors interpreted these as suggesting the potential benefits of school environment intervention. We conclude that the evidence base for this category of school environment intervention is promising but not definitive.

Two RCTs each reported on by a single report assessed interventions that combined changes to American middle schools' food and physical activity environments alongside actions that aim to empower students to contribute to achieving these changes. The studies were relatively well conducted, although the evaluation of the HYP intervention was subject to high and differential attrition. Both studies reported intervention benefits for some measures of student physical activity, but neither reported intervention benefits for any measures of student healthy eating. Although the evidence for the effectiveness of these interventions is therefore somewhat stronger than the evidence for interventions addressing sense of community and interpersonal relations, it is not possible to conclude whether or not such interventions to modify the school food/physical activity environments might have achieved the same effects even in the absence of empowered student participation. However, as reported above, mediation analysis in the HYP study did suggest that student empowerment may be important.

Three quasi-experimental evaluations reported in a total of five reports examined the effects of interventions to improve playgrounds in English elementary schools by either providing new marking and equipment or merely providing surface markings. These reported mixed findings regarding effects on students' physical activity. There were indications from one study that the benefits of improving playground physical structures may be greater for younger children and when recess was longer. The stronger of the two studies examining playground surface markings suggested that these were associated with significant effects on both VPA and MVPA immediately after intervention. Given the limitations of these three studies we must conclude that there is not currently a strong evidence base for the effects of these interventions.

Seven of the reports of intervention outcome evaluations examine subgroup effects but only one examined effects by SES. Furthermore, subgroup reporting was carried out inconsistently and three reports failed to report tests for interaction so these studies provide little information on the likely impact of school environment interventions on health inequalities. Only the AAYP intervention was delivered in an explicitly deprived area; this study supports the notion that school environment interventions addressing low-income communities are feasible.

None of the evaluations reports on cost-effectiveness. With the exception of the playground interventions reported by Ridgers et al. 124 and Stratton and Mullan,118 none of the studies reported on intervention costs. The descriptions of the interventions do not provide enough data to enable estimates of costs, particularly in terms of the costs for schools that arose as a result of their participation in the interventions.

Our focus is interventions that aim to modify the school environment without simultaneously addressing school health curricula. We have concluded that there is evidence for the potential of such action to promote student health, but that the evidence is not currently definitive. We will assess the implications of these results for our review hypotheses in Chapter 11.

These results should not be taken as suggesting that there is no definitive evidence for the effectiveness of health-promoting school interventions or co-ordinated school health programmes, which, as well as addressing school ethos or environment, include curricular and community components. A Cochrane review of the effects of health-promoting school interventions comprising changes to school environment/ethos, curricula and parent/community engagement is proceeding in parallel to our own review and was due to report later in 2012. To develop a fuller picture of the effects of school environment interventions, the results of our own review should be read in conjunction with those of the Cochrane review.

Strengths and limitations

This section addresses limitations that are particular to the review of outcome evaluations; limitations that apply across our different reviews are considered in Chapter 11. Our review excluded interventions that included health education alongside actions to address the school environment to be able to assess the specific effects on health of modifying the school environment and in particular how schools are organised, managed and built and how they provide teaching, pastoral care and discipline. However, some of the intervention studies that fit our inclusion criteria nonetheless did not quite lend themselves to these ends. The studies of whole-school approaches to promoting physical activity and healthy eating reported by Dzewaltowski et al. 63 and Sallis et al. 122 were included because they sought to enable student participation in decision-making (fitting our criterion concerning pastoral care) and did not include health education, but they also included actions altering the school food environment and opportunities for physical activity, which was not our focus. We could not establish whether or not any outcomes could be attributed to student participation, although mediation analysis in one paper suggested that this was likely.

Our criteria allowed us to make clear decisions about which study reports to include or exclude, although we sometimes needed to make difficult judgements. For example, we had to make judgements about whether or not interventions should be considered as school environment interventions. Some classroom management interventions were excluded. For example, an evaluation of the Good Behaviour Game was excluded because of its use of a highly structured classroom activity to improve students' behaviour that was delivered in certain specific sessions, which we therefore took to be a form of health education curriculum. In contrast, we included the CDP intervention because this addressed classroom management in a pervasive rather than a discrete, modularised manner, as per the Good Behaviour Game. Although the CDP intervention was not delivered in all of the classrooms in each school, we regarded it as nonetheless aiming to transform a school's overall environment, and the RCT evaluating it involved the allocation of schools not classrooms to intervention or control arms. We also excluded a study by Twemlow et al. 127 because the intervention included some martial arts training that includes self-regulation, role-play and anger management (which we regarded as a health education curriculum). In contrast, we included the study by Fonagy et al. ,120 which also involved self-defence training, because it was geared towards equipping students with the skills to deliver the intervention and did not include any reference to broader personal or social skills development.

Inclusion and exclusion criteria

As reported in the last chapter, 16 reports were included that addressed RQ2 (outcome evaluations). We included process evaluations linked to outcome evaluations reviewed in the last chapter. To identify process evaluations associated with these, the full texts of these reports were retrieved and the following exclusion criteria were applied by one reviewer and checked by a second reviewer (there were no discrepancies to be resolved):

• exclude if study is not a process evaluation

• exclude if study does not report on an intervention subject to an outcome evaluation included in stage 2

• exclude if study is not written in English.

Quality assessment

All included reports were quality assessed using the following criteria:

• whether or not study has clear RQs/aims

• whether or not sampling and sample are described

• whether or not study examines planning (using qualitative data)

• whether or not study examines delivery (using quantitative or qualitative data)

• whether or not study examines coverage (using quantitative or qualitative data)

• whether or not study examines receipt (using quantitative or qualitative data)

• whether or not study examines acceptability (using quantitative or qualitative data)

• whether or not study examines context (using quantitative or qualitative data).

These criteria used for assessing methodological quality were adapted from those used in a previous review. 94 They allowed us to assess which studies were well reported, which examined the intervention process comprehensively and which enabled examination of the process from a range of perspectives. Reports were not excluded or graded based on these quality assessment ratings. Instead, this assessment was used qualitatively when weighing up evidence from each evaluation.

The criteria were piloted on a random sample of two reports by two reviewers (CB and HW) before being applied by one reviewer (HW) and checked by another (CB), with any differences being settled by discussion without recourse to a third reviewer.

Data extraction

Because only those process evaluation studies that were linked to included outcome evaluations (RQ2) were included, we had already extracted data on study RQs/hypotheses, study site and population, sampling, data collection methods, analysis methods and results. Informed by existing tools for data extraction of process evidence126,128 we extracted data related to the following: part of the process examined (planning, delivery, receipt), aspect of the process examined (feasibility, fidelity, coverage/accessibility, acceptability) and aspect of the intervention context examined (e.g. measured need, policy, institutional and professional capacity, collaboration, ‘product champions’). Data extraction tools were piloted on a random sample of two reports by two reviewers (CB and HW) before being applied by one reviewer (CB) and checked by another (HW), with any differences being settled by discussion.

Synthesis

A narrative synthesis was conducted for process evaluation studies. We included all studies in the narrative, but made clear where studies were subject to methodological limitations informed by our quality assessment. It was aimed that the narrative synthesis would develop overarching themes, but in practice it was largely restricted to narrative summaries of the findings of each study in context. This was because the studies were too heterogeneous in design and methods to develop meaningful or very detailed overarching narrative themes. Nevertheless, a narrative overview is provided in this chapter's discussion section.

Flow of literature

Only those process evaluations that were linked to outcome evaluations included in RQ2 were considered for inclusion. Therefore, the 16 included outcome evaluation study reports were screened for accompanying process evaluations (all of the reports were written in English). Of these 16, five reports included process evaluations. From checking the references of the five included reports, we identified one further linked process evaluation that was included in the in-depth synthesis. Fagen and Flay129 reported on a process evaluation of the sustainability of the AAYP intervention but this focused only on the curriculum component and so is not considered further here. Thus, six reports (one linked)55,58–60,84,85 of four studies were included in the process evaluation in-depth synthesis (Figure 9).

FIGURE 9.

Flow of literature: process evaluation synthesis (stage 2: in-depth synthesis).

Quality assessment

Study reports varied in whether or not they set clear RQs. Clear questions were provided by Bonell et al. 59,60 and Solomon et al. 58 Nearly all described their sampling methods and samples; Solomon et al. 58 did not describe the sample. Studies varied in the extent to which they sought the perspectives of a range of stakeholders on the interventions. Battistich et al. 55 and Solomon et al. 58 relied solely on research observations of delivery. Bonell et al. 59,60 collected data from external providers, school staff and students. Dzewaltowski et al. 63 collected data from students, site co-ordinators and teachers. Flannery et al. 119 collected data solely from teachers. Only Bonell et al. 59,60 drew on qualitative and quantitative data, the others drawing only on quantitative data.

Only Bonell et al. 59 examined the planning of the intervention prior to delivery, drawing on interviews with providers. Reports from all four interventions described the fidelity of intervention delivery using observations or data from teachers, other intervention staff or students. Only Battistich et al. 55 and Solomon et al. 58 reporting on the CDP intervention, examined the fidelity of delivery by comparing intervention with control schools. Bonell et al. 59,60 and Dzewaltowski et al. 63 drew on self-report data from providers to quantify delivery. The HSE evaluation59,60 also drew on qualitative observational data on delivery and interview data on the feasibility of delivery, while Flannery et al. 119 examined quantitative data from teachers on feasibility. Bonell et al. 59,60 and Dzewaltowski et al. 63 examined coverage by determining recognition of the intervention among students. Bonell et al. 59,60 evaluated acceptability qualitatively through interviews and focus groups. Flannery et al. 119 evaluated acceptability quantitatively through surveys only of the training component for teachers. Dzewaltowski et al. 63 examined the extent to which training provided site co-ordinators with the self-efficacy to undertake their work. Only Bonell et al. 59,60 drew on qualitative data and examined how context might influence intervention delivery or uptake.

Study characteristics

Five reports examined three interventions that encouraged staff and students to develop school climates characterised by a stronger sense of community and better relationships. 55,58–60,119 One study evaluated an intervention that enabled staff and students to advocate for school environments promoting healthier eating and physical activities. 63

Narrative summary of findings from each process evaluation

Battistich et al. 55 report that, across all years of the CDP intervention, teaching practices across the five areas addressed by the programme were significantly distinctive from those in comparison schools, suggesting that the intervention was feasible to deliver with good fidelity. Solomon et al. 58 report that according to observations of teachers' there were significant differences between intervention and control schools in around half of the teaching practices and classroom activities intended to be brought about by the project, for example teachers' use of group praise and students participating in rule development. According to student reports, there were significant differences between intervention and control arms in most indicators of teaching practices and classroom activities prescribed by the project. This study examined intervention fidelity in a rigorous manner so that conclusions in this area are likely to be sound, but did not examine other aspects of or perspectives on process, or assess context.

Bonell et al. 59 report that the intervention was delivered as intended with all components implemented, although it should be noted that this study did not examine fidelity of delivery through observation sessions. Qualitative data suggest that the external facilitator enabled schools to convene an action team involving staff/students. Inputs were feasible and acceptable and enabled similar actions in both schools. Locally determined actions (e.g. peer mediators) were generally more feasible and acceptable than preset actions (e.g. modified pastoral care). This study alone used qualitative data to examine the effect of contextual factors on implementation. This suggested that implementation was facilitated when it built on aspects of schools' baseline ethos (e.g. a focus on engaging all students, formalised student participation in decisions) and when senior staff led actions, acting as ‘product champions’. Student awareness of the intervention was high. Quantitative data on students' attitudes and behaviours suggested that the intervention aims corresponded with local needs in each intervention school. Bonell et al. 60 report that some activities such as rewriting school rules involved broad participation, which was assessed through qualitative methods such as interviews.

Flannery et al. 119 report that teachers being trained to deliver the PeaceBuilders intervention found the philosophy behind the intervention easy to understand. They regarded training as clear, effective and easy to follow, that the school administration supported the intervention and that it would be easy to implement and would be effective in the classroom. Surveys with teachers also suggested that the intervention was delivered regularly; approximately half rated implementation as extensive and half as moderate, with around half using half or more of the intervention materials. Other aspects of the process, other perspectives, such as those of students, and context were not examined.

Dzewaltowski et al. 63 undertook a thorough quantitative assessment of process and reported that training for the HYP intervention occurred as planned, with site co-ordinator attendance very high in both years and self-efficacy arising from training being high. Site co-ordinators formed ‘change teams’ in each school that met regularly. Site co-ordinators reported an average of 26.5 implemented programme, policy or practice changes. Teachers implemented around two-thirds of planned lessons. Student surveys suggested that around one-third of students had heard about the intervention or its activities, around half of whom had participated on ‘change teams’. Context was not examined in this study.

Narrative overview of findings

Of the 16 included outcome evaluations, five reports included process evaluations and one further linked process evaluation paper was found by reference checking. These employed a range of research methods, most frequently drawing on quantitative data collected from students and/or teachers. These reports addressed some aspects of our third RQ more than others. Although most examined feasibility or fidelity in some way, fewer examined acceptability and only one study used a mix of quantitative and qualitative methods to examine local context and how this influenced intervention processes. Process evaluations reported largely positive results regarding intervention feasibility, fidelity, reach and acceptability, although differences in methods prevent any comparison of the delivery and uptake of each intervention. The single study that examined context suggested that this was important, facilitating implementation when this built on schools' existing ethos and when senior staff championed the intervention.

Strengths and limitations

We limited our in-depth review of process evaluations to those linked to outcome evaluation studies because in consultation with our stakeholders we deemed it most useful to synthesise evidence about the feasibility and acceptability of interventions about which we have evidence of their effects. This pragmatically limited the scope of our review by preventing us from synthesising evidence, for example, on potentially innovative interventions that have been feasibility tested but not yet been subject to outcome evaluation.

Most of our outcome evaluations were accompanied by process evaluations but these involved a diversity of methods making it impossible to compare the feasibility, coverage, acceptability or context of the interventions. The small number of heterogeneous studies made it impossible to draw conclusions about how context, processes and outcomes might inter-relate.

Inclusion and exclusion criteria

There were 285 multilevel and ecological references coded in the evidence map. We included multilevel studies examining the effects on student health or well-being outcomes (age 4–18 years) of objective school-level measures of how schools are led and managed, how they teach, support or discipline students, or schools' physical environment. Two reviewers (CB and HW) double-sifted all 285 references, independently, using the exclusion criteria shown in Table 5.

We screened all references identified as potentially relevant to RQ4 in the evidence map based on title and abstract only and full paper where necessary, not just those coded as being about relevant school-level exposures, because we could not assume that coding was accurate for the evidence map. Screening was not hierarchical or mutually exclusive, so some reports were excluded based on multiple criteria.

We included only multilevel studies and not ecological studies. Ecological studies measure the prevalence of outcomes at the school level and relate these to school-level characteristics. For example, they might conclude that smaller schools or schools with a high staff–student ratio have lower rates of smoking. Multilevel school studies measure outcomes at the individual level and explain these in terms of school-level characteristics as well as individual-level student characteristics. For example, they might conclude that student smoking is affected not only by the quality of the individual student's own relationships with teachers but also by the overall quality of relationships between students and teachers in a school. Ecological studies are vulnerable to unmeasured confounding, whereby differences in outcomes between schools reflect differences in composition rather than differences in school environments, and are also unable to disentangle the effects of school-level factors such as student–staff relations that can also be represented at the individual level (cross-level bias). 48

We decided to focus only on those studies in which measures of the school environment drew on ‘objective’ data’, that is, data were not derived from aggregates of self-reports from the same individuals (usually students) who also provided data on health outcomes. This was a pragmatic decision to limit the number of included reports as well as being informed by our view that, when data on exposures and outcomes were derived from the same individuals, any associations found might merely be the result of unmeasured characteristics of individuals unrelated to the effects of the school environment.

Quality assessment

All included reports were quality assessed using the following criteria, which were informed by the review by Aveyard et al.:48

• whether or not appropriate covariates (key potential confounders) were adjusted for

• whether or not inappropriate covariates (potential mediators) were adjusted for.

By confounders we are referring to factors such as students' SES or sex, which could vary between schools and could themselves affect health outcomes, but which are unlikely to be influenced by schools during the time period of the research. By mediators we are referring to factors such as student engagement and peer smoking, which might affect health outcomes but which themselves might be open to school-level influences. Reports of multilevel studies were not excluded from in-depth review based on judgement of their quality. We included all of the reports meeting the inclusion criteria for stage 2 when describing study context and characteristics as well as when summarising intracluster correlation coefficients for the various health outcomes reported by multilevel studies. However, we did restrict our narrative synthesis of the effects of school-level determinants on these health outcomes to studies that we judged had taken an appropriate approach to adjusting for confounding. Adjusting for confounding is important to minimise bias in estimates of the association between school-level factors and student health outcomes. Differences in the rates of health outcomes between schools might sometimes be merely attributable to differences in the characteristics of students entering the school, their families or the neighbourhoods they live in. Therefore, it is important that studies adjust or otherwise control for these potential confounders. However, as some authors have pointed out,48,49,130 it is also important that studies do not overadjust for covariates that might lie on the causal pathway underlying this association. Covariates in this category would include any factors that might be subject to school-level effects, such as attitude to school, peer behaviours and academic attainment.

First, we judged which covariates were inappropriate to adjust for. It was not possible to develop a comprehensive list a priori because of the large range of covariates that we would encounter. However, we were informed by the following lists suggested previously by Aveyard et al. 48 and used these to aid our own case-by-case judgements about which covariates are inappropriate to adjust for because they might plausibly be influenced by the school.

Factors categorised into those influenced by school and those not influenced by school – appropriate to adjust for:

• family characteristics

• family structure

• parental smoking

• parental attitudes

• sibling smoking

• family environment

• attachment to family.

Pupil characteristics not influenced by school – appropriate to adjust for:

• sex

• ethnicity

• SES

• personal income.

Pupil characteristics potentially influenced by school – inappropriate to adjust for:

• age

• school engagement

• other risk behaviour

• stress

• depression

• self-esteem

• attitudes to smoking

• attitudes towards health generally

• peer smoking

• peer attitudes and norms.

Second, we examined whether or not studies did adjust for the key potential confounders. We set two thresholds of varying strictness. This was a pragmatic decision to enable us to focus on the least biased studies while ensuring that we excluded only a few studies. Our broader, more inclusive definition required that, as well as not adjusting for any inappropriate covariates, studies adjusted for sex plus some measure of individual and/or local area social disadvantage. We treated individual or family SES or ethnicity as well as family structure and local measures of deprivation or local health indices as being crude indicators of individual or area social disadvantage.

Our stricter definition was that, as well as not adjusting for any inappropriate covariates as defined above, studies adjusted for some measure of sex plus some measure of student ethnicity, family SES or structure plus some measure of area deprivation or local health indices. We set this stricter threshold because we judged that individual and area social disadvantage would operate as independent sources of confounding.

One reviewer (WP) applied these quality assessment criteria and another reviewer (CB) checked these assessments, with any differences being settled by discussion.

Data extraction

For the reports of multilevel studies we extracted data on study RQs/hypotheses, study site and population (i.e. overall baseline characteristics), data set, sampling, types of schools, data collection methods, analysis methods, results and authors' conclusions. We also extracted data on the objective school-level measures included in the multilevel models, the types of levels included (i.e. student, class, teacher, school, city, region) and the types of covariates (and whether appropriate or inappropriate). Data extraction tools were piloted on a random sample of two reports by two reviewers (CB and HW). One reviewer (HW) extracted data on context and methods of data collection and another reviewer (WP) extracted data on methods of analysis and results, both checked by a third reviewer (CB), with any differences being settled by discussion.

Synthesis

We assessed the potential for statistical meta-analysis of multilevel studies of school effects on health by noting which studies focused on similar combinations of school-level interventions/exposures and outcomes. We set a threshold of a minimum of three such studies being required to consider meta-analysis as meta-analysis of only two studies is unlikely to produce stable effect estimates. 116 Among the included multilevel modelling studies we found possible homogeneity for Evans-Whipp et al. 131 and Piontek et al. 132 in their examination of tobacco policy comprehensiveness (although with quite different measures) and smoking in the last 30 days; and Aveyard et al. 64 and Markham et al. 65 in their examination of value-added scores as well as truancy in relation to weekly smoking. We found no homogeneous triplets of studies. We therefore subjected our studies of school-level health effects to narrative synthesis only.

We restricted our narrative synthesis to those studies that we judged as taking an approach to adjusting for covariates. First, we developed narrative summaries of the findings reported by each study in the context of study quality, setting and population. This is reported in the results section of this chapter. Second, we developed a narrative overview of findings. We noted where studies of similar influences reported convergent or divergent findings and considered the reasons for this in terms of design and context, rather than merely assessing the overall frequency of differing findings. This is reported in the discussion section of this chapter. Third, we considered the implications of the results of multilevel studies of health effects alongside other forms of evidence for this review's primary and secondary hypotheses. This is reported in Chapter 11.

Flow of literature

Of the 285 references identified from the evidence map, 236 were excluded based on the criteria in Table 5. In addition, five reports were duplicates and the full text was not available for six of the reports. A further four reports were identified from citation chasing of reports included in the evidence map. Thus, 42 reports (from 34 data sets) were included in the multilevel studies in-depth synthesis (Figure 10).

FIGURE 10.

Flow of literature: multilevel studies (stage 2: in-depth synthesis).

Quality assessment

Study characteristics

Overall, 17 reports focused on US samples,53,67,68,70–73,76,77,81,82,85,86,88,89,133,134 six were from Canada79,80,83,84,90,135 and six were from the UK,48,65,66,130,136,137 three were from Norway74,75,87 two each were from Israel78,138 and the Netherlands69,139 and one each was from Australia,97 Belgium,140 Germany,132 Spain,141 and Thailand,142 with one additional study131 reporting on data from the USA and Australia.

Two reports did not indicate what kind of schools were researched70,71 and five reports indicated only from which grades research participants were drawn (without reporting schools' full grade structure). 72,75 Of the other studies:

• 18 focused on secondary/high schools65,66,69,77,79,81–84,87,97,131,132,136,138,139–141

• three focused on middle and secondary/high schools48,76,134

• four focused on primary/junior/elementary schools86,90,135,162

• one focused on primary/junior/elementary and middle schools89

• three focused on primary/junior/elementary and secondary schools80,130,137

• two focused on middle schools85,133

• one focused on high/junior high. 67

Aveyard et al. 64 and Bisset et al. 66 both drew on data from the West Midlands Young People's Lifestyle Survey, but examined different outcomes. Two reports by Wiium and Wold74,75 both drew on Control of Adolescent Smoking (CAS) data, but examined different smoking outcomes. Two reports by West et al. 130,137 both drew on the West of Scotland 11–16 survey, but examined different outcomes and the latter involved more follow-ups. Gastic76 and Haley77 both drew on the National Longitudinal Study of Adolescent Health (NLSAH or ‘Add Health’) data, but used differing waves of follow-up and examined different outcomes. Attar-Schwartz and Khoury-Kassabri138 and Khoury-Kassabri et al. 78 both drew on the same Israeli data set, but examined different outcomes. Kairouz and Adlaf79 and Lothian80 both drew on the Ontario Student Drug Use Survey (OSDUS) data set and also examined different outcomes. Kim81 and Way82 both drew on National Education Longitudinal Study (NELS) data, but the latter used data from more follow-ups. Ishibashi83 and Murnaghan et al. 84 both drew on data from the School Health Action Planning and Evaluation System (SHAPES) study, but used different waves to examine different outcomes.

Narrative summaries of intracluster correlation coefficients of health outcomes

It is generally accepted that the intraclass correlation coefficient (ICC) of an outcome under the null model (a model containing an intercept term and no covariates) provides a good descriptive measure of the degree of clustering in data. More formally, the ICC is the proportion of total outcome variance that is ‘between groups’, that is, in this review the proportion of total outcome variance that is associated with the school level.

In reviewing study findings on ICCs we draw on all reports regardless of the approach taken to adjustment for covariates because the ICC data were drawn from unadjusted models. Reported ICCs were generally quite low, with most falling in the range of 0–10%, indicating that most of the variation in outcome measures was due to variability at the student level as opposed to the school level. Notable exceptions to this were verbal victimisation (ICC = 21%) in the study by Attar-Schwartz and Khoury-Kassabri138 and healthy food eating (ICC = 17%) and exercise (ICC = 29%) in the study by Ma. 135

Some studies report ICCs for different measures and so allow comparison of the apparent clustering of different outcomes. Attar-Schwartz and Khoury-Kassabri138 and Khoury-Kassabri et al. 78 reported that verbal victimisation was more clustered than indirect victimisation; Collins et al. 72 reported that marijuana use was more clustered than other drug use; Haley77 reported that drinking among same-sex-attracted students was more clustered than fighting; Kim81 reported that clustering for alcohol use was higher than that for smoking; and Ma135 reported that clustering was greatest for exercise but lower for healthy eating and even lower for mental health. However, these findings from single studies cannot provide a clear sense of underlying patterns of clustering for these outcomes. Because of the wide variation in outcome measures, study contexts and data collection methods, direct comparisons of ICCs across studies, even where outcomes were substantively similar, would not be valid.

Narrative summary of the results of well-adjusted studies on school-level effects

The model estimates for determinants reported in studies that were well adjusted are outlined in the following sections. We will start with the reports that met the narrower definition of adjusting for all key confounders before moving on to those that met the broader definition.

Narrative overview of findings

We included 42 reports of multilevel studies of school-level health effects (from 34 data sets) in which school-level factors were measured ‘objectively’ (i.e. not merely data from individuals from whom outcome data were collected). We confined our narrative synthesis of results to studies that adjusted for an array of key potential confounders (setting strict and less strict thresholds depending on the comprehensiveness of adjustment) and did not adjust for factors that might mediate school effects on health. Six studies met our stricter definition of being appropriately adjusted and a further four met our less strict definition. These intervention studies addressed some aspects of our fourth RQ more than others. In terms of school-level influences, the studies of value-added education are intended as indicators of teaching and of pastoral support; the studies of school policies can be regarded as examining both how schools are managed and how they instil discipline; and the studies of school campus areas and observable compared with unobservable places are examining aspects of physical environment. The studies generally reported on measures of health behaviours (such as substance use and fighting) rather than on health or ‘health promotion’ outcomes. Substance use was a predominant although not exclusive focus of the multilevel studies. These multilevel studies did not report subgroup analyses relevant to assessing health inequalities.

Reflecting an earlier review,49 we found that ICCs for health outcomes, where reported, were generally low, with notable exceptions such as for verbal victimisation138 and exercise,135 indicating that most of the variation in outcome measures was due to variability at the student level as opposed to the school level.

We found consistent evidence from cross-sectional53,64,66 and longitudinal studies53,65 of middle schools in the USA and secondary schools in the UK that schools in which attainment was higher than would be expected from the social profile of students, and truancy was lower than expected, had lower rates of substance use. The Tobler et al. study53 is particularly valuable not only in providing longitudinal evidence and examining an outcome concerning group fighting in addition to substance use, but also in examining the extent to which these effects appear to be relevant not only to predominantly white, middle-class English adolescents but also to low-income, ethnic minority young people in US cities. Gross rates of attainment and truancy were not associated with student health outcomes and the authors of these reports interpret this to mean that what matters is the value that schools add rather than gross rates of attainment and truancy because the latter merely reflects the characteristics of students at intake rather than what the school as an institution provides.

Selection bias is unlikely to explain these results because this would require schools with both high valued-added scores and high rates of substance use, and conversely schools with low value-added scores and low substance use, to have differentially tended not to participate. Reverse causality is also unlikely given that two of the studies were longitudinal and all of the studies used year-on-year averages of attainment and truancy data so that the data on exposures and outcomes reflected the experiences of different students. However, confounding might explain these results if some unmeasured or incompletely measured student or area factor was associated both with value-added scores and with student health outcomes. As Aveyard et al. 48 point out, the most obvious source of confounding arises from the same families both discouraging their children from engaging in risk behaviours and sending their children to effective schools. However, they point out that published data did not enable parents to identify exactly what value schools added, and the schools with the best attainment were not the same as the schools adding the most value. The two longitudinal studies adjusted for baseline measures of the risk behaviours being examined. Although these baseline risk behaviours might themselves be influenced by school-level factors, these do not represent cases of overadjustment for factors on the causal pathway because of the timing of the assessments. Nonetheless, a more useful approach would be to adjust for students' engagement in risk behaviours prior to their entry into the school, as West et al. 130,137 (in studies that were excluded because of their adjusting for variables we judged to be potential mediators of school effects) were able to do.

The authors suggest that authoritative schools provide better support and an appropriate level of control of students' behaviour, and that students in such schools are more likely to adopt pro-school identities and values and are less likely to invest in anti-school risk behaviours such as smoking. The authors suggest that their measures of value added are proxies for this institutional culture. We would suggest two caveats, however. First, Aveyard et al. 64 and Bisset et al. 66 use cross-sectional data and, therefore, it is difficult to be confident about causalities implied by associations. Second, these studies used routine data to develop relatively simple measures of these school-level factors and so cannot provide insight into what processes might underlie these apparent effects. Other studies might shed some light on this. As reported earlier, factors such as strong leadership, student involvement, high expectations and frequent evaluation and praise are reported to explain school differences in attainment. 18–21

Findings on the influence of school policies were mixed. A German cross-sectional study of secondary schools reported that a complete smoking ban for students at/around school was significantly associated with reduced smoking, although monitoring of students' smoking and sanction types for those caught were not associated with reduced smoking. 132 However, a cross-sectional survey of secondary schools in the USA and Australia found no association between various forms of school smoking policies and any measures of student smoking. 131 These differences may reflect a ‘ceiling’ effect for the impact of smoking bans, which have already been widely implemented across schools in the USA and Australia but not in Germany, where student smoking has traditionally been more tolerated by school authorities. Earlier studies in the USA and the UK not included in our narrative review because of methodological limitations have also suggested that school smoking policies are associated with reduced smoking. 136 The finding from a cross-sectional study of Dutch secondary schools that whether or not a school permitted alcohol use at school on special occasions, as well as a school's level of sanctioning if students were caught using alcohol at school, was not associated with heavy drinking among students is likely to reflect such a policy being unlikely to have much power given the limited number of special occasions when alcohol use would even be considered possible, such as school trips and parties. 69

These studies relied on cross-sectional data, which do not provide strong evidence of causality because of the possibility of reverse causality. However, this is unlikely in the case of these school-level measures of policy because of the stability of policies from year to year. However, as with the studies of value added discussed above, confounding by unmeasured student, neighbourhood or school factors is a potential source of bias in estimating the effects of school-level policies on smoking and other student health outcomes.

We also reviewed evidence from one cross-sectional study which reported that students in US middle schools with larger total campus and playground areas per student have higher rates of student accelerometer-measured physical activity during school hours. 133 Our review also found evidence from one cross-sectional study of US high school students that the number of unobservable and unsupervised places in and around school was associated with tenth and twelfth graders' use of alcohol in school and tenth graders' use of marijuana in school in the previous 12 months but not twelfth graders' overall use of marijuana in the past year nor twelfth graders' cigarette smoking in school in the previous month. An attractive school environment was associated with tenth and twelfth graders' use of alcohol in school in the past year but not with whether or not eighth graders used marijuana in the past year or used alcohol in school in the past year or smoked cigarettes in school in the past month, or whether or not twelfth graders were drunk in the past 30 days. The authors conclude that these results suggest that schools may be effective in addressing risk behaviours that occur inside school but may not have much impact on behaviours outside school. 68 Finally, Henry et al. 67 reported from a cross-sectional study that the following school-level factors were not associated with alcohol use among students aged 13–14 years in high, junior high and middle schools in rural USA: whether eighth graders are located within the same school as high school students or are in separate schools, school size and pupil–teacher ratio. Reverse causality is unlikely to explain the results of any of these studies because of the enduring nature of the school-level measures, but confounding by unmeasured individual or neighbourhood factors might account for the associations.

The only study that examined subgroup effects was that reported by Markham et al. ,65 which found that in secondary schools in the West Midlands, UK, there was no significant interaction of the school-level measure of value-added education with whether or not pupils were regular smokers at baseline.

Strengths and limitations

As discussed earlier in relation to intervention outcome evaluations, our review was exploratory in nature, aiming to map the possible range of health outcomes that school-level exposures might affect. We developed guidance in stage 1 about which measures would count and which would not, although in practice we found that we had to make judgements. We included measures of physical and non-physical violence and attitudes to violence but not measures of weapon carrying or non-violent delinquency such as graffiti; and measures of feeling safe at school but not successful transition to secondary school. Although these judgements can be challenged, we did at least apply them consistently.

We decided to focus only on those studies in which measures of the school environment drew on ‘objective’ data, that is, they were not derived from aggregates of self-reports from the same individuals (usually students) who also provided data on health outcomes. This was partly a pragmatic decision to limit the number of reports to review to a feasible, yet still ambitious, number. It was also informed by our view that, when data on exposures and outcomes are derived from the same individuals, any associations found might merely reflect unmeasured characteristics of individuals unrelated to the effects of the school environment. We recognise that the objective measures in question were themselves heterogeneous: researcher observations, for example, being very different from teacher reports. Such measures may be subject to information bias that differs from that found with student-derived data; for example, it may be that teacher-derived measures present a more positive picture of schools than would student-derived data. However, it is unlikely that such information bias would be differential with regard to school-level influences and so it is unlikely to have biased our findings. An alternative approach would have been to have included studies regardless of whether these used ‘objective’ or ‘subjective’ measures of school-level factors, but excluded all studies from our review (and not merely our narrative synthesis) that did not take what we judged to be an appropriate approach to adjustment for covariates.

It was sometimes difficult to judge whether a covariate was appropriate or inappropriate to adjust for. Some covariates were treated as appropriate because they were determined to be stable psychological traits, despite our thinking quite plausibly that these could be affected by schooling. These included impulsiveness in Grunseit et al. 97 and Kim88 and locus of control in Kim. 81 Varsity sports participation was also treated as appropriate as Leatherdale et al. 90 regarded that this might be a measure of intrinsic athletic ability that schools recognise but do not affect, although we again had doubts about this. Although open to challenge, decisions about these covariates were not critical in determining that these studies did not meet either our stricter or broader definitions of appropriately adjusting for covariates.

Our inclusion criteria were focused on studies of children aged 4–18 years. However, we included two multilevel studies in which the age range went up to slightly older than age 18 years, reflecting the secondary education system of that country. We included only studies in which the age range encompassed those below the age of 18 years and did not include any studies of postsecondary education.

The heterogeneity of the studies we included meant that we could not undertake statistical meta-analysis and were confined to narrative synthesis. We urge caution in the interpretation of our findings of our narrative synthesis of multilevel model studies, as with those of our intervention studies. Our review did not set specific primary and secondary outcomes. In narrating the findings of studies we have tried to give equal weight to significant and non-significant associations and comment in our summing up on the balance between these, but this is no substitute for testing hypotheses defined in terms of a small number of primary and secondary outcomes.

Inclusion and exclusion criteria

A total of 194 references included in the evidence map were relevant to RQ5: qualitative studies. We included qualitative studies in the in-depth review that explored the processes by which school-level factors relating to how schools are led and managed, or how they teach, provide pastoral care to or discipline students; student attitudes to school or relations with teachers; and/or the school physical environment might influence student health.

Two reviewers (FJ and AH) developed and applied an initial set of exclusion criteria to reports coded as addressing RQ5 in the evidence map based on the full text. Pilot screening was conducted on a random sample of 10 reports to test and refine the criteria. All reports were then double-screened by the two reviewers. The reviewers discussed any discrepancies in screening until agreement was reached. In cases in which an agreement could not be reached, a third reviewer provided input. Table 6 outlines the exclusion criteria for sift 1.

A second set of rating criteria was then applied to all reports included from sift 1 in order to limit the review to reports that provide findings that are conceptually rich enough to facilitate meta-ethnography. A scale of ‘high’, ‘medium’ and ‘low’ was used to rate each study's level of (1) conceptual richness (whether or not authors go beyond a description of the findings and interpret them to develop concepts, theories or metaphors), (2) relevance in terms of its research aims and (3) relevance of its findings to addressing our RQs. A total of 21 reports that scored ‘high’ or ‘medium’ across all three categories were included in the in-depth review.

Quality assessment

The criteria used for assessing methodological quality were built on EPPI-Centre health promotion reviews. 8 The quality criteria addressed the rigour of sampling, data collection, data analysis, the extent to which the study findings are grounded in the data, whether or not the study privileges the perspectives of children and young people, the breadth of the findings and the depth of the findings. The complete quality assessment tool is available in Appendix 5. Based on the results of these criteria, reviewers judged the study overall in terms of reliability and trustworthiness of findings on a ‘low’, ‘medium’ and ‘high’ scale. Reports were not excluded based on these quality assessment ratings; instead, they were used to inform our interpretation of the findings. A round of pilot quality assessment was conducted by four reviewers (AH, CB, FJ and HW) on a sample of two reports to test the tool and ensure consistency in applying the criteria. All reports were then split between two reviewers (FJ and HW) and assessed independently. All items were checked by another reviewer (CB or AH) and any disagreements were resolved by discussion.

Data extraction

Reviewers summarised the qualitative study findings in their own words and extracted any themes or concepts that the study authors reported. Four reviewers extracted findings from a randomly selected sample of two reports (CB, AH, FJ and HW) to pilot the tool and ensure consistency in extraction. All other reports were split between two reviewers (FJ and HW) and completed independently. All items were checked by another reviewer (CB or AH) and any disagreements were resolved by discussion.

Synthesis

Meta-ethnography is an approach to research synthesis originally developed by Noblit and Hare. 144 They argue that meta-ethnographic enquiry is driven by the desire to develop interpretative explanations and understanding from multiple cases of a given phenomenon by utilising research that is ‘grounded’ in the experiences of participants (p. 12). 144 The approach has since been applied by qualitative systematic reviewers in the field of public health, particularly for questions related to patient experiences (e.g. see references145–147).

The purpose of this meta-ethnographic synthesis is to conduct an interpretive enquiry into the themes, concepts and metaphors set out by authors investigating the ways in which the school environment might influence young people's health. We developed a general interpretation grounded in the findings of separate studies of the process by which schools might influence health by reinterpreting meaning across individual qualitative studies. We used a four-phase synthesis approach adapted from Noblit and Hare's144 phases of meta-ethnographic synthesis (pp. 26–9).

Flow of literature

In total, 194 qualitative reports examining the process by which schools might influence health were identified as potentially relevant in our map. The full texts of these reports were retrieved and screened in two sifts. In the first sift, 106 reports were excluded, the full texts of 22 reports were not available and eight reports were duplicates that were not picked up in earlier deduplication efforts (all studies were reported in English). The remaining 58 reports proceeded to the second sift. In this, 37 reports were excluded. The remaining 21 reports proceeded to the quality assessment and data extraction phase. Three of these references were ‘linked’ and thus counted as one study. Therefore, 21 reports of 19 studies were included in the in-depth review. A total of 10 studies examined aggressive behaviours, four studies examined substance use, two studies examined eating behaviours, two studies examined sexual health and one study examined the rules for going to the toilet. Figure 11 provides a summary diagram of the flow of literature through the review.

FIGURE 11.

Flow of literature: qualitative studies (stage 2: in-depth synthesis).

Quality assessment

Most of the studies in this review (n = 14) were considered to be of high91–93,148,150,152–154 or medium94,95,157–161 quality; five studies96,149,151,162,163 were considered to be of low quality based on the reliability and trustworthiness of the findings.

Three of the low-quality studies151,162,163 used ethnographic methods, such as participant observation, and were considered to be among the most conceptually rich. Reports were not excluded based on low-quality scores because reviewers felt that this could bias the review according to certain methodological approaches (e.g. interviews and focus groups rather than ethnographic approaches) and certain academic disciplines in which methods may be less transparently reported (e.g. anthropology). The evidence synthesis draws on a mixture of evidence from studies deemed to be of high, medium and low quality.

Study characteristics

Seventeen studies91–96,148–150,152,153,155–161,163 used one or more of the following methods to collect data: interviews, focus groups, participant diaries, school studies, open-ended questionnaires and/or general observation. Two studies, in the tradition of anthropology, used participant observation151,162 and two studies used participatory techniques148,155, in which researchers worked with students to map out different health behaviours using blueprints of the school environment and surrounding area, alongside interview and focus group techniques.

Of the 19 included studies, 10 were conducted in the USA,94–96,148,151–153,161,162 six in the UK,91–93,95,155,156,158,163 one in Australia,159 one in South Africa149 and one in Sweden. 160 In terms of health topics, eight of the US studies94,96,148,150–153,157,161,162 examined bullying, violence, intimidation, harassment and/or physical violence (hereafter described collectively as ‘aggressive behaviours’). The studies from Australia and South Africa also examined aggressive behaviours. The other two US studies examined sexual health94 and drug use respectively. 96 Of the six UK studies,91–93,95,155,156,158,163 three examined tobacco, alcohol and/or drug use (hereafter described collectively as ‘substance use’),91–93,95,155 two examined students' eating behaviours at school156,163 and one examined sexual health. 158 The study from Sweden examined rules for going to the toilet at school.

The majority of studies were conducted in high school/secondary school settings. Other types of schools examined were middle schools and primary/elementary schools. One study149 was conducted in an environment of chronic poverty (South Africa) in a middle-income country. Of studies undertaken in high-income countries, nine studies95,96,150–153,156,162,163 were conducted in contexts of high levels of socioeconomic disadvantage; two studies were conducted with a high SES sample;94,159 and four studies explored students' experiences across multiple schools and/or neighbourhoods that varied in terms of their socioeconomic characteristics. 91–93,148,157,161 It is difficult to assess the socioeconomic context of three of the studies. 155,158,160 The majority of studies were conducted in an urban context (n = 13) and none was focused on a rural setting, although some studies include schools from a range of different urban, suburban and rural locations.

A range of different ethnic minority groups was included in these studies. African Americans were the most studied ethnic group (seven studies148,150–153,161,162 examined at least one school with a majority African American population). Some studies focused on other ethnic minority groups such as experiences of Latina/o young people in the USA96 (n = 1) and the experiences of ‘coloured’ young people in South Africa149 (n = 1). One study focused exclusively on the experiences of white young people. 159

In summary, the qualitative synthesis is largely informed by evidence from the USA and the UK. It will also have a bias towards the views and experiences of young people from disadvantaged communities and low SES families, young people from ethnic minority groups (particularly African American students) and those students who attend schools in urban settings. In terms of specific health behaviours, qualitative researchers exploring and theorising potential school effects have focused mainly on how schools might shape aggressive behaviours and substance use.

Meta-ethnography of study findings

The first stage of the synthesis process involved grouping studies by their health topic and extracting the key concepts and themes from each individual paper. Second, reviewers conducted a thematic analysis of these themes and concepts across individual papers that addressed the same health topics. The outcome of this process is reported in Figure 12 (see Step 2: overarching themes by health topic). Finally, we developed meta-themes, which were inductively arrived at by reducing the themes from across each health topic. The outcome of this process is also reported in Figure 12 (see Step 3: meta-themes).

The findings are organised according to these meta-themes produced in step 3. This allows us to present a thick description of the key concepts that emerged and the inter-relationships between studies, comparing and contrasting across all health topics.

FIGURE 12.

Arriving at meta-themes: reciprocal translation of included studies.

Meta-theme 1: performance, collective identity and bonding

Meta-theme 2: the social importance of space

Meta-theme 3: teacher–student relationships, school policies and teacher practices

The studies consistently suggest that having good relationships with school staff, particularly teachers, is integral to creating a healthy school environment. Students who have caring and supportive relationships with teachers report feeling safer in the school and appear more resilient towards participating in risky health behaviours, and more academically and socially engaged at their school. A good example of this was a low-quality study of pre-adolescent middle school students in the USA (mostly young women) which suggested that supportive teachers and school counsellors may foster resiliency in young people regarding drug use in schools. 96

However, staff–student relationships appear to be constrained and weaker when school rules are established and enforced by teachers without student input or consultation; teachers are ‘out of touch’ with the realities of young people's lives; there is limited supervision and support from teachers outside the classroom instructional environment; and teachers apply the rules inconsistently. In the following sections we explore how these factors impact on staff–student relationships and how, in turn, they relate to student health.

Meta-theme 4: ‘escaping’ the school environment

Summary of key findings

A total of 21 reports met our inclusion criteria and were included in the review. These addressed our fifth RQ about the processes through which school effects might occur. They focused on the aspects of the school environment most salient to and of most concern to students, for example schools' physical environments, teachers' lack of involvement in pastoral care and the outsourcing of discipline to security guards and new technologies. There was less focus on how schools are organised and managed and how teaching is delivered. In terms of outcomes, our studies focused mainly on violence and substance use with some attention to diet and to teenage pregnancy. There was surprisingly little data on emotional and mental health.

Our qualitative synthesis suggests the complex pathways through which poor school organisation and management may lead to physical and mental health harms. First, students' responses to schools in which they feel educationally marginalised or unsafe may actually exacerbate school disengagement and overall lack of safety. For example, when students adopt substance use and ‘tough fronts’164 based on aggressive behaviours to facilitate a ‘safe’ identity and build protective social support, this may reinforce conflict and unhappiness at school. Second, strong positive teacher–student relationships appear to be critical in promoting student well-being and avoidance of risk behaviour. Organisational features of secondary schools in the USA, the UK and elsewhere appear to undermine such relationships. In particular, through schools' limited involvement of students in decision-making, students can lose what social control theory defines as a ‘stake’ in their school community, thus increasing the likelihood that they will look for a sense of identity and social support through some of the risk behaviours suggested above.

The meta-ethnography also therefore resonates with the findings of the synthesis of the multilevel studies in suggesting that schools in which relationships between teachers and students are not strong are less likely to protect their students from harm and are more likely inadvertently to promote substance use and other risk behaviours. Qualitative research also provides new insights regarding how students' lack of satisfaction with school life may cause them to seek sources of ‘escape’, either through heavy patterns of substance use or by physically leaving school at lunchtime or for longer unauthorised spells. While the focus of the intervention and multilevel studies was on impacts on students overall, telling us very little about subgroup effects or health inequalities, a large proportion of the qualitative studies focused on young people growing up in socially and economically disadvantaged families and communities. This qualitative evidence also suggests that the extent to which schools protect and engage students, and ensure that teachers fulfil a broader role in young people's development, may be most important in such settings, because of the higher rates of violence, drug use and other risk behaviours in those communities, inequities in school funding and higher rates of staff turnover.

Strengths and limitations

The benefit of using a meta-ethnographic approach compared with narrative approaches is that it moves beyond providing a review of the accounts of individual studies and instead develops higher-order explanations. The inductive and interpretive approach helps illuminate meaning that can be valuable to policy-makers looking to understand behaviours in context, perceptions and unintended consequences. In this review the development of meta-themes can also be used to interrogate the theory synthesis and vice versa, building on established theories and identifying conceptual gaps in the literature. The value of a meta-ethnographic approach is supported by the remarkable consistency in the findings of studies of variable quality undertaken in a wide range of settings, which differed by school system, deprivation level and ethnic make-up.

A potential limitation is that we may have covered some key concepts and themes in insufficient depth in attempting to translate the themes across studies and identify meta-themes. We attempted to preserve individual authors' interpretive frameworks by ensuring that all themes extracted from individual papers were accompanied by a narrative explaining how they are connected in order to report these relationships when synthesising the findings of multiple studies. Inevitably, however, some of this work is compromised when moving to higher levels of abstraction.

The findings presented in this chapter may not cover all of the relevant qualitative literature available. For example, we did not include reports that might address student health and the school environment when this is not the primary research aim of the study. Only those reports whose explicit and primary purpose was to address student health and its relationship with the school environment were included, which ignores a large body of work conducted by sociologists of education that may also help to understand the processes by which schools influence health outcomes.

Deviations from protocol

We deviated from protocol in five minor ways. First, because of lack of time we decided not to search for research reports that cited already-included reports. Second, because of recruitment delays, we did not, in our initial sifting of identified references, have three reviewers working in parallel comparing answers in batches. Instead, sifting was carried out by CB working in parallel with each researcher as and when they started work on the project, still comparing answers in initial batches of at least three sets of 50 until at least 99% agreement (as against 95% in the protocol) was reached. Third, we did not extract report authors' conclusions about results because we concluded that these would not inform our review and synthesis. Fourth, our protocol stated that, for coding-included reports for the evidence map, two reviewers would code an initial sample in parallel to reach consensus and subsequent coding would be carried out solely by one reviewer. However, in practice, all coding was carried out in parallel by two researchers, meeting to resolve disagreements. Fifth, we consulted with young people twice near the beginning of the project to ensure that we examined areas of priority to them rather than, as originally intended, consulting with young people once on the evidence map and once on the draft final report.

In other cases we did not strictly depart from protocol but nonetheless changed our plans, which we report here in the interests of transparency. Our protocol is only explicit about not excluding reports by language of publication with reference to our initial mapping in stage 1, which we followed. However, we did exclude reports not written in English in our in-depth review in stage 2. This applied only to 12 studies examining school-level effects on student health. These non-English reports all had abstracts written in English and from these we judged that there was little chance of them having employed multilevel modelling to examine school effects. If these reports had been published in English we would have confirmed this by obtaining the full paper; however, we did not do this because of the time and expense that this would have incurred for what we judged to be little likelihood of benefit.

Our plans for in-depth review in stage 2 were to develop a narrative synthesis of all multilevel studies focused on ‘objective’ measures of the school environment (i.e. not derived from aggregations of data from the same individuals from whom outcome data were collected). We included a greater than expected number of reports in stage 2. Our review of study methods had established that, although many studies were poorly conducted and reported, there was nonetheless a critical mass of 10 reports that were well conducted and reported, and which took what we judged to be a sound approach to adjusting for covariates. We therefore decided to limit our narrative synthesis of multilevel studies to these 10 reports.

Other limitations

This was intentionally an exploratory review that examined a relatively heterogeneous group of interventions. An alternative approach would have been to use our mapping in stage 1 to determine a more homogeneous group of interventions for in-depth review. However, this would have gone against the conclusions from our preliminary summary of theory and stakeholder consultation that what was most useful was a synthesis focused on the health effects of schools' organisation, management and physical environment, and the way that schools provide teaching, pastoral care and discipline – which necessarily involved a diverse group of interventions and school-level influences. Nonetheless, we would argue that future reviews might now focus on particular aspects of schools' social and physical environments.

Although a strength of our review is that it aimed to synthesise intervention and school-level effects on a broad range of health outcomes, this review did not examine other social, psychological and educational outcomes that might mediate effects on health. However, this would be an interesting focus for future work.

We focused our in-depth review on studies of student health, but not staff health. This was a pragmatic decision taken to ensure that we had a manageable number of reports. This is nonetheless an important area of research. One of the authors (CB) is currently engaged in a systematic review of organisational interventions to reduce teacher stress and promote well-being.

We took a very broad approach to searching electronic databases because of the multidisciplinary nature of the review and the variability in the keywords that are applied to these reports. This produced a very large number of reports to sift. It is likely that this process led to some pertinent reports being missed or accidentally excluded. We therefore complemented this approach in stage 2 by checking potentially relevant studies cited in those reports we had already included, as well as contacting authors of included reports for advice about any studies we had missed. However, we did not undertake any hand-searching of journals or web searches because the relevant literature was spread over a wide range of journals and other sources.

We coded references for our map based purely on title and abstract because of the time pressure created by the large number of reports we had to sift. Although this meant some relevant reports being excluded, as explained above, as well as some irrelevant reports being included, we nonetheless believe that the map is fit for its purpose, of describing the broad spread of research on the topic of the effects of schools and school environment interventions on health, and informing our stakeholder consultation.

Our selection of theories to inform our review hypotheses was partially subjective. We used multiple criteria to form a judgement about which theories to draw on. We did not simply require that a theory meet every criterion to be deemed primary because our judgements were necessarily more subtle than this. For example, the theory of human functioning and school organisation did not meet our criterion of simplicity. However, we judged that this theory was very strong regarding our other criteria and so included it. As a further example, several of our theories did not meet our quality criterion of addressing a range of health outcomes, but we judged them to be sufficiently useful in understanding school effects on ASB-related outcomes and so we opted to use them to inform secondary review hypotheses. We think that this balance between using clear criteria and making overall judgements is acceptable and appropriate given that these concerned the development rather than the testing of hypotheses.

Although we developed detailed guidance about which measures would ‘count’ as health outcomes, in practice we found that we had to make judgements. Although the wisdom of these judgements could be open to challenge, we did apply them consistently throughout our review, in relation to both outcome evaluations and multilevel studies of school effects on health. For example, some evaluations were excluded because what were described as health outcomes in fact involved composites that included health and educational variables together. We included conflict resolution practices but no other prosocial forms of behaviour such as sharing; measures of loneliness but not the quantity or quality of family or peer relationships; and measures of feeling safe at school but not engagement in education.

Our hypotheses did not prespecify which outcomes should be treated as primary and secondary. As a result, our narrative synthesis might run the risk of overinterpreting statistically significant findings, which may reflect the multiplicity of outcomes being reviewed rather than real intervention effects. However, to counter this we have tried within our narrative synthesis to give equal weight to findings of no significant statistical associations and in summing up the results to comment on the balance between significant and non-significant associations. Nonetheless, we recognise that this is no substitute for testing specific hypotheses defined in terms of a small number of primary and secondary outcomes and so we urge that our findings be interpreted cautiously.

Our synthesis of the theories informing empirical research in this area was highly innovative and useful in both determining the priorities for the in-depth review and interpreting empirical studies. However, it is clear from the preceding sections, which report implications of the empirical studies of school environment interventions and multilevel models of school-level health effects (see Chapters 7–9), that the empirical research cannot definitely determine the validity of our hypotheses, both because the empirical studies were not focused on testing these and because of limitations in the measures of school environment used in such studies and our focus on health outcomes, which meant that we could not examine the relevant evidence on the more proximal outcomes in the causal pathways linking the school environment to health. 60 We did not, for example, synthesise evidence examining how school environment interventions and school-level determinants affected student attendance, engagement or attainment outcomes.

It is also clear, as reported earlier, that the theories that we identified were biased towards the study of ASB. However, this simply reflected the theories that current empirical studies of the health effects of schools and school environment interventions are using. Although this might influence how our overall synthesis is presented in this chapter, it should not bias the presentation of the results of the constituent reviews presented in Chapters 7–10 because these simply present study results and their implications for our RQs rather than aiming to assess our review hypotheses.

Stage 1: identifying and describing reports

Stage 2: in-depth synthesis

After applying further exclusion criteria, our in-depth review examined 82 separate reports: two stand-alone theory papers, 16 intervention outcome evaluation reports (10 studies), one additional report of a process evaluation (four studies in all), 42 reports of multilevel studies of school-level health effects (34 data sets) and 21 reports of qualitative studies of processes of how schools might influence health.

Consultation with policy and practice stakeholders

We consulted with individuals working in policy (n = 2), research (n = 1) and practice (n = 1) on a first draft of this report, which at that point summarised findings from each of the different sections of our review but lacked an overall synthesis. This consultation aimed to get advice regarding what might be our key messages and how these might be disseminated in order to influence policy and practice. These discussions supported our aim of harnessing the evidence from the different sections to consider and refine existing theories and to ensure that any recommendations for interventions we produced stressed the importance that these be rigorously evaluated before any wider implementation.

Implications of our evidence for the primary and secondary review hypotheses

Our review of theories and stakeholder consultation in stage 1 led us to focus on how schools are organised and managed, are designed and built, and provide teaching, pastoral care and discipline. These are more ‘upstream’ determinants of student health than determinants such as what food schools provide or how they deliver PE. We then reviewed, in depth, theories about how schools influence student health cited in stand-alone papers or in empirical studies included in our review. We identified three primary and 10 secondary theories (Table 7) that were most relevant to understanding what specific aspects of schools affect health. We will now summarise what the empirical studies we reviewed suggest about the validity of these theories.

The intervention studies themselves did not set out to test any of our review hypotheses and were not informed by the theories from which these arose. Most of the interventions also employed multiple components so that it is not possible to take evidence of intervention effects as evidence of causal relations between the constructs within our review hypotheses and student health outcomes. As we will describe below, it is not clear, for example, whether the CDP's positive effects reflected its encouragement of student participation (which would support the social development model), its development of strong relationships between staff and students (which might support either social capital theory or the theory of human functioning and school organisation) or the process of giving students greater ownership – and thus a greater ‘stake’ – in their school (which would support social control theory). For these reasons, the intervention studies cannot offer very definitive evidence for or against our review hypotheses. Nevertheless, the possible ways in which the intervention studies might support or undermine our review hypotheses are considered below.

How the interventions addressed the domains specified in our primary and secondary review hypotheses is outlined in Table 8.

Three RCTs44,54,55,58,120,121,123 and one quasi-experimental study60 evaluated interventions that encouraged staff and students in a range of elementary/middle schools in the USA and in secondary schools in the UK to develop school climates characterised by a stronger sense of community. For example, the HSE intervention focused on developing better relationships; the CAPSLE intervention aimed to develop student participation in mediating conflict; and the PeaceBuilders and CDP interventions were aimed at enabling better interpersonal relations. These evaluations generally reported benefits. For example, CDP evaluations reported benefits for emotional health and conflict resolution. 54,55,58,123 The CAPSLE and PeaceBuilders findings suggested positive effects on various measures of victimisation and aggression119–121 and the HSE study reported benefits for perceived student safety. 60 However, the AAYP evaluation reported fewer significant benefits regarding violence, recent sexual intercourse and condom use when comparing school environment change plus a social skills curriculum component with a curriculum-only intervention. 44 In contrast, although not a focus for our review, when comparing the combined intervention and the curriculum-only intervention with no intervention, findings for the combined intervention were more positive, suggesting that the school environment component may have been a key active ingredient.

With the caveats noted above, this evidence might lend broad support to each of our three primary hypotheses arising from the social development model (regarding the importance for health of participation in school activities), social capital theory (regarding the effects of trusting relationships) and the theory of human functioning and school organisation (regarding the importance of eroding rigid social boundaries between staff and students and that more student-centred framing of activities will enable better health outcomes). In terms of our secondary hypotheses, these studies suggest the importance of cognitive development, bonding between students, prosocial norms and protecting students from involvement in risk behaviour, which are key concepts in the theory of triadic influence. The importance of giving students a stake in their school community also suggests the possible validity of the hypothesis arising from social control theory. These findings from interventions that did not aim to increase the severity of sanctions accorded to aggression and violence but which nonetheless appeared to improve conflict resolution provide no support for our secondary hypothesis derived from deterrence theory. Finally, these interventions' broad focus on improving the general psychosocial environment also supports the hypothesis from the ecological model of co-ordinated school health programmes. A final caveat is that these studies of interventions aiming to build the school community generally had methodological problems such as small sample sizes and high or differential attrition.

Two RCTs63,122 focus on interventions in middle schools in the USA aimed at promoting healthy eating and physical activity through greater student participation in decision-making and modifying food and physical activity environments at school. Both reported mixed findings, but more positive effects for physical activity than for healthy eating. It is difficult to assess whether or not these effects might be attributed to student participation, although a mediation analysis conducted in one study suggested that this might be the case. With this caveat, the reported benefits for physical activity offer support to our review hypotheses about the importance of participation from the social development model and the theory of human functioning and school organisation. Three quasi-experimental evaluations117,118,124–126 examine changes to playground physical environments, but report mixed findings and do not resonate with constructs in our review hypotheses.

The multilevel studies provide greater insights regarding our review hypotheses because some of these explicitly set out to test hypotheses arising from the theory of human functioning and school organisation and because other studies, although not explicitly focused on our own review hypotheses, examined the specific effects of school-level factors that are closely aligned with the constructs in some of our hypotheses. How the multilevel studies examined the domains specified in our primary and secondary review hypotheses is outlined in Table 9.

In particular, four multilevel studies53,64–66 provide insights into the theory of human functioning and school organisation in relation to secondary/middle schools in the UK and USA, particularly with regard to substance use. A cross-sectional study by Aveyard et al. 64 of students aged 11–16 years in English secondary schools reports an association between more ‘authoritative’ schools (based on evidence that these schools had higher than expected attainment and lower than expected truancy) and lower rates of regular smoking. Drawing on the same data, Bisset et al. 66 find that authoritative school status is also associated with lower rates of various measures of alcohol and drug use. Markham et al. 65 use longitudinal data tracking students aged 13–14 years for 2 years in English secondary schools and report an association of borderline significance between authoritative school status and weekly smoking. Tobler et al. 53 also report on longitudinal data tracking students aged 11–12 years for 2 years in middle schools in the USA, using similar measures to those of Aveyard et al. 64 and Markham et al. ,65 and note that authoritative schools also had lower rates of smoking, drinking and drug use as well as group fighting in this context.

These findings are offered by the authors themselves as evidence in support of their theory of human functioning and school organisation. They suggest that authoritative schools provide better social support and an appropriate level of control of students' behaviour so that students in such schools are more likely to adopt pro-school identities and values, and less likely to invest in anti-school risk behaviours such as smoking. They suggest that their measures of whether or not schools had higher than expected attainment and lower than expected truancy based on their intakes (i.e. a school's ‘value-added’ score) are proxies for this institutional culture. These studies incidentally also support the hypotheses emerging from the social development model (i.e. that schools providing students with good opportunities to participate in learning and community life are more likely to enable the development of prosocial behaviours). Although we would agree that these studies provide support to the theory of human functioning and school organisation, we reiterate the caveats listed in Chapter 9 concerning the cross-sectional nature of two of the four studies and the limited insights that such crude measures of school culture can provide into the causal pathways involved.

Evidence from the multilevel models we have reviewed is less well oriented towards testing the secondary hypothesis derived from deterrence theory that schools will limit risk behaviours by setting severe rules against such activities, and currently provides no clear evidence for or against. Evans-Whipp et al. 131 reported cross-sectional data for students aged 11–16 years at secondary schools in the USA and Australia, suggesting no association between school rules and policies on student smoking and student self-reported smoking outcomes (although they do find an association between strict enforcement of smoking policy and reduced perception of other students smoking on school grounds). Piontek et al. 132 report no associations in cross-sectional data between smoking among students aged 10–21 years and restrictions on adult smoking, monitoring of student smoking and smoking policies in German secondary schools, but they do find an association between a complete student smoking ban and reduced rates of student smoking. As discussed earlier, these differences may reflect a ceiling effect: while German schools have often tolerated smoking among staff and older students so that smoking bans have scope for impact, in the USA and Australia there is a more pervasive intolerance of smoking so that the effects of differences in smoking policies between schools are difficult to detect. Cross-sectional data from Dutch secondary schools suggest that neither whether or not a school permitted alcohol use at school on special occasions nor the severity of sanctions was associated with student alcohol outcomes. 69

Cradock et al. 133 reported cross-sectional associations between the total area of the campus and playground of US middle schools and higher levels of physical activity during school hours, providing some support for our secondary hypothesis derived from the ecological model of co-ordinated school health programmes that schools foster health by enabling physical activity within school. The finding of Kumar et al. 68 that the total number of observable and unsupervised places in US high schools was associated with twelfth graders' use of alcohol in school in the past year, tenth graders' use of alcohol in school in the past year and tenth graders' use of marijuana and other illicit drugs in school in the past year but not with measures of substance use occurring both inside and outside school provides some support for our secondary hypothesis that schools can promote health by reducing opportunities for risk behaviour derived from the theory of triadic influence.

Our meta-ethnography of qualitative evidence enables us to develop a refined theory of how the school environment influences health.

Markham and Aveyard's62 theory of human functioning and school organisation suggests that schools comprise two ‘orders’: the ‘instructional’ order (which provides students with knowledge and skills) and the ‘regulatory’ order (concerned with students' conduct, character and membership of the school community). Our meta-ethnography suggests that there are two distinct, but overlapping, ‘systems’ that operate in the school environment: the student peer system (comprising student-led structures and processes) and the school institutional system (comprising structures and processes involving school management, teachers and other school staff and their use of technologies such as CCTV). Both systems are recognised in Markham and Aveyard's62 theory of human functioning and school organisation, but the former student-led system is not greatly elaborated on by them. Markham and Aveyard62 would refer to the separation of the student and institutional systems in terms of Bernstein's105 notion of ‘classification’, which refers to the way in which boundaries between the students and the school staff are formed and modified over time. The strength of the boundaries between the student population and the school's other institutional actors is theorised as determining student capacity for affiliation and practical reasoning and, ultimately, their health outcomes. For example, students who remain uncommitted to the instructional and/or regulatory orders are theorised as more likely to become instead committed to anti-school peer groups and behaviours, but these latter processes are not elaborated. However, we aim to refine the theory of human functioning and school organisation by considering in more detail the importance of young people's agency, the student peer system and how this interacts with the school institutional system, both to define the ethos of a school and to structure students' health-related actions and outcomes.

The student peer system, like the school institutional system, is guided by a set of social norms, relationships, rituals and symbolic practices, and is also influenced by broader social and structural factors beyond the boundaries of the school. In line with Bernstein105 and Markham and Aveyard,62 we subdivide the institutional features of schools according to their ‘instructional’ and ‘regulatory’ functions. However, as our qualitative synthesis highlights, students not only react to the organisational systems put in place by the institution, but also promote their own versions of these orders. We would suggest that the importance of the student peer system is only partially theorised and that these orders should be reconceptualised to acknowledge more explicitly the student-led as well as institutionally driven nature of the instructional and regulatory orders.

The instructional order of the student peer system is apparent, for example, in the process of learning and adopting symbolic styles of dress and demeanour, and related practices such as violence and substance use. These may be critical to students' safety and social status, particularly in schools that institutionally fail to secure students' engagement or safety. The ‘tough’ performative rituals consistently reported in the qualitative studies we have reviewed are not indications of ‘anarchy’ or ‘chaos’. Rather they form the symbolic basis of the peer system's own regulatory order. Social control, mediated through rules and norms, seems to be operating among the student body, but on the terms of young people themselves. Take, for example, the rigid rules that students in some schools follow when confronted with a violent incident, such as the ‘one-to-one-fight’ rule or the practice of linking arms to block adult intrusion. In terms of social capital theory, which our theory synthesis also drew on and which our empirical evidence broadly supports, this also creates an opportunity for students to build bonding social capital102 and create shared values and obligations which may be critical in school environments that are unable to foster these by other means. In such contexts, students establish their own regulatory order with the aim of facilitating attachment to the peer group and socially regulating these seemingly ‘antisocial’ risk behaviours. These phenomena can also be understood through the lens of the social development model, which would suggest that participation in ASBs has its own reward structure.

These interpretations resonate with Bourgois'165 notion of ‘street culture’ and Dance's164 concept of ‘tough fronts’, which conceptualise young people not merely as victims of structural poverty but as agents struggling for meaning and survival, and ultimately reinforcing existing educational and health inequalities. Our argument is that this connection to the peer system's instructional order, as with that to the school's institutional system, enables young people to develop capacity for practical reasoning. In those schools lacking what Markham and Aveyard62 term ‘institutional authority’, this reasoning may provide students with the ability to develop anti-school identities as well as the knowledge and skills required to engage in symbolic practices such as violence and substance use, paradoxically to stay safe and get by. The peer regulatory order also provides a source of affiliation. In schools lacking widespread student connection with the institutional regulatory order, this affiliation may reinforce and regulate violence, substance use and other risk behaviours. Thus, rather than risk stemming from an absence of practical reasoning, affiliation and autonomy, risk actually arises from students developing the autonomy to engage in behaviour that is often regarded as antisocial but which is thoroughly social in its origins.

Finally, it is important to recognise that the way in which the student peer system operates will reflect the way in which the school institutional system operates and the degree to which it is authoritative. The peer system will also in part determine how the formal institution operates. For example, Devine151 illustrated that it is far harder for the institution to be authoritative in a context in which students have little hope of conventional social advancement and are deeply immersed in street culture.

This revised framework therefore builds on Markham and Aveyard's62 theory of human functioning and school organisation and the empirical evidence we have reviewed, which suggests that the student population not only reacts to the instructional and regulatory orders established by the school institutional system, but also produces its own set of orders that also shape school organisation and their own risk behaviours. In the qualitative research we have reviewed, the separation of these two systems represents a lack of co-operative functioning of shared norms and understanding between students and the institution, a condition most pervasive in urban contexts of poverty and disadvantage. In this context, students must protect themselves and develop relationships by means of their own intervention. The strategies they adopt, however, create a vicious circle whereby acting ‘tough’ or ‘escaping’ the school may lead to aggressive behaviours, poor eating habits, drug and tobacco use and poor sexual and bodily health practices. These behaviours in turn reinforce the boundaries between students and the authority network, which inhibits the realisation of ‘pro-school’ reasoning and affiliation for students and ultimately students' capacities for choosing to be healthy.

These suggestions are consistent with the theory of human functioning and school organisation. They are intended to underline the importance of the student peer system in defining a school's ethos in interaction with the formal institutional system, and to describe the ways in which lack of student connection to the institutional instructional and regulatory orders may predispose students' engagement in risky peer groups and behaviours. We acknowledge that the way we have refined Markham and Aveyard's62 theory inevitably reflects the range of qualitative research synthesised in this report, and the emphasis on the disconnection between the school institutional and student peer systems is likely to be partly a reflection of the fact that the majority of these studies examined schools in the most deprived urban/inner-city contexts where students and teachers may have the least in common.

Research implications

Implications for public health