The effects of Positive Youth Development interventions on substance use, violence and inequalities: systematic review of theories of change, processes and outcomes

Search terms

A sensitive search strategy using both indexed and free-text terms was developed and tested by an experienced information scientist (CS). These searches were run on 7 November 2013. Key search terms were determined by the RQ and the inclusion criteria and were developed and tested against papers already known to the research team in writing the research proposal. The search strategy involved developing strings of terms and synonyms to capture three core concepts in the review:

1. Concept 1: population (e.g. youth or young people or adolescents).

2. Concept 2: intervention (e.g. after-school clubs or community-based programme or informal education).

3. Concept 3: population and intervention (e.g. youth work or youth development or youth club).

These concepts were combined in searches as follows: concept 1 and (concept 2 or concept 3). The initial free-text search terms generated for concepts 1 and 2 were broad and could identify non-relevant literature (e.g. volunteer, development, etc.). Thus, to balance specificity with sensitivity, where possible we required that intervention terms were adjacent or near to population terms (e.g. after ‘school’ N12 ‘young people’ OR ‘adolescen*’ OR ‘youth’). An example of a search string from PsycINFO can be found in Appendix 4.

Databases

Searches were undertaken on the following 21 electronic bibliographic databases:

• Applied Social Science Abstracts via ProQuest

• Australian Educational Index via ProQuest

• BiblioMap health promotion research via EPPI-Centre

• British Educational Index via ProQuest

• Child Data via Ovid

• Cochrane Central Register of Controlled Trials via The Cochrane library

• Cumulative Index to Nursing and Allied Health Literature via EBSCOhost

• Database of Abstracts of Reviews of Effects

• Database of Promoting Health Effectiveness Reviews via EPPI-Centre

• Health Technology Assessment Database

• EconLit via Ovid

• Education Research Index Citations via ProQuest

• International Bibliography Social Sciences via ProQuest

• MEDLINE via EBSCOhost

• NHS Economic Evaluation Database

• PsycINFO via EBSCOhost

• Sociological Abstracts via ProQuest

• Social Sciences Citation Index via Web of Knowledge

• Social Policy and Practice via Ovid

• Social care online via Ovid The Health Management Information Consortium via Ovid

• Trials Register of Promoting Health Interventions (via EPPI-Centre).

Other search sources

The following websites were also searched to identify relevant studies:

• The Campbell Library

• International Clinical Trials Registry Platform

• OpenGrey

• The Health and Safety Information Centre of the International Labour Office

• Drug and Alcohol Findings Effectiveness Bank

• Dissertation Abstracts/Index to Theses

• Schools and students health education unit research archive

• Google Scholar (Google Inc., Mountain View, CA, USA)

• Google (Google Inc., Mountain View, CA, USA)

• US Center for Substance Abuse Prevention

• Northern Ireland Online Research Base

• US Food and Drug Administration–Centers for Disease Control and Prevention Youth Tobacco Prevention

• Tobacco Use Behaviour Research

• UK Clinical Research Network Study Portfolio

• Personal Social Services Research Unit Discussion Papers (see www.pssru.ac.uk/)

• Cost-effectiveness Analysis Registry

• National Youth Agency

• Social Policy Digest

• Drug database (see www.drug.org.au/)

• Smoking and Health Resource Library

• Child and Adolescent Health Research Unit

• Children in Scotland

• Children in Wales

• Children’s Research Centre

• Welsh Government Social Research

• Scottish Government website

• Young Minds

• Social Issues Research Centre

• European Commission – Community Research and Development Information Service library

• Centre for Prevention Research and Development

• Joseph Rowntree Foundation

• Childhoods Today

• The Children’s Society

• National Youth Agency

• National Criminal Justice Reference Service Abstracts.

Website searches were run between 7 and 16 January 2014. Depending on the functionality of each website interface, searches were undertaken using a combination of medical subject headings (if available) and/or free-text search terms to capture the key concepts of the intervention (e.g. ‘positive youth development’; ‘youth work’; ‘youth clubs’). Citations were screened online based on their title, title and abstract or full text when available. Potentially includable studies were cross-referenced with the electronic searches imported to EPPI-Reviewer version 4.0 (Evidence for Policy and Practice Information and Co-ordinating Centre, University of London, London, UK) to identify any unique references. As is customary with searching of this type, only included references were recorded.

We also hand-searched those journals that (1) contained studies that we included; (2) were found only via reference checking; and (3) were not indexed on databases that we had searched. We hand-searched these initially for the past 5 years and if these elicited > 1 new included study we hand-searched for a further 5 years. Our original proposal stated that we would hand-search the five journals that yield the highest numbers of studies meeting the inclusion criteria. However, we decided very early on in the review that this approach would have low specificity in identifying studies not already identified by our other methods of searching and was thus not a good use of resources. We therefore amended this aspect of our search so that hand-searching was much more targeted towards studies that were unlikely to have been identified through other means, including this in our registered protocol. 51

We also sought to contact subject experts to identify unpublished or ongoing research. We pursued this by contacting the authors of all included studies to seek their advice on other studies that we might consider for inclusion. As stated above, the International Clinical Trials Registry Platform was also searched to identify any relevant ongoing and unpublished trials on PYD. Finally, we searched reference lists of all included studies for further relevant studies.

Principles/assumptions that underlie Positive Youth Development

Included reports suggested a number of principles and assumptions that appeared to underpin PYD interventions.

Normative vision of what positive development should involve

Another subtheme within our coding template concerns what positive development should involve.

The required characteristics of Positive Youth Development interventions

Another subtheme within our theme of normative theory is the required characteristics of PYD interventions.

Challenging or contributing to the existing social order

Following on from the earlier subtheme of contributing to or challenging the existing social order, we might identify a distinction between ‘radical’ and ‘mainstream’ PYD programmes. For example, Berg et al. 86 draw on the theorists such as Gramsci,137 Bourdieu,138 Foucault139 and Freire140–142 to develop an intervention which uses a participatory action research approach to enable young people to develop a critical consciousness of, and to challenge, oppressive forces such as discrimination and racism within society. This stands in contrast to the majority of descriptions of PYD which emphasise the promotion of young people’s bonding to, and contribution towards, reproducing current societal arrangements.

Collectivist or individualistic orientation

In addition, informed by our earlier subtheme of PYD addressing individual and/or collective assets, we can contrast PYD where it is depicted as an intervention to promote individual development versus characterisations of PYD as a collective endeavour. For example, Schwartz et al. 126 argue that, for some communities, particularly in some non-western cultures, interventions need to place less emphasis on individual development and more emphasis on group empowerment. Interventions that emphasise individual agency may be less useful in some cultural contexts than others. As noted above, Berg et al. 86 make a similar argument, but one informed by a belief in the political importance of collective action.

Focusing on the individual and/or the environment

Roth and Brooks-Gunn50 contrast intervention activities that aim to modify both an individual young person as well as the context within which they live with those that focus only on the former. Referring to the asserted importance of PYD compensating for limitations in the environment within which young people develop, Roth and Brooks-Gunn50 acknowledge that the capacity for compensation by any one PYD programme is limited and what is needed is for PYD principles to inform the transformation of wider environments and services. They contrast PYD which aims to ‘prepare adolescents for the world by ensuring that they possess the 5 Cs, [versus that which aims] to shape a better world for youth by also increasing the supports available to them at home, school, and in their community’. Making a similar point from within the more radical strand of PYD theorising, Ginwright and Cammarota119 argue that a lack of focus on social, economic and political forces is a major limitation in mainstream PYD.

Breadth, depth and duration of activities

Another means of categorising PYD interventions is in terms of the breadth, depth and duration of activities. Roth and Brooks-Gunn50 report that programmes vary in which, and how many, assets they seek to enhance. Busseri et al. 117 distinguish interventions that offer a breadth of participation in many activities and those that offer more in-depth participation in fewer activities. Busseri et al. 117 argue that breadth of activities might be more important in early adolescence when young people are experimenting with different identities, whereas in-depth participation in selected activities may become more important in later adolescence as identities begin to stabilise during the transition to adulthood.

Prevention and promotion

Finally, we can distinguish between PYD programmes that aim only to promote positive assets and those that also aim to address risks. Schwartz et al. 126 suggest that the prevention of problem behaviours and the promotion of thriving can occur together: ‘prevention science and positive youth development may appear to represent incompatible approaches at worst . . . or complementary approaches at best’.

Catalano et al. 118 and Schwartz et al. 126 argue that positive assets and protective factors for the prevention of risk are in practice often the same thing, with communication skills and self-awareness being both positive assets in themselves as well as assets that provide protection against risk behaviours such as violence and substance use. In such cases, PYD interventions might reduce risks without explicitly addressing them. In other cases, PYD programmes might go beyond the promotion of positive assets so that they also address protective factors which are not themselves positive assets, such as lack of access to tobacco, as described by Catalano et al. 118

Positive Youth Development promoting intentional self-regulation

A key subtheme across a number of studies86,113–117,120,123–126 is how PYD promotes ‘developmental regulations’. 124 This notion of developmental regulations is said to be rooted in ecological systems theory, a widely used theory of how individual development is shaped by social contexts. 144 Lerner et al. 124 define developmental regulations as bi-directional interactions between the individual and their social environment (such as their peer group, family, school and neighbourhood):

These bidirectional relations may be represented as individual/context relations. These relations regulate (that is, govern) the course of development (i.e., its pace, direction, and outcomes).When these ‘developmental regulations’ involve individual/context relations benefiting both the person and his or her ecology, they may be termed ‘adaptive’.

But how might PYD interventions enable developmental regulations to occur? Busseri et al. ,117 Schwartz et al. 126 and Lerner et al. 124 all suggest that PYD improves young people’s capacities for ‘intentional self-regulation’. 124 According to Lerner et al. :124

[I]ntentional self regulation may involve the selection of positive goals (e.g. choosing goals that reflect important life purposes), using cognitive and behavioral skills (such as executive functioning or resource recruitment) to optimize the chances of actualizing ones purposes and, when goals are blocked or when initial attempts at optimization fail, possessing the capacity to compensate effectively.

Benson113 similarly suggests that PYD can help adolescents to reflect on existing behaviour, select personal goals and activities through which to pursue these, and apply available resources to pursue these goals and activities. Lerner et al. 124 suggests that if PYD programmes promote young people’s senses of hope and their ability to intentionally self-regulate and provide them with opportunities and resources such as relationships and education, then positive ‘development regulations’124 will occur. Lerner et al. ,124 as well as Benson,113 suggest that this will enable virtuous circles to occur which enable positive development in the individual as well as improvements in their social environments via the young person’s positive contribution to these.

However, although these authors argue that such processes of intentional self-regulation are required in order for young people to accrue positive assets, they are not explicit about how PYD interventions actually enable young people to develop their capacity for intentional self-regulation. Benson113 suggests that PYD interventions can optimise these processes by:

[I]ncreasing the developmental attentiveness of contexts . . . to increase their capacity to nurture, support, and constructively challenge the developing person, . . . [E]nhancing the skills and competencies of youth (to further enable their ‘natural’ capacity to engage with, connect, change, and learn from their social contexts . . . [and [c]reating processes and opportunities to invite youth to actively exercise and utilize their capacity to engage with and change their social contexts

But this quotation, as well as the theoretical literature more generally, stops short of explaining exactly how PYD programmes might promote better intentional self-regulation either with regard to specific intervention activities or more generally. However, informed by the claims in reports synthesised within our normative theme above (e.g. by Benson et al. 114), we might fill in the gaps, speculating that PYD could promote general improvements in intentional self-regulation by providing individuals with the resources, in the form of relationships and training in specific skills, that are the critical inputs upon which the use of intentional self-regulation within specific intervention-related activities depends and by enabling individuals to practise intentional self-regulation in the context of multiple activities and settings so that they improve their general ability to intentionally self-regulate.

Busseri et al. 117 come closest to making this explicit. They argue that PYD can provide a range of ‘affordances’ (opportunities) to which young people may respond. Different individuals, with different needs and goals at different points in their maturation, will make use of different affordances. Busseri et al. 117 argue that PYD should provide diverse opportunities for young people to explore different activities and then to commit to pursuing a narrower range of activities in depth. The extent to which a young person commits to an activity and to the development of specific assets may indicate growing stability in an identity.

A final way in which PYD interventions might help young people to use intentional self-regulation in order to develop positive assets is by refocusing existing capacities for intentional self-regulation away from antisocial goals and towards pro-social goals. This might occur via rewarding young people when they abandon antisocial activities and engage in specific pro-social activities. Kim et al. ,121 for example, refer to social learning theory (Akers et al. 145) to suggest that, by providing positive examples and celebrating achievements in the realm of pro-social activities, PYD programmes can reinforce positive behaviours and bonds to conventional society.

Theories of risk reduction

Despite being the central question informing our synthesis, how interventions reduce risk behaviours such as substance use and violence was a relatively minor theme in the PYD literature, a point acknowledged by Kia-Keating et al. 120 and Lerner et al. 124 Lerner et al. 124 for example recognise that ‘the PYD perspective needs to . . . focus more on what seem to be quite diverse and intricately linked pathways of thriving and of risk/problem behaviours’. The PYD literature offered a number of generic suggestions about how the development of positive assets might reduce risks, as well as a few isolated examples of how specific assets might protect against risk, but stopped short of offering a comprehensive theory of change for how PYD reduces risk.

Factors potentially moderating the impact of Positive Youth Development

Our synthesis also aimed to examine what included theory reports had to suggest about how the effectiveness of PYD interventions might be moderated by various factors. A number of disparate suggestions from the literature can be identified.

Taxonomy of Positive Youth Development

Positive Youth Development can be categorised according to:

• whether it aims to challenge or contribute to the existing social order

• whether it aims to promote the development of individuals or groups of individuals

• whether or not it aims to promote the development of the individual and/or modify the wider environment in which the individual develops

• the breadth, depth and duration of activities provided and

• whether or not it explicitly addresses the reduction of risk behaviours.

Our aim was to combine this taxonomy with categories arising from the synthesis of process evaluations and to use the combined categories to describe interventions included in the synthesis of process and outcome evaluations.

Theory of change for Positive Youth Development effects on violence and substance use

Our synthesis of how PYD is theorised to promote positive assets and reduce substance use is summarised in Figure 6. PYD interventions are intended to provide positive expectations; enduring and affective relationships with adults; diverse activities and settings; and active participation, whereby young people are empowered to choose activities and to take on responsibilities. Interventions vary according to whether or not they aim to enable young people to contribute to or challenge the existing social order; whether or not they focus on individual and/or collective development; whether or not they address the individual and/or the environment in which they live; the breadth, depth and duration of activities offered; and whether or not they address only positive development or also prevention of risk behaviours.

FIGURE 6.

Synthesised theory of change for PYD effects on substance use and violence.

Positive Youth Development aims to offer ‘affordances’,117 that is, resources that individuals make use of in the course of their development (e.g. relationships, challenges, education). Different individuals will make use of different affordances at different points in their maturation. Young people will make use of these affordances in applying intentional self-regulation to specific intervention-related activities, such as sports, arts or outdoor activities. Intentional self-regulation comprises intentionality (assessing their current skills); selection (setting goals for what they want to achieve); optimisation (using their existing skills and the new resources that PYD provides to achieve these goals); and compensation (reviewing and if necessary redirecting actions to meet goals). 117 PYD interventions aim to reward young people when they make progress with activities regarded as pro-social within that programme. Drawing on but filling in gaps in the PYD literature, we suggest that because PYD is thus enabled in a range of diverse but overlapping and mutually reinforcing intervention-related activities, young people develop a more generalised ability to engage in intentional self-regulation focused on pro-social goals.

As a result of developing intentional self-regulation, young people are better able to accrue a range of ‘positive assets’ such as the ‘5 Cs’: competence; confidence; connection; character; and caring. 124 As these accrue, young people can make better use of the opportunities available in their wider environment, which leads to positive ‘developmental regulations124 (i.e. a positive feedback cycle, whereby the individuals gain more benefit from opportunities in their environment). This enables them in turn to make a positive contribution to their communities and societies, or, as a few authors would argue, enables them to contribute by critiquing and challenging inequities present in the existing social order.

These positive assets may then reduce risk behaviours via ‘buffering’,118 whereby risk factors in a young person’s environment have less impact, or via ‘compensation’,117 whereby even if young people engage in a risk behaviour, their possession of positive assets ameliorates the impact of this on their overall development. (This use of the term ‘compensation’ is distinct from that used within the description of intentional self-regulation above.) Furthermore, positive assets may reduce risk via molecular mechanisms (whereby a specific asset exerts specific protection against a specific risk) and/or via pile-up (whereby an accumulation of multiple assets is protective regardless of the specific assets involved). However, the theoretical literature synthesised here offers only limited insights beyond these general ideas. Engagement with pro-social peer groups or institutions might lead to reductions in antisocial behaviours via greater exposure and adherence to pro-social norms. Improved emotional self-regulation, social skills and self-efficacy might contribute to better decision-making to avoid violence and substance use. However, this falls short of a systematic theory of how the accrual of assets in particular or in general contributes to reductions in substance use or violence.

Sampling

There was a general lack of studies reporting information on sampling methods. Three studies were judged to have taken steps to ensure sampling rigour. 77,84,112 These studies attempted to sample a diverse range of participants. However, most studies (n = 7) failed to provide sufficient detail about their recruitment strategies. 85–89,94,99

Data collection

Four studies were judged as rigorous in the collection of data. 77,88,89,112 These studies described how they approached data collection, with aims including putting participants ‘at ease’, providing them with opportunities to ask questions77 or employing youth as interviewers. 88 In some cases, studies attempted to increase the range of data collected by using more than one method of collection, for example, by conducting in-depth interviews as well as drawing on field notes. The remaining six studies did not report these methods clearly. 84–87,94,99

Data analysis

Half the studies (n = 5) were judged as making attempts to increase analytical rigour. 77,86,88,89,112 The remaining studies failed to provide a description of the analytical approach undertaken. 84,85,87,94,99 Detailed descriptions of methods of analysis were generally missing. Most studies were ‘thin’ in their interpretive scope. Rather than developing clear, second-order interpretations, studies often provided descriptive accounts of how contextual and participant characteristics could function as barriers to or facilitators of implementation or receipt of PYD programmes.

Grounded in/supported by data

In many cases, we found a lack of first-order constructs in the form of verbatim quotes, relying mostly on authors’ observations and descriptions to support accounts of participants’ experiences, views or beliefs. Seven studies77,84,86,88,89,99,112 were judged as being grounded and supported by the data. However, the three remaining studies did not include enough data in the form of participant quotes or author description to show how the authors arrived at their findings. 85,87,94

Breadth and depth

The majority of studies were judged as providing both good breadth and depth in their findings. 77,84,86,88,89,99,112 The remaining three studies were limited in the reporting of their findings, both overall and in relation to answering the RQ. 85,87,94 During the coding process it also became apparent that, although there were some data on contextual factors, these were often quite ‘thin’. There were even fewer data on how participant characteristics affected PYD implementation and receipt.

Perspectives

The last criterion assessed the extent to which young people’s own perspectives had been considered in the process evaluation. Half the studies77,86–88,112 were judged as giving appropriate weight to young people’s experiences of engaging in programmes alongside other informants (e.g. programme providers). Strikingly, for studies of an intervention that aims to place young people’s empowerment at the heart of their programme model, very few studies engaged with young people as co-researchers. Reflective accounts of the power dynamics that might emerge during the research process were also scarce.

Cultural sensitivity, collaboration and integration with ethnic minority communities

The importance of cultural sensitivity and collaboration when implementing programmes in ethnic minority communities was a subtheme across three studies,84,87,94 all of which were judged to be of low reliability but varying degrees of usefulness. These reported that formal and informal community engagement was a key factor in ensuring that programmes were culturally sensitive, accessible and appealing to young people and their parents. This was particularly important when programmes were targeting, or situated within, marginalised ethnic minority populations. For example, Armstrong and Armstrong84 (a study judged to be of high relevance but low reliability) reported from interviews with site coordinators delivering ASPs in a south-western US state that a programme’s cultural relevance ‘within the community was critical to successful implementation’. The authors acknowledged that being located within an ‘ethnically diverse community’ meant that it was ‘important to have a strong cultural awareness’. They focused their efforts both on ‘outreach projects with parents’ and on schools in the local area, reporting on the importance of ‘liaison . . . with a trusted member of the community who could communicate with the parents, often times in Spanish’.

The study by Lee et al. ,94 judged to be of both low reliability and relevance, supported this finding. In their evaluation of a youth development programme targeting the Aboriginal communities of the Northern Territory of Australia, the authors highlighted the importance of seeking and incorporating the views of the wider community as well as of young people or parents. This, it was suggested, could provide support through the generation of ideas and allay fears among minority ethnic groups that the programme was: ‘a non-Indigenous solution so there is little ownership of it by Indigenous people’. 94 Over the course of the programme, consultation with the community was ‘enhanced . . . with more programmes developed from community ideas and delivered with Indigenous people at the forefront’. This included delivering activities that were youth-oriented but culturally relevant, such as ‘bush hunting excursions and using computers to record traditional music’. It was found that activities, such as watching organised sports events, could bring about community changes that led to reinstating ‘understanding and respect between generations’. This study found that as the programme progressed, staff became more active in encouraging community members to get involved; this was made possible by staff working flexibly and ‘engaging in regular formal meetings and informal discussion’ with members of the community.

Lee et al. ’s94 study also highlighted the importance of increasing both the cultural relevance and participation of the local community by addressing potential language barriers through ‘translating key proceedings’ and communicating with indigenous members ‘in their language’. The US study by Armstrong and Armstrong84 also found instances in which young people were allowed to access and participate in the programme only because ‘the parents were able to communicate with, and trusted the liaison’ officer connected with the programme.

In some cases, programmes actively recruited community members as staff. For example, Lee et al. 94 found that employing a ‘respected community member with a close collaborative working relationship with non-Indigenous staff’ was perceived by many participants as ‘pivotal to the initiative’s success’. Such actions could also be seen as providing the additional benefit of promoting local role models. For example, after identifying a ‘lack of Chicano Latino adult role models’ who could ‘encourage, empower and develop leadership skills and qualities’ of local Mexican American youth from the Minnesota area of the USA, programme providers in the low-quality study by Bloomberg et al. 87 trained local community members as ‘facilitators’ who could ‘work closely with youth in the initial 2 day training’, with the aim of establishing and maintaining a bond with them.

Challenges with community engagement and establishing trust

However, challenges in relying on volunteer community engagement and establishing trust of parents was a subtheme across two studies77,99 of differing quality. These were studies of interventions that did not specifically target diverse ethnic populations but that attempted to involve parents and local members of the community. Difficulties were reported in establishing good working relationships. For example, as reported by a study of high reliability and usefulness by Schwartz et al. ,77 ensuring the successful implementation of an intervention component implemented with the support of a volunteer mentor was challenging when they were not always reliable in maintaining contact, leaving participants feeling ‘disappointed’. One young person from this study talked about her mentor, stating that she hoped that ‘she would be there more than she was. . .and, and she wasn’t’. She believed that the mentor’s lack of presence was not only because ‘she was always workin’, but also because her mother and her mentor had worked together’, and ‘whenever my mom had quit that job or whatever, I guess she just went her separate way, and didn’t come around any more’. 77 In some cases, programmes attempted to turn to their ‘back-up’ mentors, but these were still a ‘small pool’ of volunteers who were often outside the mentees’ immediate community. There was also a reluctance to draw on them too heavily, as they could not necessarily deliver a programme with good fidelity in the absence of sufficient training (see Bloom et al. ,106 a study linked to that of Schwartz et al. 77).

Building trusting and openly communicative relationships with parents could also be challenging. Regarding the QOP, an initiative consisting of intensive case management, mentoring and positive youth activities, the evaluation by Maxfield et al. 99 judged as providing highly useful findings but having low methodological reliability, found that trust and open communication were seen as important means of maintaining contact with young people and encouraging uptake of intervention activities. The case managers in this programme reported that parents who appeared ‘anxious to limit the exposure of family problems’, who seemed to experience case managers as ‘intrusive’, or who may have ‘felt threatened’ by the mentoring relationships that case managers established with their children were subsequently the ‘most difficult to reach’ compared with parents who actively supported case managers to ‘locate’ young people and ‘reinforced the value of attending program activities’. 99

Collaborating with and utilising local community resources

Another subtheme concerned with collaborations with others in the local community was the importance of collaboration with other community agencies to enable programme implementation, again echoing the suggestions from Lerner and Lerner123 cited in our synthesis of theoretical literature. This was apparent in three studies84,86,99 of variable reliability and usefulness. In Armstrong and Armstrong’s84 evaluation of ASPs in a south-western US state, it was found to be ‘extremely important for the site to utilise community resources from a programmatic standpoint’. These ‘additional resources’, made possible through links with the community, enabled providers to expand the range of activities offered, a critical element of PYD. For example, local libraries proved to be an ‘unplanned benefit’ that could help deliver a reading programme. Programme providers cited being able to host ‘occasional large-scale events’ by ‘taking advantage’ of a nearby Boys and Girls Club. Local funding bodies were considered another important local resource to support positive youth activities. This was the case in the study of medium reliability and relevance by Berg et al. ,86 in which the programme received a grant that ‘enabled the visual research group [of young participants] to receive training in photography and show their work at a photography exhibit’.

This subtheme was also apparent in the study of the QOP by Maxfield et al.,99 who reported providers maximising the range of opportunities provided in the community by ‘forming partnerships with agencies that specialised [in a range of life skills training topics] such as substance abuse prevention, conflict resolution training, date rape, and sexual abuse’. Like Armstrong and Armstrong,84 they also collaborated with the local Boys and Girls Club, which enabled them to offer young people workshops on a range of topics, ‘such as anger management, self-esteem, planned parenthood, family planning and other life skills’. The importance of being able to make use of other local services to maximise breadth of opportunities was regarded as particularly important when there were gaps in programme providers’ expertise, such as when drawing on ‘student volunteers from the local university’99 to offer tutoring to support sites at which case managers felt that they lacked the skills to provide such services.

Collaboration with schools

A final subtheme regarding community collaboration was that collaboration with schools was critical to implementation but could be time-consuming and challenging. Three studies,84,99,112 two of which were based in the USA and one of which was undertaken in the UK, examined the importance of liaising with schools to support the successful implementation of programmes. All three studies were judged to be of high usefulness but were variable in terms of methodological reliability. Site co-ordinators in Armstrong and Armstrong’s84 study (judged to be of low reliability) of ASPs in south-western USA indicated that communicating with other community stakeholders to support the development of youth ‘such as schools, had an important impact on program implementation’. This was particularly relevant in their study because they had a number of ASPs located off schools’ sites. One way of dealing with barriers to communication arising from this was to designate a school liaison, who could work across programme sites, but who was an employee of a single school. The schools then also acted as a channel to disseminate information about programme events to young people and their families in order to reach a wider audience and increase programme reach.

A study by Wiggins et al. 112 of after-school youth development targeting at-risk young people across England and judged to be of high reliability and usefulness also found that ‘working with schools was crucial’ for recruiting young people to programmes. The authors noted that liaising with staff ‘with appropriate authority within schools was critical’ and that negotiating ‘access and referral routes’ was time-consuming. However, in a context of providers aiming to meet challenging recruitment targets, some sites reformatted their programme so that young people attended as an alternative rather than a supplement to their normal schooling, a major distortion of the intended intervention model. In an evaluation of the QOP, Maxfield et al. 99 also report that collaborations with schools could raise challenges. Although providers encouraged those ‘most in need of academic remediation’ to take up professional tutoring, these young people had to travel across the city to a different school because this is where tutoring was provided. Case managers transported young people to sessions to ensure their attendance but soon found that this ‘proved too burdensome’ for them and that ‘youth ceased to attend tutoring sessions with any regularity’.

Calm and authoritative programme providers

Within this theme, one key subtheme was the importance of programme providers attending to young people in a calm and nurturing yet authoritative way, including in response to any challenging behaviour exhibited by participants. Three studies of varying reliability and usefulness described provider attitudes and responses to young people in this context. The ‘Stand Up Help Out’ programme, which aimed to develop youths’ capacity for constructive relationships with adults and peers, was evaluated by Bulanda and McCrea88 and judged to be highly reliable and of medium usefulness. This reported that successful implementation was associated with staff signalling their continued commitment to providing ‘unconditional positive regard’ when faced with challenging behaviour from young participants. It was reported that this response style was acceptable to the young people, who did not feel that they were treated ‘negatively. One participant commented:

That’s what I like about [the instructor]. Cuz, even when he don’t get all the respect he should get out of the kids in the program, he still be [himself]. You know calm and collective [sic]. 88

This was in contrast to the All Stars prevention curriculum, an enhanced ASP offering a range of social and educational activities evaluated by Cross et al. 89 and judged to be a highly reliable and relevant study. This reported that staff struggled to respond to young people’s challenging behaviour. Drawing on data from different programme sites, the authors described sites as ‘rife with behaviour problems’, whereby young people received ‘very little redirection from staff members’ when they ‘acted out’, leading the authors to be concerned about participants’ physical safety. The authors also noted that when staff from this site did interact with young people, the disciplinarian approach taken ‘appeared capricious and confusing to youth’. 89 For example, when providers applied the use of ‘quiet time’, they did so to all young people rather than merely to those engaged in misconduct. In another site, the same evaluators found staff to be ‘irritated and apathetic’, appearing to engage more with each other than with young people and failing to address young people’s challenging behaviour.

The evaluation of the QOP by Maxfield et al. 99 found evidence supporting the need for case managers to engage with young people as individuals rather than as part of a group. Maxfield et al. 99 found that ‘the most successful mentors used a balance between nurturing and discipline’ when interacting with young people. In contrast, if staff took what was described as a ‘parental or authoritarian’ approach, young people often responded with resistance; if they adopted a merely ‘friendly approach’, case managers could be taken ‘advantage of’ or simply ‘dismissed’ by some young people. Case managers in this study were also said to identify a point at which young people needed less nurturing ‘in order to encourage them to become more self-reliant’.

Positive peer relations

A further subtheme was the importance of positive and supportive peer relations, underpinned by staff and by programme structure. Three studies, of differing reliability and usefulness, examined the contextual factors that appeared to contribute to young people being potentially able to overcome difficulties or tensions between them. For example, a high-quality study by Bulanda and McCrea88 described a US ASP providing young people with an apprenticeship in social work. They reported that social differences such as membership of different ‘street alliances’, which could be a cause of conflict outside the programme, did not necessarily prevent mutual collaboration and support within the programme, as long as participants were able to ‘prioritize their connectedness over the potential discord created by differences’. The authors suggested that this connectedness was achieved by youth being able to ‘recognize relationship problems and focus on relationship strengths’. However, reporting on another US-based ASP providing access to social and educational opportunities, Cross et al. 89 argued that tensions among participants or between participants and staff could be overcome only in sites that were well managed. A lack of organisation and high turnover of staff at one site within their study was a key factor in young people not seeming ‘to enjoy each other’s company’, whereas the positive outcomes observed in another site may have been attributable to ‘the friendships among students, which were in part facilitated by stable site management’.

A third study, also judged to be of high reliability and usefulness and conducted in the USA, reported that potential discord among young people owing to gang affiliation was not considered by providers to be a reason for excluding young people from participating. Instead, Bloom et al. 106 described how the NGYCP separated participants who belonged to different gangs into different ‘squads’ and removed or obscured gang symbols such as tattoos that could function as ‘physical reminders of past affiliations’. However, problems associated with gang membership were not always easily overcome, with one site in Mississippi reporting higher incidences of ‘fights and intimidation’ and ‘gambling and extortion’ and noting that there were ‘higher levels of attrition among this subset of the population’. However, staff members did report that external problems were less likely to intrude during the residential phase of this programme, where they ‘have them 24/7’ and can instil values that young people can then take ‘home with them’. 106

Staffing continuity essential to successful implementation

Within this theme, a key subtheme was the importance of staffing continuity to intervention delivery. Site co-ordinators in the study of ASPs in south-western USA by Armstrong and Armstrong,84 which was judged to be of low reliability, felt that effective implementation and sustainability of ASPs relied on ensuring that staff turnover was kept to a minimum. This was a challenge for some programmes. For example, Cross et al. 89 report on a high-quality evaluation of a social and educational activity-based after-school initiative, finding that ‘six of the original fourteen staff members quit or were fired before the end of the year’. Similarly, Lee et al. 94 reported that turnover of staff ‘impacted significantly on program continuity and workloads’. Maxfield et al. ,99 evaluating a community-based case-management and mentoring programme in the USA, noted that they were ‘fortunate [that] turnover [in certain sites] was relatively low’. However, two instances in which staff turnover was ‘most damaging’ and led to a failure in sustaining mentoring relationships was ‘when a position remained vacant’, resulting in participants not having a ‘primary mentor for as long as two or three months’, or when participants had ‘more than two case managers’ and rarely developed ‘strong relationships with their third case manager’.

Difficulty offering full-time posts in the field of youth work

It is clear across a number of studies that the limited number of hours involved in delivering youth programmes could sometimes mean that it was not possible to offer full-time positions, thereby increasing the difficulty in securing and retaining qualified staff. To overcome this challenge, Armstrong and Armstrong84 reported how one programme aimed to recruit ASP staff who were not looking for full-time work, such as college students interested in gaining experience of youth work or other professionals who were interested in working with young people, in addition to their primary employment. Difficulties with retaining trained employees could also mean that replacement staff were not well trained. Cross et al. 89 report that youth workers who had been retained since programme initiation ‘received more than 40 h[ours] of training on average’, compared with ‘less than 6 h[ours]’ for those who had replaced them, and that sites with high employee turnover were less likely to have staff who were highly trained. They found that in some cases, high staff turnover meant that it was not possible to employ staff who had participated in the original 3-day training that was offered before the start of the programme and that in some cases it was ‘necessary for untrained replacement staff to deliver the All Stars’ programme (see Gottfredson et al. ,91 a study linked to Cross et al. 89).

Creative attempts to compensate for lack of trained staff included drawing on existing skills that happened to be held by staff members and incorporating these opportunistically into programme activities. For example, Armstrong and Armstrong84 observed that at one site an employee ‘with extensive orienteering skills’ was encouraged ‘to organize camping trips and day hikes for youth’ and that at an another site, a staff member ‘who enjoyed jazz dancing started a dance program’.

However, two studies77,99 reported that it was difficult to overcome limitations in skills owing to a lack of training and that this could mean that programmes might not be able to provide the range of activities normally expected of a PYD programme. For example, Bloom et al. 106 (a report linked to the study by Schwartz et al. 77) found that provision of ‘a one-on-one paired tutoring model to be provided an hour and a half per week’ was impossible to implement because of a lack of tutor capacity and had to be ‘abandoned midway through the year’. The authors felt that despite providing an alternative academic activity, the lack of 1 : 1 tutoring may have ‘contributed to withdrawal of youths whose parents viewed tutoring as the main draw of the program’. 106 In addition, Maxfield et al. 99 reported that programmes found it difficult to secure staff with expertise across the range of PYD domains. For example, when programmes were expanded to include an educational component, their delivery could be a challenge for staff who had been ‘hired on the basis they could be case managers not tutors or teachers’. Such staff lacked the skills needed to provide educational activities, and ‘required extensive training and technical assistance’. This proved difficult for some sites, with providers reporting that when case managers were acting as tutors, ‘more expert tutoring would have greatly improved services’. Other sites that did not provide ‘extensive in-service training to improve case managers’ tutoring skills’ relied on volunteer tutors instead. This was also associated with retention problems because these volunteers tended to work for the programme only for ‘one or two semesters’, thereby causing difficulties with ‘forging long-lasting relationships’ with staff on this basis.

Young people determining their own engagement in activities

One subtheme within this relates simply to the extent to which young people were empowered to determine in which PYD activities to participate. This was described in three studies of variable reliability and usefulness. A study by Berg et al. ,86 judged to be of medium reliability and usefulness, described ‘youth empowerment’ as a key component in the YARP programme and suggested that staff needed to be trained in ‘facilitation techniques’ to halt the tendency for staff to determine how community engagement projects are undertaken. Young people’s decision-making processes were considered more important than their final choice of activity in the study by Baker et al. 85 of the South Baltimore Youth Centre. This programme, which aimed to prevent violence, delinquency and substance abuse, attempted to provide access to a range of educational and social activities. The evaluation, judged to be of low reliability and relevance, reported that when activities were ‘imposed [in a] top down [manner, they] failed and were abandoned’. Providers instead aimed to give young people authority so that all activities provided by the centre were ‘planned and carried out by the youth’ themselves. Schwartz et al. ’s77 study of youth-initiated mentoring, which was judged to be highly reliable and useful, found that when young people were able to choose their mentors without input from family members or programme staff, there was a greater likelihood that the mentoring relationship would ‘endure’.

Limitations to choice provided

However, two studies, judged to be highly reliable, reported that young people in some particular interventions had very limited empowerment to shape and determine their involvement in positive youth activities. For example, empowerment in the ‘All Stars’ curriculum, evaluated by Cross et al. ,89 was highly restricted. In this study, young people could choose activities but were restricted to a list predetermined by the site director and programme assistant at the start of each day. They also found that ‘despite intentions to provide a wide variety of activities to youth, it is clear that only a few activities were offered consistently during leisure activity time’ (see Gottfredson et al. ,91 a study linked to Cross et al. 89). Empowerment was also restricted in the programme evaluated by Schwartz et al. 77

Tensions arising from choice

Another subtheme in terms of youth empowerment was the tensions that could arise when empowering young people to choose which activities in which to engage. Four studies, judged to be of high relevance with variable reliability, provided data on young people’s choice of activities, showing that some programme components were often rejected by young people on the basis that they were unappealing. Sometimes these were activities with an educational component. These included ‘computer-assisted instruction’ and ‘community service’, which were not received with ‘enthusiasm’, as well as standardised tests, which participants showed ‘little patience for’, particularly those young people ‘who had difficulty reading’. 99 This was also the case for ‘computerised job training’ which was ‘ignored’85 and academic assistance. 89 Wiggins et al. 112 argue that taking an academic ‘school like approach’ could alienate young people, particularly those whose lives are ‘chaotic and hard’ and suggest that young people need to be able to get involved in activities at a level that is ‘most appropriate for them at any given time’. This might suggest the importance of a diversity of provision, not only to enable choice but also to take account of the fact that different young people will have different preferences and developmental needs, a point resonating with suggestions from Busseri et al. 117 cited in our synthesis of theories of change.

However, some process evaluations, as well as much of the theoretical literature, suggested that young people’s empowerment to choose activities was central to PYD provision. The study of the QOP by Maxfield et al. 99 suggested that facilitating choice may in some cases deter engagement in the broad range of activities which is commonly regarded as a central feature of PYD. This study, judged to be of low quality but high relevance, reported that some sites offered more recreational activities (e.g. outings to the cinema, ice-skating, swimming, etc.) because they attracted ‘more enrolees than did other activities’. However, as young people got older, they resisted staff’s promotion of ‘activities with learning content’ and continued to favour the more recreational activities that providers had originally used ‘to attract youth to the program’. Participants reported that they missed doing ‘fun things’ and that ‘museums and other cultural activities were boring’. Similarly, when there was a scheduling clash between attending summer school and taking up summer employment, case managers were more likely to recommend summer school but they were ‘not able to prevent an enrolee from choosing [paid work]’. This contrasted with programme sites that provided a balanced combination but offered participants little choice, which appeared to have ‘less difficulty in maintaining interest’ among young people. In the highly reliable and useful US-based study by Schwartz et al. 77 and the linked study by Bloom et al. ,106 evaluators found that a compulsory academic component was viewed much more positively. They report that participants of the National Youth Guard mentoring programme ‘welcomed the small class size, tailored instruction, and self-paced approach’106 of the graduate education programme. One young person was reported as stating that they liked the General Educational Development (GED) programme because:

[t]hey do it, one course at a time. That’s just awesome. And also the way they treat you is like people. I thought it was going to be like Juvie, you know, where they treat you like crap all the time. It’s not. They treat you like people. That’s what I like. 106

The authors noted that a key element of the success of their educational component was that it was noticeably different from what young people were used to experiencing in school, because it combined both structure and individual support. One participant felt that he was ‘less distracted by girls’ and could focus on his goal ‘to get the work done’.

The study by Maxfield et al. 99 reported that some sites provided financial incentives to increase engagement in specific educational activities, such as computer-assisted instruction and assessment tests. However, the two sites that took these approaches found that it was ‘effective for only short periods of time and only for students already inclined to spend time on the computer’ and did not prove effective in motivating already resistant young people. Later on, stipends came to be used only for time ‘spent engaged in education, developmental, and community service activities’. The use of incentives was also reportedly problematic in the US-based study by Cross et al. 89 (cited in the linked study by Gottfredson et al. 91). To increase engagement in programme activities, young people were randomly assigned to groups which would accrue points for attendance. However, programme staff thought the system unfair and decided to place high-attending youth together ‘to ensure the attending students would receive the maximum point’, thus undermining the intended system. This ‘probably did not encourage attendance among the lower attending youth because they were placed in groups with very low probabilities for receiving points’. In both of these programmes that used incentives, there was a tension in providers’ attitudes to empowerment. Although programme providers appeared to want to enable choice, they also sometimes wanted to constrain and incentivise choices to ensure that young people engaged in the programme overall or in specific activities that staff regarded as important.

Overview of included data

After data transformation and preparation (see Appendix 11), we included 54 effect sizes addressing substance-use outcomes from 12 reports of nine studies. We did not include effect sizes from the evaluation of Stay SMART,111 as these were not clear from study reports and we obtained no further information from study authors. We were also unable to include an effect estimate for alcohol use from the evaluation of the YARP intervention,86 despite multiple attempts to contact the study’s surviving authors. The evaluation of the PYDC79 measured the difference between intervention and control groups in a change model from baseline to follow-up of risk for substance use. When intervention effects are compared across studies as standardised mean differences, outcomes comparing change scores and those comparing ‘final values’ between intervention and control groups cannot be analysed in the same model, because the standard deviations are incommensurate. 67 We did not have sufficient information to convert these outcomes into a metric completely consistent with the other studies. We decided to include findings from Tebes et al. 79 in the main analyses but also to re-estimate all relevant multilevel meta-analysis models without these findings.

All substance-use outcomes

Findings from 12 reports of nine studies (54 effect sizes) informed the analyses of all substance-use outcomes. Included PYD interventions did not have a statistically significant effect on substance-use outcomes, either across all time points (d = 0.079, 95% CI –0.025 to 0.183) or in the short term (d = 0.086, 95% CI –0.025 to 0.197) (Table 6 and Figures 8 and 9). From a perspective of public health significance, pooled effect sizes would have been considered very small. Outcomes across studies were not characterised by a large degree of heterogeneity (programme-level I² = 27% in both analyses). Both the all-points and short-term meta-analyses were robust to sensitivity analyses.

FIGURE 8.

All substance-use outcomes. ES, effect size.

FIGURE 9.

Short-term substance-use outcomes. ES, effect size.

Omnibus substance-use outcomes

Findings from seven studies (11 effect sizes) informed the analyses of omnibus substance-use outcomes (i.e. outcomes where the measure of substance use reported encompassed both illicit drug use and other substances). Although there was no statistically significant effect across all time points (d = 0.127, 95% CI –0.035 to 0.290), the meta-analysis of short-term outcomes yielded a small but statistically significant effect (d = 0.169, 95% CI 0.012 to 0.326) (see Table 6 and Figures 10 and 11). Both findings were robust to all sensitivity analyses and demonstrated little heterogeneity at the programme level (I² = 0% for both analyses). However, it is worth pointing out that the difference between the analyses reflects the inclusion in the former but not the latter analysis of one effect size: the odds of being convicted of a drug offence measured at 18 months, reported in Millenky et al. 108 as part of the evaluation of the NGYCP. Thus, this statistically significant finding should be interpreted with caution.

FIGURE 10.

Omnibus substance-use outcomes. ES, effect size.

FIGURE 11.

Short-term omnibus substance-use outcomes. ES, effect size.

Illicit drug-use outcomes

Findings from six studies informed the analyses of illicit drug-use outcomes. PYD interventions did not have a statistically significant effect on illicit drug-use outcomes either across all time points (d = 0.047, 95% CI –0.117 to 0.212) or in the short term (d = 0.050, 95% CI –0.141 to 0.242) (see Table 6 and Figures 12 and 13). Both meta-analyses were robust to sensitivity analyses, although removal of findings from Tebes et al. 79 yielded negative pooled effect sizes in both meta-analyses. These meta-analyses included 21 and 14 effect sizes, respectively. Programme-level heterogeneity was small to moderate in the first analysis (I² = 34%) and nearly negligible in the short-term meta-analysis (I² = 8%).

FIGURE 12.

Illicit drug-use outcomes. ES, effect size.

FIGURE 13.

Short-term illicit drug-use outcomes. ES, effect size.

Alcohol-use outcomes

Meta-analyses included findings from six studies, with 10 of 18 effect sizes included in the short-term meta-analysis. PYD interventions had small and statistically non-significant effects reducing alcohol use both across all time points (d = 0.050, 95% CI –0.063 to 0.163) and in the short term (d = 0.070, 95% CI –0.084 to 0.224) (see Table 6 and Figures 14 and 15). Findings were robust across sensitivity analyses. Some programme-level heterogeneity was present in the meta-analysis with all time points (I² = 21%), but heterogeneity was large in the short-term outcomes meta-analysis (I² = 46%). The small number of interventions included in the short-term outcomes meta-analysis precluded further exploration of heterogeneity.

FIGURE 14.

Alcohol outcomes. ES, effect size.

FIGURE 15.

Short-term alcohol outcomes. ES, effect size.

Smoking outcomes

Analyses for smoking outcomes included only four effect sizes from three studies and, thus, are presented with caution. PYD interventions had a small, statistically non-significant effect reducing smoking outcomes (d = 0.053, 95% CI –0.038 to 0.143) with no programme-level heterogeneity (I² = 0%) (see Table 6 and Figure 16). We do not present meta-analyses for short-term smoking outcomes, as only two effect sizes would have been included.

FIGURE 16.

Smoking outcomes. ES, effect size.

Substance-use subgroup analyses by sex

The use of covariate-adjusted models in almost all studies precluded the use of metaregression to explore group-level moderators of intervention effect. This is because covariate-adjusted models present ‘conditional’ estimates of intervention effect that apply in magnitude only to the specific population defined by the regression models used in adjustment. Moreover, the quality and number of included studies and the lack of substantial study-level heterogeneity would have made metaregression specious. As a rule of thumb, metaregression generally requires at least 10 studies for each categorical moderator tested. 67 Our metaregression models would also have been confounded by study quality and risk of bias between levels of our covariate. Instead, we present narratively the findings of studies in which subgroup analyses were undertaken for sex. Evaluations of three interventions78,96,98,103–105,112 undertook subgroup analyses by sex. Findings were mixed and do not present a clear picture.

In the evaluation of BBBS,96,98 the decrease in the likelihood of initiation of drug abuse experienced by youth in the intervention group was felt more strongly in young men (55.0% decrease in risk; p-value < 0.05) than in young women, for whom the decrease in risk (26.6%) was not statistically significant. Findings were reversed for the initiation of alcohol use, with young women experiencing twice as much decrease in risk (38.8%) than young men (19.2%), although the differences were not statistically significant for either subgroup. Although the overall decrease in risk of smoking (19.7%) was not statistically significant, young men demonstrated a greater decrease in smoking risk (24.5%) than young women (9.9%). Tests of moderation did not reveal any significant differences between subgroups in any of these analyses.

Short-term impacts reported in Schirm et al. 103 for the QOP demonstrated that, although the overall risk difference in probabilities for any past-month binge drinking between intervention and control group was a non-significant 4% (i.e. 24% in the intervention group vs. 20% in the control group, as reported above), this was driven by a marginally significant (p-value < 0.10) risk difference of 7% in males favouring the control group, compared with a risk difference of 0% in women. Risk differences for any past-month illegal drug use were the same in men and women. Tests of moderation were not significant at the p-value < 0.10 level for any of these analyses.

Medium-term impacts104 for the QOP revealed a different picture. The risk difference of 6% in past-month binge drinking, which favoured the intervention group, was driven by a risk difference of 12% in young men favouring the intervention group (p-value < 0.05), compared with a non-significant risk difference of 2% in women favouring the intervention group. The risk difference for frequent past-month binge drinking was 0% in men, but 4% favouring the control group (p-value < 0.05) in women. Finally, the 6% risk difference favouring the intervention group in past-month illicit drug use was decomposed into a marginally significant (p-value < 0.10) risk difference of 8% in men and a non-significant risk difference of 3% in women, both favouring the intervention group. Tests of moderation were not significant at the p-value < 0.10 level for any of these analyses.

The distribution of risk differences in the late-term evaluation105 of the QOP showed that risk differences for past-month tobacco use, past-month daily tobacco use and any past-month binge drinking favoured the intervention in men but favoured the control in women, although none of the effect sizes in these subgroup analyses rose to significance. The overall risk difference for past-month frequent binge drinking favoured the control group, as did the risk differences for men and women separately. As in the prior evaluations, tests of moderation were not significant at the p-value < 0.10 level for any of these analyses.

Finally for YPDP, Wiggins et al. 112 reported stratified analyses for frequency of cannabis consumption more than once a week at the 9-month follow-up. Young men (OR 2.50, 95% CI 0.86 to 7.28) and young women (OR 2.27, 95% CI 0.71 to 7.33) were not substantially different on this outcome. Moderation was not tested in these analyses.

We further intended to examine the moderation of effects by economic status, ethnicity or area deprivation, although no intervention evaluations presented analyses in these subgroups. For BBBS, Tierney98 presented subgroup analyses by combinations of sex and ethnicity. None of the moderation analyses for substance-use outcomes appeared to demonstrate that minority ethnic status was an effect modifier.

Overview of included data

We included 10 effect sizes addressing violence outcomes from four reports of three studies.

Effects on violence

Positive Youth Development interventions did not have a statistically significant effect on violence outcomes across all time points (d = 0.021, 95% CI –0.050 to 0.093) (Table 8 and Figure 17). There was no meaningful programme-level heterogeneity in this finding (I² = 0%), although this model drew on only 3 interventions and 10 effect sizes. Short-term outcomes did yield a statistically significant effect (d = 0.076, 95% CI 0.013 to 0.139), although this finding was marginally significant (p-value < 0.10) in sensitivity analysis and should thus be regarded with caution (Figure 18). Again, there was little meaningful programme-level heterogeneity (I² = 0%).

FIGURE 17.

Violence outcomes. ES, effect size.

FIGURE 18.

Short-term violence outcomes. ES, effect size.

Subgroup effects on violence

We aimed to examine how intervention effects were moderated by participants’ sex, SES and ethnicity or by area deprivation in order to assess the potential impacts of PYD interventions on health inequalities. We did not undertake these analyses for reasons similar to those presented above in our description of subgroup effects on substance use. Regardless, the inclusion of only three studies would have made any such test grossly underpowered. Instead, we present narratively the findings of BBBS, the evaluation of which98 was the only one to include subgroup analyses for violence outcomes by sex.

As described above, intervention youth hit someone else 0.85 times less frequently than did control group youth. This effect was larger in young women (1.17 times less; p-value < 0.10) than in young men (0.67 times less; p-value < 0.10), although subgroups were not significantly different from each other. In comparisons between intervention and control groups on the number of times youth were involved in a fight, young men were not different from young women (0.03 times less vs. 0.01 times less). Tests of moderation did not appear to be statistically significant.

No outcome evaluations presented analyses in terms of participants’ SES, ethnicity or area deprivation. The evaluation of BBBS98 presented subgroup analyses by categories defined by sex and ethnicity. Tests of moderation did not appear to be statistically significant.

Young people’s feedback

Young people agreed that PYD programmes can have an effect on their drug use but not on their alcohol consumption unless programmes provide an alternative to drinking in terms of social venue. They were in both agreement and disagreement that PYD programmes can influence smoking or have a beneficial influence on violent behaviour.

Assessing the quality of theory reports

It is rare for systematic reviews of theory to assess their quality; this is exemplified by those reviews reported by Campbell et al. ,146 Harris et al. 147 and Kreiger. 148 Our quality criteria were informed by previous work by Michie et al. 64 and Bonell et al. ,131 and were accompanied by guidance which was applied by two researchers who then met to discuss their scores. However, as shown in Appendix 13, reviewers found it difficult to make decisions in applying the criteria and there was very little inter-reviewer agreement on the scores.

There were a number of challenges. Few studies presented a single theory of change; many instead drew on a range of existing theories. This meant that reviewers were uncertain about whether or not they should assess each contributing theory (which would be difficult without references to primary texts) or to give an overall score for each report (which would be limited in terms of transparency). Reviewers also found it difficult to assess criteria of testability and parsimony because, although guidance was provided, this did not provide a clear benchmark against which to make such judgements, and reviewers felt that their judgements often shifted in the course of reviewing the literature. Application of the criterion of generalisability was, to some extent, easier to assess and more likely to result in agreement between reviewers simply because this was a more absolute judgement; most reports had been included because they offered an abstracted and therefore potentially generalisable framework for understanding PYD rather than, for example, reporting local data. However, we were unsure of the extent to which the theories would actually be relevant to varying structural or cultural contexts in practice. Finally, our criteria were intended to assess the quality of causal theories of change and thus were not useful in assessing the quality of the normative elements of theories.

Limitations in included theory reports

The theoretical literature did not in general focus on descriptions of causal theory of change for how PYD interventions might reduce substance use or violence among young people. Much of the literature instead aimed to assert the normative value of PYD as an approach to youth provision. This normative theory was, however, useful in understanding the goals and assumptions of PYD programming. Causal theorising was a minor and generally unsystematic element of most theory reports, with a few exceptions.

Although our synthesis of theories of change was hampered by a lack of clarity within included reports about how PYD might optimise young people’s capacity for ‘intentional self-regulation’ and by a lack of systematic consideration of how promoting positive assets might lead to reductions in risk behaviours, we nonetheless developed a synthesis of causal theory which described the mechanisms by which PYD interventions might reduce violence and substance use. In the case of theorising how PYD interventions affect intentional self-regulation, it went beyond synthesis to fill in some gaps. The synthesis was successful despite the lack of success of our quality assessment of the theoretical literature. We included reports in our synthesis regardless of their quality. Our synthesis involved bringing together theoretical fragments (which specified only certain parts of the pathway from PYD intervention to substance use or violence reduction) and would have been less comprehensive had we synthesised only theories of change that themselves set out a comprehensive and clear path from intervention to risk reduction. Because of this, quality criteria focused on, for example, parsimony would not have proven very useful even if they had been easier to apply.

However, the resulting synthesis was quite ‘thin’, particularly in terms of how positive assets might enable reductions in risk behaviours. As well as not constituting a comprehensive theory of how assets reduce risk, the suggested pathways offer little that is distinctive to that provided in more traditional psychological theories used in prevention science such as the social learning model128 and the social control theory,135 both of which PYD theorists cite.

Limitations in process and outcome evaluations

Process evaluations overall were generally of low or medium quality. Sampling and analysis methods were poorly reported. Analyses were generally descriptive and did not develop clear, second-order interpretations. Few quotes were used to substantiate the analysis. Nonetheless, we were able to develop a synthesis that provided some useful answers to our RQ concerning the characteristics of contexts and participants that could influence the implementation and receipt of PYD interventions. Only one process evaluation from the UK was included, but this did include similar themes to those reporting from the USA and Australia.

The lack of studies from the UK was also apparent in the case of outcome evaluations, as was the lack of studies of cost-effectiveness. There were also various methodological problems with outcome evaluations. Authors rarely presented data in a format that was readily analysable, which meant that our analyses required both extensive transformation of effect sizes and sensitivity analyses. We decided to exclude one study both because of the quality of the evaluation and because of the uninterpretable effect sizes it reported. Although our rationale was transparent, it is possible that another meta-analyst may have taken a slightly different approach or made somewhat different transformation decisions. Moreover, we performed a sensitivity analysis in which we excluded one study79 that did not report findings in the same standardised metric as other studies (i.e. as change from baseline rather than adjusted by baseline). The force of the conclusions did not change as a result of excluding this study.

The variable quality of evidence (including inadequate adjustment for clustering in several included studies) and the need for extensive data transformation and for sensitivity analysis for key statistical decisions must qualify the interpretation of our statistical results. Although, on balance, it was an appropriate decision to meta-analyse the included studies, challenges we faced with the data may suggest an interpretation of the pooled effect size that relies more on its general magnitude and precision than on statistical significance per se. We were also unable to test the key hypotheses derived from our consultations owing to the nature of the included evidence.

Moreover, although multilevel meta-analysis is perhaps a more robust method than those using one effect size per programme, it was not as robust as the multivariate meta-analysis originally proposed in the protocol. This is because multivariate meta-analysis uses the known variance–covariance matrix between included outcomes to account for dependencies between outcomes within interventions or studies, whereas multilevel meta-analysis achieves this by partitioning the variance between outcomes into that attributable to variation within interventions and variation across interventions.

Finally, there were problems associated with including evaluations of two interventions (PYDC79 and Stay SMART111) in the meta-analysis, for different reasons discussed above. Although we ultimately sensitivity-analysed findings with Tebes et al. ,79 our findings still may not present the most complete picture of effects on substance use.

Finally, we did not perform funnel plots because these would not have been a good guide to publication bias given the small number of studies included. 149 Our very comprehensive search methods, although not precluding the possibility of publication bias, mean that we took all reasonable steps to prevent this arising from an insensitive search.

Theories of change for Positive Youth Development effects on substance use and violence

Sixteen reports were included. We aimed to assess the quality of these theories by drawing on criteria used previously but we found that these were challenging to apply consistently to the PYD theoretical literature.

The included literature did not provide sufficient information to develop a comprehensive theory of change for the effects of PYD interventions on substance use and violence. Nonetheless, by filling in some of the gaps in the literature (in a transparent manner), we succeeded in generating a theory of change.

Positive Youth Development interventions are intended to provide a positive environment for youth in terms of positive expectations, enduring and affective relationships with adults, diverse activities and settings and active participation, whereby young people are empowered to choose activities and to take on responsibilities. Interventions vary according to whether or not they aim to enable young people to contribute to or challenge the existing social order, focus on individual or collective development, aim to transform individuals only or also the environments in which they live, provide breadth, depth and duration of activities, and address only positive assets or also risk behaviours.

In providing positive expectations and active participation in diverse activities, PYD aims to offer ‘affordances’, that is, resources individuals use in the course of their development (e.g. relationships, challenges, education; see Busseri et al. 117). We interpret that young people can make use of these affordances in learning to apply ‘intentional self-regulation’ to specific intervention activities such as sports, arts or outdoor activities. Intentional self-regulation comprises ‘intentionality’ (assessing their current skills); ‘selection’ (setting goals for what they want to achieve); ‘optimisation’ (using their existing skills and the new affordances that PYD provides to achieve these goals); and ‘compensation’ (reviewing and if necessary redirecting actions to meet goals (see Busseri et al. 117). PYD interventions aim to reward young people when they make progress with activities regarded as pro-social within that programme. Drawing on, but filing in gaps in, the PYD literature, we interpret that PYD interventions enable young people to engage in and learn from diverse, mutually reinforcing intervention activities, so that they develop a generalised intentional self-regulation focused on pro-social goals.

As a result of developing intentional self-regulation, young people are better able to develop various ‘positive assets’ such as the ‘5 Cs’: competence, confidence, connection, character and caring (Lerner et al. 124). As these accrue, young people can make better use of the opportunities available in their wider environments, which leads to positive ‘developmental regulations’: a positive feedback cycle whereby the individuals gain more benefit from opportunities in their environment (Lerner et al. 124). This enables them in turn to make a positive contribution to their communities and societies, or, as a few authors would argue, enables them to contribute by critiquing and challenging inequities present in the existing social order.

These positive assets may then reduce risk behaviours via ‘buffering’ (see Catalano et al. ,118 who state that risk factors in a young person’s environment have less impact on those with positive assets) or ‘compensation’ (see Busseri et al. ,117 who state that even if a young person engages in a risk behaviour, his or her possession of positive assets ameliorates the impacts of this on their overall health and development. (Note that this use of compensation is quite distinct from that within the description of intentional self-regulation above.) It is also argued that positive assets may reduce risk via ‘molecular’ mechanisms, whereby a specific asset exerts specific protection against a specific risk, or via ‘pile-up’, whereby accumulation of multiple assets is protective regardless of the specific assets involved. However, the theoretical literature synthesised here offers only limited insights beyond these general ideas. It suggests that engagement with pro-social peer groups or institutions might reduce antisocial behaviours via exposure and adherence to pro-social norms. It also suggests that improved emotional self-regulation, social skills and self-efficacy might contribute to better decision-making to avoid violence and substance use. But these theories fall short of a systematic theory of how the accrual of assets in particular or in general contributes to reductions in substance use or violence.

Characteristics of participants and contexts that affect implementation and receipt of PYD

Of the 10 included studies, eight were conducted in the USA, one in Australia and one in England. Study quality ranged from high reliability and usefulness77,89,112 to low reliability and usefulness. 85,87,94

A number of themes emerged from the synthesis. Community engagement was key to ensuring that programmes were culturally sensitive, accessible and appealing to young people and their parents and the wider community. Employing community members could be pivotal to successful implementation and the provision of role models. However, volunteers could be unreliable, for example in acting as mentors. Collaboration with other community agencies could be important, particularly in expanding the range of activities offered, but could also lead to a move away from original approaches.

Another theme was young people’s relationships with providers and peers. Providers should relate to young people in a calm and nurturing yet authoritative way. Skilled providers could bridge social differences between participants, but this could be undermined by poor training or retention. Retention was challenging when programmes could not offer full-time positions. A final theme concerns challenges to ensuring that young people are empowered to make decisions about programme activities while also requiring them to engage in diverse activities, including vocational or academic activities. Although studies were drawn from the USA, the themes identified were found in US, UK and Australian reports and are likely to be pertinent across different geographical settings.

Effectiveness and cost-effectiveness in reducing substance use and violence

We found 13 study reports of 10 distinct outcome evaluations and included 12 study reports of nine distinct outcome evaluations in our meta-analyses. All but one study was conducted in the USA; one study was conducted in the UK. We could not categorise interventions as per the taxonomy derived from our theory synthesis because of the lack of detail reported. We found no economic evaluations and thus cannot assess cost-effectiveness. Four studies were RCTs, five were non-randomised trials with prospectively matched control groups and one included both randomised trial and non-randomised components which were analysed together. Overall, quality of evidence for our analyses of substance use and violence outcomes was rated ‘very low’. Only one study described a theory of change for how its PYD-type intervention components might lead to reductions in substance use or violence.

Comparing PYD with usual or no treatment, our meta-analyses found a small effect for substance-use outcomes overall, which was significant neither statistically nor in terms of public health relevance. Meta-analyses found no effect for an omnibus measure of substance use across all time points but did find a small, statistically significant effect for this outcome in the short term (0–4 months post intervention). Meta-analyses of illicit drug-use and alcohol outcomes found no significant effects either across all time points or in the short term. Our meta-analysis for smoking at all time points included only four studies and so its finding of no significant effects should be treated with caution. A meta-analysis of short-term smoking was not undertaken, because only two studies could have been included.

Concerning the overall effects of PYD interventions on substance use and violence, two interpretations are possible. The first is that studies pointed to a small effect for substance-use outcomes that was significant neither in terms of statistics nor public health (i.e. that there is evidence of no effect). Another interpretation is that there is no evidence of the effectiveness of PYD interventions on substance use and violence outcomes. This interpretation is more conservative and better reflects the variable quality of included studies, the fact that some interventions, although meeting our inclusion criteria, may not have reflected a purist PYD approach as identified in our synthesis of theories of change, and the paucity of outcome evaluations included in the case of smoking and violence.

Our meta-analysis for smoking at all time points included only four studies and so its finding of no significant effects should be treated with caution. Meta-analysis of short-term smoking was not undertaken, because only two studies could have been included. Meta-analyses of illicit drug-use and alcohol outcomes found no significant effects either across all time points or in the short term. However, although our meta-analyses found no effect for an omnibus measure of substance use across all time points, we did find a small, statistically significant effect for this outcome in the short term (i.e. 0–4 months post intervention).

The data in Appendix 15 are intended to indicate the degree to which the programmes evaluated in included studies actually embodied PYD principles but are limited in their usefulness because of limitations in study reporting. Nonetheless, it is clear that some of the studies included appear to be interventions that, despite meeting our inclusion criteria, may not be exemplars of the PYD approach as suggested by our synthesis of PYD theory of change. Some involved an array of sites with intervention activities varying between them, for example, the MAP,93 the YPDP,46,112 the QOP78 and BBBS. 96,98 Furthermore, a number of programmes appeared to focus on explicit risk reduction as much as, if not more than, on positive development. This was particularly the case with the ‘All Stars’ intervention90–92 and to a lesser extent the PYDC,79 Cool Girls, Inc. 95 and Stay SMART. 111 The NGYCP,77 although focused on positive development rather than explicit risk prevention, could also be regarded as atypical of PYD interventions in adopting a quasi-military ‘boot-camp’ style of delivery.

It is of use to consider the two key deviant cases with positive effects that we included in our meta-analyses. Both the evaluation of the PYDC79 and the evaluation of BBBS96,98 reported significant effects on reducing substance use, in the case of the first evaluation, at 4-month follow-up but not at post-intervention follow-up. The authors of the first evaluation conclude that these differences are attributable to the effectiveness of the programme, but this conclusion is attenuated by the non-randomised evaluation design. Moreover, although attrition was roughly balanced by arm, attrition was approximately 40% overall at the second follow-up. This is a serious limitation that was not addressed using methods for missing data. It is likely that the positive effects at second follow-up, especially when compared with the null effects from post-intervention follow-up, are optimistic. A limited description of the intervention as implemented makes it difficult to connect the apparent successes of this intervention with the programme model, although the authors do note that the intervention combined the promotion of positive assets with ‘traditional’ risk-based prevention education.

In contrast, the evaluation of BBBS96 was randomised, although their use of complete case data analysis is an important caveat. This design, together with extensive programme description, provides some confidence about the significant intervention effects that the authors claim to demonstrate. The authors describe that what makes BBBS98 different from other mentoring-based interventions is the focus on long-term relationships between a young person, often from a disadvantaged social or economic background, and an adult who may have ‘aspirational’ characteristics (e.g. higher education). The focus on long-term, carefully selected matches also set this mentoring intervention apart from the other interventions included in this review. But, above all, those randomised were drawn from a pool of families who had approached BBBS98 for a match – that is to say, programme participants already came from families who were at least somewhat interested in participation. This is in contrast, for example, to the QOP78 or All Stars,90 where participation was ‘opt-out’ or otherwise brought to students and their families in the schools that young people were already attending. Together, all of these factors may have accounted for the success of BBBS. 98

Caution is also needed in determining the extent to which our results are transferable to settings outside the USA given that most studies were conducted there. The one UK study also reported null results but was non-random in design. 46,112

In terms of subgroup effects, it was not possible to undertake metaregressions looking at effects by sex. Narrative synthesis of three studies that examined subgroup effects found that these were mixed, with no clear pattern. We aimed to examine effects by SES, ethnicity or area deprivation but no studies reported these. One evaluation98 presented a subgroup analyses by combinations of sex and ethnicity, reporting that minority ethnic status was not a moderator of effects.

Meta-analyses suggested that PYD interventions did not have a statistically significant effect on violence outcomes across all time points but there was a beneficial effect on short-term outcomes. However, this finding was only marginally significant in sensitivity analysis and should be treated with caution.

Metaregression to look at subgroup effects was not possible. Only one evaluation98 reported effects by sex and found that this did not moderate effects. It also examined presented subgroup analyses by categories defined by sex and ethnicity, reporting that these did not moderate effects.

Characteristics of participants and contexts that determine effectiveness

We aimed to examine what characteristics of participants and contexts appear to moderate, or are necessary and sufficient for, PYD effectiveness. A synthesis of PYD theories of change and process evaluations suggested several hypotheses:

• Interventions that offer a breadth of activities may be more effective for younger adolescents, whereas those that emphasise depth may be more effective for older adolescents.

• Interventions that combine prevention and positive development may be less effective than those that focus only on positive development.

• Interventions of more than 1 year’s duration may be more effective than those of shorter duration.

• Interventions may be more effective for participants with low or moderate levels of baseline risk, because there is more scope for stimulating ‘intentional self-regulation.’124

• Interventions that have specific methods to engage communities will be more effective.

• Projects that engage with schools will achieve better recruitment.

• Interventions that are delivered by well-trained staff will be more effective.

• Interventions that have better staff retention will be more effective.

• Interventions that offer some choices but require some engagement with educational components will be more effective.

However, the limited number of studies and very low level of statistical heterogeneity or qualitative differences in the effects reported by these studies precluded our innovative use of metaregression or qualitative comparative analyses to test these hypotheses.

Background

There are increasing calls for adolescent health interventions to address multiple- rather than single-risk behaviours because such behaviours cluster together31,32 and because such interventions are potentially more feasible and efficient. 33 PYD is one such intervention to address interclustered risk behaviours among young people. PYD is the dominant paradigm in youth work in the UK. The NYA, the major youth work organisation in the UK, defines such interventions as voluntary and informal educational activities that aim to bring about generalised youth development rather than merely to remedy ‘problem behaviours’. Similarly, in the USA, PYD is defined in terms of its goal of developing bonding; resilience; social, emotional, cognitive, behavioural or moral competence; self-determination; spirituality; self-efficacy; clear and positive identity; belief in the future; recognition for positive behaviour; opportunities for pro-social involvement; and/or pro-social norms;30,44 academic, cognitive or vocational skills; confidence; connections to peers and adults; character in terms of self-control, respect and morality; and caring for others. 36

The UK government’s Positive for Youth38 report announced a multimillion pound investment in youth work, youth centres, the National Citizen Service and other youth volunteering projects. The most recent public health White Paper39 cited such work as a key element in promoting young people’s health. The Mayor of London and local government across the UK are also investing millions of pounds in various PYD interventions. 40 The devolved governments in Scotland and Wales also emphasise these principles and promote investment in PYD. 41,42

However, despite this widespread investment and potential, the evidence base for the public health benefits of such interventions is unclear. Although a systematic review examining non-health outcomes43 reported benefits for self-confidence and self-esteem, school bonding, positive social behaviours, school grades and achievement test scores, the review did not systematically examine health effects. Systematic reviews of health outcomes have so far focused only on sexual health,44,45 reporting sustained effects but demonstrating considerable unexplained variability between programmes.

Aims

To search systematically for, appraise the quality of and synthesise evidence to address the following RQs:

• RQ1: what theories of change inform PYD interventions delivered to young people aged 11–18 years addressing substance use and violence?

• RQ2: what characteristics of participants and contexts are identified as barriers and facilitators of implementation and receipt in process evaluations of PYD?

• RQ3: what is the effectiveness and cost-effectiveness of PYD compared with usual or no treatment in reducing substance use (smoking, alcohol, drugs) and violence (perpetration and victimisation)?

• RQ4: what characteristics of participants and contexts appear to moderate, or are necessary and sufficient for, PYD effectiveness?

Overview of methods

Results

Theory of PYD impacts on substance use and violence:

1. How interventions enable PYD

   1. PYD interventions provide a positive environment

      1. –positive expectations: all young people have the potential to develop healthily;

      2. –affective relationships with adults: long-term, personal and caring compared with short-term, impersonal and instrumental relationships, with a family-like programme atmosphere;

      3. –diverse activities and settings: offer diverse activities, settings and relationships; and

      4. –active participation: youth empowered to choose activities, take on responsibilities.

   2. Taxonomy of PYD

      Interventions vary according to:

      1. –the breadth, depth and duration of activities offered;

      2. –whether they aim to transform individual young people only or also the neighbourhood environments in which they live;

      3. –whether they address only positive development or also prevention of risk behaviours; and

      4. –whether they aim to enable young people to contribute to or challenge the existing social order.

   3. Interventions offer ‘affordances’ and opportunities for social learning

      The positive environment interventions provide offer:

      1. –‘affordances’: resources that individuals make use of in the course of their development (e.g. relationships, challenges, education) – different individuals at different points in their maturation will make use of different affordances (e.g. younger adolescents require breadth whereas older adolescents require depth);

      2. –opportunities for social learning: young people learn skills and pro-social conventional behaviours through (1) observing and imitating positive adult role models; and (2) the reinforcement of positive behaviour through material rewards or non-material celebrations.

   4. Young people improve their ‘intentional self-regulation’

      These opportunities enable young people to improve their ‘intentional self-regulation’ a cognitive and behavioural process which involves:

      1. –intentionality (reflecting on and learning from existing behaviour to inform intentions for future behaviour);

      2. –selection (from a range of opportunities) of those activities that provide the young person with psychological and social assets to meet their ultimate personal goals;

      3. –optimisation (acquiring, refining, coordinating and applying goal-relevant resources towards achieving the selected targets); and

      4. –compensation (redirecting behaviour if initial efforts are thwarted).

   5. Young people accrue ‘positive assets’

      Improve ‘intentional self-regulation’ enables young people to accrue positive assets, for example the ‘5 Cs’:

      1. –competence (includes self-efficacy, resilience, or social, cognitive, behavioural and moral competence);

      2. –confidence (includes self-determination and a clear and positive identity);

      3. –connection (positive, social including bonding);

      4. –character; and

      5. –caring (or compassion, character includes spirituality and a belief in the future).

   6. A virtuous cycle of ‘developmental regulation’

      As young people’s positive assets accrue, they can make better use of the opportunities available in their environment, which leads to:

      1. –‘developmental regulation’, that is, a positive feedback cycle in which individuals gain more benefit from opportunities in their environment;

      2. –this enables them in turn to make a positive contribution to their communities and societies or, as a few authors would argue, enables them to contribute by critiquing and challenging inequities present in the existing social order; and

      3. –these commitments and contributions signal the development of a stable and positive adolescent identity.

2. How PYD reduces risk of substance use and violence

   1. Relationship between positive development and risk reduction

      Different authors variously present this as complementary albeit with comparatively little theorisation of how risks are reduced. Many argue that positive assets and protective factors are often the same thing, such as communication skills and self-efficacy.

   2. Various functions

      1. –Buffering function: risk factors in a young person’s environment have less impact on those who have positive assets; and

      2. –Compensation: even if young person engages in a risk behaviour, their possession of positive assets ameliorates its impact either on their health/well-being or on their engagement in other risk behaviours (this use of ‘compensation’ is quite distinct from that within the description of intentional self-regulation in Theory of PYD impacts on substance use and violence, iv. Young people improve their ‘intentional self-regulation’) – but this function is regarded as less important than buffering.

   3. Various mechanisms

      1. –Molecular mechanism: a specific asset exerts specific protection against a specific risk; and

      2. –Pile-up mechanism: the accumulation of multiple assets is protective regardless of the specific assets involved. Pile-up can involve (1) vertical pile-up – the greater number of different assets at a point in time or over time, the less risk; and (2) horizontal pile-up – the more settings that provide the same assets, the less risk.

3. Assessment of theory of PYD effects on substance use and violence

   1. Limitations of PYD theory

      Obvious limitations are that:

      1. – the literature focuses more on asserting the normative value of PYD assets than on specifying causal mechanisms;

      2. – although notions of intentional self-regulation and developmental regulation offer a general mechanism for positive development, they offer little insight into how specific assets develop; and

      3. – there is even less attention given to how these interventions reduce risk behaviours; where causal mechanisms are specified this largely draws on theories from conventional prevention science such as social learning theory.

   2. Usefulness of PYD theory

      Despite this, the PYD literature offers three useful insights:

      1. – interventions may produce significant reductions in risk in the longer term even if effects in the short term are small because the interventions place the young person on a trajectory whereby more and more positive assets accrue leading to more significant pile up and molecular impacts on risk reduction;

      2. – the focus on pile up of multiple positive assets might ensure that PYD interventions are more likely than traditional prevention interventions to address multiple determinant of problem behaviours; and

      3. – interventions which emphasise hope in young people’s positive potential may be less likely to inadvertently reinforce risk behaviours than prevention interventions which emphasise risk.

   3. Hypotheses about what factors might affect the impact of PYD

      The above might lead us to hypothesise that:

      • interventions that offer and breadth of activities may be more effective for younger adolescents, whereas those which emphasise depth may be more effective for older adolescents;

      • interventions that combine prevention and positive development may be less effective than those that focus only on positive development;

      • interventions of more than 1 year’s duration may be more effective than those of shorter duration; and interventions may be more effective for participants with low or moderate levels of baseline risk given that there is more scope for stimulating ‘intentional self-regulation’.

Synthesis of process evaluations of Positive Youth Development interventions

Hypotheses arising from this synthesis

• Interventions that have specific methods to engage communities will be more effective.

• Projects that engage with schools will achieve better recruitment.

• Interventions that are delivered by well-trained staff will be more effective.

• Interventions that have better staff retention will be more effective.

• Interventions that offer some choices but require some engagement with educational components will be more effective.

How can we enable Positive Youth Development? The research says . . .

Provide a positive environment through:

1. positive expectations;

2. real relationships with adults;

3. diverse activities and settings;

4. active participation.

Positive Youth Development methods are distinctive because . . .

1. Breadth, depth and duration.

2. Focus on positive development not just risk behaviours.

3. Young people contribute to or challenge society.

Young people change through . . .

1. Reflecting on and learning from behaviour.

2. Selecting activities that help them to meet goals.

3. Changing behaviour if not successful.

Young people gain positive assets such as . . .

1. Competence.

2. Confidence.

3. Connection.

4. Character.

5. Caring.

Do you agree?

Introduction

Adam Fletcher came to talk to the ALPHA group about the findings of the PYD systematic review (28 March 2015). A total of 13 young people (10 boys and 3 girls), aged 14–16 years, attended.

Findings from the outcome evaluation

When considering the findings, young people agreed that PYD programmes can have an effect on their drug use but not on their alcohol consumption unless programmes provide a social venue to offer an alternative to drinking. They both agreed and disagreed that PYD programmes can influence smoking and have a beneficial influence on violence.

Findings from the process evaluation

Young people discussed the five themes and wrote messages for the research team to think about.

What are the implications of this research for youth work in the UK?

1. Youth centre closures: youth centres closures and staff turnover because youth workers need to find more permanent positions. New staff are brought in but it affects the youth work as it takes time to build a relationship and trust with the young people and can lead to a decrease in young people attending. Young people are not empowered because the people who run youth services close a centre no matter what the young people say.

2. The value of youth work: not enough people know how youth work is used, valued and appreciated by young people. The service is important because when youth clubs are shut there is nowhere to go. They need to stay open so that young people have somewhere to go and to learn about real world issues.

3. Promoting what youth services do including in the community: youth services need to better promote what they are doing so that young people attend and to ensure that services are less likely to be cut. However, if services were to open to young people in new areas, it is not clear if would be used because people already have routines in the evenings and at weekends. In some areas services do not work with community members but this should be promoted.

4. Somebody to talk to: need people other than parents and teachers to whom to talk. As being a youth worker involves good training and having the right personality, youth workers know that they are dealing with young people so need to be nurturing and caring but also authoritative if needed.

How, and to whom, should we communicate this research?

• People in the council, as they are making the cuts to youth services because they do not know the value of them.

• Communities in general need to know that the youth services are keeping young people off the streets. Then people in the community will value the youth club/services.

• Tell youth fora, youth leaders and youth centres. Send information to youth fora in Wales and they should disseminate them to individual youth workers.

• Send research directly to youth clubs/youth workers if you can find out who they are.