Social Stories to alleviate challenging behaviour and social difficulties exhibited by children with autism spectrum disorder in mainstream schools: design of a manualised training toolkit and feasibility study for a cluster randomised controlled trial with nested qualitative and cost-effectiveness components

One goal

The first criterion emphasises the main purpose and overall tone of Social Stories. It states that the goal of a Social Story is to share accurate, socially meaningful information in a positive and reassuring way. Although they are often used in response to challenging behaviour, the focus of the content of the story would not be on that particular behaviour. The focus of the Story is usually provision of social information with the expectation that it will bring about changes in behaviour that benefit the child. Having one clear and shared goal enables those around the child to support them in a co-ordinated and supportive way.

Two-step discovery

This criterion emphasises the importance of gathering information before developing the story. This is an essential, non-negotiable part of developing a Social Story. It is the process by which the child and their needs are better understood. For example, ‘why is the child frustrated?’ or ‘what is it they do not understand?’. It is suggested that people who know the child best will be the best source of information for this stage. This criterion discourages ‘off the peg’ stories being used that are only partially suitable to the situation or the child.

Three parts and a title

The third criterion simply states that a Social Story is a story and needs a title describing the content, a beginning (introduction), a middle (body) and an end (conclusion). A good title should be clear to the child and easy to understand. It describes what the story is about.

Four-mat

The fourth criterion details four characteristics of the child which should be considered to ensure the story is appropriate for their cognitive ability. This criterion emphasises the importance of tailoring the Social Story to the child. The factors to be considered are their learning style, attention span, comprehension level and reading ability.

Five factors define voice and vocabulary

This criterion recognises the importance of the use of language and how the story is told. It details five separate rules when selecting the language used that are important in providing the story with the appropriate tone. The main emphasis is for the language to be positive and for the information presented to be accurate. The five rules are:

• Write sentences in the first (I, we) and/or third person (she, it, they). Gray3 argues that ‘you’ statements are too directive and/or may be misunderstood by a child with ASD.

• Use a positive and patient tone. The tone of the story needs to be positive, non-judgemental and non-authoritarian. An example of an unhelpful sentence in a Social Story would be: ‘I should not interrupt the head teacher when she is speaking in assembly’. An example of an alternative sentence to use would be ‘Children try to listen when the head teacher is talking to the group in assembly’. This constructive language is an important central feature of a Social Story.

• Use information from the past in Social Stories. Past information can be useful in building self-esteem and remind people about previous positive events and outcomes.

• Social Stories are literally accurate. Each word, phrase or sentence can be interpreted literally without changing the intended meaning. Children with ASD often have difficulty understanding complex, metaphorical phrases or words (e.g. ‘pull your socks up’) or phrases that might have multiple meanings (e.g. ‘take your seats’).

• It is important to be accurate in meaning, especially when using verbs. For example, instead of ‘Jane will get the food from the supermarket’, a better sentence would be ‘Jane will buy the food from the supermarket’.

Six questions guide story development

The sixth criterion ensures that the appropriate content and information relevant to the situation is presented in the story. It encourages the writer to ask the six questions (who, what, where, when, why and how) about the situation that provides in-depth information about the child, and what needs to be explained in the Story to help the child. The questions are intended to explore the gaps in information for the child as the story is developed. For example, where and when does the issue occur? Who does it involve?

Seven types of sentences

The seventh criterion describes how the information included in a story should be presented by detailing seven types of sentence. Each sentence type has a different function in a story. The following are the common different types of sentences used.

• Descriptive sentences: these are sentences that describe the situation factually and objectively. They may also describe the context and the information that we often assume everybody knows.

• Perspective sentences: these sentences attempt to give the perspectives of different people. They can describe a person’s thoughts, feelings, beliefs, internal states and opinions. They can also describe a person’s sensations or physical health.

• Coaching sentences (previously called directive sentences): this category of sentence accounts for three of Gray’s3 sentence types. These sentences gently guide or coach behaviour or suggest a more effective response. They often begin with ‘I will try to . . .’ or ‘I may choose to . . .’ or they list possible responses. There are three types of coaching sentences. Those which:

  1. describe or suggest responses for the child (e.g. ‘I may choose to work on a table where there is no tinsel’ for a child who is so distressed by the presence of tinsel that they often tip the work tables over)

  2. describe or suggest responses for the care-giver (e.g. ‘The teacher will try to set up some tables with tinsel and some tables without tinsel’).

  3. are developed by the child themselves [e.g. ‘I may choose to . . .’ (the child chooses). . .]

• Affirmative sentences: these are used to reinforce the meaning of other types of sentence. They may reinforce a point about safety or a positive statement. For example, ‘Children take turns going down the slide. This is very important.’ or ‘Sometimes I may lose the game. This is OK’.

• Partial sentences: a partial sentence is one in which there are blanks that can be filled in by the child as they read.

A gr-eight formula

The eighth criterion defines the structure of the story by examining the balance of sentence types. It states that for something to be considered a Social Story, coaching sentences must amount to less than one-third of the total number of sentences in the story.

Nine makes it mine

This criterion emphasises the importance of using the child’s interests (e.g. trains or superheroes) in the story. A story should be written with knowledge of the child’s personal interests and preferences in an effort to make Stories more interesting and fun to read.

Ten guides to editing and implementation

The tenth criterion details the best practice for finalising and delivering the intervention. It emphasises the need to monitor and edit the story when necessary and the importance of having a quiet safe place to deliver it.

Evidence for the effectiveness of Social Stories

Social Stories can be effective in tackling challenging behaviour when they set out to explicitly teach social skills. 4 Until recently, research exploring efficacy and outcome has been confined to single-case studies. These case reports in children with ASD have suggested improvements in social interactions,5,6 choice making in an educational setting,6 voice volume in class7 and meal time skills. 8 Success has also been reported in addressing disruptive behaviours, including reducing tantrums9,10 and behaviours associated with frustration. 11 Non-comparative research has examined Social Story use in special education,12 mainstream education settings13 and within the home. 14

Two previous systematic reviews of Social Story effectiveness4,15 identified largely single-case designs and a paucity of good quality, comparative evidence on Social Stories. Both suggest further research, especially with the use of randomised controlled trials (RCTs) to evaluate whether or not positive outcomes seen in case series can be replicated.

Although Social Stories have much potential to improve behaviour, the comparative evidence base is limited and attributed levels of effect are based mostly on single-case research. A well-designed RCT is required to address this important gap in our knowledge, and a feasibility RCT is required to plan the appropriate experimental design. Evidence suggests that Social Stories delivered in a general education setting produced significantly larger effect sizes on behaviour than those implemented at home or in self-contained settings, such as separate schools or self-contained classrooms. 4 Therefore, this study specifically focuses on the feasibility of assessing Social Stories delivered within a mainstream schooling system.

Participants

Intervention

Comparator

Outcome

Study design

Single-case studies

As we have described above, the first aim of the review was to provide an overview of how social stories have been used in clinical and educational practice. We did this first for the 77 single-case designs, then for the seven between-groups designs and finally for the 15 other designs. Table 4 provides a summary of this information for the single-case studies and, related to this aim, Table 5 provides a summary of the extent to which the use of the social stories adhered to the criteria set out by Gray. 3

The majority of the single-case data are based on interventions conducted in the USA. Eighty-one per cent of the single-case studies were conducted in the USA, with only three studies24–26 (4%) conducted in the UK. Approximately one-half (48%) of the studies5,7,12,13,24–55 provided the intervention in a general education setting and just over one-quarter6,8,10,56–75 (29%) used a special education setting. The remainder of the studies9,11,14,76–88 used a home or other setting.

In terms of the study participants of the single-case studies, the majority were male and under the age of 10 years. Only 18% of the single-case studies8,10,31,36,43,47,49,56,59,61,63,64,69,72 had participants who were all aged 10 years or above; in only three studies8,64,69 (4%) were all the participants aged 13 years or above. Over two-thirds7–14,24,26–34,36–41,43–53,55–58,60,61,63–65,67–69,72,74,76,77,80,85,86,88 (71%), had samples that were entirely male, three studies5,66,84 (4%) were all female, and the remaining studies1,6,25,35,42,54,59,62,70,71,73,75,79,81–83,87,89,90 (25%) had a mixed sample. Over half of the studies5–7,9,10,12,13,26,28–30,32,34,37,38,41–43,46,48,51,54–67,69,73,74,76,78,79,83,84,90 had participants who were diagnosed with autism (58%). A small percentage8,24,27,36,40,44,47,52,68,72,77 (14%) had participants diagnosed with Asperger syndrome and just two14,80 (3%) with PDD-NOS. There were a mix of diagnoses in 18% of the studies25,31,33,35,39,45,49,53,68,70,81,87–89 and a small number11,50,71,75,82,85 (6%) stated that the sample were on the autistism spectrum but did not provide further details. The cognitive functioning of participants was not reported in over half (55%) of the studies,6,7,10,11,13,24–30,32,34,36,38–40,42,46,48,50–52,55,57,58,61,63,64,69–71,75,76,78,82–85,87,88,90 but when it was reported approximately half8,37,41,49,56,58–60,62,65–68,73,74,79 stated that participants had low functioning and half5,9,12,31,33,43,44,47,53,54,77,80,86,89 reported either average or above average functioning.

There was substantial variation in the characteristics of the social stories. Two-thirds of studies7,8,11–13,24–26,29,31,32,34,35,37–47,50,53,55,56,58,59,62,63,65–68,71–76,78,81–83,85,87–90 used one story per child, approximately equal numbers used two stories9,10,27,28,48,49,69,70,77,79,80 (17%) or more than two stories5,6,14,30,36,51,52,54,60,61,64,86 (16%) per child. The stories ranged in length from four to 32 sentences; the median sentence length was 11 [interquartile range (IQR) 8–16]. Over half (55%) of the studies5,6,9,10,12–14,24,26,28,29,32,34,36,38–46,51–56,58,59,63–66,70,73–78,80,85,88,89 used stories combining a written format with either illustrations or photographs, and one-quarter25,27,30,33,35,37,47–50,67–69,72,79,81–84,87,90 used a written format alone. Of the remainder, 12% of the studies41,45,60–62,75,86,89 used some form of electronic media (e.g. computer, DVD or audio) either alone or in combination with other media and 5% used a musical format,29,71,78,83 again either alone or in combination with other formats. In the studies that reported data on duration, the length of the intervention ranged from 5 to 85 days with a median duration of 28 days (IQR 16–39 days).

The researchers developed the stories alone in over one-third (36%) of studies11–14,25–28,32,35,40,41,43–45,47,49,55,56,61,64,68,70–72,78,79,81,83,86 and in the same number of studies5,10,13,24,29,31,33,34,36,38,39,42,46,50,52,59,60,63,65,66,72,73,76,82,85,87,88,90 (36%) the researchers worked with others (e.g. parents, teachers) to develop the story. In only three studies36,75,89 (4%) was the study developed by teachers without researcher input. The researchers alone delivered the story in 18% of the studies8,12,13,29,46,49,57,58,71,72,73,77,78,83 and in combination with others (e.g. researchers and teachers) in 8% of the studies. 56,64–66,69,86 Teachers alone delivered the story in 29% of the studies6,10,24–28,35,37–42,50,52,55,59,63,70,74,75 and family members alone in 12%. 11,14,44,51,76,80,81,84,85 In 19% of the studies,5,30,31,36,41,43,45,47,53,54,60–62,68,89 the story was self-administered by the child. Other combinations were used in 9% of the studies9,33,67,87,88,90 and in 5% of the studies34,48,79,82 the information was not reported.

The next stage of the overview of practice evaluated the extent to which the authors had adhered to the Social Story guidance in the development and delivery of the Social Story. Tables 5 and 6 summarise the extent to which the 77 single-case studies adhered to this guidance (see Table 1 above for a description of the Social Story guidance criteria).

For just under half of the concordance items, the studies tended to meet criteria. These included information gathered5,6,8,9,11–14,24–34,36,37,39,40,42,44–47,49,52,53,55–61,63–66,70–72,74–83,85,87–90 (74% of studies met the criterion), structured story8–12,24–27,29,32–40,42,44,46–53,55,56,58–69,71–77,79,80,82–84,86–90 (79%),’wh’ questions answered5,7–12,14,26–29,32–38,40,42,43,46–52,54–56,58–60,62,63,67,68,71–77,80,82–90 (73%), listed sentence types5,7–14,24–29,31–35,37–40,42,44–56,58–80,83,84,86–90 (88%), balance of sentence types5,7,8,12–14,24–29,33–35,37–40,42,44–53,55,56,58,59,61–67,69,71–74,76–80,82,83,85–88,90 (75%) and use of first and third person statements7–12,14,24–29,31–40,42,44,46–56,58–60,62–79,82–86,88–90 (84%). Over half of the studies7–10,12,14,25,26,28,29,31–33,35,36,38,39,42,46–51,56,58–60,62–66,68,69,71,72,74,78–80,82,83,85,86,88,90 (62%) met the criterion for avoiding the use of ‘should’ and ‘must’, although a sizeable minority11,24,27,34,37,40,44,52–55,67,73,75–77,84,87,89 (25%) did not meet this criterion.

For five of the criteria, the majority of studies were rated as unclear. Categories in which only a small proportion of studies were coded as meeting the criteria included tailored for ability5,9,12,13,25,27,29,35,39,42,44,55,59,62,63,65,73,75,76,82,85,87–90 (32%), tailored for interest5,9,27,64,65,71,72,74–77,80,83–85 (19%), story edited14,25,28,39,49,51,59,63,65,72,82,87,88 (17%), appropriate setting5,9,14,25,29,32,35,36,46,48–51,53,56,58,63,65,68,69,71,72,78,83–88 (38%) and introduced appropriately12,27,32,33,42,48,49,52,63,87,88 (14%). For the remaining three criteria (positive goal, non-authoritarian and literally accurate), approximately equal numbers of studies did and did not meet criteria.

A minority of studies either stated they did not adhere to the Social Story criteria in developing and delivering the intervention (5%)11,32,75,82 or did not report if they adhered to them (20%). 6,8,9,24,29,36,38,48,51,54,57,70,74,83–85 However, 52 studies7,9–12,14,24,27–29,31,32,34–37,39,40,42,44,46,47,50,52–55,58–60,63,65,67,68,70,73–80,82–90 (68%) were rated as not meeting at least one of the criteria. Of the remaining studies, only 1025,26,33,48,49,51,56,62,71,72 were rated as meeting 10 or more of the 15 criteria. No study met all 15 criteria.

Between-groups designs

Although the vast majority of studies used a single-case approach, we identified seven studies91–97 that used a between-groups design, of which four92–95 used some form of randomisation. Table 7 summarises the descriptive characteristics of the four randomised studies92–95 and Table 8 does the same for the non-randomised studies. 91,96,97

With the exception of one study91 conducted in Egypt, all of the studies were conducted in the USA. Three of the four randomised studies92–94 conducted the social story under experimental conditions; the remaining randomised between-group study95 did not report details of the setting. The three non-randomised designs91,96,97 were conducted in educational settings.

The gender of the participants was not reported in three studies. 91–93 In the remaining four, three94–96 of the samples were predominantly or entirely male, one study97 had an equal balance. The studies were conducted in a range of age groups. Five of the studies91–94,96 were conducted with children diagnosed with autism and two95,97 had a mix of ASD diagnoses. The participants were coded as having low cognitive functioning in all of the studies in which this was reported.

In terms of the characteristics of the social stories, all four of the randomised designs92–95 used one story per child. Two96,97 of the non-randomised designs used more than two stories per child (details not reported in the third study91). Three studies92,94,95 used a written format alone and three93,96,97 used a combination of written story and photographs (data not reported in one study91). The stories were all delivered over 1 day in the four randomised designs;92–95 this is in contrast to two of the non-randomised designs,96,97 which used interventions over 1 month. In the four randomised designs92–95 the research team developed the stories and delivered the stories. Information about who developed the stories was not given in the three non-randomised designs;91,96,97 teachers delivered the story in two96,97 of the three non-randomised studies (details were not given in the third study91).

The comparator used in the randomised designs was a story without a social loading. In the non-randomised designs, two studies96,97 did not use any form of story and details were not reported in the third. 91

Table 9 summarises the extent to which the between-group randomised designs adhered to Social Story criteria; Table 10 summarises the same for the non-randomised designs. For the randomised designs, all four studies92–95 met the criteria for at least 10 of the 15 items. For three items, the information was coded as not reported in all four studies. For the remaining criteria, the studies varied in the extent to which the criteria were met. All four studies stated that they followed Gray’s criteria; however, two of these studies94,95 did not meet the criterion for avoiding ‘should’ and ‘must’, and one study92 did not meet the criteria for correct balance of sentences.

For the three non-randomised designs,91,96,97 the level of reporting was not as clear. One of these studies91 provided little detail of the story, so all criteria were rated as not reported. For the Romano study97 the majority of items were also rated as not reported, although for the remainder the criteria were met. For the remaining study,96 the picture was more mixed.

Other studies

The remaining 15 studies described the use of a social story, but did not use either a single-case design or a between-groups design. Table 11 summarises the descriptive characteristics of these studies. Unlike the other designs, the majority of the remaining studies were not conducted in the USA and approximately half98–104 were conducted in the UK. In four studies98,101,105,106 the intervention was delivered at home and in seven100,102–104,107–109 in an educational setting. One study110 used a dental setting and three studies99,111,112 did not provide these details.

The participants that received the intervention ranged in age from 4 to 15 years. The majority of the participants were male. The participants had a diagnosis of autism in two studies,102,109 Asperger syndrome in four103,107,108,110 and PDD-NOS in two. 105,106 In six studies98–101,104,112 the participants were described as having an ASD, but no further details were given. Few of the studies provided details of the level of cognitive functioning of the participants.

In terms of the characteristics of the Social Stories, the majority (nine) of the studies98,100–103,105–107,110 had one story per child. Five of the studies103,106,108–110 used a written format, six98–101,107,111 used a combination of written story and photographs or illustrations, and three102,104,105 used an electronic medium either alone or in combination with another format (details not reported in one study108). Details of the timing of the Social Stories were not reported in five studies. 98,103,109,110,112 In the remaining studies, the stories were read once a day in five,100,105,106,108,111 more than once a day in three99,101,107 and fewer times than once a day in two studies. 102,104 Few studies provided information on how many days the stories were read to the child, although when this was reported it was for at least a week (10 days, 4 weeks and 8 weeks). The story was developed by the research team in two studies,98,106 the research team in combination with others in two studies105,111 and teachers alone in six studies. 100,102,104,107–109 Details about the delivery of the social story were not given in six studies. 99,100,103,108,110,112 In three studies,104,107,109 a teacher delivered the stories and parents alone in two studies. 98,101 Researchers delivered the story in one study106 and the research team and others in one study. 105 In the remaining studies, the story was delivered by the child themselves. 102,111

Table 12 summarises the extent to which the 15 studies classified as other designs adhered to the Social Stories guidance. Most of the studies met the six following criteria: positive goal98–101,103–105,107,112 (67%), information gathered98,101–105,107–109,111 (67%), structured story98,100,101,103–105,107,109,112 (60%), ‘wh’ questions answered98–101,103–105,107,109,110 (67%), listed sentence type98–101,103–105,107,109,111,112 (73%) and first and third person used98–101,103–105,107,109–111 (73%). For a further four concordance items, approximately half of the studies met the criteria: balance of sentence types98,100,101,103,104,107,109,111 (53%), non-authoritarian98–101,103–105,107 (63%) and avoids should/must99,101,103–105,107,109,112 (53%).

For another five criteria, the majority of studies were rated as not reported; these were tailored for ability98–100,102–104,108,110–112 (67%), tailored for interest98,99,102,103,108–112 (60%), story edited98–101,103,104,106–108,112 (67%), appropriate setting98,99,102–104,108–112 (67%) and introduced appropriately99–104,106,108–112 (87%). For the final criterion, literally accurate, four studies98,103,111,112 did not meet this criterion, six studies99,100,104,105,107,109 did and the remainder were coded as not reporting the information.

Just over half of the studies100,102–105,107,109,111 made a statement that Gray’s criteria were followed (53%). However, similar to the findings on the concordance of the other study designs to the criteria, only five studies101,102,105,107,110 (33%) had no ratings of ‘not meeting the criteria’ in any category and only three101,105,107 of these (20% of the total number of studies) were rated as meeting 10 or more of the 15 criteria. No study met all criteria.

Single-case designs

Landscape interviews

‘Landscape interviews’ were conducted with key informants in order to define and articulate the key concepts and ideas of relevance to the project113 and fill out the topic guide for the scheduled user focus groups. It was important that key stakeholders were represented – including parents/carers, clinicians, learning support workers and children with ASD. Participants were opportunistically selected from individuals known to two clinician members of the study team as having prior experience with the use of Social Stories. It was expected that a sample of five to ten participants would be sufficient to map out the key issues around the use of Social Stories, in order to familiarise the research team with the topic and to inform subsequent data collection in stage 1 of the qualitative research.

Semi-structured interviews were conducted, using a topic guide with adult participants (see Appendix 4). The interviews with children with ASD were more flexible which allowed the researcher to meet the individual needs of the child, and also focus on their areas of interest to aid responses.

Focus groups

In order to inform the development of the Social Stories training manual, focus groups were employed to gather information from key stakeholders. It was felt that two focus groups would be adequate to gain an understanding of participants’ experience with Social Stories and the features they would like included in a Social Stories manual. One group comprised key ‘professional’ stakeholders, the other parents and carers. From the outset it was felt that it would be preferential to separate professionals from parents, as this would allow for a range of discourse in which the parents felt more empowered to speak openly. This was suggested by PPI representatives on the steering group. Participants were purposively sampled to include representation from relevant professional groups (including psychology, teaching and speech and language therapy). For the parent group, it was important to include individuals with a range of expertise of Social Stories and with differing perceived outcomes.

Participants were not given anything to prepare, they were simply asked to come and talk about their experiences of Social Stories with others. A range of different activities broke up the focus group into distinct parts which focused around:

• understanding of Social Stories

• informing the content of the manual

• informing the training package.

(See Appendix 5 for further details.)