Translation into British Sign Language and validation of the Strengths and Difficulties Questionnaire

Mental health in deaf children

Why are mental health problems more common in deaf children?

There are various factors that are proposed to influence mental health negatively in deaf children, including degree of deafness, age at onset, and the presence of additional neurological difficulties. 15 Factors that are purported to increase the likelihood of behavioural and emotional problems include low intelligence quotient (IQ) and comorbid physical7 and neurological problems, severe or profound deafness,16 poor development of language and communication,17 and poor parent–child communication. 5 Factors that increase risk of ADHD are lower intelligence, acquired deafness (compared with hereditary deafness)18 and significant communication difficulties. 19 ADHD is thought to be more common in deaf children. 18 It has been suggested that people who are deafened by rubella infection have five times the likelihood of experiencing subsequent psychosis. 20

Long-term outcomes

In deaf children, mental health is associated with long-term psychological morbidity, poor educational attainment, increased unemployment, increased crime and delayed social skill development. 58,86

The rates of mental disorder in the adult deaf population are higher than for hearing people. This is particularly true for diagnoses of personality disorders102 and emotional disorders,103 which are unlikely to receive treatment in their early stages for lack of access to appropriate interventions.

Deaf people are more likely to enter the criminal justice system and are overrepresented in the prison population. 104 It has been reported that over 30% of offenders have significant hearing loss, with suggestions that this may be related to the increased risks in deaf people of school-related failure, limited social relationship networks, poverty and unemployment. 105

As a result, many organisations have highlighted the need for better access to mental health services for deaf children and young people. To identify difficulties at the earliest opportunity, in order to be able to intervene to alter the trajectory of these young people, we need to have a valid screening instrument.

Early-intervention programmes for deaf children benefit parents and their deaf children63,106–111 but most of them are focused on language development and communication. Identification of mental health problems earlier would allow clearer understanding of need, proactive development and assessment of support or therapeutic programmes, and potentially better outcomes.

Difficulties identifying mental health problems in deaf children

There is very little research with high-quality methodologies in the area of mental health in deaf children, partly because of the lack of any suitable screening or outcome measures. The research reported here will provide a foundation on which further research studies can build both in terms of translation methodologies (from spoken to visual language) and in validating a mental health instrument with deaf people, and using it in the future. It is important to identify mental health problems in deaf children for the multitude of reasons already outlined.

However, the mapping of need is not accurate because, at present, we do not have an appropriate tool to screen deaf young people for mental health problems, nor are we able to evaluate whether the specialist National Deaf Child and Adolescent Mental Health Service (NDCAMHS) in England generates positive outcomes for clients or not. Deaf children may communicate in a range of different ways: 78% of children use spoken English only, 14% speak another spoken language either on its own or in combination with spoken English and 8% use sign language either on its own or alongside spoken English. 51

Until the development of this questionnaire, there was no instrument or questionnaire that focused specifically on those young people who use BSL as their first or main language. Within child and adolescent mental health service assessments, it is standard for parents and teachers also to complete proxy questionnaires to contribute to the assessment. The BSL questionnaire resulting from this research will support clinicians in making more accurate clinical screening and assessments. It will also enable future researchers to map out the prevalence of various mental health problems in deaf young people, therefore allowing NHS services to target interventions and spending where it is most needed and most likely to be of benefit. It will allow us to measure outcomes more effectively and will lead to improved skills in developing screening and assessment instruments that are accessible to deaf children.

Strengths and Difficulties Questionnaire

The SDQ is a multi-informant mental health questionnaire that can be used as a screening tool and a treatment evaluation measure. The ‘informant-rated’ version of the SDQ can be used for children and young people aged 4–16 years, and is completed by either a parent or a teacher; the ‘self-report’ version can be completed by young people aged 11–16 years. 112 Each version of the questionnaire comprises 25 questions. All questions are scored on a three-point Likert scale, which can be divided into five subscales measuring:

• emotional symptoms

• conduct problems

• hyperactivity–inattention

• peer problems and

• pro-social behaviour.

This last subscale is a ‘strength’ subscale. The first four are ‘difficulties’ subscales. 113 The SDQ11 has been translated into over 60 spoken languages (www.sdqinfo.com) but not BSL. It is a self-report questionnaire, initially developed to improve the detection of child psychiatric disorders in the community. 1 There are three versions: one for children and young people, one for parents/carers and one for teachers. Together, the three SDQs show good sensitivity (63.3%) and specificity (94.6%). 114 The SDQ can be completed at the beginning and end of treatment to assess treatment efficacy115 and it is frequently used to evaluate Child and Adolescent Mental Health Services (CAMHS). 116 The SDQ has a satisfactory internal consistency (Cronbach’s alpha ranging from 0.80 to 0.87) and specificity ranging from 94% to 95% and sensitivity ranging from 23% to 47% for each version (parent, teacher and child). 113

Barriers to developing or validating screening instruments for deaf children

Generic CAMHS have only relatively recently started prioritising the measurement of outcomes. In the UK, the SDQ is recommended for national use within NHS mental health services by the CAMHS Outcome Research Consortium (CORC) guidelines117 and is a core part of the new national CAMHS data set required by commissioners of services and the UK government to evaluate services across the country, as from April 2013. 118 The newly funded CAMHS Increasing Access to Psychological Therapies (IAPT) initiative119 also requires better outcome monitoring. Local CAMHS often report that deaf children go to services in smaller numbers than expected. 120 There are several apparent reasons for this, including deaf families having a poor experience of services designed for hearing people with often limited understanding of Deaf culture and deaf developmental issues as well as limited and inconsistent availability of appropriately trained interpreters.

Research funding has not been sought or granted very often for deaf child mental health over the last 20 years in the UK. There are also considerable practical and theoretical boundaries to translating and validating screening instruments in a visual language, with no well-developed methodologies. 13,121

Few widely used mental health instruments are available in a format that is culturally and linguistically relevant to the Deaf population. 122 ‘Inappropriate tests, unsatisfactory administration, and the unrealistic norm referencing of results’123^{(p. 249)} remain the biggest challenges to overcome in mental health research with deaf people.

Although sign language versions of the SDQ have been produced previously,124,125 they have not been fully and rigorously translated or validated. The methodology that this study has adapted was developed for translation of spoken and written languages, which outlines the process of translation/back-translation. 14 This methodology is widely used to translate mental health questionnaires by organisations such as the World Health Organization (WHO). Additional methodological considerations that pay close attention to cultural and linguistic differences126 can result in questionnaires and screening tools that are more meaningful to deaf participants as long as this is conducted rigorously and systematically.

In clinical practice, sign language interpreters may be called upon to do on-the-spot translations of diagnostic and screening instruments, but this process can present problems. Interpreters make a linguistic and conceptual leap based on the experience of a deaf person to relay information that is contextually relevant to a hearing mental health professional. 127 Each interpreter will bring something different to this interaction, based on their own experiences, background or training, so the information given may not be consistently expressed. Working with interpreters means that variability in the translation and meaning of questionnaires is inevitable. The other reality is that deaf young people and deaf parents are likely to meet these BSL interpreters in other areas of their life, such as education meetings, financial or social care appointments. The impact of this is that it is likely that clients will not communicate as openly or as honestly as they might, because of confidentiality concerns. There is significant value in a self-report measure.

Such inconsistency could be overcome by using a validated instrument. Arguably the best way of achieving culturally sensitive instruments is to develop them empirically from first principles. However, this process is expensive and time-consuming. An alternative solution is to translate existing instruments into BSL, but the current evidence base outlining best practice is small. Rogers and colleagues gave an in-depth account of the issues regarding translation of standardised mental health assessments into BSL. 128 These issues included the direction of the signing; influence of modality; emotional state in BSL; confirmation of statements; and statements in a social context. Cornes and colleagues13 have noted that pencil/paper tests tend to underestimate prevalence of common mental health difficulties in deaf people, particularly emotional problems, hypothesising that this is partly because of the written language gap often present in deaf people in hearing environments. Linguistic delays, particularly in younger children, may invalidate standardised instruments validated for their hearing peers. For clinicians this presents additional difficulties, particularly those who have little knowledge of deafness or Deaf culture in assessment and diagnosis. The consequences of this may be that many deaf young people do not receive appropriate treatment or they may develop more complex, long-term difficulties.

Previous research has highlighted the difficulties translating into sign languages128–131 but few studies give detailed descriptions of the translation process from English to sign language, and its challenges. We describe here the process employed in translating the SDQ into BSL.

What is the service context in the NHS?

In October 2009 the NDCAMHS was set up in England, specifically to target the needs of deaf children and young people with mental health problems. Its ethos is to be accessible to all service users no matter what their cultural or linguistic background. There are four main bases for NDCAMHS across the country: York, London, Dudley and Taunton. As a national research project, recruitment was led from these sites.

The outcome of this research was a validated translated questionnaire, which was then used both in practice for clinical screening and for research to determine prevalence and types of psychiatric morbidity in deaf young people.

This research involved the collaboration of a comprehensive network of national centres (in four main centres and six outreach centres that map onto the 10 old strategic health authority regions in England). This allowed us to screen children from across England. The research will, therefore, be generalisable nationally. One of the NDCAMHS’s aims is to improve accessibility of services to deaf children and this will be a vital tool in furthering this aim. It will enable better and earlier identification of deaf children with emotional and behavioural problems.

How could a new screening instrument be used?

The development of this screening instrument for those whose first or main language is BSL will become a routine part of the NDCAMHS’s ability to screen for mental health problems and to monitor mental health outcomes. This will be part of a set of outcome measures that will meet the government’s target that high-quality services should have good measures that monitor outcomes routinely. This has been prioritised in the National Services Framework, through CORC137 and through CAMHS IAPT119 and the new ways of working in the NHS.

With more accurate estimates of the prevalence and types of psychiatric morbidity in deaf young people, the organisation and delivery of CAMHS for deaf young people can better reflect the needs of this population. This could enable the development of appropriate care pathways for deaf children, which necessarily and appropriately involve professionals at tier 1 (e.g. social workers, teachers, youth workers), tier 2 (e.g. educational psychologists, mental health workers), tier 3 (e.g. specialist mental health teams including child psychologists, clinical psychiatrists) and tier 4 (e.g. residential units). Currently, deaf children find accessing mental health support difficult,138 and more accurate mapping of mental health needs will enable more appropriate targeting of services. It will also enable the NDCAMHS teams to monitor the outcome of their interventions and build up the evidence base of what works with this population, as there is very little evidence at the present time.

Identifying translation teams

Stage 1 was built around the structure of translation/back-translation methodologies, requiring a forward translation team and a back-translation team, independent of each other. The translation teams were made up of bilingual BSL/English professionals who had experience of translation work. Beaton and colleagues14 suggested that there should be two people each in the translation and back-translation teams. We included three in each team because several factors affect variation in BSL (or differences in BSL production, including age, educational background, previous communicative and linguistic experiences, and family history of deafness139–141) and it was important to have geographical diversity in the translation and back-translation groups. BSL varies dialectically across regions. Whereas English dialect varies largely according to accent and syntax, BSL variation is more related to lexical differences. It was therefore important to find more widely used signs and phrases where possible. 142 As there are questionnaire versions of the SDQ for young people, parents, and teachers, having a range of ages represented within the translation groups was essential.

Consultation with professional deaf researchers suggested that there was benefit to having Deaf translators on both teams, as Deaf and hearing cultures are very different. We utilised the skills of bilingual deaf translators as they could more readily discriminate the sign meanings, which was more useful for translation purposes.

Beaton and colleagues suggested that having a balance of expert and lay members on a translation team assists in retaining focus on both the academic aspects of translation (e.g. reliable knowledge of the constructs being measured) and the meaning of the language as it would be perceived by a wider population. 14 This was reflected in the construction of the translation teams, which comprised equal numbers of clinical psychologists and those experienced in translation work across the teams. Equally, members of an expert panel overseeing the translation process were selected on their ability to comment on the different psychological, linguistic, psychometric and cultural aspects of the translation. This group comprised the project leaders (two psychiatrists working in the field of deaf child mental health), a linguist with sign language expertise and a range of deaf professionals experienced in mental health, research and translation work.

The team also considered the issue of agency, in that sign language has to be signed by someone, whereas a written questionnaire involves no other person. This carries with it potential transference143 issues that may resonate in terms of perceptions or feelings, and attributions or unconscious feelings about the signer. For example, a questionnaire signer that reminds a child of their critical mother may affect responses and emotions. There may be a preference of adults/young people for certain types of signers. Separate versions of the SDQ were filmed, presenting different characteristics of the signers (male/female, younger/older, etc.) with a final version chosen by the focus groups of young people. A written test is neutral in this respect.

The translation methodology

The translation process is illustrated in Figure 1. Initially, three forward translation team members each filmed independent BSL translations of the SDQ materials, resulting in an initial unbiased signed translation. The translations were collated, presented and then discussed among the three forward translators as a group. The decision-making followed a systematic process of discussion and consideration of alternatives and appropriateness for the target audience (whether young person, parent or teacher). As a result, each item was filmed again, by one of the translators at the meeting, after agreement was reached on the best version. These final versions were then passed to the back-translation team.

FIGURE 1.

The translation process.

The back-translation team, three bilingual translators who were blind to the original versions, each individually produced English translations from each BSL item. These are called back-translations because they are taking the translated versions (now in BSL) and translating them back into English. The back-translations were collated and presented to all three translators for discussion as a group, where the appropriateness of each English word and phrase selection was scrutinised, and a final version agreed. This differs from the recommended process whereby independent back-translations are used only as a validity check for ‘gross inconsistencies’. 14 However, because sign languages are visual languages using the hands, face and body, it is possible for several meanings to be conveyed simultaneously. 144,145 Translating these meanings from signs into words is challenging because some elements that may be represented by words in spoken languages are expressed in sign languages through timing, aspects of sign articulation or facial expression.

Signer characteristics

As discussed, the characteristics of the signer were considered by the focus groups and expert panel. It had been hypothesised that a male signer would elicit lower response rates for emotional disorders, whereas a female signer might cause a young person to respond to questions about conduct or aggression less readily. Despite our initial belief that there may be preferences or biases in this area, both the young people and adult groups stated that, as long as the presenter was comfortable and clear in their signing, age and gender were of limited relevance to them. It is the content of what is being said that is of most relevance. (This was qualitative feedback and was not tested in any quantitative way.)

Culture, experience and language level

Deaf people’s experiences are very different. This includes differences in family background and makeup, differences in educational experiences, and changes across time and development. 146–148 They may be part of a Deaf culture, a hearing culture or both, and this may change over time. 149,150 For example, there is a question in the self-report SDQ about ‘playing alone’. This does not capture the fact that a deaf child might play alone because they struggle to communicate with their peers, rather than play alone because they do not have the necessary social skills to play with others. Thus, they might play alone at a school where they have only hearing peers, but play quite happily with other deaf children with whom they can communicate in their home environment. This may have the effect of creating disparities between parent and teacher responses. Another example of this is the question about whether or not the child is nervous in new situations; commonly deaf children will be nervous in situations with hearing groups, but will be much more confident at the Deaf club. This makes it very important for the clinician using the screening questionnaire to understand why any particular child might answer in any particular way and to have a good understanding of that child and the deaf experience.

A major point of discussion during the initial filming and reiteration of the young person informant version of the SDQ was around the level of signing in 11- to 16-year-olds. The discussion group was carried out with deaf young people, where the level of signing ability varied within the group. The translation strategy used to overcome this was to pre-pilot the SDQ in focus groups of young people. This allowed a ‘road test’ of the BSL SDQ, highlighting issues in vocabulary choice and structure of sentences. By including bilingual deaf signers as well as two psychologists on the forward translation team, the translators were able to adjust to an appropriate language level, as they had an understanding of the types of language that young people used in real-world environments.

Feedback from the focus groups highlighted a range of issues reflecting diversity of needs in the Deaf community. Whereas, in spoken language translations, questionnaires would be piloted in the hearing population, deaf young people often grow up with a mixture of language experiences, and may use many multimodal strategies in order to gain maximal information from messages. Though the fluent signers among the young person’s focus groups understood the items with relative ease, those from oral English families found this more difficult, despite using sign language as their main mode of communication. Despite ‘successful’ back-translation before the focus groups, it became apparent that the focus group did not understand some of the more complex BSL constructions in the questionnaire. Discussion with the focus groups led the team to believe that the level of language was inappropriate for this group, and therefore some BSL items were retranslated to simplify the language, though care was taken to preserve meaning.

Non-manual features

Non-manual features of sign languages (such as facial expression) can change meaning or imply hidden meaning. This may influence the answer given or alter how a person interprets a question, similar to the way that tone of voice in a spoken language can convey different meanings. This was the case in one of the items on the SDQ, which asked ‘do you take things that are not yours?’ The first translation of this was produced with non-manual features for a question, but with a facial expression indicating that, if the respondent were to choose ‘certainly true’, there would be a negative judgement made on the person answering the questionnaire. It is important at all times for the signer to produce sentences in an attitudinally neutral and non-leading way using only obligatory grammatical features; paralinguistic features should be avoided as much as possible, unless they are required to convey clear meaning, in order not to bias responses.

However, in the translation/back-translation process these features make it harder to obtain total linguistic fidelity. For example, the English SDQ response set for the main test items is on a scale of ‘certainly true’, ‘somewhat true’ and ‘not true’. In BSL, the degree of truthfulness is expressed on a continuum by facial expressions. These represent degrees of certainty simultaneously articulated with the sign ‘TRUE’. Facial expression thus inflects the sign and alters its meaning in the way that the modifiers ‘certainly’, ‘somewhat’ and ‘not’ change meaning. Selecting a sign-for-word substitution, as in SSE, would also be to negate a natural linguistic feature of BSL production. Content-for-content translation is common in interpreting scenarios but this elaborative process means that psychometric properties of a standardised questionnaire may be compromised if not done with great care. 127 In terms of finding suitable translations, we worked according to procedures outlined in previous research. 129 Discussions about the exact meaning and context of each statement were considered first by the bilingual translation terms, then by the expert panel and the focus groups and finally by the author. This process presented a range of checks and balances to obtain cultural and linguistic fidelity in our translation that was not without difficulty, but was thorough.

Statements as questions

The original English SDQ is formatted as a series of statements with which the respondent does or does not agree. In this way, the statement reads ‘I . . .’ In BSL the sentence must instead be produced in the second person, as the presenter and the respondent are not the same person. This, therefore, means that the sentence must also be interrogative, or something to be agreed or disagreed with, in order to elicit a response. This means that there are two key changes to the statement’s linguistic structure: a change of pronominal deixis, and a change of sentence format from declarative to interrogative.

Previously, sign language translation studies have made this reference to the self clear by using a technique of finishing each statement with the index finger pointing outward with head tilted to indicate questioning. 20,24 This might be glossed as ‘YOU WHAT?’ Using this second person singular pronoun denotes that the item is referenced to the test taker, rather than the person signing on screen, and the sign glossed as ‘WHAT’ makes it clear that the sentence is designed to elicit a response. Despite this, feedback from focus groups indicated that this format of questioning seemed unnecessary and was, in some cases, confusing.

Based on this, forward translators attempted to further clarify the distinction between the presenter and the respondent. For parent and teacher statements, asking about the child, ‘THIS CHILD’ was placed at the beginning of each statement. At the end of a sentence, an appropriate sign more related to the question was chosen (such as ‘YOU HAVE?’, ‘YOU BEEN?’ or ‘YOURS?’) rather than the more generic ‘YOU WHAT?’ Although it is not then standardised throughout the questionnaire, it fits more closely with the natural characteristics of BSL sentence structure.

Sign placement

The English version of the SDQ contains the item ‘nervous or clingy in new situations, easily loses confidence’. Many signs in BSL require the location to be marked. For example, in a class of verbs known as ‘directional verbs’, the location of the subject and object determine the start and end point of the sign when articulated. This is known as placement.

In translating ‘clingy’, the forward translators had to show that a young person might be clingy to a parent or guardian. In English this might be implicit, but in BSL needs to be referenced as the sign is visual and clinginess is directional towards somebody.

General and specific wording in oral English and British Sign Language

Having an understanding of Deaf culture within the core research team is highly important, in order to understand the functional ways in which BSL is commonly used. Category words or words with ambiguous meaning can be difficult to translate without further concrete explanation of what that word or concept might include (e.g. in BSL the signing for ‘considerate of other people’s feelings’ might include noticing those feelings in others, thinking about them and moderating one’s actions based on this observation). The SDQ tends to give examples where this has been felt to be necessary. Where examples are not given in the English version, the challenge is to avoid choosing signs that narrow the options too greatly (e.g. considerate of a specific person’s feelings). Research has shown this to be a key problem for interpreting in mental health settings. 127 An additional complication here is that lack of access to situational cues and incidental learning may mean that deaf young people struggle to ‘get the gist’ or understand when context is general as opposed to specific. Without a certain degree of openness to the statement, they may believe that the question relates only to a very specific context. To overcome this, translators must be aware of how they are contextualising situations in their sign choices and placements, and try to strike an appropriate balance between clarity and scope. Having psychological and psychiatric expertise within the research and translation teams allowed choices to be made based on the original intent behind each item.

This also links with an additional concern that any concrete explanation may be a judgement by the translator that goes beyond the meaning that is intended in the original. This is a particular concern between spoken and sign languages, as some English words have a more general sense than their BSL equivalents. The SDQ has an item in the parent version ‘Often complains of headaches, stomach-aches and sickness’. The word ‘sick’ in English can be translated into BSL with two different signs, which can also be glossed as ‘VOMIT’ (which is more specific than ‘SICK’) or as ‘ILL’ (which refers to more general illness or malaise). The English word ‘sick(ness)’ is ambiguous between these two meanings, but in BSL, as in other languages, one is explicitly forced to make a choice because there is no sign that covers both meanings in the way that ‘sick’ does in English. Thus, within the translation process it was important to consult with the author as well as mental health professionals, to understand how intended meaning is received, so that it is clear to the respondent; hence the need for culturally aware translation and focus groups.

Structural characteristics of the questionnaire

A key structural difference between a language presented in writing and one presented on film/video such as BSL is that, in responding to a written test, instructions are always present at the top of the page, and a respondent can keep checking back. However, with a visual questionnaire, this may need to be reiterated within the content or technical solutions sought to readily access instructions. Time frames, scales and instructions may need to be reinforced, and it may be necessary to give a specific contextual placement in each case rather than assuming that the information will be retained through several items.

Power calculations

Inclusion/exclusion criteria

Three main groups we sought to include were:

• Deaf 11- to 16-year-old young people who are BSL users compared with the clinical interview and the SDQs from their parents and teachers (deaf and/or hearing).

• Deaf parents who are BSL users (with 4- to 16-year-old children) compared with the clinical interview and the SDQs from the young people and teachers (deaf and/or hearing).

• Deaf teachers or teaching assistants who are BSL users (teaching 4- to 16-year-old children) compared with the clinical interview and the SDQs from the young people and parents (deaf and/or hearing).

Where additional disabilities were felt to affect the language of the young person, as determined by Deaf bilingual BSL researchers, they were excluded from the study. An example of this is when a young person requested to take part in the study at school but when the Deaf researcher met him it was clear he was unable to understand BSL because of his learning difficulties. Participants currently experiencing an active episode of psychosis were also excluded, but those with an existing diagnosis of psychosis without active symptoms were included in the research.

Procedure and recruitment

To recruit the clinical sample, staff from the 10 NDCAMHS outreach centres and the residential unit Corner House in London identified which children or young people had BSL as a main language. Their clinician then sent each of these families information leaflets about the study.

To recruit from the community we used a variety of strategies. Since each child in England with a loss of more than 40 dB has been routinely referred to a ToD, each locality keeps a register of such children. Using this, teachers passed an information pack containing consent forms and an information sheet about the study to children and their parents/carers. These information packs were available in plain English and BSL on DVD.

Researchers also visited all deaf schools across the country, as well as large hearing-impaired units in mainstream schools, and known youth clubs. Information about the study was posted on NDCS newsletters. It was established very quickly that prospective participants preferred face-to-face contact to sending out letters or advertising online. This may be because of a number of factors that the team considered. These included a possible preference of the Deaf community to converse directly with researchers whom they met and recognised and were able to ask as many questions as they would like. It also may have been that, because of the delay in English language comprehension, sending letters without having first made contact to explain the research may have been confusing for several families.

In order to recruit deaf teaching staff, we actively approached all the deaf educational establishments across the country. Many centres were extremely receptive to the aims and needs of the research team and were very happy to facilitate recruitment within the school environment. This made a marked difference in levels of recruitment and emphasised the importance of getting educational establishments on board with research of this nature. Deaf education is a hotly debated area, and many schools have different stances on the promotion of BSL or sign languages, inhibiting possibilities for successful recruitment in some areas. However, collaborative relationships with staff members who understood the relevance of mental health research to the well-being of their students and a clear understanding of the commitments involved in taking part in the study was of vital importance to the recruitment of deaf young people to this study.

Once consent was obtained, young people, parents/carers and teachers were asked to complete the SDQ. All parents and young people were also asked to complete a screening questionnaire confirming their eligibility to participate in the trial, and provide basic demographic information about themselves.

For completion of the SDQ, young people were asked to complete the BSL online version or the BSL DVD version with paper fill in. Depending on their preferred language, parents/carers and teachers were able to choose between the standard version of the SDQ and the BSL online version or the BSL DVD version with paper fill in. Where necessary a clinician or researcher fluent in BSL was available to support the process, but did not influence the answering of the questions or interpret them in any way. They gave gentle prompts and encouragement.

Young people were given a £10 gift voucher in order to thank them for their participation.

To produce the online version, we used the software Select Survey version 3 (Atomic Design, Kansas City, MO, USA), which enabled us to develop an interactive online video questionnaire. The software had been used in the past by experienced Deaf researchers from the SORD unit, who are collaborators on this study. We were able to upload the final BSL translations, and participants were able to select and input corresponding answers online. This program also allowed us to obtain consent online; the personal identifiable information from this was stored separately from any questionnaire data provided for the initial test. All data were stored centrally by the University of Manchester, and only members of the research team had password-protected access. The university is an established research centre with many years of experience in research methods and information governance, and we were confident that this was a secure method of retaining our data. This method of consent was visual and preferred by deaf people who use BSL.

Statistical methodology

Overall recruited sample

Figure 2 shows the planned (and actual) recruitment in the validation process. We recruited 144 deaf children (aged 11–16 years) and their parents/carers and teachers (both hearing and deaf) to validate the deaf young person version (11–16 years) of the SDQ.

FIGURE 2.

Design of the planned (and actual) validation procedure.

We recruited 191 deaf parents who had deaf or hearing children ranging from 4 to 16 years old. We sought deaf parents of deaf children or young people but we also recruited deaf parents with hearing children or young people (because only 5% of deaf children in the general population have at least one deaf parent).

We recruited 77 deaf teachers and teaching assistants to validate this version of the SDQ.

Young people sample description

We needed 138 young people for statistical power. We recruited a total of 144 young people aged 11–16 years who completed the SDQ in BSL. The majority (116, 80%) were recruited from the community through schools, youth groups and Deaf community contacts. A further 27 (19%) were recruited from the NDCAMHS (a clinical sample). We do not have this information for one participant.

Of the schools we approached we recruited from:

• 10 of the 11 (91%) deaf residential schools (nine with primary/secondary provision and one with secondary provision)

• one of the two (50%) deaf secondary schools (day pupils)

• three of the four (75%) deaf primary schools (day pupils)

• 15 of the 54 (28%) mainstream secondary schools with hearing-impaired units

• one of the two (50%) mainstream secondary schools without a hearing-impaired unit

• eight of the 50 (16%) mainstream primary schools with hearing-impaired units

• one of the five (20%) colleges with deaf services

• one of the five (20%) specialist schools for children with complex needs (residential school)

• two mainstream primary schools without hearing-impaired units.

The main reasons for not wishing to take part included (1) not having a child or young person whose first or main language was BSL (e.g. the school reported their deaf young people used oral English); (2) that it would be too demanding for the staff; (3) that the school was too busy at the end of the school year (some were contacted in June/July); (4) not wanting to take a child out of their class; (5) having no deaf children within the age range; and (6) that there were no deaf children in their school any more.

Of the 144 deaf young people who completed the self-report BSL SDQ, 56% were male (n = 81) and 44% were female (n = 63) (Table 1).

As shown in Figure 3, the young people were aged between 11 and 16 years.

FIGURE 3.

Number of deaf young people in each age group completing the self-report SDQ.

The majority of the young people (87, 60%) were white British. Twelve (8.3%) were white non-British; 26 (18%) were Asian, mainly of Pakistani (n = 16) or Bangladeshi origin (n = 6); 8 (6%) regarded their ethnicity as black (black British, black African, black Caribbean or black other) and 7 (5%) as mixed. Four did not declare their ethnicity.

The majority of the young people (115, 80%) described themselves as profoundly deaf, 16 (11%) as severely deaf, 7 (5%) as moderately deaf and 6 (4%) were not certain or did not declare.

Approximately one-third of the young people (50, 35%) used cochlear implants, 66 (45.8%) used behind-the-ear hearing aids, 14 (10%) indicated that they did not use either and were choosing not to wear hearing aids and 7 (5%) stated that they were unable to wear hearing aids. Seven (5%) did not answer the question.

One-quarter (37, 26%) had at least one deaf parent, with 21 (15%) having two deaf parents.

Overall, 30 (21%) described their family as a deaf family with BSL as their main language, 25 (17%) said that they were from a hearing family that was fluent in BSL, 53 (37%) were from a hearing family that knew some BSL and 30 (21%) described their family as hearing and oral in its use of language with them learning and using BSL at school and/or in the community. Six young people (4%) were not clear about what language their family usually used.

Two-thirds of young people (93, 66%) said that they had no learning difficulties or disability, while 28 (19%) declared at least some learning difficulties. The remainder (23, 15%) did not declare whether or not they had learning difficulties.

All participants understood that the responses were confidential and gave fully informed consent.

Scores and validation of the self-report Strengths and Difficulties Questionnaire

Validation of parent version of the British Sign Language Strengths and Difficulties Questionnaire

The validation of the deaf parent SDQ is based on 191 respondents.

Validation of teacher version

There was a total of 77 deaf teaching staff. Of these, 27 were deaf teachers of the deaf and 50 were deaf teaching assistants.

Inter-rater agreement

Multi-informant Strengths and Difficulties Questionnaire algorithm

Table 44 shows the overall prediction of child psychiatric diagnoses from the symptom and impact scores derived from the existing SDQ multi-informant algorithm (see Appendix 1).

The following table shows the findings for the 144 deaf children. Overall, 37% were rated as having a probable psychiatric disorder. This was much lower in the community (26%) than in the NDCAMHS group (81%).

Rasch model

Exploratory factor analysis

An EFA found that, while a five-factor solution satisfied an RMSEA value of < 0.06, the pattern of factors varied between parents, teachers and children, and that only the pro-social and emotional domains, as originally specified, emerged as valid domains for parents and teachers, and for parents and children, respectively. The peer domain failed to emerge in its original format within any group.

Relaxing the criteria of RMSEA to < 0.1 in the EFA, in order to identify a potential set of unidimensional items that can be considered by the Rasch analysis, found a two-factor solution which showed the pro-social set of items loading on one factor, and all other items on the second factor (RMSEA 0.092). This gives some credence to the possibility of a total score from the remaining four domains.

The data were then fitted to the Rasch measurement model. Initially (Table 48, analysis 1) the data for the total SDQ (20 items) showed considerable misfit to model expectations. Furthermore, there was a breach of the unidimensionality and local independence assumptions. Upon examination (of the residual correlations) it was evident that locally dependent items clustered within the four domains. Consequently, the data were resolved as four testlets, and then showed good unidimensionality but still some misfit to the model expectations (see Table 48, analysis 2). The value A is a proportion of the common (non-error) variance. The bifactor solution showed that 89% of the non-error variance was common when summating the four domains into a total score; thus, 11% of the variance had to be discarded to obtain the unidimensional latent estimate.

TABLE 48 - Rasch analysis (EFA)

Bonferroni adjusted.

Not available with split items.

Note

The value A is a proportion of the common (non-error) variance.

No.	Description	Group	Chi-square	Degrees of freedom	p	Residual item		Residual person		Person Separation Index reliability	% tests > 5%	95% CI	A	n
						Mean	SD	Mean	SD
1	SDQ-20	All	701.84	180	< 0.001	0.304	2.36	−0.155	1.18	0.80	14.3	12.8 to 15.8	–	912
2	SDQ-20–4 domain	All	80.07	36	< 0.001	0.228	1.75	−0.358	1.01	0.72	3.1	1.7 to 4.5	0.89	912
3	SDQ20–4 domain split	All	92.73	72	0.051	−0.102	1.22	−0.404	1.00	0.71	–b	–	–	–
4	SDQ20–4 domain	Baseline	55.22	36	0.021	−0.101	1.80	−0.402	1.05	0.70	3.3	1.6 to 5.0	0.87	668
5	SDQ20–4 domain	Parent baseline	36.24	36	0.457	−0.111	1.32	−0.394	1.04	0.68	3.4	0.8 to 6.0	–	280
6	SDQ20–4 domain	Teacher baseline	14.87	12	0.248	−0.239	0.42	−0.398	1.20	0.58	2.7	0.5 to 5.8	–	193
7	SDQ20–4 domain	Child baseline	19.98	12	0.067	0.294	1.05	−0.319	0.99	0.66	3.7	0.5 to 6.6	–	195
8	Pro-social	Child	17.32	10	0.068	0.160	1.03	−0.311	1.47	0.23	–	–	–	165
	Ideal values				> 0.05 a		< 1.4		< 1.4	> 0.70	< 5.0	LCI < 5.0

The potential source of misfit was investigated further. Given the way in which response options are interpreted within this BSL framework, for example by judging facial expression, which may represent a continuous expression of emotional intensity, the consistency of the scoring of each item may have potential to create disordering of response thresholds, where the increase in the score of any particular item may not be reflected by an increase in the underlying trait of difficulties. This inconsistency is known as threshold disordering and may contribute to misfit. This was not the case for the SDQ, where, out of the 20 items which form the total score, only two items failed to make the transition from one category to the next in a way which reflected the increase in the underlying trait of difficulty, and this disordering was only marginal and not statistically significant. This confirms that the interpretation of magnitude of any particular item is working as expected, where an increase in score on the item reflects an increase in the underlying trait of difficulties.

Differential item functioning was observed for several domains. For example, there was significant DIF by gender on the emotional domain (Figure 4). Thus, at any level of difficulties, females were more likely to score higher on the emotion domain than were males. Likewise, teachers were likely to rate children lower than others on the conduct domain. There was also significant DIF by time on the peer domain. All the sources of DIF were subsequently accounted for by splitting domains such that, for example, two emotional domains were created, one for each gender. This resolved misfit to the model (see Table 48, analysis 3). A comparison of those who completed the SDQ with and without BSL (parent, teacher and self-report) showed that DIF by BSL was largely absent. Only for the self-completed SDQ for the peer domain did a difference occur. Those using BSL reported a higher score on this domain.

FIGURE 4.

Differential item functioning for emotional symptoms by gender.

Figure 5 shows the person–item distribution for the total sample. The total sample was then disaggregated into various groups to obtain a more in-depth understanding of how the scale worked, and where some of the problems may be found. All such analyses were undertaken with the testlet (four domains) approach. The baseline data, and each of the responder groups at baseline, showed good fit to model expectations (see Table 48, analyses 4–7). However, in each case, reliability was below that acceptable for group use. DIF was found by gender (of child) for both parental and self-report and there was some DIF by location for parental report.

FIGURE 5.

Person–item distribution of total sample (n = 971).

For completeness, the items within the pro-social subscale were examined. Only the self-report satisfied Rasch model expectations but the reliability was extremely low, and thus the scale was of little use.

Factor analysis

The original authors report a five-factor structure113 and some other studies use CFA to support this. 187 In a sample of 1686 parent-rated translated SDQs in Germany188 from a mixed community and clinical sample, principal components analysis extracts five factors accounting for 53.9% of the variance. Our study had similar findings, with a five-factor model accounting for 55% of the variance for the teacher version, 50% for the parent version and 48% for the self-report version. Some studies have suggested some modifications189 to the five-factor structure. 190 A Hebrew version finds five factors but in both the parent and the self-report version the peer problems factor disappears, with the five items going into four different factors on each and the hyperactivity and inattention factor splitting into two. 191 Five factors are also found in the Russian version but with low enough factor loadings for the authors to suggest that inadequate psychometric characteristics make it ‘unsatisfactory’ for use in Russian adolescents. 192 This is a problem not only for translated versions. A large US cohort found a three-factor structure was a better fit for the self-report version of the English SDQ,192 and that factor loadings and subscale reliabilities were low for the five-subscale version. A group considering the US self-report SDQ suggested a three-factor solution based on pro-social behaviours, internalising problems and externalising problems,193 and some similarities to this model were found in a large Finnish study of 1458 13- to 17-year-olds where, for boys, symptoms of conduct problems, emotional problems and hyperactivity/inattention are ‘fused’ on factor analysis. 194 An Italian study with 1394 children aged 8–10 years also found that this three-factor model fitted best. 195 As an example, this three-factor model yields Cronbach’s alpha coefficients for internal consistency of 0.67 (0.63–0.69) for pro-social behaviours, 0.63 (0.60–0.66) for externalisation problems and 0.54 (0.49–0.57) for internalisation problems for the overall sample. Goodman and colleagues196 reporting on an epidemiological study of 5- to 16-year-old children in the UK suggested that a three-factor model may be useful in low-risk samples. A study comparing the self-report version in five European countries (Italy, Sweden, Cyprus, Germany and the UK) found that a three-factor model (with internalising, externalising and pro-social subscales) fitted better in Cyprus, with five factors fitting better in Germany, but fit indices for both models were ‘poor’ for the UK, Sweden and Italy. 197 They suggested that removing the inversely expressed items improved the goodness of fit. The internal consistency and validity of the SDQ total difficulties score were good for most countries, and this accords with our findings.

Our factor analysis suggests that a good use of the BSL SDQ is as a screening tool to identify young people with mental health problems with sensitivity and specificity both above 70% when using the upper threshold on the multi-informant algorithm. Looking at concurrent validity, the BSL SDQ versions were able to identify attachment and conduct disorders in all three versions, ADHD in parent and teacher versions and anxiety disorders in child and parent versions.

In summary, the psychometrics for the BSL version of the SDQ are not dissimilar to other translated versions.

There are several reasons why differences in factor structure from the original version may exist. It has been proposed that the reverse items in the SDQ may be confusing for some and that the psychometrics improve when they are removed. 197 We should always consider the possibility that a translation from the written form of an oral/written language to a visual language carries with it complexities that make accurate translation of meaning more difficult, as we have discussed at length in Chapter 2, Stage 1. However, our findings are not very different from other written to written translation studies reported and the variance of factor solutions are very similar (as described in Factor analysis above).

Deaf cultural norms are significantly different from spoken English. 148 This means that there may be different expectations from deaf adults regarding the behaviour of children and young people, with respect to what is ‘normal’. Examples of this were seen within the clinical validation interviews, where parents reported on emotional symptoms that levels of social confidence/worries depended very much on the environment, and they found the question difficult to answer. They reported that the child could present very differently in a Deaf environment from a hearing one. The same might be true for some of the conduct subscale items. It is our clinical experience that a lot of behavioural presentations are in fact due to underlying communication difficulties for a deaf child in a hearing environment, rather than a primary behaviour disorder.

Other studies show strong associations between different factors, for example between parent-rated conduct and hyperactivity/inattention191,198 and between conduct and emotional scores191 in Hebrew-speaking children. Another example of this was a study in Norway that found that, across ethnic groups, the correlation coefficients between the hyperactivity/inattention factor and the conduct problems factor were high (range 0.76–0.92), suggesting that they did not reliably represent distinct constructs in adolescents. 199 In our findings, the fact that emotion and conduct items load together in both the self-report and parent BSL SDQs suggests a complex relationship between emotional problems/anxiety and behaviours associated with frustration such as temper tantrums. The finding that theory of mind is delayed in many deaf children82 may be related, as empathy delays can be associated with emotional, social and behavioural problems. Put another way, empathy problems may lead to internalising problems in some children and externalising problems in others. This could be explored in further research.

Rasch factor analysis

Using the Rasch analysis, the CFA failed to support the five-factor structure of the SDQ, as well as the four-domain-based total score. An initial EFA failed to derive any meaningful structure that was consistent across the parent–teacher–child response structures. Various reports on the SDQ based upon this approach have been published with different degrees of support for the original structure, sometimes arguing for a three- or two-domain structure. 192,194,197,200,201 Furthermore, there are suggestions that the scale lacks cultural invariance, working differently in different countries, even where the language is the same. 193 The finding in the current study that the structure varies by parent–teacher–child completion has also been reported elsewhere. 189 The various issues arising from analysis of the scale have led some to argue that the SDQ total score should be used in preference to the various subscale scores. 199 Consequently, the findings in the current study are not atypical and may not be a function of the novel approach of delivering the scale through BSL.

Some of the reasons for that lack of stability may have emerged in the Rasch analysis. Other than substantive breaches of the local independence assumption, dealt with by creating testlets, item bias emerged in the DIF tests. Thus, the emotional domain needed to be split for gender, and also there were some differences in the way in which teachers assessed their pupils, and also between the baseline and follow-up study in the test–retest component. All these biases can be accommodated within the Rasch measurement framework through the item split technique, and thus unbiased latent estimates can be obtained which provide valid ways to compare, for example, gender or test–retest. Nevertheless, outside this framework these biases may contribute to the lack of stability of the scale. Furthermore, the use of small subscales with cut-points where gender bias is present may lead to misinference about diagnostic allocation, where, for example, girls are more likely to achieve an above cut-point score on the emotional domain than boys, at the same level of overall difficulties. The absence of DIF for completion by BSL confirms the success of the adaptation procedure and the comparability of the BSL version with the original. Only the self-completed version for the peer domain showed a significant (higher) difference in response at any level of the trait.

Another concern is the level of reliability reported in this study, and in others. 113,202 Where clinical cut-points are used, which can be viewed as high stakes, reliability should be in excess of 0.9. 203 These levels are never achieved for this scale, and some reports have subscales well below that required even for group comparisons.

Given fit to the Rasch model, then, the raw score of the SDQ-20 (total score) is a sufficient statistic to estimate magnitude of difficulties. It is valid for each group (parents, teachers and self-completion) independently and when pooled together after adjustment for item bias (e.g. the conduct domain by teacher). Consequently, the differences in estimates of difficulties between the different assessors can be evaluated without the risk of bias caused by the DIF (in this case domain functioning). Thus, all assessments can be placed on the same unbiased metric.

The low reliability may be a function of the sample having similar levels of difficulties. Figure 2 shows that the majority of assessments allocate subjects into a range of just one logit (between –1 and 0 logits). This typically reduces reliability, as the scale cannot separate out this large group over this narrow range. This is commonly seen in other samples (e.g. undergraduate examinations) where all the respondents are clustered together over a very narrow range of marks. Thus, it is possible that the reliability may be depressed because of this concentration of location.

As BSL is accessible only through someone’s production, as it has no written form, prosodic features such as speed and intensity of articulation and facial expression are necessarily present. In written language, such features are not present (except as, for example, punctuation marks or writing style), and so a written questionnaire can present as more neutral, as it has no inherent ‘tone of voice’. Using the Rasch analysis we were able to explore if there was any disordering of response thresholds, where the increase in the score of any particular item may not be reflected by the increase in the underlying trait of difficulties. This inconsistency is known as threshold disordering. This was not the case for the SDQ, where, out of the 20 items which form the total score, only two items failed to make the transition from one category to the next in a way which reflected the increase in the underlying trait of difficulty, and this was not significant. This suggests that respondents were able to see questions as distinct and an increase in score on the item reflected an increase in the underlying trait of difficulties.

Subscales and internal consistency

Several studies report specific subscales being below acceptable limits in the specific psychometrics (Cronbach’s alpha coefficients). 202,204 It is common in studies from different countries for items to load on to different subscales from the ones intended from the original. For example, in the Spanish teacher version205 two of the hyperactivity/inattention items load better onto the conduct subscale and two of the items for the peer problems scale load better onto the conduct problems subscale. Similar problems are found with the parents subscale. In the Italian version195 Cronbach’s alpha coefficients (internal consistency/reliability) improve when certain poorly performing items are removed. Such studies discuss possible problems with translation, subtle meaning or cultural interpretation of behaviour. Five items show factor loadings less than 0.35 and three of these items are from the peer subscale. Other studies from other languages and cultures find similar problems with loading of items. 188,206,207

The original reports of the English SDQ described Cronbach’s alpha scores of 0.73 for the total difficulty scale. 113 This study had total difficulty Cronbach’s alphas of 0.735 for the self-report version, 0.842 for the parent version and 0.738 for the teacher version, which are not dissimilar. The mean and range from a review of 26 studies208 in various languages for parents showed 0.67 (0.54–0.84) for pro-social behaviour, 0.76 (0.58–0.85) for hyperactivity/inattention, 0.66 (0.46–0.76) for emotional problems, 0.58 (0.46–0.76) for conduct problems, 0.53 (0.30–0.76) for peer problems and 0.80 (0.69–0.87) for total difficulties. For teachers, weighted mean internal consistency and ranges are 0.82 (0.79–0.86) for pro-social behaviours, 0.83 (0.66–0.89) for hyperactivity/inattention, 0.73 (0.63–0.80) for emotional problems, 0.70 (0.63–0.84) for conduct problems, 0.63 (0.35–0.77) for peer problems and 0.82 (0.62–0.85) for total difficulties. In a more recent study of five European countries,197 self-report Cronbach’s alphas for some subscales for conduct disorders were 0.45 for German 12- to 17-year-olds but 0.12 in Cyprus and 0.02 in Sweden. Similarly the peer problems subscale showed coefficients of 0.54 in Germany and 0.5 in Italy, but 0.15 in Cyprus. In the Spanish version for teachers, the subscale Cronbach’s alphas range from 0.64 to 0.85 and in the parent version from 0.58 to 0.77. 205 In other translated versions, the internal consistency Cronbach’s alpha coefficient is very low,206 ranging between 0.30 and 0.83. For subscales, only parent and teacher hyperactivity/inattention and teacher pro-social subscales had Cronbach’s alpha greater than 0.7 in this study. Again, this is not dissimilar from other studies. In the Israeli version only the total difficulties scale and the hyperactivity/inattention subscale for parent and adolescent had Cronbach’s alpha scores of 0.7 or above. 191 The Russian self-report version finds Cronbach’s alpha of 0.7 or above in only the emotional subscale. 192 The remaining subscales have scores ranging from 0.44 to 0.60. The Swedish version finds scores between 0.51 and 0.76, with the latter the highest score for the total score. 209 Other studies also show low reliability. 194,198,210 One study suggests that one of the problems is that none of the subscales are truly unidimensional. 190 Di Riso and colleagues195 suggest that instruments should be striving for very high Cronbach’s alphas of 0.90 or above. Few studies achieve this and our Cronbach’s alpha scores are broadly in the ranges of other translation studies.

Cultural differences in SDQ research

In the current study the mean female self-report scores were higher for all subscales and the total score than those for UK children. 162 For example, the UK mean total score for 11- to 16-year-olds is 10 and our study found 15.5 in the community sample. The male scores were similar between Deaf and UK mean scores, except that both males and females had higher peer problem scores than the UK young people. It is not uncommon to find differences between cultures such as studies reporting differences between UK children and Arab,211 Norwegian212 or Russian192 children. The Norwegian study suggested under-reporting of SDQ emotional symptoms by Norwegian adults (compared with British adults) in those children with emotional disorders. Parents of deaf children appear to report higher rates of problems than their teacher or self-report equivalents.

Compared with British hearing children,1 Dutch,204 Norwegian212 and Hebrew children191 have lower mean scores on emotional subscales, while the Russian teacher version reports higher means for emotional problems than the UK sample. 213 Deaf girls in particular seem to have higher rates of emotional problems in both our community and clinical samples. Any cultural differences could be related to differences in actual morbidity, attributions, levels of tolerance, expectation/fear of being judged in differing societies or other factors. This has yet to be more comprehensively researched.

One study that explored an ethnic minority (Pakistani) in its country (Norway) found that, while questionnaire items showed loadings above 0.4 on all the subscales, six items were below 0.30 in the Pakistani group199 when using the same Norwegian version. There was no apparent difference between first- and second-generation ethnic minority responses, suggesting the language was not solely responsible for problems with the subscales. This is a clue that cultural factors are relevant.

Some authors suggest real cultural differences in behaviours, or language differences or response style differences. 199 There may also be some kind of measurement artefact214 related, for example, to social desirability and openness about symptoms or questions. Translation issues discussed in Chapter 2 have also been highlighted by others, including the non-equivalence of translated items. 214

Other authors have made reference to subtle differences in meaning or severity when words or phrases are translated in different languages in different cultures. 191,214 Response styles may also vary across cultures. 197

Gender

In this study we found that girls in both community and clinical samples scored higher on the total difficulties scale when scored by teachers or parents. This is similar to Finnish,194 Hebrew191 and Dutch202 studies showing higher informant total scores for girls, although the French214 and Chinese206 translations had higher scores for boys. The Italian study of 8- to 10-year-olds shows girls and boys roughly equivalent for total difficulties but girls higher for emotional problems and on the pro-social subscale195 and this study also found that girls had higher scores on the emotional problems and pro-social subscales. Most studies score boys higher for hyperactivity/inattention and girls higher for pro-social behaviour, but our sample showed no statistically significant difference between genders. As discussed above, these findings suggest that deaf girls struggle more than boys with emotional problems in both the community and clinical samples.

Test–retest reliability

In this study the test–retest reliability ICC is 0.830 for the total difficulties scores, and for the subscale scores ranges between 0.664 and 0.831. Interestingly the best ICC, of 0.831, is for the pro-social subscale. This may reflect the fact that deaf teaching staff know their students better and can observe pro-social activity on a regular basis. Broadly speaking, test–retest results are equivalent to those in the original and other versions of the SDQ.

The original study reporting the UK version of the SDQ reported test–retest reliability of 0.62. 113 The lowest mean from six studies of informants for 4- to 10-year-old children208 is 0.65 for parent-rated pro-social behaviour, with means of 0.66 for the emotional, conduct and peer problems scales. Teacher test–retest correlations are better, with the means for different subscales between 0.72–0.85. Some studies have shown low test–retest reliability, with for example a range of 0.40–0.79 in the Chinese study. 206 It found ICCs between 0.45 and 0.71 for the self-report version, between 0.60 and 0.77 for the parent version and between 0.65 and 0.88 for the teacher version. This accords with other studies in that teacher versions have better test–retest reliability than other versions and that the self-report pro-social subscale has the lowest retest scores of the self-report subscales.

Inter-rater reliability (cross-informant correlations)

In general, Goodman113 reports good cross-informant correlations compared with other instruments. However, this study found that subscale agreement between parents and teachers ranged from 0.214 to 0.531, with 0.513 for the total score and 0.317 for the impact score. The weighted mean correlations between subscales for parent and teacher versions in a review of eight studies of 4- to 12-year-olds ranged from 0.26 to 0.47. 208 Other studies looking at inter-rater reliability correlations also found similar ranges, such as 0.23–0.49 reported for subscales of the Chinese version of the SDQ. 206

This study found similar paired sample correlations between 0.175 (hyperactivity subscale) and 0.264 (emotional subscale) for the deaf child and parent comparisons and between 0.150 (pro-social behaviour) and 0.294 (emotional subscale) for the deaf child and teacher comparisons. Deaf parent comparisons with self-report versions range from 0.039 (pro-social subscale) to 0.560 (peer subscale) and with teacher versions range from 0.167 (pro-social subscale) and 0.450 (conduct subscale). Our study found 0.039–0.560 for deaf parent subscale comparisons with self-report and 0.130–0.341 for deaf teacher comparisons with self-report. All of these are very similar, suggesting that our inter-rater agreement statistics are very similar to those of a range of other cross-cultural translations of the SDQ. The pro-social subscale had poor inter-rater reliability. This is not dissimilar to other studies but could also point to a lack of awareness of parents and teachers about the experiences of deaf young people.

For the SDQ self-report version, the only subscale that showed significant difference from the parent SDQ was the hyperactivity subscale. Where parents scored significantly higher, this may be because of differences in awareness, attitudes, expectations or fund of knowledge issues, or it may be because there are cultural differences in behaviour. Young people may regard their behaviour as less problematic or more acceptable and not have wider groups of peers to compare against. This hypothesis may be strengthened by the finding that, for the deaf parent SDQ, there were significant differences from the young person self-report SDQ on emotional, conduct and peer subscales, with parents scoring higher on those subscales. They also scored higher than the teachers in all subscales other than pro-social scale.

The parents consistently scored higher than the teachers.

This raises some interesting questions about how deaf parents perceive their children (both deaf and hearing offspring), and also how deaf young people are perceived. It may be a cultural phenomenon, or linked to fund of knowledge deficits, or related to relative social isolation, or changes in expectations, awareness or attitudes. We are unable to be certain, but this is an interesting area for future research.

Validity and discriminative power in identifying a clinical sample

There were significant differences between the clinical and community samples for total score and for the emotional and conduct subscales.

The standard multi-informant SDQ algorithm for generating a final score for caseness found that 81% of the clinical samples were identified and 26% of the community samples were identified. When using this algorithm against the gold standard clinical assessment we found good sensitivity (81%) and low specificity (42%) when using ‘probable’ or ‘possible’ caseness as a threshold, and good sensitivity (76%) and good specificity (73%) when using only ‘probable’ caseness as a threshold. Both of these have better sensitivities than the original UK studies but lower specificities. This means higher numbers of false positives if this tool is used as a screening instrument.

As discussed through the report, our hypothesis about this would be relating it to the clinical deaf population. It is our clinical experience, for example, that deaf young people referred for a behaviour problem often have underlying emotional, developmental or communication disorders.

The SDQ has been translated into over 60 languages. There are therefore numerous papers describing validation. Some examples include research from Germany,188 Israel,191 Russia192 and the Islamic Republic of Iran. 215 Many of these do not use gold standards to validate the child versions but may use other questionnaire-based instruments such as the Child Behaviour Checklist. 188 Some studies explore cross-informant agreement between different versions of the SDQ, for example comparing the child version with parent or teacher versions. 188,215 Some use semistructured clinical interviews such as the Development and Well-Being Assessment (DAWBA) inventory,191 while some, like our study, have used clinical interviews. 215

We chose to use clinical assessment to make it ecologically valid but also because there are no other gold standards available for validation in deaf children, meaning we had limited options.

Recruitment within the Deaf community

The process of recruitment was difficult for a variety of reasons. In the UK, the Deaf community is a close-knit group. The most recent prevalence rates for BSL users in England estimates that there are approximately 125,000 deaf adults who use BSL as their main form of communication. 222 The 2012 Consortium for Research into Deaf Education survey51 suggests that only 5–8% of deaf young people are BSL users; the others rely on spoken English or multimodal forms of communication, such as SSE. SSE is not in itself a language and, although the translators who developed the BSL SDQ were keen to be inclusive, the team were conscious that they were developing a signed language version of the SDQ, available in full BSL. Around 90% of deaf young people are born to hearing families, and therefore struggle to acquire language at the same rate as their hearing peers, because the necessary conditions for language acquisition are less likely to be met.

There were concerns from some members of the Deaf community about confidentiality, which in some cases seemed to be a barrier to participation. Although reassurances were given, some families struggled with the concept and clearly felt uncomfortable with who knew what about their child, particularly in regard to the necessity for a clinical interview; several families expressed concerns about this and questioned the need for it. There was a suspicion among many, in particular deaf parents, that the clinicians were looking for problems with their child. In these circumstances, it was highly beneficial that several of the research assistants were deaf themselves, and able to allay fears about the purpose and nature of the clinical interviews and the security of the study data.

Using online delivery

It was found that the online system was particularly useful for retesting participants. In these cases, respondents were already familiar with the research process and had experienced answering the questionnaire once on a previous occasion. However, for participants new to the research it was less beneficial, as researchers found that almost all participants involved in the study appreciated the opportunity to ask questions face to face, and discuss their involvement in the research.

There were additional drawbacks in that it was not possible to enter data offline and upload them at a later date when internet connection had been established, meaning that all participants had to have access to internet connections to make use of this system. While in large cities this was not a problem, during recruitment in the south-west and other rural areas of the country where internet access is poor it did cause problems. These stemmed from not being able to establish internet connection at all, or it not being at fast enough speeds to stream SDQ BSL video questions smoothly enough to be understood.

Equally, because the videos were hosted on the Manchester video library service (VLS), occasions when this site was closed for maintenance caused disruption to the streaming of the videos within the questionnaires. For example, at one point the VLS was completely updated over several weeks, meaning that every video had to be re-embedded within the questionnaires, as the coding had changed. As the research team was not based within the University of Manchester, the team was unaware of this for a significant period of time.

There were considerable limitations within setting up and securing the SelectSurvey database, in that there was a lack of close technical support available to the team, and it is a complex system to use and manage, with several different components. Funding was not sought for a part-time study technician, or technical support costs; this team would recommend that future studies include such funding in setting up similar projects. While it is clearly a useful piece of software that can bring many benefits to the recruitment phase of such a study, great care must be taken in the design and maintenance of such a study, and teams must plan effective ways of setting up and managing the challenges associated with using online systems in their research.

The online system was, for some participants, difficult to use without researcher contact. Participants are required to enter some basic information about themselves and their enrolment in the research (e.g. date of birth, participant number). Although these instructions are available in both BSL and English, following the research procedures has been much easier when supervised by a researcher who can help explain the process in a responsive way. This is particularly true for the younger participants in the research, who may have significant lack of access to a ‘fund of knowledge’. Thus, many of the benefits that one may expect such a system to reap became limitations. While in some senses the system was able to save considerable time and travel costs, the quality of information was poorer and participants were less easily engaged to stay involved with all the individual steps of the research process. This may have been partly a result of design and presentation, and has yielded useful information for future studies.

Non-responders and potential sample bias

While some previous, clinic-based, studies were able to obtain very high rates of parent report alongside child/young person self-report,188 other studies found high levels of non-response, particularly with the parent version. 204 This Dutch study also found (when analysing the teacher and self-report questionnaires) that the children of parents who did not return a SDQ had significantly more conduct, hyperactivity/inattention and peer problems.

We found that schools were particularly keen to become involved if they were deaf residential schools. By contrast, mainstream schools with deaf children had much lower take-up rates. This may have created a bias, but it is not possible to know the direction of this bias without further study.

Mental health problems are thought to occur in about 10% of 5- to 16-year-olds1 in the UK. Our study cannot claim to examine prevalence in a population because it is not a population sample. Our sample has come mainly from two groups. The first is from a clinical group of young people being seen by NDCAMHS (a highly specialised nationally commissioned service), and the second is from the community. Those recruited from the community have come largely through approaches to primary and secondary schools that include deaf children, but also through our network of contacts in the Deaf community and through teachers of the Deaf who see most children with deafness of 40 dB and above. There are therefore likely to be selection biases. The clinical group includes mainly children and young people who have been unable to access generic CAMHS, or who have been referred to us because of our expertise in seeing children and families whose first language is BSL. In the community setting there are likely to be a number of selection biases, and some of these have been identified in earlier sections. Biases can operate in both directions. For example, this may include culturally Deaf families avoiding the study because of perceived stigma and worried parents seeking out participation in the hope of eliciting support for their child.

Mean scores of total difficulties are reported across five different countries by Essau and colleagues. 197 Adolescents in the UK and Germany report significantly higher problems than in Sweden, Cyprus and Italy, and there are various statistically significant differences in the subscales across countries. High means (equivalent to those in Germany and the UK in this study) have previously been found in China223 and Holland. 202 These studies, alongside studies of different ethnic groups in the same country, for example Israel224 and Norway,199 suggest that it is not safe to use similar thresholds for total or subscales across different ethnic groups.

Gold standards

Even when the recruitment figures are met, there have still been issues in terms of how to validate the BSL SDQs, as there are no other validated mental health instruments (either questionnaires or semistructured interviews) designed specifically for this population. The current gold standard is a full clinical interview with a semistructured questionnaire, administered by a clinician with specialist knowledge of both Deaf culture and mental health, with a qualified interpreter used to working in child mental health services. All clinicians who administered clinical interviews as part of this study had at least BSL level 3, but used interpreters at all times. However, this has meant much greater resources have needed to be introduced to the project, as the recruitment centres cover large geographical areas, and the clinicians must be accompanied by qualified BSL/English interpreters in order to conduct the level of assessment required. The whole interview was interpreted from English to BSL (clinician to family) and then interpreted back from BSL to English (family to clinician). As the research was carried out nationwide, there were problems establishing consistency in interpreters being used; for instance, it would not be cost-effective having an interpreter based in Newcastle travelling down to London for the interview. There has been nationwide training for interpreters working with NDCAMHS, and interpreters who had received this training were used whenever possible. All clinicians had at least 5 years’ experience of working with deaf children in child mental health services, and some had considerably more. There are no deaf BSL-using clinicians working within the NDCAMHS teams who were available to undertake these interviews.

Language and linguistic challenges

One issue that is likely to be influential in the interpretation of the data concerns the highly heterogeneous nature of the language used by the deaf children who participated. This is partly related to the influence of the age of acquisition of a signed language, and also the lack of early childhood experiences of visual language. There will also be issues with the relative knowledge of certain aspects of language, such as social and emotional language. Thought was given to the practicalities of assessing the children’s language level using standardised tools. However, given the extreme shortage of experienced language therapists who have experience and knowledge of sign language, this was not possible. It is also not likely to be possible in practice, for example in population screening. The researchers noted the highly heterogeneous life histories of the children who participated. They differed not only in age of acquisition of a sign language, but also in the extent to which they could be considered bilingual. This is not simply an issue about language competence; some aspects of language (emotions, attitudes, etc.) may be more readily expressed in one language than another, depending on the child’s experiences. We had no way of controlling for this, but it remains an important issue for future research. Some mechanism for controlling language experiences and competencies would yield richer data.

Deaf researchers

One advantage of our research team was the presence of five Deaf researchers (three research assistants, one collaborator and one clinical studies officer) who were an integrated part of the team. They could communicate one to one with deaf children and young people as well as deaf teachers and parents, and this meant direct communication was possible without the need for an interpreter. Both deaf and hearing researchers supported each other and brought a range of skills into the team. Deaf researchers also made high-quality links with the Deaf community and had great credibility in environments such as Deaf youth clubs, where they were seen as positive role models. Similarly, many hearing families valued contact with hearing and Deaf researchers who often visited in pairs. The management of the study was highly collaborative.

There were some important issues around confidentiality. Some Deaf communities are small, and Deaf members of the research team often knew deaf families who were participating. We took special care to be clear about confidentiality with participants and to have high standards of information governance.

A positive outcome to one visit was noted, as one of the children involved in the research was visibly affected by the exposure to a Deaf professional being respected among hearing professionals.

Working with interpreters

British Sign Language interpretation was also vital in the management of the research project. While the main research centre was in York, Deaf research assistants in other areas of the country working independently from the clinical NDCAMHS staff were given regular supervision. BSL interpreters facilitated a number of telephone calls, face-to-face supervisions and steering groups throughout the entirety of the study, aiming to ensure good communication among all members of the research team.

There were, however, certain barriers when working within different NHS trusts, which sometimes made communication difficult for all researchers, as the use of video-conferencing facilities such as Skype and FaceTime is not allowed between some NHS trusts. This meant that communication was often confined to e-mail, or telephone calls only when BSL interpreters were available. This raises some important learning points for future studies, in thinking about how such teams should be organised. It was felt at times that it would have been considerably more beneficial if everyone had shared a base and travelled across the country as needed. This facilitates much better communication in a mixed hearing/deaf environment, but also presents a greater burden of travel times and costs.

Sociocultural issues

Most of the young people who filled in the self-report questionnaire were profoundly deaf (79.9%). This is as expected, as children with mild to moderate deafness often attend mainstream schools and choose oral communication over BSL. We did, however, have some moderately and severely deaf young people who regarded themselves as fluent in BSL. Some of these young people were in deaf families, had close associations with other deaf young people who used BSL fluently, or were in schools where BSL was used regularly.

Deafness presents in many different ways and it is not possible to define all deaf people as one group. 225 The Deaf community is important to many deaf people, especially the 5–10% born to a deaf parent or parents. Deaf parents can be strong positive role models, which benefits their children. 226 Deaf culture refers to a sense of belonging and shared values that communities experience, where sign language is central. 227 Deaf is spelt with a capital D by some to signify cultural identity, and is not necessarily related to level of hearing loss. Many deaf people are sensitised to power imbalances between deaf and hearing people. Many dislike terms such as ‘hearing impairment’, ‘hearing disability’ or even ‘hearing problems’, seeing these terms as pejorative. It has been suggested that mental health difficulties in deaf people are not due to being deaf per se, but are largely to do with living in an environment that is not deaf aware or deaf friendly. 15

Deaf parents usually bring children up within the Deaf culture. Deaf children in hearing families face a range of experiences, ranging from a bicultural/bilingual upbringing to those who avoid Deaf culture and choose spoken language as the main mode of communication that they pursue for their child. 228 This may change when the child becomes older and makes choices of their own.

Focus groups

As part of the PPI, participants from the focus groups were given opportunities to comment on the processes and materials.

Steering groups

Co-applicants from voluntary organisations such as the NDCS and BATOD were regularly invited to steering groups, both in the development phase and throughout the running of the study. They were able to provide several important contributions, particularly with respect to the recruitment phase of the study. The NDCS in particular gave valuable support in accessing their groups, providing information about the feasibility of recruitment in terms of areas where there were particularly strong BSL groups, and the idea of giving a small reimbursement to participants for taking part. All these ideas were eventually taken forward and integrated into the study protocol in time for the recruitment phase of the research.

Deaf service advisors

The NDCAMHS employs several Deaf service consultants, whose job it is to provide support to the clinical teams in making sure that the service is accessible, and that clinical and strategic developments are considered from a Deaf perspective. These professionals were able to provide regional and national support to the research assistants, particularly with regards to recruiting community sample participants. As well-connected members of the Deaf community, they were able to facilitate access to the Deaf community as well as a multitude of deaf professional contacts, who in turn were able to promote the study throughout England. This proved extremely valuable in terms of recruitment, as people were much more willing to engage when approached through a previously known contact. The value of having strong professional networks should not be underestimated when conducting research within tight-knit communities such as the Deaf community.

Web-based screening and surveys

Web-based surveys are much cheaper than equivalent face-to-face surveys,229 but the questionnaire is completed in full less often, leading to concerns about prevalence estimates of psychopathology using this methodology. 229

Feedback from deaf parents

Many parents reported problems with question number 1. They were unclear whether the placement meant that the child’s or other people’s views needed to be considered. Another question was about being ‘unhappy’. The presentation of the sign in this was reported to be ambiguous, and so parents were unsure whether it referred to being happy or unhappy. However, other parents thought that it was a good presentation of the information, and one parent reported that it had made her think twice about her child, who was already known to CAMHS. Deaf parents reported that they sometimes felt questions were difficult to answer, as their answers would be different depending on whether their child was in a hearing or Deaf environment. Deaf parents also commented that they appreciated the presentation in BSL, but they would have greatly appreciated more examples.

Feedback from deaf teachers

There was a mixed response in terms of feedback from the teachers. Some reported that the SDQ in BSL was very clear and understandable. Some stated that the structure was not clear; for example, in the first question it was unclear whether the child was to think of others or it was asking what others thought about the child. Others reported that they would have liked more pronounced lip patterns, as, when a verb and adverb were created in one sign, for example to ‘WALK-QUICKLY’, it was ambiguous exactly what adverb the verb was conveying.

One researcher noted a general response to their arrival within a mainstream context as being that of surprise from the young people that a professional visiting the school would be a deaf person. This was noteworthy to the researcher, as they in turn were surprised to find this response in 2013. It appeared to be a negative response to encounter, but alternatively it was encouraging for the researcher to see that their presence did offer the opportunity for the young people to observe for themselves a potential role model in a deaf person who was a professional, and to gain a sense that this was indeed possible.

Feedback from deaf young people

Many deaf young people reported problems with questions 2 and 10. They stated that they were unsure what the person was referring to, and it would have been helpful to have more lip pattern to help identify this. For example, a BSL mouth pattern was used in a question about stealing, but some of the young people were unsure if this then meant taking and if so taking what, leading to confusion in some young people. In addition, they stated that they would have liked to see more examples being used to help create the concept in a way they clearly understood. A number reported that they felt it was difficult to connect with the presenter because of the style of presenting in BSL that they had adopted. Some mentioned that they found the regional variation a problem. Others, however, stated that they felt the presentation was clearly understandable. In summary, the feedback was very mixed, with many finding the BSL version straightforward to understand.

Priority 1

Use the SDQ BSL version alongside other validated versions of the SDQ to scope the mental health needs of deaf children in England.

This study has shown that the BSL SDQ is an adequate screening instrument for deaf children and young people, particularly as an overall screening instrument for mental health problems.

The translation and validation of the SDQ in BSL enables comparative research to take place with deaf children who are BSL users. A research priority would be to carry out a national study of deaf children to scope the mental health needs of deaf children and to compare different subpopulations. This will help us better understand the prevalence of mental health problems and the needs of deaf children in different settings or backgrounds, and plan services to meet those needs.

Priority 2

There is a need to develop better instruments for deaf children from basic principles (i.e. not translated versions). These need to be designed specifically for deaf children and be culturally and linguistically appropriate.

Anxiety disorders, behaviour problems230 and low self-esteem231 have been proposed as more common in deaf children, and the development of new instruments for deaf young people, through research, would enable us to explore this further. A priority therefore is to develop further instruments specifically for deaf children for particular disorders. These could focus around depression, quality of life and anxiety. It would allow better monitoring of interventions for this group.

Priority 3

We need to understand better why it is that deaf children have higher rates of mental health problems and to develop early interventions that prevent children who are deaf having mental health problems.

Further research should take place to understand the high rates of mental health problems evident in deaf young people found in this and other research. We need to urgently research why this increase continues against a backdrop of changes in education policy, newborn screening and technological advances (e.g. cochlear implantation). Qualitative research with deaf children, teachers of the deaf and parents will help us understand the experiences of deaf children and the impact of a range of factors on their mental health.

A large follow-up cohort of deaf children would be helpful. This would be methodologically robust with rich information across a range of ages and domains. This could include detailed epidemiological information and birth-related data as well as social, family, communicative and educational information. This would help us better understand associations with mental health problems. From this it would be possible to develop early intervention research including clinical trials (e.g. parenting interventions) and policy-based research (e.g. educational interventions) that seeks to improve child mental health outcomes for deaf children.