Interventions designed to improve therapeutic communications between black and minority ethnic people and professionals working in psychiatric services: a systematic review of the evidence for their effectiveness

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 10/141/02. The contractual start date was in February 2012. The draft report began editorial review in November 2013 and was accepted for publication in March 2014. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Professor Bhui is Director of Master of Science programmes in mental health including transcultural mental health care. Professor Weich is a member of the commissioning panel for the National Institute for Health Research Health Technology Assessment programme.

Copyright statement

© Queen’s Printer and Controller of HMSO 2015. This work was produced by Bhui et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Culture and communication

The ability of mental health service staff to communicate effectively with any particular patient, and in a manner appropriate to that person’s culture, underpins successful diagnosis and therapy. The proposed value of understanding and evaluating communications is that they improve therapeutic outcomes through multiple mechanisms, hence the term therapeutic communications (TCs). For example, patients with a mental illness who cannot speak good (or perhaps any) English are likely to experience anxiety and uncertainty during assessment; moreover, diagnosis and clinical decision-making will be hampered. The use of family or friends as interpreters to address this issue is often inappropriate, and may undermine the clinical assessment. Instead, the use of bilingual professionals or interpreters with expertise in mental health is recommended. 8–10 Problems arise from language differences and the interconnection between language and concepts of health and illness (often called explanatory models or illness perceptions) that are used to communicate distress. These explanatory models vary across cultures, and this makes it difficult for the patient and professional to gather the same understanding of their cultural origins, and therefore concepts of health and illness, differ. These concepts may be expressed through particular idioms or metaphors, and these may differ across languages. The use of professional interpreters or other forms of formal language support is rare, but when the spoken word is the primary source of information for diagnosis and also the format of some forms of therapy, these cultural and linguistic influences on communication have a significant impact. 11,12 It is necessary to ensure that both aspects of diversity – language and culture – are addressed. 13 However, dissatisfaction and inequalities are also prominent among anglophone migrants and other people from BME groups who speak English. 9,10 There are wider issues of communication beyond translation and interpretation, and it is these that we focus on in this review.

Overall, BME groups are more likely than the white British population to report dissatisfaction with care, to fail to engage with services, to decline treatment and to have fears for their own safety while in treatment. This may be explained not only by inherent communication problems, but also by different (cultural) assumptions about the causes and treatments of mental and emotional distress. 14 Ineffective communication because of these differences may then lead to a feeling of not being understood, omissions of important information from the clinical assessment, conflict with staff, disengagement and/or a failure to take up interventions14,15 – sometimes termed failed negotiation. This may lead to more severe and more frequent episodes of illness and, in turn, the use of coercion, which is itself associated with a higher rate of adverse incidents. Such a cycle undermines the therapeutic potential of established care practices and processes, and further burdens BME users of mental health of services. Thus, improving TCs may permit maximum benefits to be realised from existing care and services, improve safety and avoid adverse incidents in care.

Effective communication is central to psychiatric assessment, diagnosis, engagement and treatment, and, ultimately, recovery. 14,15 Effective communication has proven more difficult to achieve where there are differences in culture or language between those delivering and receiving care. 6 Of course, communication difficulties might arise in any encounter between a patient and professional owing to differences in age, gender, social status or perceived power. However, cultural differences between patient and professional increase the challenges. For example, when attending to a patient from a different culture, the following abilities of a professional will be reduced:

• the ability to identify with and empathise with the patient, and to assess that patient’s emotion accurately16–20

• the ability to understand symbolic and metaphorical language, as this varies greatly across cultures21

• the ability to understand the patient’s expectations of the health-care professional, as professional roles differ widely across countries and cultures (e.g. authoritarian versus egalitarian approaches and the preferential use of medication as opposed to ‘talking therapies’ to try to resolve emotional issues)22,23

• the ability to appreciate the differences in illness perceptions and explanatory models of the illness. 14,24

Cultural factors amplify the limitations of TCs, and are of importance because they have the potential to compound inequalities in the social determinants of illness and to perpetuate inequalities in health-care outcomes following contact with health systems. 23,25,26 Conversely, good TC can reduce the inequalities. For example, Lorenz and Chillingerian27 argued that the use of visual supports to improve communication helps to address inequalities and gender disadvantage by introducing a more ‘fair process’ of assessment. They defined a fair process as one that involves patients in a collaborative approach to the exploration of diagnostic issues and treatments, explains the rationale for decisions, sets expectations about roles and responsibilities, and implements a core plan and ongoing evaluation. Fair process opens the door to bringing patient expertise into the clinical setting and the work of developing health-care goals and strategies. Although improved TC is at the heart of fair process, the evidence base to support its professional use is scattered across a number of disciplines, each of which uses a different theoretical model. There is therefore a need to pull together the evidence on interventions that improve TCs, appraise its quality and identify the components of effective interventions.

Cultural competency

One proposed solution to the problems stemming from poor communication between mental health services and their BME patients is the training of the professional in ‘cultural competency’. 28 A review of the international literature on cultural competency suggests that it is best conceptualised as a systemic and deep-seated process of change in both organisations and professional practice. 29 It often requires a change in the attitudes of staff, and a change in the way they assess, diagnose and treat people whose expectations and perceptions of both illness and recovery differ from their own. At an organisational level, the changes required include developing values that are more welcoming of culturally diverse populations and changes in management styles and human resource practices that reflect an understanding of the influence of culture on communication. Alongside these macro-level interventions, educational solutions have included training to address the attitudes and stereotypes held by individual members of staff. However, the introduction of long-term change at an individual and organisational level has not been widely achieved in the UK, perhaps because of the complexity of the task. Short-term educational solutions have been more popular and, therefore, more widely reported in the literature. These have varied in quality and focus, with some attending to communication; some to clinical skills and practices; some to the attitudes of practitioners and their cultural biases; and some to specific groups, such as faith groups, refugees, migrants, gypsies or other racialised groups. The lack of long-term programmes and the wide variability of the short-term interventions have made the development of a robust evidence base problematic.

Some cultural competency training has included information on race equality and compliance with recruitment legislation. The Department of Health rolled out a ‘race equality and cultural competency framework’ to address stigma, race equality and cultural factors. 30 This highlighted some of the communication issues that arise with BME patients and the need for sensitivity to stereotypes according to race and culture, but had rather little to say about clinical assessment, diagnosis or specific treatment strategies. Bennett and Keating28 mapped cultural competency training and its content in the UK and concluded there was insufficient attention to clinical interventions and to racial issues; they suggested instead that non-TC issues were more often reported. A systematic review of the international literature on cultural competency interventions in mental health settings similarly found few evaluations and none of these reported patient outcomes. 29 A systematic review of TCs with BME patients is necessary to synthesise the findings across the many approaches tried, and to identify lessons for policy, practice and research. Inevitably, such a systematic review will require characterisation of a wide range of interventions in diverse contexts and for diverse populations, rather than a meta-analysis of trials of a single intervention as there appears to be no single intervention that is known to be of superior efficacy.

Narratives, ethnography and diagnosis

The meaning a person assigns to an illness may be quite different from that assigned by the health professional. 31 This issue is not confined to the UK, and it reflects fundamental differences across national, cultural, ethnic and religious groups in the way mental distress and illness are understood and defined, and is related to expectations of recovery and treatment. 32,33 If psychiatric professionals have a good appreciation of a patient’s beliefs and expectations, then that can only be beneficial. Some pioneering services have sought to facilitate such an appreciation through, for example, the use of ‘patient narratives’ (the account of how the patient came to present to the professional, and what this might mean for the formulation of the treatment plan), what has been termed ‘ethnography’ (an understanding of, and sensitivity to, a patient’s ethnic background), and negotiations of meaning (the elucidation of the patient’s beliefs concerning the cause and likely outcome of the illness and, again, the account taken to inform the diagnosis and treatment plan).

Canales25 describes ‘narrative interaction’ – that is, the sharing of personal stories – as a form of TC that permits the inequalities by gender to be addressed in nursing practice. 25 Narratives and story-telling as part of assessment and treatment have a long history in cross-cultural psychology. 34 Narratives are ways of expressing and exploring personal identity, group histories and securing group cohesion35 that can influence the onset and course of psychiatric disorders. Assessing narratives in practice and research brings together idiographic and ethnographic methods of assessment and research. 36 Narrative methods are also valuable in addressing inequalities by engaging socially excluded groups, for example teenage mothers, so that they can receive nursing interventions,37 and in understanding challenges to the implementation of policy and practice. 38,39

A more detailed assessment of patients’ illness models is advocated by some medical anthropologists as a way of improving assessment; for example, ‘mini-ethnography’ has been used in the clinical assessment. 40 Studies of this approach have demonstrated improvements in diagnostic precision, diagnostic depth and care plans. Attempts to introduce ‘ethnography’ into the diagnostic process (i.e. to increase its ethnic sensitivity) have led to support for a ‘cultural formulation’, and indeed this has been highlighted in a previous and the most recent edition of the diagnostic manual produced by the American Psychiatric Association [Diagnostic and Statistical Manual of Mental Disorders–Fifth Edition (DSM-V)]. 41–43 This advocates that assessment includes ethnography and narrative, by the psychiatrist asking questions about cultural identity and explanatory models. Explanatory models in the anthropology literature are similar to illness perceptions reported in the psychology literature, as both refer to concepts about what causes illness, what it is called, who might help in recovery and what expectations there are of potential carers. 44 In addition, the cultural formulation covers psychosocial factors, and brings the clinician’s perspective into play by openly seeking the clinician’s reflections on interpersonal interactions before reaching an overall diagnosis and narrative about the problem and its resolution (also called a formulation). Although a cultural formulation has been reported to be helpful in clinical practice, the published papers have been mainly qualitative and descriptive, such as case reports; evaluative studies seem to appear only in the grey literature, although perhaps now just emerging in the scientific literature that tests effectiveness.

Other developments in the UK have included conflict resolution and mediation45 and a cultural consultation service (CCS) that is collecting pilot data on cultural competency and organisational narratives of care and communication with BME patients. 39

Participants

We were interested in all studies that provide evidence on how to improve TC with BME psychiatric patients in the setting of specialist psychiatric care. Specialist psychiatric care is delivered by many professional disciplines. We adopt the term psychiatric services rather than mental health services as the latter is a very broad term that includes public health and social interventions in the community, for example housing and other services, that were not within the scope of the commissioned review.

Key populations included all age groups (young people, adults and the elderly) and all ethnic groups known to be prominent in health-care settings in the UK, i.e. people from Indian, Pakistani, Bangladeshi, Sri Lankan, black Caribbean, black British, black African, Irish and Chinese backgrounds. We used these terms in the search strategy but complemented them with other terms in order to identify the broadest literature of relevance to the UK. Although commissioned as a review of BME groups, our analysis specifies the groups of importance to the UK setting and reveals the ethnic and cultural specificity of interventions.

Although originally commissioned to include only UK studies of the larger ethnic groups, we amended this aspect of the protocol during the study wherever relevant as useful information might otherwise be overlooked, for example if a study included a diaspora population of relevance to the UK (e.g. east Europeans) or if an intervention was judged to be transferable to the UK (e.g. an intervention for African Americans). However, we included such studies only where the other inclusion criteria were met.

During the review we came across some studies located at the interface of specialist psychiatric care and other sectors of care, but requiring specialist input; we included those studies in the review.

Interventions

At the outset, we defined TC as:

[A]ny conversation (face-to-face or technology-assisted) that is undertaken using a pre-defined model that seeks to improve understanding, engagement and therapeutic outcomes. For communication in health care to be therapeutic, it must involve a relationship and exchange of ideas between a patient and professional helper, be patient centred and engaging in order to influence the patient’s emotional world, and directed by the professional using expertise and skill. Therapeutic communications include all interactions that enable people in distress to resolve conflicts, divergent expectations, traumatic histories and adverse life events, and to take up and to overcome distress and also take up offers of help.

Bhui et al. 46

In this review we were specifically interested in all interventions seeking to improve TC with BME patients receiving psychiatric care, and expected a broad range of such interventions, for example conflict resolution, cultural consultancy, cultural competence and others as yet undefined.

These interventions might be aimed at either individuals or populations.

Care could be delivered by psychiatrists, general practitioners (GPs), psychologists, nurses or any other professional as long as it was located in specialist psychiatric settings.

From existing knowledge of this field, interventions to promote TC include those that:

• employ mediation to enhance mutual understanding and to improve engagement with care

• seek to manage divergent views, conflict and differing explanatory models and illness perceptions through negotiation and mediation

• include narrative-based interventions (i.e. that place the service user and patient perspectives at the heart of consultation, assessment and treatment)

• employ cultural consultation as a process of gathering narratives

• apply cultural competence interventions focused on communication

• any other new method or process for improving TCs that is not captured by the above, but is suited for BME populations in psychiatric care.

Any of the above processes could be delivered face to face or through two-way real-time communication technologies (e.g. NHS Direct or other support systems, telemedicine or e-mail).

We did not review the literature on interventions that are considered to be (generic) TCs themselves, such as psychological therapies or music therapies, unless the research evidence focused on interventions that might improve TCs with the specified target group(s), while meeting the other inclusion criteria (see Box 2).

Furthermore, given the evidence base already available,13 we did not include studies that were purely testing models of translation or interpretation as an intervention. The reasons for this were twofold. First, there is already a concurrent enquiry into interpretation, translation and language support (ITALS) in mental health care;12 and, second, without a control group who are denied access to an interpreter (which itself would be deemed unethical), it is difficult to determine any effect on outcome that is produced by the interpretation as distinct from the effect of the psychiatric intervention of which the interpretation is a component.

Review procedures and processes

A systematic review was carried out in accordance with the methods outlined in guidance issued by the Centre for Reviews and Dissemination. The full peer-reviewed protocol submitted in the funding application has been published in an open access journal for scrutiny. 46 During the course of the review, the original protocol was changed in two ways, as indicated in Participants. First, we included studies from other countries of interventions that showed evidence of transferability to the UK and the ethnic groups were relevant to the UK (e.g. by being from the same diaspora), if other inclusion and exclusion criteria were met. Second, we discovered many studies that seemed not to include evaluations but only to describe an intervention; these were assessed more carefully for evaluative statements and conclusions to create a category of near misses (A– rather than A+).

Data sources and search strategy for published literature

The following databases were searched: MEDLINE, PsycINFO, EMBASE, Applied Social Sciences Index and Abstracts (ASSIA), Cochrane Database of Systematic Reviews, The Campbell Collaboration, ACP (American College of Physicians) Journal Club, The Cochrane Central Register of Controlled Trials, Cochrane Methodology Register, Allied and Complementary Medicine Database (AMED), Cumulative Index to Nursing and Allied Health Literature (CINAHL), British Nursing Index, Health Management Information Consortium, Social Science Citation Index (SSCI), Social Care Online (www.scie-socialcareonline.org.uk) and NHS Evidence collection on ethnicity and health. We also searched university databases for Doctor of Philosophy (PhD) theses (ProQuest assisted) and Master of Science theses from specialist centres on ethnicity and health.

The terms provided by Anderson et al. 47 to understand culturally competent care were adapted as a framework from which to generate search terms that might help identify publications on the effectiveness of interventions. Search strategies were constructed around BME groups using search terms refined through a number of systematic reviews29,48,49 and terms related to key descriptors of TCs: (1) aspect of TC; (2) types of mental disorder; (3) the professionals involved; and (4) aspects of clinical success.

These terms were tested and then applied to the electronic databases from database inception to 4 April 2012. All searches were rerun in early January and February 2013. The search strategy was designed to identify a broad range of literature on TC with BME patients and staff working in specialist psychiatric services (see Appendix 1 for the final search strategy). This strategy evolved as it was iteratively tested using a range of keywords and refined during pilot searches to provide a maximum yield.

The remit of the current systematic review is necessarily very broad: interventions for improving TC might be direct (i.e. between patient and professional) or indirect (through structural modifications to services to create more space for conflict to be resolved). Studies in both community and institutional settings were eligible. Studies of interventions designed to improve TC in our target population might not have been explicitly labelled as studies of TC. Indeed, relevant studies could be classified in many ways, for example as studies of communication, cultural competence or awareness, or just training to improve outcomes in ethnically diverse areas. Therefore, an inclusive strategy was felt most appropriate to capture all potentially relevant literature; specificity was sacrificed to maximise sensitivity.

The search strategy was iteratively developed to test its sensitivity to capture gold standard papers that were known to the research team. The final search strategy was settled following further discussion with the review management group (all investigators, listed as authors of this review) and carer representatives (Patrick Vernon/Afiya Trust). The complexity of the concept of TCs and the range of BME groups that might be relevant required us to develop this block search approach to ensure we captured all possible papers. The initial search strategy was trialled on different databases to assess the number of hits and relevant papers generated. We found a separate search strategy was needed to identify papers that included dementia, as these seemed not to be picked up by the generic search strategy. An example of the blocks of terms entered is shown in Box 1.

Hand searches of the following journals were completed for the time period April 2007 to May 2012: Transcultural Psychiatry; Culture, Medicine and Psychiatry; International Journal of Social Psychiatry; Journal of Cross-Cultural Psychology; Ethnicity and Health; Ethnicity and Disease; and Diversity in Health & Care. In addition, two special issues of journals were also screened:

1. Psychotherapy: Theory, Research, Practice, Training, a special issue on culture, race, and ethnicity in psychotherapy, Volume 43, pp. 377–560, winter 2006. 50

2. Journal of Counselling Psychology, Volume 58, pp. 457–646, October 2011. 51

Grey literature: data sources and search strategy

An important body of relevant evidence was expected in the grey literature: unpublished reports and papers containing practice and community-based information on interventions. For this material, standard database searches were replaced by a variety of strategies: hand-searching more recent issues of journals on ethnicity and health (those that had appeared in the last 10 years), and journals on communications; cascade-searching; and searching specialist collections at the Centre for Evidence in Ethnicity, Health and Diversity, The King’s Fund, the NHS library on ethnicity and health, National Institute for Health Research (NIHR)’s Health Technology Assessment (HTA) programme, the National Institute for Health and Care Excellence (NICE), the Royal College of Psychiatrists and the Medical Foundation for the Care of Victims of Torture. We also made use of various web-based resources [e.g. Google (Google Inc., Menlo Park, CA, USA), NHS Evidence, JISCMail] to search for reports that were not published in conventional research or professional journals and research in progress.

Search strategies for these grey literature databases were also derived by an information scientist and researcher. They built on the terms used for searching the published literature (see Appendix 1). An initial stage consisted of testing several key terms and search strategies, followed by filtering by eye and then iterative refinement of the original searches.

Economic literature

A series of systematic MEDLINE searches was undertaken by the information scientist to identify economic materials that were potentially relevant to interventions (see Appendix 10). Abstracts, were assessed and a total of 21 items were short-listed for full examination (see Appendix 10), as a result of which two studies with economic data were entered into the review; however, both had already been identified, one from the published literature search57 and one from the grey literature search. 58

Quality assessment

Overall quality score

The approach taken to quality rating was to use the core criteria and add the specialised criteria according to study design, and for the trials to include the economic score. For all other study types, the core and specialised criteria were used. The scores were summed and presented as percentages of the maximum score for each of the core items, the aggregated score of the specialised items and then as a total overall quality score. These were then categorised into low, medium or high quality on the basis of percentage of the maximum score (i.e. < 33% low, 33–66% medium and > 66% high). These are presented in Tables 2–5 to give an overall visual impression of the quality ratings according to study design, including how key elements of quality were rated, rather than relying only on a single total score. In undertaking the scoring, details about the methodological and design issues were evaluated using the scoring schedules. Specific methodological strengths or weaknesses were considered in the synthesis.

The B papers were:

1. reviews with no primary data

2. descriptions of potential interventions on theoretical grounds, but without an evaluation; often these were case studies for teaching purposes, without an evaluative and critical element or conclusion.

Studies included

The database searches yielded 7329 hits, and 3733 records were found to be potentially relevant after removing duplicates. An extensive search for grey literature yielded 608 sources, including six PhD theses, two of which could not be located from the original universities. Figure 1 shows the selection of papers at each stage of the review.

FIGURE 1.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram showing selection of papers at each stage.

A total of 21 publications were deemed to be relevant and met all the inclusion criteria. These comprised 12 trials52,57,60–69 one of which was from the grey literature,52 two observational studies,70,71 three case series58,72,73 (one of which was from the grey literature58 and one of which had a qualitative component that was separately extracted72), two qualitative studies40,74 (which included a case study in each) and two pure case studies. 14,75

Methods of analysis and synthesis

The findings are presented below in two groups (trials and non-trials) given that within the hierarchies of evidence it is the trials that provide the most definitive evidence of effectiveness. 83 We set out the interventions, study design, ethnic groups and service setting, and outcomes. Given the diversity of study settings, interventions and outcomes, the studies were not suitable for a meta-analysis and thus the data were subjected to a narrative synthesis. Popay et al. 84 define narrative synthesis as:

[A]n approach to the systematic review and synthesis of findings from multiple studies that relies primarily on the use of words and text to summarise and explain the findings of the synthesis. Whilst narrative synthesis can involve the manipulation of statistical data, the defining characteristic is that it adopts a textual approach to the process of synthesis to ‘tell the story’ of the findings from the included studies. As used here ‘narrative synthesis’ refers to a process of synthesis that can be used in systematic reviews focusing on a wide range of questions, not only those relating to the effectiveness of a particular intervention.

It is part of a larger review process that includes a systematic approach to searching for and quality[-]appraising research[-]based evidence as well as the synthesis of this evidence. 84

Narrative synthesis includes the following elements: textual description, tabulation, grouping and thematic analysis. We undertook this process and by thematic analysis contrasted, within- and non-trial designs, elements of intervention, the outcomes used in studies, and direct or indirect measures of effectiveness, taking account of the elements contributing to the quality score and the perspectives of patients and carers who reviewed the emergent evidence (see Patients’ and carers’ views).

Data extraction

Patients’ and carers’ views

A key element of the study was to ascertain user views from the outset; in the design and operation of the literature review, as well as in the evaluation of the interventions and themes identified as having potential to improve TC. In other words, we asked service users and their carers what they regarded (or would regard) as TC and what they would see as better practice. The involvement of minority ethnic users (i.e. BME people currently using health services for mental health needs, or significantly involved as carers of such patients) was operationalised by a collaboration with the national charity The Afiya Trust (www.afiya-trust.org) and its links to the National Black Carers Workers Network (NWBCN), CatchAFiya, and the National Survivor User Network, which support and enable the voices of such individuals through a panel of users who have had basic training in self-representation and some experience of supporting similar research and service development.

The Afiya Trust circulated a call for expressions of interest and selected a panel of eight people who had relevant experience and covered a range of ethnicities and mental health needs. The panel members, chaired by an Afiya Trustee, then worked closely with the research team in advising on the literature search, considering the emergent themes and assisting with the rating, validation and dissemination of the emergent models of good practice.

The core user panel comprised two men and six women. Three self-identified as being of African/Caribbean/black British origin and five were of South Asian background, with Hindu, Sikh, Muslim and other religious affiliations, and from a variety of the major South Asian national/linguistic groups. Three had experience as carers of mental health service users (some having also been patients themselves) and most had also some experience of working with community-based third-sector support groups.

Initially, a series of three user workshops were held, to elicit key themes and to discuss perspectives on TC as a process and what issues were seen as likely to be of concern. The panel’s views were then compared and contrasted with those emerging from the review of the published scientific literature and the grey literature. This background exploration was important for the patients and carers in order to prepare themselves to consider the outputs from the review, so that they were more active in deliberating the value of the interventions identified in the review.

The interventions or other solutions identified by the literature review were grouped and presented as model responses, using a technique of vignettes that included quotes from the publications and a brief description of key elements. A series of Delphi technique consultation rounds were engaged in using e-mail and workshops to highlight the most significant issues and to draw up a list of priorities for desired changes or best practice, as well as a list of issues of concern, responding to the research team’s initial questions and planned recommendations. Although overall agreement did emerge, there was a degree of variation between panel members, and many of the model interventions were ranked as ‘most desirable’ by some members but ‘least important’ by one or more others, illustrating the heterogeneity of need and the requirement for a variety of options to be available to service providers and users.

The patient and carers commented on the proposed interventions and ranked them as high or low priority, and these judgements were included in the synthesis alongside the quality of design and methodological issues, and the strength of the findings.

The trials

For trials Table 6 includes details of the study authors, title, intervention and the country, ethnic groups and sample sizes, diagnostic groups and service settings, and details on the professionals involved and the outcome measures used. The findings of the trials are presented in Table 7, which gives both the statistical details against each of the main outcomes used and a narrative summary of the findings for each study.

Description of interventions

The studies included in the review had interventions that could be broadly categorised as seven types:

1. preparing patients beforehand for their expected role in a psychiatric therapy before they received a therapeutic intervention60,69

2. enhancements of existing therapies in terms of technical content and attention to beliefs57,61,62,64,68

3. influencing wider social systems before and during therapy57,67

4. peer or professional to patient psychoeducation to encourage acceptability and effective use of therapy52,66

5. communication training for professionals to partner better with families, and increase expectations62

6. an audiovisual pre-therapy training to help patients make best use of the therapy, described as role induction60

7. the ethnic matching of the psychiatrist to the patient, achieved through the use of telepsychiatry. 65

The various components of each intervention were subjected to thematic analysis in order to capture the proposed effective elements. These are tabulated in Table 8. We also applied the framework for cultural adaptations of psychotherapies proposed by Tseng to assess if the following types of cultural adaptation were evident: philosophical, technical, theoretical or practical alternatives. 82

Outcomes

Symptoms were assessed as outcomes by 8 of the 11 trials. 57,61–68 One study included insight,61 one impairment,62 two functioning,57,67 six studies medication use and adherence,61,65–69 one patient experience,57 one perceived helpfulness,66 two patient satisfaction,65,69 one knowledge and attitude towards treatment,60 and one perceived stigma. 66 All of these outcomes can be understood as reflecting an inferred relationship between the intervention and improved TCs, rather than a direct measure of TC. Only one study included working alliance as an outcome. 65

Synthesis of findings

The trials did not exhibit sufficient similarity of diagnostic groups, samples, contexts or settings (see Tables 7 and 8), outcomes or interventions to warrant a meta-analysis that would provide meaningful pooling of data. They are briefly described below, categorised by their quality rating.

Non-trial designs

The data from the non-trial designs were extracted and tabulated (Table 9). Details extracted were the study title, design, country, interventions, sample sizes, ethnic groups, settings, professional groups and the outcomes used. In addition, a summary of findings with statistical and narrative conclusions is included.

Thematic analysis of interventions

The evidence on the interventions and how they were constructed as set out in Tables 6–8, and detailed in terms of components in Tables 8 and 10, was synthesised using thematic analysis. This analysis had a particular focus on the aims and components of different interventions. Overwhelmingly, interventions aimed to improve engagement of the patient in treatment, while some also explicitly aimed to empower patients. These empowering interventions were especially valued when our patient and carer group reviewed the interventions. Most interventions aimed to engage or empower by increasing professional understanding of patients’ belief systems about illness, either through formal training or through adapting the content and/or structure of treatment to accommodate differences arising from ethnic diversity. A minority of interventions focused rather on patients’ expectations of treatment. Compared with interventions evaluated in non-trials, those evaluated in trials were less likely to be specific to a single ethnic group (8/10 and 5/12, respectively) and were more likely to focus on accessibility of treatment for patients (1/10 vs. 4/12). In two of the trials57,67 and two of the non-trials,40,58 the wider social system was also considered in the intervention. The elements of the interventions that we have isolated might also be valuable in future evaluations of complex interventions, where these might be adaptations of the interventions we have identified or composite interventions that combined the best of the interventions we have identified with new approaches to improve TC.

There were some studies that included a description of an intervention and did not include an evaluation, or the populations of interest were not relevant to the UK context and there was not any obvious transferability of the knowledge; these did not enter the review and were assessed as meeting our A– criteria. These included the following interventions: cultural formulation,93 restructuring illness narratives,94 cultural sensitivity,95 multicultural competence and evidence-based group therapy practice,96 a community-based treatment for Native Americans,97 adapted CBT for traumatised refugees (other papers from the same author were included in the review as they provided evidence),98 culturally sensitive CBT for psychosis in ethnic minorities (other papers from the same author were included in the review as those provided evidence for this intervention),99 components of cultural competence in three mental health programmes (service intervention across programmes and not focused on the TCs),100 culturally relevant psychoeducation for Korean American families,101 psychosocial counselling in Nepal,102 cultural considerations in CBT with Chinese people,103 and adaptation of solution-focused therapy in Chinese culture. 102 These interventions were described often as serving an educative role about constructing interventions or sharing how to deliver an intervention and so did not enter the review. A full reference list is available in Appendix 16.

Patients’ and carers’ ranking of interventions

This broader discussion served to contexualise the work and provide insights about interventions that patients and carers valued. This seeking of views from a general discussion, to grade and then to comment on the evidenced interventions, was considered the most useful and creative process to engage patients and carers, rather than just presenting them with a set of outcomes and expecting a passive approval or disapproval of identified interventions. In initial discussions, a number of aspects of TC and experience of mental health services were discussed and some key ‘likes’ and ‘dislikes’ were identified (Tables 11 and 12). In all cases, it was clear that a personalised experience was being sought and that tokenism or being reduced to stereotypical categories was an experience that had adverse effects. Some possible actions or solutions to a failure of communication were also discussed, and the panel members were able to identify some ideas which they were aware of, or which emerged in discussion, and which were felt to be less helpful, while greater enthusiasm was expressed for other ideas which reflected a degree of interest in their community, culture or situation, and which they regarded as being general good practice. This included choice and continuity of care, and attention to human rights and patient rights approaches.

Subsequently, the panel members were presented with the key emergent themes from the grey literature, which had been published on the subject of access to, and improvement of, mental health communication with BME communities. A list of the emergent themes is provided in Box 3.

In addition to these, the literature reported a number of concerns apparently specific to particular ethnic groups, noting in particular:

• African Caribbean community members fear medication or ‘strong drugs’ which may alter personality.

• Men want to see male practitioners where issues to do with masculinity are valued.

• Chinese people have very culturally specific understandings of needs and a specific world view.

• South Asian people have commonly reported concerns about three main issues: languages, stigma (or shame) and effects on family status (referred to as Izzat).

• Asylum seekers were reported to avoid contact with services and to suffer additional ill health because of stresses arising from their contacts with the UK Borders Agency and others in connection with their immigration status.

• Migrants of Eastern European origin (of recent years) are known to have increased health needs owing to the long and unsocial hours typically worked by members of such groups.

• Religious Jews are also reported to wish for Kosher (Kishrut) rules to be observed, in particular around diet, but also want their support to be from people who are not members of their religious community, to preserve confidentiality.

The panel discussed these matters and applied the model to their own experiences, understanding and community concerns. There was some agreement on these, and also recognition that many of them were reflected in each other’s cultural setting and were not as restricted to one cultural group as initially perceived, especially issues in relation to shame and cultural rules. However, they were adamant that these rules should not become accommodated in new fixed guidelines that might imprison people within a stereotypical culture-bound set of behaviours and treatments, but people should have the ability to choose a level of observance or derogation from such ‘norms’ and not to be treated ‘as if’ without their explicit agreement. In other words, practitioners should be aware and informed about these matters, but to ask before acting on the basis of perceived cultural norms. Their key principles are listed in the next section, Users’ final perspective. Solutions that were proposed in the grey literature included Box 4, which were again subjected to a critique by the panel members.

Users’ final perspectives

The following were the issues identified from our workshops and presented as a set of ‘key recommendations’ by the panel members that might be central to TCs:

• acknowledgement of pain and hurt, and encouragement to share through talking

• ‘ask me what I want and work with me to tailor-make a plan’

• stable, consistent relationships between patient and clinician.

These three themes appear to be universal truths appropriate to all service users, but were stated to have additional salience and cultural specificity for BME populations, in which pain and distance between user and provider may be greater and there is a friction or strain involved in having to explain the additional issues of cultural identity ‘again and again’. As suggested in some of the ‘grey literature’, the panel members noted the importance of care plans, including attention to housing and financial benefits (i.e. not just mental health treatment and advice) and opportunities for relaxation and social life. It was noted that users may need to be helped to know what forms of additional support are available, but clinicians need to be open to the user’s own solutions, even if these are not in the NICE guidelines. In addition, the following suggestions and understandings were offered as guidelines for training and improved practice:

• Good listening skills and reassuring body language.

• Different ethnic groups have different attitudes towards looking steadfastly in the eyes, touch and gaze, and clinicians should be aware of these matters.

• How do people show attention and respect?

• ‘Don’t lean and gesticulate!’

• The centrality of practical, non-clinical support.

• The treatment needs to be looked at as a whole, over a long-term ‘recovery relationship’, not a single episode.

• Equally, training and growth by experience cannot be achieved by a one-off training course but comes from sustained immersion in working with difference.

• Respect – non-critical listening is crucial – don’t send out signals of alarm or startle or even dismissal – this can close down the conversation.

• Characteristics that might help:

  • ‘someone like me’ may mean ‘not too much like me’, as it could lead to assumptions

  • someone who understands and can show it!

  • someone ‘who likes me’

  • signalling some prior knowledge of culture.

One African-heritage member of the group noted that some white professionals had been very helpful, and showed they were ‘trying to understand’, but her current Asian professional was even better, in showing a caring attitude and calling back (by telephone) to ask about any side-effects of treatment (outside consultation).

Prioritisation of ‘good practice’ models and interventions

The panel was finally presented with vignettes based on the 10 main types of intervention identified in the literature review, and a brief description of each. These are included in Table 13.

Following two workshops in which these themes and interventions were discussed and described, the users were asked to ‘rate’ each of the proposed interventions, and to make any comments that would explain their judgement. Then, in a final round, they were asked specifically to select the three they most felt worthy of investment and development or implementation, and the three they regarded as being of least interest or value. The ratings, and some key comments, are shown in Table 14.

It is clear that there was a degree of consensus on the top four ideas, while some were agreed to be of less salience from the point of view of service users and their carers. There was not, though, total agreement, since some models were rated by different members of the panel as both ‘top priority’ and ‘least value’. As one of the members of the group commented:

In general, the ranking exercise was more challenging than I anticipated because no single intervention is perfect, and even the worst interventions have at least one positive attribute.

Overall, there was a strong feeling of support for both the need for health-care professionals to learn more about how to communicate with BME patients, and for the use of consultation approaches that were modified to take account of specific cultural sensitivities (such as stigma, language and somatisation).

There was less support for approaches which were seen as ignoring the skills or insights of the user, and providing ‘briefing’ or training in ‘how to be a patient’, but this may have depended on how far such support was deemed to be sympathetic to their expertise and culture: some of the examples found reported in the professional literature had quite detailed vignettes or descriptions of case studies which provide a more accessible way to understand how the intervention was put into practice.

Somewhat unexpectedly, the use of community workers providing advocacy (see Community workers practical advice and advocacy in Table 14), despite its attention to practical needs such as entitlement to benefits, was not well regarded, even though this was a theme that had been highlighted in earlier discussions. It may be that the specific study was felt to be less applicable to the situation of the panel members, or that it did not show enough care to provide confidentiality and protection against community spread of ‘gossip’, which was another very real concern. Members of the group certainly felt it was overcomplicated in practice and might lead to problems in communication or leadership between different professionals. A degree of medical professionalism was seen as appropriate, in addition to ‘cultural competence’ – which they felt should be expected of any professional, and not seen as an ‘add-on’ capability.

None of the proposed models of service development achieved total support, and there were a number of issues – such as public perceptions of communities and the impact of racism or media coverage of ‘immigrant concerns’ – that were not addressed by any of the proposed solutions located in the published papers reviewed. Some of these issues are addressed in the next section, User priorities.

User priorities

Many of the points raised in the user group discussions incorporated several elements of the suggested ‘model’ interventions located in the published literature. A key theme was: ‘asking me what I want and then doing it! Working with me to tailor-make a plan.’ Many participants had had negative experiences of treatment:

It stays with and changes you. Staff with the right attitude – [we are] looking for . . . respect.

It was noted that empathy and understanding need to be unconditional and not subject to imposed rules and the assertion of dominance by professionals. Clients were looking for flexibility in application of rules and standards. Some expressed the feeling that professionals selected ‘good’ clients and that people have to ‘merit’ help. This was related often in terms of cultural attitudes to timekeeping and authority: people find having to conform to ‘being here at a fixed time/place’ hard, even punitive. This was said by more than one user to have caused them distress and got in the way of therapy. As one of the group noted:

Staff often speak to you and treat you like a child. Over time, this has the effect of disempowering you and reduces your ability to trust yourself. When staff complain about people being dependent, they don’t acknowledge then they in fact created this dependency by taking away free will and choice.

An element of ‘the right attitude’ was ‘understanding and valuing community and cultural and religious connections and sensitivity.’ It was expected that (or at least hoped that) professionals would have some sensitivity to, and prior knowledge of, the communities of origin of their clients, and be aware of culturally influenced issues such as honour and witchcraft, as well as of gender relations (including the risks of domestic violence) and expectations and fears around confidentiality and stigma. Users reported sensitivity to issues of ‘double discrimination’ where, on top of the stigma of mental illness, they might be discriminated against on the grounds of their culture or religion, or their ‘colour’ or race. Such processes were noted also to occur within minority communities. Consequently, it was felt that professionals should have some knowledge of where to refer clients in order to find culturally specific support from appropriate and supportive community-based organisations, while avoiding those which would be judgemental or which would otherwise expose the community member to disapprobation and stigma.

Finally, people in BME groups – especially but not exclusively those of recent migrant origin – pleaded for professionals to: ‘be patient when I don’t understand things – language, forms and entitlement.’ They reported the stress of trying to fit into checklists of symptoms, noting that people have different ways of expressing themselves – but also were anxious to be seen to be showing respect to doctors and other important persons. ‘Not being able to express myself makes me feel there is something wrong with me’. This was exacerbated by frequent changes in personnel and referral between teams.’ Overall, there was a strong wish for ‘consistency and continuity and stable patient–clinician relationships.’

Discussion

This review identified several types of effective interventions in trial designs, including adapted cognitive–behavioural treatments,61,63,64 adapted psychotherapies more generally,57,67,68 complex interventions that engage with social systems and stepped care,57 different interviewing techniques like ethnographic and motivational interviewing,67 behavioural activation within an adapted cognitive–behaviour paradigm,68 and role adoption that included information and training for patients to make best use of their services and better use of encounters with professionals. 60,69 The telepsychiatry intervention65 showed a favourable response in that patients were willing to pay more for it than the standard treatment, although there was not much evidence of advantage in term of symptoms.

Each of these showed benefits in terms of an improvement in outcomes.

Adapted cognitive–behavioural therapy

What these studies offer is a range of effective interventions with differing emphases all showing benefit. Thus adapted CBT for several ethnic groups61 worked as well as CBT adapted for a single ethnic group. 64 Service users reviewing the evidence were supportive of these. Importantly, the methods of adaptation paid attention to technical, theoretical, philosophical and practical aspects,87 and in that respect both Rathod et al. 61 and Hinton et al. 63,64 were similarly adapted. These adapted interventions were promising as they showed benefit in terms of both symptoms and patient-reported outcomes, and were evolved by including elements of culturally determined communication of distress in the CBT manual; Rathod et al. ’s61 study contrasted with Hinton et al. ’s63,64 studies as the former was applied across ethnic groups in the UK, whereas Hinton et al. 63,64 offered a more profound adaptation for specific East Asian ethnic/cultural groups with PTSD, where these populations held markedly contrasting beliefs and practices regarding health and illness when compared with the majority US population. We omitted from our review studies that applied a non-adapted CBT intervention in other countries, as the concept of ‘improving the intervention’ was not visible and the attention to cultural or ethnic groups was minimal other than recruiting a sample in another country; some of these excluded studies, even if of high quality, paid little attention to therapeutic relationships and so do not feature in our review.

Complex interventions

There were two trials that included a complex intervention based on social systems in the community. 57,67 One of these engaged ethnographic and motivational interviewing as part of the assessment to empower pregnant African American single women who were depressed,67 and this preceded interpersonal therapy. One of the key components of this intervention was the emphasis on empowerment through personal stories being understood. This contrasted with another complex intervention: a stepped care model of community social venues acting as points of access to an adapted CBT intervention for black African, black Caribbean and black British people living in south London. 57 Both of these were effective at reducing symptom outcomes, yet service users in our review of the evidence favoured the US intervention, perhaps because of its emphasis on belief systems, ethnography and empowering interviews, which gathered knowledge and stories from participants in their community contexts before recommending an intervention. They were less keen on a stepped care approach or too complex a system of health care, which they felt they would find difficult to negotiate. The trial by Afuwape et al. 57 was the only one to include an economic evaluation suggesting the cost-effectiveness of the approach. The economic review indicated this evidence to be weak given the small samples and single centre in a pilot study. These studies emphasising the patients’ perspectives share the empowerment and motivational elements with other non-trial designs, specifically the small case series on cultural consultation,58 more details of which were published in an educational paper following the completion of our review. 104

Training, education and communications skills

There were a few studies of training and education for professionals, including on communication skills with family members. One study of specialist psychiatric services outreach focused on the treatment of children62 and showed less impairment in black compared with white children for the sites in which training took place, but there were no differences in symptoms, and the training also led to fewer parent symptoms, irrespective of ethnicity. White parent symptoms diminished while ethnic minority parent symptoms increased. Of the two other trials that included an educational intervention, one tested role induction69 and the other an audiovisual programme that prepared patients for psychotherapy. 60 The first of these, scoring low on quality, was associated with higher premature dropout, but more satisfaction in the intervention group and greater expectancy of what therapy might offer which is a positive sign when approaching a therapy. 69 The second of these, a medium-quality study, increased patients’ information about psychotherapy and improved attitudes so they expected more from therapy. 60 Although less favoured by patients who commented on the evidence we generated, these preparatory interventions seem attractive to practitioners as a way of enhancing the benefit patients receive from interventions that are already provided. Yet, these have not been assessed for cost-effectiveness and value to a wider range of patients; these may be especially unhelpful for those who are socially isolated or who have difficulty seeing the value of travelling to a hospital or therapy centre, where the more complex ethnographic and motivational interview-based interventions may achieve more success.

Despite the body of literature on cultural competence29 of professionals as a key way of improving health outcomes and communication between BME patients and professionals, none of our trials tested a specific model of cultural competence, or at least that is not what the authors called their interventions. Given the nature of the review, we may be demonstrating that cultural competence is too broad a framework, better suited to policy and aspirations for practitioners, while more concrete and specific examples of interventions are needed. This review illustrates just that. Tables 6 and 8 show the shared and non-shared components across the interventions. Future interventions might select particular components and test them either individually or be able to discern which are more important for a positive care experience and a positive health outcome (the two may not always be closely linked).

Only one study included telepsychiatry; this was of Mexican patients and psychiatrists living in the USA close to the Mexican border, and essentially relied on ethnic matching and the telepsychiatry intervention. 65 This showed more antidepressant use, and greater working alliance and satisfaction, but no difference in depressive symptoms. Patients were willing to pay more for this intervention, which suggests its convenience was valued, but the relative benefits of ethnic matching and a telepsychiatry intervention cannot be disentangled. Although many studies of ethnic matching can be found in the literature, these are often observational and show an association between ethnic matching and outcomes in administrative data sets rather than testing an intervention in a trial. 105,106 However, this study does illustrate how just a few psychiatrists can provide services to geographically remote or scattered populations who need a mental health professional with a similar ethnic background, or, for that matter, linguistic skills. The wider issue of using e-technology and social media as a way of encouraging engagement and delivering interventions was not investigated in the studies that entered the review.

Other study designs

The studies with a non-trial design did not add more to the evidence base. They assessed psychoeducation, ways of adapting psychotherapies, including a family orientated intervention, ethnographic interviewing and case studies concerning ethnographic principles. Given that these interventions were at an earlier stage of development, it is reasonable to suggest that their elements, especially their ethnographic elements, may prove suitable for testing in trials. 14,15,40,58,75 Two of these studies nicely showed how to involve the community before adapting an intervention. 72,74 Many excluded papers presented accounts of the cultural formulation, and its use as part of the formal psychiatric assessment process, yet none evaluated the cultural formulation, or offered a sufficiently critical evaluative statement. 87,107 In the new edition of its diagnostic manual Diagnostic and Statistical Manual of Mental Illness-Fifth Edition (DSM-V),108 the American Psychiatric Association has included a section on the ‘Cultural Interview’ that promotes the assessment of cultural factors (at least this area is no longer in an appendix, as it was in previous editions of the manual) and sets out guidelines. However, this approach needs evaluation and some have expressed a concern that the approach will not be well known or widely adopted by practitioners. 43

UK compared with US studies

Two trials,57,61 one case series58 and one case study14 were based in the UK. All the remaining studies were based in the USA. Given the differences in the cultural heritage and history of migrant groups in the UK and the USA, we were careful not to include studies that did not hold lessons for UK populations. Otherwise, where important principles of adaptation were studied, and where the US populations had similar ancestry, cultural history, or experiences of racial and ethnic discrimination and social exclusion, studies were included. All studies need to consider adaptations if the proposed interventions have not previously been used; usefully, the studies reviewed included the adaptation method for a range of study designs which are useful for future researchers working on new health outcomes or ethnic or culturally distinct populations for which little evidence exists. Although we have focused here on USA versus UK comparisons, this was not the original intention. The studies that met our inclusion criteria happened to be from the USA or the UK, reflecting the attention given to this in these countries and that more work is needed from other countries.

Economic evaluations

The review found only two studies offering economic evidence: a report on the CCS commissioned by the NHS in part of London,58 and a pilot trial of a community-based intervention for individuals from BME groups with common mental disorders. 57 The initial economic analysis of the CCS in Tower Hamlets suggested that there could be service use-related cost savings over a 3-month period following a clinical cultural consultation. And there was less emergency care needed in the intervention group. The Cares of Life Project (CoLP)57 was a small randomised controlled trial comparing rapid access and delayed access (by 3 months) to a pragmatic, needs-led care package consisting of practical advice and assistance, advocacy for social needs, health education, mentoring and one-to-one brief therapies based on principles of CBT and brief solution-focused therapy.

The rapid-access group had better depression outcomes, and the service was found to be more culturally appropriate and acceptable among users, and costs were slightly higher, but not statistically significantly for the rapid-access group. Both of these studies can be described as exploratory rather than definitive – partly because of their small sample sizes, short follow-up periods, and recruitment from a single locality – but both studies offer encouraging economic evidence. The stronger message coming from this review, however, is the absence of economic evidence in this area, making it impossible at this stage to draw conclusions as to cost-effectiveness.

Conclusions and future research

Implications for clinical practice

Our review found a range of interventions designed to improve TCs with BME patients in psychiatric services. Furthermore, we identified specific elements of these interventions that are likely to prove amenable to incorporation with clinical practice and to be effective. Practitioners might map their existing provision against these. They are:

• patient centred

• professional centred

• cultural adaptation of established therapies

• the involvement of social systems and the community in the patient’s care

• patient empowerment and engagement.

Clinicians may wish to adapt interventions, and our evidence gives examples of the processes by which this can be undertaken with families and individuals. Two issues were emphasised in our consultations with patients and carers: they didn’t like it when they were passive recipients of information in an impersonal service; and communications matter, even if these are not directly related to therapy. If professionals better understand patients’ causal explanations and health beliefs, they will be better able to assess patients and to deliver effective information to patients and carers. In particular, pre-therapy education can improve therapeutic outcomes for people from diverse cultural and ethnic backgrounds when it is tailored to their needs and expectations. Given the emphasis in policy and practice on equity and reducing inequalities, commissioners and policy-makers need to consider the recommendations set out here on what types of interventions are helpful. In summary, interventions that include outreach and engagement with the patients’ social world, cultural beliefs about health and illness, overcoming difficulties accessing services and stigma, motivational and ethnographic interview and cultural consultation processes are important foundations that empower and offer a more flexible experience that is tailored to individual patients. The may wish to map existing provisions, taking more notice of specific types of services and whether or not they offer effective intervention and service models that are acceptable and attractive for ethnically and culturally diverse populations.

Author contributions

Kamaldeep Bhui was the principal investigator, managed the project, was lead on the analyses and writing of the report, and responsible for sharing consecutive drafts with the team. Kamaldeep Bhui supervised the research fellow at Queen Mary University of London, extracted data with Rabbea’h W Aslam and Andrea Palinski, and was lead on writing the final report.

Rabbea’h W Aslam was a research fellow on the project, collected the publications, acted as second searcher and first extractor.

Andrea Palinski appraised the grey literature, assessed against criteria, extracted data, and prepared tables for the report with Kamaldeep Bhui and Rabbea’h W Aslam.

Rose McCabe was an applicant, helped extract some papers, assess against inclusion and exclusion criteria and adjudicate on disagreements on meeting inclusion criteria and characterised interventions with Kamaldeep Bhui and Rabbea’h W Aslam.

Mark RD Johnson was an applicant, assembled and helped facilitate the service user group, helped identify grey literature sources, and web resources, and supervised the survey of experts.

Scott Weich was an applicant, critiqued research processes, clarified methods and inclusion and exclusion criteria, and reviewed outputs with Kamaldeep Bhui.

Swaran Preet Singh was an applicant and reviewed outputs.

Martin Knapp critiqued economic outputs, prepared economics sections of text, and reviewed outputs.

Vittoria Ardino critiqued economic outputs, prepared economics text and reviewed outputs.

Ala Szczepura was an applicant, managed grey literature and web searches, early economic searches and helped facilitate the service user group.

All authors commented on consecutive drafts and reviewed the final draft.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health.

Sample search strategy run on Ovid MEDLINE and adapted for other databases

Searched: 31 March 2012.

Date range of searches: from inception until 31 March 2012.

Organisations contacted

1. Advocacy Project.

2. African Caribbean Mental Health Services.

3. AnxietyUK (formerly the National Phobics Society).

4. Asian Family Counselling Service.

5. Asian Resource Centre.

6. B-eat.

7. Better Health.

8. BHA.

9. Black and Asian Therapists Online.

10. Black Orchid.

11. Breathing Space Scotland.

12. British Association for Behavioural and Cognitive Psychotherapies.

13. British Association for Counselling and Psychotherapy.

14. British Association of Psychotherapists.

15. British Psychoanalytic Council.

16. British Psychological Society.

17. Care Quality Commission.

18. Carers Northern Ireland.

19. Carers Scotland.

20. Carers UK.

21. Carers Wales.

22. CAUSE (Carers and Users Support Enterprise).

23. Centre for Mental Health (Sainsbury).

24. Chinese Mental Health Association.

25. Confederation of Indian Organisations (UK).

26. Cruse Bereavement Care.

27. Depression Alliance Scotland.

28. Depression UK.

29. Equilibrium.

30. Fanon Women’s Centre.

31. Greenwich Mind.

32. ICAP.

33. Institute of Family Therapy.

34. Jewish Association for the Mentally Ill.

35. Kuumba Centre.

36. MDF – The BiPolar Organisation.

37. MECOPP (Minority Ethic Carers of Older People Project).

38. Mental Health Foundation.

39. Mental Health Shop.

40. MIND.

41. Muslim Youth Helpline.

42. Nafsiyat – The Intercultural Therapy Centre.

43. Narcotics Anonymous.

44. Narcotics Anonymous Northern Ireland.

45. National Mental Health Development Unit.

46. National Self-Harm Network.

47. Northern School of Child and Adolescent Psychotherapy.

48. OCD-UK.

49. Pakistani Resource Centre.

50. Qualb Centre.

51. Recovery Research Network.

52. Refugee Council.

53. Rethink Mental Illness.

54. Royal College of Psychiatrists.

55. Sahara – Asian Women’s Group.

56. Samaritans.

57. Sane.

58. Sathi – Asian Men’s Group.

59. Savera.

60. Scottish Association for Mental Health.

61. Sheffield African Caribbean Mental Health Association.

62. Society of Analytical Psychology.

63. South London Family Centre.

64. Southall Black Sisters.

65. Tamarind Centre.

66. The Afiya Trust.

67. The Black Women’s Mental Health Project.

68. The Harmony Family Centre.

69. The Naz Project.

70. Triumph Over Phobia (TOP UK).

71. Tulip Mental Health Group.

72. UK Council for Psychotherapy.

73. Waltham Forest Black People’s Mental Health Association.

74. Women’s Therapy Centre.

Experts found in ongoing research

Therapeutic communications in black and minority ethnic populations (THERACOM)

Searches of index to theses

A search of the index of all doctoral dissertations and some master’s theses from most North American and many European colleges and universities was undertaken covering nearly 3 million searchable citations to dissertation and theses from around the world to 4 February 2013. The search used the terms Psychiatr* AND Communication AND Ethnic.

Rudell K. Bart’s Explanatory Model Inventory – The Exploration of Cross-cultural Variations in Perceptions of Mental Distress. PhD thesis. London: Queen Mary University of London; 2006.

Outcome: A study of explanatory models in primary care and in the population, not in specialist-psychiatric care. And did not assess TCs or outcomes therefore does not meet inclusion criteria.

Hines AL. Perceptions, Preferences, and Partnerships: A Patient-Centred Approach to Understanding Racial and Ethnic Differences in Mental Healthcare. PhD thesis. Baltimore, MD: The Johns Hopkins University; 2010.

Outcome: This thesis uses routine data from the collaborative psychiatric epidemiology studies, including the national comorbidity survey replication, the national survey of American life and the national Latino and Asian American study and patient-centred care for African Americans and depression study. There is no intervention or any evaluation of an intervention so it does not meet inclusion criteria.

Mickens-English P. The Efficacy of an Afrocentric/Holistic Group Psychotherapy Approach for Black Women. PhD thesis. Kent, OH; Kent State University; 1996.

Outcome: Not available for order or from originator university.

Tom LM. Psychoeducational Approach With Chronically Mentally Ill Chinese Americans: A Cultural Special Issue: Cultural Adaptions 547 Framework. PhD thesis. New York, NY: The City University of New York; 1989.

Outcome: Not available from originator university.

Raymond MJ. Analysis of Native American Cultural Practices Used as a Treatment Modality for Alcohol Addiction. PhD thesis. Palo Alto, CA: Pacific Graduate School of Psychology; 1996.

Outcome: This was an observational study on retrospective data on treatment outcomes following alcohol programme, with a newly constructed variable of traditionalism assessed against outcomes: physical health, alcohol use, emotional status, cultural status, spiritual status. There was no intervention to improve TCs between ethnic minority populations and specialist psychiatric staff; indeed this was a study of indigenous groups in the population. Thus the thesis did not meet criteria for inclusion.

Schwarz DA. The Effect of a Spanish Pre-Therapy Orientation Videotape on Puerto Rican Clients’ Knowledge About Psychotherapy, Improvement in Therapy, Attendance Patterns and Satisfaction with Services. PhD thesis. Philadelphia, PA: Temple University; 1989.

Outcome: Originator university unable to locate microfiche of thesis.

Health, health policy, evidence-based medicine and health services research websites searched

A wide array of health, health policy, evidence-based medicine, health services research and other websites were searched in November 2013. These included the following:

1. BME Health Forum. URL: www.bmehf.org.uk

2. Bristol Mind. URL: www.bristolmind.org.uk

3. Centre for Mental Health. URL: www.centreformentalhealth.org.uk

4. Healing Circles. URL: www.healingcircles.org

5. Hogg Foundation for Mental Health. URL: www.hogg.utexas.edu

6. Mental Health Commission of Canada. URL: www.mentalhealthcommission.ca

7. Mental Health Foundation. URL: www.mentalhealth.org.uk

8. Mind. URL: www.mind.org.uk

9. Mind in Croydon. URL: www.mindincroydon.org.uk

10. National Alliance on Mental Illness. URL: www.nami.org

11. National Mental Health Development Unit. URL: www.nmhdu.org.uk

12. Network for Mental Health. URL: www.nsun.org.uk

13. Race Equality Foundation. URL: www.raceequalityfoundation.org.uk

14. Rethink Mental Illness. URL: www.rethink.org

15. Social Care Institute for Excellence. URL: www.scie.org.uk/

16. World Federation for Mental Health. URL: www.wfmh.org

Black or minority ethic-relevant websites searched

1. The Afiya Trust. URL: www.afiya-trust.org

2. Better Health. URL: www.better-health.org.uk/

3. The Black and Asian Therapist Network. URL: www.baatn.org.uk/

4. Bolton Council of Mosques. URL: www.thebcom.org

5. British Asian Medical Association. URL: www.thebama.co.uk/

6. Friends, Families and Travellers. URL: www.gypsy-traveller.org

7. Health Development Authority (HDA) – closed 2007. URL: www.nice.org.uk/aboutnice/whoweare/aboutthehda/hdapublications/hda_publications.jsp

8. Institute of Race Relations. URL: www.irr.org.uk

9. The King’s Fund. URL: www.kingsfund.org.uk

10. Meri Yaadain – Recognising Dementia in South Asian Communities. URL: www.meriyaadain.co.uk

11. Migrant Health. URL: http://mighealth.net

12. Minority Fellowship Programme. URL: www.emfp.org

13. Mosaics of Meaning. URL: www.mosaicsofmeaning.info

14. MultiKulti – closed 2008. URL: www.multikulti.org.uk/

15. Race Equality Foundation. URL: www.raceequalityfoundation.org.uk/

16. South Asian Health Foundation (SAHF). URL: www.sahf.org.uk/

17. South East and Migrant Health (SEMH) Network. URL: www.migranthealthse.co.uk

18. The Urban Indian Health Institute. URL: www.uihi.org/

Other websites searched

1. Aethos Consultancy and Training. URL: www.aethos.org.uk

2. Care Quality Commission. URL: www.cqc.org.uk

3. Child and Maternal Health Observatory. URL: www.chimat.org.uk

4. CLAHRC (Collaboration for Leadership in Applied Health Research and Care) for South Yorkshire. URL: http://clahrc-sy.nihr.ac.uk/

5. The College of Emergency Medicine. URL: www.collemergencymed.ac.uk

6. Department of Health. URL: www.dh.gov.uk

7. Economic and Social Data Service. URL: www.esds.ac.uk/

8. Health Services Research and Development Service (HSR&D). URL: www.hsrd.research.va.gov

9. Health Talk Online. URL: www.healthtalkonline.org

10. Informa Healthcare. URL: http://informahealthcare.com/

11. Joseph Rowntree Foundation. URL: www.jrf.org.uk

12. National Association of Social Workers. URL: www.socialworkers.org

13. National Center for Biotechnology Information. URL: www.ncbi.nlm.nih.gov

14. NHS Camden. URL: www.camden.nhs.uk/

15. NHS Confederation. URL: www.nhsconfed.org

16. NHS Norfolk. URL: www.norfolk.nhs.uk

17. NHS North West. URL: www.northwest.nhs.uk

18. Offender Health Research Network. URL: www.ohrn.nhs.uk

19. The Psychologist. URL: www.thepsychologist.org.uk

20. Scope. URL: www.scope.org.uk

21. UC Davis Health System. URL: www.ucdmc.ucdavis.edu

Closed websites

UK sources

All websites were accessed in November 2013.

Non-UK sources

Search strategy for NHS Evidence

URL: www.evidence.nhs.uk/

Date of search: January 2012.

All URLs were accessed in November 2013.

Search terms used

Search terms used (always used in pairs both ways if two given).

• Mental and communication.

• Therapeutic and mental.

• Therapeutic and communication.

• Mental and counselling.

• Mental and therapy.

• Mental and intervention.

• Advocacy.

Projects identified in UK Clinical Research Network study portfolio

Projects identified on the National Institute for Health Research Evaluation, Trials and Studies Coordinating Centre website

Projects identified on the Economic and Social Research Council website

Outputs identified for completed projects From UK Clinical Research Network study search

Outputs identified from National Institute for Health Research Evaluation, Trials and Studies Coordinating Centre search

Outputs identified from Economic and Social Research Council search

Search strategies

Sample search strategy run on Ovid MEDLINE and adapted for other mental disorders.

Searched: inception to 4 April 2012 (re-run January/February 2013).

Other economic articles shortlisted

How clearly each study indicates that there is an intervention to improve therapeutic communication could be a quality indicator (1–4)

1. Intervention clearly shown, and it clearly is to improve TC, for relevant outcomes of interest. (4)

2. Intervention clearly shown and relevant to outcomes of interest. (3)

3. Intervention vague and or multicomponent, so difficult to discern whether the mediating effect is truly through improved TC. (2)

4. Inferred by reviewer given elements of decision-making, assessment and conversation needed and efforts to improve these through adaptation of interventions. (1)

Outcome of therapeutic communication (1–3)

1. Direct measure of TC by a reliable and valid scale, for example alliance, reduced conflict, greater trust. (3)

2. Proxy measure: attendance, premature termination. (2)

3. Narrative outcome. (1)

4. No outcome: exclude study.

Ethnic groups (0–5)

1. Groups of relevance to the UK and described in a manner consistent with a specific classification scheme for ethnicity (not just race). (5)

2. Groups of relevance to the UK. (3)

3. Not of relevance to the UK but well described in terms of ethnicity. (1)

4. Not of relevance not the UK. (0)

Types of study: (1–4)

1. Quantitative: randomised controlled trial. (4)

2. Observational: a series, or service evaluation data, or pre–post type evaluations. (3)

3. Qualitative/narrative evaluation of a series. (2)

4. Case study/studies: in-depth narrative information. (1)

5. Experiences and personal reports with no methodological framework. (0)

Economic evaluation (0–4)

1. Cost-effectiveness. (4)

2. Impact and interventions cost/benefits. (3)

3. Intervention costs. (1)

4. Financial gains and losses. (1)

Quality of randomised controlled trial (0–30)

1. Adequate sample size (n per group): 0 = inadequate, 1 = moderate and 2 = large or specified by power calculations.

2. Appropriate duration of trial including follow up: 0 = too short, 1 = reasonable length and 2 = long enough for assessment of long-term outcomes.

3. Power calculation: 0 = not reported, 1 = mentioned without details and 2 = details of calculations provided.

4. Method of allocation: 0 = unrandomised and likely to be biased, 1 = partially or quasi-randomised with some bias possible and 2 = randomised allocation.

5. Source of subjects described and representative sample recruitment: 0 = source of subjects not described, 1 = source of subjects given but no information on sampling or use of unrepresentative sample (e.g. volunteers) and 2 = source of subjects described plus representative sample taken (e.g. all consecutive admissions or referrals, or random sample taken).

6. Use of diagnostic criteria (or clear specification of inclusion criteria): 0 = none, 1 = diagnostic criteria or clear inclusion criteria and 2 = diagnostic criteria plus specification of severity.

7. Record of exclusion criteria and number of exclusions and refusals reported: 0 = criteria and number not reported, 1 = criteria or number of exclusions and refusals not reported and 2 = criteria and number of exclusions and refusals reported.

8. Blinding of assessor: 0 = not done, 1 = done but no test of blind and 2 = done and integrity of blind tested.

9. Assessment of compliance with experimental treatments (including attendance for therapy): 0 = not assessed, 1 = assessed for some experimental treatments and 2 = assessed for all experimental treatments.

10. Record of number and reasons for withdrawal: 0 = no information on withdrawals by group, 1 = withdrawals by group reported without reason and 2 = withdrawals and reason by group reported.

11. Information on comparability and adjustment for differences in analysis: 0= no information on comparability, 1 = some information on comparability with appropriate adjustment and 2 = sufficient information on comparability with appropriate adjustment.

12. Inclusion of all subjects in analyses: 0 = less than 95% of subjects included (intention-to-treat analysis) and 2 = 95% or more included.

13. Presentation of results with inclusion of data for re-analysis of main outcomes (e.g. standard deviations): 0 = little information presented, 1 = adequate information and 2 = comprehensive.

14. Appropriate statistical analysis (including correction for multiple tests where applicable): 0 = inadequate, 1 = adequate and 2 = comprehensive and appropriate.

15. Conclusions justified: 0 = no, 1 = partially and 2 = yes.

Quality assessment for case series

Scoring: yes = 2, unclear = 1 and no = 0 (total score = 0–38).

• Is the hypothesis/aim/objective of the study clearly described?

  • Yes: the hypothesis/aim/objective of the study is clearly reported.

  • Unclear: the hypothesis/aim/objective of the study is vague or unclearly reported.

  • No: the hypothesis/aim/objective is not reported.

• Are the characteristics of the participants included in the study described?

  • Yes: the most relevant characteristics of the participants are reported (e.g. the total number, age, and gender distribution). Ethnicity, severity of disease/condition, comorbidity, or aetiology should also be included, if relevant.

  • Partially reported: only the number of participants was reported.

  • No: none of the relevant characteristics of the participants is reported.

• Were the cases collected in more than one centre?

  • Yes: cases are collected in more than one centre (multicentre study).

  • Unclear: unclear where the patients come from (i.e. single or multicentre study).

  • No: cases are collected from one centre.

• Are the eligibility criteria (i.e. inclusion and exclusion criteria) for entry into the study clearly stated?

  • Yes: both inclusion and exclusion criteria are reported.

  • Partially reported: only one, the inclusion or exclusion criteria are reported.

  • No: neither inclusion nor exclusion criteria are reported.

• Were participants recruited consecutively?

  • Yes: there is a clear statement or it is clear from the context that the participants were recruited consecutively or study stated that all eligible patients were recruited.

  • Unclear: the method used to recruit participants is not clearly stated or no information is provided about the method used to recruit participants in the study.

  • No: the cases studied were a subgroup of those treated with no evidence to show that they were selected consecutively. The participants were recruited based on other criteria such as access to intervention determined by the distance or availability of resources.

• Did participants enter the study at a similar point in the disease?

  • Yes: there is a clear description about all participants entering the study at a similar point in the condition/disease based on their clinical status, duration of condition or exposure before the intervention, severity of disease, and presence of comorbidities or complications.

  • Unclear: there is no description of the characteristics of participants before entering the study or there is no statement about entering the study at a similar point in the disease.

  • No: participants did not enter the study at a similar point in the condition/disease. This can be revealed by a wide range of disease durations before entering the study or different levels of severities or comorbidities or complications due to progression of their condition/disease.

• Were additional interventions (co-interventions) reported in the study?

  • Yes: participants received additional co-intervention(s).

  • Unclear: it is suspected that a co-intervention was administered but the information is not reported.

  • No: there is a clear statement or it is clear from the context that a co-intervention was not administered.

• Are the outcome measures established a priori?

  • Yes: all relevant outcome measures are reported in the introduction or methods section (e.g. accomplished, measurable improvements or effects, symptoms relieved, improved function, improved test scores, and quality-of-life measures).

  • Partially reported: some of the relevant outcomes are briefly reported in the introduction or methods section.

  • No: the outcome measures are reported for the first time in the results, discussion, or conclusion section of the study.

• Were the relevant outcomes measured with appropriate objective and/or subjective methods?

  • Yes: all relevant outcomes are measured with appropriate methods, which are described in the methods section. These measures might be objective (e.g. gold standard tests or standardised clinical tests), subjective (e.g. self-administered questionnaires, standardised forms, or patient symptoms interview forms), or both.

  • Unclear: it is unclear how the relevant outcomes were measured. No information is provided on the methods used to measure the study’s relevant outcomes.

  • No: the methods used to measure outcomes were inappropriate.

• Were the relevant outcomes measured before and after the intervention?

  • Yes: the relevant outcomes are measured before and after applying the intervention.

  • Unclear: it is unclear when the outcomes were measured.

  • No: the study reported only outcomes measured after applying the intervention.

• Was the study conducted prospectively?

  • Yes: it is clearly stated that the study was conducted prospectively.

  • Unclear: the design of the study is not mentioned or it is unclear if the study was conducted prospectively.

  • No: the authors clearly stated that it was a retrospective study.

• Were the relevant outcomes assessed blinded to intervention status?

  • Yes: the relevant outcomes were analysed by individuals who were not aware of the intervention status.

  • Unclear: the study did not report whether the outcome assessors were aware of the intervention status.

  • No: it is clearly stated or obvious that the relevant outcomes were analysed by individuals who were aware of the intervention status.

• Were the statistical tests used to assess the relevant outcomes appropriate?

  • Yes: the statistical tests are clearly described in the methods section of the study and are used appropriately (e.g. parametric test for normally distributed population vs. nonparametric test for non-Gaussian population). The reviewer should assign a yes score if no statistical analysis was performed but reasons for this were stated.

  • Unclear: the statistical tests are not described in the methods section of the study or there is no information about the statistical analysis.

  • No: the statistical tests were used inappropriately.

• Was the length of follow-up reported?

  • Yes: the length of follow-up is clearly reported (mean, median, range, standard deviation).

  • Unclear: the duration of follow-up is not clearly reported.

  • No: the length of follow-up is not reported.

• Was the loss to follow-up reported?

  • Yes: the number or proportion of participants lost to follow-up is clearly reported or authors report outcome results on all participants included initially, or number lost to follow-up can be subtracted from the number enrolled and number analysed.

  • Unclear: it is not clear from the information provided how many participants were lost to follow-up or it is an inconsistence of reporting lost to follow-up (e.g. discrepancies between information from tables and text).

  • No: the number or proportion of participants lost to follow-up is not reported.

• Does the study provide estimates of the random variability in the data analysis of relevant outcomes?

  • Yes: the study reports estimates of the random variability (e.g. standard error, standard deviation, confidence interval for parametric data, and range and interquartile range for nonparametric data) for all relevant outcomes.

  • Unclear or partially reported: the presentation of the random variability is unclear (e.g. the measure of dispersion is reported without indicating if it is a standard deviation or standard error). Estimates of the random variability are not reported for all relevant outcomes.

  • No: the study does not report estimates of the random variability.

• Are the adverse events related with the intervention reported?

  • Yes: the undesirable or unwanted consequences of the intervention during the study period or within a pre-specified time period are reported. The absence of adverse event(s) is acknowledged in the study.

  • Partially reported: it is deducible that only some but not all potential adverse events are reported.

  • No: there is no statement about the presence or absence of adverse events.

• Are the conclusions of the study supported by results?

  • Yes: the conclusions of the study (in terms of patient, intervention, outcomes) are supported by the evidence presented in the results and discussion sections.

  • Partially reported: not all components of the patient, intervention, outcomes are supported by the evidence presented in the results and discussion section.

  • No: the conclusions are not supported by the evidence presented in the results and discussion section.

• Are both competing interests and sources of support for the study reported?

  • Yes: both competing interests and sources of support (financial or other) received for the study are reported, or the absence of any competing interest and source of support is acknowledged.

  • Partially reported: only one of these elements is reported.

  • No: neither competing interests nor sources of support were reported.

Quality score for qualitative studies

One point for each endorsed statement based on judgement (total score = 0–87).

1. Findings: how credible are the findings?

   1. Findings/conclusions are supported by data/study evidence (i.e. the reader can see how the researcher arrived at his/her conclusions; the ‘building blocks’ of analysis and interpretation are evident).

   2. Findings/conclusions ‘make sense’/have a coherent logic.

   3. Findings/conclusions are resonant with other knowledge and experience (this might include peer or member review).

   4. Use of corroborating evidence to support or refine findings (i.e. other data sources have been used to examine phenomena; other research evidence has been evaluated: see also statement 14, Reporting: how clear are the links between data, interpretation and conclusions, i.e. how well can the route to any conclusions be seen?).

2. Findings: how has knowledge/understanding been extended by the research?

   1. Literature review (where appropriate) summarising knowledge to date/key issues raised by previous research aims and design of study set in the context of existing knowledge/understanding; identifies new areas for investigation (for example, in relation to policy/practice/substantive theory).

   2. Credible/clear discussion of how findings have contributed to knowledge and understanding (e.g. of the policy, programme or theory being reviewed); might be applied to new policy developments, practice or theory.

   3. Findings presented or conceptualised in a way that offers new insights/alternative ways of thinking.

   4. Discussion of limitations of evidence and what remains unknown/unclear or what further information/research is needed.

3. Findings: how well does the evaluation address its original aims and purpose?

   1. Clear statement of study aims and objectives; reasons for any changes in objectives.

   2. Findings clearly linked to the purposes of the study – and to the initiative or policy being studied.

   3. Summary or conclusions directed towards aims of study.

   4. Discussion of limitations of study in meeting aims (e.g. are there limitations because of restricted access to study settings or participants, gaps in the sample coverage, missed or unresolved areas of questioning; incomplete analysis; time constraints?).

4. Findings: scope for drawing wider inference – how well is this explained?

   1. Discussion of what can be generalised to wider population from which sample is drawn/case selection has been made.

   2. Detailed description of the contexts in which the study was conducted to allow applicability to other settings/contextual generalities to be assessed.

   3. Discussion of how hypotheses/propositions/findings may relate to wider theory; consideration of rival explanations.

   4. Evidence supplied to support claims for wider inference (either from study or from corroborating sources).

   5. Discussion of limitations on drawing wider inference (e.g. re-examination of sample and any missing constituencies: analysis of restrictions of study settings for drawing wider inference).

5. Findings: how clear is the basis of evaluative appraisal?

   1. Discussion of how assessments of effectiveness/evaluative judgements have been reached (i.e. whose judgements are they and on what basis have they been reached?).

   2. Description of any formalised appraisal criteria used, when generated and how and by whom they have been applied.

   3. Discussion of the nature and source of any divergence in evaluative appraisals.

   4. Discussion of any unintended consequences of intervention, their impact and why they arose.

6. Design: how defensible is the research design?

   1. Discussion of how overall research strategy was designed to meet aims of study.

   2. Discussion of rationale for study design.

   3. Convincing argument for different features of research design (e.g. reasons given for different components or stages of research; purpose of particular methods or data sources, multiple methods, time frames, etc.).

   4. Use of different features of design/data sources evident in findings presented.

   5. Discussion of limitations of research design and their implications for the study evidence.

7. Sample: how well defended is the sample design/target selection of cases/documents?

   1. Description of study locations/areas and how and why chosen.

   2. Description of population of interest and how sample selection relates to it (e.g. typical, extreme case, diverse constituencies, etc.).

   3. Rationale for basis of selection of target sample/settings/documents (e.g. characteristics/features of target sample/settings/documents, basis for inclusions and exclusions, discussion of sample size/number of cases/setting selected, etc.).

   4. Discussion of how sample/selections allowed required comparisons to be made.

8. Sample: sample composition/case inclusion – how well is the eventual coverage described?

   1. Detailed profile of achieved sample/case coverage.

   2. Maximising inclusion (e.g. language matching or translation; specialised recruitment; organised transport for group attendance).

   3. Discussion of any missing coverage in achieved samples/cases and implications for study evidence (e.g. through comparison of target and achieved samples, comparison with population, etc.).

   4. Documentation of reasons for non-participation among sample approached/non-inclusion of selected cases/documents.

   5. Discussion of access and methods of approach and how these might have affected participation/coverage.

9. Data collection: how well was the data collection carried out?

   1. Discussion of:

      1. –  who conducted data collection.

      2. –  procedures/documents used for collection/recording.

      3. –  checks on origin/status/authorship of documents.

   2. Audio or video recording of interviews/discussions/conversations (if not recorded, were justifiable reasons given?).

   3. Description of conventions for taking field notes (e.g. to identify what form of observations were required/to distinguish description from researcher commentary/analysis).

   4. Discussion of how fieldwork methods or settings may have influenced data collected.

   5. Demonstration, through portrayal and use of data, that depth, detail and richness were achieved in collection.

10. Analysis: how well has the approach to and formulation of the analysis been conveyed?

    1. Description of form of original data (e.g. use of verbatim transcripts, observation or interview notes, documents, etc.).

    2. Clear rationale for choice of data management method/tool/package Evidence of how descriptive analytic categories, classes, labels, etc., have been generated and used (i.e. either through explicit discussion or portrayal in the commentary).

    3. Discussion, with examples, of how any constructed analytic concepts/typologies, etc. have been devised and applied.

    4. Discussion, with examples, of how any constructed analytic concepts/typologies, etc., have been devised and applied.

11. Analysis: contexts of data sources – how well are they retained and portrayed?

    1. Description of background or historical developments and social/organisational characteristics of study sites or settings.

    2. Participants’ perspectives/observations placed in personal context (e.g. use of case studies/vignettes/individual profiles, textual extracts annotated with details of contributors).

    3. Explanation of origins/history of written documents.

    4. Use of data management methods that preserve context (i.e. facilitate within case description and analysis).

12. Analysis: how well has diversity of perspective and content been explored?

    1. Discussion of contribution of sample design/case selection in generating diversity.

    2. Description and illumination of diversity/multiple perspectives/alternative positions in the evidence displayed.

    3. Evidence of attention to negative cases, outliers or exceptions.

    4. Typologies/models of variation derived and discussed.

    5. Examination of origins/influences on opposing or differing positions.

    6. Identification of patterns of association/linkages with divergent positions/groups.

13. Analysis: how well has detail, depth and complexity (i.e. richness) of the data been conveyed?

    1. Use and exploration of contributors’ terms, concepts and meanings.

    2. Unpacking and portrayal of nuance/subtlety/intricacy within data.

    3. Discussion of explicit and implicit explanations.

    4. Detection of underlying factors/influences.

    5. Identification and discussion of patterns of association/conceptual linkages within data.

    6. Presentation of illuminating textual extracts/observations.

14. Reporting: how clear are the links between data, interpretation and conclusions, i.e. how well can the route to any conclusions be seen?

    1. Clear conceptual links between analytic commentary and presentations of original data (i.e. commentary and cited data relate; there is an analytic context to cited data, not simply repeated description).

    2. Discussion of how/why particular interpretation/significance is assigned to specific aspects of data – with illustrative extracts of original data.

    3. Discussion of how explanations/theories/conclusions were derived – and how they relate to interpretations and content of original data (i.e. how warranted); whether alternative explanations explored.

    4. Display of negative cases and how they lie outside main proposition/theory/hypothesis, etc.; or how proposition, etc. revised to include them.

15. Reporting: how clear and coherent is the reporting?

    1. Demonstrates link to aims of study/research questions.

    2. Provides a narrative/story or clearly constructed thematic account.

    3. Has structure and signposting that usefully guide reader through the commentary.

    4. Provides accessible information for intended target audience(s).

    5. Key messages highlighted or summarised.

16. Reflexivity and neutrality: how clear are the assumptions/theoretical perspectives/values that have shaped the form and output of the evaluation?

    1. Discussion/evidence of the main assumptions/hypotheses/theoretical ideas on which the evaluation was based and how these affected the form, coverage or output of the evaluation (the assumption here is that no research is undertaken without some underlying assumptions or theoretical ideas).

    2. Discussion/evidence of the ideological perspectives/values/philosophies of research team and their impact on the methodological or substantive content of the evaluation (again, may not be explicitly stated).

    3. Evidence of openness to new/alternative ways of viewing subject/theories/assumptions (e.g. discussion of learning/concepts/constructions that have emerged from the data; refinement and restatement of hypotheses/theories in light of emergent findings; evidence that alternative claims have been examined).

    4. Discussion of how error or bias may have arisen in design/data collection/analysis and how addressed, if at all.

    5. Reflections on the impact of the researcher on the research process.

17. Ethics: what evidence is there of attention to ethical issues?

    1. Evidence of thoughtfulness/sensitivity about research contexts and participants.

    2. Documentation of how research was presented in study settings/to participants (including, where relevant, any possible consequences of taking part).

    3. Documentation of consent procedures and information provided to participants.

    4. Discussion of confidentiality of data and procedures for protecting.

    5. Discussion of how anonymity of participants/sources was protected.

    6. Discussion of any measures to offer information/advice/services, etc. at end of study (i.e. where participation exposed the need for these).

    7. Discussion of potential harm or difficulty through participation, and how avoided (guides, observation templates, data management frameworks, etc.).

18. Auditability: how adequately has the research process been documented?

    1. Discussion of strengths and weaknesses of data sources and methods.

    2. Documentation of changes made to design and reasons; implications for study coverage.

    3. Documentation and reasons for changes in sample coverage/data collection/analytic approach; implications.

    4. Reproduction of main study documents (e.g. letters of approach, topic guides, observation templates, data management frameworks, etc.).