Improving the effectiveness of multidisciplinary team meetings for patients with chronic diseases: a prospective observational study

Article history

The research reported in this issue of the journal was funded by the HS&DR programme or one of its proceeding programmes as project number 09/2001/04. The contractual start date was in November 2010. The final report began editorial review in December 2013 and was accepted for publication in March 2014. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

none

Copyright statement

© Queen’s Printer and Controller of HMSO 2014. This work was produced by Raine et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Background

Multidisciplinary team (MDT) meetings are central to the management of chronic disease and they have become widely established across the NHS1–4 and internationally. 5 The proliferation of MDT meetings in health care has occurred against a background of increasingly specialised medical practice, more complex medical knowledge, continuing clinical uncertainty and the promotion of the patient’s role in their own care. In this environment, it is believed that MDT meetings ensure higher quality decision-making and improved outcomes,6–8 for example by providing a better assessment of treatment strategies and safeguarding against errors. 9

However, the evidence underpinning the establishment of MDT meetings is limited and mixed, and the degree to which they have been absorbed into clinical practice varies widely across conditions and settings. 10–12 One recent trial found multidisciplinary care is associated with improved survival for patients with breast cancer. 13 However, another trial comparing multidisciplinary memory clinics with general practitioner care found no evidence of improved effectiveness. 14 Several recent systematic reviews of MDTs working in cancer, mental health and other disciplines have concluded that there is insufficient evidence to determine their effectiveness. 15–17 In the light of this ambiguity, there have been calls for empirical research on MDT meeting decision-making in routine practice to understand how and under what conditions MDT meetings produce effective decisions. 11

Purpose of the research

Chapter summary

In this chapter we have outlined the background to the study and described the relevance to MDT team effectiveness of the national and local contexts within which MDTs operate, of their distinctive features and group processes, and of patient characteristics. We have discussed the complexity of measuring effectiveness in this context, and outlined our rationale for investigating treatment plan implementation. Finally, we outlined the aims and objectives of the study. In Chapters 2 and 3, we discuss the methods through which these aims and objectives were addressed.

Ethics and research governance permissions

This study was approved by East London Research Ethics Committee (10/H0704/68) and the National Information Governance Board (NIGB) for Health and Social Care [Ethics and Confidentiality Committee (ECC) 6-05 (h)/2010]. The NIGB ECC provided permission, under Section 251 of the NHS Act 2006,87 to process patient identifiable information without consent. We gained permissions from the relevant research and development (R&D) departments to collect data following submission of site-specific information forms (using the Integrated Research Application System) at each of the NHS organisations that participated in the study.

Recruitment of multidisciplinary teams

We aimed to include a diverse range of teams and chronic conditions to allow examination of key issues about MDT decision-making and implementation. We sought to include teams which varied in terms of health-care context, professional mix of team, conditions which affected patients from different parts of the adult life range, and fatal versus lifelong conditions. When deciding which and how many chronic diseases to include, we also had to take a pragmatic approach to ensure feasibility, because the data collection phase involved extended periods of observation at multiple sites.

Teams which might have been interested in participating in the study were identified by our clinical coapplicants. The principal investigator (PI) wrote to the lead clinician of each MDT inviting them to take part. This was followed by a discussion with each MDT lead to clarify any issues or concerns.

Following this, the research team visited each MDT meeting to introduce the study. We provided participant information sheets and obtained members’ signed consent to observe, record data from and audiotape MDT meetings (see Appendix 1). Other professionals occasionally attend MDT meetings on an ‘ad hoc’ basis, so we also displayed a printed notice at the entrance to the meeting room during every meeting we recorded. This explained the nature of the observation and provided the researcher’s contact details. We made it clear that if individuals did not wish to take part in the study we would delete their contributions from any transcripts. Nobody requested this.

We initially aimed to recruit 10 teams. This was based on estimates made by our clinical coapplicants of the number of patients discussed in each specialty. In practice, we found considerable differences in the number of patients discussed at each MDT (even within the same specialty), so we had to vary the observation period in each team to maximise data collection in those teams which discussed fewer patients (Table 1). In addition, one mental health team disbanded during the observation period. We therefore asked one of our coapplicants (with a background in mental health) to identify other suitable mental health MDTs. We subsequently recruited an additional two mental health teams (in order to increase our capability of achieving the required sample size), taking the total number of teams in the study to 12. Three field researchers observed a total of 394 meetings (including two meetings for each of the 12 teams where the data collection tools were piloted) and collected quantitative and qualitative data from 370 meetings.

We also planned to include semiurban and metropolitan teams but, in the event, only one MDT from outside London accepted our invitation to participate.

Quantitative methods

We collected both quantitative and qualitative data. In the rest of this section (Quantitative methods) and the next (Qualitative methods) we first describe how we collected and then analysed the quantitative data, and then describe how we collected and analysed the qualitative data.

Qualitative methods

The qualitative aspect of the study served two purposes: firstly, to provide insights into possible explanations for the quantitative findings, and, secondly, to identify areas where there were diverse beliefs or practices with respect to MDT meetings.

To achieve this we collected qualitative data using three data collection methods:

• non-participant observation of MDT meetings

• interviews with MDT members

• interviews with patients.

The steps taken to collect and analyse the qualitative data are described below.

Chapter summary

We began this chapter by outlining the ethics and recruitment procedures undertaken. We then described the data collection and analysis methods for the quantitative components (i.e. non-participant observation of meetings and medical record review) and the qualitative components (i.e. non-participant observation of meetings, interviews with patients and professionals) of the research. Finally we explained how we integrated these data to address the study aims. Chapters 3 and 4 present the quantitative and qualitative results.

Team characteristics

We observed 370 MDT meetings during which 3184 patients were discussed. Characteristics of the 12 teams and their meetings are summarised in Table 7. There was considerable variability among the 12 teams in the number of patients discussed at each meeting. The average for each team ranged from 4 to 49 patients. The average meeting size across the 12 MDTs ranged between 5 and 28 members, with one to seven professional groups represented (an overall median of four). Of the patients discussed, 83% had a least one treatment plan made, but this proportion was notably lower for the mental health teams.

Team Climate Inventory and additional questions

Of the 211 TCIs issued, 161 were returned, giving a response rate of 76%, ranging from 50% (mental health team that disbanded) to 100% (skin cancer).

Across all teams, 155 members responded to the additional question, with almost all agreeing to a great or very great extent with the statement, ‘I believe that the [team name] MDT meetings are an effective use of my time’ (Figure 1).

FIGURE 1.

MDT member response to ‘I believe that MDT meetings are an effective use of my time’ across 12 MDTs.

Treatment plan implementation

Overall, 2654 patients had a treatment plan. The number of decisions making up the plan ranged between 1 and 11 (Figure 2) with an average of 1.6 decisions per plan.

FIGURE 2.

Number of decisions making up the first treatment plan recorded for each patient. Average number of decisions = 1.6, number of decisions for first treatment plans = 4127.

Of the patients with treatment plans, 2512 (94.6%) had adequate implementation data recorded for implementation status to be classified, and 1967 (78.3%) of these had implemented plans. Treatment plan implementation is summarised by patient, MDT and discussion characteristics in Table 8. By disease type, implementation was highest in the gynaecological cancer team and lowest in the CMHTs, although implementation also varied by team. There was a trend for non-implementation with increasing patient deprivation.

Adjusted associations (Table 9) showed no evidence of a relationship between treatment plan implementation and patient’s age or sex, or with discussion of patient preferences or health behaviours/other clinical factors. ATI and number of professional groups were highly correlated (correlation coefficient 0.8) but did not appear to have an association when fitted together (model 2). When included individually in the model, however, each showed a relationship such that an increase in team diversity or number of professional groups was associated with a reduced odds of treatment plan implementation.

Factors with an association (p < 0.3) in models 1 and 2 were fitted together in the final model (model 3). The adjusted odds of implementation increased by 7% (95% CI 1% to 13%) for a 0.1-unit TCI score increase (indicating improved team climate). In contrast, the adjusted odds of implementation were reduced by 25% for each additional professional group represented, and were lower for patients living in more deprived IMD quintiles. Adjusted odds of implementation also varied by disease type.

We had sufficient data on 2512 (92%) patients to include them in the final model. Data were missing for age (2%), IMD quintile (6%) and treatment plan implementation (5%), mainly because of missing patient notes. The only characteristic associated with ‘missingness’ was MDT disease type. Given the small proportion of missing values, we did not consider it necessary to account for this in our analysis.

Sensitivity analyses for model 3 included adjusting for the number of decisions making up the first treatment plan (which varied between 1 and 11 decisions); examining the impact of using the first recorded treatment plan in analysis by refitting the model based on a randomly chosen treatment plan for each patient; collapsing the five IMD quintiles to two groups (IMD 1–3 and IMD 4–5) to produce an eight-coefficient model; and altering the definition of an implemented treatment plan such that implementation is when > 80% of component decisions are implemented. None of these analyses substantially changed our conclusions (see Appendix 3).

We further explored the observed trend between implementation and number of professional groups to examine whether this was dependent on disease type. Tabulations suggested that this association was mainly apparent in memory clinic and mental health teams, and an interaction term added to model 3 indicated some evidence of a differential effect by type of disease (p = 0.06) (Table 10). We also considered whether the relationship between IMD and implementation differed for cancer and non-cancer specialties. This interaction term was not significant (interaction p = 0.13) (Table 11).

The reasons for non-implementation of treatment plans are summarised in Table 12 by disease type. Although 15% of non-implementations were recorded as for patient, family or carer reasons, ‘other’ reasons for non-implementation were most commonly recorded. The ‘other’ reasons were diverse, and included cases where the decision was implemented outside the 3-month follow-up period or when new information or test results emerged after the MDT meeting, for example where it was subsequently decided that a referral had been inappropriate or that a patient should have been referred to a team in a different catchment area. Patient-led reasons were recorded most frequently in CMHTs (e.g. because of patient non-attendance) while patient death was reported most frequently for heart failure MDTs. We also collapsed the five IMD quintiles to two groups (IMD 1–3 and IMD 4–5) and found that reasons for non-implementation were similar, although patient death was a more frequent reason for non-implementation in the least deprived group while patient non-attendance and changes in circumstances were recorded less for this group (Table 13).

Chapter summary

In this chapter we have quantitatively described characteristics of the teams studied and presented the results of the primary quantitative analysis, assessing factors associated with decision implementation. We also described some of the reasons for non-implementation. Chapter 4 presents the qualitative results. This includes a qualitative description of team characteristics, a qualitative exploration of the quantitative findings, and a qualitative analysis investigating areas of diversity in beliefs and practices relating to MDT meetings.

Data sources

Overview of the multidisciplinary team meetings and their context

Here we present a brief analysis of our observation and interview data to provide an overall description of the teams, and the contexts within which they operated.

In terms of meeting characteristics, overall there was considerable variation between teams, in the size of the teams, the number of patients discussed (as mentioned in Chapter 3) the length of the MDT meetings, the chairing arrangements and the administrative support provided (Table 16).

The context within which each of the 12 teams operated, and attendance and structural characteristics, are described below.

Qualitative exploration of the quantitative results

The main findings from the quantitative analysis formed the basis for the following five deductive themes, which we explored further using our qualitative data:

1. theme 1: influence of disease specialty on implementation

2. theme 2: influence of multidisciplinarity on implementation

3. theme 3: influence of team climate on implementation

4. theme 4: influence of socioeconomic characteristics on implementation

5. theme 5: roles that patient preference, health behaviours and other clinical factors may have in explaining the socioeconomic variations in implementation.

Qualitative investigation of areas of diversity in beliefs and practices

Our synthesis of the observation, professional and patient interview data resulted in 16 metathemes relating to areas of diverse beliefs or practices in MDT meetings (Box 4) (see Chapter 2, Summary measures, for a description of this analysis). Each of the 16 metathemes is discussed below.

Chapter summary

In this chapter we have provided an overview of the teams studied and their different contexts. We then presented findings from our qualitative exploration of the quantitative findings, and from our investigation of areas of diversity in beliefs and practice across the teams. In Chapter 5 we discuss the findings from study 1, consider its limitations and draw conclusions.

Identifying factors that influence decision implementation

Rates of implementation varied between specialties, with gynaecological cancer having the highest implementation rate and CMHTs the lowest. Examining the policy context within which these teams operate provides some insights into these differences. Mental health policy and guidance is much less detailed than that in cancer, in terms of the expected attendance, structure and administrative support at MDT meetings. Cancer teams use clear guidelines regarding the MDT purpose and processes, which translated into clearly structured MDT meetings. These include guidelines on documentation,112 attendance,42 chairing113 and administrative support. 1

The clearly structured, guideline-driven cancer MDT meetings are in no small part a consequence of the historic priority given to cancer research, which has produced a robust evidence base for the care of cancer patients. In contrast to mental health, which received 3.5% of Medical Research Council (MRC) funding in 2008/9, cancer research received 14.6% of MRC funds. 114 This is despite the fact that mental illness accounts for 23% of the overall disease burden in the UK, while cancer accounts for 16%. 48 There have been recent efforts to redress this balance, with MRC spending in 2012/13 rising to 21.5% for neurological and mental health research, and falling to 6.2% for cancer. 115 In time, this shift is likely to strengthen the evidence base for decision-making by mental health MDTs.

Another factor which could be argued to influence rates of implementation for teams focusing on different conditions is the type of patient outcome which the teams focus on. The International Classification of Functioning, Disability and Health (ICF) identifies three levels of human functioning: functioning at the level of body or body part (body functions and structures), the whole person (engagement in activities) and the whole person in a social context (participation in society). 116 The clinical conditions which we studied could be argued to vary with respect to the extent to which the MDT meetings would focus on managing and treating diseases and on facilitating patients’ meaningful participation in society. For those teams whose primary focus is to improve physical functioning, universally relevant treatment guidelines are more commonly available. Teams which also place a heavy emphasis on assisting patients’ participation in society may need to develop more individualised treatment plans. Treatment plans relating to an individual’s participation in society may also be more inherently difficult to implement, because they are more likely to rely on multiagency collaboration and complex social and cultural factors.

Contrary to previous reports and expectations, greater multidisciplinarity was not necessarily associated with more effective decision-making. This has important implications, as a major assumption underpinning MDT meetings is that more multidisciplinary input leads to better and more informed decisions for patients, with each team member bringing relevant information for the development of a cohesive care plan. 117 Our data suggest that the relationship between multidisciplinarity and MDT decision implementation is mediated by other factors such as clarity of purpose and agreed processes. This is supported by evidence which points to the benefits of shared objectives to guide and structure communication,118 focused leadership119 and team reflexivity. 21

It is also noteworthy that the teams which implemented the most decisions tended to be the most medically dominated in terms of both attendance and participation. Higher implementation rates among more homogeneous (medically qualified) groups are perhaps unsurprising given a higher likelihood of a shared biomedical perspective which drives the decisions made and implemented. However, the inclusion of additional perspectives in the decision-making process may improve decision quality with respect to patient experience, choice and patient-centred outcomes. It was beyond the scope of this study to examine such components of decision-making, but it remains an important area for future investigation.

Effective teamwork is therefore necessary for the potential benefits of multidisciplinarity to be achieved. MDT members identified difficulties in working across disciplinary boundaries, in terms of both status differences and differing professional perspectives. In line with other research, we found that not all professional groups participated equally in MDT discussions. 120 Cancer and heart failure teams in particular were characterised by a strong medical dominance which was manifest both in the degree of participation and in the seating arrangements. We found some evidence that conflicting disciplinary perspectives made multidisciplinary working a challenge, an issue that has been identified in previous research. 21,121 For example, mental health professionals identified differences in how social workers and nurses viewed their roles (e.g. as healers or as human rights advocates), which sometimes made it difficult to collaborate. In the absence of clear national guidance, team reflection on the assumptions underpinning each disciplinary perspective and clarity regarding the role of each professional group may be necessary to develop a shared perspective and agreement on the purpose of the meetings.

We also found that a positive team climate was an important predictor of treatment plan implementation across teams. This finding corroborates previous research which has demonstrated that a good team climate can translate to better decisions. 23,122 It also adds to the body of evidence demonstrating the effects of local context on team performance34 and supports research that has highlighted the importance of positive relationships, teamwork and supportive systems to effectiveness. 36 It was clear from our qualitative data that local context, in this case restructuring in mental health teams, had a significant impact on staff morale and many believed it hindered team performance. In examining the different team structures and processes in the light of their varied national guidance and standards, our data also give an indication of how the broader policy context impacts on effectiveness.

Patient preferences, comorbidities and socioeconomic variation

Despite clear policy commitments to promote equality in the NHS58 we found that there were variations in treatment by patients’ socioeconomic circumstances. This was evident in all specialties and could not be explained by consideration of patient preference, comorbidities or other health-related factors in team meetings. Previous research reporting socioeconomic inequalities in use of adjuvant therapy for cancer suggests that non-implementation is due to non-uptake of care for cancer patients. 72 This is likely to partly explain the continuing socioeconomic variations in cancer survival. 123 There is little research on treatment inequalities for people with dementia, although a small study indicates that these may occur124 and the evidence in mental health has been mixed. 65,67 Recent models have demonstrated equitable provision of cardiovascular care which translated to similar proportions of deaths averted across deprivation quintiles. 125 However, non-implementation may impact upon morbidity rather than mortality, which was not addressed in these cardiovascular models.

Cancer policy states that patients’ comorbidities, preferences and circumstances are important for decision-making and that MDTs should take these into consideration wherever possible. 113 Guidelines for mental health, memory and heart failure state that patient preferences should be taken into account when managing their care, though they do not relate this specifically to the MDT meeting. 49,77,126 Research in this area has called for routine discussion of patient preferences and comorbidities, as evidence suggests that MDT decisions are often not implemented because of a failure to consider patient preferences during the meeting. 127,128 Inadequate information on comorbidities in MDT meetings has also been associated with failure to make decisions. 129,130

Contrary to these previous studies, we found no association between discussion of patient preferences or comorbidities and treatment plan implementation. However, the qualitative results showed that these were complex issues and there was diversity in how readily this information was available, and that there were differing opinions regarding the most appropriate way of incorporating this information into decision-making. Overall, both MDT members and patients stated that patient preferences and comorbidities were very important for decision-making and that they should be included in MDT discussions when possible. However, some suggested that treatment recommendations should initially be independent of patients’ preferences, which was, in line with previous research,131 partly attributed to the fact that patient preferences can change during the management journey. The belief that patients’ views should be sought after the MDT meeting (to ensure clarity and feasibility of options) rather than beforehand has also been highlighted in a previous study. 120

Areas of diversity in beliefs and practices

We identified substantial diversity across teams in the purpose, structure, processes and content of MDT meetings, and in levels of patient involvement. While such diversity may be appropriate to the various needs of different teams, it may also indicate a lack of clarity regarding best practice and a need for further guidance.

Meetings served a range of explicit and implicit functions, from decision-making to team bonding and emotional support. Clarity of purpose has been identified as an essential to effective teamworking and high quality patient care. 132 In line with this previous research, we found that in mental health teams the purpose of meetings was implicit rather than explicit and that there was rarely a written agenda. Though almost all interviewees found the meetings valuable, several mental health interviewees noted the need for greater clarity around the purpose of meetings. Previous research has also identified the potential benefits of using the meeting for functions such as teaching and recruiting patients to trials. 25,133 We identified mixed views regarding these functions. While generally these were considered beneficial, there were practical issues relating to time constraints and resource issues that limited their potential.

The structure and processes of meetings varied widely between specialties. This partly reflects the different policy contexts described above in which the teams operate. We also found wide variation within specialties where there were no national guidelines for MDT meetings, particularly memory clinic and heart failure teams. These variations highlighted the importance of certain team features, including the availability of administrative support.

The importance of administrative support for MDT meetings was clear from both interviews and observations. Previous research in cancer has emphasised the essential role of the MDT co-ordinator role in MDT decision-making134,135 and they were resourced in the cancer teams we observed. In heart failure, memory clinics and mental health, administrative duties were undertaken by managers, health-care professionals and administrators. Many MDT members in these teams highlighted difficulties in having to undertake administrative tasks while participating as professionals. This had an impact on how patients were selected for discussion, and how decisions were documented, among other things. Previous research has found that meeting processes are crucial to quality-monitoring and effectiveness, for example in terms of the documentation and follow-up of meeting outcomes. 31,113,136 Previous research has also identified advantages in collating lists of patients for discussion in advance of the meeting. Registering patients for MDT discussions can form part of a monitoring system that can reduce delays to treatment. 137

The content of discussions also varied by specialty. There was diversity between teams in how much time they spent discussing patients’ psychosocial issues and preferences. This was much more common in memory clinic and mental health teams (both staffed by mental health professionals), while cancer and heart failure meetings were more focused on biomedical information. This was despite the fact that previous research in cancer has found that 98% of professionals surveyed believed that psychosocial care issues should always be considered in MDT meetings. 25 Where patient preferences were mentioned, there were differences between specialties in the kinds of preferences considered. For example, in cancer, discussions of preferences focused on different treatment alternatives, while, in mental health, discussions of choice tended to relate to the timing and delivery of treatment rather than choosing between treatments. Again, this may reflect contextual differences, as lack of resources was a common issue in mental health team discussions.

There was variation in the role of the patient in MDT meetings. While cancer patients are routinely informed of MDT processes, several patient interviewees from other specialties were unaware of the MDT meeting. Policy variations are also noteworthy here. Cancer policy states that patients should be given a written description of the MDT meeting at which they are discussed,42 and should be informed of the purpose of the MDT, when it meets,113 its membership and the responsibilities of each team member. 41 It also states that patients should be informed of the outcome of discussions within a locally agreed timeframe. 113 In contrast, policy in mental health, heart failure and memory clinics does not specify information that should be given to patients on MDT meetings. The idea that their treatment plans were discussed by a team gave some patients confidence, while others were concerned that they might be misrepresented at meetings. In line with previous research, most patients and staff believed it would be impractical for patients to attend the meetings25 and patients varied in their preferences regarding the format and content of the information they would like fed back to them after the meeting. While previous research has found that clinicians tend to underestimate the amount of information that patients require,138,139 it has also been found that these preferences vary at different stages of illness,140 meaning an ongoing dialogue is required to determine a patient’s current informational needs.

Study limitations

The 12 participating teams were based in a single geographical area and, with the exception of one, were all in London. This may limit the generalisability of our results. However, when designing this research, we had to address the issue of feasibility. The researchers attended weekly MDT meetings and spent long periods of time at hospital sites collecting interview and medical record data. They therefore needed to be located within a reasonable distance of the study sites. It would have been difficult to provide close and constructive supervision if researchers were based in geographically dispersed parts of the country. The alternative would have been for the researchers to travel from London, but this would have impacted on the time and cost of the study.

The use of decision implementation as an indicator of effectiveness raises several issues. Firstly, we recognise that measuring treatment plan implementation does not address the issue of whether or not the ‘right’ decisions have been made. It is possible that poor decisions are implemented. In order to examine whether or not implemented decisions are ‘good decisions’ that do indeed lead to patient benefit, it would be necessary to analyse carefully identified disease-specific health outcomes that can be plausibly and directly related to the MDT decision. Alternatively, a subset of decisions about which there are specific best practice guidelines could be assessed according to whether or not they conform to this guidance.

Secondly, the factors we considered in our analyses were identified as potentially important based on previous research. We appreciate that there may be other, unmeasured factors that could also have influenced implementation.

Thirdly, we did not directly explore determinants of decision implementation or the extent to which ‘good’ decisions were made during our interviews with health professionals. This was because interviews took place towards the end of the observation period for each team and making participants aware that implementation was under investigation may have influenced their decision-making processes and practice.

Fourthly, in our regression analysis, we combined decisions where implementation was ‘not documented’ with those decisions that were ‘not implemented’. This was because, on the basis of our clinical experience, non-implementation is commonly not explicitly recorded in patients’ records. Our conclusions are therefore limited by the accuracy of record-keeping.

Finally, multiple treatment plans recorded for the same patient over the observation period could not be satisfactorily included in our models. Therefore, our main analysis considered only the first treatment plan recorded for each patient. We examined the impact of this in our sensitivity analysis.

It is also possible that team members behaved differently when under observation. However, this was mitigated by the 2 weeks of observation before data were collected. In addition, MDT members were aware that decision-making was being studied, but not that implementation was being examined.

The field researchers were not clinicians, but there were several quality assurance procedures in place to safeguard against this potential limitation, as described in Chapter 2, Quality assurance: clinician validation.

We did not achieve the target sample size of 3000 individual patients with treatment plans. This is because fewer patients than expected had treatment plans formulated at each meeting. However, even with our smaller sample, investigation into model overfitting did not raise concerns.

There is no consensus on the best measure of skill mix or team diversity. 98 We therefore categorised professionals into coherent groups on the basis of our clinical expertise. It was necessary to group allied health professionals together as a result of low numbers. We did not account for the seniority or qualifications of members in the different professional groups, which may have had a bearing on the degree to which each profession participated in discussions.

We also used a well-established area-based measure of socioeconomic characteristics, commonly used where individual level indicators are not available. This method rests on the assumption that individuals conform to the socioeconomic profile of their residential area. We recognise that misclassifications can cause under- or overestimates of the relation between socioeconomic circumstances and implementation.

Mental health patients are under-represented in the interview component of our research because, despite strenuous attempts by the research team, we were able to recruit only two mental health patients. Their experiences were distinct from the other patients we interviewed; for example, neither was aware that their care was being discussed in MDT meetings. An in-depth comparison of our interview data from cancer and heart failure patients with mental health and memory clinic patients further explores patients’ experiences. 141

Finally, we did not collect cost or waiting time data and so we cannot comment on the implications of MDT diversity with respect to MDT cost-effectiveness or consequences for patients.

Conclusions

Multidisciplinary team meetings for the management of chronic diseases are widely established in the NHS and internationally. This study builds on previous research in a number of important ways. It is the first to include MDTs for a range of different conditions, generating findings with relevance across health services for a diverse range of patients. We found that the effectiveness of MDT meetings (as indicated by decision implementation) depends on a range of contextual factors, team features and patient characteristics. Having a clear purpose, adequate administrative support and a team atmosphere that facilitates involvement, task orientation and support for improvement is key to ensuring team decisions are implemented. The benefits of multidisciplinarity per se should not be assumed; clear goals and procedures are crucial. These findings corroborate previous work which points to the benefits of shared objectives to guide and structure communication,118 focused leadership119 and team reflexivity. 21

We believe this is the first study to explore the influence of patients’ socioeconomic circumstances on MDT decision implementation. We found that patients from more deprived areas are less likely to have their MDT treatment plans implemented, highlighting continuing inequalities in NHS care provision. This occurred despite the routine reference to treatment guidelines by cancer teams, and did not seem to be explained by consideration of patient preference, comorbidities or other health-related factors. Further research is necessary to understand this. Routine monitoring of decision implementation is one approach that could be used to ensure the equitable provision of care.

We also found many areas in which there is a diversity of views and practices across MDT meetings. While this diversity may reflect appropriate flexibility with respect to the purpose of meetings and their value for team members, it may also indicate uncertainty regarding the most successful ways to conduct MDT meetings.

Chapter summary

In this chapter we have discussed the findings from study 1 and related them to previous research. We have also considered the limitations of the study. The next chapter describes how we used these data, in combination with other research and policy documents, to derive a series of recommendations that would be generalisable across MDTs for different chronic diseases.

Introduction

The aim of study 2 was to apply a modified formal consensus technique to derive a set of feasible recommendations for improving the effectiveness of MDT meetings for patients with chronic diseases. This chapter sets out how the research team, under the guidance of the steering group, used the quantitative and qualitative findings from study 1 to derive a series of recommendations that would be generalisable across MDTs for different disease types.

Formal consensus methods are structured facilitation techniques that explore levels of consensus among a group of experts by synthesising their opinions. 142 They are designed to minimise some of the limitations associated with group decision-making. In contrast to informal decision-making groups such as committees, they follow explicit methodological steps that can be replicated. They have been used to formulate clinical practice guidelines143 and to establish national research priorities. 144 They are particularly useful in defining levels of agreement where there is an insufficient or contradictory evidence base,145 and where there is uncertainty about the value of different options. 146 These methods therefore constitute a valuable tool for agreeing recommendations for improving MDT meetings, where the empirical evidence informs but does not clearly stipulate the action to be taken to improve effectiveness.

The three consensus methods most commonly used in health-care research are the nominal group technique (NGT), the Delphi method and the RAND/UCLA appropriateness method (RAM). 147 In the NGT, experts independently generate ideas, meet to discuss each of the ideas generated and then privately rank them in order of preference. 148 In contrast, the Delphi method does not require the participants to meet. It involves two or more rounds of questionnaires and may start with the generation of ideas. Responses from each round are aggregated and fed back, giving participants the opportunity to revise their answers in the light of the responses of the other participants. 149 The RAM was developed to determine the appropriateness of particular health-care interventions by combining the best available evidence with collective expert judgements. 150 This method involves sending a literature review and list of possible indications for intervention to participants, who independently rate each item. They then meet to discuss areas of discrepancy, with the aid of a second-round questionnaire showing both their own initial rating and the distribution of all first round ratings. Following this, they rerate the items privately and individually, in the light of the discussion. In practice, formal consensus studies often adapt elements from each of these methods to optimally address specific research objectives. 146

Methods

We focused on those areas where the research evidence did not provide clear recommendations on practice to improve effectiveness and where we identified diversity in both MDT practice and in beliefs about best practice. We used a two-round process illustrated in Figure 3 and described in detail below.

FIGURE 3.

Overview of the consensus development method.

We use the term ‘recommendation’ throughout the report because this was the terminology used in discussion with the expert panel members. We have therefore continued to use the term throughout the methods and results sections of this report. However, for our final conclusions, we have named the recommendations that were scored with strong agreement and low variation in agreement as ‘indications of good practice’. This reflects the fact that they are designed to be helpful pointers, rather than policy recommendations.

Generating statements for the expert panel to rate

We used the following sources to generate statements for the expert panel to rate.

Study 1: qualitative findings

We drew upon the 16 metathemes which described areas of diversity in beliefs or practices (see Chapter 4, Qualitative investigation of areas of diversity in beliefs and practices) to generate a list of issues and questions which could be turned into single-item statements. This took place during an analytic conference attended by the field researchers, the chief investigator, two coapplicants and two patient representatives. When exploring these issues we focused on:

• identifying specific practices or procedures that were considered effective in some teams but not used in others

• identification of specific arguments for and against current MDT practices, issues relating to the content of discussion, and reflections on team structure that were made by MDT professionals and patients

• proposals made by MDT professionals and patients for improving MDT meetings and the experience of patients

• suggestions made by MDT professionals and patients for incorporating patient preferences into MDT discussions.

Other published research and guidance

We conducted a review of UK-based research literature published between 1995 and May 2013 and of national policy and guidance (for cancer, heart failure, old age psychiatry and mental health MDT teams) on issues related to the 16 metathemes.

Study 1: quantitative findings

We also drew upon information from our quantitative results, for example we found no association between discussion of patient preferences and treatment plan implementation.

Using these data sources we developed an initial series of statements describing potential ways to improve the effectiveness of MDT meetings. These statements were refined in further analytic conferences, with all members of the research team present to ensure that they were supported by the data. We also ensured that each statement was clear and precise and included only one issue for consideration.

In total we generated 68 statements for the expert panel to rate and four open-ended questions where we did not have enough evidence to propose a statement. In these cases we asked the panellists to provide suggestions on feasible methods to improve effectiveness, for example how to incorporate teaching into MDT meetings.

The questionnaire

The final questionnaire was divided into 16 sections, each relating to a theme (a sample theme is available in Appendix 11; all questionnaire items are presented in the results tables in Appendices 12–15 and the entire questionnaire pack is available from the authors on request). Each section summarised the information outlined above, that is relevant policy and guidance, published research literature and our results. This was followed by a series of statements. We instructed panellists to rate their level of support for each statement by drawing on their own knowledge and experience in addition to the information provided. 151 Ratings were on a Likert scale, from 1 to 9, where a rating of 1 indicated that the panellist strongly disagreed with the statement, a rating of 5 indicated neither agreement nor disagreement (i.e. depends on circumstances), a rating of 9 indicated strong agreement, and ‘don’t know’ indicated that participants did not think they were informed enough to answer the question. An example of a statement is provided in Figure 4.

FIGURE 4.

A questionnaire item from round 1.

We piloted the questionnaire with a member of the steering group to ensure that the information was presented coherently and clearly and to provide an estimate of the time required to complete it.

Formal consensus development process

Expert panel round 1: consensus development questionnaire

Ten weeks before the consensus meeting, all members of the panel were sent the first round questionnaire pack with instructions and a stamped addressed envelope. They were asked to return the completed questionnaire within 1 month (see Appendix 17 for the invitation and instructions).

We contacted panellists who returned questionnaires with missing data and asked them to complete the remaining questions.

Expert panel round 2: consensus development meeting

The ratings from round 1 were used to develop a personalised version of the questionnaire for each panel member (Figure 5). These showed the participant’s own responses (in green) and the distribution of responses for all panellists for each item (in italics above the Likert scale).

FIGURE 5.

A questionnaire item from round 2 showing the distribution of round 1 responses in italics above the Likert scale, and the respondent’s own round 1 rating in green.

We grouped together all the responses from the four open-ended questions, and included a summary of these in the second-round questionnaire. For one of the questions, we used the suggestions to amend an existing statement, and flagged this up to participants so they could rate it during the consensus development group meeting.

This information was distributed to the panellists when they attended the consensus meeting in central London, chaired by the chief investigator, who is experienced at facilitating formal consensus meetings. The purpose of the meeting was to discuss those statements where there was a lack of consensus, to explore causes of divergent responses and to identify any statements where lack of consensus was secondary to different interpretations of the statements. Therefore, statements were ordered for discussion so that those with the least consensus (defined using RAND guidelines, see Data entry and analysis below) were discussed first, and those which had strong consensus were considered last. Thus, during the meeting, priority was given to discussing the statements where there was most variation in scoring among expert panel members. For these statements, panellists were encouraged by the facilitator to discuss reasons for differences in ratings before rerating each item privately. The facilitator ensured that all participants had an opportunity to contribute during the meeting, and made it clear that participants did not need to conform to the group view. 145

Where it became apparent that differences in first-round ratings had resulted partly from ambiguity in the wording of the statement, the group agreed a revised wording before making their second-round rating. Where there was consensus in the first-round ratings, statements were not discussed individually, although the broader discussion usually touched on the themes in these statements and panellists were given the opportunity to comment on any of these statements before rerating.

With the consent of all the panellists, the meeting was audiotaped and field notes were taken to ensure we could correctly identify the specialty associated with each justification for variations in agreement with statements.

The meeting lasted 3.5 hours. At the end of the meeting, we ensured that panellists had completed the entire questionnaire.

Results

Results from the second round of ratings are summarised here and presented in full in Appendices 12–15, 18 and 19. Of the 68 potential modifications to improve the effectiveness of MDT meetings, only one needed to be reworded before rerating.

There were 21 statements for which there was both strong agreement (median ≥ 7) and low variation in the extent of agreement (MADM score of < 1.11). This included six recommendations relating to the purpose of the meetings, ten relating to meeting processes, two relating to content of the discussion and three relating to the role of the patient. Panellists from all specialties agreed that these recommendations were desirable and feasible. The list of these final indications of good practice is provided in Box 5.

Discussion

We used formal consensus development methods to produce a list of indications of good practice for ensuring the effectiveness of MDT meetings, which expert panellists agreed were both feasible and desirable. Of 68 potential modifications proposed, almost one-third (31%), were rated with ‘strong agreement’ by experts from all three disease types (cancer, heart failure and mental health). While the research to date on improving the effectiveness of MDT meetings has focused on teams within one disease type,9,17,154 our findings illustrate that there is scope for learning between specialties and the potential to determine a significant number of indications of good practice that are applicable in the varied contexts within which MDTs operate. Our group-level findings with respect to those 17 potential recommendations with an overall panel rating of ‘uncertain’ suggest that, in these areas, disease-specific recommendations may be more appropriate.

The largest category of indications of good practice where there was cross-specialty agreement related to MDT processes (10 of the 21 recommendations). Our findings concur with other studies that have demonstrated the importance of clear documentation of meeting outcomes (recommendations 13, 15 and 16, Box 5)113,136 and regular review of meeting objectives (recommendations 11 and 12, Box 5). 21,118 While cancer MDTs already follow, and are audited against, national guidelines that explicitly address these issues by requiring, for example, recording of patients’ treatment plans, other MDTs do not. 10,42 This clarity of structure and processes is reflected in our findings from study 1, with similarities between the cancer teams but greater variation between teams in other disease specialties. Our findings from study 2 demonstrate agreement that it is feasible and desirable for other chronic disease MDTs to adhere to a number of these processes too. Notably, many of these indications of good practice require minimal additional financial resources, indicating that improvement is possible even in resource-stretched teams.

Our findings from study 1 highlighted clarity of purpose as an important area of diversity in MDT practices. Clarity of purpose has also been identified in previous research as a key feature of effective teamworking in health care. 21,154 In study 2 we showed that, while consensus was achieved about the primary objective of MDT meetings, and about the need to include both locally and nationally determined goals, there was uncertainty or disagreement about other functions such as teaching.

Previous research has also emphasised the importance of considering patient preferences, arguing that they are associated with MDT decision implementation rates. 120,127,128 The expert panel discussed the importance of knowing patient preferences before MDT meetings and of shared decision-making. Much of the discussion highlighted the complexity of these issues in terms of the most appropriate and sensitive times to involve patients, practical constraints (such as urgent, last-minute inclusion of patients in MDT discussions where there has been no time to obtain their preferences) and disease-specific concerns (such as difficulties faced by some mental health patients in making decisions about their treatment). The panel therefore recommended that, when making a decision, the MDT should actively seek all possible treatment options and discuss these with the patient after the meeting.

The feasibility and acceptability of giving patients the option to attend MDT meetings has been explored in the Australian healthcare system. 155,156 However the consensus panel rejected this proposal because of logistical difficulties and because these meetings were not considered to be an appropriate forum for breaking bad news.

The calculation of condition-specific medians highlighted 17 areas where disease-specific recommendations may be required. For example, while the cancer panellists argued strongly for national agreement on at least a common core of required professionals at every MDT meeting, the heart failure panellists suggested that it was more appropriate for their MDT attendees to reflect the flexibility inherent in the variable composition of MDTs across the country. Similarly, mental health panellists considered it to be imperative for someone with personal knowledge of a patient be present when that patient is discussed by the MDT, but this was believed to be unnecessary in cancer and heart failure MDT meetings.

Study limitations and strengths

Important strengths of our study include our calculation of both the strength and extent of agreement for each recommendation and our ability to understand the reasons given for each rating by conducting a thematic analysis of the panel discussions. Our consideration of results by disease group allowed us to distinguish between recommendations which might be applied generically across conditions and those which might be more appropriately dealt with on a disease-specific basis. However, these results should be treated with caution, as they represent the views of small numbers of disease-specific participants.

The three consensus methods most commonly used in health-care research are the Delphi method, the NGT and the RAM. In practice, formal consensus studies often adapt elements from each of these methods to optimally address their specific research objectives. 146 In the Delphi method, participants rate recommendations over a series of rounds using questionnaires. They do not meet to discuss their responses, but a summary of previous ratings is provided in each successive round to allow them to take them into consideration into their ratings. In contrast, the NGT convenes a panel of participants who list their ideas on a particular subject and then share them with the group in a structured meeting. After discussion, participants individually rank-order each topic. 146,148 We chose to use the RAM because the forum for detailed discussion allowed us to gain valuable data on panellists’ reasoning, and their ratings allowed us to statistically summarise both the strength and extent of agreement for each recommendation discussed.

Another strength of our study was the size and diversity of our consensus group. The inclusion of 16 participants meant that we were able to include a range of representatives from each disease speciality. Had we convened a larger panel, it is likely that the ability of every panellist to contribute fully to the discussion of each recommendation would have been diminished. However, we recognise that a consensus panel of this size raises questions around the reliability of the results. Another group of different experts might have produced different recommendations. 150 This is particularly relevant for mental health and heart failure MDTs, which, unlike cancer MDTs, are not required to follow guidance about core membership. 41,42 We therefore aimed to achieve a balance between including as many stakeholders as possible (including patient and policy representatives) and ensuring that the size of the panel allowed all participants sufficient opportunities to be heard. In the event, not all the professionals whom we approached accepted our invitation to participate, which meant that we were unable to represent all relevant professional groups (e.g. psychologists). We could have improved within- and between-group reliability by using the results of this study as the basis for a large-scale Delphi survey. 157 However, we did not have the resources to do this.

In terms of patient representatives (of which there were three), the larger proportion of health-care professionals in attendance may have acted as a barrier and prevented the views of patient representatives from being heard. Given the practical challenges described above relating to the size and diversity of the panel, the chairperson sought to overcome this limitation by actively involving all panellists in discussion. It was also made clear throughout the meeting that there were no ‘right’ or ‘wrong’ answers. In addition, by reporting variation in agreement as well as strength of agreement, we were able to take account of differences between panellists, and our final 21 indications of good practice included only those where there was agreement and low variation in agreement.

It is also possible that the framing of the evidence and recommendations could influence judgements. 158 We attempted to address this by using the meeting to identify any differing interpretations of the evidence and recommendations. This resulted in the rewording of just one of the recommendations to clarify its meaning (recommendation 19, see Box 5).

Finally, the 68 recommendations were derived from research evidence and policy guidance. Had we chosen the NGT, in which panellists begin by listing their ideas, additional and perhaps more innovative potential recommendations might have been discussed. However, this approach might also have led to less consensus.

Conclusions

Multidisciplinary team meetings for patients with chronic diseases are well established across the NHS. However, evidence for their effectiveness is mixed and their structures and processes vary widely. Given that MDT meetings are resource-intensive their value to the NHS and patients should be maximised. Previous research on improving MDT meetings has examined specific diseases in isolation,9,17,154 limiting the potential for learning across different specialties. In addition, much of the evidence regarding features of effective teamworking in health care has not been translated into practical recommendations. We used consensus development methods to build on our findings from study 1, and our expert panel identified 21 feasible and desirable indications of good practice to improve the effectiveness of MDT meetings which are applicable to a range of chronic diseases. These can be directly applied in NHS practice and quality-monitoring initiatives. The fact that no single team from the 12 teams that we observed in study 1 met all of the indications agreed suggests that there is room for improvement in all specialties, even those that already have clear policies and procedures. We also identified uncertainty and disagreement regarding a number of issues, including the best ways to involve patients, which patients should be discussed and meeting membership. These issues require further investigation, and disease-specific or locally determined guidance may be necessary.

Chapter summary

This chapter has presented study 2 in its entirety, describing how we used a formal consensus development method to derive a set of feasible recommendations for improving the effectiveness of MDT meetings for chronic diseases. In Chapter 7 we discuss the project as a whole, bringing studies 1 and 2 together to consider the key findings and implications for future research.

Summary of key findings

• We found that greater multidisciplinarity is not necessarily associated with more effective decision-making; rather it is mediated by having a clear purpose, agreed processes and a team atmosphere that facilitates inclusion and improvement.

• Overall, 78% of MDT decisions across the 12 chronic disease MDTs under study were implemented.

• CMHTs implemented fewer decisions than did other teams. Staff in these teams reported a wide array of functions of MDT meetings in addition to decision-making; however, some reported that meetings lacked clarity of purpose.

• Teams differed widely in relation to the format and structure of meetings, documentation and audit procedures, choice of patients for discussion, the content of discussions and the use of technology.

• While team members reported that they valued hearing a range of disciplinary perspectives, not all disciplines were perceived to have an equal ‘voice’.

• Some teams were characterised by a strong medical dominance in terms of attendance and participation. While these teams typically made and implemented high numbers of treatment plans, those plans were less likely to have incorporated the full range of disciplinary perspectives.

• Patients from more deprived areas were less likely to have their treatment plans implemented and this occurred despite the routine reference to treatment guidelines by cancer teams. Consideration of patient preference, comorbidities or other health-related factors did not seem to explain this, and we were unable to account for these findings.

• Stakeholders with expertise in cancer, mental health and heart failure agreed on 21 indications of good practice, which were applicable to all the chronic diseases considered. They included recommendations relating to the purpose of the meetings (e.g. that agreeing treatment plans should take precedence over other objectives); meeting processes (e.g. that MDT decision implementation should be audited annually); content of the discussion (e.g. that information on comorbidities and past medical history should be routinely available); and the role of the patient (e.g. concerning the most appropriate time to discuss treatment options). Panellists from all specialties agreed that these recommendations were both desirable and feasible.

• No single team from the 12 teams that we observed in study 1 met all of the recommendations agreed on by the expert panel. Our findings illustrate that there is scope for learning between specialties and the potential to agree a significant number of indications of good practice that are applicable in the varied contexts within which MDTs operate.

Future research priorities

Our results highlight the following areas for further study.

Contributions of authors

Rosalind Raine, Anne Lanceley, Archie Prentice, Jane M Blazeby and Michael King conceived the idea for this study.

Rosalind Raine led the funding application and all components of the study. As the chief investigator and guarantor, she managed the study overall and had final responsibility for the analysis and report content.

Rosalind Raine, Penny Xanthopoulou, Isla Wallace, Caoimhe Nic a’ Bháird and Julie Barber prepared the first draft of this report. All other authors contributed to subsequent drafts and approved the final draft.

Isla Wallace, Caoimhe Nic a’ Bháird and Penny Xanthopoulou contributed to study design, review of the literature, recruitment of participants, data collection, analysis and interpretation.

Julie Barber collaborated in the funding application and was responsible for the statistical aspects of the study, including involvement in the study design, analysis and interpretation.

Jane M Blazeby, Alex Clarke, J Simon R Gibbs, Michael King, Anne Lanceley, Gill Livingston, Susan Michie and Archie Prentice collaborated in developing the idea for the research and in the funding application. They contributed to study design and provided clinical expertise throughout the project.

David Ardron, Miriam Harris, Alex Clarke, Anne Lanceley, Gill Livingston and Archie Prentice contributed to participant recruitment.

Anne Lanceley and Alex Clarke contributed to the qualitative analysis and interpretation of the findings.

David Ardon and Miriam Harris contributed to study design and provided guidance from a patient perspective throughout the project.

Ewan Ferlie contributed to the funding application and provided expertise on the influence of context.

Publications

Raine R, Xanthopoulou P, Wallace I, Nic a’ Bháird C, Lanceley A, Clarke A, et al. Determinants of treatment plan implementation in multidisciplinary team meetings for patients with chronic diseases: a mixed-methods study [published online ahead of print June 9 2014]. BMJ Qual Saf 2014. http://dx.doi.org/10.1136/bmjqs-2014-002818

O’Driscoll W, Livingston G, Lanceley A, Nic a’ Bháird C, Xanthopoulou P, Wallace I, et al. Patient experience of MDT care and desicion-making. Mental Health Review Journal 2014; in press.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health.

Participant consent form to observe

(PDF download)

Participant information leaflet

[hospital logo]

PARTICIPANT information leaflet

Improving the effectiveness of Multi-Disciplinary Team Meetings (MDMs) for patients with chronic diseases

We would like to invite you to take part in a research study. Please read this leaflet which tells you about the study and what it involves and ask one of our team if there is anything that is not clear. Take time to decide whether or not you wish to take part.

What is the purpose of the study?

Multi-Disciplinary Team Meetings (MDMs) are widely used across the NHS for managing chronic diseases. We want to find out more about the factors that influence clinical decision-making at MDMs. The findings will be used to improve treatment decisions in your MDM and in other MDMs across the NHS by recommending possible improvements in the way that MDMs work.

Why have I been chosen?

We wish to examine a number of MDMs covering a wide range of clinical conditions affecting a diverse range of patients. All members of your multi-disciplinary team have been provided with this information sheet because we wish to observe your MDM and to invite some MDM members to participate in an interview. We will invite MDM members from each core professional group represented on the MDM, including both regular and infrequent attenders, to be interviewed.

What does the research involve?

If all multi-disciplinary team members consent, a researcher will attend and observe a number of consecutive MDMs. She will not be an active participant in these meetings, but will both audiotape the meeting and take notes, using a structured form. She will collect information on the structure of the meeting (including the number of patients discussed, the professional mix of members attending); processes (including the roles of each of the members); patient related factors that are discussed and the clinical decision made. We expect to attend and observe between 22 and 41 meetings, depending on the number of patients discussed. The first eight meetings will be audiotaped. In this part of the study, you will not be required to do anything outside of, or in addition to your normal day to day activities.

Patients discussed at each MDM will be given the opportunity to ‘opt out’ of having their medical information included in the study. We will ask each patient’s key worker/treating clinician to give their patients an ‘opt out’ form at the earliest appropriate and feasible opportunity. These forms will be available at every MDM where patients eligible for inclusion in this research are discussed.

In addition, the researcher will conduct face-to-face interviews with a selection of multi-disciplinary team members. The purpose of these interviews is to explore members’ perceptions of MDM strengths and weaknesses; factors influencing MDM decisions; their professional role and value to the MDM. The questions will be flexible and open-ended, to allow you the chance to raise the issues that you feel are important. If you are approached to be interviewed, we will ask you to sign a further consent form. All information given during these interviews will be kept strictly confidential and no names will be attached to the information provided. The interview will be conducted at a convenient time and place of your choosing. The interview will last between 30 minutes and one hour and may be ended by you at any time. It will be tape-recorded, if you consent, but the tapes will be destroyed after analysis has been completed.

Do I have to take part?

It will be entirely up to you to decide whether or not to take part in the study and you can withdraw from the study at any time without having to give a reason. If you decide to participate, you will be asked to sign a consent form, and given a copy to keep. A decision not to take part or a decision to withdraw from the study will not affect your work in any way.

What are the possible disadvantages of being interviewed?

It is expected that this study does not have any disadvantages, but the interview will take between thirty and sixty minutes of your time.

What are the possible benefits of being interviewed?

The information we get from this study will improve the MDM decision-making process by highlighting areas of excellence and possible weaknesses. If you take part in an interview, your anonymised views will contribute to our findings and any resulting recommendations for change.

Will what I say be confidential?

Yes. We will follow ethical and legal practice and all information about you will be handled in confidence. If you are interviewed, this will take place in private, and the recording will not contain your name or any personal information, only a study identification number. Recordings will be encrypted and held in a computer in the Department of Epidemiology and Public Health, UCL. Only those members of the research team who are directly involved in analysing the information will have access to the recordings. In publications and reports, the identity of participating MDMs will not be revealed, only basic descriptive information on the conditions covered and regional location of the MDM will be given. Professor Rosalind Raine is the Chief Investigator and she has overall responsibility for confidentiality and data security.

What will happen to the results of the study?

Once the study has finished the results will be analysed and conclusions drawn about how treatment decisions are reached, and how this process might be improved. Findings will be published in scientific journals, but the MDM and all individuals will be referred to in anonymised form. Quotes from the interviews may be used, but again will be anonymised. Any quotes where the individual concerned could be identified by another team member, or anyone else, will not be used. We will also visit your MDM and provide a summary of our findings. Again, interview quotes will only be used as long as the speaker’s anonymity can be preserved. MDM members will have the opportunity to discuss the findings and give their views on the recommendations.

Who is organising and funding the research

Professor Rosalind Raine is the Chief Investigator and the study is funded by the National Institute of Health Research. No payments are made to the researchers conducting this study.

Who has reviewed this study?

All research in the NHS is looked at by an independent group of people, called a Research Ethics Committee, to protect your interests. This study has been reviewed and given a favourable opinion by East London Research Ethics Committee.

What do I do if I wish to make a complaint about the research?

If you wish to complain about any aspect of the research, you should contact the Chief Investigator, Professor Rosalind Raine or the researcher. If you feel you do not receive a satisfactory response and you wish to take the matter further you should contact the Complaints Manager (see below) giving the project title and the Principal Investigator’s contact details.

Contact details

Please contact [researcher] if you would like to ask questions about the study or for any other reason:

By telephone:

By email:

By post:

You can also contact:

Professor Rosalind Raine on [telephone] or by email [email address]

Contact details for the Complaints Manager are:

[local service address and telephone]

Thank you very much for taking the time to read this information about the study.

Quantitative data collection

(PDF download)

Sensitivity analysis 1

Sensitivity analysis 2

Sensitivity analysis 3

Sensitivity analysis 4

Qualitative data collection

MDT STUDY: Participant interview TOPIC GUIDE

Version 1, 28.3.12

INTRODUCTION FOR PARTICIPANTS

The purpose of this interview is to explore your views on the multidisciplinary team meetings that you attend for [team name], and to discuss some of the factors influencing MDM decision-making.

The questions will be flexible and open-ended, to allow you the chance to raise the issues that you feel are important. Your responses will be confidential, and anonymised in any findings we publish. They will not be shared with your team. There are no ‘right’ or ‘wrong’ answers and I stress that this is not about individuals, but about the system. There is conflicting evidence in the literature about whether MDTs are effective or not, and this is what we would like to explore.

Interviewer will then talk through patient information sheet if needed and answer any questions

MDT STUDY: Patient interview TOPIC GUIDE

VERSION 1, 28.3.12

INTRODUCTION FOR PATIENTS

This research aims to understand clinical decision-making processes through the eyes of the patient. There will be three sections to the interview. First, we’d like to hear about your experience of your care so far. Then we’ll move on to how you and your healthcare team make decisions about your care. Finally, we’ll discuss more generally how you feel patients should be involved in these kinds of decisions.

We would like to emphasise that the study is being conducted by UCL and we are not employed by your Trust or the NHS.

We wish to reassure you that you do not have to answer any question you feel uncomfortable with and you can stop the interview at any time. This will in no way affect your care.

This is about getting your views across. Your responses will be confidential: they will not be shared with your healthcare team. Quotes from the interviews may be used in our findings, but it will not be possible to identify you or your healthcare team from these.

Interviewer will then talk through patient information sheet if patient/carer wishes

Warm up - I’d like to start by asking you about your condition and your care

1. Could you briefly tell me about your [condition] and the most important ways that it affects you?

   1. Encourage focus on main condition (focus of MDM) & impact on quality of life

2. What types of care do you currently have for your [condition]?

   1. Encourage brevity & focus on major health (+/– social care) interventions

3. Can you describe how decisions were/are made about your care?

   • Who are the main people involved in making decisions about your care? i.e. are respondents aware of multidisciplinary team (MDM) vs. individual consultant making decision?

   • What involvement did you have in making decisions/planning care?

   • What were your preferences/wishes regarding your care?

   • Can you describe any times when you were invited to discuss or choose any aspects of your care?

   • Did you want to make choices? Does it depend on the type of decision?

4. Please can we talk in more detail about a particularly important decision that was made about your care? This may include a treatment, diagnostic procedure, something about the timing, setting (inpatient/outpatient) or anything else such as a second opinion.

5. Can you describe how you made your decision to/not to receive this care? Can you tell us the factors that influenced the decision?

   • Any perceived concerns/benefits e.g. side effects; time in hospital; convalescence time; enormity of procedure; quality of life etc.

   • Where they received information from (e.g. family, friends, reading, clinical staff)

   • Which of these factors made a difference to the decision made? Why?

   • Extent to which their choice/decision was influenced by personal characteristics e.g. belief systems (fatalism, faith, ‘willing to try anything’, autonomy [want to retain personal control], delegation [want their Dr to make important decisions] etc.)

   • Extent to which personal characteristics interacted/were mediated by context e.g. their relationship with key health professionals / the importance of the decision/uncertainty surrounding management options etc.

6. Is there anything about your care that you know now, that you would like to have known before?

   • Side effects, time in hospital, recuperation issues, specified risks etc.

   • How might this information have changed your care preferences/decision?

MDT meetings - increasingly in the NHS, patient care is managed by a team of professionals with different skills, rather than one person. For example, [insert relevant mix of professionals] may meet together once a week to discuss different cases. Patient care decisions are often made in these in ‘multidisciplinary team meetings’, by the whole (multidisciplinary) team.

1. What do you know about these meetings?

   • Do you know that your care is discussed?

   • Were decisions fed back to you?

   • What do you think about this way of working?

   • How do you feel about your care being discussed in these meetings?

2. What were/are the important things about you that you would want the multidisciplinary team to consider when they meet to discuss your care/management?

3. What were/are the important things about your care that you would want the multidisciplinary team to consider when they meet to discuss your care/management?

   • Defined benefits and concerns e.g. side effects; time in hospital; convalescence time; enormity of procedure; quality of life etc.; importance of short versus long term effects

   • Timing; setting etc.

4. When is the most appropriate time to discuss this information with you?

   • How should it be done & how often? e.g. before or after the MDT

   • Who would you want to represent your views at a multidisciplinary meeting (key health professional; patient advocate; patient/carer)?

5. Are there issues that you would wish to remain confidential (not to be shared) when the team is discussing management options for you?

   1. As appropriate, probe, with sensitivity:

   • Comorbidities; risky behaviours; personal circumstances (do NOT press to define these if the patient does not volunteer the information).

In this final section, I’d like to talk more generally about how you think multidisciplinary teams should work

1. Do you think it is important for multidisciplinary teams to always consider patients’ views when making a decision about care?

   1. As appropriate, probe the influence of:

   • Context e.g. major decisions (to have a major intervention or not etc.) versus decisions about process (scheduling, setting etc.);

   • Intervention specific issues e.g. where there is clinical uncertainty about options; quality of life (pain, impact on mobility/independence etc.)

2. What information should be fed back to patients about the multidisciplinary team meeting decision-making process about their care?

   1. As appropriate, probe:

   • How the MDT decided what to recommend e.g. whether it was protocol/evidence based; the specialties involved in making the decision; options discussed; factors which influenced the decision etc.

3. Do you have any suggestions for ways for patients’ views to be represented at the multidisciplinary team meeting?

   1. As appropriate, probe:

   • Some suggestions include: staff known to patient attend; patient advocate attends; preparatory meeting with patient; patient preferences written down in advance; patient submits a written statement; patient/carer attends meeting; formalised feedback after the meeting; decision options discussed with patient after meeting etc.

   • How would this work in practice? e.g. would they find it upsetting, would they understand the discussion, would it be intimidating; is it feasible (work & family commitments)?

   • What about for patients who feel unable to express their view/ask questions?

Finally, is there anything else about how patients are involved in decisions about their healthcare that you would like to add?

Cancer team 1

This team meeting was one of the longest MDT meetings we observed, often running for almost 3 hours. Between 35 and 50 patients were discussed each week. It was a large team, with over 30 core members, including surgeons, oncologists, psychologists, palliative care specialists, pathologists, radiologists, research nurses, a clinical trials manager, a physiotherapist and an MDT co-ordinator. There were also often a number of observers present each week who did not participate in discussion, for example medical students or visiting academics.

The meeting was chaired by the MDT lead, who was a surgeon. On occasion, other surgeons would be asked to chair the meeting on their behalf.

A full list of patients to be discussed was provided at the start of the meeting by the MDT co-ordinator. The list was divided into different groups for discussion:

• radiology and integrated pathology patients (these tended to be lengthy discussions)

• complex cases (these tended to be a small number of lengthy discussions)

• new patients referred to the centre (these were brief updates, often feedback only)

• pathology cases (this tended to be a long list but covered quickly).

During the meeting, the chairperson updated an ‘MDT sheet’ for each patient discussed. This recorded the findings presented by the radiologist and/or pathologist, and captured the plan agreed for each patient. These records were stored on a team shared drive, and were also subsequently uploaded to individual electronic patient files.

Cancer team 2

There were 17 members in this team, including haematologists, an oncologist, a pathologist, a radiologist, clinical nurse specialists, chemotherapy nurses, specialist registrars, clinical trials practitioners and an MDT co-ordinator. Most team members attended the weekly MDT meeting regularly. Three or four members of the team were connected via video link from a second hospital site. The meeting lasted for 1 hour, with between 10 and 20 patients discussed each week. A consultant haematologist chaired the meeting.

A patient list was circulated to all team members a day or two in advance of the meeting. This was used as an agenda to structure the meeting.

Decisions were clearly summarised for the MDT co-ordinator, who made notes of the treatment plan agreed for each patient. These notes were subsequently checked by the MDT lead before being uploaded onto the electronic records system.

At the start of each meeting an IT representative set up the equipment for the video link. However, there were frequent problems with IT, which disrupted the meeting or delayed the start time.

Cancer team 3

This was the largest team we observed, with as many as 35 to 40 people attending some meetings. There was wide input from a number of those present, although many people attending did not contribute at all. Attendees included haematologists, clinical oncologists, pathologists, radiologists, specialists in nuclear medicine, cytogenetics and molecular biology, research nurses, trials co-ordinators, pharmacists, specialists registrars and an MDT co-ordinator. A consultant haematologist chaired the meeting.

The meeting was clearly structured and a patient list was circulated in advance which grouped patients by disease subtype (e.g. lymphoma, myeloma). This was used as an agenda to guide the meeting.

Decisions were recorded for each patient directly into an electronic patient record, which was projected onto a screen for other team members to see. The decision was usually recorded by a specialist registrar or a junior doctor and checked by the chairperson.

Cancer team 4

This team included both medical and clinical oncologists, dermatologists, plastic surgeons, a clinical nurse specialist, pathologists, a radiologist, a nuclear medicine specialist and a research nurse. The pathologists and the radiologist were more actively involved in discussion and decision-making than in the other cancer teams.

A patient list was circulated by e-mail to the team in advance of each meeting. This included information on patient’s sociodemographic characteristics; the consultant responsible for their care; imaging or histology to be reviewed; and a note of the issue for discussion.

There was a clear structure to the meeting, during which between 35 and 50 patients were discussed, in the following order:

• melanoma cases (radiology done first, allowing the radiologist to leave afterwards)

• patients from another hospital (done via video link)

• squamous cell carcinoma cases

• basal cell carcinoma cases.

The meeting was chaired by a consultant oncologist, with a surgeon taking the lead in their absence.

There was a wide pattern of participation during this meeting, as the focus shifted between different specialties depending on the cases being discussed. Most people attending presented at least one case (with the exception of the clinical nurse specialist, who rarely presented cases), although the consultants usually made the decisions.

Clinical decisions were recorded on a paper proforma by the presenting clinician. They were subsequently photocopied by the MDT co-ordinator at the end of the meeting and filed in both the paper and electronic notes. During the observation period the team began to complete this information electronically.

Heart failure team 1

This was a large team which included cardiologists, surgeons, palliative care specialists, specialist registrars and junior doctors, heart failure specialist nurses, research nurses and imaging consultants. In addition, a small number of observers often attended, but they did not participate in discussion.

There was a patient list, distributed before the meeting, which consisted of inpatients and any other patients flagged by a member of the team as needing discussion. This list was collated by a Senior House Officer (SHO) and included the patient’s name and hospital number, and sometimes included the reason for discussion, for example ‘review of echo’. Each patient was presented to the meeting by the SHO using a PowerPoint presentation (Microsoft Corporation, Redmond, WA, USA). The electrocardiogram, echo and other test results were also projected on a large screen.

These presentations provided the structure for the meeting. While the matter of chairing was not explicitly discussed, discussions tended to be moved along by one of the consultant cardiologists or a specialist registrar.

A specialist registrar made hand-written notes on each case and wrote up a summary of each patient discussion, including a list of named members who had contributed to the discussion. After the meeting this was given to an administrator, who added it to the patient’s electronic record.

Heart failure team 2

This was a new team and attendance changed over the period of observation when additional professionals were asked to join the team. Initially the meeting was attended by six to eight members; however, after the fourth week of observation the team was regularly attended by just three to five members. This coincided with the MDT meeting moving from its initial location to a different building. The meeting was attended by cardiology consultants, specialist registrars and heart failure nurses regardless of the number of attendees. On some occasions meetings were cancelled because the consultant was unable to attend. In response to this the lead consultant suggested that other consultants should attend the meetings.

The meeting ran for approximately 1 hour with an average of six patients discussed each week. The heart failure nurses decided which patients to bring to the meeting, and presented these, sometimes in conjunction with a specialist registrar. Patients’ cases were presented on a computer screen and the discussion focused mainly on the echo findings.

Decisions were not always clear, but when they were made they tended to be very specific, for example medication or surgery. Usually the nurses presented their suggestions and then confirmed with the doctor that the treatment they had chosen was correct.

Memory clinic team 1

Between 8 and 12 staff attended every meeting. Attendance was regarded as compulsory for all members unless there was a good reason for not attending. Team members included nurses, psychiatrists, psychologists, a specialist registrar, a team manager, social worker, an occupational therapist and a dementia support worker. The meeting was chaired by the team manager, who was a senior nurse.

At the beginning of the meeting all new referrals received that week were allocated to team members. Following this, between five and eight existing patients were discussed in the hour-long meeting, during which the results of an assessment were fed back to the team, specific queries were raised or patient updates were provided.

There was no predetermined patient list for this meeting. Instead, all team members were given the opportunity to raise cases on an ad hoc basis. Decisions made by the team were typed directly into electronic patient records during the meeting by the chairperson.

Memory clinic team 2

This team consisted of psychiatrists, psychologists, specialist registrars and a memory specialist nurse. It was a relatively small team with three to eight people attending each week to discuss three or four patients. Specialist registrars would usually attend only if they had a case to present. Not all cases were discussed during the meeting, in which case they were either brought back the following week or discussed outside the meeting by one of the consultants and another member of the team. The meeting was chaired by a consultant psychiatrist.

As with the other memory team, there was no predetermined patient list for the meeting, and team members were given the opportunity to raise patients they had seen most recently. The structure of discussion was uniform, with full presentation of test results, including cognitive assessments and a medical assessment made by a specialist registrar. Discussions would then focus on making a diagnosis based on these assessments. Decisions were recorded informally by the team member who raised the case.

Community mental health team 1

This team was observed for 14 weeks, before it disbanded. It included a consultant psychiatrist, specialist registrar, nurses and social workers, and a team manager. A locum psychologist joined the team for the last few weeks before the team disbanded. The meeting lasted for between 2.5 and 3 hours, during which approximately 30 patients were discussed. The meeting was chaired by the team manager (a social worker).

The structure of the meeting was usually not made explicit, but tended to cover:

• discussion and allocation of new referrals

• feedback from assessments (outpatient or community assessments)

• discussion of patients shared with other teams, if the inpatient/home treatment consultant was in attendance

• ongoing cases, where the team was given the opportunity to discuss any issues they wanted to raise.

A patient list was not distributed in advance of the meetings. Instead, team members raised patients as they remembered them. They fed back assessments from memory, and did not use IT or patient notes during the meeting.

At the start of the observation period, the team began recording handwritten notes into a hard-backed A4 copybook to capture team discussions and decisions made during the meeting.

Community mental health team 2

The team included nurses, social workers (one of whom was the team manager), a psychologist, a psychiatrist (who did not always attend), a specialist registrar and a junior doctor. Other occasional attendees included a rotating specialist registrar, an SHO, an administrator, who took minutes but did not contribute to discussions, and a representative from the crisis team. The nurses, social workers and psychologists took turns to chair the meeting. The meeting lasted 1 to 1.5 hours, with the team discussing around 25 patients.

The rotating chairperson tended to go through a format of:

• matters arising from the minutes of the last meeting

• crisis team liaison

• feedback from assessments

• clinical liaison (taking turns to raise issues of concern)

• any other business (e.g. new staff arriving or updates on the service-restructuring plan).

In common with CMHT 1, a patient list was not distributed in advance of the meetings; instead, team members raised patients as they remembered them. In addition, they did not use IT or patient records during the meeting and fed back assessments from memory.

During the observation period there was considerable anxiety within the team due to service line-restructuring, and many of the team were being re-interviewed for their jobs. There were sometimes extended discussions about this restructuring during MDT meetings, when people spoke about their anxieties. These discussions often ended with a reminder that restructuring issues should be discussed at the weekly ‘business meeting’ held on a different day of the week.

Minutes of the meeting were hand-written by the team administrator (or another team member, if the administrator was absent), who later typed them up. Details of discussions were not systematically entered into patients’ electronic medical notes, although some team members occasionally mentioned the MDT discussion in the notes (e.g. ‘discussed with the team and decided to discharge’). Members did not usually bring paper with them or write their own action points down during the meeting.

Community mental health team 3

This team included a consultant psychiatrist, community psychiatric nurses, social workers (including the team manager) and a clinical psychologist. Other attendees at the meeting included a rotating specialist registrar and/or SHO, an administrator, who took minutes but did not contribute to discussions, a representative from the crisis team and occasionally trainee psychologists or student nurses. The nurses, social workers and psychologists took turns to chair the meeting.

The weekly meeting lasted approximately 1 hour, though this ranged from 30–90 minutes. The team discussed around 25 clients per week. Because the job of chairperson rotated, the style of chairing varied from week to week, though it loosely followed the structure:

• apologies

• minutes of last meeting

• Mental Health Act assessments (mention of any patients sectioned or plans to do so this week)

• feedback from new assessments

• crisis team liaison

• clinical liaison (taking turns to raise any issues of concern).

Like the other CMHTs, there was no patient list made in advance of the meeting. Instead, team members raised patients about whom they had concerns (due, for example, to relapse, crisis or risky behaviour). They also brought up specific medical queries (e.g. asking the psychiatrist about medication dosages). In addition, when the chairperson went through the previous week’s minutes, senior members occasionally asked for feedback (e.g. ‘how did that appointment go?’, team manager, observation). Mental Health Act assessments planned for the following week were mentioned.

Minutes of the meeting were hand-written by the team administrator, who later typed them up. Details of the discussions were not systematically entered into the patient’s electronic medical notes, although some team members occasionally mentioned the MDT discussion in the notes (e.g. ‘discussed with the team and decided to discharge’). Members did not usually bring paper with them or write their own action points down during the meeting.

Community mental health team 4

The team consisted of a consultant psychiatrist and specialist registrar, social workers (including the team manager), community psychiatric nurses and occupational therapists. In addition, a forensic psychiatrist and a forensic nurse attended on a monthly basis and there were other occasional attendees including student nurses and student psychologists. Approximately 16 team members met each week for between 2 and 3 hours, during which approximately 49 patients were discussed. Discussions were guided by a list distributed at the start of meetings by the manager, which listed all patients currently on the team’s caseload. Most team members attended every week, although a small number did not attend for the full duration of the meeting.

The meeting was usually chaired by the team manager (a social worker with no caseload) or the deputy manager (a social worker with a reduced caseload), and was structured around the following categories:

• inpatients

• new assessments (discussion of patients in their initial 6-week assessment period while the team decided whether or not to ‘take them on’)

• clients of concern (each team member read through their list of patients and commented on anyone they were concerned about).

On one occasion the team went through a team member’s whole caseload. They made reference to the fact that they had intended to go through a team member’s whole caseload each week but there was rarely time for this.

During the ‘clients of concern’ session, the team members used their discretion to decide whom to mention, although sometimes the manager asked them about particular clients whom they had not mentioned of their own accord.

There was a computer in one corner, where one member of the team typed notes directly into the patient electronic records system or into a Word (Microsoft Corporation, Redmond, WA, USA) document to be uploaded later. This was usually done by the deputy manager or a nurse.

Theme: Discussing comorbidities at MDT meetings

Questions

Approaches to incorporate information on comorbidities in the MDT meeting:

Do you have any other suggestions about how to incorporate comorbidities into MDT meeting discussions?

Conducting a study across different NHS organisations

Conducting the study across eight NHS trusts was not a streamlined process. The hurdles we describe are familiar to every researcher undertaking multisite research in the NHS. Though the field researchers had NHS Research Passports, trusts varied in the approvals they required (e.g. honorary contracts, letters of access, licences to attend) and in the procedures for obtaining them. One site required a site-specific agreement to be drawn up which involved extensive liaison between the researchers, representatives of the study’s sponsoring organisation (University College London) and the contracts team of the trust in question.

Similarly, it was time-consuming to find and follow the local procedures for acquiring the necessary security passes and IT access privileges at each trust. Some trusts required that the researchers go on specific training before granting access to certain computer programs used to access parts of medical records (e.g. software for viewing magnetic resonance imaging scans). This occurred even where the training was not relevant to the researcher’s role – for example, in order to gain access to certain medication records, one of our researchers was obliged to undergo training on electronic prescribing, even though this was clearly not a task she was going to undertake.

Difficulties in reaching the planned sample size for the quantitative analysis

We originally recruited 10 MDTs in eight NHS trusts. The necessary ethical and R&D approvals were obtained by the chief investigator before the project start date to allow us to begin data collection immediately. For a number of reasons, however, it became apparent that these teams would not provide sufficient data for us to reach our target sample size of 3300 patients. Although we drew on the experience of all our clinical coapplicants, the idiosyncratic nature of MDT meetings soon became apparent: there were extensive differences in the number of patients discussed both within and between teams concerned with different chronic conditions. We encountered particular problems in identifying heart failure teams with sufficient patient throughput to allow us to achieve the necessary sample size in the required time frame. Having met with heart failure consultant leads across London we identified two teams, but with low patient throughput. We therefore identified a third heart failure team, which agreed to participate in the study. Unfortunately, despite our efforts and timely production of all relevant documentation, delays in the R&D procedures at the site meant that after 6 months of liaison it was no longer going to be possible to recruit the required number from that site. Instead, we compensated for the small numbers discussed in some teams by observing these teams for longer than originally planned and we also observed extra meetings at the three MDTs discussing the highest numbers of patients.

One of the mental health teams disbanded shortly into the data collection phase because of service-restructuring within their trust. We were unaware of any plans to disband when we started data collection. We therefore recruited another team. As there was not enough time to go through the process of acquiring R&D approval from a new NHS trust, we recruited a new team from a trust which had already approved the study in relation to another team. This allowed us to recruit the new team and begin data collection relatively quickly.

Another challenge related to the nature of discussions in different teams. In our clinical coapplicants’ experience, each patient discussion resulted in a management decision. This information was used to calculate the sample size required. While this did occur in some teams, in practice several teams did not make decisions in every patient discussion, particularly in mental health teams. This meant that, even where teams were discussing sufficient patients, they were not making enough decisions to allow us to reach the target sample size. We therefore recruited a fourth mental health team to compensate for this.

Another unanticipated and time-consuming task related to quality assurance of our data. We originally planned to audiotape the first six meetings observed for each team. However, because meetings often involved team members talking simultaneously, we found that it was necessary to listen to the recordings of all meetings observed to ensure accurate completion of the observation proformas (i.e. to make sure we had captured all decisions and other relevant information). This meant listening to all 370 meetings at least twice (once in person and once on the recording). We amended the participant information sheet to reflect this.

Because of the recruitment of additional teams, we needed to employ an additional research assistant to attend and collect data from these teams. We applied for and were awarded service support costs from the Central and East London Comprehensive Local Research Network to fund this.

The NHS National Research Ethics Service granted approval for these minor amendments to the study.

Challenges relating to the interviews

The need to extend the data collection period in order to achieve a sufficient sample size had consequences in terms of our ability to undertake the planned number of interviews. We considered focusing only on the professional interviews; however, on the advice of our funders and members of our steering group, we proceeded with both professional and patient interviews as planned, though with a smaller sample and with carefully targeted recruitment to retain a diversity of disciplinary perspectives in the professional interviews.

We were assisted in the conduct of some of the interviews with mental health patients by a medical student and two specialist registrar psychiatrists (currently under the supervision of one of our coapplicants). The specialist registrar psychiatrists performed additional analyses of these interviews, and these have resulted in a research article which is currently under review. We invested considerable time to ensure that the medical student and trainee psychiatrists were fully cognisant of the study. Crucially, we trained them in qualitative interview techniques and observed their technique in pilot interviews with volunteer patients identified by our coapplicant PPI representatives before they began data collection. We also helped them to obtain the necessary approvals and submitted a substantial amendment (concerning the additional researchers) to the Research Ethics Committee.

Recruitment of mental health service users for interviews proved very challenging for a number of reasons. We recruited patients who had been recently discussed at MDT meetings, purposively sampling for a diversity of age, sex and ethnicity. For each selected patient, we asked their key worker to contact the patient and, if the patient agreed, we then made direct contact with the patient to explain the study, respond to their questions and arrange the interview. We agreed exclusion criteria with MDT members. These were patients who were acutely unwell, who had been discharged and were no longer contactable, and who were in prison.

The MDT teams from which these mental health patients were drawn were undergoing restructuring. Though only one team disbanded during the observation period (no patients from this team were interviewed), the others were preparing for staff changes and morale was very low because many staff members were concerned about losing their jobs. Thus, the research was understandably not a high priority for them. The key workers were also very busy and did not always have time to speak to the patients they had agreed to contact about the study. Several key workers did not respond to our e-mail and voicemail reminders.

Even where patients had been approved as appropriate by their key workers, by the time they had contacted them to explain the study and let us know, many patients had experienced deterioration in their health or had moved address and were no longer contactable.

Several patients agreed to be interviewed during our initial conversation but we were subsequently unable to contact them. One patient agreed to participate but then rescheduled the date of the interview and subsequently changed their mind and declined. One patient did not attend two appointments that we had agreed for the interview and so we decided not to pursue them for a third appointment. Both of the mental health patients we did interview had rescheduled at least once.

Managing large qualitative data sets

The qualitative aspect of the study involved 73 interviews and observation of 370 meetings (6053 patient discussions and approximately 530 hours of audiotape). Managing this volume of data was challenging, in terms of confidential storage, data reduction, ensuring consistency of data analysis across researchers and reporting the results concisely. At one point we unexpectedly ran out of electronic storage space and had to reduce the size of our audio files. This had a detrimental effect on the sound quality. The large volume of data also significantly slowed the running of the analytic software.

We adopted a number of strategies to facilitate the processing and analysis of the data. We outsourced transcription of the interviews to an external company that was recommended by qualitative researcher colleagues and that signed a confidentiality contract. We did not outsource transcription of the audio recordings of the meetings because they were too complex, with large numbers of speakers and variable audio quality.

Ensuring that each of the three field researchers coded the data reliably was a learning process necessitating close and regular communication between the researchers. Each researcher had different expectations regarding a ‘normal’ number of codes, how broadly or narrowly to interpret the research question, how to divide the task between them and which data management software to use, if any. Each code is open to different interpretations and it was a slow and laborious process to agree unambiguous definitions which coders could use consistently.

However, we found that the benefits of performing the analysis as a group far outweighed the costs. We believe it improved the robustness of the analysis because we needed to scrutinise, challenge and defend our ideas throughout. It led to more accurate and standardised coding and gave us insights into the assumptions underlying our interpretations of events. It allowed us to learn from each other and provided more ideas for a richer conceptualisation. Finally, it allowed us to analyse far more data than an individual could have managed alone.

We began to code data from the different sources as they became available, rather than waiting for the data collection phase to be complete before beginning the analysis. Observation of the different teams finished at different times. Thus we began the basic inductive coding and selective transcription of the observational data for the first team when it was ready, rather than waiting until observation of all the teams had finished. This meant that one field researcher could work on the coding framework and feed back to the team before the next researcher used it on the next ‘batch’ of data. This avoided the potential problem of researchers simultaneously coding different sections of the data and then meeting to find that their coding frames were incommensurable or redundant. This phased approach also meant that we were already very familiar with many of the data by the time data collection was finished and we could proceed relatively quickly to the integration stages.

Conclusions

Studies of this nature, involving collaboration across multiple sites, are inevitably subject to factors that are beyond the control of the research team. Responding to these challenges requires a willingness to be flexible and responsive to unanticipated events. Maintaining an open dialogue with our funders throughout and proactively seeking the advice of our steering committee were also essential to successfully navigating these challenges.