Notes
Article history
The research reported in this issue of the journal was funded by the HS&DR programme or one of its preceding programmes as project number 11/1022/01. The contractual start date was in November 2012. The final report began editorial review in February 2014 and was accepted for publication in October 2014. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.
Declared competing interests of authors
none
Permissions
Copyright statement
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Blank et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
Chapter 1 Background
Demand management defines any method used to monitor, direct or regulate patient referrals. This includes the methods by which patients are referred from primary care to specialist, non-emergency care provided in hospital. This interface between primary and secondary care is a pivotal organisational feature in many health-care systems, including the NHS. In the UK, primary-care physicians act as the gatekeeper for patient access to secondary care and are responsible for deciding which patients require referral to specialist care. Similar models are found in health-care systems throughout the developed world, for example Australia, Denmark and the Netherlands. 1 Elsewhere, self-referral dominates (e.g. France), or the colocation of primary and specialist services leads to a variety of referral pathways (e.g. the USA). As demand outstrips resources in the UK, the volume and appropriateness of referrals from primary care to specialist services has become a key concern within the NHS. Worldwide, shifts in demographics and disease patterns, accompanied by changes in societal expectations and the relationship between professionals and patients (including the influence of the internet), are driving up treatment costs. As a result of this, several strategies have developed to manage the referral of patients to secondary care, with interventions that target primary care, specialist services or infrastructure (such as referral management centres).
Recent reviews of referral management interventions
The effectiveness of interventions to improve outpatient referrals from primary to secondary care has been the subject of a Cochrane review. 1 The Cochrane review searched for only high-quality, controlled studies and found 17 published papers. The authors concluded that there was insufficient evidence on organisational and financial interventions aimed at primary care, and also inconclusive evidence on effective educational interventions. They did, however, suggest that focusing on potentially effective interventions such as secondary care provider-led education activities, structured referral management sheets, enhancement of primary care and in-house second opinions should guide further research. A previous review on the effects of service innovation on the quality and pattern of referrals from primary care predates recent innovations such as referral management centres. 2 This previous review concluded that professional interventions such as guidelines and education, although able to affect clinical behaviour, had limited effect on referral rates, whereas organisational innovations were more likely to affect referral rates. Further to this, Dunst and Gorman3 reanalysed the Faulkner review along with the previous Cochrane review4 and concluded that interventions that more actively involved primary-care physicians were more effective in influencing rates and patterns of referral.
More recently, referral management in the general practitioner (GP) context has been the subject of work funded by The King’s Fund. 5 Their report highlights the concerns of many with regard to the risks of managing demand without taking account of patient safety, acknowledging that referral management has the capacity to increase clinical risk as well as to reduce it. In considering whether or not one approach to referral management is ‘better’ than another, they suggest that ‘light touch interventions’ such as peer review and feedback, alongside the use of guidelines and structured referral sheets, may offer the most cost-effective approach. However, although the report contributes important insights, it does not suggest best practice examples of these interventions or how they would best be implemented in practice.
Theoretical/conceptual framework
It is increasingly recognised that most interventions in health care can be considered to be complex, with individual and organisational factors affecting how and if interventions lead to improved outcomes. 6 This recognition of the complexity of interventions has been accompanied by a corresponding growth in the challenges for standard methods of evaluation and synthesis. Evidence-based practice requires policy-makers and practitioners to have readily available access to information on interventions that have been shown to work or not work, or indeed have the potential to cause harm. Systematic reviews are an established way of exploring the effectiveness of interventions and a cornerstone of evidence-based practice in order to identify, evaluate and summarise the findings of all available research evidence. Methods for carrying out systematic reviews have become increasingly refined, led by Cochrane, the National Institute for Health and Care Excellence (NICE) and the Centre for Reviews and Dissemination which details the formal procedures required. Conventional systematic review methods, however, face challenges in establishing clear intervention-outcome links when complex multifactorial processes are operating, and there are few experimental studies to draw upon.
As much of the international evidence in the area of referral management is observational in nature and lacks control comparators, our work builds on previous reviews by taking broader inclusion criteria (to include all study designs and grey literature, as well as evidence from other industries). The review findings are presented via a conceptual model (a logic model), which details the range of interventions identified, evidence of their effectiveness and factors which may influence how and if interventions lead to demand management outcomes. The work not only explores the effectiveness of interventions for demand management, but also aims to uncover detail of the processes whereby interventions may lead to an impact on health-care systems in order to determine applicability to the UK context.
Logic models
Logic model methods are a form of theory-based evaluation that focus on relating hypothesised links between an intervention and its constituent parts to its outcomes and long-term impacts. Logic models are concerned with examining the processes of implementation, mechanisms of change and participant responses in order to develop hypothesised links or a ‘theory of change.’7 In order to develop a theory of change, it is necessary to understand the moderator and mediator variables in the process. 8 These factors are the key to understanding how an intervention works and how interventions may work in different health-care contexts. Logic model evaluation methods begin by mapping out an intervention and then examining conjectured links between the intervention activities and anticipated outcomes to develop a summarised theory of how an intervention works, usually in diagrammatic form. Outcomes are conceptualised as being the end of a chain of intermediate changes which the evaluation process seeks to track, with each intermediate point predicting the outcomes that may occur in the future. 9 Logic models have been suggested as a means to help to provide a strategic perspective on complex programmes and to understand the relationships between various elements of an intervention and outcomes. 10 In particular, they are recommended for evaluating highly complex, multisite interventions with multiple and/or indeterminate outcomes. 11
The area of referral/demand management has many of the same challenges as other complex interventions. A key issue relates to the diversity of the many different referral management approaches that have been investigated, which involve varying degrees of active intervention in referral systems and processes. Understanding how these interventions operate is important when evaluating applicability between different systems and contexts. Logic model methods are underpinned by a systems perspective and provide a mechanism for evaluating system impacts, and for supporting managers in presenting a logical argument for how and why an intervention will address a specific need. There has been growing interest in applying the approach to evaluation of health care. It has been highlighted, for example, that hospitals need to look at the logistics of their patient-pathway processes and use a systems perspective to examine flows through the process. Referral management entails moving from a system that reacts in an ad-hoc way to meet increasing needs to one that is able to plan, direct and optimise services in order to optimise demand, capacity and access across an area. Uncovering the assumptions and processes within a referral management intervention, therefore, requires an understanding of system operation and assumptions which the logic model methodology is well placed to address.
Research questions
This research was designed to conduct an inclusive systematic review and develop a logic model to answer the following research questions:
-
What can be learned from the international evidence on interventions to manage referral from primary to specialist care?
-
How can international evidence on interventions to manage referral from primary to specialist care be applied in a UK context?
-
What factors affect the applicability of international evidence in the UK?
-
What are the pathways from interventions to improved outcomes?
Chapter 2 Review methods
A review protocol was developed for the project and can be found at www.nets.nihr.ac.uk/__data/assets/pdf_file/0007/81178/PRO-11–1022–01.pdf.
Inclusion and exclusion criteria
Participants: all primary care medical physicians, hospital specialists and their patients.
Interventions: interventions that aim to influence and/or affect referral from primary care to specialist services by having an impact on the referral practices of the primary physician; in addition, interventions that aim to improve referral between specialists or have the potential to impact on primary care to specialist referrals.
Comparators: the main comparator condition for intervention studies was the usual method of referral practice which is undertaken in the location where the intervention is being implemented. However, alternative comparators have not been excluded. We also included studies with no concurrent comparator (e.g. non-controlled before-and-after studies), as well as qualitative studies where comparators are not relevant.
Outcomes: all outcomes relating to referral were considered, including referral rate, referral quality, appropriateness of referral, impact on existing service provision, costs, mortality and morbidity outcomes, length of stay in hospital, safety, effectiveness, patient satisfaction, patient experience and process measures (such as referral variation and conversion rates). All qualitative outcomes were also considered for the relevant papers.
Study design: with the increasing recognition in the literature that a broad range of evidence is needed to inform review findings, no restrictions were placed on study design. The criterion for inclusion in the review was that a study is able to answer or inform the research questions. We have, however, taken note of how quality of study design and execution may affect the reliability of the results generated, as discussed below.
Identification of evidence
Search strategy
Searches were limited by date (January 2000 to July 2013). Articles generated by our searches that consisted of English abstracts only, with full papers published in other languages, were considered for translation, but none was found to meet the inclusion criteria for the review. Our international collaborators did not identify any key articles in other languages, which might have required translation.
All of the literature identified using the above methods were imported into Reference Manager Version 12 (Thomson ResearchSoft, San Francisco, CA, USA) and key-worded appropriately. An audit table of the search process was kept, with date of search, search terms/strategy, database searched, number of hits, keywords and other comments included, in order that searches were transparent, systematic and replicable. Searches took place between November 2012 and July 2013. Search strategies and a full list of data sources are given in Appendices 3 and 4.
At the outset of the project a steering group of our international collaborators, relevant patient representatives and other stakeholders was formed. This group had the opportunity to suggest terms to be considered for inclusion in the initial search strategy as well as identifying key articles for potential inclusion.
Initial search
Systematic searches of published and unpublished (grey literature) sources from health care and other industries were undertaken to identify recent, relevant studies. An iterative (i.e. a number of different searches) and emergent (i.e. the understanding of the question develops throughout the process) approach was taken to identify evidence. 12,13
An initial search was generated to address the project research questions, with free-text and subject-heading terms combined to address the concepts of ‘primary care’ and ‘referral’. A broad range of electronic database, including MEDLINE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO and Health Business Elite, was searched in order to reflect the diffuse nature of evidence (see Appendices 3 and 4).
Databases that focus on health management literature, such as the Health Management Information Consortium and Health Business Elite, and management databases such as Business Source Premier and Emerald Management Reviews, were also searched using the initial search strategy.
Additional searches
After the initial search a phrase search was undertaken for ‘referral management centres’ in MEDLINE and CINAHL (for full details of data sources see Appendix 3). This was to make sure that papers had not been missed which described this particular referral method.
As the work progressed, further searches were required in order to seek additional evidence where there were gaps and implicit assumptions that particular outcomes would result following interventions described later.
Citation searches
Citation searches of included articles and systematic reviews were undertaken in the Science Citation Index and Social Science Citation Index and respective conference papers indices. Where a search returned no results, a search in Scopus was undertaken to double check for any registered citations. Relevant reviews articles were also used to identify studies.
Grey literature
Grey literature (in the form of published or unpublished reports, or data published on websites, in government policy documents or in books) was searched for using the OpenGrey (www.opengrey.eu), Greysource (www.greynet.org) and Google Scholar (http://scholar.google.com; Mountain View, CA, USA) electronic databases.
Reference list checking
Hand-searching of reference lists of all included articles was also undertaken, including relevant systematic reviews.
Selection of papers and data extraction
Citations were uploaded to Reference Manager, and titles and abstracts (where available) of papers were independently screened for inclusion by two reviewers, with disputes resolved by consulting other team members. Full-paper copies of potentially relevant articles were retrieved for systematic screening. A data extraction form was developed using the previous expertise of the review team, trialled using a small number of papers and refined for use here. Data extractions were completed by one reviewer and checked by a second.
Extraction data included country of the study, study design, data collection method, aim of the study, detail of participants (number; any reported demographics), study methods/intervention details, control details, length of follow-up, response and/or attrition rate, context (referral from what/who to what/who), outcome measures, main results and reported associations between elements for the logic model.
Data synthesis
The heterogeneity of the interventions’ aim, design and outcome measures used precluded a meta-analysis of their results. We therefore completed a narrative synthesis of the data, primarily in terms of type of intervention and outcomes. In addition, we built on our previous methodological work14,15 and thematic synthesis methods,16 and used the data to develop a diagrammatic representation (logic model) of the factors that may influence the pathway from interventions to system-wide impacts. The model aimed to portray how interventions operate in order to change practice at individual, local and system-wide levels.
Quality appraisal
Individual studies
The critical appraisal of included evidence is a key part of the review process; however, it is the subject of debate in the field, with no single recognised tool. There is also variation in views regarding the use of scoring systems, with Cochrane discouraging the use of systems which total elements on a checklist, as a single item may jeopardise an entire study. In this review, the quality of studies was assessed using a checklist based on work by Cochrane (see Appendix 2). This approach considers risk of bias and, as it is usually used with experimental studies, required some modification for use with our wider range of study designs. Qualitative papers were evaluated using an adaptation of the Critical Skills Appraisal Program tool. Each paper was assessed by one reviewer and checked for accuracy by a second. Each paper was graded on a three-point scale as being at higher risk of bias, lower risk of bias or unclear risk of bias. The rating was based on not only an aggregate (the number of items) but also an overall judgement of risk of bias. It is important to note that our rating was comparative (higher vs. lower) across the set of papers, with a study classed as being at lower risk not meaning that it was necessarily low risk (see the assessment of each study detailed in Appendix 2). Study design criteria for inclusion in the review were not set as the work was intended to be broad-based and inclusive. Inclusion required only that the paper was able to answer the research question; however, we took account of quality standards in the synthesis and presentation of the evidence as will be outlined below.
Appraising the strength of the evidence
Although there is debate regarding rating of quality of individual studies, there is also considerable variation in views regarding methods for appraising strength of evidence across studies, with a higher number of papers in an area indicating not necessarily greater strength of evidence but only that more work has been carried out. We adopted a system that combined consideration of volume of evidence, and also consistency of evidence, with quality of evidence, based on work by Hoogendoorn et al. 17 Evidence strength appraisal was undertaken by the research team at a series of meetings to establish consensus. Each group of papers was graded as (i) stronger evidence, (ii) weaker evidence or (iii) inconsistent/no evidence.
Stronger evidence (i) was defined as generally consistent findings in multiple higher-quality studies.
Weaker evidence (ii) was defined as generally consistent findings in one higher-quality study and lower-quality studies, or in multiple lower-quality studies.
No evidence or inconsistent evidence (iii) was defined as only one study available or inconsistent findings in multiple studies. Study findings were considered to be inconsistent if fewer than 75% of studies reported the same conclusions.
Validation and applicability of the findings
Following completion of the evidence appraisal and draft logic model synthesis, we undertook a period of stakeholder consultation to seek feedback on the evidence that we had identified and the applicability of the findings to the UK health-care context. This consultation was carried out via presentations to practitioners and patient representatives, via individual meetings to discuss the findings, and by circulating the model to experts in the field (including practitioners, commissioners and academics). In total, 44 individuals contributed to this validation stage. In order to assess how our findings resonated with other work in the field, we also carried out a review of other reviews in the area.
Chapter 3 Results of the review
Quantity of the evidence available
In total, our searches generated a database of 8327 unique papers. Of these, 580 papers were selected for consideration at the full-paper stage. After considering these, searching reference lists and completing the validation stage of the project, 290 full papers were included in the review (Table 1). 18–308 The included papers consisted of 140 intervention papers and 150 non-intervention papers (looking at the views of patients and professionals on the referral process, and factors which predict referral). The 150 non-intervention papers included qualitative studies (n = 33) and non-intervention quantitative studies such as surveys and research reporting associations (n = 117). Grey literature searches generated 69 potentially relevant articles but no additional articles were subsequently found to be within the scope of the review. This was probably due to the fact that a number of grey literature reports had already been identified in the previous searches.
| Source | Number of hits | Number of papers included |
|---|---|---|
| Initial searches | 6431 | 253 |
| Additional searches | 876 | 7 |
| Citation searches of included papers | 814 | 16 |
| Reference list of included papers and systematic reviews | 137 | 12 |
| Grey literature | 69 | 0 |
| Validation stage | 1 | 1 |
| Total | 8328 | 290 |
Of the intervention papers, 114 were identified through the initial database searches, 14 were identified through citation searches, one was identified through additional targeting searching and 10 additional papers were identified through scrutinising reference lists (including those of systematic reviews). One further study was identified at the validation stage of the logic model.
Of the non-intervention studies, 140 were identified through the initial database searches, two were identified through citation searches and six were identified through additional targeting searching, with two additional papers identified through scrutinising reference lists.
In addition, 30 systematic review papers in relevant topics were identified and a synthesis of these was developed in parallel with, but independently to, the logic model development. Comparison with the logic model synthesis is considered in Appendix 6 of this report as part of the validation stage.
We excluded a total of 286 papers which were obtained as full papers but were subsequently found to be outside the scope of the review. A list of these papers and the reasons for their exclusion are given in Appendix 5. Figure 1 details the process of identification of studies.
FIGURE 1.
The process of identification of studies.
Quality of the evidence available
Of the 140 intervention studies, the vast majority (n = 126) were considered to be at lower risk of bias. 19,21–24,27–40,43–71,73–89,92–96,98–100,102–139,141,142,144–150,152,156–160 Fifteen intervention studies were considered to be at higher risk of bias,25,26,42,72,90,91,97,129,140,143,151,153,154 including two studies where the risk of bias was unclear. 19,41 The main risks for bias related to a lack of participant details, only narrative results, percentages reported without supporting statistics, data reported as charts only, inconsistencies in data reporting, poor response rates, attrition rate not reported, weak outcome measures, unclear study design, and evaluation tools which asked questions that strongly led respondents towards positive answers.
Of the 33 qualitative studies, 32 were considered to be at lower risk of bias. 176,177,182,192,194,201,204,207,209,210,212,213,217,218,221,226,228–230,232,237,239,249,252,253,256–258,273,293,306 Only one was considered to be at higher risk of bias due to unclear aim, unclear process for selection of participants and data not clearly distinguished from report of other authors’ work. 20
Of the 117 non-intervention qualitative studies (surveys, etc.), 96 were considered to be at lower risk of bias,98,101,138,161,163–181,183,187–189,191,193,195–200,206,211,215,216,219,220,222,223,225,231,234,235,238,240–243,245–248,250,251,254,259–270,272,274–276,278–294,297,299–305,307 with 21 studies considered to be at higher risk of bias. 162,165,184–186,190,202,203,205,208,214,224,227,233,236,244,255,271,277,295,306,308 The main risks for increased bias were attributable to studies being completed in one small sample only, limited recruitment details, poor response rate, leading questions, recall bias, unpiloted survey tools, unclear methods, limited data presentation, possible overstatement of findings and over-reliance on self-reported outcomes.
Although the higher-risk studies were not excluded from the synthesis and model, the risk of bias was accounted for in assessing the strength of evidence for each element of the model. The detailed quality assessment for each study is provided in Appendix 2.
Study designs
Of the 140 intervention studies, there were 44 randomised controlled trials (RCTs)23,26,27,29–32,36,39,53,54,58–60,63–68,76,77,79,82,85–87,92–95,107,109,111,114,116,117,120,125,126,131,135,144,159 (including 19 of cluster design30–32,39,53,58,63,65–68,77,79,86,111,114,117,120,131), five non-RCTs (nRCTs),62,108,127,130,134 43 before-and-after studies (without a concurrent control group),24,33–35,38,42,43,45,47–52,55,57,69,72–74,89,90,102,103,105,110,112,115,119,122,129,133,136,137,143,145,146,149,154,156–158,160 three controlled before-and-after studies,56,70,81 one case–control study,57 one economic analysis,151 five cohort studies28,46,71,104,128 and 38 evaluation studies (described variably as audits, review, evaluation and retrospective data analysis). 18,19,21,22,25,27,40,41,44,61,75,78,80,83,84,88,91,97–99,106,113,118,121,123,124,132,135,138,140–142,147,148,152,153,155,158
Of the non-intervention views and predictors studies, the 33 qualitative studies consisted of qualitative interview studies (n = 2520,163–165,171,177,178,180,183,192,194,196,201,204,207,210,212,213,237,239,245,249,253,258,260), focus group studies (n = 5217,230,232,252,257), studies using both interviews and focus groups (n = 2196,239) and one study which used transcriptions of video tapes. 182 The non-intervention quantitative studies (n = 117) were mostly cross-sectional surveys (n = 8229,108,161,168–175,178,179,181,183–185,187–191,193,195,198,200,202,203,205,206,208,209,211,214–216,219,220,222,224,225,227,231,232,234–236,238–240,242,244,246,248,250,251,259,261,263,264,268–282,284–287,289,291,292). In addition, one study employed a follow-up survey; two studies used surveys and interviews,176,186 and one further study also included a focus group. 233 There were also 29 studies which consisted of an analysis of patient records, documents, case notes, admissions data and referral forms. 138,166,167,173,197,219,223,235,241,243,254,256,263,265–267 Most of these studies (n = 23) were retrospective designs, but four employed a prospective cohort design. 173,223,254,266 In addition, one study employed Delphi methods196 and one final study used a group-based assessment of referral appropriateness. 255
Populations and settings
Of the 140 interventions, the majority were conducted in the UK (n = 8218,19,21–23,26,28,30–32,34,37,38,41–62,64,65,68,70,71,73,74,76–80,82–85,94,96,99,103,104,106,109,114,116,117,119,122,124–126,128,129,131,133,139,140,142,143,152–157,159,160) or the USA (n = 2024,33,63,87,89,93,98,100,102,112,115,121,132,138,144–147,155,158). There were 10 studies from the Netherlands36,67,86,90,120,123,134,135,141,149 and nine from Australia. 49,72,91,97,105,111,118,136,148 Additional studies were conducted in Canada (n = 327,107,110), Israel (n = 3130,137,150), Italy (n = 369,113,127), Denmark (n = 229,92), Spain (n = 235,75), Finland (n = 195), Norway (n = 1151), Hong Kong (n = 181) and UK/China (n = 125), with one final study where the country of origin was unclear. 101
Of the non-intervention views and predictors studies, the 33 qualitative studies were conducted mostly in the UK (n = 18177,180,182,192,194,201,204,207,209,210,218,228,229,249,252,253,257,258), with additional studies from Australia (n = 5169,176,221,226,245), USA (n = 5170,183,200,202,208), the Netherlands (n = 3212,237), Norway (n = 2164,217), New Zealand (n = 120) and Belgium (n = 1230). The non-intervention quantitative studies (n = 117) were mostly from the UK (n = 35157,174,175,177,187,189,190,193,195,197,198,207,220,224,233,236,241–243,247,251,254–256,265,266,272,273,279,282,284,285,287,291,294) and USA (n = 3198,108,138,171,172,178,184,200,205,214,216,218,219,222,223,225,231,232,235,238,240,246,260,264,267,270,271,274,277,283,286,290,304,305,307), with additional studies from Canada (n = 13107,165,179,196,203,206,227,234,248,263,275,292,299), Australia (n = 1040,91,105,148,162,185,186,188,215,268), the Netherlands (n = 4163,191,212,250), Norway (n = 4164,168,239,244), Israel (n = 3167,261,269), Germany (n = 2173,211), Denmark (n = 229,181), New Zealand (n = 2288,302), France (n = 1161), Ireland (n = 1280), Belgium (n = 1209), Lithuania (n = 1166) and Spain (n = 1276). In addition, two studies were conducted in more than one country, namely the UK/Australia (n = 1169) and USA/Canada/Puerto Rico (n = 1183).
Types of interventions
In total 140 intervention papers were identified and used to create a typology of studies by intervention type. The intervention studies identified may be grouped into four categories: GP education interventions (n = 4919,21,22–69); process change interventions (n = 4770–87,98–120); system change interventions (n = 4118,121–157); and patient-focused interventions (n = 3158–160). It is accepted that this grouping of interventions may have some overlap; however, focus is on the content. Table 2 provides a summary of the intervention studies grouped by typology.
| Intervention category | Intervention type | Studies reporting a positive effect on referral outcomes (first author and year) | Studies reporting no effect on referral outcomes (first author and year) | Strength of evidence |
|---|---|---|---|---|
| GP education | Peer review and training/feedback | Cooper 2012,19 Evans 2009,21 Evans 2011,22 Jiwa 200423 | i | |
| GP training: professional development | Adams 2012,33 Bennett 2001,30 Donohoe 2000,31 Hands 2001,34 Hilty 2006,24 Kousgaard 2003,29 Ramsay 2003,27 Suris 2007,35 Watson 2001,32 Wolters 200536 | Bhalla 2002,37 Ellard 2012,38 Emmerson 2003,40 Lam 2011,25 Lester 2009,39 Rowlands 2003,26 Shariff 201028 | iii | |
| Guidelines (no training/feedback) | Cusack 2005,43 Idiculla 2000,44 Lucassen 2001,45 Malik 2007,41 Imkampe 2006,47 Potter 2007,46 Twomey 200342 | Fearn 2009,48 Hill 2000,49 Matowe 2002,50 Melia 2008,51 West 200752 | iii | |
| Guidelines with training/feedback/specialist support | Banait 2003,53 Eccles 2001,54 Elwyn 2007,55 Glaves 2005,57 Griffiths 2006,58 Julian 2007,62 Kerry 2000,59 Robling 2002,60 Walkowski 2007,63 White 2004,61 Wright 200656 | Dey 2004,66 Engers 2005,67 Jiwa 2006,68 Morrison 2001,64 Spatafora 2005,69 Wilson 200665 | iii | |
| Process change | Direct access to screening/diagnostic testing | DAMASK 2008,76 Shaw 2006,77 Simpson 2010,78 Thomas 2003,79 Thomas 2010,80 Wong 200081 | Dhillon 2003,82 Eley 2010,83 Gough-Palmer 200984 | iii |
| Designated appointment slots/fast-track clinic | Bridgman 2005,70 Hemingway 2006,73 Khan 2008,71 Sved-Williams 201072 | McNally 2003,74 Prades 201175 | iii | |
| Specialist consultation prior to referral | Eminovic 2009,86 Harrington 2001,93 Hockey 2004,91 Jaatinen 2002,95 Knol 2006,90 Leggett 2004,85 McKoy 2004,89 Nielsen 2003,92 Tadros 2009,96 Wallace 2004,94 Whited 200287 | i | ||
| Electronic referral | Chen 2010,100 Dennison 2006,99 Gandhi 2008,108 Jiwa 2012,105 Kim 2009,98 Kim-Hwang 2010,102 Nicholson 2006,97 Patterson 2004,104 Stoves 2010103 | Kennedy 2012106 | i | |
| Decision support tool | Akbari 2012,110 Emery 2007,111 Junghams 2007,109 Knab 2001,112 Mariotti 2008,113 McGowan 2008107 | Greiver 2005,114 Magill 2009,115 Slade 2008,117 Tierney 2003116 | iii | |
| Waiting list review | Stainkey 2010118 | King 2001,119 van Bokhoven 2012120 | iii | |
| System change | Community provision of ‘specialist’ services by GPs | Callaway 2000,121 Ridsdale 2008,124 Salisbury 2005,125 Sanderson 2002,126 Sauro 2005,127 Standing 2001,122 Van Dijk 2011123 | Levell 2012,129 Rosen 2006128 | i |
| Additional primary care staff | Simpson 2003,143 Van Dijk 2010,141 White 2000142 | i | ||
| Outreach: community provision by specialists | Campbell 2003,131 Felker 2004,132 Gurden 2012,133 Hermush 2009,137 Hughes-Anderson 2002,136 Leiba 2002,130 Schulpen 2003,134 Vlek 2003135 | Johnson 2008,139 Pfeiffer 2011138 | i | |
| Return of inappropriate referrals | Tan 2007,140 Wylie 200118 | ii | ||
| Gatekeeping | Ferris 2001,145 Ferris 2002,146 Joyce 2000,147 Schillinger 2000144 | iii | ||
| Payment system | McGarry 2009148 | Iversen 2000,151 Van Dijk 2013,149 Vardy 2008150 | iii | |
| Referral management centre | Maddison 2004,154 Watson 2002,152 Whiting 2011153 | Cox 2013,156 Ferriter 2006,157 Kim 2004155 | iii | |
| Patient inventions | Patient education | Lyon 2009160 | Heaney 2001159 | iii |
| Patient concerns and satisfaction | Albertson 2002158 | iii |
General practitioner education interventions
The GP education intervention group included peer-review and feedback (n = 4) interventions, which consisted of formal GP training (including continued professional development) (n = 17) and the issuing of guidelines [with (n = 18) and without (n = 11) additional formal training and support for practitioners].
Peer review
Peer-review training/feedback was offered to GPs (plus advanced health-care practitioners and practice managers) in one study19 either in face-to-face meetings19,21,22 or via written feedback. 23 Follow-up was for a minimum of 1 year in all cases. Details of each study are outlined in Table 3.
| Study | Intervention | Design | Country | Specialty | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Cooper 2012 19 | Face-to-face peer review | Audit | UK | Orthopaedics | NR | 5 years |
| Evans 200921 | Face-to-face peer review | Audit | UK | Emergency, orthopaedics | Nine GP practices | 1 year |
| Evans et al. 201122 | Face-to-face peer review | Audit | UK | Seven specialties | 10 GP practices (53 GPs) | 1 year |
| Seven specialties | ||||||
| 21 female GPs, median aged 44 years | ||||||
| Jiwa et al. 200423 | Written peer review | nRCT | UK | Specialists | 26 GPs in intervention group | 18 months (6 months) |
Two studies were at lower risk of bias. Evans21 reported, on average, a significant drop in referrals between the first and fourth quarters (z = 2.25, p = 0.025). The quality of referrals as judged by doctors’ peers improved and referral rates in orthopaedics showed a reduction of up to 50%. However, variability between practices decreased and referral to local services increased. In 2011 they further reported a reduction in variation in individual GP referral rates (from 2.7–7.7 to 3.0–6.5 per 1000 patients per quarter), and a related reduction in overall referral rates (from 5.5 to 4.3 per 1000 patients per quarter). 22 Although the highest individual referrers showed a decrease, the lowest referrers may show an increase in referrals [and a significant negative correlation comparing the first month’s data with the change from first to last month (r = 0.719, p = 0.019)]. 22 Jiwa et al. 23 reported a difference of 7.1 points [95% confidence interval (CI) 1.9 to 12.2 points] in the content scores between the feedback group and the controls after adjusting for baseline differences between the groups. There was a considerable improvement in the content of the referral letters from the feedback group from before to after feedback (mean score 34.1 vs. 39.5). There was no improvement in the scores for the control group in the same period [mean score 34.1 vs. 28.2; mean difference 5.3 (95% CI 1.5 to 9.2)/mean difference 0.55 (95% CI –1.4 to 2.5); t-test degrees of freedom (df) 20/36; p = 0.008/0.6].
One further study was at higher risk of bias. Cooper19 conducted a peer-review scheme for referrals with two guiding principles: the review would benefit the practice and the commissioning group; and there was no blame. GPs, nurses, advanced health-care practitioners and practice managers attended a workshop event and each practice bought two or three trauma and orthopaedic referral letters. Participants worked at mixed tables to understand each practice’s referral profile, share how each practice would handle each situation and then identify any gaps or areas of changed needed. As a result they reported that trauma and orthopaedic expenditure in 2010–11 was 17% less than in 2006–7; in addition, one practice cut ear, nose and throat (ENT) referrals by 20% in the first year and 40% overall.
Formal general practitioner training
Seventeen interventions consisted of formal GP training. Overall, 11 studies reported a positive impact on referral,24,27–36 with six showing no effect or a negative change. 25,26,37–40 Three studies were considered to be at higher risk of bias. 24–26 Overall, the strength of this evidence was graded as inconsistent.
The interventions themselves were varied and it was challenging to separate them further for analysis given the diversity of the interventions delivered. However, seven interventions were delivered in one single session (Table 4) and 10 sessions were delivered over a number of weeks or months (Table 5). The single-session interventions consisted of educational reminders added to radiographs requested by GPs;27 an educational module and 12-page printed guide;28 a structured information pack sent to GPs when their patients attended the department of oncology for the first time;29 an education video;30 in-practice education session plus information pack;31,32 and a 1-day interactive chronic obstructive pulmonary disease (COPD) programme. 33
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Adams 201233 | One-day CME | BA | USA | COPD | 351 primary care clinicians | (3–6 months) |
| Bennett 200130 | Video; checklist | cRCT | UK | ENT (glue ear) | 50 practices | (1 year) |
| 177 GPs | ||||||
| Donohoe 200031 | Practice visits; leaflets | cRCT | UK | Diabetic foot | 10 towns | (6 months) |
| 1939 patients | ||||||
| Aged 18+ years | ||||||
| Kousgaard 200329 | Information pack to GPs on first referral | RCT (unblind) | Denmark | Oncology | 248 patients | NR |
| 199 GPs | ||||||
| Ramsay 200327 | Educational reminders on radiographs | RCT | Canada | Radiology (knee and spine) | 81 GP practices | 12 months |
| 2324 referrals | ||||||
| Shariff 201028 | Educational module | Cohort | UK | Oncology (skin cancer) | 460 referrals | 15 months (12 months) |
| Watson 200132 | Practice education session ± information pack | cRCT | UK | Oncology (familial breast/ovarian cancer) | 170 GP practices | 9 months |
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Bhalla 200237 | Three or four ENT sessions over a 2-week period once a year | Case control | UK | Otolaryngology (ENT) | Two GP practices | 3 years |
| 1073 referrals | ||||||
| One partner in each GP practice | ||||||
| Ellard 201238 | Six 2-hour interactive sessions on common skin conditions | CBA | UK | Dermatology | 30 GPs from 26 practices | (3 months) |
| Emmerson 200340 | Psychiatric appointments in primary care | Audit | Australia | Psychiatry | Five psychiatrists, 200 GPs | 1 year |
| Hands 200134 | GPs trained at outpatient sessions | BA | UK | All specialties | 22 consultants, 21 GPs | (6 months) |
| Hilty 2006 24 | Regular CME peer review; consultation notes for GPs | BA | USA | Psychiatry | 400 consultations | NR |
| Lam 2011 25 | Diploma in Community Geriatrics | CX | UK/China | Geriatrics | 98 GPs | 1 year |
| Lester 200939 | Video, question and answer, two refresher sessions | cRCT | UK | Psychiatry | 179 patients | (4 months) |
| Two GP practices | ||||||
| Rowlands 2003 26 | Educational referral meetings | CX (part of RCT) | UK | All specialists | 13 GP practices | NR |
| Four or more partners | ||||||
| Suris 200735 | Biweekly educational sessions by specialists | BA | Spain | Rheumatology | 117 GPs | 1 year |
| Wolters 200536 | Distance-learning programme | RCT | Netherlands | Urology | 142 GPs | (14 months) |
Six of the ‘one-session’ interventions (see Table 4) showed positive effects on referral outcomes and were at lower risk of bias.
Adams et al. 33 delivered a 1-day interactive COPD continuing medical education programme. Knowledge/comprehension significantly improved {mean [standard deviation (SD)] pre-test percentage correct, 77.1% (16.4%); 95% CI 76.2% to 78.9%; and mean (SD) post-test percentage correct, 94.7% (8.7%); 95% CI 94.2% to 95.2%; p < 0.001)}, with an absolute percentage change of 17.6% (13.2%). Of the follow-up survey respondents, 92 of 132 (69.7%) reported completely implementing at least one clinical practice change, and only 8 of 132 (6.1%) reported inability to make any clinical practice change after the programme.
Bennett et al. 30 delivered a training video, a checklist or both to three intervention groups. At 1 year post intervention, there was significant improvement in the positive predictive value, adjusted for patient waiting time between GP referral and appointment at the ENT department. The improvement in positive predictive value pre and post intervention was 15% (95% CI –12.1 to 41.7) for the practices receiving both interventions, compared with 20% (95% CI –32.9 to –6.4) for practices receiving only one intervention and a degradation of 34% for those receiving no intervention.
Donohoe et al. 31 delivered an educational intervention aimed at clarifying management of the diabetic foot, referral criteria and the responsibilities of professionals. The intervention included practice visits and education of the whole practice team. Leaflets outlining patients’ role and responsibility were disseminated to the practices. Appropriate referrals from intervention practices to the specialist foot clinic rose significantly (p = 0.05), compared with control practices (p = 0.14).
Kousgaard et al. 29 provided a structured information pack to GPs when their patients attended the department of oncology for the first time. Intervention group practitioners gave a significantly higher score to the information value of the discharge letter than did control group practitioners. The most pronounced difference was seen for psychosocial conditions (p = 0.001) and information about what the patient had been told at the department (p = 0.001).
Ramsay et al. 27 reported that after 6 months of adding educational reminders to radiographs (adjusting for seasonal variation) the frequency of knee radiographs showed a relative risk (RR) reduction of 0.65 and lumbar spine radiographs showed one of 0.64. The mean number of referrals per practice per month for the control group was 2.97 (SD 3.22) knee and 2.88 (SD 3.05) spine, compared with intervention group mean referrals of 1.87 (SD 2.4) knee and 1.76 (SD 2.38) spine.
Watson et al. 32 randomised 170 practices to group A (receiving an in-practice educational session plus information pack), group B (receiving an information pack alone), or group C (receiving neither an educational session nor a pack). There was a 40% (95% CI 30 to –50, p < 0.001) improvement in the proportion of GPs who made the correct referral decision on at least five of six vignettes in group A (79%) compared with the control group (39%) and a 42% (95% CI 31 to 52%, p < 0.001) improvement in group B (81%) compared with the control group (39%). There was no significant difference between groups A and B.
A further ‘one-session’ intervention was not effective. Shariff et al. 28 delivered an educational module that was aimed at building confidence in the diagnosis of lesions not requiring an urgent referral, especially basal cell carcinomas and seborrhoeic keratoses, referred through the ‘2-week wait’ route. After 11 months, the proportion of appropriately referred skin cancers (squamous cell carcinomas and melanomas) was 20.6%, compared with 23.2% before the intervention. The remaining 10 interventions were delivered over several sessions (see Table 5), although the exact number and timing of sessions was not always well described.
Hands et al. 34 reported an intervention where GPs attended outpatient sessions in different clinical specialties of their choice. GPs reported changes in their clinical behaviour which appear to have been maintained at 6 months. GPs stated that referral was discussed/taught in 83% of interactions. Immediately after the session, 25% of GPs reported that this would change their referral behaviour. After 6 months, 29% reported behaviour change in reference to referral.
Hilty et al. 24 implemented the following educational strategies. (1) Regular continuing medical education lectures. (2) GP participation in consultations: GPs present their patients at the beginning of the sessions, and get direct feedback at the end. (3) Consultation notes for GPs: a note by the psychiatrist was sent within 10 minutes of each consultation in a deliberately educational style. A dictation of two to three pages was sent in about 5 working days. (4) Telephone consultations with the psychiatrist. Among the first 200 consultations, only 47.4% of the medication doses for depressive and anxiety disorders were adequate, according to national guidelines. Among the second 200 consultations, dosing adequacy improved to 63.6% (p < 0.001). GPs rated the quality of consultation as significantly higher over time (95% CI 4.45 to 4.83, p < 0.001), as with overall satisfaction (95% CI 4.49 to 4.73, p < 0025). This study was considered to be at higher risk of bias.
Suris et al. 35 carried out biweekly educational sessions with GPs for 1 year (a total of 120 sessions carried out by four rheumatologists). At the end of the pilot year the total number of GP referrals was 31% lower than the previous year (1141 vs. 1652, no significance levels reported). The referral rate to the rheumatology unit decreased significantly from 8.13 per 1000 to 5.53 per 1000 (2.6, 95% CI 2.09 to 3.10; p < 0.001).
Wolters et al. 36 delivered a distance-learning programme accompanied with educational materials or a control group only receiving mailed clinical guidelines. The distance-learning programme comprised: (1) a package for individual learning developed by the Dutch College of General Practitioners; (2) consultation supporting materials: a voiding diary, the international prostate symptom score (IPSS) and Bother score; (3) the guideline summarised into two decision trees [one on clinical management of lower urinary tract symptoms (LUTS) and one on prostate-specific antigen (PSA) testing] and a brief explanation; and (4) two information leaflets for patients (on PSA testing and on treatment for LUTS). The intervention group showed a lower referral rate to a urologist [odds ratio (OR) 0.08, 95% CI 0.02 to 0.40], but no effect on PSA testing or prescription of medication.
Six further studies delivered over several sessions did not show a clearly positive effect on referral outcomes. Four of these were at lower risk of bias: Bhalla et al. 37 delivered three or four clinical ENT sessions over a 2-week period, once a year for 3 years to one partner in a GP practice. There was no statistical difference in referral rates (Kruskal–Wallis: p = 0.63) for the trained partner when compared with the other three partners in the same practice. There was also no statistical difference in referral patterns between the intervention and the control practice (Mann–Whitney U-test p = 0.50).
Ellard et al. 38 completed six 2-hour interactive sessions on common skin conditions in early 2011. Appropriate referrals from participants increased from 37.2% in 2010 to 51.8% after training, accompanied by an increase in the mean number of referrals from 20.7 to 25.7. Furthermore, the overall number of appropriate referrals increased from 37.8% to 49.5% at participating surgeries. However, these results were compared with the 36 other local GP practices that did not participate in the training programme, which also displayed an increase in appropriate referrals from 40.8% to 56.4% from 2010 to 2011.
Lester et al. 39 reported an intervention consisting of a 17-minute video, a 15-minute question-and-answer session, and two refresher educational sessions conducted over 4 months. Ninety-seven people with a first episode of psychosis were referred by intervention practices and 82 people from control practices during the study: RR of referral 1.20 (95% CI 0.74 to 1.95, p = 0.48). No effect was observed on secondary outcomes except for ‘delay in reaching early-intervention services’, which was statistically significantly shorter in patients registered in intervention practices (95% CI 83.5 to 360.5, p = 0.002).
Emmerson et al. 40 developed a psychiatric assessment and advisory service for local GPs. Five full-time psychiatrists dedicated a 1-hour appointment per week in their hospital private practice clinics to assess patients referred by local GPs. After 12 months referrals to the clinic were disappointing (n = 30, with 10 referrals from one GP). Feedback from GPs who had used the service showed high levels of satisfaction with the service (mean score 6.2 out of 7). Feedback from GPs who had not used the service showed a strong endorsement of the concept (94%), but there was poor awareness of the service’s existence (26%).
There were also two studies of interventions delivered over several sessions which were at higher risk of bias. Lam et al. 25 conducted an evaluative study to examine the impact of a 1-year part-time Postgraduate Diploma in Community Geriatrics. The diploma includes the components of clinical attachment (20 sessions of clinical geriatric teaching and five sessions of rehabilitation and community health services), interactive workshops, locally developed distance-learning manual, written assignments and examination as well as a clinical examination. Most respondents did not refer elderly patients to private geriatricians and would refer them to public geriatricians or other specialists. After the course, the average percentage of elderly patients being referred to private geriatricians increased from 2.8% to 6.1% and to other specialists decreased from 53.4% to 49.1%. The changes in the referrals to private geriatricians and other specialists were statistically significant. However, no significant change was found in the referrals to public geriatricians. The average percentage remained around 44%. It is unclear which of those outcomes were beneficial or how this study could be applied in a UK context.
Finally, Rowlands et al. 26 implemented an educational intervention consisting of referral meetings. Fewer than half of doctors became involved with development of formal referral or clinical protocols. Eighty-eight per cent noted a change in their referral practice. Overall, there was no change on referral rate in the intervention group. This study was considered to be at higher risk of bias.
Guidelines (no training or feedback)
Interventions that consisted of guidelines mailed to GPs (with no further training, support or feedback) were reported in 12 studies (Table 6). 41–52 The guidelines were for a range of referral conditions and procedures including genetic screening, orthopaedics, complications of diabetes, dementia, dermatology (two studies43,49), radiography (two studies42,50) and cancer (three studies41,46,47). Overall, seven studies reported at least some positive impact on referral,41–47 with five showing no effect or a negative change. 48–52 Two of the positive impact studies were considered to be at higher risk of bias41,42 with all other studies at lower risk of bias. Overall, the strength of this evidence was graded as inconsistent.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Cusack 200543 | NICE guidelines and a pro forma | BA | UK | Dermatology | 36 GPs | (18 months) |
| 150 referrals | ||||||
| Fearn 200948 | QOF Depression Indicators | BA | UK | Dementia clinic | NR | (18 months) |
| Hill 200049 | Local guidelines | Audit | UK | Dermatology | 33 GP practices | (2 years) |
| 422 patients | ||||||
| Idiculla 200044 | Local guidelines | RCT | UK | Outpatient infertility clinic | 214 GP practices | 1 year |
| 689 referrals | ||||||
| Most aged over 34 years, 84% female only | ||||||
| Imkampe 200647 | Pro forma for breast cancer referral | BA | UK | Oncology (breast cancer) | 2354 referrals | (8 months) |
| Lucassen 200145 | Local guidelines | BA | UK | Regional genetics service | NR | 14 months (6 months) |
| Malik 2007 41 | 2-week wait cancer guidelines | Audit | UK | Oncology (bone or soft tissue tumour) | 40 patients | 2 years |
| Matowe 200250 | Royal College of Radiology referral guidelines | BA | UK | Radiology | 376 GPs in 87 practices | (3 years) |
| 117,747 referrals | ||||||
| Melia 200851 | Prostate Cancer Risk Management Programme guidelines | BA | UK | Urology | 200 GP partners in 48 practices | 1–2 years |
| Male patients aged 45–84 years, n = 1520 | ||||||
| Potter 200746 | 2-week wait cancer guidelines | Cohort | UK | Oncology (breast cancer) | 24,999 new referrals | (7 years) |
| Twomey 2003 42 | Local guidelines | BA | UK | Radiology | NR | 2 years |
| West 200752 | Local guidelines | BA | UK | Orthopaedic outpatient department | 471 referrals | 29 weeks |
Seven studies showed a positive effect on at least one referral outcome (although results were often borderline or mixed). Five of these studies were considered to be at lower risk of bias.
Cusack and Buckley43 analysed dermatology referral letters from GPs prior to guidelines and 60 following guideline introduction. NICE guidelines and a pro forma for future referrals were sent to GPs. The percentage of referrals in accordance with NICE guidelines increased from 31% to 45% after introduction of guidelines (p = 0.041). The percentage of inappropriate referrals decreased from 69% to 55%, and 22% of GPs (8 of 36) fully complied with guidelines. However, over 50% of referrals were still inappropriate. The pro forma was used in only 23% of referrals and the provision of data in referral letters remained poor. The number of referrals per month only marginally decreased.
Idiculla et al. 44 analysed 200 GP referral letters submitted before (set 1) and 200 submitted after (set 2) local guidelines on the management of adult diabetes had been issued to local GPs. Following the distribution of the guidelines there was no significant change in the frequency with which specific conditions were documented in referral letters (set 1 vs. set 2): for example, hypertension 72% versus 79%, cerebrovascular disease 89% versus 80%. However, the guidelines did appear to have encouraged the active treatment of hyperglycaemia by GPs before referral.
Lucassen et al. 45 sent referral guidelines for a regional genetics service family cancer clinic to GPs and subsequent content of referral letters was analysed and compared with the previous 6 months. Post guidelines, more referrals met the criteria than before (χ2 = 15.79, p < 0.001). Fewer lower-risk referrals were made: 34% of letters (36/103) were high risk pre guidelines, whereas 47% (46/110) were high risk post guidance (not significant: χ2 for change in proportion of low risk pre and post = 1.34; p = 0.24, and for high risk χ2 = 3.33, p = 0.07). The description of the risk in the GP letter improved so that a greater proportion of generic clinic risks agreed with those described in the GP letter.
Potter et al. 46 used routine data to consider the effect of the introduction of the 2-week wait guideline for cancer referrals. The annual number of referrals increased over 7 years from 3499 in 1999 to 3821 in 2005, a significant increase of 1.6% (95% CI 1.0% to 2.2%). The number of 2-week wait referrals increased by 42% (n = 739) from 1751 in 1999 to 2490 in 2005, an estimated increase of 5.8% per year (5.0% to 6.7%, p = 0.001). By contrast, the number of routine referrals has declined over the same period by an estimated 4.3% a year (3.3% to 5.2%, p < 0.001), giving an apparent reduction of 24% (n = 417) from 1999 to 2005. The percentage of patients diagnosed with cancer in the 2-week wait group decreased from 12.8% (224/1751) in 1999 to 7.7% (191/2490) in 2005 (p < 0.001), whereas the number of cancers detected in the ‘routine’ group increased from 2.5% (43/1748) to 5.3% (70/1331) (p < 0.001) over the same period. About 27% (70/261) of people with cancer are currently referred in the non-urgent group. Waiting times for routine referrals have increased with time.
Imkampe et al. 47 determined whether or not GP grading of referrals into urgent and non-urgent had improved after the introduction of the 2-week rule was introduced. A retrospective review of GP referrals over 8 months, between September 2003 and April 2004, with regard to their urgency, subsequent diagnosis and the use of standardised referral formats was carried out. The results were compared with the 1999 audit. Eighty-two of 1178 patients referred by GP had breast cancer versus 115 of 1176 patients referred in 1999. Sixty-eight per cent (56/82) of breast cancer patients were referred as urgent, compared with 47% (54/115) in 1999 (p = 0.005). A pro forma was used in 47% (548/1178) of GP referrals, while no pro forma was used in 1999. Sixty-five of the 82 cancer patients were referred with a pro forma and 85% (55/65) were referred as urgent.
Two further studies which showed a positive effect on at least one referral outcome were at higher risk of bias. Malik et al. 41 determined if the 2-week wait referral guidelines for suspect cancer referrals had been followed and what proportion of patients referred under the guideline had malignant tumours. Referral letters were evaluated to see if they met Department of Health guidelines for referral of a suspected bone or soft tissue tumour. Most (31 of 40: 78%) ‘2-week’ referrals met the published referral guidelines. However, in 9 of the 40 cases, the patient did not meet the criteria for urgent referral, and none of the nine patients had malignant tumours. Of 40 patients referred under the guideline, 10 of these patients (25%) had malignant tumours, but this was compared with 243 of 507 (48%) of those referred from other sources. Twomey42 assessed GP referral for plain radiography in the areas of hip, knee, cervical spine and lumbar to establish a procedure for the development of care pathways. The proposed guidelines were circulated to all GPs. GP referrals to radiology for plain radiography declined from 2365 the year before the intervention to 1077 the year after intervention, a total reduction of 288 (54%). Similarly, referrals for plain radiography requests declined from 6650 to 4291, a reduction of 2359 (35.5%).
Five further studies (all at lower risk of bias) of dissemination of referral guidelines showed no effect, or a negative effect, on referral outcomes.
Fearn et al. 48 looked at whether or not the introduction of Quality and Outcomes Framework (QOF) Depression Indicators changed the pattern of referrals from primary care to a dedicated dementia clinic. The percentage of all referrals originating from primary care was about half in both time periods and did not differ significantly between the two time periods (χ2 = 0.88, df = 1, p > 0.1; z = 0.77, p > 0.05). Of the referrals from primary care, about one-third referred in both time periods had dementia. The RR of a diagnosis of dementia in a primary care referral pre and post QOF was 0.55 (95% CI 0.40 to 0.74) and 0.66 (95% CI 0.49 to 0.89), respectively. The proportion of patients referred from primary care with dementia was the same in the cohorts seen both before and after introduction of the QOF Depression Indicator (χ2 = 0.54, df = 1, p > 0.05), a finding corroborated by the z-test (z = 0.60, p > 0.05).
Hill et al. 49 evaluated referral guidelines for dermatology compiled by the dermatologist at the Royal Surrey County Hospital in consultation with local GPs. A 40% increase was seen in the numbers of referrals recorded by the dermatologist as appropriate immediately after the guidelines were sent (from 57% to 80%). The 2-year follow-up audit, however, demonstrated that the improvement had not been sustained, with a decline to 48% appropriate referrals.
Matowe et al. 50 mailed copies of the Royal College of Radiology referral guidelines for chest, limb and joint, and spine radiographs to GPs. There were no significant effects of the intervention on total number of general practice imaging requests. Total referrals decreased by 32 (95% CI –226.7 to 291.4) in the month following guideline dissemination, while the trend decreased by –1.82 requests per month (95% CI –11.8 to 8.2 requests per month). Referral only decreased by average 1.2 per month for the entire 35-month period.
Melia et al. 51 disseminated the Prostate Cancer Risk Management Programme (guidelines for GPs on age-specific PSA cut-off levels in asymptomatic men). One year after intervention, awareness of the pack was acknowledged by 112 (56%) GPs (24 were unaware and 64 did not know if they had seen it). The proportion of asymptomatic men referred who had raised antigen levels did not increase significantly from baseline to intervention (24% pre intervention, 29% post intervention; p = 0.42) There was no significant difference in referral rate by area (p = 0.33).
West et al. 52 completed a 13-week audit of referral letters for six specific orthopaedic complaints, namely anterior knee pain, back pain, carpal tunnel syndrome, in-toeing in children, sciatica and tennis elbow. Paper copies of referral guidelines produced by orthopaedic consultants were distributed to all local GPs. After a period of 4 weeks for distribution, the process was repeated for a further 13 weeks. The first 13-week period had 195 (64%) referrals that consisted of patients who had not received the recommended management or to whom this had not been mentioned in the referral letter. The second period had 103 (61%). There was no statistically significant difference between the two (p = 0.49).
Guidelines with additional training or feedback
Interventions consisting of guidelines with additional training or feedback were reported in 18 studies (all lower risk of bias), of which 11 showed a positive association with referral outcomes53–63 and six did not (Table 7). 64–69 The guidelines were for a range of referral conditions and procedures including mental health, infertility clinic, dermatology, gynaecology, oncology, colorectal surgeon, urology, cardiology (two studies56,63), low-back pain (two studies66,87), endoscopy (two studies53,55) and radiology (four studies54,57,59,60).
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Banait 200353 | Educational outreach/dyspepsia management guidelines | cRCT | UK | Open-access endoscopy (GI) | 114 practices | (6 months) |
| 233 GPs | ||||||
| Dey 200466 | RCGP guidelines plus outreach visits | cRCT | UK | Low-back pain | 24 health centres | (8 months) |
| 2187 patients; age 18–64 years (mean 42.2 years, SD 12.1) | ||||||
| 54% female | ||||||
| Eccles 200154 | RCGP guidelines, audit and feedback, or educational messages | RCT | UK | Radiology | Six radiology departments; 244 general practices | (1 year) |
| Elwyn 200755 | NICE guidelines plus feedback | BA | UK | Endoscopy (dyspepsia) | 215 GPs | (5 months) |
| Three endoscopy units | ||||||
| Engers 200567 | National guidelines plus workshop | cRCT | the Netherlands | Low-back pain | 41 GPs | NR |
| 531 patients | ||||||
| Glaves 200557 | Guidelines plus return of referrals | BA | UK | Radiology (spine and knee) | Three community hospitals | (1 year) |
| Griffiths 200658 | Local guidelines and training sessions | cRCT | UK | Dermatology | 165 health centres | NR |
| Patients 18+ years with psoriasis n = 188 | ||||||
| Jiwa 200668 | Local guidelines plus visit | cRCT | UK | Colorectal surgeon | 44 practices | (6 months) |
| 180 GPs | ||||||
| 504 patients | ||||||
| GPs 30–60 years | ||||||
| Julian 200762 | Shared care guidelines | nRCT | UK | Gynaecology | 193 GP practices | (8 months) |
| One hospital | ||||||
| Kerry 200059 | Royal College of Radiology guidelines plus feedback | RCT | UK | Radiology (spinal exam) | 69 GP practices | 2 years (9 months) |
| Morrison 200164 | Local guidelines plus meeting | RCT | UK | Outpatient infertility clinic | 214 GP practices | 1 year |
| 689 referrals | ||||||
| Age 34+ years | ||||||
| 84% female | ||||||
| Robling 200260 | Local guidelines plus seminar or newsletter | RCT | UK | Radiology (MRI) | 121 GP practices | NR |
| 182 referrals | ||||||
| Spatafora 200569 | Local guidelines plus meeting | BA | Italy | Urology (outpatients) | 45 urological centres, 263 GPs | NR |
| GPs’ mean age 47 years | ||||||
| 18% female | ||||||
| Walkowski 200763 | Local guidelines, telephone call, e-mail, or in-person visit | cRCT | USA | Cardiology | Five US states | 15 months (3 months) |
| White 200461 | Local guidelines plus implementation strategy | Audit | UK | Mental health | NR | (2 years) |
| Wilson 200665 | Local guidelines plus education meetings and outreach | cRCT | UK | Oncology (familial breast cancer) | GP in Grampian | 4 years (11 months) |
| Wright 200656 | Guidelines, educational meetings, outreach visits | CBA | UK | Cardiology (post TIA for stroke prevention) | One PCT | 50 months (22 months) |
Eleven studies showed a positive relationship between the intervention and referral-related outcomes. 53–63
Banait et al. 53 implemented educational outreach as a strategy for facilitating the uptake of dyspepsia management guidelines in primary care for open-access endoscopy. All groups received the guidelines by post and the intervention groups began to receive education outreach 3 months later. The outreach included practice-based seminars with hospital specialists at which guidelines recommendations were appraised and implementation plans formulated, and was reinforced by visits after 12 weeks. The proportion of appropriate referrals was higher in the intervention group in the 6-month post-intervention period (practice medians: control = 50%, intervention = 63.9%; p < 0.05). The proportion of major findings at endoscopy did not alter significantly, but there was an overall rise in acid-suppressing drugs in the intervention, compared with the control group (+ 8% vs. + 2%, p = 0.005).
Eccles et al. 54 compared two methods of reducing GP requests for radiological tests in accordance with the UK Royal College of Radiologists’ guidelines on lumbar spine and knee radiographs. GPs and consultant radiologists wrote referral guidelines and educational messages for lumbar spine and knee radiographs [based on the Royal College of Radiologists’ guidelines and the Royal College of General Practitioners’ (RCGP) back-pain guidelines]. The referral guidelines were then sent by post to all study GPs. Each practice was randomly allocated to receive audit and feedback or control; and educational messages or control. Feedback covered the previous 6 months’ referrals and was sent to GPs at the start of the intervention period and 6 months later. Educational messages were attached to the reports of every knee or lumbar spine radiograph requested during the intervention. The effect of educational reminder messages (i.e. the change in referral rate after intervention) was an absolute change of 1.53 (95% CI 2.5 to 0.57) for lumbar spine and of 1.61 (2.6 to 0.62) for knee radiographs (relative reductions of ≈20%). The effect of audit and feedback was an absolute change of 0.07 (1.3 to 0.9) for lumbar spine and 0.04 (0.95 to 1.03) for knee radiograph requests (relative reductions of 1%). Requests from doctors who had received audit and feedback were no more likely to be appropriate than requests from other doctors: OR 0.75 (95% CI 0.52 to 1.07) for lumbar spine radiographs and 0.82 (0.50 to 1.33) for knee. For doctors who had received educational reminder messages, the equivalent values were 0.95 (0.63 to 1.67) and 1.36 (0.86 to 2.23).
Elwyn et al. 55 evaluated a system of providing feedback to clinicians following referral requests not adhering to NICE guidelines. Letters were sent to GPs stating that two GPs would be employed part-time to assess all endoscopy letters and referrals for dyspepsia and they would be judged against recently issued NICE guidelines. Where referrals did not meet the criteria, the referring doctor would be informed by letter giving a reason for non-adherence to guidelines. The All Wales Dyspepsia Guidelines based on NICE criteria were circulated to GPs 2 weeks earlier. Adherence to NICE guidelines for referral criteria increased significantly among GPs following the intervention (mean 55% to 75%; 95% CI 13.6 to 26.4; p < 0.001). No similar effect was seen for hospital doctors. The number of gastroscopy referrals for dyspepsia declined after the intervention, but not significantly after inclusion of seasonal effects (p = 0.065). Intervention significantly reduced the referral to procedure time for gastroscopy (mean 52.1 to 39.4 days, 95% CI 6.6 to 18.6 days; p < 0.001).
Wright et al. 56 completed an evaluation of a quality improvement programme for transient ischaemic attack (TIA) referral in three primary care trusts (PCTs). Four local consensus group meetings for relevant stakeholders (including service users and carers) were used to adapt national guidelines to local context and identify barriers and incentives for changing practice. Guideline reminders for clinicians included laminated posters, desktop coasters and electronic referral templates. Guidelines were disseminated via education meetings in each PCT and further education outreach visits to 19 practices. Guidelines were disseminated by post to other practices not requesting a visit. There was a 41% increase in referrals from trained practices, compared with control practices (RR 1.41, p = 0.018). Adherence to best-practice standards was significantly higher in practices that had received the training programme than in the controls.
Glaves57 undertook an intervention where GPs referring to three community hospitals and a district general hospital were circulated with referral guidelines for radiography of the cervical spine, lumbar spine and knee. All requests for these three examinations were checked and requests that did not fit the guidelines were returned to the GP with an explanatory letter and a further copy of the guidelines. If the GP maintained the opinion that the examination was indicated, they had the option of supplying further information in writing or speaking to a consultant radiologist to reach agreement. The total number of examinations fell by 68% in the first year (95% CI 67% to 69%) and 79% in the second year (95% CI 78% to 80%). Knee radiographs fell by 64% in the first year (95% CI 62% to 65%) and 77% in the second year (95% CI 75% to 79%). Lumbar spine radiographs fell by 69% in the first year (95% CI 68% to 71%) and 78% in the second year (95% CI 77% to 80%). Cervical spine radiographs fell by 76% in the first year (95% CI 74% to 78%) and 86% in the second year (95% CI 84% to 88%) (p = 0.001 for all measures).
Griffiths et al. 58 evaluated the effectiveness of guidelines and training sessions on the management of psoriasis in reducing inappropriate referrals from primary care. Guidelines on the management of psoriasis in primary care, developed by local dermatologists, were sent to health centres in the intervention arm, and supplemented by the offer of a practice-based nurse-led training session. Patients in the intervention arm (82/105) were significantly more likely to be appropriately referred than patients in the control arm (49/83), a difference of 19.1% [OR 2.47; 95% CI 1.31 to 4.68; intracluster correlation coefficient (ICC) 0]. Only 25 (30%) health centres in the intervention arm took up the offer of training sessions. There was no significant difference in outcome between health centres in the intervention arm that received a training session and those that did not (OR 1.28, 95% CI 0.50 to 3.29; ICC 0).
Kerry et al. 59 evaluated the introduction of radiological guidelines into general practices, together with feedback on referral rates, to see whether or not this reduced the number of GP radiological requests over 1 year. A GP version of the Royal College of Radiologists guidelines was sent to each GP in the 33 practices in the intervention group. Guidelines for examination of chest, hips, knees, spine, skull and sinuses were printed verbatim on two sides of a sheet of A4 paper, which was then laminated. After 9 months’ intervention, practices were sent revised guidelines with individual feedback on the number of examinations requested in the past 6 months. A total of 43,778 radiological requests were made during the 2-year intervention. The number of referrals for all spinal examinations fell by 18% in the intervention group, compared with a 2% rise in the control group (p = 0.05). Taking requests for the lumbar spine alone, there was a reduction of 15% in the intervention group, compared with a rise of 5% in the control group, giving a difference of 20% between the groups (95% CI 3% to 37%). Overall, an 8% reduction in total numbers of radiological requests was observed in the intervention group, compared with a 2% increase in the control group (10% between the two groups, not significant).
Robling et al. 60 investigated whether or not method of access or method of guideline dissemination affects GP compliance with referral guidelines for magnetic resonance imaging (MRI) in two sequential trials: (1) one group of practices requesting MRI by telephone was compared with a second group requesting in writing using a standard request form. A third group could refer as wished; and (2) one group of practices receiving guidelines via a seminar was compared with a second group who received feedback via a newsletter with practice-specific data on referrals. A third group received both a seminar and feedback, and a fourth group received guidelines only by post. The seminars were facilitated by an academic GP and a researcher. In trial 1, 65% of requests were judged to be compliant with the guidelines and there were no statistical differences between the three groups. Telephone access proved unpopular among participants and written access more cost-effective. In trial 2, 74% of referrals were judged to be compliant with the guidelines and there was no association between method of dissemination of guidelines and compliance. Requests made after dissemination of guidelines were more likely to be compliant: 74% versus 65% (OR 1.62, p < 0.005).
White et al. 61 aimed to use guidelines to improve communication between GPs and community mental health teams (CMHTs). Following a baseline audit of referrals and assessment letters, locally agreed good practice protocols were developed and shared widely, accompanied by a dissemination and implementation strategy (updates at 6-monthly intervals throughout the project). Significant improvements occurred in both the GP and the CMHT letters. These were most dramatic after 1 year but tailed off considerably in the second year despite continued efforts to implement the protocol’s standards. Annual GP referrals (percentage of total) reduced from 661 (63%) to 550 (58%), p-value not significant, and new referrals completing CMHT assessment increased from 369 (66%) to 423 (89%) (p < 0.001).
Julian et al. 62 examined the outcomes of an integrated model. Women attending the new ‘Bridges’ pathway were compared with those attending a consultant-led one-stop menstrual clinic. The Bridges pathway involved the use of shared care evidence-based guidelines for the management of dysmenorrhoea patients in primary and secondary care, which determined the timings for investigations and surgical treatment. Management decisions were made by GPs in all but atypical/complex cases. At 8 months, there were no significant differences between the groups in terms of surgical and medical treatments of in the use of GP clinic appointments. Significantly fewer hospital outpatient appointments were made in the Bridges group than in the one-stop menstrual clinic (p < 0.001). Patient diaries demonstrated a significant improvement in the Bridges group for patient information, ease of access (p < 0.001), choice of doctor (p < 0.002), waiting time (p < 0.001) and less ‘limbo’ between primary and secondary care (p < 0.001).
Walkowski et al. 63 tested the effect of different strategies to inform GPs of the high performing cardiac specialists in their community and facilitate increased referrals to these specialists. This initiative involved sending letters to primary care physicians which requested that when the physician had a patient needing referral to a cardiac specialist or facility, they refer that patient to a physician or facility that had earned the ‘United Health Premium designation for both Quality and Efficiency of care’. To facilitate those referrals, the primary care physicians were provided with a hard-copy referral list of cardiac specialists and hospitals. Participants were divided into four test groups: (1) letter and referral list (LRL) only (n = 3537); (2) LRL plus follow-up telephone call from the local health plan (n = 252); (3) LRL plus e-mail reminder (n = 1187); or (4) LRL plus in-person follow-up visit from the local market medical director (n = 65). The initial 3-month pilot data showed an overall 6.3% increase of patients referred to United Health Premium-designated quality and efficient cardiac specialists overall, compared with a baseline period of 12 months prior to the mailing. Intervention effects ranged from 17% change (letter plus call) to 22% change (letter plus visit), versus 0.3% change in the control group. The applicability of this study in the UK may be limited.
Six further studies (all at lower risk of bias) of dissemination of referral guidelines with additional support or training showed no effect or a negative effect on referral outcomes. 64–69
Morrison et al. 64 evaluated the effect of clinical guidelines on the management of infertility in general practice. Local guidelines were developed and a management pack was sent to intervention practices with an invitation to attend a meeting to discuss using the guidelines. Seventeen per cent of doctors attended a meeting. Individual visits were also offered but were taken up by only two practices. There was no difference between the control and intervention practices with regard to whether or not a management plan was made (OR 1.239, 95% CI 0.869 to 1.765; p = 0.236). There was also no difference in duration between first appointment and date of management plan, no difference in mean number of outpatient visits before a management plan was put in place, and no significant difference in total costs to the NHS (£349.78 vs. £327.48, p > 0.05).
Wilson et al. 65 considered the effectiveness of an intervention to improve GP confidence in managing patients concerned about genetic risk of breast cancer. Components of the intervention software included a list of the key patient information needed in order to use the guidelines. A risk assessment module was presented as a set of short checklists, in which the Scottish referral guidelines for breast, ovarian and colorectal cancer were embedded. This was provided along with the following: background information on cancer genetics and the evidence underlying the guidelines (prepared by local geneticists); printer-ready, locally customised patient information leaflets; selected web-links for professionals and patients; and a contact e-mail link with the Cancer Genetics Service, with a guaranteed response time. The system automatically produced a draft referral letter using the regionally recommended template. All partners in intervention practices were invited to interactive workshops on cancer genetics designed to complement the software. No statistically significant differences were observed between intervention and control arms in the primary or secondary outcomes. Only a small proportion of intervention GPs attended the educational session, were aware of the software or made use of it in practice. In the pre-intervention period, intervention GPs were less likely than control GPs to refer patients who were eventually assessed as having elevated genetic risk (0.70, 95% CI 0.50 to 0.99), with the opposite trend observed in the post-intervention period (1.18, 95% CI 0.88 to 1.37), although these results did not reach statistical significance.
Dey et al. 66 evaluated the impact on patient management of an educational strategy to promote the RCGP’s low-back-pain guidelines among GPs. Practices in the intervention arm were offered outreach visits to promote national guidelines on acute low-back pain, as well as access to fast-track physiotherapy and to a triage service for patients with persistent symptoms. At least two members of the guideline team attended each visit; these included senior representatives from the musculoskeletal directorate, physiotherapy services and the health authority. Members of the guideline team facilitated a structured interactive discussion with the GP to raise awareness of the RCGP guidelines, adapted to the local context; emphasise the key messages in the guidelines; identify potential barriers to implementation; and suggest strategies for overcoming the barriers identified. GPs were given a poster reinforcing guideline recommendations and a copy of a text recommended by the RCGP for patients. The estimated annual consultation rate for acute low-back pain was 35 per 1000 adults in the intervention group, compared with 38 per 1000 in the control group. There were no significant differences between study groups with respect to the proportion of patients who were referred for radiography (a difference of 1.4%, 95% CI –4.1% to 6.8%), issued with a sickness certificate (a difference of –1.5%, 95% CI –10.3% to 7.3%), prescribed opioids or muscle relaxants (a difference of –0.03%, 95% CI –5.5% to 5.4%) or referred to secondary care (a difference of 1.1%, 95% CI –0.3% to 2.6%). Significantly more patients in the intervention group were first referred to physiotherapy or to educational programmes at the back pain unit than in the control group (a difference of 12.2%; ICC = 0.0563; χ2 = 6.49; 1 df; p = 0.01; 95% CI 2.8% to 21.6%).
Engers et al. 67 assessed the effectiveness of the Dutch low-back-pain guideline for GPs with regard to adherence to guideline recommendations. GPs in the intervention group received a tailored interventions consisting of the Dutch low-back-pain guideline for GPs, a two hour educational and clinical practice workshop; two scientific articles on low-back-pain management; the guideline for occupational physicians; a tool for patient education; and a tool for reaching agreement on low back care with physical, exercise and manual therapists. The participating GPs were asked to recruit consecutive patients with a new episode of low-back pain as the main reason for consultation. The intervention was delivered by a psychologist-physiotherapist. Forty-one of the 67 randomised GPs reported on a total of 616 consultations for 531 patients with non-specific low-back pain. The advice and explanation provided by the GPs, the prescription of paracetamol (33% vs. 21%) or non-steroidal anti-inflammatory drugs (54% vs. 62%), and prescription of pain medication on a time contingent (70% vs. 69%) or a pain contingent basis (30% vs. 31%), showed no statistically significant differences between the intervention and control groups. There were also no differences in overall referral rate (23% vs. 28%; OR 0.8, 95% CI 0.5 to 1.4). However, in follow-up consultations fewer patients were referred to a physical or exercise therapist by the GPs in the intervention group than in the control group (36% vs. 76%; OR 0.2, 95% CI 0.1 to 0.6).
Jiwa et al. 68 evaluated a referral guideline intervention for lower bowel symptoms. GP practices were offered one of an electronic interactive referral pro forma, an educational outreach visit by a local colorectal surgeon, both or neither. They developed and piloted an interactive electronic pro forma for processing referrals to colorectal surgeons (General Practice Referral Assessment Facilitator or G-RAF). The interactive pro forma requested information on drop-down menus for 15 clinical signs and symptoms previously identified by GPs and colorectal surgeons as those of significant colorectal disease. The interactive software offered the practitioner guidance on which cases needed urgent referral with reference to current UK Department of Health guidelines. A referral letter was automatically produced seeking an appropriate appointment at a hospital clinic. The educational outreach visit was delivered by a colorectal surgeon. During the 45-minute meeting, the presenter summarised the features of significant organic colorectal disease and encouraged questions. There were 716 consecutive referrals recorded over a 6-month period, for which a diagnosis was available for 514. There was no significant difference in proportion of cases with significant pathology for either intervention or compared with no intervention. In the combined software arms 14% (37/261) had significant pathology, compared with 19% (49/253) in the non-software arms: RR 0.73 (95% CI 0.46 to 1.15). In the combined educational outreach arms 15% (38/258) had significant pathology, compared with 19% (48/256) in the non-educational arms (RR 0.79, 95% CI 0.50 to 1.24).
Spatafora et al. 69 developed a short algorithm on procedures to be used with men with LUTS. The algorithm was developed by urologists and approved by a panel of experts. It was presented at a meeting with local GPs and revised in line with feedback, and the revised protocol was presented at each centre. The protocol was a clinical report form containing history, examination, use and outcome of tests, and diagnosis. Sixteen per cent of centres accepted the original protocol with no changes. There was no significant change in referral pattern from baseline to intervention: 51.2% of patients were managed entirely by their GP, 44.3% were referred to urologist after some diagnostic procedures and 4.5% were referred without any diagnostic testing. Use of digital rectal exams increased significantly from 32% to 41% (p < 0.001) and this was predominantly in centres that endorsed this test.
Process change interventions
We defined process changes as small-scale changes to some aspect of the individual referral process which did not involve the movement of staff or relocation of clinics, the methods in which referrals were triaged at hospital or financial arrangements for referral.
Process change interventions included designated appointment slots and fast-track clinics for primary care referrals (n = 6), interventions that provided direct access to screening (n = 9), specialist consultation prior to referral (n = 11), electronic referral systems (n = 10), the provision of decision support tools to assist GPs in making referrals (n = 10) and interventions that consisted of waiting list review or watchful waiting (n = 3).
Designated slots/fast-track clinics
The provision of designated appointment slots and fast-track clinics for primary care referrals were reported in six studies (Table 8). 70–75 The speed of referral varied from the same day to within 2 weeks (to meet the 2-week cancer referral guidelines). Four studies showed a positive effect,70–73 with two studies showing a negative or no effect. 74,75 One effective study was considered to be at higher risk of bias. 72 The evidence overall was rated as inconsistent.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Bridgman 200570 | Quota appointment slots | CBA | UK | Orthopaedic (outpatients) | 36 practices | (18 months) |
| 33 GPs | ||||||
| 30 full-time | ||||||
| Three single practices | ||||||
| Hemingway 200673 | Protocol-driven fast-track referral system | BA | UK | Colorectal cancer screening | Eight surgeons, 10 GI physicians | (2 years) |
| Khan 200871 | Direct referral to Hot Clinic | Cohort | UK | Respiratory (COPD) | 173 patients | 6 months |
| 97 (57%) men, 75% current or ex-smokers | ||||||
| McNally 200374 | Fast-track clinic | BA | UK | Oncology (ovarian cancer) | 242 patients | 6 years |
| Prades 201175 | Seven fast-track hospital indicators | Mixed method | Spain | Oncology (breast, lung, colorectal) | 56,020 patients | NR |
| 83 health professionals from 18 clinics | ||||||
| 38% GPs | ||||||
| Sved-Williams 2010 72 | Single entry point for psychiatry | BA | Australia | Psychiatry | 45 psychiatrists | 28 months |
| 301 GPs | ||||||
| 824 patients |
Bridgman et al. 70 evaluated a slot system for referrals. GPs and orthopaedic consultants were invited to a meeting to discuss and input into the design of the system. The number of slots available was based on the registered practice population. Quota of slots and their use was fed back to practices on a monthly basis. If a practice went beyond their quota they were told that they might not be allowed to refer any more patients that month. GPs guaranteed a maximum of 8 weeks’ assessment for patients and the backlog of waiting patients was removed. GPs received guidelines on appropriate referrals and routes of referrals for musculoskeletal problems. A clerical officer was appointed to answer queries and make appointments. After a modification to the hospital software, referrals were made using a special pro forma, which included a prioritisation score. In total, 15,439 referrals were made, and 90% attended their appointments. The mean monthly referral rate in the intervention group declined 22% in year 1 and was maintained in year 2. The difference in mean referral rate between the control and intervention was –1.59 intervention; –2.61 control; and –4.39 other comparator. The relative mean rate in reductions in mean referral rates were: 14.5%, –23.7% and –39.5% in period 0, year 1 and year 2, respectively.
Khan et al. 71 evaluated the efficacy of direct GP referral to a hospital respiratory specialist team to a ‘Hot Clinic’ in avoiding hospital admissions. GPs and community nurses directly referred patients threatening an acute hospital admission, by fax, for a rapid assessment. The Hot Clinic service operated Monday to Friday, 09:00–16:00 hours. Patients were seen within 24 hours of the receipt of the referral letter. The consultation included clinical assessment, chest radiograph, laboratory data and a decision whether to treat the patient in the community or to admit the patient to the hospital. The GP would be informed by a typed and faxed letter returned the same day. In total, 27 patients (16%) were admitted directly from the Hot Clinic and 146 (84%) were treated in the community. Of those 146 patients, nine (5%) were later admitted within 1 week and 12 (7%) admitted over 1 week to 1 month after the Hot Clinic appointment. Overall, 125 (72%) were treated successfully in the community without the need for hospitalisation. However, it is unclear if all would have been hospitalised without the clinic.
Sved-Williams and Poulton72 described and evaluated a service that provided a single point of entry for GPs wishing to refer their patients for one-off psychiatric consultations. All psychiatrists in the region were invited to provide reserved appointments to an administrative officer based at the Department of General Practice. They could specify the number of appointments and withdraw unfilled appointments at any time. To make an appointment, a GP or practice nurse phoned a dedicated number Monday to Friday 09:00–17:00 hours. There was no paperwork, and the GP was supplied with the appointment time, along with the name and contact details of the psychiatrist over the phone. From August 2005 to March 2007, 84% of offered appointments were filled. Use of the service rose from six referrals to 10 per week over the course of the study, and 55% of psychiatrists continued to provide regular appointments after the study period. This study was at higher risk of bias.
Hemingway et al. 73 evaluated a protocol-driven rapid-access referral system for colorectal cancer tests. The Leicester Colorectal Test Protocol included a list of presenting symptoms, age criteria for test and the appropriate diagnostic test for each symptom. Patients had investigations either before seeing an outpatient clinician or on the day of the clinic. Referrals were processed by ‘2-week wait’ administration staff using the protocol and assessments booked by these administration staff. There was protection of time slots within the testing suites. Referrals not complying with protocol were redirected to appropriate test without referral back to GP. The data that relate to the intervention period are not clear as they are reported by year rather than before and after. At baseline, the year 1 median time to diagnosis for non-emergencies was 35 days (interquartile range 13–80 days), compared with fast-track (categorised as 2-week wait or ‘soon’) 21 days (interquartile range 10–48 days). Sixty-two per cent of cancers referred as either 2-week wait or ‘soon’ were diagnosed within 31 days. After introduction of the intervention (pilot and full implementation) year 3 median time to diagnosis for non-emergencies was 20 days (interquartile range 10–59 days) and for emergencies was 13 days (interquartile range 8–29 days) [year 4 non-emergencies 20 days (interquartile range 10–51 days) and emergencies 13 days (interquartile range 9–23 days)]. During the 2-month full implementation period in year 3, the service received 256 referrals: 64% came through the 2-week wait protocol office and 36% were referred directly to consultants. In these referrals 70% were diagnosed with a pathology and 19 patients were diagnosed with cancer, all within 31 days. Overall during year 3, 79% of patients with colorectal cancer diagnosed who were referred as 2-week wait or ‘soon’ were diagnosed within 31 days. In year 4, the figure was 82%.
Two studies showed no association with referral outcomes. McNally et al. 74 implemented clinic appointments within 2 weeks to a fast-track breast cancer clinic. GPs were informed of the clinic and referral criteria by individual letter, GP newsletter and meetings. The median waiting time for referral to specialist was 3 days (range 0–188 days). This did not change significantly after clinic introduction (p = 0.05). The impact of fast-track clinic on referral and diagnosis time variables was not significant.
Prades et al. 75 analysed the implementation and effectiveness of a fast-track referral system for cancer which included clinical criteria for primary care referral and patient pathway management in hospital. There was an increase in completeness of hospital data during the intervention period (74% to 96%). Adherence to clinical criteria for including patients in the fast-track system was more than 70% (no specific data reported). About half of all new patients were diagnosed via the fast-track system and the cancer rate declined during the period. The mean time to treatment from primary care was 32 days for breast cancer, 30 for colorectal cancer and 37 for lung cancer. There are no data for patients not referred via the programme to compare these results with.
Direct access to diagnostic testing
Nine interventions provided direct access to diagnostic testing for a range of conditions and, as such, included a range of screening tests [including MRI, dual-energy X-ray absorptiometry (DXA) scanning, computerised tomography (CT) brain scanning, audiology screening, endoscopy, CT (for chronic daily headache) and an open-access urology unit] (Table 9). 76–84 Six studies showed a positive effect,76–81 but three studies were very unclear as to the effect on referral outcomes82–84 and all studies were considered to be at lower risk of bias. The strength of the evidence was graded as inconsistent.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| DAMASK 200876 | GP direct referral for MRI | RCT | UK | Radiology (MRI) | 386 patients | NR |
| Dhillon 200382 | Direct access to DXA scan | RCT | UK | Rheumatology | 330 patients | NR |
| 18 practices | ||||||
| Patients aged 31 to 89 years | ||||||
| Eley 201083 | Direct audiology referrals | Audit | UK | ENT (audiology) | 353 patients | (4 months) |
| 178 female, 175 male | ||||||
| Mean age 77 (60–96) years | ||||||
| Gough-Palmer 200984 | Direct MRI access | Audit | UK | MRI | 1798 scans | 12 years |
| 209 GPs | ||||||
| Shaw 200677 | Open-access serology | cRCT | UK | Serology (dyspepsia) | 47 practices | NA |
| Simpson 201078 | Direct access to head CT | Audit | UK | Neurology | 4404 referrals | NA |
| 986 GPs | ||||||
| Thomas 201080 | Direct access to head CT | Audit | UK | Neurology | 232 referrals | (1 year) |
| 72 practices, 309 GPs | ||||||
| Patient age range 20–85 years | ||||||
| Thomas 200379 | Open-access urology | cRCT | UK | Urology | 66 GPs | (12 months) |
| 959 patients | ||||||
| Wong 200081 | Open-access endoscopy | CBA | Hong Kong | Endoscopy (dyspepsia) | 1334 patients | (2 years 10 months) |
| Mean age 74 years |
DAMASK76 looked at a process to allow direct referral from general practice to a local radiology department for MRI to allow early access to imaging. Early MRI was associated with higher NHS cost by £294 per patient and a larger number of quality-adjusted life-years (QALYs) by 0.05. There was an incremental cost per QALY gained of £5840 below the cost threshold of £20,000 per QALY commonly used in the NHS. This was, therefore, considered to be a cost-effective use of NHS resources.
Shaw et al. 77 evaluated the effect of providing a Helicobacter pylori serology service for GPs who requested open-access endoscopy. General practices were stratified by endoscopy referral rate and randomised into two groups. The intervention group was provided with access to H. pylori serology testing and encouraged to use it in place of endoscopy for patients aged < 55 years with dyspepsia. They were sent written information promoting the use of the serology service in place of endoscopy for patients aged < 55 years suffering from dyspepsia without alarm symptoms and were issued with a summary of the Maastricht consensus statement on the management of H. pylori. The GPs remained free to refer for open-access endoscopy as they felt necessary. The number of endoscopy referrals fell in both groups during the study period, but fell by a greater amount in the intervention group than in the control group. During the 2-year study period, 626 referrals were received from the intervention group, compared with 771 from the control group. This accounted for a significant reduction in referrals for endoscopy in the intervention group compared with the control group: 18.8% difference (95% CI 5.0% to 30.6%, p = 0.009).
Simpson et al. 78 assessed GP direct access to CT for patients with chronic daily headache; 10.5% of scans indicated abnormalities. GPs reported that if direct-access CT had not been available then 44% would have referred to neurology and 38% to general medicine. Ten per cent of patients would not have been referred. Following scans, 86% did not require further specialist referral. Sixty-seven per cent of reports issued following scans were received in 1 week and 79% were received within 14 days. Without direct access, 90% of patients would have been referred at a cost estimate of £503,428. The cost of scans and outpatient appointments for patients in the study was estimated to be £602,026. A specialist headache clinic where, typically, 29% are referred for scans would cost £131,991, with a further review appointment costing total £688,708. Therefore, the cost saving of this intervention was estimated at £86,681.
Thomas et al. 79 evaluated the effectiveness and efficiency of a guideline-based open-access urological investigation service. General practices were randomised to receive either referral guidelines and access to the investigation service for LUTS or referral guidelines and access to the investigation service for microscopic haematuria. Participating GPs were offered a 2-hour educational meeting and were mailed a guideline package which included a guideline booklet, a quick reference flow chart and structured referral checklists. GPs’ compliance with referral guidelines increased (difference in means 0.5, 95% CI 0.2 to 0.8; p = 0.001). Approximately 50% of eligible patients were referred through the new system. The number and case mix of referrals were similar. The intervention reduced the waiting time from referral to initial outpatient appointment (ratio of means 0.7; 95% CI 0.5 to 0.9, patients with LUTS only) and increased the number of patients who had a management decision reached at initial appointment (OR 5.8, 95% CI 2.9 to 11.5; p < 0.001, both conditions). Patients were more likely to be discharged within 12 months (OR 1.7, 95% CI 0.9 to 3.3; p = 0.11). There were no significant changes detected in patient outcomes.
Thomas et al. 80 evaluated the referral rate of patients with chronic headache to open-access CT and the effect on neurology referral rates at three sites. Scanned patients had a lower referral rate to neurology immediately and in the year following the scan. The referral rate to open-access service was 1.2% of headache consultations by GPs. Open-access scans accounted for 4% of the annual number of scans. Of 215 scans, three lesions were identified which may have caused chronic headache (1.4% yield for significant findings and 10.2% for non-significant findings), and 88.4% of scans were normal. The service was used by 45% of GPs from 82% of practices. At 1-year follow-up, 14% (30) were subsequently referred to neurology clinic because of headaches; of these, 40% were referred at the same time as the CT scan request and 60% were referred after their brain-scan CT. Of these later referrals, 17 of the 30 had normal CT findings.
Wong et al. 81 evaluated a system of open-access endoscopy for dyspepsia. Family physicians were able to arrange upper endoscopy directly with the endoscopy unit in addition to conventional referrals. Extra sessions each week were allocated to open-access requests to ensure waiting time not affected. Waiting time for the intervention group was a mean of 6 weeks. For the control group the mean waiting time was 17.5 weeks to consultation and then another 4.5 weeks to procedure (a total of 22 weeks). During this waiting time only antacids were prescribed. There were abnormal findings in 19% of patients from the intervention group and 22% from consultant referral (difference not significant). Only two patients (0.2%) referred via open access were considered inappropriate. There were no significant differences in intervention versus control in peptic ulcer and cancer detection rate, but significantly more non-ulcer non-cancer abnormal findings in referrals via consultant (0.5% vs. 5%, p < 0.005). Of the intervention patients, 76% required no further consultation for at least 4 weeks after endoscopy, 12% attended a GP, and 12% were referred to specialist or were admitted to hospital.
Three further studies of direct-access screening interventions showed no clear effect on referral outcomes. Dhillon et al. 82 evaluated the impact of GP direct access to DXA scanning for patients at risk of osteoporosis; no specific guidelines were issued. They reported mostly clinical outcomes, but included some limited referral rate data. Before intervention, the range of number of referrals for scanning was 0.01% to 0.6% (median 0.2%). The number of referrals to a specialist clinic was 24 in the intervention group, compared with 12 in the control group. The study also concludes that direct access is more economically efficient, but it is not fully explained in the data how this is evaluated. Eley et al. 83 assessed the effectiveness of direct referral to audiology clinics on ENT appointments and appropriate GP use of the clinics. Direct-referral audiology clinics (DRACs) for the assessment and provision of hearing aids in those > 60 years were introduced as a means of decreasing outpatient waiting times and demand on ENT appointments. Of the 353 patients seen within the DRAC clinics, 320 were ultimately provided with a hearing aid. Fifty-five patients require review by an otolaryngologist, either by direct referral or via their GP. The greatest lack of adherence to the referral criteria for DRAC appointments related to appropriate treatment of wax within the community. Gough-Palmer et al. 84 looked retrospectively at GP access to MRI scans. There was no protocol, guidance or formal consultant or radiologist vetting in place. GP-requested scans, as a percentage of the workload of the department, were low (around 2.6%). While workload of the department increased over the study period, this percentage remained stable. Forty-eight per cent of scans requested were normal or minor degenerative changes; 26% demonstrated serious pathology warranting hospital referral.
Enhanced referral information
These interventions were dominated by studies conducted in dermatology where images were sent electronically or by post to the specialist to assist in determining whether or not a referral was necessary (Table 10).
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Knol 2006 90 | Electronic consultation with images | BA | Netherlands | Dermatology | 505 consultations | (2 years) |
| 29 GPs | ||||||
| Leggett 200485 | Referral letter with images | RCT | UK | Dermatology | 136 referrals | NR |
| 20 GPs | ||||||
| McKoy 200489 | Electronic consultation with images | BA | USA | Dermatology | 52 patients | NR |
| Aged 25–89 years | ||||||
| 46% female | ||||||
| Tadros 2009 96 | Electronic referral with images | RCT | UK | Oncology (skin cancer) | 300 referrals | NR |
| Whited 200287 | Electronic consultation with images | RCT | USA | Dermatology | NR | NR |
Leggett et al. 85 compared outcomes of referral for dermatology appointments between patients whose referral letters did and did not include instant photograph(s) taken by the GP. The GP took photograph(s) of the skin condition and sent them with a referral letter to the dermatologist in a numbered, sealed envelope. If a diagnosis was not possible, patients were given an appointment. If diagnosis was possible, a letter was sent to the GP with advice on management; some patients were also given an appointment for further management. Control group patients were given outpatient appointments in the usual way. For 63% of the study group (45/71), a diagnosis and a management plan were made without the patient requiring an appointment. This included 38% (27/71) of patients who, after diagnosis and initial management, needed an appointment, and 25% (18/71) who did not. The remainder of the study group (37%: 26/71) required a face-to-face consultation. This reduced the numbers requiring an outpatient appointment by 25% compared with the control group. The mean time for formulation of a management plan for patients without an appointment was 17 days (SD 11 days); waiting times for appointments in study and control groups were similar (mean 55 days; SD 40 days).
Whited et al. 87 compared usual care (text-based electronic consultation) with teledermatology (usual care plus digital images and standardised history). The standardised history included demographic information, patient-reported medical history, dermatology history, lesion location, size and duration of presence. The consultant answered by scheduling an appointment or by relaying a diagnosis and management plan back to the GP. Patients in the intervention arm reached time to initial defined intervention sooner than those in the usual-care arm (median 41 vs. 127 days, p < 0.001) and 18.5% of patients in the intervention arm avoided the need for a dermatology clinic visit, compared with no patients in the usual-care arm (p < 0.001). A further satisfaction survey as part of the RCT was also reported. 88
McKoy et al. 89 evaluated the accuracy, access time, cost and acceptance by patients and physicians of an asynchronous teledermatology referral intervention in primary care. GPs in a multispecialty group referred patients for teledermatology consultation. Same-day history and digital images taken by a nurse were electronically sent to a dermatologist who returned a diagnosis to the referring physician. History was adequate for diagnosis in 81% of cases and images were adequate in 75% of cases. Accuracy of the teledermatology diagnosis in cases with adequate images was 97%; accuracy for all cases was 92%. A dermatology visit was recommended in 26% of cases with adequate images and in 42% of all cases. Access time for a teledermatology opinion was 1.9 days, compared with 52 days for a regular dermatology appointment.
Knol et al. 90 aimed to reduce dermatology referrals using teledermatology. One overview and two detailed digital photographs of the skin problems were taken on a digital camera and attached to an e-mail message containing standard clinical information. The e-mail was sent to a dermatologist who replied after evaluation. Using teledermatology, 163 patients were not referred, a reduction of 163 out of 306 or 53%. There was no significant difference between dermatologists for secondary referral (χ2 = 1.6, p = 0.45), and patient sex did not affect secondary referral (χ2 = 0.8, p = 0.36). This study was at higher risk of bias.
Hockey et al. 91 examined the feasibility of a low-cost store-and-forward teledermatology service for GPs in regional Queensland. GPs were required to decide whether to refer for electronic consultation with the hospital or whether to refer to outpatients as usual. Electronic communication with the hospital was through a secure web-based system. Over 6 months, 63 referrals were processed by the teledermatology service. In the majority of cases, the referring doctors were able to treat the condition after receipt of e-mail advice from the dermatologist. In 10 cases (16%) additional images or biopsy results were requested because image quality was inadequate. The average time between a referral being received and clinical advice being provided was 46 hours. This study was at higher risk of bias.
Specialist consultation prior to referral
Specialist consultation prior to referral was the basis of six interventions (Table 11). 86,89,92–95 The interventions varied from a shared care programme with an oncologist, a system to contact a spine orthopaedist for red flag symptoms, to a virtual outreach intervention to share medical records between GP and specialist. All interventions showed a positive effect on at least one referral-related outcome, although results were sometimes mixed. The evidence for these interventions was rated as stronger.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Eminovic 200986 | Teledermatology to confer with specialist | cRCT | the Netherlands | Dermatology | 85 GPs from 35 practices | (1 month) |
| Five dermatologists | ||||||
| Harrington 200193 | Flow chart/algorithm for care | Case series | USA | Orthopaedics (low-back pain) | 581 patients | 3 years |
| Jaatinen 200295 | Consideration of teleconsultation | RCT | Finland | Specialists | 78 patients | 5 months |
| McKoy 200489 | Electronic consultation with images | BA | USA | Dermatology | 52 patients | NR |
| Aged 25–89 years | ||||||
| 46% female | ||||||
| Nielsen 200392 | Knowledge transfer GP/oncologist | RCT | Denmark | Oncology | 248 referrals | (3 and 6 months) |
| Wallace 200494 | Virtual outreach between GP and specialist | RCT | UK | Specialists | 134 GPs from 29 practices and 20 consultant specialists | (6 months) |
Eminovic et al. 86 determined whether or not teledermatological consultations can reduce referrals to a dermatologist by GPs. The GPs randomised to the intervention used a teledermatological consultation system to confer with a dermatologist, whereas those in the control group referred their patients according to usual practice. A training programme for the intervention GPs included instructions on taking digital images, downloading images to the computer, managing files and using the website. Dermatologists were taught how to use the website and complete the study forms. All patients, regardless of their condition, were seen in the office by a dermatologist after approximately 1 month. The five dermatologists considered a consultation preventable for 39.0% of patients who received teledermatological consultation and 18.3% of 169 control patients, a difference of 20.7% (95% CI 8.5% to 32.9%). At the 1-month dermatologist visit, 20.0% of patients who received teledermatological consultation had recovered, compared with 4.1% of control patients. No significant differences in patient satisfaction were found between groups.
Nielsen et al. 92 conducted an intervention to determine the effect of a shared care programme on the attitudes of newly referred cancer patients towards the health-care system and their health-related quality of life and performance status, and to assess patients’ reports on contacts with their GP. The shared care programme included transfer of knowledge from the oncologist to the GP, improved communication between the parties and active patient involvement. The shared care programme had a positive effect on patient evaluation of co-operation between the primary and secondary health-care sectors. The effect was particularly significant in men and in younger patients (18–49 years) who felt that they received more care from the GP and were left less in limbo. Younger patients in the intervention group rated the GP’s knowledge of disease and treatment significantly higher than younger patients in the control group. The number of contacts with the GP was significantly higher in the intervention group. The quality of life questionnaire and performance status showed no significant differences between the two groups.
Harrington et al. 93 developed an algorithm for referral to a spine orthopaedists which included a flow chart for care and a system for separating urgent cases from others. GPs were encouraged to contact the surgeon or physician manager for advice on patients with red flag symptoms. This resulted in a receptionist taking information which was verified by a nurse co-ordinator. The physician manager then reviewed the information to determine a care plan, which was instigated by the nurse co-ordinator. Following introduction of the guidelines little change was documented from traditional referral patterns (no other information provided on this). Three years later, in response to long waiting lists, the referral management programme was put in place, resulting in a shift of care from spine orthopaedists to primary physicians. Before implementation, 28% of patient visits for low-back pain were referred to a specialist and 72% were treated in primary care. During the transition year, 13% of patient visits were referred to a specialist and 87% were treated in primary care. In the year after implementation, 17% were referred to a specialist care and 83% treated in primary care.
Wallace et al. 94 considered whether or not virtual outreach would reduce offers of hospital follow-up appointments and reduce numbers of medical interventions and investigations, reduce numbers of contacts with the health-care system, have a positive impact on patient satisfaction and enablement, and lead to improvements in patient health status. Joint teleconsultation between GPs, specialists and patients prior to referral was compared with standard outpatient referral. Fifty-two per cent of patients in the virtual outreach group were offered a follow-up appointment, compared with 41% in the standard outpatient group. The overall proportion of patients receiving an offer of follow-up was 46% in the virtual outreach group and 42% in the standard outpatient group (OR 1.19, 95% CI 0.99 to 1.44), but significant heterogeneity remained for both site and specialty (p = 0.001 and p < 0.001, respectively). Fewer tests and investigations were ordered in the virtual outreach group, by an average of 0.79 per patient. In the 6-month period following the index consultation, there were no significant differences overall in number of contacts with general practice, outpatient visits, accident and emergency contacts, inpatient stays, day surgery and inpatient procedures or prescriptions between the randomised groups.
Jaatinen et al. 95 considered teleconsultation as a replacement for referral to an outpatient clinic. GPs had to decide whether to refer for electronic consultation with the hospital or whether to refer to outpatients as usual. Electronic communication with the hospital was through a secure web-based system. All patients treated by teleconsultation said that they wanted the same procedure in the future and 63% of the control group said that they would prefer a teleconsultation next time (p = 0.02), although they were nearly as satisfied as those who had received a teleconference (p = 0.37). The doctors quickly learned to exploit the telecommunication model. The responsibility for treatment was maintained, with the primary-care centre in 52% of cases using teleconsultation without any hospital visit required. The GPs and doctors agreed on follow-up treatment.
Tadros et al. 96 compared referral of suspect skin cancers as well as non-malignant symptomatic skin lesions using high-quality digital images transferred via a secure electronic referral system versus conventional pathways. A comparison of the diagnoses made from digital images with the diagnoses confirmed on pathology reports for lesions excised is described using a random selection of patients’ images and referrals. The study concludes that digital image referral for skin malignancy and other cutaneous lesions reduced the interval between referral and diagnosis by 81% and referral to commencement of treatment in suspect lesions by 30%. Diagnostic accuracy in a random sample of 30 patients was comparable with that reported for patients seen in face-to-face consultations. High levels of GP and patient satisfaction were recorded. This study was at higher risk of bias.
Electronic referral systems
Electronic referral systems were reported in 10 studies (Table 12). 97–106 Although each system differed, and included referral to different specialties, they all consisted of referral via an online system as opposed to via letter or e-mail. In one case a clinical reviewer assessed the referral for appropriateness,100 and in a second study a referral pro forma was included to try to guide appropriate referral, but in all other studies all referrals were accepted. Two studies103,104 also included aspects of specialist consultation but this was not the main focus of the intervention. Nine of the interventions reported a positive effect97–105 and all studies were considered to be at lower risk of bias. The evidence was rated as stronger.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Chen 2010100 | E-referral consultation requests compared with paper | BA | USA | Specialists | One hospital: 500,000 outpatients annually | 6 months |
| GPs in five hospitals: 11 community GPs and 10 independent GPs | ||||||
| Dennison 200699 | Electronic surgical referral | Cross-sectional | UK | Colorectal and gastroenterology clinics | 243 referrals | NR |
| 22 GPs in four practices | ||||||
| 54 patients electronically referred, 189 referred on paper | ||||||
| Gandhi 2008108 | Electronic referral tool | RCT | NR | Specialists | 430 referrals | 2 years |
| Jiwa 2012105 | Referral Writer software | BA | Australia | Six specialties | NR | (4 months) |
| Kennedy 2012106 | Electronic referral system | Audit | UK | Oncology (head and neck cancer) | 190 patients | NR |
| 55% female, aged 19–92 years, mean age 58 years | ||||||
| Kim 200998 | Impact of electronic referrals | Cross-sectional | USA | Clinical care | 298 GPs | NR |
| Kim-Hwang 2010102 | E-referral compared with paper | BA | USA | Specialists | 505 specialists | 2 years |
| Nicholson 200697 | Design and delivery of electronic referral system | Audit | Australia | Oncology | NR | 1.5 months |
| Patterson 2004104 | Structured form for neurology referrals | Cohort | UK | Neurology | 76 referrals | 14 months (6 months) |
| 27 male, 48 female | ||||||
| Mean age 44 years, range 16–80 years | ||||||
| Stoves 2010103 | Electronic sharing of health records | BA | UK | Nephrology | 17 practices | NR |
Nicholson et al. 97 completed an evaluation of an online referral and booking system for oncology referrals which included the design, development and deployment of the software in a new approach to information management (similar to choose-and-book system) for suspected cancer referrals. GP satisfaction with the new system was high. Hospital specialists were supportive; however, they noticed little difference in the processes from their perspective. All participants agreed that the system had meant that referrals were being efficiently actioned and that it made the process easy for patients. Patients perceived no major disadvantage.
Kim et al. 98 evaluated GPs to assess the impact of electronic referrals on workflow and clinical care. They distributed an 18-item, web-based questionnaire to 368 GPs who had the option of referring to San Francisco General Hospital. They asked participants to rate the time spent submitting a referral, guidance of work-up, wait times and change in overall clinical care compared with prior referral methods using five-point Likert scales. Over half (55.4%) worked at hospital-based clinics, 27.9% worked at county-funded community clinics and 17.1% worked at non-county-funded community clinics. Most (71.9%) reported that electronic referrals had improved overall clinical care. Providers from non-county-funded clinics (OR 0.40, 95% CI 0.14 to 0.79) and those who spent more than 6 minutes submitting an electronic referral (OR 0.33, 95% CI 0.18 to 0.61) were significantly less likely than other participants to report that electronic referrals had improved clinical care.
Dennison et al. 99 implemented an electronic surgical referral pro forma system, including patient details, symptoms, urgent/routine, provisional diagnosis and a free-text box, for referral to colorectal and gastroenterology clinics. Patients were 21% less likely to change their appointment when referred electronically. Time from referral to appointment was 8 weeks for the electronic system and 10 weeks for the paper system. Time from referral to booking was 0 days for the electronic system, compared with 7 days for the paper system (significantly different; data not given). There was an 8.5% rate of non-attendance in the electronic system, compared with 22.5% in the paper system (significantly different; data not given).
Chen et al. 100 evaluated a new consultation request process, called e-Referral, which was integrated into a hospital’s electronic health record. Clinician reviewers screen requests to evaluate urgency, choice of specialties, whether or not sufficient workup information is provided, and whether a specialist needs to see the patient or can guide the primary care clinician through the e-Referral system. Waiting times for non-urgent visits declined in seven of eight medical specialty clinics by up to 90% during the first 6 months of use. The percentage of referrals deemed inappropriate by medical and surgical specialists was cut by more than half (no data given). For clinics that had been plagued by long waiting times, implementation of e-Referral resulted in dramatic improvements. For example, in rheumatology, the median waiting time for a non-urgent appointment initially dropped from 126 days to 29 days. The majority of primary care clinicians reported that e-Referral improved patient care, but those with poorer access to the electronic health record found it more time-consuming than the previous paper-based system.
Gandhi et al. 108 reported on implementation of an electronic referral tool to analyse its impact on communication between primary care and specialists. They studied one practice site that implemented the referral tool and one that did not, and surveyed affiliated specialists, GPs and patients about referral communication. Specialists more often received information before the referral visit from intervention GPs versus non-intervention GPs (62% vs. 12%, p < 0.001), a finding that persisted after adjustment (RR = 3.3, p = 0.008). Intervention GPs more often received communication from specialists (69% vs. 50%, p = 0.08). Patients of intervention GPs were more likely than patients of control GPs to report that specialists had received information before their visit (70% vs. 43%, p = 0.007).
Kim-Hwang et al. 102 aimed to determine the impact of ‘e-Referral’, compared with paper-based referral, on specialty referral rates. The study was based on a visit-based questionnaire appended to new patient charts at randomly selected specialist clinic sessions before and after the implementation of e-Referrals (using a web-based system). A specialist reviewer (physician or nurse) reviewed the referrals and determined whether or not it was appropriate to schedule an appointment. It was difficult to identify the reason for referral in 19.8% of medical and 38.0% of surgical visits using paper-based methods versus 11.0% and 9.5% of those using e-Referral (p = 0.03 and p < 0.001). Of those using e-Referral, 6.4% and 9.8% of medical/surgical referrals using paper methods versus 2.6% and 2.1% were deemed not completely appropriate (p = 0.21 and p = 0.03). Follow-up was requested for 82.4% and 76.2% of medical and surgical patients with paper referrals versus 90.1% and 58.1% of e-Referrals (p = 0.06 and p = 0.01). Follow-up was considered avoidable for 32.4% and 44.7% of medical/surgical follow-ups with paper-based methods vs. 27.5% and 13.5% with e-Referral (p = 0.41 and p < 0.001).
Stoves et al. 103 evaluated an intervention where the electronic sharing of primary care electronic health records with the nephrology service was introduced to intervention practices. Participating GPs attended education workshops and received paper and e-guidance about the new service. The service allowed GPs to send electronic referrals and share patient electronic health records with a renal specialist after first obtaining verbal patient consent. GPs use criteria agreed in local guidelines to ‘request advice’ or ‘question the need’ for hospital clinic review. There was a significant reduction in paper referrals from intervention practices. The mean [standard error (SE)] interval between the GP sending an e-consultation referral and the renal specialist submitting an electronic response was 7 (0.8) days. This contrasted with a mean wait of 55.1 (1.6) days between the GP sending a paper referral and the patient attending a hospital clinic. When GPs were requesting clinic review by letter, only 56% of referrals were appropriate according to local criteria (71% and 52% for intervention and non-intervention practices, respectively), but 98% of these were accepted for hospital clinic review. By contrast, 90% of e-consultations that questioned the need for clinic review were appropriate, and clinic assessment was recommended in only 27% of cases.
Patterson et al. 104 conducted an intervention to determine if an e-mail triage system between GPs and a neurologist for new outpatient referrals was feasible, acceptable, efficient, safe and effective. A structured form was devised for GPs to refer patients. This set out the required history and examination and was either sent as an e-mail attachment or incorporated into the body text of the e-mail. When the neurologists received the e-mail referral they decided whether or not advice alone was appropriate, whether or not investigations were needed, or whether or not a clinic visit was necessary. When the investigation results were available, either a clinic appointment was made or further advice was given. Forty-three per cent of participants required a clinic appointment, 45% were managed by e-mail advice alone and 12% were managed by e-mail plus investigations. Forty-four per cent of the neurologist’s time was saved, compared with conventional consultation; total time spent was, therefore, 1270 minutes (mean of 16.7 minutes per patient). No deaths or significant changes in diagnosis were recorded during the 6-month follow-up period.
Jiwa et al. 105 explored if increasing the amount of relevant information in referral letters between GPs and hospital specialists helps in the scheduling of appointments for patients. They used Referral Writer software, a software system to assist referral writing, consisting of a pro forma that selects relevant information from the electronic patient record and requests the doctor to choose one of six specialties for referral: urology, breast, gynaecology, upper gastrointestinal (GI), colorectal and respiratory. The doctors were finally prompted to enter details about the patient’s condition. Each GP referred 5.6 patients on average (range 1–14) before the intervention and 4.8 patients (0–14) after it. The amount of relevant information in the referrals improved substantially (mean difference 37%, 95% CI 30% to 43%; p < 0.001). For 91% of referrals after the intervention both specialists in each specialty were confident or very confident that they had enough information to decide when the patient should come to their clinic; this was an increase from 50% before the intervention (p = 0.001). There was no association between the amount of relevant information and the final diagnosis.
One further study of an electronic referral system showed no effect on referral. Kennedy et al. 106 evaluated a fast-track electronic referral system (including referral guidelines) for suspected head and neck cancer. Fifty-two per cent of urgent referrals required no further investigation following assessment and were discharged. Head and neck cancer detection rate (percentage of patients with confirmed diagnosis from total number of referrals) was 8%. Overall cancer detection rate was 15%. During the time period of system operation (1 year), only 14% of the total number of head and neck cancers diagnosed were referred via the electronic system. All others had been referred by non-urgent referral channels (by the same group of practitioners). Twenty-seven different GP practices used the system to refer; however, one city-centre practice accounted for 17% of referrals. Therefore, in this case, 86% of patients diagnosed with cancer bypassed the system.
Decision support tools
The 10 decision support tools all aimed to assist GPs in making referrals and included real-time computer or internet-based systems, as well as a librarian consultation service, the effect of patient-specific ratings versus conventional guidelines, and automatic reporting of estimated glomerular filtration rate (eGFR) to inform referral decision (Table 13). Six studies showed a positive effect,107,109–113 but four reported a negative effect or no effect. 114–117 All studies were considered to be at lower risk of bias. The strength of the evidence was graded as inconsistent.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Akbari 2012110 | Automatic reporting eGFR | BA | Canada | Nephrology | 2672 patients | 2 years (1 year) |
| 12.5% aged 65+ years | ||||||
| Emery 2007111 | Computer decision support system | cRCT | Australia | Regional cancer genetics service | 45 GPs practice teams | (12 months) |
| Greiver 2005114 | PDA software to diagnose angina | cRCT | UK | Cardiology (angina) | 18 GPs | (7 months) |
| 65 patients; patients aged 30–75 years | ||||||
| Junghams 2007109 | Patient-specific ratings | RCT | UK | Cardiology (angina) | 145 GPs | NR |
| Knab 2001112 | Computer-based decision support | BA | USA | Chronic pain referral | 100 patients | 1 year |
| Magill 2009115 | Computer-based referral enhancing | BA | USA | Colonoscopy | NR; patients aged 50+ years | NR |
| Mariotti 2008113 | Prioritisation by GP and specialist | Audit | Italy | Gastroscopy colonoscopy | 438 outpatients | 7 months |
| McGowan 2008107 | Librarian consultation | RCT | Canada | Specialists | 82 GPs; five nurses; one specialist | (24 hours) |
| Slade 2008117 | Referral threshold assessment | cRCT | UK | Mental health | 281 GPs | NR |
| 1061 referrals | ||||||
| Tierney 2003116 | Computer-based care suggestions | RCT | UK | Cardiology | 706 patients | 1 year |
McGowan et al. 107 evaluated whether or not information provided by librarians to answer clinical questions positively impacted time, decision-making, cost savings and satisfaction. The ‘just-in-time information’ librarian consultation service was designed to provide a rapid response to clinical questions during patient visit hours. The questions were submitted by the participants and each question was randomly assigned to the intervention (librarian information) or control (no librarian information) group. If the question was randomised to the control group, participants received a message within 1 minute that their question would not be answered. Each participant had clinical questions randomly allocated to both intervention (librarian information) and control (no librarian information) groups. Participants were trained to send clinical questions via a hand-held device. The average time for ‘just-in-time information’ librarians to respond to all questions was 13.68 minutes per question (95% CI 13.38 to 13.98 minutes). The average time for participants to respond their control questions was 20.29 minutes per question (95% CI 18.72 to 21.86 minutes). Using an impact assessment scale rating cognitive impact, participants rated 62.9% of information provided to intervention group questions as having a highly positive cognitive impact. They rated 14.8% of their own answers to control question as having a highly positive cognitive impact, 44.9% as having a negative cognitive impact and 24.8% as having no cognitive impact at all.
Junghams et al. 109 assessed the effect of patient-specific ratings versus conventional guidelines on appropriate investigation of angina. Intervention physicians received patient-specific ratings (online prompt stating whether the specific vignette was considered appropriate or inappropriate for investigation, with access to detailed information on how the ratings were derived) and control physicians received conventional guidelines from the American Heart Association and the European Society of Cardiology. Physicians made recommendations on 12 web-based patient vignettes before and on 12 vignettes after these interventions. Decisions for exercise electrocardiography were more appropriate with patient-specific ratings [819/1491 (55%)], compared with conventional guidelines [648/1488 (44%)] (OR 1.57, 95% CI 1.36 to 1.82). The effect was stronger for angiography [1274/1595 (80%) with patient-specific ratings compared with 1009/1576 (64%) with conventional guidelines (OR 2.24, 95% CI 1.90 to 2.62)]. Within-arm comparisons confirmed that conventional guidelines had no effect but that patient-specific ratings significantly changed physicians’ decisions towards appropriate recommendations for exercise electrocardiography (55% vs. 42%; OR 2.62, 95% CI 2.14 to 3.22) and for angiography (80% vs. 65%; OR 2.10, 95% CI 1.79 to 2.47).
Akbari et al. 110 assessed whether or not automatic reporting of the eGFR, along with an ad hoc educational component for primary care physicians, would increase the number of appropriate referrals to nephrology. Concurrent with the introduction of automatic reporting of the eGFR, the nephrology service mailed an algorithm to all primary care physicians in the Champlain Local Health Integration Network. This algorithm explained the interpretation of the eGFR and appropriate parameters for referrals to nephrology, based on the value. In addition, ad hoc educational sessions (lectures and workshops) were provided to the primary care physicians to discuss interpretation of the eGFR results and parameters for referral to nephrology. In the year after automatic reporting began, the number of referrals from primary care physicians increased by 80.6% (95% CI 74.8% to 86.9%). The number of appropriate referrals increased by 43.2% (95% CI 38.0% to 48.2%). However, there was no significant change in the proportion of appropriate referrals between the two periods (−2.8%, 95% CI −26.4% to 43.4%). In the year after automatic reporting of the eGFR was introduced, the total number of referrals increased significantly among patients ≥ 80 years (percentage-point change 8.0, p < 0.001) and among women (percentage-point change 12.6, p < 0.001).
Emery et al. 111 evaluated the effect of an assessment strategy using the computer decision support system [the Genetic Risk Assessment on the Internet with Decision Support (GRAIDS) software] on the management of familial cancer risk in British general practice in comparison with best current practice. Training in the new assessment strategy and access to the GRAIDS software (GRAIDS arm) was conducted and compared with an educational session and guidelines about managing familial breast and colorectal cancer risk. All GPs and practice nurses attended a 45-minute educational session on cancer genetics, delivered at their general practice. They were also introduced to the principles of the GRAIDS intervention. There were more referrals to the Regional Genetics Clinic from GRAIDS than to control practices (mean 6.2 and 3.2 referrals per 10,000 registered patients per year; mean difference 3.0 referrals; 95% CI 1.2 to 4.8; p = 0.001). Referrals from GRAIDS practices were more likely to be consistent with referral guidelines (OR 5.2, 95% CI 1.7 to 15.8; p = 0.006). Patients referred from GRAIDS practices had lower cancer worry scores at the point of referral (mean difference 1.44, 95% CI 0.23 to 2.64; p = 0.02).
Knab et al. 112 determined whether or not computer-based decision support (CBDS) could enhance the ability of GPs to manage chronic pain. Structured summaries were generated for 50 chronic pain patients referred by GPs to a pain clinic. A pain specialist used a decision support system to determine appropriate pain therapy and sent letters to the referring physicians outlining these recommendations. Separately, five GPs used a CBDS system to ‘treat’ the 50 cases. One year later, the hospital database provided information on how the actual patients’ pain was managed and the number of patients rereferred by their GP to the pain clinic. On the basis of CBDS recommendations, the GP subjects ‘prescribed’ additional pain therapy in 213 of 250 evaluations (85%), with a medical appropriateness score of 5.5 ± 0.1. Only 25% of these chronic pain patients were subsequently rereferred to the pain clinic within 1 year. The use of a CBDS system may improve the ability of GPs to manage chronic pain and may also facilitate screening of consults to optimise specialist utilisation.
Mariotti et al. 113 evaluated a new method of prioritisation of patients suffering from significant GI disorders needing rapid access to diagnostic procedures. GPs used a ranking of waiting times for different levels of clinical priority called homogenous waiting groups. Specialists assigned a priority level for each patient as well as evaluating the appropriateness of the referral and the presence of significant endoscopic disorders. Agreement between GP and specialist was evaluated. Most referrals (74.4%) were deemed low priority by GPs, with no maximum waiting time assigned. The level of agreement between GPs and specialists with regard to patient priorities was poor to moderate; for gastroscopy the kappa was 0.31 and for colonoscopy it was 0.44. There was an association between the proportion of significant disorders identified with endoscopy and the priority assigned to the referral (χ2 = 18.9; 1 df; p < 0.001). The overall proportion of referrals deemed inappropriate by specialists was 22.1%.
Four further decision support studies showed no positive association with referral outcomes. Greiver et al. 114 determined the effectiveness of a personal digital assistant (PDA) software application to help family physicians to diagnose angina among patients with chest pain. Intervention GPs received a Palm PDA (which included the angina diagnosis software). They prospectively recorded the process of care for patients presenting with suspected angina over seven months. Fourteen of the 28 patients in the control arm (50%) and 30 of the 37 patients in the PDA arm (81%) were referred for cardiac stress tests (p = 0.007), an absolute difference of 31% (95% CI 8% to 58%). There was a trend towards more appropriate use of stress testing (48.6% with the PDA vs. 28.6% control), an increase of 20% (95% CI –11.54% to 51.4%; p = 0.284). There was also a trend towards more appropriate use of nuclear cardiology following cardiac stress testing (63.0% vs. 45.5%), an absolute increase of 17.5% (95% CI –13.9% to 48.9%; p = 0.400). Referrals to cardiologists did not increase (38.2% with the PDA vs. 40.9%, p = 0.869). A referral was more likely to have been made if the final diagnosis was angina (likelihood ratio for referral 15.455, 95% CI 2.124 to 112.431), so GPs appeared to refer appropriately.
Magill et al. 115 evaluated a computer-based system to enhance referral for colonoscopy. The intervention had three components: (1) a pop-up prompt for screening colonoscopy on electronic medical records (EMRs) was modified; (2) education sessions for primary care providers comprising epidemiology of colon cancer, strategies for early detection, how to use EMRs and optimal clinic workflow to facilitate screening were provided; and (3) medical assistants were asked to discuss screening with eligible patients before they were seen by a physician and to initiate preliminary orders for tests. There were also best practice alerts, computerised documentation of referral status and individual physician feedback, which were implemented later. Individual site providers experienced very different local conditions and changes during the course of the project, for example relocation, new services, personnel change, and introduction of revenue for screening site and physician from referrals. At baseline, monthly referral rates were 5–7%. The pop-up prompt and provider education introduced over a 2-month period showed little or no immediate correlation with referral. Initiation of medical assistant workflow change 2 months later was associated with an 11% increase in referral rate. Small increases were observed after best practice alerts and computerised documentation of referral status was implemented 2.5 years after the initial intervention (no details given of these intervention methods). At 4 years, referral rates remained above baseline.
Tierney et al. 116 assessed the effects of computer-based cardiac care suggestions. Evidence-based cardiac care suggestions, approved by a panel of local cardiologists and general internists, were displayed to GPs and pharmacists as they cared for enrolled patients. Evidence-based guidelines published by the Agency for Health Care Policy and Research and national professional organisations were used to develop the cardiac care rules. The cardiac care suggestions were printed at the end of the medication list on the encounter form and displayed as ‘suggested orders’ on GPs’ workstations. GPs could view the guidelines and references via the ‘help’ key. Subjects were followed for 1 year, during which they made 3419 primary care visits and were eligible for 2609 separate cardiac care suggestions. The intervention had no effect on physicians’ adherence to the care suggestions (23% for intervention patients vs. 22% for controls). There were no intervention–control differences in quality of life, medication compliance, health-care utilisation, costs or satisfaction with care.
Slade et al. 117 investigated whether or not introducing a standardised assessment of severity improved referral agreement. Prior to a mental health referral, GPs completed a threshold assessment grid, a one-page assessment of mental health severity, which was then attached to the referral form/letter. Implementation was low and the grid was used with only 25% of referrals. There were no significant differences between trial arms (p = 0.05) for any of the comparisons: appropriateness of referral was 64% versus 60% (intervention vs. control, p = 0.41 adjusted), rating of urgency was 81% intervention versus 76% control (p = 0.15), identification of appropriate professional was 89% intervention versus 87% control (p = 0.46), and time to discuss referral by mental health team was 2.08 versus 2.15 minutes (p = 0.37).
Waiting list interventions
We identified three interventions that consisted of waiting list review (reviewing the condition of patients awaiting a specialist appointment to see if that appointment was still appropriate and required) or watchful waiting (delaying referral to see how a condition developed) (Table 14). 118–120 Only one of these interventions was shown to be effective, although all were considered to be at lower risk of bias. 118 The strength of the evidence was graded as inconsistent.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| King 2001119 | Review of waiting list | BA | UK | Any specialty | 109 referrals | NR |
| Stainkey 2010118 | Review of waiting list | Audit | Australia | Five specialties | 872 patients | NR |
| van Bokhoven 2012120 | Watchful waiting | cRCT | the Netherlands | 498 patients, 63 GPs | NR | |
| Patient mean age 43 years, 28% male | ||||||
| GP mean age 45 years, 74% male |
The effective intervention118 evaluated a specialist appointment service for long-waiting patients. Letters were sent to patients who had been waiting for hospital appointments for 2 years or more (triaged by the hospital as non-urgent). Patients responded and, if they felt that the appointment was still needed, they were seen at specially arranged clinics. In the first wave 16 patients required procedures (of the 101 who had responded to the letter and been seen in a clinic). In the second wave 532 patients responded to the letter and were seen in a clinic. One hundred and seventy-seven patients had surgical procedures resulting from these appointments.
Two further waiting list interventions had no effect on referral: the first119 considered whether or not, in practices with high referral rate, an invitation to review referrals could identify patients on the waiting list who considered their referral unnecessary, leading to a negotiated cancelling of their appointment. Four to seven weeks after referral, selected patients were sent a questionnaire and an invitation to a review their appointment. Exclusion criteria were symptoms that raised the possibility of significant disease; patient’s mental state precluded consent or co-operation; the referring doctor preferred the patient not to participate; and such urgency that an outpatient appointment could be expected within 3 weeks. Of those patients who were contacted, 77 (72%) responded and, of those, 10 (13% of responders) indicated uncertainty that a referral was still needed. Eight of these attended for review, but in none of these cases was the appointment subsequently cancelled. Therefore, taking cancellation of a hospital appointment as an end point, the effect shown is 0 out of 435 referrals and 0 out of 109 in the intervention group (95% CI 0 to 3).
The second study120 evaluated the feasibility of watchful waiting compared with immediate blood test ordering in patients presenting with unexplained complaints that did not cause alarm for the GP, including fatigue, abdominal complaints, weight change, musculoskeletal complaints and itch. Group A took a watchful-waiting approach. Group B included watchful waiting plus a ‘quality improvement strategy’, which consisted of two small group meetings including an explanation of the diagnostic value of tests, a discussion of the difficulties in dealing with patients with unexplained complaints, and goal setting to change GPs’ behaviour. There was no statistically significant difference between the two intervention groups in terms of the number of patients for whom tests were ordered, or GP performance (performs adequate examination, explains findings to patient). First consultation GPs ordered a mean of seven tests in the control group and trained intervention group, and six tests in the untrained intervention group. Fifty-two of the 498 patients returned to the GP after 2 weeks for a further consultation.
System change interventions
We defined system changes as large changes impacting on all referrals made which involved the movement of staff or relocation clinics, the methods in which all referrals were triaged at hospital or financial arrangements for referrals.
System change interventions included the community provision of specialist services by GPs (n = 9), outreach or community provision by specialists (n = 10), return of inappropriate referrals (n = 2), the provision of additional primary care staff (n = 3), the addition or removal of gatekeeping systems (n = 4), changes to payment systems (n = 4), and referral management centre or other major triage systems (n = 6).
Community provision of specialist services by general practitioners
Community provision of specialist services by GPs was reported in nine studies (Table 15). 121–129 The services provided included dermatology services delivered by primary care, ambulatory electrocardiogram (ECG) monitoring in general practice, GP providing minor surgery, a GP headache service, GP with special interest (GPwSI) clinics in primary care, spirometry, and loop electrical excision procedure (LEEP) for cervical dysplasia. Seven studies showed a positive effect on referral outcomes, but two reported a negative effect or no effect. Eight studies were considered to be at lower risk of bias,121–128 with only one study at higher risk of bias (this study showed no effect on referral outcomes). 129 The strength of the evidence was graded as stronger.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Callaway 2000121 | LEEP training for GPs | Audit | USA | Gynaecology | 272 patients; female | (6 years) |
| Levell 2012 129 | Dermatology clinics | BA | UK | Dermatology | NR | 6 years |
| Ridsdale 2008124 | GPwSIs in headache | Audit | UK | Neurology | 117 patients | NR |
| Mean age 41.1 years, 57% female | ||||||
| Rosen 2006128 | GPwSIs | Cohort | UK | All specialties | Four sites | NR |
| Salisbury 2005125 | Primary dermatology service | RCT | UK | Dermatology | 30 practices | (9 months) |
| 556 patients | ||||||
| Sanderson 2002126 | Dermatology in primary care | RCT | UK | Dermatology | 556 patients | (9 months) |
| Sauro 2005127 | GP spirometry | nRCT | Italy | Respiratory (COPD) | 24 GPs | NR |
| 32,785 patients | ||||||
| Standing 2001122 | ECG monitoring by GPs | BA | UK | Cardiology | 73 patients | NR |
| 26 male, 47 female | ||||||
| 71% no cardiac history | ||||||
| Van Dijk 2011123 | Minor surgery by GPs | Audit | the Netherlands | Surgical specialties | 14,202 patients | NR |
| Mean age 39 years; 51% female |
Seven studies showed a positive association between the intervention and referral outcomes. 121–127
The first from the USA121 evaluated whether or not LEEP training for family physicians could impact on referral to gynaecology. Prior to training all patients were referred. After training, the LEEP for cervical dysplasia was carried out by family physicians in a cervical dysplasia clinic. During the study period, 283 women were seen in the clinic, and 26 individuals (9%) were referred by the GP to a consulting gynaecologist. Of the 9% referred to gynaecologist, all but one were subsequently treated with a laser or a combination of a laser and LEEP.
A UK study122 investigated whether or not ambulatory ECG monitoring in general practice could decrease unnecessary referrals and pick up unsuspected cardiac abnormalities. Patients were recruited to use a novel ambulatory ECG machine designed to detect arrhythmias in general practice. Patients were selected if they had signs and symptoms indicative of cardiac abnormalities including dizzy spells, fainting, palpitations or pounding chest, as well as considering their medical history and general profile. Patients made two GP visits. On the first they underwent a normal consultation and the GP recorded any diagnosis made, whether he or she would refer the patient and, if so, what test he or she would request. The ECG device was fitted and the patient was given a diary card and general advice about the equipment. The patient’s ECG signal was then analysed for 24 hours. The patient was instructed to return to the surgery the next day where the GP reviewed the report generated by the equipment and decided whether or not to refer the patient to the cardiology clinic. Following GP assessment prior to using the ECG machine, GPs were intending to refer 49 (68%) to cardiology outpatients for further tests. Of these, three cases were considered to need urgent appointments. The ECG data identified 22 patients with cardiac abnormalities. In seven patients no abnormality was detected, and three further cases gave non-diagnostic results (probably attributable to poor fitting). The number of patients the GPs decided to refer to cardiology outpatients reduced by 60%, from 49 to 19 patients. However, the number of patients identified as urgent increased from three to seven. Thirty-six (of 49) were unlikely to need cardiology referral.
The most recent study123 retrospectively examined associations between the number of minor GP surgical interventions undertaken and hospital referral rates. Electronic medical record data were examined for patients where benign neoplasm skin/naevus, sebaceous cyst or laceration/cut and/or minor surgery was performed by GPs. GP practices that performed more minor surgery had a lower referral rate for patients with a laceration/cut (–0.38, 95% CI –0.6 to –0.11) and for patients with a sebaceous cyst (–0.42, 95% CI –0.63 to –0.16) but not for those with benign neoplasm skin/naevus (–0.26, 95% CI –0.51 to 0.03). Minor surgery was more often performed in older patients. The presence of a primary care nurse only affected referral for benign neoplasm. There was a significant negative correlation between minor surgery intervention and referrals at a practice level (no data given). For laceration/cut and sebaceous cysts, GP practices that perform more minor surgery interventions refer fewer patients to a medical specialist. Performing five more minor surgery interventions per 100 care episodes would result in 4.3 fewer referrals for sebaceous cyst.
Another UK study124 evaluated the training of GPwSIs in headache and the setting up of a GPwSI clinic in general practice, compared with the existing neurology service. A questionnaire survey was conducted, measuring headache impact, satisfaction and cost estimates. There was no significant difference in headache impact between hospital (mean score 61.2, SD 10.4) and GPwSI clinic attendees (mean score 64.3, SD 9.3) after adjustment for age, sex and ethnicity (mean difference 2.7, 95% CI 1.6 to 7.0). Patients were significantly more satisfied with the GPwSI service, particularly that the service was effective in helping to relieve their symptoms (89% vs. 76%; OR 7.7, 95% CI 2.7 to 22.4). The cost per first appointment was estimated to be £136, with £68 for subsequent contacts. These are lower than costs for neurologist contacts.
A further study from the UK125 investigated the effectiveness, cost-effectiveness, accessibility and acceptability of a primary care dermatology service (PCDS) in comparison with a hospital outpatient clinic for dermatology. The PCDS was staffed by two GPwSIs and a specialist nurse, and provided from a suburban health centre. Patients were referred by their GPs to the outpatient dermatology department as usual. Those who appeared on the basis of their referral letter to be suitable for management in the PCDS were given an appointment there rather than at the outpatient department. There were no marked differences between the PCDS and hospital care in respect of clinical outcome (ratio of geometric means 0.99, 95% CI 0.85 to 1.15; p = 0.9, adjusting for baseline and stratification). The PCDS was more accessible [the difference between means on the access scale (scored out of 100) was 14, 95% CI 11 to 19; p < 0.001] and patients had reduced waiting times by a mean of 40 days (95% CI 35 to 46 days, p < 0.001). Fewer PCDS patients (6%) than hospital patients (11%) failed to attend their initial appointment, but overall did-not-attend rates for new and follow-up appointments were similar in both sites (PCDS 8%; hospital 11%). Of those patients seen initially at PCDS, 12% were referred to the hospital for one or more follow-up appointments.
The fourth UK study in this group126 assessed the effectiveness, accessibility and acceptability of a GPwSI service for skin problems compared with a hospital dermatology clinic. The GP clinic was staffed by two GPwSIs and a specialist nurse. A consultant dermatologist provided clinical support for two sessions per month. No noticeable differences were found between the groups in clinical outcome (median dermatology life quality index score of 1 both arms, ratio of geometric means 0.99, 95% CI 0.85 to 1.15). The GPwSI service was more accessible (difference between means on access scale 14, 95% CI 11 to 19) and patients waited a mean of 40 (95% CI 35 to 46) days less. Patients expressed slightly greater satisfaction with consultations with a GPwSI (difference in mean satisfaction score 4, 95% CI 1 to 7), and at baseline and after 9 months 61% said that they preferred care at the service.
An Italian study127 considered the effect of training GPs to perform spirometry on the management of COPD and asthma. There were three study groups (it is not clear if they were randomly allocated): group 1 GPs received a spirometer and practice training in its use, including information on guidelines (n = 11,050); group 2 received only guidelines (no spirometer or training) (n = 11,040); and group 3 was the control group (n = 1049). COPD was diagnosed in 5.8% of group 1, 1.5% of group 2 and 2.3% of group 3 (p < 0.001). Group 1 performed the test in 65.7% cases of COPD or asthma. Group 2 referred 7.8% of patients. The control group requested the test in 96.8% of the cases. There were significant differences between prescribing and/or utilising spirometry between all three groups (p < 0.001, data not given). Group 1 referred 7.5% to a specialist and diagnosis was confirmed in 91.8% of cases. Group 2 sent 7.8% to the specialist and diagnosis was confirmed in 75.8%. The control group referred 96.8% of patients, of whom 27.2% only had a confirmed diagnosis.
Two other UK studies showed no association with referral outcomes (one showed a strong negative effect on referral numbers). 128,129
The first128 compared referrals from GP practices that had access to GPwSI clinics and those that did not. They found that the association between the introduction of GPwSI clinics and hospital referral rates was variable and unpredictable. There were no significant changes in hospital referral rates following the introduction of GPwSI clinics in any of the sites studied. Overall referrals to hospital and GPwSI clinics combined increased in the three sites for which data were available. The likelihood of referral, calculated as the RR, adjusted for baseline and linear time trend, did not change after the launch of the GPwSI clinics in any of the sites studied. Small changes in risks of referral from studying control practices did not reach statistical significance. In one site, where all practices had access to GPwSI clinics, there was a significant (p = 0.08) 13% increase in overall referrals.
The second129 assessed the effect of introducing dermatology integrated intermediate care services on the numbers of dermatology referrals to secondary care. The dermatology intermediate care service was set up in 2005, providing services in two locations by two GPwSIs in dermatology. The GPwSIs were supported by experienced dermatology nurses and in total six clinics weekly were held, seeing approximately 30 new patients weekly. The numbers of dermatology new patients seen in secondary care, which had been stable for 5 years, showed an increase in 2007 followed by a substantial increase in 2008 and then 2009. The mean number of new patients seen in dermatology in 2004–6 was 6927 patients per year; in 2007, the mean number was 7844 patients; and the mean number of new patients seen between 2008 and 2010 was 11,535 patients per year. This was an increase of 67% in the number of new patients seen. Overall, over this period, there was a 23% increase in new dermatology patients seen in secondary-care dermatology in England. This study was at higher risk of bias.
The majority of interventions in which GPs were trained to provide specialist services in the community were effective at preventing referrals to secondary care. The two studies which did not show a positive effect consisted of GPwSIs rather than GPs who were trained to undertake a specific procedure. However, three other GPwSI interventions were shown to be effective. This could not be separated by condition, as of the two GPwSIs in dermatology studies, one was shown to be effective and one was not.
Community provision by specialists
Community provision by specialists was reported in 10 studies (Table 16). 130–139 The interventions consisted of specialist outreach clinics for diagnosis and treatment where appropriate, development of multidisciplinary mental health teams in primary care, acupuncture in primary care, manual therapy as part of a community-based musculoskeletal service, and an outreach surgical service offering open-access endoscopy to rural areas. Of the 10 studies, eight reported positive effects,130–137 with two reporting a negative effect or no effect138,139 (including one at higher risk of bias139). Nine of the studies were considered lower risk for bias,130–138 and the strength of the evidence was graded as stronger.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Campbell 2003131 | Specialist outreach clinic | cRCT | UK | Cancer genetics service | 203 GPs | NR |
| Women: family breast cancer | ||||||
| Felker 2004132 | Multidisciplinary mental health team | BA | USA | Mental health | 9656 patients | 2 years (1 year) |
| Mean age 53 years; 90% male | ||||||
| Gurden 2012133 | Community musculoskeletal service | BA | UK | Musculoskeletal | 696 patients | (≈8 weeks) |
| Back or neck pain | ||||||
| Mean age 52 years; 66% female | ||||||
| Hermush 2009137 | Caring for the elderly in the community | BA | Israel | Geriatrics | 512 patients | (3 years) |
| Mean age 79 years; 66% female | ||||||
| Hughes-Anderson 2002136 | Outreach endoscopy | BA | Australia | Endoscopy | 4400 patients | 5 years |
| Mean age 50.8 (15–94) years; 45% female | ||||||
| Johnson 2008 139 | Acupuncture in primary care | Audit | UK | Acupuncture | 109 practices | NR |
| Leiba 2002130 | Specialist outreach clinic | nRCT | Israel | All specialists | 136 patients; 20 GPs | NR |
| Pfeiffer 2011138 | Primary mental health services | Audit | USA | Mental health | 49,957 patients | NR |
| Mean age 55.7 years; 93% male | ||||||
| Schulpen 2003134 | Joint consultation sessions | nRCT | the Netherlands | Rheumatology | 17 GPs | 2 years |
| Mean age 48.5 years, 12% female | ||||||
| Vlek 2003135 | Joint consultation sessions | RCT | the Netherlands | Cardiology | 49 GPs | 1 year |
| 13 cardiologists | ||||||
| 306 patients; mean patients age 58 years |
A study from Israel130 evaluated a specialist outreach clinic established in a home-front military primary-care clinic. Patients were initially referred, but no further referral was required for continuity of specialist care. The same analysis was applied to a similar clinic employing only GPs, which refers to military specialist centres or hospital outpatient clinics. The incorporation of specialists did not result in a significant increase in the overall consumption of medical services (p < 0.05). It reduced the number of referrals out of the clinic to specialist centres from 1449 to 421 per month (p < 0.05). In the control clinic, referrals to distant specialist centres and outpatient clinics showed a slight and non-significant increase. Number of work-days lost was reduced from 2891 days per month to 1938 days per month (p < 0.001). The total cost of all medical interactions and referrals did not significantly increase after the introduction of the outreach specialist clinic (p < 0.05). Primary physicians graded their satisfaction with the new clinic as 4.5 (out of 5).
Campbell et al. 131 evaluated specialist outreach clinics in rural Scotland. Women with a family history of breast cancer were referred to a clinic held in a community setting near to the GP practice rather than receiving an appointment to see a consultant geneticist and breast surgeon at a regional centre. Referral rates rose from 2 years before the trial to during the trial (0.21 to 0.31), a 48% increase in referral rate (p < 0.001). Forty-three per cent of women asked to be referred and younger women were more likely to have taken the initiative to request referral (p = 0.001). There was a substantially greater increase in referral rates to community clinics than to the regional centre (64% increase vs. 38% increase), suggesting that providing a service in the community resulted in a change in GP referral behaviour. This was particularly apparent in practices in relatively deprived communities. There were higher referral rates from practices with more female partners before and during the trial (p < 0.005 and p < 0.02).
A study from the USA132 evaluated the effect of a multidisciplinary mental health care team in primary care. A multidisciplinary mental health team was created consisting of a psychologist, a psychology intern, psychiatry residents, clinical social workers and a chaplain. Before implementation 543 consultations occurred over the year. Of these, 543 (38%) were subsequently referred to specialty mental health care services. The following year, 560 consultations occurred, but only 81 (14%) were referred. The change in referral rate was significant (χ2 = 77.85, df = 1; p < 0.001).
The most recent study133 evaluated a community-based musculoskeletal service. Patients still having pain after 4–6 weeks of ‘usual GP care’ were offered a course of manual therapy and referred to a private provider of their choice for chiropractic, osteopathy and physiotherapy services. The percentage change in scores from baseline to discharge were as follows: Bournemouth Questionnaire, 64.6% patients categorised as improved; Bothersomeness scale, 69.9% patients categorised as improved; and Global Improvement Scale, 67.8% patients categorised as improved. Overall, 99.5% were satisfied or very satisfied with the treatment and only 3% were referred back to the GP with a recommendation for referral to secondary-care services (97% were given self-management advice and recommended for discharge).
Schulpen et al. from the Netherlands134 evaluated joint consultation sessions between GPs and a consultant held 6-weekly which consisted of three GPs and one visiting rheumatologist at the practice of a host GP. The GPs presented each patient, and the consultant examined the patient and formulated a diagnosis and therapy policy together with the GP. Prior to intervention there was an increasing referral rate to the hospital rheumatology department. By the end of the study period, the number of patients referred by each GP per year differed by –62% in the intervention group, compared with the controls. The average reduction in referral rate to rheumatology was –2.8 (SD 3.9) at the end of the second year of the intervention period, compared with the first year in the intervention group. In the control group the referral rate difference was zero (SD 2.1). The difference in referral rate between the intervention and control groups both before and after the intervention was significant (p = 0.024, Mann–Whitney U-test). Based on referral rates prior to the intervention, if all patients had been referred to a normal outpatient clinic they would have taken 307.8 hours of consultant time. If all referrals during the study period had been seen via the joint clinic system this would have used 166.7 hours. The authors argue that there was, therefore, a decrease of 46% in time spent by rheumatologist consultants.
A second study from the Netherlands in this group135 evaluated monthly joint consultation sessions between GPs and cardiology specialists held over 18 months in the surgery of the GP. Three to four patients could be examined and discussed at each session and there were an average of seven sessions per GP (range 2–13 sessions per GP). Fewer patients in the intervention group than the control group were referred to a cardiologist (33% vs. 52%, p = 0.001). The difference in referral rates showed an average decrease of referrals to cardiology of 6 per 1000 patients in the GPs from the intervention group. Further diagnostic procedures were required for 7% in the intervention group versus 16% in control group (p = 0.013).
Hughes-Anderson et al. 136 assessed whether or not an Australian outreach surgical service offering open-access endoscopy to rural areas was being overutilised. Indications for referral between the GPs and the visiting surgeons were reviewed in patient records and assessed for compliance with guidelines. Two groups of patients were defined: those referred directly for open-access endoscopy and those selected by the surgeons. A total of 772 endoscopies were performed and 75% were booked as open-access services. The referral rate for procedures was greater for GPs (583: 75%) than for the visiting surgeons (189: 25%). The overall compliance rate for approved indications using the guidelines for both groups was 92%. There was no significant difference in pathology found between groups. The difference between GPs and visiting surgeons for the number of appropriate indications for endoscopy was 3.2% (95% CI 1.8% to 8.2%; p = 0.348, not significant). The difference between GPs and visiting surgeons (appropriate indications) for colonoscopy was 6.8% (95% CI 1.8% to 15.4%; p = 0.148, not significant).
A study from Israel137 evaluated a new model used in caring for the elderly in the community. GPs referred difficult or complex cases to a geriatrician who carried out a clinic in the same primary-care location. Referrals to a geriatrician increased significantly from 133 at baseline to 207 2 years later (p = 0.01). The number of visits to GPs decreased in the 6 months following the consultation with the geriatrician (p < 0.01).
Two further studies did not show clearly positive association with referral outcomes: the first138 determined whether or not the implementation of primary care mental health services is associated with differences in specialty mental health clinic use. The US Veterans Health Administration is a primary care mental health service providing collocated collaborative mental health specialists and managers for screening and managing common mental health conditions (e.g. depression or alcohol misuse). Initiation of treatment at the specialty mental health clinic did not differ between primary-care services with mental health facilities and those without (5.6% vs. 5.8%). Attendance at a primary-care service for mental health was not a predictor of total number of specialist mental health clinic visits.
The second study, from the UK,139 evaluated the provision of acupuncture in primary care and whether or not it resulted in a reduced need for referral to secondary care. They found ‘no evidence from the data that provision of acupuncture is associated with lower referral rates’. The data presented outline mean referral rates for practices providing acupuncture clinics and ‘some’ versus ‘higher’ numbers of acupuncture appointments, but not for practices with no acupuncture, so this conclusion may need modification. They also report a wide variation between different PCTs, possibly associated with local differences in referral patterns and sociodemographic characteristics. This study was at higher risk of bias.
Return of inappropriate referrals
Interventions consisting of the return of inappropriate referrals were reported in two UK studies (Table 17). 18,140 The interventions consisted of a restricted-referral guideline issued to GPs for dermatology, including a list of conditions for which the dermatology service would no longer see patients, and a clinic returning patients referred for erectile dysfunction to the referrer (either in writing or by telephone). Both studies showed positive effects, with one at lower risk18 and the other at higher risk of bias. 140 The strength of the evidence was graded as weaker.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Tan 2007 140 | Refuse referral for certain conditions | Audit | UK | Dermatology | NR | NR |
| Wylie 200118 | Return of referrals for erectile dysfunction | Audit | UK | Erectile dysfunction | 796 referrals | NR |
The first study18 compared the prescribing pattern and attitude of GPs in response to a clinic returning a patient referred for erectile dysfunction with the referrer by two different methods. Referrals on a waiting list for an assessment of erectile dysfunction were reviewed and a subgroup of patients was identified who had criteria enabling them to be eligible for a prescription under the NHS. The GP was informed either in writing or by telephone that the clinic had written to the patient, suggesting that he make direct contact with his GP. The long waiting time for assessment had led to 35% of patients having already tried drug therapy, and by the time the questionnaire was completed, 57% of patients had tried drug therapy. Ten times as many referrers indicated that they were happy to initiate a prescription for drug therapy than not to do so, for those men eligible for an NHS prescription. More GPs who had received a letter returned the completed questionnaire (80%) than those who had received a courtesy telephone call (64%). There were no differences between the groups of GPs in their attitude to contact with their patient and no difference in prescribing pattern.
The second study140 evaluated the impact of a restricted-referral guidance issued to GPs for dermatology referrals inspired by the Oregon Health Plan, a rationing policy. A list of conditions that the service would no longer treat or treat only in exceptional circumstances was circulated to all GPs. Referrals for these conditions were returned. Following the introduction of the new policy, a reduction in the rate of referrals occurred. For a further 3–4 years post intervention the volume of new referrals remained static. The data are presented only in the form of a chart; there was a peak of 800 new referrals per year before the intervention, falling to around 600 referrals per year post intervention. This study was at higher risk of bias.
Additional primary care staff
Three studies reported on the provision of additional primary care staff: primary care nurses, and counsellors (Table 18). 141–143 However, all showed no effect (or very limited effect) on referral outcomes, with one graded as being at higher risk of bias143 and the other two being graded as lower risk. 141,142 The strength of evidence was graded as stronger, but it is important to note that the evidence was in a negative direction here, that is, more staff adversely impacted on demand management outcomes.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Simpson 2003 143 | Counsellors in primary care | BA | UK | Mental health | 85 practices | 8 years |
| Van Dijk 2010141 | Primary care nurses | Audit | the Netherlands | Diabetes referral to internists, ophthalmologists, cardiologists or mental health care | 54 practices | NR |
| 751 patients; 50% male; mean age 61 years | ||||||
| White 2000142 | Counsellors in primary care | Cross-sectional | UK | Mental health | 180 referrals | NA |
A study from the Netherlands141 assessed whether or not the introduction of primary care nurses affected referral rate for diabetes-related hospital treatment (referrals to internists, ophthalmologists, cardiologists or mental health care). Referral rate to internists for newly diagnosed patients decreased for practices both with and without a practice nurse between the two time points (7.3% vs. 3.3%). The trend in referral patterns to internists for known diabetic patients was lower in general practices with primary care nurses than those without (OR 0.59, 95% CI 0.31 to 1.11; p < 0.1). The number of diabetes-related contacts did not differ between practices with and without primary care nurses.
The first of two UK studies142 evaluated the impact of counsellors in primary care on referrals to mental health services. A counsellor was present at 20.3% of practices. A random sample of 180 referrals to community mental health teams was reviewed: 76 (42.2%) from practices that employed a counsellor and 104 (57.8%) from practices that did not. There was a significantly higher referral rate from practices that employed a counsellor (p = 0.003). However, there was no evidence of a difference in rates of appropriateness of referrals between practices that employed a counsellor and those that did not.
The second UK paper143 also investigated the effect of employing counsellors in general practice on referral rates to mental health services. The practice-employed counsellors were well established and practices were allocated 6–12 hours per week. The findings suggest that the cost of the counsellor could be offset elsewhere. The provision of counselling had no statistically significant effect on referrals or the volume and cost of prescribing.
Gatekeeping systems
Interventions that involved the addition or the removal of gatekeeping systems (primary-care control of hospital referral) were reported in four studies (Table 19). 144–147 In two studies (by the same author),145,146 multispecialty primary-care gatekeeping was removed so that patients were able to schedule an appointment directly with any specialist. The other two studies144,147 compared open-access with physician-approved referral. Overall, the studies showed no significant effect (or only a borderline significant effect) on referrals irrespective of whether gatekeeping was added or removed in the intervention. One study was at higher risk of bias,146 with the other three being rated as lower risk of bias. 144,145,147 The strength of the evidence was rated as stronger; however, it is important to note the bidirection of evidence and that all studies were from the USA.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Ferris 2001145 | Removal of gatekeeping | BA | USA | All specialists | 59,997 patients | 6 months |
| Mean age 41.7 years, 53% female | ||||||
| Ferris 2002 146 | Removal of gatekeeping | BA | USA | All specialists | 59,952 patients | NR |
| Joyce 2000147 | Open access vs. gatekeeping | Audit | USA | All specialists | 53,011 patients, working age | 2 years |
| Schillinger 2000144 | Open access vs. physician approved | RCT | USA | All specialists | 2293 patients | 1 year |
The first paper144 evaluated the effect of open-access versus physician approval of referral to specialist services (and to emergency departments). Intervention patients required prior approval from their primary-care physician in order to receive specialty care at the local hospital. A computer programme blocked the scheduling of unapproved appointments for these patients. Primary-care physicians were required to complete a consultation form including clinical information and number of visits requested prior to the unlocking of the system. For control patients, physician approval was not required prior to accessing services, and both self-referral or physician referral were permitted. Intervention patients decreased specialty use by 0.57 visits per year more than control patients (95% CI –1.05 to –0.01; p = 0.04). The intervention group increased primary-care use; however, this change was not significant. Changes in patient satisfaction with care, perceived access to specialists and use of services were similar between the two groups.
The second paper145 evaluated the elimination of a gatekeeping system. The need for referral from a primary-care provider was removed and patients were able to call and schedule an appointment with any specialist in the group. Rates of visits to specialists were stable during the baseline period and during the intervention period. However, first visits to specialists increased slightly from 0.19 to 0.22 per patient per 6-month period (p < 0.001). The average proportion of visits to eligible specialists as a percentage of all visits was 29% during the year before the removal of gatekeeping and 29.6% during the year afterwards (p = 0.39).
The third paper146 also evaluated the elimination of a gatekeeping system in a separate population. Elimination of gatekeeping was not associated with changes in the mean number of visits to specialists (0.28 visits per 6 months before and after gatekeeping was removed), or the percentage of all children visits to specialists (11.6% vs. 12.1%, 95% CI 29.4% to 31.8%, vs. 11.8% to 12.4%). However, new patient visits to specialists by children with chronic conditions as a percentage of all specialist visits increased from 28.1% (95% CI 25.9% to 30.2%) to 32.2% (95% CI 30.1% to 34.5%). This study was at higher risk of bias.
The fourth paper in this group147 assessed utilisation of visits to primary-care physicians and to specialists in two different managed care models: a closed-panel gatekeeper model and an open-panel point-of-service model. Both plans shared the same physician network. There were more annual visits to primary care and a greater number of total physician visits in the gatekeeper model than in the point-of-service plan. However, there was no difference in rates of specialist visits between the systems.
Payment systems
Changes to payment systems were reported in four studies (Table 20). 148–151 The system changes were described as (1) change from a contract system (whereby the GP receives a fixed practice allowance plus charges fee per item to each patient) to a capitation system where GP income is based on the number of patients on their list; (2) all GPs regardless of training or practice location receive higher Medicare rebates to complete GP mental health plans and for mental health consultations; (3) replacing separate remuneration systems for publicly insured patients (capitation) and privately insured patients (fee-for-service) with a combined system of capitation and fee-for service for all; and (4) introducing a co-payment system – patient payment for attending specialist consultation. One study showed a positive effect on referral outcomes, with three studies showing a negative/no effect. One study was graded as being at higher risk of bias151 (with the other three being judged as lower risk of bias148–150). The strength of the evidence was graded as inconsistent, with none of this group reporting UK data.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Iversen 2000 151 | Payment system for GPs | Economic analysis | Norway | All specialties | 150 GPs | NR |
| McGarry 2009148 | Government spending on mental health | Audit | Australia | Mental health | 44 GPs | 5 years |
| Van Dijk 2013149 | Changes to payment systems | BA | the Netherlands | All specialties | 39,828 patients, 52 GPs | 7 years |
| Vardy 2008150 | Copayment system | Audit | Israel | All specialties | 3745 patients, 48 GPs | NR |
| GPs: 54% female, mean age 45.5 years |
McGarry et al. 148 examined changes in patient management and referral for care following the Better Outcomes in Mental Health Care (BOiMHC) programme initiative in Australia. The BOiMHC programme allows all GPs to refer patients for psychological health care under Medicare. GPs working in accredited practices who had completed accredited mental health training were able to receive service incentive payments for providing care to patients with International Classification of Diseases, Tenth Edition (ICD-10)-diagnosed mental illness. All GPs regardless of training or practice location receive higher Medicare rebates to complete GP mental health plans for patients with ICD-10-diagnosed mental illness, as well as higher rebates for mental health consultations. Significantly higher rates of referral for psychological treatments were reported in 2006 than in 2002. Significantly higher proportions of responders in 2006 reported referring half or more of their patients with mild to moderate depression (p < 0.001) for cognitive–behavioural therapy (p < 0.001).
A study from the Netherlands149 investigated the effects of replacing separate remuneration systems for publicly insured patients and privately insured patients with a combined system of payment. Guideline adherence increased between 2002 and 2008 by 7% for (formerly) publicly insured patients and 10% for (formerly) privately insured patients. In general, there were no significant differences in the trends for guideline adherence between privately and publicly insured patients, indicating the absence of an effect of the remuneration system on guideline adherence.
Vardy et al. 150 evaluated a copayment system in Israel which consisted of a payment per patient for attending a specialist consultation. The payment was described only as ‘a relatively low fixed sum to be paid prior to the appointment’. Attendance at planned appointments was 85% for specialist appointments in the community and 91.7% for specialist hospital appointments in the time period when copayment was in operation. There was no difference in self-referral and physician referral rates. Only 2% reported copayment as the reason for not attending, compared with 19% who stated that copayment was a reason for not attending an appointment in the past. Physicians stated that a need for copayment influenced their referral decision, especially with elderly or lower-income patients.
A Norwegian paper151 explored whether or not a payment system for GPs has an impact on referral. The intervention consisted of a change from a contract system (whereby the GP receives a fixed practice allowance, plus charges a fee per item to each patient) to a capitation system where each person registers with a particular GP and GP income is based on the number of patients on their list. In the capitation system where GP income is determined by the number of patients on the list, the GP referral rates to specialists increased by 42%. It was hypothesised that it is less profitable for the GP to provide services themselves and more profitable for them to let the specialists provide the services.
Referral management centres
Referral management centres or other major triage systems were reported in six papers (Table 21). 152–157 All but one155 reported UK studies. The interventions included two city-wide gateways for triage from general practice to specialist care, single-point referral systems for adult learning disability health services and old age psychiatry, a common pathway for all musculoskeletal referrals, and a gatekeeping and appropriateness review for diabetes referral. Three studies showed a positive effect on referral outcomes,152–154 with three studies showing a negative/no effect. 155–157 Two of the studies that showed a positive effect were graded at higher risk of bias153,154 (with the other four being lower risk for bias152,155–157), and the strength of the evidence was graded as inconsistent.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Cox 2013156 | Introduction of referral management centres | BA (retrospective) | UK | All specialties | 376,000 patients | 3 years |
| 85 practices | ||||||
| Ferriter 2006157 | Single assessment process | BA | UK | Psychiatry | 20 referrals | NA |
| Kim 2004155 | Diabetes referral management centre | Audit | USA | Diabetes specialists | 6941 patients | (1 year) |
| Mean age 61 years; 54% female | ||||||
| Maddison 2004 154 | Early access to musculoskeletal services | BA | UK | Musculoskeletal | NR | 18 months |
| Watson 2002152 | Single-point referral system | Audit | UK | Adult learning disability services | NR | NR |
| Whiting 2011 153 | Manchester referral gateway | Audit | UK | Eight specialties | Four practices | 5 months |
The first paper152 evaluated the impact of introducing a multidisciplinary single point of referral (SPR) system for dedicated adult learning disability health services. They completed a retrospective case note review comparing referrals to a SPR system with those to the old referral system. The SPR system used common referral criteria and a streamlined information system. A new referral form and information leaflet were developed and copies distributed to social workers, data centre managers, GPs and colleges of further education. With the introduction of the SPR system, the mean waiting time for referral to assessment was reduced from 46 (15–67) days to 6 (2–9) days. The proportion of inappropriate referrals halved from 26% to 13%. The proportion of appropriate referrals that involved more than one dedicated learning disability health professional increased from 63% to 80%.
Whiting153 evaluated development of a Manchester-wide referral gateway for triage from general practice to specialist care (including referrals to general surgery, ophthalmology, cardiology, ENT, trauma/orthopaedics, gynaecology, urology and dermatology). Referrals were electronically screened at three stages using a single standard referral letter template. At stage 1, GP referrals were checked for completeness (NHS number, date of birth, etc.), and checked against local non-commissioned policy. At stage 2, if data were missing, or the procedure was not commissioned, an electronic advice note was sent back to the GP practice. Stage 3 was clinical triage consisting of three outcomes: referral continues; referral diverted to an alternative service or advice and guidance from Map of Medicine, NICE or the local commissioner; or referral sent back to the GP to encourage more work-up or increase management in primary care. The process was completed within 2 working days. There was a 1.2% reduction in outpatient activity (compared with the 3.8% growth predicted before the intervention). No further data were reported. This study was at higher risk of bias.
The third paper in this group154 assessed the impact of a Targeted Early Access to Musculoskeletal Services (TEAMS) programme on accessibility to musculoskeletal services. The intervention established (with central clinical triage) a common pathway for all musculoskeletal referrals so that patients attended the appropriate department. A back pain pathway led by physiotherapists was developed, and GPwSIs and physiotherapists were trained to provide services for patients with uncomplicated musculoskeletal problems in the community. After the introduction of intervention, there was a major increase (116%) in the total number of referrals for musculoskeletal problems. In contrast, the number of orthopaedic referrals was slightly reduced. Over 18 months the total number of referrals more than doubled. Despite this, waiting times for musculoskeletal services fell; this was noticeable for rheumatology and pain management (primary data not given).
The only non-UK study155 examined the effect of referral management on diabetes care by evaluating Translating Research Into Action for Diabetes (TRIAD), a multicentre US study of managed-care enrolees with diabetes. Prospective referral management consisted of gatekeeping and mandatory authorisation from the management office. Retrospective referral management consisted of referral profiling and appropriateness reviews. Referral management was commonly used by health plans (55%) and provider groups (52%). In adjusted analysis, there were no associations between any of the referral management strategies and any of the referral outcome measures.
The most recent paper156 reported an evaluation to establish whether or not the introduction of referral management centres was associated with a reduction in hospital outpatient attendance rates. Eighty-five GP practices formed five groups to manage referrals. Two groups also carried out peer review of referrals. The referral management interventions were more complex than internal peer-review controls, involved a wider range of activities, and included activities not directly related to referral management (no further information on these differences is given). Four groups showed statistically significant increases in attendance rates, ranging from 0.41 to 1.20 attendances per 1000 persons per month. After correction, only one group (a referral management centre) remained significant (1.05 attendances per 1000 persons per month, 95% CI 0.64 to 1.64; p < 0.005).
The final paper in this group157 aimed to identify changes in the quality of information in referrals to an old age psychiatry service before and after the introduction of the single assessment process. The single assessment process was introduced in response to the National Service Framework for Older People, to facilitate referrals between agencies and reduce duplication for patients, carers and clinicians. All referrals between agencies were expected to be made on designated forms. The referral form consists of several free-text sections: identity of patient and carer; identity of referrer; reason for referral; assessment of urgency; risk factors; current services provided to patient; diagnosis and recent history; current medication; and signature of referrer. Two senior clinicians performed independent and masked rating of each referral, using a five-point Likert scale. The authors report that referrals were worse in all areas of quality of referral information after implementation of the single assessment process. Word count decreased from 240 (SD 120) to 129 (SD 39) (p = 0.005). Time to read in seconds increased from 96 seconds (SD 40 seconds) to 124 seconds (SD 41 seconds) (p = 0.001). Illegible sections (% of) increased from 2 (10%) to 6 (30%) (p = 0.011). The number of raters who strongly agreed or agreed with the statement: ‘I am able to judge the appropriateness of the referral’ decreased from 19 to 5 (p = 0.001). ‘I would need to seek further information before processing this referral’ increased from 3 to 17 (p = 0.001). ‘Overall I think the referral is useful’ decreased from 17 to 3 (p = 0.001).
Patient-focused interventions
We found few examples of patient-focused interventions. The papers we identified comprised two evaluating the provision of health information/education, and one intervention aiming to address patient concerns and satisfaction (Table 22). 158–160 The first study showed no effect and was scored at higher risk of bias. The second showed a positive effect and scored lower risk of bias. The education interventions were graded as inconsistent and the small number of papers led to a strength of evidence grading as ‘no evidence’.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Albertson 2002158 | Recognition of patient concerns | BA | USA | All specialists | 12 GPs | NR |
| 495 patients | ||||||
| Heaney 2001 159 | Patient information booklets | RCT | UK | All specialists | 4878 patients | 12 months |
| 20 GPs | ||||||
| Lyon 2009160 | Raising community awareness | BA | UK | Cancer | NR | 12 months |
A US study158 determined whether or not a brief pre-visit questionnaire about referral concerns could improve primary-care provider recognition of patient concerns and satisfaction with care. Patients were given a pre-visit questionnaire about referral need and rationale and a post-visit questionnaire about referral concern and visit satisfaction. Providers were given a post-visit questionnaire asking whether a referral was discussed or made, and about visit satisfaction. In the control phase, patient pre-visit questionnaires remained confidential, whereas in the intervention phase GPs were shown the pre-visit questionnaire at the time of the encounter. The intervention significantly increased GP referral recognition from 61% to 81% (p < 0.001) and was associated with increased visit satisfaction (p = 0.05). Satisfaction of GPs with the referral discussion, overall rate of referral and visit duration was not affected by the intervention.
The first UK paper159 investigated the effect of patient information booklets on overall use of health services. One of two booklets was posted to participants in intervention groups. Patients randomised to the control group did not receive a booklet. ‘What Should I Do?’ was part of a patient education programme that had been implemented in the Netherlands. The booklet outlines 40 common health problems and provides information on when to consult a doctor and when self-care is appropriate. The ‘Health Care Manual’ was developed by a GP and a practice nurse in Scotland. It outlines 50 common health problems and also provides information about keeping healthy. Receipt of either booklet had no significant effect on health service use, compared with the control group (difference 0.14, 95% CI –0.18 to 0.45).
Lyon et al. 160 conducted a UK intervention which involved local people working in partnership in their communities to raise awareness of cancer symptoms and promote early presentation. The teams worked with primary care, with other statutory organisations and with the voluntary sector. The specific contribution of the local people was in the identification of hard-to-reach groups and the tailoring of effective health messages. Interim results showed an increase in the number of urgent 2-week referrals and the proportion of new cancer cases diagnosed through the urgent 2-week referral route (from 43% to 51%) for all breast, lung and bowel cancers. These results were statistically significant for the bowel cancer (χ2 = 22.193, df = 1; p < 0.001) and lung cancer pathways (χ2 = 8.886, df = 1; p = 0.003). There was also an increase in the proportion with no spread at the time of diagnosis for bowel cancer (38% to 43%) and breast cancer (41% to 44.5%), but these results did not reach statistical significance.
Intervention outcome measures
In addition to synthesising the evidence by intervention type, we examined the main outcome measures reported in each intervention study (Table 23). As with the types of interventions outlined above, we evaluated the strength of evidence which supported interventions having an effect on this range of outcomes (see Figure 2). The outcomes reported in the literature were as follows.
| Primary outcome | Studies reporting positive effect on outcome (first author and year) | Studies reporting no effect on outcome (first author and year) | Strength | ||
|---|---|---|---|---|---|
| Controlled study/RCT/cRCT/nRCT/CBA | Other | Controlled study/RCT/cRCT/nRCT/CBA | Other | ||
| Referral | Bridgman 2005,70 Julian 2007,62 Kerry 2000,59 Ramsay 2003,27 Salisbury 2005,125 Sauro 2005,127 Schulpen 2003,134 Shaw 2006,77 Vlek 2003,135 Wolters 2005,36 Wright 200656 | Albertson 2002,158 Callaway 2000,121 Campbell 2003,131 Cooper 2012,19 Cusack 2005,43 Elwyn 2007,55 Glaves 2005,57 Gurden 2012,133 Hands 2001,34 Hermush 2009,137 Hockey 2004,91 Kim 2009,98 Knol 2006,90 Lam 2011,25 Levell 2012,129 Maddison 2004,154 McGarry 2009,148 McKoy 2004,89 Simpson 2010,78 Standing 2001,122 Stoves 2010,103 Suris 2007,35 Tan 2007,140 Thomas 2010,80 Twomey 2003,42 Van Dijk 2011,123 Whited 200287 | Bhalla 2002,37 Campbell 2003,131 Dhillon 2003,82 Dey 2004,66 Engers 2005,67 Lester 200939 | Emmerson 2003,40 Gough-Palmer 2009,84 Iversen 2000,151 Johnson 2008,139 Joyce 2000,147 Magill 2009,115 Matowe 2002,50 Potter 2007,46 Rosen 2006,128 Rowlands 2003,26 Simpson 2003,143 Van Dijk 2010,141 White 2000142 | iii |
| Attendance rate | Leiba 2002,130 Schillinger 2000,144 Wallace 2004,94 Whited 200287 | Harrington 2001,93 Khan 2008,71 Stainkey 2010,118 Whiting 2011153 | Eley 2010,83 Heaney 2001159 | Cox 2013,156 Ferris 2001,145 Ferris 2002,146 Kim 2004,155 King 2001,119 Pfeiffer 2011,138 Sved-Williams 2010,72 Vardy 2008150 | iii |
| Appropriateness of referral | Banait 2003,53 Bennett 2001,30 Donohoe 2000,31 Griffiths 2006,58 Junghams 2007,109 Walkowski 2007,63 Watson 2001,32 Wong 200081 | Akbari 2012,110 Chen 2010,100 Evans 2009,21 Hughes-Anderson 2002,136 Imkampe 2006,47 Kim-Hwang 2010,102 Knab 2001,112 Lucassen 2001,45 Watson 2002152 | Ellard 2012,38 Greiver 2005,114 Slade 2008117 | Hill 2000, Kennedy 2012,106 Melia 2008,51 Shariff 201028 | ii |
| Appropriate actioning of referral | Eccles 2001,54 Emery 2007,111 Robling 2002,60 Thomas 200379 | Malik 2007,41 Mariotti 2008,113 Van Dijk 2013149 | Julian 2007,62 Tierney 2003116 | Cusack 200543 | i |
| Adequate referral information provided | Jiwa 2004, Jiwa 2012,105 Kousgaard 2003,29 McGowan 2008,107 Gandhi 2008101 | Idiculla 2000,44 White 200461 | Jiwa 200668 | Ferriter 2006,157 West 200752 | i |
| Waiting time | Leggett 2004,85 Morrison 2001,64 Wong 200081 | Dennison 2006,99 Hemingway 2006,73 Tadros 2009,96 Watson 2002152 | McNally 200374 | i | |
| QALYs/cost | Damask 2008,76 Leiba 2002,130 McGowan 2008,107 Morrison 2001,64 Robling 2002,60 Salisbury 2005125 | Harrington 2001,93 Ridsdale 2008,124 Simpson 201078 | Tierney 2003116 | Rosen 2006,128 Spatafora 200569 | iii |
| Satisfaction/attitudes | Jaatinen 2002,95 Kousgaard 2003,29 Leiba 2002,130 McGowan 2008,107 Nicholson 2006,97 Salisbury 2005,125 Sanderson 2002,126 Schillinger 2000, 144 Wallace 2004,94 Whited 2002,87 Wong 200081 | Albertson 2002,158 Gurden 2012,133 Harrington 2001,93 Hilty 2006,24 Maddison 2004,154 Patterson 2004,104 Ridsdale 2008,124 Simpson 2010,78 Stoves 2010,103 Suris 2007,35 Tadros 2009,96 Wylie 200118 | Eminovic 2009,86 Tierney 2003116 | Emmerson 2003,40 Rosen 2006128 | i |
-
Referral outcomes (n = 62). These were outcomes that related to measuring the number of referrals which had been made. The specific outcomes reported by individual papers included referral, number of referrals/number of patients referred, change/differences in referral rates, referral to a particular specialty, referred back to GP with recommendation for referral to secondary-care services, achieving target referral levels and referrals avoided. These measures were usually used in a context in which a reduction was the target of the intervention. However, there were some instances where an intervention aimed to increase referrals (e.g. early diagnosis and referral).
-
Attendance rate/service use outcomes (n = 18). These outcomes related to use of specialist services as a result of referrals from primary care. Very often this outcome measure was reported in the absence of a more direct measure of referral. The specific outcomes reported by individual papers were described as service use; attendance rate; new visits to the clinic; number of patients requiring a clinic appointment; appointment cancellation; admission avoidance/readmission; non-attendance; and self-reported visits to specialist. This measure could be used in both a positive and a negative way, in that an increase in attendance could be the target outcome (decreasing non-attendance), or, conversely, a decrease in service use could be the anticipated effect.
-
Appropriateness of referral outcomes (n = 24). These outcomes relate to measuring the amount or proportion of referrals considered to be ‘appropriate’. Both adequacy of referral (suitable level of urgency and timing) and accuracy of referral (patients referred to the most suitable place) were considered. Most outcome measures were described simply as the amount or proportion of appropriate referrals, but others were also described as the number of inappropriate referrals, quality of referral, proportion of GPs making the correct referral decision, proportion correctly referred, and proportion of asymptomatic referrals. This outcome measure reportedly has some limitations in that consideration of appropriateness could vary between practitioners.
-
Referral quality outcomes (adequate referral information provided) (n = 10). These outcomes included measures of the quality of information provided in the referral. The outcomes were focused on whether or not the information provided by the GP to the specialist was adequate for the specialist’s needs. The outcomes were described in the individual studies as referral quality, referral letter quality, referral letter content, the quality of the referral information, relevant information in the referral, impact of the information provided and referral communication.
-
Appropriate actioning of referral (n = 10). These outcomes related to guidelines and measured compliance with, or adherence to, referral guidelines. Individual studies described outcome measures as compliance/concordance with guidelines, proportion of referrals meeting guidelines, adherence to care suggestions, number of requests for treatment/appointments, and GP/specialist agreement. As with the appropriateness of referral outcome (above), this measure has some limitations owing to variations in judgements of what is considered appropriate.
-
Waiting time (n = 8). These outcomes included all measures of time from the GP making the referral to some subsequent point in the diagnostic process. Most frequently, this was the time from the GP making the referral to the patient seeing the specialist for the first time. Individual papers described the outcomes as waiting time, time from presentation to referral appointment, waiting time from referral to appointment, time from referral to diagnosis, time to diagnosis and speed of referral.
-
Costs (n = 12). Although few papers focused specifically on the cost/cost-effectiveness of an intervention, 12 papers did report cost outcomes along with other measures. The cost-related outcomes reported included cost, cost to the NHS, cost of testing, health-care costs, cost-effectiveness, QALYs and cost saving.
-
Satisfaction/attitudes (n = 27). These outcomes looked at positive impacts on decision-making and patient satisfaction, and therefore include satisfaction of the patient, the referrer or both. Many studies included satisfaction outcomes as secondary measures. The specific outcomes reported were patient satisfaction, user satisfaction, satisfaction of patients and health professionals, practitioner satisfaction and GP attitude.
Non-intervention papers: immediate effects
The non-intervention papers consisted of qualitative studies and papers reporting associations. We scrutinised data from these papers and carried out additional searching to uncover any further evidence regarding the process whereby the different types of interventions we had identified may lead to change in referral outcomes. The key gaps in evidence from the intervention literature related to, firstly, the process whereby providing GP education interventions would change referral outcomes. We carried out further targeted searching to identify evidence here termed the ‘clinical reasoning search’. The second gap related to the process whereby interventions that change processes and systems would impact on referral behaviours and outcomes, and additional searches for this evidence were named the ‘systems search’. Full search strategies are provided in Appendix 4. Full extractions of these papers are to be found in Appendix 1.
Scrutiny of this literature identified two key sections of data, which had not been described in the intervention papers. Firstly, the literature described effects resulting from an intervention at a more immediate or micro level for individuals and, secondly, the papers described a range of predictors that may influence whether or not interventions which achieve effects in the short term lead to long-term change.
The outcomes described could be considered as measuring the ‘active ingredients’ in the interventions; these are the elements that would underpin the intended changed referral practice. These factors are, therefore, of importance in influencing if and how an intervention has an effect. The outcomes described in the literature were change in the doctor’s or patient’s knowledge, attitudes or beliefs, and change in the doctor–patient relationship (Table 24). As with the intervention and outcomes data, we assessed the strength of evidence underpinning these factors being associated with referral outcomes.
| Factor | Studies reporting association with referral outcomes (first author and year) | Studies reporting no association with referral outcomes (first author and year) | Strength |
|---|---|---|---|
| Increased GP knowledge | |||
| Additional training in condition/knowledge level or familiarity with a condition | More referral: Delva 2011,161 Fucito 2003162 | Jorgensen 2001,181 Lakha 2011,179 Montgomery 2006,180 Rowlands 2001,182 Rushton 2002,183 Wassenaar 2007178 | i |
| Less referral: Elhayany 2000,167 Freed 2003,172, Kvaerner 2007,168 Naccarella 2008,169 O’Neill 2005, Ringard 2010,164 Scheerers 2007,163 Swarzrauber 2002,171 Townsley 2003,165 Zielinski 2008166 | |||
| Direction unclear: Dodds 2004,174 Knight 2003,177 Lambert 2001,175 Pomeroy 2010,176 Tzaribachev 2009173 | |||
| Increased knowledge of services/systems | Angstman 2009,184 Coulston 2008,187 Kisely 2002,185 Mitchell 2012186 | ii | |
| Use/awareness/availability of referral guidelines | Blundell 2011,189 Clarke 2010,190 Kasje 2004,191 Ramanathan 2011188 | Abel 2011,20 Baker 2006,192 Bederman 2010,196 Belgamwar 2011,197 Jiwa 2008,193 Ruston 2004194 Tucker 2003,198 Watson 2001195 | iii |
| Use or awareness of specialist service quality indicators | Morsi 2012200 | iii | |
| Increased knowledge of patient responsiveness to treatment | Philichi 2010,202 Sigel 2004201 | ii | |
| Changed GP attitudes/beliefs | |||
| Confidence in management/perceived expertise | Anthony 2010,208 Bruynincksx 2009,209 Calnan 2007,207 Knight 2003,177 Moore 2000,205 Morgan 2007,210 Nandy 2001,204 Olson 2012,206 Rosemann 2005,211 Steele 2012,203 Van der Weijden 2002,212 Wilkes 2009213 | Ahluwalia 2009,214 Pryor 2001215 | i |
| Tolerance of uncertainty/risk | Abel 2011,20 Bruynincksx 2009,209 Calnan 2007,207 Cornford 2004,218 Espeland 2003,217 Franks 2000,216 Morgan 2007,210 Rosemann 2005,211 Rushton 2002,183 Van der Weijden 2002212 | Forrest 2006283 | i |
| Belief regarding peer opinion | Bruynincksx 2009,209 Green 2008,220 Van der Weijden 2002212 | i | |
| Perception of role | Abel 2011,20 Calnan 2007,207 Knight 2003,177 Nandy 2001,204 Young 2010221 | i | |
| Changed views of specialist service: familiarity with service/referral relationship including communication | Allareddy 2007,232 Barnett 2011,222 Beel 2008,226 Berendsen 2007,237 Chew-Graham 2008,229 Clemence 2003,228 Dagneaux 2012,230 Dale 2000,224 Delva 2011,161 Forrest 2002,223 Gandhi 2000,108 Grace 2008,292 Harland 2009,231 Jorgensen 2001,181 Kinchen 2004,238 Knight 2003,177 Massey 2004,236 McKenna 2005,225 Mitchell 2012,186 Morsi 2012,200 Pomeroy 2010,176 Ringard 2010,164 Samant 2007,234 Sigel 2004,201 Taggarshe 2006,233 Thorsen 2012,239 Wakefield 2012,227 Xu 2002235 | Ahluwalia 2009 214 | i |
| Changed GP referral behaviours | |||
| Optimal time of referral | Greer 2011240 | iii | |
| Optimal content of referral | Gandhi 2000,108 Jiwa 2009, Jiwa 2004, Jiwa 2012,105 Kousgaard 2003,29 McGowan 2008107 | Ferriter 2006,157 Harvey 2005242 | i |
| Pre-testing and ability to triage | O’Byrne 2010243 | iii | |
| Changed doctor–patient interaction | |||
| Doctor–patient relationship | Baker 2006,192 Berendsen 2007,237 Carlsen 2008,244 Forrest 2007,246 Hyman 2001,248 Johnson 2011,245 Knight 2003,177 Nandy 2001,204 Ramchandiani 2002,247 Rosemann 2005211 | i | |
| Shared decision-making | Carlsen 2008,244 Clarke 2010,190 Knight 2003,177 Nandy 2001204 | i | |
| Appropriate response to patient pressure | Calnan 2007,207 Little 2004,251 Morgan 2007,210 Rosen 2007,252 Stavrou 2009,249 Vulto 2009250 | i | |
| Changed patient attitudes/beliefs | |||
| Patient wishes/patient pressure | Albertson 2000,270 Anthony 2010,208 Bekkelund 2001,259 Berendsen 2007,237 Blundell 2010,253 Brien 2008,258 Dale 2000,224, Davies 2007,256 Edwards 2002,257 Espeland 2003,217 Forrest 2002,223 Glozier 2007,254 Gross 2000,261 Knight 2003,177 Lakha 2011,179 Lewis 2000,260 Little 2004251 Morgan 2007,210 Morsi 2012,200 Musila 2011,255 Philichi 2010,202 Pomeroy 2010,176 Stavrou 2009,249 Townsley 2003165 | i | |
| Appropriate service use: number of patient visits to GP/previous referral | Albertson 2000,270 Bertakis 2001,264 Cohen 2013,267 Dearman 2006,265 Harris 2011,268 Morgan 2007,210 Ridsdale 2007,266 Shadd 2011263 | Pfeiffer 2011,138 Vinker 2007269 | i |
General practitioner knowledge
The first immediate effect of an intervention described in the literature was a change in the referrer’s level or type of knowledge. Within this, a number of subfactors were categorised.
Additional training in the presenting condition
Additional training in the presenting condition (resulting in a higher knowledge level or familiarity with the patients’ symptoms) was reported in 23 studies (Table 25). 161–183 Of these, 17 studies reported a positive association between greater knowledge of the presenting condition and better referral outcomes (including only one study at higher risk of bias,165 the others being at lower risk of bias). 161–177 A further six studies (at lower risk of bias) reported no association. 178–183 The evidence for this association was graded as inconsistent.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and participant details where reported | Response |
|---|---|---|---|---|---|
| Delva 2011161 | Survey | France | Oncology | 436 GPs | NR |
| 75% male | |||||
| Dodds 2004174 | Survey | UK | Oncology | 331 GPs; 80% practices with four or more doctors | 65% |
| Elhayany 2000167 | Audit | Israel | All specialties | 44 GPs | NA |
| 67,577 patients | |||||
| Freed 2003172 | Survey | USA | Juvenile RA | NR | 49% |
| Jorgensen 2001181 | Survey | Denmark | Physiotherapy | 38,231 referrals | 90% |
| 410 GPs | |||||
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Two practices | |||||
| Kvaerner 2007168 | Survey | Norway | ENT | 1633 GPs | 48% |
| Lakha 2011179 | Survey | Canada | Pain clinic | 47 GPs | 32% |
| Lambert 2001175 | Survey | UK | Epilepsy | 312 GPs | 67% |
| Montgomery 2006180 | Interviews | UK | Nephrology | 51 GPs | 65% |
| Naccarella 2008169 | Survey | Australia | Mental health | 89 projects | 81% |
| O’Neill 2005170 | Survey | USA | All specialties | 2455 GPs | NR |
| Pomeroy 2010176 | Interviews/survey | Australia | Dietitian | 248 GPs (survey) | 30% |
| Ringard 2010164 | Survey | Norway | All specialties | 3493 GPs | 48–50% |
| Rowlands 2001182 | Video transcript | UK | All specialties | NR | NA |
| Rushton 2002183 | Survey | USA, Canada, Puerto Rico | Psychosocial services | 4012 patients | NR |
| Scheerers 2007163 | Survey | the Netherlands | Mental health | 301 GPs | NR |
| Swarzrauber 2002171 | Survey | USA | Neurology | 609 GPs | NR |
| 1116 specialists | |||||
| Townsley 2003 165 | Survey | Canada | Oncology | 2089 GPs | 24% |
| Tzaribachev 2009173 | Cohort | Germany | Paediatric rheumatology | 132 patients | NA |
| Wassenaar 2007178 | Survey | USA | Oncology | 672 GPs | 59.4% |
| Zielinski 2008166 | Audit | Lithuania | All specialties | 18 practice | NA |
Seventeen studies presented data suggesting an association between GP knowledge from training in the presenting condition and referral patterns. 161–177 Three studies suggested that GPs with training in a particular condition would refer more. 161–163 A study from France161 presented data which suggested that GPs’ attendance at a training course was associated with being more likely to refer for advanced cancer (OR = 1.85, 95% CI 1.01 to 3.38). Fucito et al. 162 reported that GPs who stated they regularly obtained information (training) about drug and alcohol use were more likely to refer patients for these problems (χ2 = 7.0, p < 0.01). Scheerers et al. 163 found that, in the Netherlands, GPs who received written training materials encouraging them to refer for chronic fatigue syndrome had higher referral rates.
However, nine studies suggested that GPs with training (increasing knowledge level or familiarity) in a particular condition would refer less. 164–172 The first164 reported that frequency of GPs attending formal meetings (training) and the GPs’ level of expertise were associated with lower referral rate in the Netherlands. A Canadian paper165 reported that GPs with extra training in geriatrics and those in practice longer were likely to refer regardless of tumour stage. Zielinski et al. 166 reported that being a specialist in family medicine, training and experience correlated with lower referral rates in Lithuania. A study in Israel167 found that GPs without any postgraduate training or specialty designation were likely to refer 2.5 times more often than primary paediatricians or family physicians. Kvaerner et al. 168 found that GPs in Norway who had received specialty training in general medicine made 6% fewer referrals than those who did not. Naccarella et al. 169 found that informing and training Australian GPs was the most popular demand management strategy to reduce referrals in a survey of project officers who had carried out demand management projects. The first of two US studies170 reported that GPs who were ‘board certified’ (trained) were associated with lower factor referral scores. The second171 found that GPs who preferred to manage patients without specialty involvement had higher knowledge scores than primary care physicians who preferred to refer to a specialist (p < 0.001). The final paper in this group172 reported a study on referral for juvenile rheumatoid arthritis and reported that 61% of GPs referred only to confirm diagnosis and guide initial therapy.
Another five studies suggested a link between training (or obtaining knowledge) and referral, but the direction of effect was unclear. Tzaribachev et al. 173 reported that a statistically significant predictor of delayed referral was the primary physician’s subspecialty training (p = 0.016). Dodds et al. 174 reported that GPs described that training for the 2-week wait guidance for cancer referrals created a rigid and inflexible system which did not offer scope for GP own judgement and experience. A UK study175 found that 64% of the GPs they surveyed would welcome teaching on epilepsy. Pomeroy and Cant176 reported that GP previous experience and knowledge of service were associated with referral. A second UK paper177 reported that GP expertise was one of 12 ‘doctor-related factors’ which could influence referral decisions.
There were a further six studies which reported that training in a particular condition (and the increased knowledge level, or familiarity with symptoms as a result of this) was not associated with referral. Wassenaar et al. 178 reported no difference in referral patterns related to those who had more or fewer patients with cancer (differing levels of familiarity with condition) in their US practice. Another North American study179 found that the more chronic pain patients a physician saw, the less he or she tended to refer them to pain clinics, but the relationship was not significant. A third UK paper in this group180 reported that referral rate did not differ by experience with renal patients. Jorgensen et al. 181 reported that the GP having frequent contact with a physiotherapist explained only a very small amount of referral variation, leaving the greatest majority of variation unexplained. Rowlands et al. 182 reported no alteration of practice referral rate following a UK education intervention. Rushton et al. carried out a survey across three countries183 and found that providing training in behaviour management did not change rate of referral for child psychosocial services.
Increased knowledge of services or systems for referral
Increased knowledge of services or systems for referral was reported in four studies (Table 26). 184–187 Although three of the four studies showed a positive association between increased knowledge of services or systems and better referral outcomes, three of the studies in the group were at higher risk of bias. 184–186 The evidence for this association was, therefore, graded as weaker.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Angstman 2009 184 | Survey | USA | Viral specialist | 56 GPs | NR |
| Coulston 2008187 | Survey | UK | Hernia surgery | 86 GPs | 72% |
| Kisely 2002 185 | Survey | Australia | Mental health | 74 GPs | 45% |
| Mitchell 2012 186 | Survey/interviews | Australia | Dietitian | 90 survey | 20–22% |
Angstman et al. 184 found that GPs reported that they often forgot that viral specialist consultations were an option, suggesting that increased knowledge would increase referral to the service. The first of two Australian papers185 reported that 80% of participants found the intervention duty officer useful as a point of first contact for the consultation-liaison service. The second186 reported that GP relationships with dieticians were believed to be the primary influencing factor on referral. The fourth study in this group187 found that only 17% of GPs were aware of any specialist consultant surgeons in South Wales performing laparoscopic groin hernia repair. Of those who were aware, 80% had at some time referred to this service.
Greater use or awareness of referral guidelines
Greater use or awareness of referral guidelines was reported in 12 studies (Table 27). 20,188–198 Of these, only four showed an association with better referral outcomes188–191 (one of which was at higher risk of bias190). A further eight studies showed no association between these factors (all at lower risk of bias). 20,192–198 The evidence for this association was, therefore, graded as conflicting.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where available | Response |
|---|---|---|---|---|---|
| Abel 201120 | Interviews | New Zealand | Colorectal cancer | 15 GPs | NA |
| 11 specialists | |||||
| Baker 2006192 | Interviews | UK | Lumbar spine X-ray | 29 GPs | NA |
| 24 male | |||||
| Bederman 2010196 | Delphi | Canada | Lumbar spine disease | 10 GPs/specialists | NA |
| Belgamwar 2011197 | Audit | UK | Anxiety/depression | Seven GPs | NA |
| 204 referrals | |||||
| Blundell 2011189 | Survey | UK | Elective surgery | 310 GPs | 41.6% |
| Clarke 2010 190 | Survey | UK | Elective surgery | 324 GPs | 40% |
| Jiwa 2008193 | Survey | UK | Lower bowel symptoms | 260 GPs | 52% |
| 50% male | |||||
| Aged 40 + years | |||||
| Kasje 2004191 | Survey | the Netherlands | All specialties | 197 GPs | GPs 75% |
| 34 general internists | Internists 50% | ||||
| Ramanathan 2011188 | Survey | Australia | Gynaecology/oncology | 140 GPs | 45.5% |
| Ruston 2004194 | Interviews | UK | All specialties | 85 GPs | NA |
| 49 male | |||||
| Tucker 2003198 | Interview | UK | Paediatric rheumatology | 171 GPs | 68% GP |
| Survey | 158 midwives | 77% midwives | |||
| Watson 2001195 | Survey | UK | Regional genetics service | 50 GPs | 94% |
Ramanathan et al. 188 reported greater variation in referral practice for endometrial cancer for which there are no Australian guidelines: 68% of vignettes with high probability of cancer were referred compared with 83% for ovarian cancer and 80% for cervical cancer for which guidelines are available. Blundell et al. 189 reported that most responding GPs indicated support for UK referral guidelines but 18% reported that they had never used them and < 3% reported use for most or all referral decisions. The odds of using guidelines decreased with increasing GP age, with a 10-year increase in age associated with halving odds of use (OR 0.53, 95% CI 0.29 to 0.90). Another UK study190 similarly found that although there was overall support from GPs for referral guidelines, these were rarely used in practice. Kasje et al. 191 reported that in the Netherlands most hospital specialists relied for their prescribing on international guidelines and agreements within their own department, whereas GPs relied more on national and regional guidelines.
Another UK study192 reported that both high and low referrers were aware of the X-ray guidelines for lumbar spine. Jiwa et al. 193 concluded that the application of guidelines by UK GPs is moderated by the influence of the characteristics of the patients only. Ruston et al194 echoed this lack of influence of UK guidelines, finding that none of responding GPs reported using referral guidelines as they considered them to be of theoretical rather than practical relevance. A Canadian study196 similarly found poor concordance of both predicted GP preferences and guideline recommendations with actual referral. Watson et al. 195 supported these limitations in their finding that, despite UK guidelines, many GPs did not know which patients warranted referral to a genetics service. Belgamwar et al. 197 reported that exactly half of all referrals (32/64) did not follow guidelines. Another study found that for paediatric rheumatology referrals intended management was most often referral or admission to a specialist hospital (59%, 132/224), both courses of action beyond guideline recommendations. 198 Abel and Thompson explored possible reasons underpinning this limited use of guidelines. 20 They reported that GPs perceived that rigid adherence to guidelines was inappropriate when working for the benefit of the patient.
Awareness of quality indicators
The use or awareness of quality indicators was reported in only one study200 at lower risk of bias (Table 28) in which publicly available quality measures were found to be ‘not at all important’ to referral decisions. The evidence from this study was, therefore, graded as no evidence of an association between awareness of quality indicators and referral outcomes.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where available | Response |
|---|---|---|---|---|---|
| Morsi 2012200 | Survey | USA | All specialties | 10 GPs | NR |
Knowledge of patient responsiveness to treatment
Increased knowledge of patient responsiveness to treatment/suitability for treatment was reported in two studies,201,202 one at higher risk of bias (Table 29). 202 The evidence for this association was graded as weaker.
The first study201 reported that referral decisions were made when GPs perceive that they have reached the limits of their capabilities for treating a problem, taking account of patient suitability for therapy and access to services. The second paper202 suggested that the most frequently identified reason for referral was patient unresponsiveness to treatment.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size | Response |
|---|---|---|---|---|---|
| Philichi 2010 202 | Survey | USA | Paediatric gastroenterology | 237 primary care | 38% |
| Paediatricians and nurse practitioners | |||||
| Sigel 2004201 | Interviews | UK | Psychological problems | 10 GPs | 40% |
| Seven male | |||||
| Age 38–60 years |
General practitioner attitudes and beliefs
The following elements were identified within the category of GP attitudes and beliefs which influenced referral decision-making.
Confidence in management of the patient
Increased confidence in management of the patient, or own perceived expertise, was reported in 14 studies (Table 30). 177,203–215 Of these, 12 showed a positive association between increased confidence and better referral outcomes203–213 (three of these were higher risk of bias203,205,208), and two showed no association between the factors214,215 (one of which was at higher risk of bias214). Therefore, the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size | Response |
|---|---|---|---|---|---|
| Ahluwalia 2009 214 | Survey | USA | Palliative care | 145 GPs | 85% |
| 58% female | |||||
| Anthony 2010 208 | Interviews/survey | USA | Depression care | 40 physicians, 15 GPs, 10 nurse practitioners | NA |
| 27 male | |||||
| Bruynincksx 2009209 | Survey | Belgium | All specialties | 163 GPs | NA |
| Calnan 2007207 | Interviews | UK | Immediate care | 15 GPs | NA |
| 10 male | |||||
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Eight male | |||||
| Moore 2000 205 | Survey | USA | Neurology | 504 GPs | NR |
| Morgan 2007210 | Interviews | UK | Headache | 20 GPs | 50% |
| Nandy 2001204 | Interviews | UK | Mental health | 23 GPs | 67% |
| Olson 2012206 | Survey | Canada | Palliative radiotherapy | NR | 33% |
| Pryor 2001215 | Survey | Australia | Psychology | 105 GPs | 66% |
| 69% female | |||||
| Rosemann 2005211 | Survey | Germany | All specialties | 26 GPs | NR |
| Steele 2012 203 | Survey | Canada | Psychiatry | 847 GPs | 24.9% |
| Remote areas | |||||
| Male aged 41–60 years | |||||
| Van der Weijden 2002212 | Interviews | the Netherlands | Unexplained symptoms | 21 GPs | NA |
| Wilkes 2009213 | Interviews | UK | Infertility | 12 GPs, five specialists | NA |
| 13 patients |
Steele et al. 203 reported that lower GP confidence in managing mental health patients was associated with referral. Nandy et al. 204 associated lower referral rates with GPs having an interest in mental health and having confidence in dealing with mental health. Moore et al. 205 found that GPs who rated themselves as comfortable with seizure patients tended to refer fewer of these patients. Olson et al. 206 similarly found a strong relationship between family physician referral and self-assessed or tested knowledge and confidence (p < 0.001 and p < 0.010). One of five UK studies in this group207 reported that low referrers were more confident in their decisions and less often worried afterwards. Anthony et al. 208 reported that a clinician’s comfort in treating depression was identified by 80% as a very important factor for referral. Bruynincksx et al. 209 reported that whether or not the GP was uncertain of the diagnosis was associated with referral.
A second UK study177 reported that GPs needing advice affected referral. A further UK paper210 reported that GP clinical confidence in identifying risks of brain tumour affected referral. Rosemann et al. 211 reported that GPs’ experiences were more positive if their purpose was to reduce diagnostic uncertainty (p < 0.001) or if the purpose was to exclude serious illness (p < 0.010). Van der Weijden et al. 212 reported that GP uncertainty affected referral in the Netherlands. Wilkes et al. 213 found that UK GPs often reported a lack of skills or lack of confidence over infertility referrals. Ahluwalia et al. 214 found that having personal experience with palliative care was not statistically related to the likelihood of referral (OR 2.13, 95% CI 0.95 to 4.98). The final paper in this group215 reported that perception of professional competency was not a barrier to referral in Australia.
Tolerance of uncertainty and risk
Tolerance of uncertainty and risk in diagnosis and referral was reported in 11 studies (Table 31). 20,183,207,209–212,216–219 Of these, 10 reported a positive association between risk tolerance and better referral outcomes20,183,207,209–212,216–218 (with only one of these being at higher risk of bias20). The remaining study showed no association (and was at lower risk of bias). The evidence for this association was, therefore, graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details were reported | Response |
|---|---|---|---|---|---|
| Abel 2011 20 | Interviews | New Zealand | Colorectal cancer | 15 GPs | NA |
| 11 specialists | |||||
| Bruynincksx 2009209 | Survey | Belgium | All specialties | 163 GPs | NA |
| Calnan 2007207 | Interviews | UK | Immediate care | 15 GPs | NA |
| 10 male | |||||
| Cornford 2004218 | Interviews | UK | Breast cancer | 20 GP/other | NA |
| Surgeons, nurses | |||||
| Espeland 2003217 | Focus groups | Norway | All specialties | 14 GPs | NA |
| Forrest 2003219 | Audit | USA | All specialties | 139 GPs | NA |
| 14,709 visits | |||||
| Franks 2000216 | Survey | USA | All specialties | 173 GPs | 66% |
| Morgan 2007210 | Interviews | UK | Headache | 20 GPs | 50% |
| Rosemann 2005211 | Survey | Germany | All specialties | 26 GPs | NR |
| Rushton 2002183 | Survey | USA, Canada, Puerto Rico | Psychosocial services | 4012 patients | NR |
| Children | |||||
| van der Weijden 2002212 | Interviews | the Netherlands | Unexplained symptoms | 21 GPs | NA |
Franks et al. 216 found that greater malpractice fear was associated with greater likelihood of referral in the USA. Bruynincksx et al. 209 reported that referral in Belgium was affected by GP uncertainty or anxiety. A UK paper207 reported that high referring GPs tended to express anxiety about the consequences of a decision. A paper reporting a survey across three countries183 described defensive GP referral strategies where there was risk to the woman of not referring when breast cancer was a serious disease and risk of the patient resorting to litigation if not referred and a problem was found later. Morgan et al. 210 found that in the UK referral was related to personal tolerance of uncertainty. This was echoed by a paper from the Netherlands,212 which also reported that GPs’ handling of uncertainty or error tolerance influenced referral. Rosemann et al. 211 reported that GPs’ experiences of referral were more positive if the GP’s purpose was to reduce diagnostic uncertainty (p < 0.001). Abel and Thompson20 found that GPs considered emotional or subjective concerns for the patient more relevant than subjective measures of risk. Espeland et al. ,217 similarly to the above studies, found that GP uncertainty influenced referral. Cornford et al. 218 reported that UK GPs varied in the extent to which they could accept the uncertainty about diagnosis. However, Forrest et al. ,219 in a US study, reported that anxiety as a result of to clinical uncertainty did not influence referral.
Peer opinion
An association between beliefs regarding peer opinion and referral was reported in three studies (Table 32). 209,212,220 All three showed a positive association between positive beliefs regarding peer opinion and referral outcomes (all were at lower risk of bias). The evidence for this association was, therefore, graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where available | Response |
|---|---|---|---|---|---|
| Bruynincksx 2009209 | Survey | Belgium | All specialties | 163 GPs | NA |
| 55% female | |||||
| Green 2008220 | Survey | UK | Eating disorders | 88 GPs | 33% |
| van der Weijden 2002212 | Interviews | the Netherlands | Unexplained symptoms | 21 GPs | NA |
A UK study220 found that intention to refer was significantly related to subjective norms (believing that a referral would be recommended by colleagues) and cognitive attitudes (r = 0.917 and 0.0896, p < 0.001). Bruynincksx et al. 209 found that GP referral was influenced by a perceived negative attitude towards the GP by specialists they had previously referred to. Van der Weijden et al. 212 also highlighted the influence of social norms on referral.
Role perception
The influence of the GP having a specific perception of their own role (perception of role as gatekeeper, responsibility for the patient, or referring for patient reassurance) was highlighted in five studies (Table 33), all of which reported an association between role perception and referral outcomes. As only one study was graded at higher risk of bias,20 the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Abel 2011 20 | Interviews | New Zealand | Colorectal cancer | 15 GPs | NA |
| 11 specialists | |||||
| Calnan 2007207 | Interviews | UK | Immediate care | 15 GPs, 10 male | NA |
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Eight male | |||||
| Nandy 2001204 | Interviews | UK | Mental health | 23 GPs | 67% |
| Young 2010221 | Interviews | Australia | All specialists | 10 GPs | NA |
One of three UK papers201 reported that some GPs saw their role as preventing burden on other agencies and thus tended not to refer, whereas others perceived that their role was diagnostic and patients were best managed by others (and thus tended to refer). A second UK study207 explored GP role perception and reported that low referrers saw hospitals as places to be avoided and that their role was to prevent admission. The other UK paper177 found that low referrers to mental health services might take more responsibility for patients and have more interest in treating psychological problems. Young et al. 221 found that processes of referral were influenced considerably by the degree to which GPs had taken on broader chronic care models rather than a more traditional care approach. Abel and Thomspon20 found that GPs perceived that referral and getting patients seen was part of their duty to do the best for the patient.
Views of a specialist service
The potential influence of a GP having specific views of a specialist service (as a result of increased familiarity with service or a better referral relationship, including communication with the specialist) was reported in 29 studies (Table 34). 108,161,164,176,177,181,186,200,201,214,222–239,292 Of these, 28 studies reported an association between a better GP view of a service and positive referral outcomes (one study reported no association). 214 Four studies were reported as having a higher risk of bias. 224,227,233 Despite this, the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Ahluwalia 2009 214 | Survey | USA | Palliative care | 145 GP | 85% |
| 58% female | |||||
| Allareddy 2007232 | Focus groups | USA | Chiropractic | NR | NA |
| Barnett 2011222 | Survey | USA | All specialties | 386 GPs | 63% |
| 64% male | |||||
| Beel 2008226 | Interviews | Australia | Psychology | 12 GPs | NA |
| Eight male | |||||
| Berendsen 2007237 | Interviews | the Netherlands | All specialists | 21 GPs | NA |
| Chew-Graham 2008229 | Interviews | UK | Mental health | GPs (no number) | NA |
| Clemence 2003228 | Interviews | UK | Musculoskeletal conditions | 22 GPs | NR |
| Dagneaux 2012230 | Focus groups | Belgium | Geriatricians | NR | NA |
| Dale 2000 224 | Survey | UK | Paediatric neurology | 50 GPs | NR |
| Delva 2011161 | Survey | France | Oncology | 436 GPs | NR |
| 75% male | |||||
| Forrest 2002223 | Survey | USA | All specialties | 141 GPs | NR |
| Gandhi 2000108 | Survey | USA | Orthopaedics, cardiology and gastro | 48 GPs | 53–56% |
| 400 specialists | |||||
| Grace 2008292 | Survey | Canada | Cardiology | 510 GPs/specialists | 36% |
| Harlan 2009231 | Survey | USA | Paediatrics | 10 paediatricians | NR |
| 12 GPs | |||||
| Jorgensen 2001181 | Survey | Denmark | Physiotherapy | 38,231 referrals | 90% |
| 410 GPs | |||||
| Kinchen 2004238 | Survey | USA | All specialists | 1252 GPs | 59.1% |
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Two practices | |||||
| Eight male, one female | |||||
| Massey 2004236 | Survey | UK | Physiotherapy | 50 GPs | 65% |
| McKenna 2005225 | Survey | USA | All specialties | 460 GPs | 46% |
| Mean age 48 years | |||||
| Mitchell 2012186 | Survey/interviews | Australia | Dietitian | 90 surveys | 20–22% |
| 52 interviews | |||||
| Morsi 2012200 | Survey | USA | All specialties | 10 GPs | NR |
| Pomeroy 2010176 | Interviews/survey | Australia | Dietitian | 248 GPs (survey) | 30% |
| 30 GPs interviewed: 14 male | |||||
| Ringard 2010164 | Survey | Norway | All specialties | 3483 GPs | 48–50% |
| Mean age 48 years | |||||
| Samant 2007234 | Survey | Canada | Radiotherapy | 400 GPs | 50% |
| Sigel 2004201 | Interviews | UK | Psychological problems | 10 GPs | 40% |
| Seven male | |||||
| Age 38–60 years | |||||
| Taggarshe 2006 233 | Focus group/survey | UK | All specialists | NR | 99% |
| Thorsen 2012239 | Focus groups | Norway | All specialists | 31 GPs | NA |
| 17 female | |||||
| Age 29–61 years | |||||
| Wakefield 2012 227 | Survey | Canada | Cardiology | 91 GPs | 19.9% |
| Xu 2002235 | Audit | USA | All specialties | 2572 GPs | NA |
| 79% male |
Barnett et al. 222 reported that GPs initiated referrals to 66% of their professional network colleagues. Delva et al. 161 associated referral with whether or not the GP was used to collaborating with the oncologist. Morsi et al. 200 reported that 70% of GPs said that familiarity with the hospital influenced referral. Ringard164 reported that referral was affected by having a formal arena for co-operation and exchange of information. Forrest et al. 223 reported that personal knowledge of the specialist was the most important reason for selecting a specific specialist. Jorgensen et al. 181 found that having frequent contact with a physiotherapist explained a small variation in referral rates (6.7% to 9.2%). Dale and Goodman224 reported that reasons for referral were having prior knowledge of the service and having previously referred to the service. McKenna225 found that GPs with greater understanding of the practice of the specialists were more likely to refer (p = 0.003). Sigel and Leiper201 found that referral decisions were influenced by professional interactions with psychologists. Knight177 found that previous experience with service influenced referral. Pomeroy and Cant176 found that GP knowledge of local services affected referral. Beel et al. 226 found that GP dissatisfaction with professional communications from psychologists affected referral. Wakefield et al. 227 reported that previous experience with a facility affected referral. Clemence et al. 228 found that GPs’ past experience of physiotherapy significantly affected referral. Chew-Graham et al. 229 found that lack of direct doctor-to-doctor communication was perceived to contribute to referral issues. Dagneaux et al. ,230 in areas with few geriatric services, found that doctors knew little of other professionals and reported suspicion and even conflicts. Harlan et al. 231 found that specialists and GPs acknowledge that significant barriers to optimal communication currently exist. Mitchell et al. 186 found that GPs’ relationships with dieticians were believed to be the primary influencing factor on referral by 81% of dieticians. Allareddy et al. 232 reported that GPs expressed a lack of understanding of chiropractic care and did not have any relationship with practitioners. Gandhi et al. 108 found that 28% of GPs and 43% of specialists were dissatisfied with information received from the other group. Grace et al. 292 found that GP lack of familiarity with cardiology site locations negatively impacted referral (p < 0.001). Taggarshe et al. 233 found almost four out of five GPs made referrals specifically to a named surgeon and valued personal rapport with the consultant. Samant et al. 234 reported that physicians who referred patients for radiotherapy were more likely to have sought advice from a radiation oncologist in the past. Xu et al. 235 found that the most significant determiner of perceived ability to refer was GP satisfaction in their communication with specialists. Massey et al. 236 found that those GPs not previously aware of a physiotherapy service would refer in the future. Berendsen et al. 237 found that ‘developing personal relationships’ and ‘gaining mutual respect’ dominated when the motivational factors for referral were considered. Kinchen et al. 238 found that previous experience with the specialist affected referral. Thorsen et al. 239 reported that GPs wished for improved dialogue with the hospital specialists. However, Ahluwalia et al. 214 found that having personal experience with palliative care was not statistically significantly associated with referral (OR 2.13, 95% CI 0.95 to 4.98).
General practitioner referral behaviour
A number of behaviour effects were reported following interventions which may be associated with changed referral pathways. Factors that were categorised as elements of GP referral behaviour associated with referral included the following subfactors.
Optimal timing of referral
One study was found that considered the potential significance of this aspect of the referral process. 240 This US paper reported that the enhanced use of optimal tests for kidney function by GPs could be associated with timely referral (Table 35). 240 The study was at low risk of bias and this evidence was graded as no evidence (evidence from only one study).
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Greer 2011240 | Survey | USA | Nephrology | 178 GPs and specialists | NR |
Optimal content of referral
The impact of optimal referral content was reported in eight studies (Table 36). 23,29,105,107,108,157,241,242 Of these, six showed an association between referral content and outcome23,29,104,107,108,241 (two studies reported no association156,242). All of the studies were at lower risk of bias and the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Ferriter 2006157 | BA | UK | Psychiatry | 20 referrals | NR |
| Gandhi 2000108 | Survey | NR | All specialties | 430 referrals | NR |
| Harvey 2005242 | Survey | UK | Psychiatry | 107 GPs | 94% |
| Jiwa 200423 | nRCT | UK | All specialties | 26 GPs | 100% |
| Jiwa 2009241 | Audit | UK | Gastroenterology | 207 referrals | NA |
| Jiwa 2012105 | BA | Australia | All specialties | NR | NR |
| Kousgaard 200329 | Survey | Denmark | Oncology | 199 GPs | 88.3% |
| McGowan 2008107 | RCT | Canada | All specialties | 82 GPs | 93.2% |
In the first of three papers by the same author, Jiwa et al. 241 reported that the cases that could be triaged from the letter were those where the letter contained more information (mean 66.38 vs. 49.86, mean difference 16, 95% CI 1.3 to 31.7; p < 0.001). The second paper23 reported that feedback improves the content of GP referral letters and may also impact on the type of patients referred for investigation by specialists. The third104 found that standardising and using electronic communications to refer facilitates the scheduling of specialist appointments. Kousgaard et al. 29 reported that better information provision before and after referral improved co-operation between the specialist department and the GP. McGowan et al. 107 found that providing timely information to clinical questions had a highly positive impact on decision-making and a high approval rating from participants. Gandhi et al. 108 echoed these other authors, highlighting that electronic referral can improve referral content and communication.
However, Harvey et al. ,242 in contrast, found no difference between higher- or lower-quality referral letters and referrals to psychiatric services. Ferriter et al. 156 suggested that the introduction of a single assessment process impaired clinical communication between GPs and psychiatrists.
Use of pre-referral testing
One paper243 reported that in 72% of cases an alteration to the diagnostic investigations thought to be necessary by GPs was required when the patient was seen by a specialist (Table 37). The paper highlighted the importance of accurate referral information in order to select tests prior to consultation. The study was at lower risk of bias and this evidence was graded as no evidence.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size | Response |
|---|---|---|---|---|---|
| O’Byrne 2010243 | Audit | UK | Respiratory consultants | 50 referrals | NA |
Doctor–patient interaction
Outcomes relating to changing the doctor–patient interaction and the association between this and referral practice were described in a large body of work. Elements of the doctor–patient interaction included the following subfactors.
Optimal relationship
Having a positive doctor–patient relationship (optimal relationship) was reported to be positively associated with referral outcomes in 10 studies (Table 38). 177,192,204,211,237,244–248 As only two were considered to be at higher risk of bias, the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Baker 2006192 | Interviews | UK | Lumbar spine X-ray | 29 GP | NA |
| 24 male | |||||
| Berendsen 2007237 | Interviews | the Netherlands | All specialists | 21 GPs | NA |
| Carlsen 2008 244 | Survey | Norway | All specialties | 41 GPs | 46% |
| 66% male | |||||
| Forrest 2007246 | Survey | USA | All specialties | 776 patients | NR |
| 133 GPs | |||||
| Hyman 2001248 | Survey | Canada | Mammography | 64 GPs | NA |
| 40% female, age range 29–71 (42.16) years | |||||
| Johnson 2011 245 | Interviews | Australia | Oncology | 40 GPs | NA |
| Mean age 47 (30–60) years | |||||
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Two practices | |||||
| Eight male, one female | |||||
| Nandy 2001204 | Interviews | UK | Mental health | 23 GPs | 67% |
| Ramchandiani 2002247 | Survey | UK | Ophthalmology | 50 GPs | 64% |
| 776 specialists | |||||
| 85 patients, 55 female. Mean age 75.5 years | |||||
| Rosemann 2005211 | Survey | Germany | All specialties | 26 GPs | NR |
Baker et al. 192 found a greater emphasis on the fragility of the doctor–patient relationship in higher referrers, and reported the use of referral for radiography as a method of attempting to preserve this relationship. Nandy et al. 204 reported that poor rapport with a patient was a reason for referral. Carlsen et al. 244 found that the more the doctor and patient differ in attitude towards patient involvement, the more often the GP refers to specialist care (p = 0.001). Knight177 also found that the quality of the doctor–patient relationship influenced referral decisions. Johnson et al. 245 similarly reported that communication and interpersonal issues affected referral. Forrest et al. 246 found that longer duration of the doctor–patient relationship was a positive predictor of referral completion. Rosemann et al. 211 found that experiences with the referral were more positive if the initiative for the referral came from the physician (beta = 0.365, p < 0.001). Ramchandiani et al. 247 reported that pooled lists were unpopular as they devalued the doctor–patient relationship. Berendsen et al. 237 reported that 81% of patients thought it was important that the GP gave them advice on which hospital or specialist to go to. Hyman et al. 248 found that physicians who spent more time on patient education were more likely to refer.
Shared decision-making
Shared decision-making between the GP and the patient was reported to be positively associated with referral outcome in four studies (Table 39). 177,190,204,244 As only two were at higher risk of bias,190,244 the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Carlsen 2008 244 | Survey | Norway | All specialties | 41 GPs | 46% |
| 66% male | |||||
| Clarke 2010 190 | Survey | UK | Elective surgery | 324 GPs | 40% |
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Two practices | |||||
| Eight male, one female | |||||
| Nandy 2001204 | Interviews | UK | Mental health | 23 GPs | 67% |
Clarke et al. 190 found that the view that patients should be involved in referral decision-making was strongly supported by UK GPs. Another study from the UK177 found that patient wishes and preferences influenced referral decisions. Carlsen et al. 244 reported a significant negative correlation between GP score and referral rate (–0.46, p = 0.002), indicating that GPs with a preference for patient involvement in Norway are less likely to refer. Nandy et al. , in a third UK study in this group,204 reported that the patient desire to be referred was important.
Appropriate response to patient pressure
Response to patient pressure was reported to be associated with referral outcomes in six studies (Table 40). 207,210,249,250,251,252 All studies were at lower risk of bias and the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample/treatment | Response |
|---|---|---|---|---|---|
| Calnan 2007207 | Interviews | UK | Immediate care | 15 GPs | NA |
| 10 male | |||||
| Little 2004251 | Survey | UK | Depression | 30 GPs | NA |
| 847 patients aged 16–80 years | |||||
| Morgan 2007210 | Interviews | UK | Headache | 20 GPs | 50% |
| Rosen 2007252 | Interviews, focus groups | UK | All referrals | GPs (no number) | NA |
| Stavrou 2009249 | Interviews | UK | Mental health | 14 GPs | 47% |
| Seven male. Mean age 39 years | |||||
| Vulto 2009250 | Survey | the Netherlands | Palliative radiotherapy | 489 GPs | 45.5% |
| 65% male |
Calnan et al. 207 found that low referrers described themselves as more able to resist pressure from family or carers. Stavrou et al. 249 found that no GP refused if a patient asked to be referred. The one non-UK study in this group250 found that most GPs in the Netherlands reported that they reacted to the wishes of the patient regarding referral. Little et al. 251 found that doctor’s perception of moderate or definite patient pressure was a predictor of referral behaviour: perceived slight patient pressure to be referred – 19% referred, 5% not referred (OR 8.99, 95% CI 4.91 to 16.46; p = 0.994); perceived moderate or definite pressure – 44% referred, 1% not referred (OR 125.3, 95% CI 51.3 to 306.5; p = 0.005). Morgan et al. 210 reported that GPs showed variations in an individual’s willingness or ‘resistance’ to refer, reflecting differences in clinical confidence and views of patients’ ‘right’ to referral. Rosen et al. 252 reported that most GPs make choices on the patient’s behalf (with or without Choose and Book) unless the patient expresses a preference.
Patient factors
Although we found a large body of evidence regarding the potential influence of doctor–patient interaction on referral, we found no studies that reported patient knowledge outcomes and associated these with referral outcomes. Although patient knowledge outcomes were not reported, literature describing an association between patient attitude/belief elements and referral were found. Factors that were categorised as relating to patient attitude or beliefs included two subfactors: patient pressure and service use.
Patient pressure
The association between strong patient wishes (or the amount of patient pressure imposed on the GP) was reported as being associated with referral outcomes in 24 studies (Table 41),165,176,177,179,200,202,208,210,217,223,224,237,249,251,253–261,270 of which five were at higher risk of bias. 165,202,208,224,255 Therefore, the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Albertson 2000270 | Survey | USA | All specialists | 12 GPs | NR |
| 822 patients | |||||
| Anthony 2010 208 | Survey | USA | Depression | 40 physicians; 15 general internists, 15 GPs, 10 nurse practitioners; 27 female, 13 male | NR |
| Bekkelund 2001259 | Survey | Norway | Neurology | 105 patients | 75% |
| Berendsen 2007237 | Interviews | the Netherlands | All specialists | 21 GPs | NA |
| Blundell 2010253 | Interviews | UK | Surgical | 22 GPs | 96% |
| Brien 2008258 | Interviews | UK | CAM | 10 GPs | 30% |
| Dale 2000 224 | Survey | UK | Paediatric neurology | 50 GPs | NR |
| Davies 2007256 | Audit | UK | Endoscopy | 33 referrals | NA |
| Edwards 2002257 | Focus group | UK | All specialists | 86 GPs/nurses | 51–90% |
| Espeland 2003217 | Focus groups | Norway | Radiography | 14 GPs | NA |
| Forrest 2002223 | Cohort | USA | All specialists | 141 GPs | NA |
| Glozier 2007254 | Cohort | UK | Orthopaedics | 188 referrals | NA |
| Gross 2000261 | Survey | Israel | All specialists | 1084 patients | 81% |
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Two practices | |||||
| Lakha 2011179 | Survey | Canada | Pain clinic | 47 GPs | 32% |
| Lewis 2000260 | Interviews | USA | All specialists | 314 patients | NR |
| Little 2004251 | Survey | UK | Depression | 30 GPs | NA |
| Morgan 2007210 | Interviews | UK | Headache | 20 GPs | 50% |
| Morsi 2012200 | Survey | USA | All specialties | 10 GPs | NR |
| Musila 2011 255 | Referral audit | UK | Chronic knee pain | 12 members including patients, GPs, orthopaedic surgeons and other health-care professionals | NA |
| Philichi 2010 202 | Survey | USA | Paediatric gastro | 237 GPs | 38% |
| Pomeroy 2010176 | Interviews/survey | Australia | Dietitian | 248 GPs (survey) | 30% |
| Stavrou 2009249 | Interviews | UK | Mental health | 14 GPs | 47% |
| Townsley 2003 165 | Survey | Canada | Oncology | 2089 GPs | 24% |
Blundell et al. 189 reported that the extent of patient involvement in the referral decision affected referral. Morsi et al. 200 found that patient preference was considered important in referral decisions by 62% of respondents. Forrest et al. 223 reported that patient request was the reason for 13.6% of referrals. Townsley et al. 165 found that a patient’s desire to be referred influenced GPs’ decision to refer. Stavrou et al. 249 found that referral was influenced by patient request and interest in referral; no GP refused if a patient asked to be referred. Dale and Goodsman224 reported that 78% of GPs, in making a referral, were responding to parental concerns. Little et al. 251 found that patient pressure affected referral; where patient wish to be referred was slight, 16% were referred and 8% were not referred (OR 3.34, CI 1.88 to 5.93; p = 0.796), and where patient pressure was moderate or definite, 28% were referred and 5% were not referred (OR 8.51, CI 4.97 to 14.6; p = 0.028). Glozier et al. 254 found that greater personal control (patient) was associated with referral; assertive patients better able to influence and control their lives were more successful at obtaining an urgent referral. Anthony et al. 208 found that patient preference and resources (willingness to see a mental health specialist, and ability to pay) affected referral. Knight177 found that patient wishes and preferences influenced referral decisions. Lakha et al. 179 reported patient preference for other treatments influenced referral decisions. Philichi and Yuwono202 described that the second most frequently identified reason for referral to paediatric gastroenterology was parents wanting a second opinion (15%). Pomeroy and Cant176 found that patient choice of treatment and willingness to attend affected referral. Musila et al. 255 found that ratings of referral appropriateness were strongly influenced by patients’ referral preferences. Morgan et al. 210 found that readiness to refer in response to pressure was influenced by characteristics of the consultation, including frequent attendance, communication problems and time constraints. Davies et al. 256 found that patients also identified problems with communication, information and support about diagnosis when being referred for endoscopy. Edwards et al. 257 reported the importance of patient psychosocial factors in referral. Brien et al. 258 found that a match between the doctor’s attitude and treatment preferences and patient views was important. Espeland et al. 217 found that patient wishes for radiography and the GP’s response affected referral. Albertson et al. 270 found that continuity of care and familiarity with their GP are associated with patients initiating a referral discussion with their GP.
There were a few international studies where applicability in the UK was questionable: Bekkelund et al. 259 found less Norwegian patient satisfaction (52% dissatisfied) with self-referral than with doctor referral (42% dissatisfied). Lewis et al. 260 found that, in the USA, patients valued the freedom to choose their doctor and have unencumbered access to specialists. Gross et al. 261 reported that one-third of Israeli respondents preferred self-referral to a specialist. Forty per cent preferred their family physician to act as a gatekeeper, and 19% preferred the physician to co-ordinate care but to refer themselves to a specialist. Berendsen et al. 262 reported that 81% of patients in Norway thought that it was important that the GP gave them advice on which hospital or specialist to go to when they referred the patient.
Service use
Appropriate patient behaviour in terms of appropriate service use (number of patient visits to GP and previous referral) was reported as being associated with referral outcomes in eight studies, all at lower risk of bias (Table 42). 210,263–268,270 A further two lower-risk studies showed no association between these factors. 138,269 Overall, the strength of evidence was graded as weaker.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Albertson 2000270 | Survey | USA | All specialists | 12 GPs | NR |
| 822 patients | |||||
| Bertakis 2001264 | Survey | USA | All specialties | 509 patients | NR |
| 26 GPs | |||||
| 79 specialists | |||||
| 38% male patients | |||||
| Cohen 2013267 | Audit | USA | Otolaryngology | 149,653 patients | NA |
| Dearman 2006265 | Audit | UK | Psychiatry | 1089 patients | NA |
| Elderly | |||||
| Harris 2011268 | Survey | Australia | Heart disease/hypertension | 26 practices | NR |
| Patient mean age 61.6 (19 to 90) years; 55% female | |||||
| Morgan 2007210 | Interviews | UK | Headache | 20 GPs | 50% |
| Pfeiffer 2011138 | Audit | USA | Mental health | 49,957 patients | NA |
| Mean age 55.7 years | |||||
| 93% male | |||||
| Ridsdale 2007266 | Cohort | UK | Neurology | 488 patients | NA |
| Shadd 2011263 | Audit | Canada | All specialties | 33,998 patients, 10 GPs | NA |
| Vinker 2007269 | Survey | Israel | Ophthalmology, orthopaedics, ENT, dermatology | 257 referrals | NR |
Shadd et al. 263 found that 92% of the variance in referral rates was attributable to the patient (rather than to the practice). Bertakis et al. 264 reported that after controlling for physical status, patient sex and age, more visits to the GP was associated with more specialist referrals. Dearman et al. 265 reported that patients referred to psychiatry had consulted their GP more frequently in the past year. Morgan et al. 210 found that GP readiness to refer for headache was influenced by frequent attendance. Ridsdale et al. 266 found that, for migraine patients, referred patients consulted more frequently than those not referred in the 3 months before referral (p = 0.003). Albertson et al. 157 found that patients were significantly more likely to have initiated the referral discussion when they had seen the GP previously; there was a trend for patient initiation of the referral discussion when the patient had known the GP for more than 1 year (p = 0.08) Cohen et al. 267 reported that a greater number of GP visits was related to a lower hazard ratio for referral and more days to referral. Harris et al. 268 found that previous referral was associated with the likelihood of subsequent referral.
However, in contrast to these papers reporting an association, Pfeiffer et al. 138 found that attendance at a primary care service for mental health was not a predictor of total number of specialist mental health clinic visits; and Vinker et al. 269 found that the length of time the patient was with the GP did not affect referral.
Non-intervention papers: predictors of changed practice
The second group of factors described in the non-intervention literature were elements which may moderate or mediate the outcomes described above, and act as predictors of whether of not an intervention will lead to long-term change in referral practice. Here, we examined evidence regarding the potential barriers or facilitators to the interventions changing practice at a local level and/or a health-care system level.
Moderating and mediating factors described in the literature related to the GP, the patient or the service in which the referral was taking place (Table 43). The complexity of the evidence here is further increased by many of the identified factors as operating in both directions, for example older age increases referral or older age decreases referral.
| Factor | Studies reporting association (first author and year) | Studies reporting no association (first author and year) | Strength |
|---|---|---|---|
| GP factors | |||
| Years in practice | Longer = fewer: Calnan 2007,207 Fucito 2003,162 Townsley 2003165 | Albertson 2000,270 Delva 2011,161 Johnson 2008,271 Jorgensen 2001,181 Lakha 2011,179 Vulto 2009,250 Wakefield 2012227 | iii |
| Longer = more: Elhayany 2000,167 Franks 2000,216 Ramanathan 2011,188 Harvey 2005242 | |||
| Age | Younger = more: Hugo 2000,272 Jiwa 2008,193 Pryor 2001,215 Balduf 2008274 | Albertson 2000,270 Bolanos-Carmona 2002,276 Delva 2011,161 Elhayany 2000,167 Johnson 2008,271 Jorgensen 2001,181 Lakha 2011,179 Pomeroy 2010,176 Ringard 2010,164 Rushton 2002,183 Wakefield 2012,227 Wassenaar 2007178 | iii |
| Older = more: Bowling 2006,273 Chan 2003,275 Franks 2000,216 Fucito 2003,162 O’Neill 2005,170 Swarzrauber 2002171 | |||
| Ethnicity | Ache 2011,277 Kinchen 2004,238 Navaneethan 2010278 | Lakha 2011179 | iii |
| UK-qualified | Hugo 2000272 (more), O’Neill 2005170 (fewer) | iii | |
| Sex | Females refer more: Bowling 2006,273 Calnan 2007,207 Chan 2003,275 Cooper 2001,279 Coyle 2011,280 Feeney 2007,282 Franks 2000,216 Gruen 2002,281 Hugo 2000,272 Jorgensen 2001,181 McKenna 2005225 | Albertson 2000,270 Bolanos-Carmona 2002,276 Delva 2011,161 Elhayany 2000,167 Forrest 2006,283 Johnson 2008,271 Lakha 2011,179 Montgomery 2006,180 Ringard 2010,164 Rushton 2002,183 Wakefield 2012,227 Wassenaar 2007178 | iii |
| Males perceive barriers: Hyman 2001248 | |||
| Previous experience/familiarity with service | Ahluwalia 2009,214 Allareddy 2007,232 Balduf 2008,274 Barnett 2011,222 Beel 2008,226 Berendsen 2007,237 Brien 2008,258 Chew-Graham 2008,229 Clemence 2003,228 Cornford 2004,218 Dagneaux 2012,230 Dale 2000,224 Delva 2011,161 Dodds 2004,174 Forrest 2002,223 Gandhi 2000,108 Grace 2008,292 Harlan 2009,231 Holley 2010,293 Jorgensen 2001,181 Kier 2012,294 Kinchen 2004,238 Knight 2003,177 Massey 2004,236 McKenna 2005,225 Mitchell 2012,186 Morsi 2012,200 Pomeroy 2010,176 Ringard 2010,164 Samant 2007,234 Sigel 2004,201 Taggarshe 2006,233 Thorsen 2012,239 Wakefield 2012,227 Watson 2001,195 Xu 2002235 | Chan 2003,275 Harris 2011268 | i |
| Satisfaction with specialist service | Beel 2008,226 Johnson 2011,289 Knight 2003,177 Nandy 2001,204 Pryor 2001,215 Ringard 2010,164 Sigel 2004201 | Guevara 2009290 | i |
| Emotional response | Bowling 2000,291 Espeland 2003,217 Nandy 2001204 | i | |
| Ability to judge own referral | Baker 2006192 | iii | |
| Patient factors | |||
| Ethnicity | Chen 2005,286 Greer 2011,240 Navaneethan 2010,278 Chauhan 2012284 | Johnson 2011289 | ii |
| Age | Older referred more: Bertakis 2001,264 Chan 2003,275 Chauhan 2012,284 Cohen 2013,267 Forrest 2006,283 Gruen 2002,281 Harris 2011,268 Jorgensen 2001,181 Sullivan 2005,285 Ramanathan 2011,188 Ringard 2010,164 Shadd 2011,263 Van der Weijden 2002,212 Zielinski 2008166 | Bruynincksx 2009,209 Delva 2011,161 Glozier 2007,254 Johnson 2008,271 Montgomery 2006,180 Pomeroy 2010,176 Townsley 2003,165 Vulto 2009250 | iii |
| Older referred less: Chen 2005,286 McBride 2010,287 Navaneethan 2010,278 Robinson 2010,288 Samant 2007,234 Todman 2011295 | |||
| Children more: Chan 2003275 | |||
| Urgent referral younger: Vinker 2007269 | |||
| Sex | Females referred more: Bertakis 2001,264 Chauhan 2012,284 Jorgensen 2001,181 Sullivan 2005,285 Shadd 2011,263 Zielinski 2008166 | Vinker 2007269 | iii |
| Males referred more: Bruynincksx 2009,209 Chen 2005,286 Cohen 2013,267 Forrest 2006,283 Gruen 2002,281 McBride 2010,287 Navaneethan 2010278 | |||
| Level of education | Berendsen 2010,262 Ringard 2010164 | Johnson 2008 271 | iii |
| General patient-related social/clinical factors | General: Bolanos-Carmona 2002,276 Delva 2011,161 Forrest 2006,283 Harris 2011,268 Johnson 2011,245 Knight 2003,177 Rushton 2002,183 Shadd 2011,263 Vulto 2009,250 Wakefield 2012227 | Glozier 2007254 | i |
| Socioeconomic: Baker 2006,192 Soomro 2000,296 McBride 2010,287 Mulvaney 2005,297 Soerensen 2009,298 Van der Weijden 2002212 | |||
| Clinical specialty/condition | Anthony 2010,208 Bertakis 2001,264 Chan 2003,275 Chen 2005,286 Harris 2011,268 Johnson 2011,289 Johnson 2011,245 Knight 2003,177 Little 2004,251 Musila 2011,255 Sullivan 2005,285 Rushton 2002,183 Shadd 2011263 | Calnan 2007207 | i |
| Comorbidity/complexity of condition | Anthony 2010,208 Bertakis 2001,264 Cohen 2013,267 Dearman 2006,265 Forrest 2006,283 Gruen 2002,281 Harris 2011,268 McBride 2010,287 Navaneethan 2010,278 Pomeroy 2010,176 Ridsdale 2007266 Rushton 2002,183 Zielinski 2008166 | Glozier 2007254 | i |
| Responsiveness to treatment/suitability/likely benefit | Anthony 2010,208 Baker 2006,192 Blundell 2010,189 Green 2008,220 Johnson 2011,245 Knight 2003,177 Nandy 2001,204 Philichi 2010,202 Pomeroy 2010,176 Samant 2007,234 Sigel 2004,201 Stavrou 2009,249 Steele 2012203 | Ahluwalia 2009 214 | i |
| Self-reported health | Harris 2011268 | iii | |
| Service and organisational factors | |||
| Practice location | Greater distance to specialist: Jorgensen 2001,181 Swarzrauber 2002,171 Tzaribachev 2009173 | Delva 2011,161 Gruen 2002,281 Johnson 2011,289 Love 2005,302 Pryor 2001,215 Rushton 2002183 | iii |
| Local more: Franz 2010,307 Hugo 2000,272 Johnson 2011,289 Jorgensen 2001,181 Lakha 2011,179 Todman 2011,295 Wakefield 2012227 | |||
| More deprived location: Chan 2003,275 Rosen 2007252 | |||
| Rural more: Shadd 2011,263 Tucker 2003198 | |||
| Rural less: Jiwa 2008,193 Ramanathan 2011,188 Townsley 2003,165 Zielinski 2008166 | |||
| Size of practice | Large practice more: Chauhan 2012,284 Forrest 2006,283 Harris 2011,268 Navaneethan 2010,278 Trude 2003199 | Ashworth 2002,303 Johnson 2008,271 Johnson 2011,245 Jorgensen 2001,181 Montgomery 2006,180 Rushton 2002,183 Xu 2002235 | iii |
| Single GP more: O’Neill 2005170 | |||
| Other practice characteristics | Managed care higher: Forrest 2006,283 Navaneethan 2010,278 Sullivan 2005,285 Walders 2003300 | Ownership, managed care: Burns 2000,301 Shadd 2011263 | iii |
| Private higher: Hugo 2000,272 Zielinski 2008166 | Fundholding: Ashworth 2002,303 Soomro 2000296 | ||
| Admin resources higher: Boulware 2006,304 Walders 2003300 | Having onsite service: Greenaway 2006306 | ||
| Assistants/nurses: Chung 2010305 | |||
| Financial arrangements in smaller practices only: Xu 2002235 | |||
| Care group/role in practice: Bolanos-Carmona 2002276 | |||
| Gatekeeping: Forest 2003 | |||
| Physician burden/time pressure | Anthony 2010,208 Franz 2010,307 Guevara 2009,290 Kim 2009,98 Knight 2003,177 Kvaerner 2007,168 Morgan 2007,210 Nandy 2001,204 Philichi 2010,202 Trude 2003,199 Van der Weijden 2002212 | Albertson 2000,270 Bolanos-Carmona 2002,276 Hyman 2001248 | i |
| Waiting time | Barnett 2011,222 Bowling 2006,273 Knight 2003,177 Lakha 2011,179 Ramchandiani 2002,247 Ringard 2010,164 Samant 2007,234 Stavrou 2009,249 Steele 2012,203 Taggarshe 2006,233 Todman 2011295 | i | |
| Availability of specialist | Alexander 2008,308 Anthony 2010,208 Franz 2010,307 Guevara 2009,290 Holley 2010,293 Johnson 2011,289 Johnson 2011,245 Kvaerner 2007,168, Morgan 2007,210 Ramanathan 2011,188 Trude 2003,199 Wakefield 2012227 | Forrest 2006,283 Malcolm 2008299 | i |
General practitioner factors
Moderating factors which were categorised as GP factors include the following subfactors.
The number of years a GP had been in practice was reported in 14 studies. 161,162,165,167,179,181,188,207,216,242,250,270,271,277 Of these, three studies reported that a GP who had been in practice longer referred less frequently. 162,165,207 Conversely, three studies reported that GPs who had been in practice longer referred more frequently. 167,188,216 One further study reported better-quality referral letters in those GP who had recently qualified. 242 An additional seven studies reported no association between number of years in practice and referral rate. 161,179,181,227,250,270,271 Overall, three of these studies162,227,271 were at higher risk of bias and the evidence was graded as conflicting.
General practitioner age as a factor associated with referral outcomes was reported in 21 studies. 161,162,164,167,169,171,176,179,181,183,193,215,216,227,270–276 Again, the picture was very mixed, with four studies reporting higher rates of referral for younger GPs193,215,272,274 and six studies reporting higher rates of referral for older GPs. 162,169,171,216,273,275 Thirteen studies reported no association. 161,164,167,176,178,179,181,183,227,270,271,272,276 Three of these studies162,227,271 were at higher risk of bias and the evidence was graded as conflicting.
The ethnicity of the referring GP, or the country of their medical training, was reported by four studies. 179,238,277,278 Of these, three showed an association with referral outcomes. 238,277,278 The fourth study showed no association between ethnicity/country of training and referral. 179 One study was considered to be at higher risk of bias277 and the evidence overall was graded as conflicting.
The sex of the GP was reported by 24 studies. 161,164,167,178–181,183,207,216,225,227,248,270–273,275,276,279–283 Eleven studies suggested that females refer more frequently,181,207,216,225,272,273,275,279–282 with one further study248 discussing perceived male barriers to referral. However, 12 studies reported no association between sex and referral outcomes. 161,164,167,178–180,183,227,270,271,276,283 Two were at higher risk of bias227,271 and overall the evidence was graded as conflicting.
Previous experience or familiarity with the service referring to was reported as a factor associated with increased likelihood of referral outcomes in 38 studies. 108,161,164,174,176,177,181,186,195,200,201,214,218,222–239,258,268,274,275,292–294 Of these, 36 studies reported that previous experience of familiarity with a service was associated with an increase likelihood of referral. 108,161,164,174,176,177,181,186,195,200,201,214,218,222–239,258,274,292–294 Six of these studies186,214,224,227,233,236 were considered at higher risk of bias. Only two studies reported no association268,275 and so this evidence was graded as stronger.
Previous satisfaction with specialists, reported in eight studies (all at lower risk of bias), was shown to be associated with increased likelihood of referral in all but one. 164,177,201,204,215,226,289 The final study showed no association. 290 This evidence was, therefore, also graded as stronger.
The GP’s emotional response to the patient was reported to be associated with referral in three studies of lower risk of bias. 204,217,291 A GP who had greater awareness of their own referral rate and who was able to judge their referral level as lower or higher was reported to be associated with likelihood of referral in one study. 192 As only one study reported this outcome, the evidence for this factor was graded as no evidence.
Patient factors
Moderating factors which were categorised as patient factors include the following subfactors.
Patient age as a factor associated with referral outcomes was reported in 30 studies. Twenty-five studies showed an association between age and referral rate, of which 14 studies reported higher rates of referral for older patients164,166,181,188,212,263,264,267,268,275,281,283–285 and six studies reporting higher rates of referral for younger patients. 234,278,286–288,295 Chan et al. 275 also reported that children were referred more often than adults and Vinker et al. 269 reported more urgent referral for younger patients. A further eight studies reported no association between age and referral. 161,165,176,180,209,250,254,271 Three of these studies165,271,295 were at higher risk of bias and the evidence was graded as conflicting.
An association between the ethnicity of a patient and referral was considered by five studies. 240,278,284,286,289 Of these, four showed an association with referral outcomes. 240,278,284,286 Three studies showed lower referral rates for non-white patients,278,284,286 with one further study reporting improvement in timing of referrals for white patients compared with African Americans. 240 Johnson et al. 289 reported no association between ethnicity and referral. The studies were all considered to be at lower risk of bias and the evidence overall was graded as weaker.
The sex of the patient was reported to be associated with referral rate in 15 studies. 166,181,209,263,264,267,278,281,283–287 Six studies suggested that females were referred more frequently,166,181,263,264,284,285 and seven further studies reported that males were referred more frequently. 209,267,278,281,283,286,287 One further study suggested no association between patient sex and referral. 269 All studies were at lower risk of bias and overall the evidence was graded as conflicting.
Patient level of education was reported in three studies. 164,262,271 Two studies reported an association between being more educated and being more likely to be referred. 164,262 and one study reported no association. 271 The third of these studies271 was at higher risk of bias and the evidence was graded as conflicting.
A further 17 studies reported an association with referral outcomes (including one at higher risk of bias). General patient characteristics were reported to be associated with referral in 11 studies. 161,177,181,183,227,245,250,263,268,276,283 Socioeconomic characteristics of the patient were reported to be associated with referral decisions in a further six studies,192,212,287,296–298 with lower deprivation leading to more referral. One further study reported no association between sociodemographic characteristics and urgent referral requests. 254 Overall, for this association, the evidence was graded as stronger.
Fourteen studies considered whether the clinical specialty being referred to, or the particular condition which the patient presented with, were associated with referral outcomes. 177,183,207,208,245,251,255,263,264,268,275,285,286,289 Thirteen studies reported that referral was moderated by clinical specialty177,183,208,245,251,255,263,264,268,275,285,286,289 and only one did not. 207 In this group only one study was at higher risk of bias,208 and the evidence was graded as stronger.
The presence of comorbidity or the complexity of the presenting condition was further reported as being associated with referral outcomes in 14 studies. Thirteen studies reported that referral was moderated by the complexity of the clinical presentation166,176,183,208,264–268,278,281,283,287 and only one did not. 254 This evidence was graded as stronger.
Related to this, patient responsiveness to treatment, suitability for treatment or likely benefit of referral (perceived by the referring doctor) was reported in 14 studies176,177,182,189,201–204,208,214,220,234,245,249 (four at higher risk of bias202,203,208,214), and suggested as being associated with referral outcomes in 13 studies. 176,177,182,189,201–204,208,220,234,245,249 Only one study (at higher risk of bias) showed no association. 214 The evidence for this association was graded as stronger.
Patient self-reported health was reported in one (lower risk of bias) study. Harris et al. 268 reported that patients with lower self-reported health were more likely to be referred. The evidence was, therefore, graded as no evidence.
Service factors
A number of elements were identified within the category of service factors, as follows.
The location of the GP practice (including the distance to service being referred to and whether urban or rural) was reported in 24 studies (three papers at higher risk of bias). 161,165,166,171,173,179,181,183,188,193,198,215,227,252,263,272,275,281,289,295,302,307 Of these, 18 studies165,166,171,173,179,181,188,193,198,227,252,263,272,275,289,295,307 reported an association with referral outcomes but the directions of association were very mixed. Greater distance to the specialist was reported to be associated with a reduced likelihood of referral in three studies171,173,181 and greater likelihood of referral to more local services was reported in a further seven studies. 179,181,227,272,289,295,307 A more deprived location was also associated with a reduced likelihood of referral. 252,275 Rural practices were associated with more referral in two studies198,263 but less referral in a further four studies. 165,166,188,193 Five further studies reported no association between location of GP practice or distance to the specialist service and referral outcomes. 161,183,215,281,302 The strength of this evidence was graded as inconsistent.
An association between size of the GP practice and referral outcome was reported in six studies,170,199,268,278,283,284 with no association reported by a further seven studies180,181,183,235,245,271,303 (including one study at higher risk of bias271). Of those reporting association, five reported that larger practices were associated with higher referral rates199,268,278,283,284 but one paper reported that single GP practices were associated with higher referral. 169 This evidence was graded as inconsistent.
A further 17 studies (at lower risk of bias) reported on other GP practice characteristics associated with referral outcomes, mostly relating to the fundholding or ownership of the practice. Thirteen studies reported an association166,219,235,272,276,278,283,285,300,304–307 and four studies reported no association263,296,301,303 for the following factors. Four studies reported that managed care practices were associated with higher rates of referral278,283,285,300 but two studies reported no association with referral outcomes. 263,301 Two studies reported no association between fundholding practices and rates of referral. 296,303 The other factors associated with referral outcomes were private practice associated with higher referral,166,272 greater administration resources associated with higher referral,300,304 and practice nurses or assistants associated with higher referral. 305 In addition, financial arrangements in smaller practices were associated with referral outcomes (no direction reported);235 links with a care group associated with referral outcomes (no direction reported);276 and gatekeeping role associated with referral outcomes (no direction reported). 219 One final study reported no association with referral outcomes for having an on-site service to refer to. 306 Given this complexity, the association between additional practice characteristics and referral outcomes was graded as inconsistent.
General practitioners perceived to be under greater burden or time pressure were associated with referral outcomes in 11 studies98,168,177,199,202,204,208,210,212,290,307 (including two studies at higher risk of bias202,208). Three studies reported no association. 248,270,276 This evidence was graded as stronger.
A perceived longer waiting time for the referral was associated with lower referral rates in 11 studies164,177,179,203,222,233,234,247,249,273,295 (including three at higher risk of bias203,233,295). There were no studies reporting no association and so this evidence was graded as stronger.
Greater perceived availability of the specialist was associated with more frequent referral in 12 studies168,188,199,208,210,227,290,245,289,293,307,308 (including three at higher risk of bias208,227,308). Only two studies reported no association between availability of the specialist and referral decisions. 283,299 This evidence was, therefore, graded as stronger.
Chapter 4 Summary of the evidence
What can be learned from the evidence on interventions to manage referral from primary to specialist care?
We firstly examined the overall evidence regarding referral management interventions by typology, and the overall rating of evidence of effectiveness for each group of studies which were described in the earlier sections (Figure 2).
FIGURE 2.
Summary of strength of evidence for referral management interventions. Green text, stronger evidence; blue text, weaker evidence; bold green text, conflicting evidence; standard black text, no evidence. a, Evidence in both directions (i.e. that these interventions may reduce referral but also that they may increase referral).
In the first group (practitioner education interventions), the peer-review and feedback interventions were all shown to be effective to some degree in reducing referrals, although the appropriateness of that reduction was not always considered. Although there was a higher risk of bias for one study (Cooper19), the other three studies were considered to be at lower risk of bias. 21–23 The strength of evidence for effectiveness of this type of intervention was graded as stronger. The evidence indicated that this type of intervention with individual staff had the most potential to effect change. The evidence of effectiveness for the other approaches was more mixed, perhaps owing to the variation in the training provided in terms of aim, duration and intensity. It was not possible overall to draw patterns from the data in terms of a particular type of training which may be more effective than another. Nor is it clear whether longer training programmes have greater impact than short-term or one-off interventions. However, although the evidence of effectiveness was not strong, there was some evidence that GP training could be effective in moderating referral outcomes in some contexts.
The review suggests that only in some limited situations does dissemination of guidelines have any positive effect on referral outcomes, and this is only seen over the short term. Dissemination of referral guidelines with further training, support or feedback seemed to have an inconsistent effect, with no clear patterns in terms of method of guideline development (e.g. local vs. national guidelines), or type of support provided or duration of study. There seemed to be some relationship between outcome and type of specialty; for example, low-back pain guidelines were ineffective,66,67 but those interventions that focused on cardiology (two studies56,63), endoscopy (two studies52,55), and radiology (four studies42,50,54,57) were all effective, possibly suggesting that referral guidelines may be more effective in specialties where referral criteria are clearer and more consistent between patients.
With regard to the second group of interventions (process change), there were three types where evidence of effectiveness appeared to be stronger: firstly, improving the referral information provided to specialists; secondly, enabling a community practitioner to have contact with a specialist prior to the referral; and thirdly, the introduction of electronic referral. All interventions that were focused on improving referral information were shown to be effective in improving referral-related outcomes. It was particularly apparent (given the volume of studies) that pre-referral consultation via teledermatology (where images of the skin condition were sent) was effective in moderating referral and ensuring that those referrals which were made were appropriate. Although four of the five studies here were in the specialty of dermatology,85,87,89,90 a cancer referral intervention using images was also effective,96 suggesting that the use of sending images pre referral could be used more widely than in dermatology. All interventions that reported the introduction of consultation with a specialist seemed to be effective in improving referral-related outcomes. Similarly, in nearly all cases, electronic referral systems were shown to be effective in moderating referral-related outcomes. In the ineffective study in this group,106 uptake and use of the new referral system was very low, which will have impacted on its effectiveness. These interventions seem to share a common purpose: all are designed to provide better-quality information to the specialist (either before or as part of a formal referral process).
From analysis of the studies with less clear evidence, it seemed that designated appointment slots and fast-track clinics may be effective in improving referral outcomes in some cases. The two interventions that were not effective were both focused on oncology referrals to meet the 2-week wait guidelines (although a third system for colorectal cancer referrals, in contrast, was shown to be effective). The evidence indicated that direct access to testing also might be effective in moderating referral outcomes in some cases. Of the three interventions here that did not show a clear effect, it was not possible to distinguish them from the effective interventions in terms of diagnostic test, specialty or length of the study. Decision support tools appeared to be somewhat effective in improving referral outcomes in around half of all the studies identified. We were unable to make distinctions between those studies that were effective and those that were not in terms of the content of the intervention or the specialty and/or location of the study (e.g. of three cardiology studies, one was effective109 and the other two were not114,116). Only one of the waiting list interventions was shown to have a positive effect on referral outcomes. 118 The effectiveness of this study may be due to the fact that patients had been waiting considerably longer than in the other two studies (more than 2 years).
The overall picture for interventions which aim to moderate referral outcomes by wider change at the level of the health-care system is mixed. The evidence was strongest for two types of interventions: first, community practitioners being trained to carry out additional procedures, and, second, outreach clinics. Training GPs to provide a specific procedure in the community (such as LEEP training, ECG monitoring, minor surgery or spirometry) seemed to be effective, but the GPwSI programme was shown to be more variable. Community provision by specialists in outreach clinics was generally shown to have positive effects on referral outcome measures. Two further types of intervention are highlighted in the figure as having stronger evidence of effect; however, these effects were not in a positive direction. The addition of extra nurses or counsellors in primary care did not show either clear positive effects on referral outcomes (with referral rates being no different from controls) or referral rates increasing (although it was not clear whether this increase was due to appropriate or inappropriate referrals). All four studies that evaluated the removal of gatekeeping or compared gatekeeping with an open-access system showed no (or very little) effect on referral outcome or an increase in community physician visits. 144–147 These studies highlight the potential impact on other elements of a system resulting from change to referral practice.
Other intervention types in this group had more mixed or limited evidence underpinning their use. Two interventions were identified which consisted of the return of inappropriate referrals; both showed a positive effect on referral outcomes by reducing further inappropriate referrals. 18,140 Further studies would be needed to understand whether or not this type of intervention could be effective more widely. The evidence for the effectiveness of referral management centres was very mixed, as the studies were divided in terms of whether or not they showed a positive effect on referral outcomes. In addition, of the studies showing a positive effect, two were considered to be at higher risk of bias, which may affect the reliability of their findings. The final category of interventions was patient-focused interventions. The available evidence here was limited, with the role of patients in the referral process seemingly under-researched. Of the three studies we identified, two had a significant effect on referral outcomes.
In terms of the outcomes that may result from these interventions, we found a wide range of measures of effectiveness used by studies. The outcomes divided into those earlier in the referral process, which could be considered to be at the level of the primary-care referrer, and those that were intended to have an impact at a whole-service or system-wide level. Figure 3 provides a summary of the measures and the strength of evidence underpinning interventions having an effect on that outcome. As can be seen, the areas where there was stronger evidence that interventions may have an effect were mostly in the first group: improving the provision of referral information; reducing waiting time; and increasing practitioner and patient satisfaction. There was stronger evidence of an impact on waiting times; however, there was conflicting or weaker evidence of any interventions impacting at a system-wide level on referral rates, attendance rate, cost or appropriateness.
FIGURE 3.
Reported outcomes and system impacts. Green text (uppercase and underlined), stronger evidence; blue text (uppercase), weaker evidence; bold green text (lowercase), conflicting evidence.
What are the pathways from interventions to outcomes?
We used all the elements identified and described in the preceding sections to compile an evidence-based logic model which illustrates the pathway from interventions to system-wide demand management outcomes (Figure 4). The model was constructed by listing the typology of interventions in the first column and the immediate/short-term effects that may result from interventions in the second column, and describing predictors of change (barriers or facilitators) in the third column. The final two columns detail the outcomes for demand management described in the literature at an individual level and then finally a system-level impact.
FIGURE 4.
The pathway between interventions and intended impact. Note: the arrows indicate the direction of the pathway and do not imply causality. Green text (uppercase and underlined), stronger evidence; blue text (uppercase), weaker evidence; bold green text (lowercase), conflicting evidence; no evidence, standard black text.
The model provides a detailed summary of the evidence found in the review relating to the effectiveness of interventions, the key outcomes resulting from interventions, potential obstacles to interventions effecting changed practice, and where there is stronger or weaker evidence of effect on demand management outcomes. The model highlights the complexity of the intervention change pathway, with the influence of individual-, context- and system-level factors acting as barriers and facilitators to any intervention achieving its intended impact in a particular health-care context.
The diagram also illustrates the broad scope of demand management interventions, and the range of outcomes that may result. In particular, it indicates the central role of intermediate factors in the pathway to broad system-wide impact. Currently, the majority of the intervention literature tends to use measures of system impact, whereas analysis of the pathway suggests that these intermediate elements are key important markers of change that should be evaluated in any assessment of intervention effectiveness. The model also highlights the challenges of identifying simple cause–effect relationships between individual interventions and a referral management impact.
How can evidence on interventions to manage referral from primary to specialist care be applied in a UK context, and what factors affect the applicability of international evidence in the UK?
We considered how the findings of the review were applicable and transferable to the UK NHS context. Of the 141 intervention papers we included, the largest group (n = 83) were from the UK, with an additional 19 from countries with similar systems (the Netherlands, Australia). There were a comparatively small number of North American papers (n = 24), with those countries’ very different health-care systems suggesting that the findings of the review regarding the effectiveness of interventions are applicable to the UK without special consideration. The outcomes measured and reported by studies similarly have no particular issues of applicability.
Examination of the spread of country of origin across the intervention types indicates that there is representation of UK evidence for all but two interventions approaches (none of the four gatekeeping or four payment system papers was from the UK). The following papers originated in the UK: all of the papers regarding GP peer review, 9 of 17 papers regarding GP education, all 12 papers regarding guideline dissemination, 14 of 18 papers regarding guidelines plus training/feedback, four of six papers regarding fast-track interventions, all but one of nine papers on direct access interventions, 3 of 11 papers on specialist consultation, 4 of 10 papers on electronic referral, 4 of 10 papers on decision support, one of three papers on waiting lists, six of nine papers on GP provision of specialist services, 3 of 10 papers on specialists in the community, both papers on the return of referrals, two of three papers on additional staff, all but one of six papers on referral management centre, and two of three patient-focused papers. There is UK-based evidence, therefore, across individual, process and system typologies. The review finding that, with regard to individual-level interventions, the evidence was stronger only for peer-review and feedback approaches is significant given the dominance of UK papers evaluating training or guideline provision for individual practitioners.
The non-intervention papers were more spread in terms of country of origin, with around one-third (53 of 154) originating from the UK and almost the same number (n = 52) from North America. Examination of the spread of UK studies across the factors, however, showed that there was representation in all but two categories (the one awareness of quality study was from the USA, and the one optimal timing of referral study was from the USA). This underpinning of the evidence by UK data suggests that the findings regarding these influential factors in the pathway from intervention to outcomes is applicable in the UK context.
In order to further assess how the findings of the review may be applied in the UK context, and the extent to which the diagrammatic representation of the evidence resonated with the real-life experiences of practitioners and commissioners of services, we carried out a phase of evaluation and feedback. We sought the views of key stakeholders from primary care, specialist services and NHS commissioners via a series of presentations and one-to-one meetings. In total, 44 individuals contributed to the validation stage of the work, including 15 GPs, five commissioners, seven members of the public and 17 hospital specialists. The specialties represented included infectious diseases (n = 1), gynaecology (n = 1), neurology (n = 5), palliative care (n = 2), haematology (n = 1), cardiology (n = 3), speech and language therapy (n = 1), orthopaedics (n = 1), oncology (n = 1) and respiratory medicine (n = 1).
Overall, most participants reported that they had clearly understood the logic model, with 38 respondents giving a positive first response (100% of GPs, 100% of commissioners, 76% of specialists and 71% of public respondents). Of those who felt they did not understand it (n = 6), four specialists described the model as too complex and two members of the public found it confusing.
All GPs reported that the model was a good fit with their experience of the way in which referrals are managed. In particular, they discussed how successfully the model was able to portray the ‘chaos’ of general practice. GPs also described how the model had highlighted the role of both the GPs’ and the patients’ attitudes and beliefs, the doctor–patient interaction, and especially the emotional response to the patient, which resonated very much with their experiences as a doctor. Most specialists also reported that the model was a good fit with their experience of the way referrals are managed. However, three specialists criticised the model as being a model of the literature and that this was not the same as the referral process. These participants wished to highlight that referral is often a non-linear process and also that it may be necessary to have a different model for different conditions.
The commissioners reported that the model would be useful when analysing the demand management pathway when commissioning, and for comparing what was being commissioned with what was evidence-based. GPs and commissioners also highlighted that it would be useful for people who educate GPs, and for GPs undergoing training. One GP also was positive regarding the potential of the model as a teaching aid for undergraduates. Patient and public representatives described it as useful for directing research in poorly evidenced areas, and in discussion with GP practices. However, three patient and public representatives reported that they could not see any obvious use for the model.
Many respondents mentioned factors which they thought were missing from the model but which were in fact embedded within the terms used. Although the model was able to convey a vast amount of information, some grouping and categorising had inevitably masked individual subelements. There were a small number of factors mentioned which were not included in the reviewed literature (and, therefore, not represented in the model). One specialist noted that the presence of a locum GP might affect the likelihood of referral. A patient and public representative mentioned that the influence of carers (both family members and nursing home staff) might be important in determining whether or not an elderly person was referred. A number of GPs in the focus group discussed the impact of disease burden, although it was ultimately agreed that this would fall outside the model remit, probably sitting to the left of the intervention column. A number of specialists commented on the amount of information that was sent back to the GPs after a referral, echoing comments about the referral process not always being linear. Several respondents suggested that there should be ‘feedback loops’ included in the model.
Overall, as a result of this feedback process, several changes were made to the draft model including categorising ‘conflicting evidence’ and ‘no evidence’ separately, ensuring consistency of terms throughout the model, and alterations to the arrows between the boxes.
In addition, during this validation and evaluation phase of the work, we sought to compare our findings with other international systematic reviews of evidence. A separate analysis of systematic review papers was undertaken to compare our review findings with previous reviews of the area to further explore generalisability of the current findings. Further detail on the individual systematic review papers is provided in Appendix 6.
The review of reviews focused on systematic and narrative reviews of interventions to manage referral from primary to specialist care. In total, 30 unique reviews met the inclusion criteria and were included in this review of reviews. The review papers identified both factors that influence the referral process and interventions to manage demand. Seventeen of the reviews were judged to be at lower risk of bias, with the remaining 13 at higher risk of bias, largely because of the inadequate description of the search methodologies adopted, or the primary studies included in the review.
Although the number of reviews identified and included suggests a large body of evidence of interventions to manage referrals from primary to secondary care, there was considerable duplication among the reviews identified. A number of the identified reviews drew their findings solely from previous reviews, resulting in the duplication of primary studies, and therefore of findings, across the reviews. Another consequence of this was that a relatively small number of unique primary studies included in the reviews were of relevance here.
Overall, we found that many of the other reviews had been unable to reach firm conclusions about what interventions were effective or not effective. Many reviews were unable to draw robust conclusions because of the inconsistent findings between the primary studies considered. Therefore, their findings echoed our review in concluding that interventions with similar approaches could result in different outcomes in different contexts. Other reviews also concurred with our finding that the passive dissemination of guidelines is unlikely to change referral behaviour (although there was some evidence to suggest that guidelines with education/structured checklists, and feedback and training for GPs, may improve the pre-referral management of patients). Other reviews we examined also highlighted that there was very little review-level evidence on the effectiveness of referral management centres or evidence of interventions aimed at patients and public around changing behaviour, self-care or self-appraisal, together with a lack of evidence around cost-effectiveness of interventions and their sustainability. Some of the other studies highlighted the risk of stimulating demand with interventions that provide an alternative way of accessing a service, for example outreach or attaching specialists to primary care. Many of the other reviews similar to the current work highlighted the need for a whole-systems approach to referral management.
Limitations of the study
Our systematic review and logic model synthesis has added to the existing literature by moving beyond the assumptions about outcomes, to detail fully the pathway between interventions and system-wide impact. The review was also inclusive in terms of study design and considered a large volume of literature. The potential limitations of the work, however, relate firstly to our population inclusion criteria, with the review focusing on demand management within community medical services. We recognise that other services such as community dental practices make referral to specialist services, and therefore our exclusion of studies examining these services may have omitted potentially useful data.
A key potential area of debate concerns our adopted system of quality appraisal and assessment of strength of evidence. There are many available systems for critical appraisal of studies. Our selection of a tool was based on the requirement that it would be applicable to multiple study designs; also, given the substantial volume of literature that we included, that the tool would be assess the range of potential sources of bias while not being overly long to complete. Although there were many quality appraisal systems to select from, we found considerably fewer options for examining overall strength of evidence. Although a simple tally of numbers of studies has been used by some authors, the system we selected, while we recognise its limitations, was able to consider not only quantity but also consistency in evidence and quality of evidence.
Our grouping of interventions may also be a source of limitation in interpretation of the findings. Although the typology was able to distinguish different forms of content successfully, it should be recognised that there may be overlap between categories and, although we identified the core constituents, some interventions may have included several elements.
In terms of evaluation of outcomes, at times it was difficult to establish which outcome measures should be considered as positive, for example where interventions encourage referrals irrespective of their appropriateness, or where quicker referral processes are created and are, therefore, positive; however, this has a cost implication for the system. The concept of appropriateness as an outcome was particularly challenging to interpret as views may differ between community and specialist practitioners, and also, for example, patient satisfaction and/or mental well-being may be increased by a referral; however, the referral may be considered unnecessary.
Although the evidence identified here is international in nature and some of it originates from countries with different health-care systems and processes from the UK, as we have outlined in other sections, the vast majority of studies have relevance in the UK within a NHS setting. It is likely that differences between specialties, UK demographic variation and the impact of individual patients and practitioners will have a stronger impact on the effectiveness of the interventions in a given location than will their country of origin.
We chose to use logic model methods to synthesise the review findings as they have been suggested as useful explanatory tools. The process of evaluation that we undertook following completion of the synthesis indicated that the method was able to provide a detailed illustration of multiple elements of interventions and outcomes which was viewed positively by the majority of stakeholders. The model was able to summarise a complex set of data in a single diagram; however, for some this complexity was viewed as being confusing and overcomplicated. Some practitioners pointed out that this ‘messiness’ represented the reality of endeavouring to manage referral demand, and, although the method may have limitations, it perhaps serves to confirm the challenges inherent in designing and implementing effective complex interventions.
Implications for health care
Our systematic review of the literature and logic model synthesis suggests that no one level of intervention (GP training, process change, system change or patient intervention) stands out as being much more successful than any other in producing successful referral outcomes. However, some groups of interventions may have greater potential for development, given the existing evidence that they can be effective in specific contexts.
The findings suggest that, although individual-level interventions may be popular, the stronger evidence relates to only the effectiveness of GP peer-review and feedback interventions, with evidence underpinning the implementation of formal training and referral guidelines less clear. Providing training (or reinforcement) of guidelines may aid their use.
Process change interventions appear to be most effective when the changes result in the specialist being provided with more or better-quality information about the patient – whether that is provided electronically (electronic referral) as part of the referral process, or via specialist consultation prior to the formal referral being made. The evidence is less strong for the effectiveness of process interventions which do not result in earlier interaction with a specialist (designated appointment slots/fast-track clinics, direct access to screening, and decision support tools).
With regard to system changes, the community provision of specialist services by GPs (having been previously trained by specialists), outreach or community provision by specialists, and the return of inappropriate referrals, all engage the specialist and show the stronger evidence of effect on referral outcomes. However, the evidence suggests that the addition of other primary care staff (e.g. nurses, counsellors) into a GP practice can have a negative effect on referral outcomes including referral rate and appropriateness of referral (although the amount of evidence here was limited).
The evidence for gatekeeping systems overall was very inconsistent and appeared to suggest that adding or removing a gatekeeping system had no positive impact on referral (although there were possibly small negative effects). The evidence here was weaker and originated from countries with different health-care systems from that of the UK.
Despite additional targeted searches, we found a significant lack of an evidence base to support referral management centres or other large triage systems. We were also surprised to find an almost complete lack of patient-focused interventions. This is particularly relevant given the evidence highlighting the impact that the doctor–patient relationship, and the role of patient factors, may have on the referral decision.
A key contribution of this review has been the highlighting of elements that act as mediators and moderators to intervention outcomes. We found a considerable volume of literature which endeavoured to link particular practitioner demographics to referral patterns; however, no clear associations were apparent. Instead, the factors that appeared to be important related to practitioner views and knowledge of the service which was being referred to (previous experience or familiarity with service, and satisfaction with service), and their emotional response to the patient. The importance of understanding that the GP is an individual and that each referral decision is unique was voiced strongly in feedback on the review findings during our validation work. This is further underpinned by evidence regarding the influence of individual patient factors relating to clinical condition (clinical specialty/condition, comorbidity/complexity of condition, and suitability for referral/likely benefit from referral). These factors were important in predicting whether or not referrals would be made.
In terms of service factors, the particular characteristics of the GP practice (location, size and ownership) seemed less important than factors associated with the service referred to (waiting time and availability of specialist). The burden imposed on GPs’ time by the service they were working in was also important in influencing the referral process. These local factors will influence the success and applicability of any interventions.
In interpreting the findings of this review it is important to recognise that a number of the interventions we have identified are condition specific (such as sending photographs with dermatology referrals) and that the same intervention may not be transferable across different conditions or diseases. What may be less clear but equally important is that the same also applies to the moderating and mediating factors. For example, the effect of one patient demographic factor such as age may be a strong predictor of referral in certain conditions.
This review has highlighted the value of overall consideration of the entire referral system rather than examined individual components. To tackle demand management of primary care services, the focus cannot be on primary care alone – a whole-systems approach is needed as the introduction of interventions in primary care is often just the starting point of the referral process. Patton309 has emphasised a ‘systems perspective as becoming increasingly important in dealing with and understanding real-world complexities’ (p. 120). With the introduction of interventions in primary care there are likely to be implications for secondary care. Furthermore, in a climate of ‘payment by results’, any intervention that reduces secondary care activity means a loss of income to secondary care and the implications of this would require consideration. When considering potential interventions to influence referral management, too little regard may be given to the whole referral system, including ensuring that people are referred to the most appropriate destination, that referrals are timely, that all necessary pre-referral tests have been done and that referral letters include all pertinent information. Authors such as Anderson310 argue that health-care organisations should be seen as ‘unpredictable and disorderly’, seen as ‘complex, adaptive systems’ rather than ‘a well-oiled machine’. Complexity theory suggests that it is the interaction and interdependency among elements as well as the unity as a whole that needs to be studied, with the key to understanding a health-care system being ‘patterns of relationships and interactions’. 311
In addition, many of the most complex interventions require culture change as well as a change in individual attitudes. However, often the interventions and strategies have been implemented without due regard to the challenges of changing culture or engaging individuals. This review and model detailing the pathway of change should help to emphasise the role of individuals in the change process.
Recommendations for research
-
More research is needed to develop and evaluate interventions that acknowledge the role of the patient in the referral decision. We found a lack of interventions aimed at providing health information or education to patients or to moderate their concerns or satisfaction regarding a referral. This is an important aspect to tackle, as our model showed that both patient pressure and their relationship with their GP can affect whether or not a referral occurs.
-
Research is also required to better understand the relationship between GP knowledge and GP attitudes and beliefs in terms of how an intervention is framed and how responsive practitioners are to change. The review findings indicate that attitudes and beliefs of the patient and the GP, as well as the doctor–patient interaction, are potentially important mechanisms of change which interventions should seek to impact and should measure in outcome evaluations. It is suggested that interventions focusing on these have a greater potential for impacting on referral demand outcomes. This suggests that interventions which act only to change knowledge may not be as effective as those acting on attitudes and beliefs. This is particularly important for future intervention study design as knowledge is relatively easy to measure and therefore more likely to be included, whereas change in attitudes and beliefs may be more challenging to evaluate.
-
This work highlights that intermediate outcomes such as the content of the referral provided to the specialist are important in the referral pathway. It is only by recognising and evaluating these individual outcomes that the intervention change pathway can be understood. It is recommended that researchers include measures of these intermediate outcomes in their evaluation of intervention effectiveness in order to determine where blocks or facilitators to system-wide impact may be occurring.
Acknowledgements
We would like to acknowledge the members of our steering group for their invaluable contribution to this research: Dr Martin McShane, NHS England; Professor Danuta Kasprzyk (Department of Global Health, University of Washington); Professor Helena Britt (Family Medicine Research Centre, University of Sydney); Ellen Nolte (RAND Corporation Europe/international); Jon Karnon (School of Population Health and Clinical Practice, University of Adelaide); Christine Allmark [patient and public involvement (PPI)]; and Brian Hodges (PPI). Thanks also to all of those who contributed to the validation stage of the work.
Contributions of authors
Dr Lindsay Blank (Research Fellow) was the principal investigator and study manager for this project. She acted as lead reviewer and drafted the final report.
Dr Susan Baxter (Research Fellow) led the logic model development aspect of the work, contributed as a reviewer to all aspects and revised the report following reviewer feedback.
Helen Buckley Woods (Information Specialist) took the lead in the searching aspects of the work.
Professor Elizabeth Goyder (Professor of Public Health) provided oversight and guidance to the project.
Dr Andrew Lee (Senior Clinical University Teacher in Public Health/GP) provided a strategic overview to the project from a NHS perspective and provided expertise in translation of the findings to the NHS context.
Professor Nick Payne (Honorary Professor of Public Health) provided oversight and guidance to the project in terms of review methods and the NHS context.
Melanie Rimmer (Research Assistant) contributed to data collecting during the validation stage and proofreading the report.
All authors have read and approved the final report.
Publications
Baxter SK, Blank L, Woods HB, Payne N, Rimmer M, Goyder E. Using logic model methods in systematic review synthesis: describing complex pathways in referral management interventions. BMC Med Res Methodol 2014;14:62.
Blank L, Baxter S, Woods BH, Goyder E, Lee A, Payne N. Referral interventions from primary to specialist care: a systematic review of international evidence. Br J Gen Pract 2014;64:e765–74.
Disclaimers
This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health.
References
- Akbari A, Mayhew A, Al-Alawi MA, Grimshaw J, Winkens R, Glidewell E, et al. Interventions to improve outpatient referrals from primary care to secondary care. Cochrane Database Syst Rev 2008;4. http://dx.doi.org/10.1002/14651858.CD005471.pub2.
- Faulkner A, Mills N, Bainton D, Baxter K, Kinnersley P, Peters TJ, et al. A systematic review of the effect of primary care-based service innovations on quality and patterns of referral to specialist secondary care. Br J Gen Pract 2003;53:878-84.
- Dunst CJ, Gorman E. Practices for increasing referrals from primary care physicians. Cornerstones 2006;2:1-10.
- Grimshaw JM, Winkens RA, Shirran L, Cunningham C, Mayhew A, Thomas R, et al. Interventions to improve outpatient referrals from primary care to secondary care. Cochrane Database Syst Rev 2005;3.
- Imison C, Naylor C. Referral Management – Lessons for Success 2010. www.kingsfund.org.uk/sites/files/kf/Referral-management-lessons-for-success-Candace-Imison-Chris-Naylor-Kings-Fund-August2010.pdf (accessed 24 January 2014).
- Rees K, Bennett P, West R, Davey Smith G, Ebrahim S. Psychological interventions for coronary heart disease. Cochrane Database Syst Rev 2004;2. http://dx.doi.org/10.1002/14651858.CD002902.pub2.
- Weiss C, Connell J, Kubisch A, Schorr L, Weiss C. New Approaches to Evaluating Community Initiatives Vol 1: Concepts, Methods and Contexts. Washington, DC: Aspen Institute; 1995.
- Weiss C. Theory-based evaluation: past, present and future. New Dir Eval 2007;76:68-81.
- Dyson A, Todd L. Dealing with complexity: theory of change evaluation and the full service extended schools initiative. Int J Res Method Educ 2010;33:119-34. http://dx.doi.org/10.1080/1743727X.2010.484606.
- Blamey A, Mackenzie M. Theories of change and realistic evaluation: peas in a pod or apples and oranges. Evaluation 2007;13:439-55. http://dx.doi.org/10.1177/1356389007082129.
- Connell J, Kubisch A, Fullbright-Anderson K, Kubisch A, Connell J. New Approaches to Evaluating Community Initiatives, Vol 2: Theory, Measurement and Analysis. Queenstown: The Aspen Institute; 1998.
- EPPI-Centre Methods for Conducting Systematic Reviews. 2010.
- Grant MJ, Brettle A, Long AF. Developing a Review Question: A Spiral Approach to Literature Searching. Poster Presentation n.d.
- Baxter S, Killoran A, Kelly M, Goyder E. Synthesising diverse evidence: the use of primary qualitative data analysis methods and logic models in public health reviews. Public Health 2010;124:99-106. http://dx.doi.org/10.1016/j.puhe.2010.01.002.
- Allmark P, Baxter S, Goyder E, Guillaume L, Crofton-Martin G. Assessing the health benefits of advice services: using research evidence and logic model methods to explore complex pathways. Health Soc Care Comm 2013;21:59-68. http://dx.doi.org/10.1111/j.1365-2524.2012.01087.x.
- Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 2008;8. http://dx.doi.org/10.1186/1471-2288-8-45.
- Hoogendoorn WE, van Poppel MNM, Bongers PM, Koes BW, Bouter LM. Physical load during work and leisure time as risk factors for back pain. Scand J Work Environ Health 1999;25:387-403. http://dx.doi.org/10.5271/sjweh.451.
- Wylie KR, Davies-South D. Returning a referral for erectile dysfunction to the referrer by two different routes. BJU Int 2001;87:846-8. http://dx.doi.org/10.1046/j.1464-410x.2001.02188.x.
- Cooper R. How peer review reduced GP referrals by 25% in two months. Pulse 2012;72.
- Abel G, Thompson L. General practitioners, specialists and surveillance guidelines: Interpreting the socio-clinical context of decision-making. Health Risk Soc 2011;13:547-59. http://dx.doi.org/10.1080/13698575.2011.615826.
- Evans E. The Torfaen referral evaluation project. Qual Prim Care 2009;17:423-9.
- Evans E, Aiking H, Edwards A. Reducing variation in general practitioner referral rates through clinical engagement and peer review of referrals: a service improvement project. Qual Prim Care 2011;19:263-72.
- Jiwa M, Walters S, Mathers N. Referral letters to colorectal surgeons: the impact of peer-mediated feedback. Br J Gen Pract 2004;54:123-6.
- Hilty DM, Yellowlees PM, Nesbitt TS. Evolution of telepsychiatry to rural sites: changes over time in types of referral and in primary care providers’ knowledge, skills and satisfaction. Gen Hosp Psychiatry 2006;28:367-73. http://dx.doi.org/10.1016/j.genhosppsych.2006.05.009.
- Lam TP, Chow RWM, Lam KF, Lennox IM, Chan FHW, Tsoi SLT. Evaluation of the learning outcomes of a year-long postgraduate training course in community geriatrics for primary care doctors. Arch Gerontol Geriatr 2011;52:350-6. http://dx.doi.org/10.1016/j.archger.2010.05.019.
- Rowlands G, Sims J, Kerry S, Keene D, Hilton S. Within-practice educational meetings and GP referrals to secondary care: an aid to reflection and review of clinical practice. Educ 2003;14:449-62.
- Ramsay CR, Eccles M, Grimshaw JM, Steen N. Assessing the long-term effect of educational reminder messages on primary care radiology referrals. Clin Radiol 2003;58:319-21. http://dx.doi.org/10.1016/S0009-9260(02)00524-X.
- Shariff ZR, Roshan A, Williams AM, Platt AJ. 2-week wait referrals in suspected skin cancer: does an instructional module for general practitioners improve diagnostic accuracy?. Surgeon 2010;8:247-51. http://dx.doi.org/10.1016/j.surge.2010.03.004.
- Kousgaard KR, Nielsen JD, Olesen F, Jensen AB. General practitioner assessment of structured oncological information accompanying newly referred cancer patients. Scand J Prim Health Care 2003;21:110-14. http://dx.doi.org/10.1080/02813430310001725.
- Bennett K, Haggard M, Churchill R, Wood S. Improving referrals for glue ear from primary care: are multiple interventions better than one alone?. J Health Serv Res Policy 2001;6:139-44. http://dx.doi.org/10.1258/1355819011927387.
- Donohoe ME, Fletton JA, Hook A, Powell R, Robinson I, Stead JW, et al. Improving foot care for people with diabetes mellitus – a randomised controlled trial of an integrated care approach. Diabet Med 2000;17:581-7. http://dx.doi.org/10.1046/j.1464-5491.2000.00336.x.
- Watson E, Clements A, Yudkin P, Rose P, Bukach C, Mackay J, et al. Evaluation of the impact of two educational interventions on GP management of familial breast/ovarian cancer cases: a cluster randomised controlled trial. Br J Gen Pract 2001;51:817-21.
- Adams SGP, Pitts J, Wynne J, Yawn BP, Diamond EJ, Lee S, et al. Effect of a primary care continuing education program on clinical practice of chronic obstructive pulmonary disease: translating theory into practice. Mayo Clin Proc 2012;87:862-70. http://dx.doi.org/10.1016/j.mayocp.2012.02.028.
- Hands S, Holbrook J. Are one-to-one outpatient teaching sessions a useful way for GPs to undertake their continuing education?. Educ 2001;12:392-400.
- Suris X, Cerda D, Ortiz-Santamaria V, Ponce A, Simon JL, Calvo E, et al. A rheumatology consultancy program with general practitioners in Catalonia, Spain. J Rheumatol 2007;34:1328-31.
- Wolters R, Wensing M, Klomp M, Lagro-Jansen T, van Weel C, Grol R. Effects of distance learning on clinical management of LUTS in primary care: a randomised trial. Patient Educ Couns 2005;59:212-18. http://dx.doi.org/10.1016/j.pec.2004.11.009.
- Bhalla RK, Unwin D, Jones TM, Lesser T. Does clinical assistant experience in ENT influence general practitioner referral rates to hospital?. J Laryngol Otol 2002;116:586-8. http://dx.doi.org/10.1258/00222150260171542.
- Ellard R, Gulati A, Hubbard V, Bull R, Pozo-Garcia L, Goldsmith P. The impact of physician education on primary care referrals in dermatology. Br J Dermatol 2012;167.
- Lester H, Birchwood M, Freemantle N, Michail M, Tait L. REDIRECT: cluster randomised controlled trial of GP training in first-episode psychosis. Br J Gen Pract 2009;59:e183-90. http://dx.doi.org/10.3399/bjgp09X420851.
- Emmerson B, Frost A, Powell J, Ward W, Barnes M, Frank RE-MA, et al. Evaluating a GP consultative psychiatric service in an Australian metropolitan hospital district. Australas 2003;11:195-8. http://dx.doi.org/10.1046/j.1039-8562.2003.00547.x.
- Malik A, Wigney L, Murray S, Gerrand CH. The effectiveness of ‘two-week’ referrals for suspected bone and soft tissue sarcoma. Sarcoma 2007;11:1-3. http://dx.doi.org/10.1155/2007/23870.
- Twomey P. Making the best use of a radiology department: an example of implementation of a referral guideline within a primary care organisation. Qual Prim Care 2003;11:53-9.
- Cusack CM, Buckley CC. Efficacy of issuing guidelines on acne management to general practitioners. Br J Dermatol 2005;152:1392-3. http://dx.doi.org/10.1111/j.1365-2133.2005.06641.x.
- Idiculla JM, Perros P, Frier BM. Do diabetes guidelines influence the content of referral letters by general practitioners to a diabetes specialist clinic?. Health Bull (Edinb) 2000;58:322-7.
- Lucassen AW, Watson E, Harcourt J, Rose P, O’Grady J. Guidelines for referral to a regional genetics service: GPs respond by referring more appropriate cases. Fam Pract 2001;18:135-40. http://dx.doi.org/10.1093/fampra/18.2.135.
- Potter S, Govindarajulu S, Shere M, Braddon F, Curran G, Greenwood R, et al. Referral patterns, cancer diagnoses, and waiting times after introduction of two week wait rule for breast cancer: prospective cohort study. BMJ 2007;335:288-90. http://dx.doi.org/10.1136/bmj.39258.688553.55.
- Imkampe A, Bendall S, Chianakwalam C. Two-week rule: has prioritisation of breast referrals by general practitioners improved?. Breast 2006;15:654-8. http://dx.doi.org/10.1016/j.breast.2006.02.002.
- Fearn S, Larner J. Have Quality and Outcomes Framework Depression Indicators changed referrals from primary care to a dedicated memory clinic?. Ment Health Fam Med 2009;6:129-32.
- Hill VAW, Wong E, Hart CJ. General practitioner referral guidelines for dermatology: do they improve the quality of referrals?. Clin Exp Dermatol 2000;25:371-6. http://dx.doi.org/10.1046/j.1365-2230.2000.00665.x.
- Matowe L, Ramsay CR, Grimshaw JM, Gilbert FJ, Macleod MJ, Needham G, et al. Effects of mailed dissemination of the Royal College of Radiologists’ guidelines on general practitioner referrals for radiography: a time series analysis. Clin Radiol 2002;57:575-8. http://dx.doi.org/10.1053/crad.2001.0894.
- Melia J, Coulson P, Coleman D, Moss S. Urological referral of asymptomatic men in general practice in England. Br J Cancer 2008;98:1176-81. http://dx.doi.org/10.1038/sj.bjc.6604291.
- West YRK, Kendrick BL, Williamson DM. Evaluation of the impact of orthopaedic guidelines on referrals from primary care to a specialist department. Qual Prim Care 2007;15:27-31.
- Banait G, Sibbald B, Thompson D, Summerton C, Hann M, Talbot S. Modifying dyspepsia management in primary care: a cluster randomised controlled trial of educational outreach compared with passive guideline dissemination. Br J Gen Pract 2003;53:94-100.
- Eccles M, Steen N, Grimshaw J, Thomas L, McNamee P, Soutter J, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet 2001;357:1406-9. http://dx.doi.org/10.1016/S0140-6736(00)04564-5.
- Elwyn G, Owen D, Roberts L, Wareham K, Duane P, Allison M, et al. Influencing referral practice using feedback of adherence to NICE guidelines: a quality improvement report for dyspepsia. Qual Saf Health Care 2007;16:67-70. http://dx.doi.org/10.1136/qshc.2006.019992.
- Wright J, Harrison S, McGeorge M, Patterson C, Russell I, Russell D, et al. Improving the management and referral of patients with transient ischaemic attacks: a change strategy for a health community. Qual Saf Health Care 2006;15:9-12. http://dx.doi.org/10.1136/qshc.2005.014704.
- Glaves J. The use of radiological guidelines to achieve a sustained reduction in the number of radiographic examinations of the cervical spine, lumbar spine and knees performed for GPs. Clin Radiol 2005;60:914-20. http://dx.doi.org/10.1016/j.crad.2005.02.013.
- Griffiths C, Taylor H, Collins SI, Hobson JE, Collier PA, Chalmers RJ, et al. The impact of psoriasis guidelines on appropriateness of referral from primary to secondary care: a randomised controlled trial. British Association of Dermatologists 86th Annual Meeting, abstract O-2. Br J Dermatol 2006;155. http://dx.doi.org/10.1111/j.1365-2133.2006.07343.x.
- Kerry S, Oakeshott P, Dundas D, Williams J. Influence of postal distribution of the Royal College of Radiologists’ guidelines, together with feedback on radiological referral rates, on X-ray referrals from general practice: a randomised controlled trial. Fam Pract 2000;17:46-52. http://dx.doi.org/10.1093/fampra/17.1.46.
- Robling MR, Houston HL, Kinnersley P, Hourihan MD, Cohen DR, Hale J, et al. General practitioners’ use of magnetic resonance imaging: an open randomised trial comparing telephone and written requests and an open randomised controlled trial of different methods of local guideline dissemination. Clin Radiol 2002;57:402-7. http://dx.doi.org/10.1053/crad.2001.0864.
- White T, Marriott S. Using evidence-based dissemination and implementation strategies to improve routine communication between general practitioners and community mental health teams. Psychiatr Bull 2004;28:8-11. http://dx.doi.org/10.1192/pb.28.1.8.
- Julian S, Naftalin NJ, Clark M, Szczepura A, Rashid A, Baker R, et al. An integrated care pathway for menorrhagia across the primary–secondary interface: patients’ experience, clinical outcomes, and service utilisation. Qual Saf Health Care 2007;16. http://dx.doi.org/10.1136/qshc.2005.016782.
- Walkowski K, Peel C, Sandy L. Effect of academic detailing on primary care referral patterns to high performing cardiac specialists. J Gen Intern Med 2007;22:196-7.
- Morrison J, Carroll L, Twaddle S, Cameron I, Grimshaw J, Leyland A, et al. Pragmatic randomised controlled trial to evaluate guidelines for the management of infertility across the primary care–secondary care interface. BMJ 2001;322:1282-4. http://dx.doi.org/10.1136/bmj.322.7297.1282.
- Wilson BJ, Torrance N, Mollison J, Watson MS, Douglas A, Miedzybrodzka Z, et al. Cluster randomised trial of a multifaceted primary care decision-support intervention for inherited breast cancer risk. Fam Pract 2006;23:537-44. http://dx.doi.org/10.1093/fampra/cml026.
- Dey P, Simpson CW, Collins SI, Hodgson G, Dowrick CF, Simison AJ, et al. Implementation of RCGP guidelines for acute low back pain: a cluster randomised controlled trial. Br J Gen Pract 2004;54:33-7.
- Engers AJ, Wensing M, van Tulder MW, Timmermans A, Oostendorp RAB, Koes BW, et al. Implementation of the Dutch low back pain guideline for general practitioners: a cluster randomised controlled trial. Spine 2005;30:595-600. http://dx.doi.org/10.1097/01.brs.0000155406.79479.3a.
- Jiwa MS, Skinner P, Coker AO, Shaw L, Campbell MJ, Thompson J. Implementing referral guidelines: lessons from a negative outcome cluster randomised factorial trial in general practice. BMC Fam Pract 2006;7. http://dx.doi.org/10.1186/1471-2296-7-65.
- Spatafora S, Canepa G, Migliari R, Rotondo S, Mandressi A, Puppo P, et al. Effects of a shared protocol between urologists and general practitioners on referral patterns and initial diagnostic management of men with lower urinary tract symptoms in Italy: the Prostate Destination study. BJU Int 2005;95:563-70. http://dx.doi.org/10.1111/j.1464-410X.2005.05340.x.
- Bridgman S, Li X, Mackenzie G, Dawes P. Does the North Staffordshire slot system control demand of orthopaedic referrals from primary care? A population-based survey in general practice. Br J Gen Pract 2005;55:704-9.
- Khan A, Smith DA, Whittaker J, Williams A, Khan D, Harvey JE, et al. Effectiveness of direct GP referrals to hospital specialist respiratory teams in avoiding acute admissions. Thorax 2008;63.
- Sved-Williams A, Poulton JE. Primary care mental health consultation-liaison: a connecting system for private psychiatrists and general practitioners. Australas Psychiatry 2010;18:125-9. http://dx.doi.org/10.3109/10398560903469783.
- Hemingway DM, Jameson J, Kelly MJ. Leicester Colorectal Specialist Interest Group Project Steering Committee. Straight to test: introduction of a city-wide protocol driven investigation of suspected colorectal cancer. Colorectal Dis 2006;8:289-95. http://dx.doi.org/10.1111/j.1463-1318.2005.00935.x.
- McNally OM, Wareham V, Flemming DJ, Cruickshank ME, Parkin DE, McNally OM, et al. The impact of the introduction of a fast track clinic on ovarian cancer referral patterns. Eur J Cancer Care (Engl) 2003;12:327-30. http://dx.doi.org/10.1046/j.1365-2354.2003.00419.x.
- Prades J, Espinas JA, Font R, Argimon JM, Borras JM. Implementing a cancer fast-track programme between primary and specialised care in Catalonia (Spain): a mixed methods study. Br J Cancer 2011;105:753-9. http://dx.doi.org/10.1038/bjc.2011.308.
- DAMASK (Direct Access to Magnetic Resonance Imaging: Assessment for Suspect Knees) Trial Team . Cost-effectiveness of magnetic resonance imaging of the knee for patients presenting in primary care. Br J Gen Pract 2008;58:e10-16. http://dx.doi.org/10.3399/bjgp08X342660.
- Shaw IS, Valori RM, Charlett A, McNulty CA. Limited impact on endoscopy demand from a primary care based ‘test and treat’ dyspepsia management strategy: the results of a randomised controlled trial. Br J Gen Pract 2006;56:369-74.
- Simpson GC, Forbes K, Teasdale E, Tyagi A, Santosh C. Impact of GP direct-access computerised tomography for the investigation of chronic daily headache. Br J Gen Pract 2010;60:897-901. http://dx.doi.org/10.3399/bjgp10X544069.
- Thomas RE, Grimshaw JM, Mollison J, McClinton S, McIntosh E, Deans H, et al. Cluster randomised trial of a guideline-based open access urological investigation service. Fam Pract 2003;20:646-54. http://dx.doi.org/10.1093/fampra/cmg605.
- Thomas R, Cook A, Main G, Taylor T, Galizia CE, Swingler R, et al. Primary care access to computed tomography for chronic headache. Br J Gen Pract 2010;60:426-30. http://dx.doi.org/10.3399/bjgp10X502146.
- Wong BC, Chan CK, Wong KW, Wong WM, Yuen MF, Lai KC, et al. Evaluation of a new referral system for the management of dyspepsia in Hong Kong: role of open-access upper endoscopy. J Gastroenterol Hepatol 2000;15:1251-6. http://dx.doi.org/10.1046/j.1440-1746.2000.2353.x.
- Dhillon V, Creiger J, Hannan J, Hurst N, Nuki G. The effect of DXA scanning on clinical decision making by general practitioners: a randomised, prospective trial of direct access versus referral to a hospital consultant. Osteoporos Int 2003;14:326-33. http://dx.doi.org/10.1007/s00198-002-1371-2.
- Eley KA, Fitzgerald JE. Direct general practitioner referrals to audiology for the provision of hearing aids: a single centre review. Qual Prim Care 2010;18:201-6.
- Gough-Palmer AL, Burnett C, Gedroyc WM. Open access to MRI for general practitioners: 12 years’ experience at one institution – a retrospective analysis. Br J Radiol 2009;82:687-90. http://dx.doi.org/10.1259/bjr/88267089.
- Leggett P, Gilliland AE, Cupples ME, McGlade K, Corbett R, Stevenson M, et al. A randomised controlled trial using instant photography to diagnose and manage dermatology referrals. Fam Pract 2004;21:54-6. http://dx.doi.org/10.1093/fampra/cmh112.
- Eminovic N, de Keizer NF, Wyatt JC, ter Riet G, Peek N, van Weert HC, et al. Teledermatologic consultation and reduction in referrals to dermatologists: a cluster randomised controlled trial. Arch Dermatol 2009;145:558-64. http://dx.doi.org/10.1001/archdermatol.2009.44.
- Whited JD, Hall RP, Foy ME, Marbrey LE, Grambow SC, Dudley TK, et al. Teledermatology’s impact on time to intervention among referrals to a dermatology consult service. Telemed J E Health 2002;8:313-21. http://dx.doi.org/10.1089/15305620260353207.
- Whited JD, Hall RP, Foy ME, Marbrey LE, Grambow SC, Dudley TK, et al. Patient and clinician satisfaction with a store-and-forward teledermatology consult system. Telemed J E Health 2004;10:422-31. http://dx.doi.org/10.1089/tmj.2004.10.422.
- McKoy KC, DiGregorio S, Stira L. Asynchronous teledermatology in an urban primary care practice. Telemed J E Health 2004;10:S70-80. http://dx.doi.org/10.1089/1530562042632001.
- Knol A, van den Akker TW, Damstra RJ, de Haan J. Teledermatology reduces the number of patient referrals to a dermatologist. J Telemed Telecare 2006;12:75-8. http://dx.doi.org/10.1258/135763306776084365.
- Hockey AD, Wootton R, Casey T. Trial of low-cost teledermatology in primary care. J Telemed Telecare 2004;10:44-7. http://dx.doi.org/10.1258/1357633042614221.
- Nielsen JDP, Palshof T, Mainz J, Jensen AB, Olesen F. Randomised controlled trial of a shared care programme for newly referred cancer patients: bridging the gap between general practice and hospital. Qual Saf Health Care 2003;12:263-72. http://dx.doi.org/10.1136/qhc.12.4.263.
- Harrington JT, Dopf CA, Chalgren CS. Implementing guidelines for interdisciplinary care of low back pain: a critical role for pre-appointment management of specialty referrals. Jt Comm J Qual Improv 2001;27:651-63.
- Wallace P, Barber J, Clayton W, Currell R, Fleming K, Garner P, et al. Virtual outreach: a randomised controlled trial and economic evaluation of joint teleconferenced medical consultations. Health Technol Assess 2004;8. http://dx.doi.org/10.3310/hta8500.
- Jaatinen T, Aarnio P, Remes J, Hannukainen J, Koymari S. Teleconsultation as a replacement for referral to an outpatient clinic. J Telemed Telecare 2002;8:102-6. http://dx.doi.org/10.1258/1357633021937550.
- Tadros A, Murdoch R, Stevenson JH. Digital image referral for suspected skin malignancy – a pilot study of 300 patients. J Plast Reconstr Aesthet Surg 2009;62:1048-53. http://dx.doi.org/10.1016/j.bjps.2008.02.005.
- Nicholson C, Jackson CL, Wright B, Mainwaring P, Holliday D, Lankowski A, et al. Online referral and OPD booking from the GP desktop. Aust Health Rev 2006;30:397-404. http://dx.doi.org/10.1071/AH060397.
- Kim Y, Chen AH, Keith E, Yee HF, Kushel MB. Not perfect, but better: primary care providers’ experiences with electronic referrals in a safety net health system. J Gen Intern Med 2009;24:614-19. http://dx.doi.org/10.1007/s11606-009-0955-3.
- Dennison J, Eisen S, Towers M, Ingham CC. An effective electronic surgical referral system. Ann R Coll Surg Engl 2006;88:554-6. http://dx.doi.org/10.1308/003588406X130642.
- Chen AH, Kushel MB, Grumbach K, Yee HF. A safety-net system gains efficiencies through ‘eReferrals’ to specialists. Health Affairs 2010;29:969-71. http://dx.doi.org/10.1377/hlthaff.2010.0027.
- Gandhi TK, Sittig DF, Franklin M, Sussman AJ, Fairchild DG, Bates DW, et al. Communication breakdown in the outpatient referral process. J Gen Intern Med 2000;15:626-31. http://dx.doi.org/10.1046/j.1525-1497.2000.91119.x.
- Kim-Hwang JE, Chen AH, Bell DS, Guzman D, Yee HF, Kushel MB. Evaluating electronic referrals for specialty care at a public hospital. J Gen Intern Med 2010;25:1123-8. http://dx.doi.org/10.1007/s11606-010-1402-1.
- Stoves J, Connolly J, Cheung C. Electronic consultation as an alternative to hospital referral for patients with chronic kidney disease: a novel application for networked electronic health records to improve the accessibility and efficiency of healthcare. BMJ Quality and Safety 2010;19:1-4. http://dx.doi.org/10.1136/qshc.2009.038984.
- Patterson V, Humphreys J, Chua R. E-mail triage of new neurological outpatient referrals from general practice. J Neurol Neurosurg Psychiatry 2004;75:617-20. http://dx.doi.org/10.1136/jnnp.2003.024489.
- Jiwa MD, Dhaliwal S. Referral Writer: preliminary evidence for the value of comprehensive referral letters. Qual Prim Care 2012;20:39-45.
- Kennedy AM, Aziz A, Khalid S, Hurman D. Do GP referral guidelines really work? Audit of an electronic urgent referral system for suspected head and neck cancer. Eur Arch Otorhinolaryngol 2012;269:1509-12. http://dx.doi.org/10.1007/s00405-011-1788-3.
- McGowan J, Hogg W, Campbell C, Rowan M. Just-in-time information improved decision-making in primary care: a randomised controlled trial. PLOS ONE 2008;3. http://dx.doi.org/10.1371/journal.pone.0003785.
- Gandhi TK, Keating NL, Ditmore M, Kiernan D, Johnson R, Burdick E, et al. Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 3: Performance and Tools). Rockville, MD: Agency for Healthcare Research and Quality; 2008.
- Junghans C, Feder G, Timmis AD, Eldridge S, Sekhri N, Black N, et al. Effect of patient-specific ratings vs conventional guidelines on investigation decisions in angina – appropriateness of referral and investigation in angina (ARIA) trial. Arch Intern Med 2007;167:195-202. http://dx.doi.org/10.1001/archinte.167.2.195.
- Akbari A, Grimshaw J, Stacey D, Hogg W, Ramsay T, Cheng-Fitzpatrick M, et al. Change in appropriate referrals to nephrologists after the introduction of automatic reporting of the estimated glomerular filtration rate. CMAJ 2012;184:E269-76. http://dx.doi.org/10.1503/cmaj.110678.
- Emery J, Morris H, Goodchild R, Fanshawe T, Prevost AT, Bobrow M, et al. The GRAIDS trial: a cluster randomised controlled trial of computer decision support for the management of familial cancer risk in primary care. Br J Cancer 2007;97:486-93. http://dx.doi.org/10.1038/sj.bjc.6603897.
- Knab JH, Wallace MS, Wagner RL, Tsoukatos J, Weinger MB. The use of a computer-based decision support system facilitates primary care physicians’ management of chronic pain. Anaesth Analg 2001;93:712-20. http://dx.doi.org/10.1097/00000539-200109000-00035.
- Mariotti G, Meggio A, de Pretis G, Gentilini M. Improving the appropriateness of referrals and waiting times for endoscopic procedures. J Health Serv Res Policy 2008;13:146-51. http://dx.doi.org/10.1258/jhsrp.2008.007170.
- Greiver M, Drummond N, White D, Weshler J, Moineddin R. Angina on the Palm: randomised controlled pilot trial of Palm PDA software for referrals for cardiac testing. Can Fam Physician 2005;51:382-3.
- Magill MK, Day J, Mervis A, Donnelly SM, Parsons M, Baker AN, et al. Improving colonoscopy referral rates through computer-supported, primary care practice redesign. J Healthc Qual 2009;31:43-52. http://dx.doi.org/10.1111/j.1945-1474.2009.00037.x.
- Tierney WMO, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, et al. Effects of computerised guidelines for managing heart disease in primary care: a randomised, controlled trial. J Gen Intern Med 2003;18:967-76. http://dx.doi.org/10.1111/j.1525-1497.2003.30635.x.
- Slade M, Gask L, Leese M, McCrone P, Montana C, Powell R, et al. Failure to improve appropriateness of referrals to adult community mental health services – lessons from a multi-site cluster randomised controlled trial. Fam Pract 2008;25:181-90. http://dx.doi.org/10.1093/fampra/cmn025.
- Stainkey LA, Seidl IA, Johnson AJ, Tulloch GE, Pain T. The challenge of long waiting lists: how we implemented a GP referral system for non-urgent specialist’ appointments at an Australian public hospital. BMC Health Serv Res 2010;10. http://dx.doi.org/10.1186/1472-6963-10-303.
- King RHB, Bateman H. Piloting an approach to the identification of avoidable referrals in a general practice with a high referral rate. J Clin Excel 2001;2:209-13.
- van Bokhoven MA, Koch H, van der Weijden T, Weekers-Muyres AH, Bindels PJ, Grol RP, et al. The effect of watchful waiting compared to immediate test ordering instructions on general practitioners’ blood test ordering behaviour for patients with unexplained complaints; a randomised clinical trial (ISRCTN55755886). Implement Sci 2012;7. http://dx.doi.org/10.1186/1748-5908-7-29.
- Callaway P, Frisch L. Does a family physician who offers colposcopy and LEEP need to refer patients to a gynaecologist?. J Fam Pract 2000;49:534-6.
- Standing PD. Changes in referral patterns to cardiac out-patient clinics with ambulatory ECG monitoring in general practice. Br J Cardiol 2001;8:394-9.
- van Dijk CE, Verheij RA, Spreeuwenberg P, Groenewegen PP, de Bakker DH, van Dijk CE, et al. Minor surgery in general practice and effects on referrals to hospital care: observational study. BMC Health Serv Res 2011;11. http://dx.doi.org/10.1186/1472-6963-11-2.
- Ridsdale L, Doherty J, McCrone P, Seed P. Headache Innovation and Evaluation Group. A new GP with special interest headache service: observational study. Br J Gen Pract 2008;58:478-83. http://dx.doi.org/10.3399/bjgp08X319440.
- Salisbury C, Noble A, Horrocks S, Crosby Z, Harrison V, Coast J, et al. Evaluation of a general practitioner with special interest service for dermatology: randomised controlled trial. BMJ 2005;331:1441-6. http://dx.doi.org/10.1136/bmj.38670.494734.7C.
- Sanderson D. Evaluation of the GPs with Special Interests (GPwSIs) Pilot Projects within the Action On ENT Programme: Final Report. Department of Health Modernisation Agency, York Health Economics Consortium; 2002.
- Sauro AS, Scalzitti F, Buono N, Siringano R, Petrazzuoli F, Diodati G, et al. Spirometry is really useful and feasible in the GPs’ daily practice but guidelines alone are not. Eur J Gen Pract 2005;11:29-31. http://dx.doi.org/10.3109/13814780509178015.
- Rosen R, Jones R, Tomlin Z, Cavanagh M. Evaluation of General Practitioners with Special Interests: Access, Cost Evaluation and Satisfaction with Services. NIHR Service Delivery and Organisation; 2006.
- Levell NJ, Penart-Lanau AM, Garioch JJ. Introduction of intermediate care dermatology services in Norfolk, England was followed by a 67% increase in referrals to the local secondary care dermatology department. Br J Dermatol 2012;167:443-5. http://dx.doi.org/10.1111/j.1365-2133.2012.10850.x.
- Leiba AM, Martonovits G, Magnezi R, Goldberg A, Carroll J, Benedek P, et al. Evaluation of a specialist outreach clinic in a primary healthcare setting: the effect of easy access to specialists. Clinician Manag 2002;11:131-6.
- Campbell H, Holloway S, Cetnarskyj R, Anderson E, Rush R, Fry A, et al. Referrals of women with a family history of breast cancer from primary care to cancer genetics services in South East Scotland. Br J Cancer 2003;89:1650-6. http://dx.doi.org/10.1038/sj.bjc.6601348.
- Felker BL, Barnes RF, Greenberg DM, Chaney EF, Shores MM, Gillespie-Gateley L, et al. Preliminary outcomes from an integrated mental health primary care team. Psychiatr Serv 2004;55:442-4. http://dx.doi.org/10.1176/appi.ps.55.4.442.
- Gurden M, Morelli M, Sharp G, Baker K, Betts N, Bolton J, et al. Evaluation of a general practitioner referral service for manual treatment of back and neck pain. Prim Health Care Res Dev 2012;13:204-10. http://dx.doi.org/10.1017/S1463423611000648.
- Schulpen GJ, Vierhout WP, van der Heijde DM, Landewe RB, Winkens RA, van der Linden S, et al. Joint consultation of general practitioner and rheumatologist: does it matter?. Ann Rheum Dis 2003;62:159-61. http://dx.doi.org/10.1136/ard.62.2.159.
- Vlek JF, Vierhout WP, Knottnerus JA, Schmitz JJ, Winter J, Wesselingh-Megens AM, et al. A randomised controlled trial of joint consultations with general practitioners and cardiologists in primary care. Br J Gen Pract 2003;53:108-12.
- Hughes-Anderson W, Rankin SL, House J, Aitken J, Heath D, House AK, et al. Open access endoscopy in rural and remote Western Australia: does it work?. ANZ J Surg 2002;72:699-703. http://dx.doi.org/10.1046/j.1445-2197.2002.02535.x.
- Hermush V, Daliot D, Weiss A, Brill S, Beloosesky Y. The impact of geriatric consultation on the care of the elders in community clinics. Arch Gerontol Geriatr 2009;49:260-2. http://dx.doi.org/10.1016/j.archger.2008.09.007.
- Pfeiffer PN, Szymanski BR, Zivin K, Post EP, Valenstein M, McCarthy JF, et al. Are primary care mental health services associated with differences in specialty mental health clinic use?. Psychiatr Serv 2011;62:422-5. http://dx.doi.org/10.1176/ps.62.4.pss6204_0422.
- Johnson G, White A, Livingstone R. Do general practices which provide an acupuncture service have low referral rates and prescription costs? A pilot survey. Acupunct Med 2008;26:205-13. http://dx.doi.org/10.1136/aim.26.4.205.
- Tan E, Levell NJ, Garioch JJ. The effect of a dermatology restricted-referral list upon the volume of referrals. Clin Exp Dermatol 2007;32:114-15.
- van Dijk CE, Verheij RA, Hansen J, van d V, Nijpels G, Groenewegen PP, et al. Primary care nurses: effects on secondary care referrals for diabetes. BMC Health Serv Res 2010;10. http://dx.doi.org/10.1186/1472-6963-10-230.
- White M, Bijlani N, Bale R, Burns T. Impact of counsellors in primary care on referrals to secondary mental health services. Psychiatr Bull 2000;24:418-20. http://dx.doi.org/10.1192/pb.24.11.418.
- Simpson S, Corney R, Fitzgerald P. Counselling provision, prescribing and referral rates in a general practice setting. Prim Care Psychiatry 2003;8:115-19. http://dx.doi.org/10.1185/135525703125001857.
- Schillinger D, Bibbins-Domingo K, Vranizan K, Bacchetti P, Luce JM, Bindman AB, et al. Effects of primary care coordination on public hospital patients. J Gen Intern Med 2000;15:329-36. http://dx.doi.org/10.1046/j.1525-1497.2000.07010.x.
- Ferris TG, Chang Y, Blumenthal D, Pearson SD. Leaving gatekeeping behind – effects of opening access to specialists for adults in a health maintenance organisation. N Engl J Med 2001;345:1312-17. http://dx.doi.org/10.1056/NEJMsa010097.
- Ferris TG, Chang Y, Perrin JM, Blumenthal D, Pearson SD. Effects of removing gatekeeping on specialist utilisation by children in a Health Maintenance Organisation. Arch Pediatr Adolesc Med 2002;156:574-9. http://dx.doi.org/10.1001/archpedi.156.6.574.
- Joyce GF, Kapur K, Van Vorst KA, Escarce JJ. Visits to primary care physicians and to specialists under gatekeeper and point-of-service arrangements. Am J Manag Care 2000;6:1189-96.
- McGarry HH, Hegarty K, Johnson C, Gunn J, Blashki G. Managing depression in a changing primary mental healthcare system: comparison of two snapshots of Australian GPs’ treatment and referral patterns. Ment Health Fam Med 2009;6:75-83.
- van Dijk CE, Verheij RA, Spreeuwenberg P, Van den Berg MJ, Groenewegen PP, Braspenning J, et al. Impact of remuneration on guideline adherence: empirical evidence in general practice. Scand J Prim Health Care 2013;31:56-63. http://dx.doi.org/10.3109/02813432.2012.757078.
- Vardy DA, Freud T, Sherf M, Spilberg O, Goldfarb D, Cohen AD, et al. A co-payment for consultant services: primary care physicians’ referral actualisation. J Med Syst 2008;32:37-41. http://dx.doi.org/10.1007/s10916-007-9105-9.
- Iversen T, Luras H. The effect of capitation on GPs’ referral decisions. Health Econ 2000;9:199-210. http://dx.doi.org/10.1002/(SICI)1099-1050(200004)9:3<199::AID-HEC514>3.0.CO;2-2.
- Watson E, Clements A, Lucassen A, Yudkin P, Mackay J, Austoker J. Education improves general practitioner (GP) management of familial breast/ovarian cancer: findings from a cluster randomised controlled trial. J Med Genet 2002;39:779-81. http://dx.doi.org/10.1136/jmg.39.10.779.
- Whiting M. Introducing a GP-led referral gateway. Pulse 2011;71:29-30.
- Maddison P, Jones J, Breslin A, Carton C, Fleur J. Improved access and targeting of musculoskeletal services in northwest Wales: targeted early access to musculoskeletal services (TEAMS) programme. BMJ 2004;329:1325-7. http://dx.doi.org/10.1136/bmj.329.7478.1325.
- Kim C, Williamson DF, Herman WH, Safford MM, Selby JV, Marrero DG, et al. Referral management and the care of patients with diabetes: the Translating Research into Action for Diabetes (TRIAD) study. Am J Manag Care 2004;10:137-43.
- Cox JMS, Steel N, Clark AB, Kumaravel B, Bachmann MO. Do referrral management schemes reduce hospital outpatient attendances?. Br J Gen Pract 2013;63:299-300. http://dx.doi.org/10.3399/bjgp13X668177.
- Ferriter K, Gangopadhyay P, Nilforooshan R. Quality of referrals to old age psychiatry following introduction of the single assessment process. Psychiatr Bull 2006;30. http://dx.doi.org/10.1192/pb.30.12.452.
- Albertson G, Lin CT, Schilling L, Cyran E, Anderson S, Anderson RJ, et al. Impact of a simple intervention to increase primary care provider recognition of patient referral concerns. Am J Manag Care 2002;8:375-81.
- Heaney D, Wyke S, Wilson P, Elton R, Rutledge P. Assessment of impact of information booklets on use of healthcare services: randomised controlled trial. BMJ 2001;322:1218-21. http://dx.doi.org/10.1136/bmj.322.7296.1218.
- Lyon D, Knowles J, Slater B, Kennedy R. Improving the early presentation of cancer symptoms in disadvantaged communities: putting local people in control. Br J Cancer 2009;101:49-54. http://dx.doi.org/10.1038/sj.bjc.6605390.
- Delva F, Marien E, Fonck M, Rainfray M, Demeaux JL, Moreaud P, et al. Factors influencing general practitioners in the referral of elderly cancer patients. BMC Cancer 2011;11. http://dx.doi.org/10.1186/1471-2407-11-5.
- Fucito LMG, Gomes B, Murnion B, Haber P. General practitioners’ diagnostic skills and referral practices in managing patients with drug and alcohol-related health problems: implications for medical training and education programmes. Drug Alcohol Rev 2003;22:417-24. http://dx.doi.org/10.1080/09595230310001613930.
- Scheeres K, Wensing M, Mes C, Bleijenberg G. The impact of informational interventions about cognitive behavioural therapy for chronic fatigue syndrome on GPs referral behaviour. Patient Educ Couns 2007;68:29-32. http://dx.doi.org/10.1016/j.pec.2007.04.002.
- Ringard A. Why do general practitioners abandon the local hospital? An analysis of referral decisions related to elective treatment. Scand J Public Health 2010;38:597-604. http://dx.doi.org/10.1177/1403494810371019.
- Townsley CA, Naidoo K, Pond GR, Melnick W, Straus SE, Siu LL, et al. Are older cancer patients being referred to oncologists? A mail questionnaire of Ontario primary care practitioners to evaluate their referral patterns. J Clin Oncol 2003;21:4627-35. http://dx.doi.org/10.1200/JCO.2003.06.073.
- Zielinski AH, Håkansson A, Jurgutis A, Ovhed I, Halling A. Differences in referral rates to specialised health care from four primary health care models in Klaipeda, Lithuania. BMC Fam Pract 2008;26. http://dx.doi.org/10.1186/1471-2296-9-63.
- Elhayany A, Shvartzman P, Regev S, Reuveni H, Tabenkin H. Variations in referrals to consultants: a study of general practitioners’ characteristics in southern Israel. J Ambulatory Care Manage 2000;23:45-54. http://dx.doi.org/10.1097/00004479-200001000-00005.
- Kvaerner KJ, Helgaker AB. Otitis media referrals – the general practitioner perspective. Int J Pediatr Otorhinolaryngol 2007;71:1219-24. http://dx.doi.org/10.1016/j.ijporl.2007.04.012.
- Naccarella L, Pirkis J, Morley B, Kohn F, Blashki G, Burgess PE-MA, et al. Managing demand for psychological services within an Australian primary mental healthcare initiative. Primary Care Community Psychiatry 2008;13:126-33.
- O’Neill L, Kuder J. Explaining variation in physician practice patterns and their propensities to recommend services. Med Care Res Rev 2005;62:339-57. http://dx.doi.org/10.1177/1077558705275424.
- Swarztrauber K, Vickrey BG, Mittman BS. Physicians’ preferences for specialty involvement in the care of patients with neurological conditions. Med Care 2002;40:1196-209. http://dx.doi.org/10.1097/00005650-200212000-00007.
- Freed GL, Jee S, Stein L, Spera L, Clark SJ. Comparing the self-reported referral and management preferences of pediatricians and family physicians for children with juvenile rheumatoid arthritis. J Rheumatol 2003;30:2700-4.
- Tzaribachev NB, Benseler SM, Tyrrell PN, Meyer A, Kuemmerle-Deschner JB. Predictors of delayed referral to a pediatric rheumatology center. Arthritis Rheum 2009;61:1367-72. http://dx.doi.org/10.1002/art.24671.
- Dodds W, Morgan M, Wolfe C, Raju KS. Implementing the 2-week wait rule for cancer referral in the UK: general practitioners’ views and practices. Eur J Cancer Care 2004;13:82-7. http://dx.doi.org/10.1111/j.1365-2354.2004.00447.x.
- Lambert MV, Bird JM. The assessment and management of adult patients with epilepsy – the role of general practitioners and the specialist services. Seizure 2001;10:341-6. http://dx.doi.org/10.1053/seiz.2001.0520.
- Pomeroy SE, Cant RP. General practitioners’ decision to refer patients to dietitians: insight into the clinical reasoning process. Aust J Prim Health 2010;16:147-53. http://dx.doi.org/10.1071/PY09024.
- Knight L. How do GPs make referral and treatment decisions when patients present with mental health problems?. Counsell Psychol Q 2003;16:195-221. http://dx.doi.org/10.1080/09515070310001610092.
- Wassenaar TR, Eickhoff JC, Jarzemsky DR, Smith SS, Larson ML, Schiller JH. Differences in primary care clinicians’ approach to non-small cell lung cancer patients compared with breast cancer. J Thorac Oncol 2007;2:722-8. http://dx.doi.org/10.1097/JTO.0b013e3180cc2599.
- Lakha SF, Yegneswaran B, Furlan JC, Legnini V, Nicholson K, Mailis-Gagnon A. Referring patients with chronic noncancer pain to pain clinics: survey of Ontario family physicians. Can Fam Physician 2011;57:e106-12.
- Montgomery AJ, McGee HM, Shannon W, Donohoe J. Factors influencing general practitioner referral of patients developing end-stage renal failure: a standardised case-analysis study. BMC Health Serv Res 2006;6. http://dx.doi.org/10.1186/1472-6963-6-114.
- Jørgensen CK, Olesen F. Predictors for referral to physiotherapy from general practice. Scand J Prim Health Care 2001;19:48-53. http://dx.doi.org/10.1080/028134301300034684.
- Rowlands G, Willis S, Singleton A. Referrals and relationships: in-practice referrals meetings in a general practice. Fam Pract 2001;18:399-406. http://dx.doi.org/10.1093/fampra/18.4.399.
- Rushton JB, Bruckman D, Kelleher K. Primary care referral of children with psychosocial problems. Arch Pediatr Adolesc Med 2002;156:592-8. http://dx.doi.org/10.1001/archpedi.156.6.592.
- Angstman KB, Adamson SC, Furst JW, Houston MS, Rohrer JE. Provider satisfaction with virtual specialist consultations in a family medicine department. Health Care Manag (Frederick) 2009;28:14-8. http://dx.doi.org/10.1097/HCM.0b013e318196def8.
- Kisely S, Horton-Hausknecht J, Miller K, Mascall C, Tait A, Wong P, et al. Increased collaboration between primary care and psychiatric services. A survey of general practitioners’ views and referrals. Aust Fam Physician 2002;31:587-9.
- Mitchell L, Macdonald-Wicks L, Capra S. Increasing dietetic referrals: perceptions of general practitioners, practice nurses and dietitians. Nutrition Dietetics 2012;69:32-8. http://dx.doi.org/10.1111/j.1747-0080.2011.01570.x.
- Coulston JE, Williams GL, Stephenson BM. Audit of referral patterns for hernia repair – are general practitioners aware of the changing face of herniorrhaphy?. Ann R Coll Surg Engl 2008;90:140-1. http://dx.doi.org/10.1308/003588408X261573.
- Ramanathan SA, Baratiny G, Stocks NP, Searles AM, Redford RJ. General practitioner referral patterns for women with gynaecological symptoms: a randomised incomplete block study design. Med J Aust 2011;195:602-6. http://dx.doi.org/10.5694/mja10.10867.
- Blundell N, Taylor-Phillips S, Spitzer D, Martin S, Forde I, Clarke A, et al. Elective surgical referral guidelines--background educational material or essential shared decision making tool? A survey of GPs’ in England. BMC Fam Pract 2011;12. http://dx.doi.org/10.1186/1471-2296-12-92.
- Clarke A, Nusila M, Le M. The REFER Project: Realistic Effective Facilitation of Elective Referral for Elective Surgical Assessment 2010. www.netscc.ac.uk/hsdr/files/project/SDO_FR_08–1310–072_V01.pdf (accessed 14 January 2014).
- Kasje WN, Denig P, De Graeff PA, Haaijer-Ruskamp FM. Physicians’ views on joint treatment guidelines for primary and secondary care. Int J Qual Health Care 2004;16:229-36. http://dx.doi.org/10.1093/intqhc/mzh038.
- Baker R, Lecouturier J, Bond S. Explaining variation in GP referral rates for X-rays for back pain. Implement Sci 2006;1. http://dx.doi.org/10.1186/1748-5908-1-15.
- Jiwa M, Gordon M, Arnet H, Ee H, Bulsara M, Colwell B, et al. Referring patients to specialists: a structured vignette survey of Australian and British GPs. BMC Fam Pract 2008;9. http://dx.doi.org/10.1186/1471-2296-9-2.
- Ruston A. Risk, anxiety and defensive action: general practitioner’s referral decisions for women presenting with breast problems. Health Risk Soc 2004;6. http://dx.doi.org/10.1080/1369857042000193066.
- Watson E, Austoker J, Lucassen. A study of GP referrals to a family cancer clinic for breast/ovarian cancer. Fam Pract 2001;18:131-4. http://dx.doi.org/10.1093/fampra/18.2.131.
- Bederman SS, McIsaac WJ, Coyte PC, Kreder HJ, Mahomed NN, Wright JG, et al. Referral practices for spinal surgery are poorly predicted by clinical guidelines and opinions of primary care physicians. Med Care 2010;48:852-8. http://dx.doi.org/10.1097/MLR.0b013e3181e3588b.
- Belgamwar B, Bates C, Goes C, Talyor N. Appropriateness of GP referrals of patients with anxiety and depression. Progress Neurol Psychiatry 2011;15:27-9. http://dx.doi.org/10.1002/pnp.188.
- Tucker JF, Farmer J, Stimpson P. Guidelines and management of mild hypertensive conditions in pregnancy in rural general practices in Scotland: issues of appropriateness and access. Qual Saf Health Care 2003;12:286-90. http://dx.doi.org/10.1136/qhc.12.4.286.
- Trude S, Stoddard JJ. Referral gridlock: primary care physicians and mental health services. J Gen Intern Med 2003;18:442-9. http://dx.doi.org/10.1046/j.1525-1497.2003.30216.x.
- Morsi E, Lindenauer PK, Rothberg MB. Primary care physicians’ use of publicly reported quality data in hospital referral decisions. J Hosp Med 2012;7:370-5. http://dx.doi.org/10.1002/jhm.1931.
- Sigel PL, Leiper R. GP views of their management and referral of psychological problems: a qualitative study. Psychol Psychother 2004;77:279-95. http://dx.doi.org/10.1348/1476083041839394.
- Philichi L, Yuwono M. Primary care: constipation and encopresis treatment strategies and reasons to refer. Gastroenterol Nurs 2010;33:363-6. http://dx.doi.org/10.1097/SGA.0b013e3181f35020.
- Steele M, Zayed R, Davidson B, Stretch N, Nadeau L, Fleisher W, et al. Referral patterns and training needs in psychiatry among primary care physicians in Canadian rural/remote areas. J Can Acad Child Adolesc Psychiatry 2012;21:111-23.
- Nandy S, Chalmers-Watson C, Gantley M, Underwood M. Referral for minor mental illness: a qualitative study. Br J Gen Pract 2001;51:461-5.
- Moore JL, McAuley JW, Mott D, Reeves AL, Bussa B. Referral characteristics of primary care physicians for seizure patients. Epilepsia 2000;41:744-8. http://dx.doi.org/10.1111/j.1528-1157.2000.tb00237.x.
- Olson R, Lengoc S, Tyldesley S, French J, McGahan C, Soo J. Relationships between family physicians’ referral for palliative radiotherapy, knowledge of indications for radiotherapy, and prior training: a survey of rural and urban family physicians. Radiation Oncol 2012;7. http://dx.doi.org/10.1186/1748-717X-7-73.
- Calnan MP, Payne S, Kemple T, Rossdale M, Ingram J. A qualitative study exploring variations in GPs’ out-of-hours referrals to hospital. Br J Gen Pract 2007;57:706-13.
- Anthony JS, Baik SY, Bowers BJ, Tidjani B, Jacobson CJ, Susman J, et al. Conditions that influence a primary care clinician’s decision to refer patients for depression care. Rehabil Nurs 2010;35:113-22. http://dx.doi.org/10.1002/j.2048-7940.2010.tb00286.x.
- Bruyninckx R, Van den Bruel A, Aertgeerts B, Van CV, Buntinx F. Why does the general practitioner refer patients with chest pain not-urgently to the specialist or urgently to the emergency department? Influence of the certainty of the initial diagnosis. Acta Cardiol 2009;64:259-65. http://dx.doi.org/10.2143/AC.64.2.2036147.
- Morgan M, Jenkins L, Ridsdale L. Patient pressure for referral for headache: a qualitative study of GPs’ referral behaviour. Br J Gen Pract 2007;57:29-35.
- Rosemann T, Wensing M, Rueter G, Szecsenyi J. Referrals from general practice to consultants in Germany: if the GP is the initiator, patients’ experiences are more positive. BMC Health Serv Res 2005;6. http://dx.doi.org/10.1186/1472-6963-6-5.
- van der Weijden T, van Bokhoven M, Dinant GJ, van Hasselt CM, Grol RPTM. Understanding laboratory testing in diagnostic uncertainty: a qualitative study in general practice. Br J Gen Pract 2002;52:974-80.
- Wilkes S, Murdoch A, Steen N, Wilsdon J, Rubin G. Open Access Tubal aSsessment for the initial management of infertility in general practice (the OATS trial): a pragmatic cluster randomised controlled trial. Br J Gen Pract 2009;59:329-35. http://dx.doi.org/10.3399/bjgp09X420590.
- Ahluwalia SC, Fried TR. Physician factors associated with outpatient palliative care referral. Palliat Med 2009;23:608-15. http://dx.doi.org/10.1177/0269216309106315.
- Pryor AMR, Knowles AA. The relationship between general practitioners’ characteristics and the extent to which they refer clients to psychologists. Aust Psychol 2001;36:227-31. http://dx.doi.org/10.1080/00050060108259659.
- Franks P, Williams GC, Zwanziger J, Mooney C, Sorbero M. Why do physicians vary so widely in their referral rates?. J Gen Intern Med 2000;15:163-8. http://dx.doi.org/10.1046/j.1525-1497.2000.04079.x.
- Espeland A, Baerheim A. Factors affecting general practitioners’ decisions about plain radiography for back pain: implications for classification of guideline barriers – a qualitative study. BMC Health Serv Res 2003;3. http://dx.doi.org/10.1186/1472-6963-3-8.
- Cornford CS, Harley J, Oswald N. The ‘2-week rule’ for suspected breast carcinoma: a qualitative study of the views of patients and professionals. Br J Gen Pract 2004;54:584-8.
- Forrest CB, Nutting P, Werner JJ, Starfield B, von Schrader S, Rohde C. Managed health plan effects on the specialty referral process – results from the ambulatory sentinel practice network referral study. Med Care 2003;41:242-53. http://dx.doi.org/10.1097/01.MLR.0000044903.91168.B6.
- Green H, Johnston O, Cabrini S, Fornai G, Kendrick T. General practitioner attitudes towards referral of eating-disordered patients: a vignette study based on the theory of planned behaviour. Ment Health Fam Med 2008;5:213-18.
- Young CE, Mutch AJ, Boyle FM, Dean JH. Investigating referral pathways from primary care to consumer health organisations. Aust J Prim Health 2010;16:260-7. http://dx.doi.org/10.1071/PY09081.
- Barnett ML, Keating NL, Christakis NA, O’Malley AJ, Landon BE. Reasons for choice of referral physician among primary care and specialist physicians. J Gen Intern Med 2012;27:506-12. http://dx.doi.org/10.1007/s11606-011-1861-z.
- Forrest CB, Nutting PA, Starfield B, von SS. Family physicians’ referral decisions: results from the ASPN referral study. J Fam Pract 2002;51:215-22.
- Dale N, Godsman J. Factors influencing general practitioner referrals to a tertiary paediatric neurodisability service. Br J Gen Pract 2000;50:131-2.
- McKenna CF, Farber NJ, Eschbach KS, Collier VU. Primary care practitioners’ understanding of psychiatric practice: effects on intention to refer. Arch Phys Med Rehabil 2005;86:881-8. http://dx.doi.org/10.1016/j.apmr.2004.09.014.
- Beel JV, Gringart E, Edwards MEE-MA, Beel JV. Western Australian general practitioners’ views on psychologists and the determinants of patient referral: an exploratory study. Families Systems Health 2008;26:250-66. http://dx.doi.org/10.1037/a0012913.
- Wakefield P, Randall G, Fiala JM. Competing for referrals for cardiac diagnostic tests: what do family physicians really want?. JMIRS 2012;43:155-60. http://dx.doi.org/10.1016/j.jmir.2012.04.001.
- Clemence ML, Seamark DA. GP referral for physiotherapy to musculoskeletal conditions – a qualitative study. Fam Pract 2003;20:578-82. http://dx.doi.org/10.1093/fampra/cmg515.
- Chew-Graham C, Slade M, Montâna C, Stewart M, Gask L. Loss of doctor-to-doctor communication: lessons from the reconfiguration of mental health services in England. J Health Serv Res Policy 2008;13:6-12. http://dx.doi.org/10.1258/jhsrp.2007.006053.
- Dagneaux I, Gillard I, De Lepeleire J. Care of elderly people by the general practitioner and the geriatrician in Belgium: a qualitative study of their relationship. J Multidiscip Healthc 2012;5:17-25. http://dx.doi.org/10.2147/JMDH.S27617.
- Harlan G, Srivastava R, Harrison L, McBride G, Maloney C. Paediatric hospitalists and primary care providers: a communication needs assessment. J Hosp Med 2009;4:187-93. http://dx.doi.org/10.1002/jhm.456.
- Allareddy V, Greene BR, Smith M, Haas M, Liao J. Facilitators and barriers to improving interprofessional referral relationships between primary care physicians and chiropractors. J Ambulatory Care Manage 2007;30:347-54. http://dx.doi.org/10.1097/01.JAC.0000290404.96907.e3.
- Taggarshe D, Haldipur N, Singh S. Generic outpatient referrals: why don’t GPs make them?. J Public Health (Oxf) 2006;28:218-20. http://dx.doi.org/10.1093/pubmed/fdl027.
- Samant RS, Fitzgibbon E, Meng J, Graham ID. Barriers to palliative radiotherapy referral: a Canadian perspective. Acta Oncol 2007;46:659-63. http://dx.doi.org/10.1080/02841860600979005.
- Xu T, Rohrer J, Borders T. The impact of managed care and practice size on primary care physicians’ perceived ability to refer. J Health Serv Res Policy 2002;7:143-50. http://dx.doi.org/10.1258/135581902760082445.
- Massey J. Referral criteria to improve access for patients with incontinence/pelvic floor dysfunction from primary care to a physiotherapy continence service. J Assoc Chartered Physiother Womens Health 2004;95:29-35.
- Berendsen AJ, Benneker WH, Meyboom-de JB, Klazinga NS, Schuling J. Motives and preferences of general practitioners for new collaboration models with medical specialists: a qualitative study. BMC Health Serv Res 2007;7. http://dx.doi.org/10.1186/1472-6963-7-4.
- Kinchen KS, Cooper LA, Levine D, Wang NY, Powe NR. Referral of patients to specialists: factors affecting choice of specialist by primary care physicians. Ann Fam Med 2004;2:245-52. http://dx.doi.org/10.1370/afm.68.
- Thorsen O, Hartveit M, Baerheim A. General practitioners’ reflections on referring: an asymmetric or non-dialogical process?. Scand J Prim Health Care 2012;30:241-6. http://dx.doi.org/10.3109/02813432.2012.711190.
- Greer RC, Powe NR, Jaar BG, Troll MU, Boulware LE. Effect of primary care physicians’ use of estimated glomerular filtration rate on the timing of their subspecialty referral decisions. BMC Nephrol 2011;12. http://dx.doi.org/10.1186/1471-2369-12-1.
- Jiwa M, Arnet H, Bulsara M, Ee HC, Harwood A. What is the importance of the referral letter in the patient journey? A pilot survey in Western Australia. Qual Prim Care 2009;17:31-6.
- Harvey NSG, Gill PV, Kimlim J. A survey of general practitioners’ preferences, when referring to mental health services, and the implications for electronic outpatient booking. Prim Care Community Psychiatry 2005;10:51-6.
- O’Byrne L, Darlow C, Roberts N, Wilson G, Partridge MR. Smoothing the passage of patients from primary care to specialist respiratory opinion. Prim Care Respir J 2010;19:248-53. http://dx.doi.org/10.4104/pcrj.2010.00028.
- Carlsen B, Aakvik A, Norheim OF. Variation in practice: a questionnaire survey of how congruence in attitudes between doctors and patients influences referral decisions. Med Decis Making 2008;28:262-8. http://dx.doi.org/10.1177/0272989X07311751.
- Johnson C, Paul C, Girgis A, Adams J, Currow D. Australian general practitioners’ and oncology specialists’ perceptions of barriers and facilitators of access to specialist palliative care services. J Palliat Med 2011;14:429-35. http://dx.doi.org/10.1089/jpm.2010.0259.
- Forrest CB, Shadmi E, Nutting PA, Starfield B. Specialty referral completion among primary care patients: results from the ASPN Referral Study. Ann Fam Med 2007;5:361-7. http://dx.doi.org/10.1370/afm.703.
- Ramchandani M, Mirza S, Sharma A, Kirkby G. Pooled cataract waiting lists: views of hospital consultants, general practitioners and patients. J R Soc Med 2003;95:598-600. http://dx.doi.org/10.1258/jrsm.95.12.598.
- Hyman I, Singh M, Ahmad F, Austin L, Meana M, George U, et al. The role of physicians in mammography referral for older Caribbean women in Canada. Medscape Womens Health 2001;6.
- Stavrou S, Cape J, Barker C. Decisions about referrals for psychological therapies: a matched-patient qualitative study. Br J Gen Pract 2009;59. http://dx.doi.org/10.3399/bjgp09X454089.
- Vulto A, van Bommel M, Poortmans P, Lybeert M, Louwman M, Baart R, et al. General practitioners and referral for palliative radiotherapy: a population-based survey. Radiother Oncol 2009;91:267-70. http://dx.doi.org/10.1016/j.radonc.2009.01.009.
- Little P, Dorward M, Warner G, Stephens K, Senior J, Moore M. Importance of patient pressure and perceived pressure and perceived medical need for investigations, referral, and prescribing in primary care: nested observational study. BMJ 2004;328:444-46. http://dx.doi.org/10.1136/bmj.38013.644086.7C.
- Rosen R, Florin D, Hutt R. An Anatomy of GP Referral Decisions: A Qualitative Study of GPs’ Views on Their Role in Supporting Patient Care. London: The King’s Fund; 2007.
- Blundell N, Clarke A, Mays N. Interpretations of referral appropriateness by senior health managers in five PCT areas in England: a qualitative investigation. Qual Saf Health Care 2010;19:182-6. http://dx.doi.org/10.1136/qshc.2007.025684.
- Glozier N, Prince M. Psychological determinants of primary care requests for urgent outpatient appointments in elective referrals. J Eval Clin Pract 2007;13:236-41. http://dx.doi.org/10.1111/j.1365-2753.2006.00685.x.
- Musila N, Underwood M, McCaskie AW, Black N, Clarke A, van der Meulen JH, et al. Referral recommendations for osteoarthritis of the knee incorporating patients’ preferences. Fam Pract 2011;28:68-74. http://dx.doi.org/10.1093/fampra/cmq066.
- Davies E, van der Molen B, Cranston A, Davies E, van der Molen B, Cranston A. Using clinical audit, qualitative data from patients and feedback from general practitioners to decrease delay in the referral of suspected colorectal cancer. J Eval Clin Pract 2007;13:310-17. http://dx.doi.org/10.1111/j.1365-2753.2006.00820.x.
- Edwards M. BRIDGE Study Group . Responses of primary health care professionals to UK national guidelines on the management and referral of women with breast conditions. J Eval Clin Pract 2002;8:319-25. http://dx.doi.org/10.1046/j.1365-2753.2002.00335.x.
- Brien S, Howells E, Leydon G, Brien Ssacu. Why GPs refer patients to complementary medicine via the NHS: a qualitative exploration. Prim Health Care Res Development 2008;9:205-15. http://dx.doi.org/10.1017/S1463423608000789.
- Bekkelund SI, Salvesen R. Are headache patients who initiate their referral to a neurologist satisfied with the consultation? A population study of 927 patients – the North Norway Headace Study (NNHS). Fam Pract 2001;18:524-7. http://dx.doi.org/10.1093/fampra/18.5.524.
- Lewis CL, Wickstrom GC, Kolar MM, Keyserling TC, Bognar BA, Dupre CT, et al. Patient preferences for care by general internists and specialists in the ambulatory setting. J Gen Intern Med 2000;15:75-83. http://dx.doi.org/10.1046/j.1525-1497.2000.05089.x.
- Gross R, Tabenkin H, Brammli-Greenberg S. Who needs a gatekeeper? Patients’ views of the role of the primary care physician. Fam Pract 2000;17:222-9. http://dx.doi.org/10.1093/fampra/17.3.222.
- Berendsen AJ, de Jong GM, Schuling J, Bosveld HE, de Waal MW, Mitchell GK, et al. Patient’s need for choice and information across the interface between primary and secondary care: a survey. Patient Educ Couns 2010;79:100-5. http://dx.doi.org/10.1016/j.pec.2009.07.032.
- Shadd J, Ryan BL, Maddocks H, Thind A. Patterns of referral in a Canadian primary care electronic health record database: retrospective cross-sectional analysis. Inform Prim Care 2011;19:217-23.
- Bertakis KD, Callahan EJ, Azari R, Robbins JA, Bertakis KD. Predictors of patient referrals by primary care residents to specialty care clinics. Fam Med 2001;33:203-9.
- Dearman SP, Waheed W. Management strategies in geriatric depression by primary care physicians and factors associated with the use of psychiatric services: a naturalistic study. Aging Ment Health 2006;10:521-4. http://dx.doi.org/10.1080/13607860600637984.
- Ridsdale LC, Clark LV, Dowson AJ, Goldstein LH, Jenkins L, McCrone P, et al. How do patients referred to neurologists for headache differ from those managed in primary care?. Br J Gen Pract 2007;57:388-95.
- Cohen SM, Jaewhan K, Nelson R, Courey M. Factors influencing referral of patients with voice disorders from primary care to otolaryngology. Laryngoscope 2014;124:214-20. http://dx.doi.org/10.1002/lary.24280.
- Harris M, Jayasinghe U, Chan B. Patient and practice characteristics predict the frequency of general practice multidisciplinary referrals of patients with chronic diseases: a multilevel study. Health Policy 2011;101:140-5. http://dx.doi.org/10.1016/j.healthpol.2010.10.019.
- Vinker S, Kaiserman I, Karni A, Kitai E, Kasinetz LM, Elhayany A, et al. Urgent referrals to a specialist by family physicians – is the ‘urgency’ real: a prospective study. Eur J Gen Pract 2007;13:37-9. http://dx.doi.org/10.1080/13814780600881003.
- Albertson GA, Lin CT, Kutner J, Schilling LM, Anderson SN, Anderson RJ. Recognition of patient referral desires in an academic managed care plan. J Gen Intern Med 2000;15:242-7. http://dx.doi.org/10.1111/j.1525-1497.2000.02208.x.
- Johnson CE, Danhauer JL, Koch LL, Celani KE, Lopez IP, Williams VA, et al. Hearing and balance screening and referrals for Medicare patients: a national survey of primary care physicians. J Am Acad Audiol 2008;19:171-90. http://dx.doi.org/10.3766/jaaa.19.2.7.
- Hugo P, Kendrick T, Reid F, Lacey H. GP referral to an eating disorder service: why the wide variation?. Br J Gen Pract 2000;50:380-3.
- Bowling A, Harries C, Forrest D, Harvey N. Variations in cardiac interventions: doctors’ practices and views. Fam Pract 2006;23:427-36. http://dx.doi.org/10.1093/fampra/cmi125.
- Balduf LM, Farrell TM. Attitudes, beliefs, and referral patterns of PCPs to bariatric surgeons. J Surg Res 2008;144:49-58. http://dx.doi.org/10.1016/j.jss.2007.01.038.
- Chan BT, Austin PC. Patient, physician, and community factors affecting referrals to specialists in Ontario, Canada: a population-based, multi-level modelling approach. Med Care 2003;41:500-11. http://dx.doi.org/10.1097/01.MLR.0000053971.89707.97.
- Bolaños-Carmona V, Ocaña-Riola R, Prados-Torres A, Gutiérrez-Cuadra P. Variations in health services utilisation by primary care patients. Health Serv Manage Res 2002;15:116-25. http://dx.doi.org/10.1258/0951484021912888.
- Ache KA, Shannon RP, Heckman MG, Diehl NN, Willis FB, Ache KA, et al. A preliminary study comparing attitudes towards hospice referral between African American and white American primary care physicians. J Palliat Med 2011;14:542-7. http://dx.doi.org/10.1089/jpm.2010.0426.
- Navaneethan SD, Kandula P, Jeevanantham V, Nally JV, Liebman SE, Navaneethan SD, et al. Referral patterns of primary care physicians for chronic kidney disease in general population and geriatric patients. Clin Nephrol 2010;73:260-7. http://dx.doi.org/10.5414/CNP73260.
- Cooper SM, Wojnarowska F. The influence of the sex of the general practitioner on referral to a vulval clinic. Journal Obstet Gynaecol 2001;21:179-80. http://dx.doi.org/10.1080/01443610020026155.
- Coyle E, Hanley K, Sheerin J. Who goes where? A prospective study of referral patterns within a newly established primary care team. Ir J Med Sci 2011;180:845-9. http://dx.doi.org/10.1007/s11845-011-0724-2.
- Gruen RL, Knox S, Britt H. Where there is no surgeon: the effect of specialist proximity on general practitioners’ referral rates. Med J Aust 2002;177:111-15.
- Feeney E, Noble H, Waller G. Referral patterns to a specialist eating disorder service: the impact of the referrer’s gender. Eur Eat Disord Rev 2007;15:45-9. http://dx.doi.org/10.1002/erv.759.
- Forrest CB, Nutting PA, von SS, Rohde C, Starfield B. Primary care physician specialty referral decision making: patient, physician, and health care system determinants. Med Decis Making 2006;26:76-85. http://dx.doi.org/10.1177/0272989X05284110.
- Chauhan M, Bankart MJ, Labeit A, Baker R. Characteristics of general practices associated with numbers of elective admissions. J Public Health 2012;34:584-90. http://dx.doi.org/10.1093/pubmed/fds024.
- Sullivan CO, Omar RZ, Ambler G, Majeed A. Case-mix and variation in specialist referrals in general practice. Br J Gen Pract 2005;55:529-33.
- Chen FM, Fryer GE, Norris TE. Effects of comorbidity and clustering upon referrals in primary care. J Am Board Fam Pract 2005;18:449-52. http://dx.doi.org/10.3122/jabfm.18.6.449.
- McBride D, Hardoon S, Walters K, Gilmour S, Raine R. Explaining variation in referral from primary to secondary care: cohort study. BMJ 2010;341. http://dx.doi.org/10.1136/bmj.c6267.
- Robinson PC, Taylor WJ. Time to treatment in rheumatoid arthritis: factors associated with time to treatment initiation and urgent triage assessment of general practitioner referrals. J Clin Rheumatol 2010;16:267-73. http://dx.doi.org/10.1097/RHU.0b013e3181eeb499.
- Johnson CE, Girgis A, Paul CL, Currow DC. Palliative care referral practices and perceptions: the divide between metropolitan and non-metropolitan general practitioners. Palliat Support Care 2011;9:181-9. http://dx.doi.org/10.1017/S1478951511000058.
- Guevara JP, Greenbaum PE, Shera D, Bauer L, Schwarz DF. Survey of mental health consultation and referral among primary care pediatricians. Acad Pediatr 2009;9:123-7. http://dx.doi.org/10.1016/j.acap.2008.12.008.
- Bowling A, Redfern J. The process of outpatient referral and care: the experiences and views of patients, their general practitioners, and specialists. Br J Gen Pract 2000;50:116-20.
- Grace SL, Grewal K, Stewart DE. Factors affecting cardiac rehabilitation referral by physician specialty. J Cardiopulm Rehab Prev 2008;28:248-52. http://dx.doi.org/10.1097/01.HCR.0000327182.13875.07.
- Holley CD, Lee PP. Primary care provider views of the current referral-to-eye-care process: focus group results. Invest Ophthalmol Vis Sci 2010;51:1866-72. http://dx.doi.org/10.1167/iovs.09-4512.
- Kier A, George M, McCarthy P. Referral patterns to spinal manipulative therapy by Welsh general practitioners. Clinical Chiropractic 2012;15:91-2. http://dx.doi.org/10.1016/j.clch.2012.06.015.
- Todman JP, Law J, MacDougall A. Attitudes of GPs towards Older Adults Psychology Services in the Scottish Highlands. Rural Remote Health 2011;11.
- Soomro GM, Burns T, Majeed A. Socio-economic deprivation and psychiatric referral and admission rates: an ecological study in one London borough. Psychiatr Bull 2002;26. http://dx.doi.org/10.1192/pb.26.5.175.
- Mulvaney C, Coupland C, Wilson A, Hammersley V, Dyas J, Carlisle R, et al. Does increased use of private health care reduce the demand for NHS care? A prospective survey of general practice referrals. J Public Health (Oxf) 2005;27:182-8. http://dx.doi.org/10.1093/pubmed/fdi013.
- Soerensen TH, Olsen KR, Vedsted P. Association between general practice referral rates and patients’ socioeconomic status and access to specialised health care: a population-based nationwide study. Health Policy 2009;92:180-6. http://dx.doi.org/10.1016/j.healthpol.2009.03.011.
- Malcolm JC, Liddy C, Rowan M, Maranger J, Keely E, Harrison C, et al. Transition of patients with type 2 diabetes from specialist to primary care: a survey of primary care physicians on the usefulness of tools for transition. Can J Diabetes 2008;32:37-45. http://dx.doi.org/10.1016/S1499-2671(08)21009-9.
- Walders N, Childs GE, Comer D, Kelleher KJ, Drotar D, Walders N, et al. Barriers to mental health referral from pediatric primary care settings. Am J Manag Care 2003;9:677-83.
- Burns T. Socio-economic deprivation and psychiatric referral and admission rates: an ecological study in one London borough. Psychiatr Bull 2002;26:175-8. http://dx.doi.org/10.1192/pb.26.5.175.
- Love T, Crampton P, Salmond C, Dowell A. Patterns of medical practice variation: variability in referral for back pain by New Zealand general practitioners. N Z Med J 2005;118.
- Ashworth M, Clement S, Sandhu J, Farley N, Ramsay R, Davies T, et al. Psychiatric referral rates and the influence of on-site mental health workers in general practice. Br J Gen Pract 2002;52:39-41.
- Boulware LE, Troll MU, Jaar BG, Myers DI, Powe NR. Identification and referral of patients with progressive CKD: a national study. Am J Kidney Dis 2006;48:192-204. http://dx.doi.org/10.1053/j.ajkd.2006.04.073.
- Chung K, Yang D, Lee JH. Determinants of primary care physicians’ referral pattern: a structural equation model approach. Int J Public Pol 2010;5:259-71. http://dx.doi.org/10.1504/IJPP.2010.030607.
- Greenaway R, Fortune L. General practitioners’ views of psychology services: a comparison of general practitioners who refer to onsite and offsite services. Prim Care Ment Health 2006;4:245-54.
- Franz CE, Barker JC, Kim K, Flores Y, Jenkins C, Kravitz RL, et al. When help becomes a hindrance: mental health referral systems as barriers to care for primary care physicians treating patients with Alzheimer’s disease. Am J Geriatr Psychiatry 2010;18:576-85. http://dx.doi.org/10.1097/JGP.0b013e3181a76df7.
- Alexander C, Fraser J. General practitioners’ management of patients with mental health conditions: the views of general practitioners working in rural north-western New South Wales. Aust J Rural Health 2008;16:363-9. http://dx.doi.org/10.1111/j.1440-1584.2008.01017.x.
- Patton M. Qualitative Research and Evaluation Methods. Thousand Oaks, CA: Sage; 2002.
- Anderson R, Crabtree B, Steele D, McDaniel R. Case study research: the view from complexity science. Qual Health Res 2005;15:669-85. http://dx.doi.org/10.1177/1049732305275208.
- Capra F. The Web of Life. New York, NY: Anchor-Doubleday; 1996.
- Alvarez MP, Agra Y. Systematic review of educational interventions in palliative care for primary care physicians. Palliat Med 2006;20:673-83. http://dx.doi.org/10.1177/0269216306071794.
- Bazian Ltd . Specialist outreach into primary care: is it better than standard care?. Evid-Based Healthc Public Health 2005;9:294-301.
- Bower P, Sibbald B. Systematic review of the effect of on-site mental health professionals on the clinical behaviour of general practitioners. BMJ 2000;320:614-17. http://dx.doi.org/10.1136/bmj.320.7235.614.
- Bower P, Sibbald B. Do consultation-liaison services change the behaviour of primary care providers? A review. General Hospital Psychiatry 2000;22:84-96. http://dx.doi.org/10.1016/S0163-8343(00)00058-X.
- Brocklehurst PR, Baker SR, Speight PM. Primary care clinicians and the detection and referral of potentially malignant disorders in the mouth: a summary of the current evidence. Prim Dent Care 2010;17:65-71. http://dx.doi.org/10.1308/135576110791013749.
- Evaluation of Referral Management Pilots in Wales. NHS Wales and National Leadership and Innovation Agency for Healthcare; 2006.
- Referral Management Pilots in Wales – Follow Up Review. National Leadership and Innovation Agency for Healthcare; 2007.
- Clarke A, Blundell N, Forde I, Musila N, Spitzer D, Naqvi S, et al. Can guidelines improve referral to elective surgical specialties for adults: a systematic review. Qual Saf Health Care 2008;19:187-94. http://dx.doi.org/10.1136/qshc.2008.029918.
- Delva F, Soubeyran P, Rainfray M, Mathoulin-Pelissier S. Referral of elderly cancer patients to specialists: action proposals for general practitioners. Cancer Treatment Reviews 2012;38:935-41. http://dx.doi.org/10.1016/j.ctrv.2012.03.010.
- Foot C, Naylor C, Imison C. The Quality of GP Diagnosis and Referral. London: The King’s Fund; 2010.
- Forrest CB. Primary care gatekeeping and referrals: effective filter of failed experiment?. BMJ 2003;326:692-5. http://dx.doi.org/10.1136/bmj.326.7391.692.
- Gruen RL, Weeramanthri TS, Knight SS, Bailie RS. Specialist outreach clinics in primary care and rural hospital settings. Cochrane Database Syst Rev 2003;4. http://dx.doi.org/10.1002/14651858.CD003798.pub2.
- Harkness EF, Bower PJ. On site mental health workers delivering psychological therapy and psychosocial interventions to patients in primary care: effects on the professional practice of primary care providers. Cochrane Database Syst Rev 2009;1. http://dx.doi.org/10.1002/14651858.CD000532.pub2.
- Herrington P, Baker R, Gibson SL, Golden S. GP referrals for counselling: a review and model. J Interprof Care 2003;17:263-71. http://dx.doi.org/10.1080/1356182031000122889.
- Jiwa M, Dadich A. Referral letter content: can it affect patient outcomes?. Br J Health Care Manage 2013;19:140-7. http://dx.doi.org/10.12968/bjhc.2013.19.3.140.
- Lin CY. Improving care coordination in the specialty referral process between primary and specialty care. N C Med J 2012;73:61-2.
- Martin A, Macleod C, Naqui SAR. Effectiveness and Cost-Effectiveness of Targeted Interventions to Reduce Unnecessary Referrals and Improve the Quality of Referrals from Primary Care to Secondary Care. NHS Evidence Adoption Centre East of England; 2010.
- Mead N, Bower P. Patient centred consultations and outcomes in primary care: a review of the literature. Patient Educ Couns 2002;48:51-6. http://dx.doi.org/10.1016/S0738-3991(02)00099-X.
- Mehrota A, Forrest CB, Lin CY. Dropping the baton: specialty referrals in the United States. Millbank Q 2011;89:39-68. http://dx.doi.org/10.1111/j.1468-0009.2011.00619.x.
- Navaneethan SD, Aloudat S, Singh S. A systematic review of patient and health system characteristics associated with late referral in chronic kidney disease. BMC Nephrology 2008;9. http://dx.doi.org/10.1186/1471-2369-9-3.
- O’Donnell CA. Variation in GP referral rates: what can we learn from the literature?. Fam Pract 2000;17:462-71. http://dx.doi.org/10.1093/fampra/17.6.462.
- Piterman L, Koritsas S. Part II General practitioner-specialist referral process. Intern Med J 2005;35:491-6. http://dx.doi.org/10.1111/j.1445-5994.2005.00860.x.
- Powell J. Systematic review of outreach clinics in primary care in the UK. J Health Serv Res Policy 2007;7:177-83. http://dx.doi.org/10.1258/135581902760082490.
- Qureshi NA, van der Molen HT, Schmidt HG, Al-Habeeb TA, Magzoub MEM. Criteria for a good referral system for psychiatric patients: the view from Saudi Arabia. Eastern Mediterr Health J 2009;15:1580-95.
- Roland M, McDonald R, Sibbald B, Boyd A, Fotaki M, Gravelle H, et al. Outpatient Services and Primary Care: A Scoping Review of Research into Strategies for Improving Outpatient Effectiveness and Efficiency. National Primary Care Research and Development Centre, and Centre for Public Policy and Management of the University of Manchester; 2006.
Appendix 1 Extraction tables
Intervention papers
| Adams 201233 Country: USA Study design: Before-and-after Data collection method: Audience response system Aim: To describe the development and implementation process and assess the effect on self-reported clinical practice changes of a multidisciplinary, collaborative, interactive CME/continuing education (CE) programme on COPD Detail of participants (number, any reported demographics): 351 participants |
Method: Multidisciplinary subject matter experts and education specialists used a systematic instructional design approach and collaborated with the American College of Chest Physicians and American Academy of Nurse Practitioners to develop, deliver and reproduce a 1-day interactive COPD CME/CE program for 351 primary-care clinicians in 20 US cities from 23 September 2009 to 13 November 2010 Control: NA Length of follow-up: 3- to 6-month follow-up survey Response and/or attrition rate: 132 of 271 participants (48.7%) Context (from what/who to what/who): GP referral for COPD |
Outcome measures: Self-confidence Knowledge and comprehension |
Main results: Clinician self-confidence improved after the course in all areas measured. In addition, clinician knowledge/comprehension significantly improved [mean (SD) pre-test percentage correct, 77.1% (16.4); 95% CI 76.2% to 78.9%; and mean (SD) post-test percentage correct, 94.7% (8.7%); 95% CI 94.2% to 95.2%; p < 0.001], with an absolute percentage change of 17.6% (13.2%) Of the five knowledge (recall) questions, the mean (SD) improvement in pre-test vs. post-test scores was 14% (5.0%) (95% CI 9.6% to 18.4%), from 83.1% to 97.1% (p < 0.001). The mean improvement in the five comprehension/application questions was 2.7% 17.5% (95% CI 7.4% to 38.0%), from 68.8% to 91.5% (p < 0.001) Of the follow-up survey respondents, 92 of 132 (69.7%) reported completely implementing at least one clinical practice change, and only 8 of 132 (6.1%) reported inability to make any clinical practice change after the programme |
Reported associations between elements for logic model: A carefully designed, interactive, flexible, dynamic, and reproducible COPD CME/CE programme tailored to clinicians’ needs that involves diverse instructional strategies and media can have short-term and long-term improvements in clinician self-confidence, knowledge/comprehension, and clinical practice |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Akbari 2012110 Country: Canada Study design: Before-and-after Data collection method Aim: To show whether or not automatic reporting of the eGFR, along with an ad hoc educational component for primary care physicians, would increase the number of appropriate referrals Detail of participants (number, any reported demographics): All referrals to nephrologists received at the centre during the year before and the year after automatic reporting of the eGFR was introduced were included. The area served by the Champlain Local Health Integration Network has a population of 1,176,600, of whom 12.5% are 65 years or older, 17% are immigrants and 13% are from visible minorities |
Method: Automatic reporting of eGFR to inform referral decision Concurrent with the introduction of automatic reporting of the eGFR, the nephrology service mailed an algorithm to all primary care physicians in the Champlain Local Health Integration Network. This algorithm explained the interpretation of the eGFR and appropriate parameters for referrals to nephrology, based on the value. In addition, ad hoc educational sessions (lectures and workshops) were provided to the primary care physicians to discuss interpretation of the eGFR results and parameters for referral to nephrology Control: None Length of follow-up: 1 year Response and/or attrition rate: NR Context (from what/who to what/who): GP to nephrologist |
Outcome measures: Number and appropriateness of referrals to nephrologists |
Main results: A total of 2672 patients were included in the study. In the year after automatic reporting began, the number of referrals from primary care physicians increased by 80.6% (95% CI 74.8% to 86.9%) The number of appropriate referrals increased by 43.2% (95% CI 38.0% to 48.2%) There was no significant change in the proportion of appropriate referrals between the two periods (−2.8%, 95% CI −26.4% to 43.4%) In the year after automatic reporting of the eGFR was introduced, the total number of referrals increased significantly among patients 80 years and older (percentage-point change 8.0; p < 0.001) and among women (percentage-point change 12.6; p < 0.001) |
Reported associations between elements for logic model: The total number of referrals increased after automatic reporting of the eGFR began, especially among women and elderly people The number of appropriate referrals also increased, but the proportion of appropriate referrals did not change significantly |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Albertson 2002158 Country: USA Study design: Sequential prospective study before-and-after Data collection method: Questionnaire Aim: To determine whether or not a brief pre-visit questionnaire about referral concerns can improve primary care provider recognition of patient concerns and satisfaction with care Detail of participants (number, any reported demographics): 12 PCP, 1495 patients |
Intervention: Patients were given a pre-visit questionnaire about referral need and rationale and a post-visit questionnaire about referral concern and visit satisfaction. Providers were given a post-visit questionnaire asking whether a referral was discussed, or made, and about visit satisfaction. In the control phase patient pre-visit questionnaires remained confidential, whereas in the intervention phase PCPs were shown the pre-visit questionnaire at the time of the encounter Control: As above Length of follow-up: None Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to specialist |
Outcome measures: Referral Satisfaction |
Main results: The intervention significantly increased PCP referral recognition from 61% to 81% (p < 0.001) and was associated with increased visit satisfaction (p = 0.05). Satisfaction of PCPs with the referral discussion, overall rate of referral, and visit duration were not affected by the intervention |
Reported associations between elements for logic model: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Banait 200353 Country: UK Study design: Cluster RCT Data collection method: NR Aim: To test the effectiveness of educational outreach as a strategy for facilitating the uptake of dyspepsia management guidelines in primary care Detail of participants (number, any reported demographics): 114 general practices (233 GPs) in Salford and Trafford |
Method: All groups received the guidelines by post. The intervention groups began to receive education outreach 3 months later Including: Practice-based seminars with hospital specialists at which guidelines recommendations were appraised and implementation plans formulated, reinforcement visits after 12 weeks Control: Guidelines received by post Length of follow-up: 6 months post intervention Response and/or attrition rate: One practice dropped out Context (from what/who to what/who): GP to open-access endocscopy (GI) |
Outcome measures: Appropriateness of referral for open-access endoscopy |
Main results: The proportion of appropriate referrals was higher in the intervention group in the 6 months’ post-intervention period (practice medians: control = 50%, intervention = 63.9%; p < 0.05) The proportion of major findings at endoscopy did not alter significantly, but there was an overall rise in acid suppressing drugs in the intervention group, compared with the control group (+ 8% vs. + 2%, p = 0.005) |
Reported associations between elements for logic model: Outreach may be more effective that passive guideline dissemination in changing clinical behaviour, but unexpected outcomes may emerge (increased drug prescription) |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Bennett 200130 Country: UK Study design: cRCT Data collection method: Routine data on ENT referrals and diagnostic results Aim: To evaluate the effect of a risk factor checklist and training video for GPs in reducing inter-practice variation and improving the appropriateness of referrals Detail of participants (number, any reported demographics): 50 practices (177 GPs) from NHS Trent and West Scotland |
Intervention: Practices were cluster randomised to either the control group (n = 12) or to one of three intervention groups (training video n = 16, checklist n = 11, or both n = 11) Data on all paediatric ENT referrals and diagnostic results were collected for 1 year pre and post intervention. Referral rates for otitis media with effusion (glue ear) and for closely related conditions were calculated for children aged 0–15 years based on practice size. Positive predictive value was defined as the proportion of referrals resulting in bilateral hearing loss of > 20 dB at the ENT outpatient department Number of hours: NA Delivered by who? NA Control: No intervention Length of follow-up: 1 year Response and/or attrition rate: NR Context (from what/who to what/who): GP referral to ENT for glue ear |
Outcome measures: Appropriateness of referrals Variation in referral rate |
Main results: There was significant improvement in the positive predictive value, adjusted for patient waiting time between GP referral and appointment at the ENT department. The improvement in positive predictive value pre and post intervention was 15% (95% CI –12.1% to 41.7%) for the practices receiving both interventions, compared with 20% (95% CI –32.9 to –6.4%) for practices receiving only one intervention and a degradation of 34% for those receiving no intervention |
Reported associations between elements for logic model: Disseminating a risk factor checklist and training video improved quality of ENT referral for glue ear by more than administering only one of these interventions |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Bhalla 200237 Country: UK Study design: Case control Data collection method: Annual otolaryngology referral rates from individuals in two demographically matched general practices to the two ENT consultants were gathered prospectively over a 6-year time period Aim: To determine if the provision of clinical ENT training to a GP influenced the subsequent rates of referral to specialist clinics, and if the presence of this practitioner within the practice made any difference to overall referrals by other partners Detail of participants (number, any reported demographics): Two demographically twinned practices of similar size (demographics not reported). Intervention practice had four partners (no other details) |
Intervention: One partner, within practice A, attended three or four clinical ENT sessions over a 2-week period, once a year for 3 years, where he would work alongside the consultant otolaryngologist Number of hours: Three or four clinical ENT sessions over a 2-week period, once a year for 3 years Delivered by who? Consultant otolaryngologist Control: Second practice with no training intervention Length of follow-up: 3 years. Data gathered over 6-year period Response and/or attrition rate: NA Context (from what/who to what/who): Referral from general practice to otolaryngology (ENT) |
Outcome measures: Referral rates |
Main results: There was no statistical difference in referral rates (Kruskal–Wallis: p = 0.63) for the trained partner when compared with the other three partners in the same practice No statistical difference in referral patterns between the intervention and the control practice (Mann–Whitney U-test: p = 0.50) Intervention practice 552 referrals over 6 years, control practice 521 referrals. No difference in each year either Sharp increase in number of referrals from both practices in third year of data collection TABLE I: Numbers of ENT referrals for individual partners within the same practice YearDUJSMLST1994–542461995–615101481996–7403622391997–8383337311998–9332227321999–200026192826Total156122132142 There was also no statistical difference (Mann–Whitney U-test: p = 0.50) in the referral patterns between the two practices TABLE II: Numbers of ENT referrals from all partners within two demographically matched general practices YearPractice APractice B1994–516211995–647381996–71371191997–81391421998–9114901999–200099111 |
Year | DU | JS | ML | ST | 1994–5 | 4 | 2 | 4 | 6 | 1995–6 | 15 | 10 | 14 | 8 | 1996–7 | 40 | 36 | 22 | 39 | 1997–8 | 38 | 33 | 37 | 31 | 1998–9 | 33 | 22 | 27 | 32 | 1999–2000 | 26 | 19 | 28 | 26 | Total | 156 | 122 | 132 | 142 | Year | Practice A | Practice B | 1994–5 | 16 | 21 | 1995–6 | 47 | 38 | 1996–7 | 137 | 119 | 1997–8 | 139 | 142 | 1998–9 | 114 | 90 | 1999–2000 | 99 | 111 | Reported associations between elements for logic model: The presence of a partner in receipt of continued ENT training made no difference to the specialist referral rates over a 6-year period when compared with the other partners within the same general practice. Furthermore, the presence of this partner did not influence the number of referrals from the practice as a whole when compared with a demographically matched general practice |
| Year | DU | JS | ML | ST | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1994–5 | 4 | 2 | 4 | 6 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1995–6 | 15 | 10 | 14 | 8 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1996–7 | 40 | 36 | 22 | 39 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1997–8 | 38 | 33 | 37 | 31 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1998–9 | 33 | 22 | 27 | 32 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1999–2000 | 26 | 19 | 28 | 26 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Total | 156 | 122 | 132 | 142 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Year | Practice A | Practice B | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1994–5 | 16 | 21 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1995–6 | 47 | 38 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1996–7 | 137 | 119 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1997–8 | 139 | 142 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1998–9 | 114 | 90 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1999–2000 | 99 | 111 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Bridgman 200570 Country: UK Study design: Controlled before-and-after Data collection method: Routine data Aim: To evaluate a slot system Detail of participants (number, any reported demographics): General practices in Staffordshire – 12 intervention, 24 controls, 63 others. Intervention group – mix of practices in area to include large and small, urban and rural, and champions and likely critics. Practices represented 14% of population, size varied from 17,000 to 14,000 and from one to six partners. 33 GPs involved, 30 full-time, three single handed and one job-share, one practice a training practice. Eleven practices were average referrers and one was a high referrer |
Intervention: System developed by MD input. GPs and orthopaedic consultants invited to a meeting to discuss and input to design. Postal survey to GPs regarding need and views Number of slots available based on registered practice population. Not adjusted for age. Quota of slots and their use fed back to practice on monthly basis. If they went beyond their quota they might not be allowed to refer any more patients that month. GPs guaranteed maximum of 8-week assessment for patients and backlog of waiting patients removed. GPs received guidelines on appropriate referrals and routes of referrals for musculoskeletal problems Clerical officer appointed to answer queries, make appointments. Modification to hospital software; referrals made using a special pro forma, which included a prioritisation score. New referrals triaged to most appropriate clinic by clinical director Number of hours: NA Delivered by who? NA Control: 24 control practices and all other practices in area (631) Length of follow-up: Up to second half of second year Response and/or attrition rate: All selected practices agreed to participate Context (from what/who to what/who): GP to orthopaedic outpatient clinic |
Outcome measures: Difference in referral rates per 10,000 population per month by practice |
Main results: 15,439 referrals made, 90% attended appointments Mean monthly referral rate in the intervention group declined 22% in year 1 and was maintained in year 2. From baseline to intervention year 1 [9.4 (SE 0.41) to 7.29 (SE 0.31) and in intervention year 2 [7.31 (SE 0.21)] Rates for two non-intervention groups were stable/slight decrease in year 1 [baseline 10.99 (SE 0.52) and 9.50 (SE 0.29) to year 1 9.9 (SE 0.39) and 9.31 (SE 0.36). Referrals increased in year 2 [11.7 (SE 0.48) and 10.33 (SE 0.36)] Difference in mean referral rate control to intervention = –1.59 intervention, –2.61 control, –4.39 other comparator Relative mean rate in reductions in mean referral rates were 14.5%, –23.7% and –39.5% in period 0, year 1 and year 2, respectively Linear regression indicated that the interaction between practice group and the time period was statistically significant (not reported) |
Reported associations between elements for logic model: A slot system can reduce referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Callaway and Frisch 2000121 Country: USA Study design: Service data review Data collection method: Clinic log review Aim: To evaluate whether LEEP training for family physicians impacts on referral to gynaecology Detail of participants (number, any reported demographics): n = 272 women attending a cervical dysplasia clinic |
Intervention: LEEP for cervical dysplasia carried out by family physicians in a cervical dysplasia clinic Number of hours: NA Delivered by who? Family physicians Control: None Length of follow-up: Evaluation over 6 years Response and/or attrition rate: NA Context (from what/who to what/who): Family physician to gynaecologist |
Outcome measures: Number of women referred to gynaecology |
Main results: During the study period, 283 women were seen in the clinic, and 26 individuals (9%) were referred by the family physician colposcopist to a consulting gynaecologist Of the 9% referred to gynaecologist, all but one subsequently treated with laser or combination of laser and LEEP |
Reported associations between elements for logic model: Family physicians who are well-trained in LEEP can manage cases without referral Physicians need to be thoroughly trained in cognitive and technical aspects of electrosurgery (and presumably have necessary equipment) |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Campbell 2003131 Country: UK Study design: Cluster RCT Data collection method: Referral data Aim: To gather referral data as part of RCT evaluating specialist outreach clinics Detail of participants (number, any reported demographics): 203 GP practices in SE Scotland Women with a positive family history of breast cancer |
Intervention: Referral to a clinic held in community setting near to the GP practice Number of hours: NA Delivered by who? NA Control: Existing service – an appointment to see a consultant geneticist and breast surgeon at a regional centre Length of follow-up: NA Response and/or attrition rate: 84% of practices agreed to take part Context (from what/who to what/who): GPs to cancer genetics advisors |
Outcome measures: Referral rates before and during the trial |
Main results: The referral rate rose from 2 years before the trial to during the trial (0.21 to 0.31). A 48% increase in referral rate (p < 0.001) 43% of women asked to be referred. Younger women were more likely to have taken the initiative to request referral (p = 0.001 chi-squared). Substantially greater increase in referral rates to community clinics than to regional centre (64% increase vs. 38% increase), suggesting that providing a service in the community resulted in a change in GP referral behaviour. This was particularly apparent in practices in relatively deprived communities. Higher referral rates from practices with more female partners before and during the trial (p < 0.005 and p < 0.02) |
Reported associations between elements for logic model: Provision of specialist service in community Sex of referrer |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Chen 2010100 Country: USA Study design: Before-and-after Data collection method: Clinical data Aim: To evaluate a new consultation request process, called e-Referral Detail of participants (number, any reported demographics): Primary care clinicians practising in five hospital-based, 11 community-based, and 10 independent non-profit community clinics in San Francisco San Francisco General Hospital, which provides more than 500,000 outpatient visits annually |
Intervention: A new consultation request process, called e-Referral, was integrated into the hospital’s electronic health record. Clinician reviewers screen requests to evaluate urgency, choice of specialties, whether or not sufficient work-up information is provided, and whether or not a specialist needs to see the patient or can guide the primary care clinician through the e-Referral system Control: None Length of follow-up: NR Response and/or attrition rate: NR Context (from what/who to what/who): GP referral to secondary care |
Outcome measures: Waiting times Appropriate referrals |
Main results: Waiting times for non-urgent visits declined in seven of eight medical specialty clinics by up to 90% during the first 6 months of use. Expedited visits accounted for up to one-third of all visits in some specialties. The percentage of referrals deemed inappropriate by medical and surgical specialists was cut by more than half For clinics that had been plagued by long wait times, implementation of e-Referral resulted in dramatic improvements. For example, in rheumatology, the median wait time for a non-urgent appointment initially dropped from 126 days to 29 days Access to a common electronic health record and participation by specialists who are salaried, and thus not financially dependent on generating visits, were critical to this programme’s success, but generally neither is an option in most practice settings. Success also depends on how well reviewers interact with primary care clinicians. The majority of primary care clinicians reported that e-Referral improved patient care, but those with poorer access to the electronic health record found it more time-consuming than the previous paper-based system |
Reported associations between elements for logic model: The percentage of referrals deemed inappropriate by medical and surgical specialists was cut by more than half |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Cooper 201219 Country: UK Study design: Evaluation/audit report Data collection method: NR Aim: To identify the underlying drivers for variance in referral rate and make plans to address them Detail of participants (number, any reported demographics): Not clear |
Intervention: Peer review scheme for referrals. Two guiding principles – the review would benefit the practice and the commissioning group; there was no blame GPs, nurses, advanced health-care practitioners and practice managers attended a workshop event. Each practice bought two or three trauma and orthopaedic referral letters. Participants worked at mixed tables to understand each practice’s referral profile, and share how each practice would handle each situation. Then to identify any gaps or areas of changed needed in terms of: Information needs Training needs Commissioning needs Needs for guidelines This led to a joint health training programme and individual practices were encouraged to tackle specialties where they were outliers Number of hours: Unclear Delivered by who? Unclear Control: None Length of follow-up: NR Response and/or attrition rate: NA Context (from what/who to what/who): GP referrals to trauma/orthopaedics and any specialty |
Outcome measures: Achieving target referral levels (incentivised) Target calculated by working out average rate per 1000 of weighted captitation for all GP referrals. Practices were awarded 20p per registered patient for referring below that level |
Main results: Trauma and orthopaedic expenditure in 2010–11 was 17% lower than in 2006–7 when the first workshop was conducted. One practice cut ENT referrals by 20% in the first year and 40% overall |
Reported associations between elements for logic model: Using clear communication and a professional approach can lead doctors to accept that peer review in not a ‘blame game’ but an opportunity to share and build experience In this context peer review can have a positive effect in reducing referral rates |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Cox 2013156 Country: UK Study design: Retrospective before-and-after Data collection method: Internal peer review of referral data Aim: To establish whether or not the introduction of referral management centres was associated with a reduction in hospital outpatient attendance rates Detail of participants (number, any reported demographics): 376,000 outpatient attendances from 85 practices |
Method: 85 practices formed five groups to manage referrals. Two groups also carried out peer review of referrals The directly age standardised GP referred first outpatient monthly attendance rate was calculated for each group from April 2009 to March 2012 using 5-year age bands. Linear regression tested for association between the introduction of referral management and change in outpatient attendance rate The RM interventions were more complex than internal peer-review interventions, involved a wider range of activities and including activities not directly related to referral management (no info given) Control: None Length of follow-up: Unclear. Study 3 years Response and/or attrition rate: NA Context (from what/who to what/who): GP to hospital outpatients |
Outcome measures: Attendance rate |
Main results: Four groups showed statistically significant increases in attendance rates ranging from 0.41 to 1.20 attendances per 1000 persons per month After correction, only one group (3, referral management centre) remained significant (1.05 attendances per 1000 persons per month, 95% CI 0.64 to 1.64; p < 0.005) There were no decreases in attendance rate |
Reported associations between elements for logic model: The introduction of referral management centres was not associated with a reduction in hospital attendance rates in any group |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Cusack and Buckley 200543 Country: UK Study design: Before-and-after Data collection method: Examination of referral letter quality Aim: To investigate the impact of issuing guidelines on acne referral Detail of participants (number, any reported demographics): 90 referral letters from GPs prior to guidelines and 60 following guideline introduction. 36 GPs |
Intervention: Guidelines and pro forma for future referrals sent to GPs Number of hours: NA Control: None Length of follow-up: 18 months Response and/or attrition rate: NA Context (from what/who to what/who): GP to dermatology |
Outcome measures: Number of referrals |
Main results: The number of referrals in accordance with NICE guidelines increased from 31% to 45% after introduction of guidelines (p = 0.041). The number of inappropriate referrals decreased from 69% to 55% 22% of GPs (8 of 36) fully complied with guidelines. Over 50% of referrals still inappropriate. Pro forma used in only 23% of referrals and provision of data in referral letters remained poor. Number of referrals per month only marginally decreased |
Reported associations between elements for logic model: Guideline provision and limited impact on referral |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| DAMASK 200876 Country: UK Study design: Cost-effectiveness analysis alongside RCT Data collection method: Costs estimated in terms of QALYs, patient responses to EQ-5D questionnaire Aim: To investigate cost-effectiveness of GP referral to early MRI scan Detail of participants (number, any reported demographics): n = 386 patients consulting GP about a knee problem |
Intervention: Referral from GP to local radiology department for MRI. Early access to imaging Number of hours: NA Delivered by who? NA Control: Normal care – referral to local orthopaedic department Length of follow up: NA Response and/or attrition rate: Data available for 70% (386) of sample Context (from what/who to what/who): GP to MRI (knee problems) |
Outcome measures: QALYs Cost Health outcomes Knee-related NHS usage reported by patients |
Main results: Early MRI was associated with higher NHS cost by £294 per patient and a larger number of QALYs by 0.05. Incremental cost per QALY gained of £5840 was below cost threshold of £20,000 per QALY commonly used in the NHS; therefore, is cost-effective use of NHS resources Higher cost in intervention group was partly due to higher number of primary care consultations and more use of physiotherapy but largely due to results from greater use of MRI |
Reported associations between elements for logic model: Early access to imaging cost-effective |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Dennison 200699 Country: UK Study design: Cross-sectional evaluation Data collection method: Electronic data analysis Aim: To assess the effect of the intervention on waiting times and attendance Detail of participants (number, any reported demographics): 54 patients referred electronically and 189 referred on paper; London hospital. 22 GPs in four practices |
Intervention: Electronic surgical referral pro forma system including patient details, symptoms, urgent/routine, provisional diagnosis, free-text box Number of hours: NA Delivered by who? NA Control: Paper referrals Length of follow-up: NA Response and/or attrition rate: All referrals over a 1-year period Context (from what/who to what/who): GP to colorectal and gastroenterology clinics |
Outcome measures: Waiting time referral to appointment booked Waiting time referral to clinic appointment Patient attendance rate |
Main results: Patients were 21% less likely to change appointment when referred electronically. Referral to appointment was 8 weeks for electronic system, 10 weeks for paper system. Referral to booking was 0 days for electronic system, 7 days for paper system – significant difference (Mann–Whitney U-test no other details) 8.5% non-attendance electronic system, 22.5% paper system – significant difference (χ2 no other details) |
Reported associations between elements for logic model: Electronic referral can improve efficiency of service |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Dey 200466 Country: UK Study design: cRCT Data collection method: NR Aim: To investigate the impact on patient management of an educational strategy to promote these guidelines among GPs Detail of participants (number, any reported demographics): 24 health centres. Two thousand, one hundred and eighty-seven eligible patients presented with acute low-back pain during the study period – 1049 in the intervention group and 1138 in the control group. Aged 18–64 years Mean age in years (SD) – 42.2 (12.1) intervention and 41.3 (12.5) control Female sex (%) –568 (54.1) intervention and 618 (54.3) control |
Intervention: Practices in the intervention arm were offered outreach visits to promote national guidelines on acute low-back pain, as well as access to fast-track physiotherapy and to a triage service for patients with persistent symptoms At least two members of the guideline team attended each visit; included senior representatives from the musculoskeletal directorate, physiotherapy services and the health authority. Members of the guideline team facilitated a structured interactive discussion with the GP, which was based on the ‘elaboration likelihood model of persuasion’. This discussion was used to raise awareness of the RCGP guidelines, adapted to the local context; emphasise the key messages in the guidelines; identify potential barriers to implementation; and suggest strategies for overcoming the barriers identified. GPs were given a poster reinforcing guideline recommendations and a copy of a text recommended by the RCGP for patients. Referral forms for access to fast-track physiotherapy were distributed at this session, as were forms for direct access to the back clinic of patients who had failed to respond to conservative management within 6 weeks Number of hours: NA Delivered by who? Guidance team (as above) Control: No intervention Length of follow up: 8 months Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to secondary care for low-back pain |
Outcome measures: Radiography referral, sickness certificate issued, prescribed opioids or muscle relaxants, referred to secondary care, physiotherapy, or educational programmes |
Main results: The estimated annual consultation rate for acute low-back pain was 35 per 1000 adults in the intervention group, compared with 38 per 1000 in the control group. There were no significant differences between study groups with respect to the proportion of patients who were referred for radiography (difference = 1.4%; 95% CI –4.1% to 6.8%), issued with a sickness certificate (difference = –1.5%; 95% CI –10.3% to 7.3%), prescribed opioids or muscle relaxants (difference = –0.03%; 95% CI –5.5% to 5.4%) or referred to secondary care (difference = 1.1%; 95% CI –0.3% to 2.6%). Significantly more patients in the intervention group were first referred to physiotherapy or to educational programmes at the back pain unit than in the control group (26.0% and 13.8%, respectively; difference = 12.2%; ICC = 0.0563; χ2 = 6.49, 1 df; p = 0.01; 95% CI for difference in proportion = 2.8% to 21.6%). A total of 121 (11.5%) patients in the intervention arm were referred to the triage service within the follow-up period. Of the 273 referrals to physiotherapy or the back pain unit by GPs in the intervention group, 110 (40.3%) were directed to these services by the back clinic triage service |
Reported associations between elements for logic model: The management of patients presenting with low-back pain to primary care was mostly unchanged by an outreach educational strategy to promote greater adherence to RCGP guidelines among GPs |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Dhillon 200382 Country: UK Study design: RCT Data collection method: NR Aim: To evaluate the impact of GP direct access to scanning Detail of participants (number, any reported demographics): 330 patients aged 31 to 89 years, Edinburgh. 18 general practices |
Intervention: GP direct access to DXA scanning for patients at risk of osteoporosis, no specific guidelines issued Number of hours: NA Delivered by who? NA Control: Routine rheumatology clinic appointment and scan Length of follow-up: NA Response and/or attrition rate: NR Context (from what/who to what/who): GP to rheumatology specialist |
Outcome measures: Frequency of change of GP management following scan |
Main results: Mostly clinical outcomes, but includes limited referral rate data. Before intervention the range of number of referrals for scanning was 0.01% to 0.6%, median 0.2%. Number of referrals to specialist clinic was 24 in intervention group, compared with 12 in control group Study concludes direct access more economically efficient; not fully explained in the data how this is evaluated |
Reported associations between elements for logic model: Access to scanning and efficiency? |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Donohoe 200031 Country: UK Study design: cRCT Data collection method: Questionnaires Aim: To evaluate a model of integrated diabetic foot care on identification and clinical management Detail of participants (number, any reported demographics): 10 towns from mid and East Devon. Total of 1939 patients with diabetes (aged over 18 years) |
Intervention: The intervention consisted of the integrated care model where the patient’s feet are examined by the GP or practice nurse. It includes complementary educational interventions aimed at clarifying management of the diabetic foot, referral criteria, and the responsibilities of professionals The intervention included practice visits and education of the whole practice team Leaflets outlining patients’ role and responsibility were disseminated to the practices Control: Current foot care arrangements and a practice visit where an alternative education package (on diabetic neuropathy) was given Length of follow-up: 6 months Response and/or attrition rate: Patients – intervention 68%, control 65% Professionals – intervention 80%, control 81% Context (from what/who to what/who): GP referral for diabetic foot care |
Outcome measures: Patient attitudes to value and importance of foot care Foot care knowledge (patient and professional) Use of services |
Main results: Attitudes to foot care improved in both groups (mean change 3.91, 0.68) with SD in change of 3.18 (95% CI 1.29 to 5.07) between groups Patient knowledge about foot problems improved significantly in both groups (mean percentage change 1.09, 1.32, but with no significant difference in change –0.09 (95% CI –1.81 to 18.62) between groups Appropriate referrals from intervention practices to the specialist foot clinic rose significantly (p = 0.05), compared with control practices (p = 0.14) |
Reported associations between elements for logic model: Integrated care arrangements can lead to an increase number of appropriate referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Eccles 200154 Country: UK Study design: RCT Data collection method: A random subset of GP patients’ records were examined for concordance with the guidelines Aim: To assess two methods of reducing GP requests for radiological tests in accordance with the UK Royal College of Radiologists’ guidelines on lumbar spine and knee radiographs Detail of participants (number, any reported demographics): six radiology departments and 244 general practices |
Intervention: A group of GPs and consultant radiologists wrote referral guidelines and educational messages for lumbar spine and knee radiographs (based on the Royal College of Radiologists’ guidelines and the RCGPs’ back-pain guidelines). The referral guidelines were sent by post to all GPs. Each practice was then randomly allocated to: audit and feedback, or control; and educational messages or control Feedback was prepared by the research team from routine data provided by the radiology departments. It covered the previous 6 months and was sent to GPs at the start of the intervention period and 6 months later. Feedback contained the number of requests for lumbar spine and knee radiographs made by the whole practice compared with requests made by all GPs in the study Educational messages were attached to the reports of every knee or lumbar spine radiograph requested during the 12-month intervention (e.g. ‘in adults with knee pain, without serious locking or restriction in movement, radiograph is not routinely indicated’) Number of hours: NA Delivered by who? Radiologists Control: Usual care Length of follow-up: 1 year Response and/or attrition rate: Of 247 practices, three dropped out Context (from what/who to what/who): GP to radiology |
Outcome measures: Number of radiograph requests per 1000 patients per year |
Main results: The effect of educational reminder messages (i.e. the change in request rate after intervention) was an absolute change of 1.53 (95% CI 2.5 to 0.57) for lumbar spine and of 1.61 (2.6 to 0.62) for knee radiographs, both relative reductions of about 20% The effect of audit and feedback was an absolute change of 0.07 (1.3 to 0.9) for lumbar spine of 0.04 (0.95 to1.03) for knee radiograph requests, both relative reductions of about 1% Concordance between groups did not differ significantly. Requests from doctors who had received audit and feedback were no more likely to be appropriate than requests from other doctors. The OR for lumbar spine radiographs was 0.75 (95% CI 0.52 to 1.07) and for knee was 0.82 (0.50 to 1.33). For doctors who had received educational reminder messages, the equivalent values were 0.95 (0.63 to 1.67) and 1.36 (0.86 to 2.23) |
Reported associations between elements for logic model: In this study 6-monthly feedback of audit data was ineffective but the routine attachment of educational reminder messages to radiographs was effective and did not affect quality of referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Eley 201083 Country: UK Study design: Audit Data collection method: Retrospective review of electronic records Aim: To assess effectiveness of direct referral audiology clinics on ENT appointments and appropriate GP use of clinics Detail of participants (number, any reported demographics): 353 patients [178 female, 175 male, mean age 77 (60–96) years] |
Intervention: DRACs for the assessment and provision of hearing aids in those over 60 years were introduced as a means of decreasing outpatient waiting times and demand on ENT appointments Number of hours: NA Delivered by who? Audiologist Control: None Length of follow-up: 4 months Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to audiology clinics or ENT |
Outcome measures: ENT appointments and appropriate GP use of clinics |
Main results: Of the 353 patients seen within the DRAC clinics, 320 were ultimately provided with a hearing aid. 55 patients require review by an otolaryngologist, either by direct referral or via their GP. The greatest lack of adherence to the referral criteria for DRAC appointments related to appropriate treatment of wax within the community |
Reported associations between elements for logic model: DRAC continues to provide a cost benefit to the NHS by reducing demand on ENT appointments |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Ellard 201238 Country: UK Study design: Before-and-after study (also control condition) Data collection method: All adult dermatology referrals seen between 1 April and 30 June in 2010 and 2011 (after the teaching programme) were examined using clinic letters or pro formas completed by clinicians after consultations based on NICE guidelines as criterion for appropriateness Aim: To assess the benefit of a consultant-led dermatology training programme for GPs on the number and quality of referrals to a local university teaching hospital Detail of participants (number, any reported demographics): 30 GPs from 26 practices attended the teaching programme |
Intervention: All local GPs were invited to participate in six 2-hour interactive sessions on common skin conditions in early 2011 Number of hours: 6 × 2 hours Delivered by who? NR Control: 36 other local GP practices that did not participate in the training programme Length of follow up: 3 months Response and/or attrition rate: NR Context (from what/who to what/who): GP referrals to dermatology |
Outcome measures: Appropriate referrals |
Main results: During the 3-month study period in 2010, 542 patients were seen, of whom 39% were appropriate referrals. After the teaching programme, 478 patients were seen during the same time period in 2011, of whom 58% were appropriate. More appropriate referrals were seen in all conditions except basal cell carcinoma, where there was a fall from 100% in 2010 to 93.8% in 2011 30 GPs from 26 practices attended the teaching programme. Appropriate referrals from these GPs increased from 37.2% in 2010 to 51.8% after training, accompanied by an increase in the mean number of referrals from 20.7 to 25.7. Furthermore, the overall number of appropriate referrals increased from 37.8% to 49.5% at these 26 surgeries. These results were compared with the 36 other local GP practices that did not participate in the training programme, which also displayed an increase in appropriate referrals from 40.8% to 56.4% from 2010 to 2011 This may reflect local referral priorities, patient factors and underlying differences between the practices, suggested by the observation that the mean number of referrals in 2010 from practices attending teaching was 12.37 (SD 9.02), compared with 4.83 (SD 3.54) in those that abstained. This study has limitations in its design and size, but the results suggest that further investigation into the benefits of GP education in dermatology would be worthwhile |
Reported associations between elements for logic model: During the 3-month study period (post GP training), the number of appropriate referrals increase, but control GPs also saw an increase in appropriate referral |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Elwyn 200755 Country: UK Study design: Before and after Data collection method: NR Aim: To evaluate a system of providing feedback to clinicians following referral requests not adhering to NICE guidelines Detail of participants (number, any reported demographics): 215 GPs in catchment area of three endoscopy units |
Intervention: Letter to GPs stating that two GPs would be employed part-time to assess all endoscopy letters and referrals for dyspepsia and they would be judged against recently issued NICE guidelines. Letter said that where referrals did not meet criteria, the referring doctor would be informed by letter giving reason for non-adherence to guidelines. All Wales Dyspepsia Guidelines based on NICE criteria circulated to all GPs 2 weeks earlier to this letter Referrals were processed in usual way Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 5 months Response and/or attrition rate: NA Context (from what/who to what/who): GP to endoscopy clinic |
Outcome measures: Adherence to NICE guidelines Number of referrals for gastroscopy Time referral to procedure |
Main results: Adherence to NICE guidelines for referral criteria increased significantly among GPs following the intervention (mean 55% to 75%, 95% CI 13.6 to 26.4; p < 0.001). No similar effect for hospital doctors Number of gastroscopy referrals for dyspepsia declined after the intervention; however, not significantly for GPs after inclusion of seasonal effects (p = 0.065) Intervention significantly reduced the referral to procedure time for gastroscopy (mean 52.1 days to mean 39.4 days, p < 0.001, 95% CI 6.6 to 18.6 days) Need to consider demand generated across all health care – hospital doctor referrals accounted for many more referrals than GPs (1720 bed hospital doctors referred more than 215 GPs) |
Reported associations between elements for logic model: Feedback to referrers can improve adherence to referral guidelines |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Emery 2007111 Country: Australia Study design: cRCT Data collection method Aim: To evaluate the effect of an assessment strategy using the computer decision support system (the GRAIDS software), on the management of familial cancer risk in British general practice in comparison with best current practice Detail of participants (number, any reported demographics): 45 general practice teams in East Anglia Randomised to GRAIDS support [intervention (n = 23) or comparison (n = 22)] |
Method: Training in the new assessment strategy and access to the GRAIDS software (GRAIDS arm) was conducted, compared with an educational session, and guidelines about managing familial breast and colorectal cancer risk (comparison) were mailed All GPs and practice nurses attended a 45-minute educational session on cancer genetics, delivered at their general practice. They were also introduced to the principles of the GRAIDS intervention Control: Current practice Length of follow-up: 12 months Response and/or attrition rate: 45/170 participated. All 45 practice teams were in the trial for a minimum of 12 months and none withdrew Context (from what/who to what/who): GP to regional cancer genetics service |
Outcome measures: Proportion of referrals made to the Regional Genetics Clinic for familial breast or colorectal cancer that were consistent with referral guidelines Practitioner confidence in managing familial cancer (GRAIDS arm only) and cancer worry, risk perception and knowledge about familial cancer |
Main results: There were more referrals to the Regional Genetics Clinic from GRAIDS than comparison practices (mean 6.2 and 3.2 referrals per 10,000 registered patients per year; mean difference 3.0 referrals; 95% CI 1.2 to 4.8; p = 0.001); referrals from GRAIDS practices were more likely to be consistent with referral guidelines [(OR) 5.2; 95% CI 1.7 to 15.8; p = 0.006]. Patients referred from GRAIDS practices had lower cancer worry scores at the point of referral (mean difference 1.44; 95% CI 2.64 to 0.23; p = 0.02) There were no differences in patient knowledge about familial cancer. The intervention increased GPs’ confidence in managing familial cancer |
Reported associations between elements for logic model: Compared with education and mailed guidelines, assessment including computer decision support increased the number and quality of referrals to the Regional Genetics Clinic for familial cancer risk, improved practitioner confidence and had no adverse psychological effects in patients |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Eminovic 200986 Country: the Netherlands Study design: cRCT Data collection method Aim: To determine whether or not teledermatologic consultations can reduce referrals to a dermatologist by GPs Detail of participants (number, any reported demographics): 85 GPs from 35 general practices in two regions in the Netherlands (Almere and Zeist); five dermatologists from two non-academic hospitals were also included in the study |
Method: The GPs randomised to the intervention used a teledermatologic consultation system to confer with a dermatologist, whereas those in the control group referred their patients according to usual practice The improved training programme for all intervention GPs included instructions on taking digital images, downloading images to the computer, managing files, and using the website. Dermatologists were taught how to use the website and complete the study forms All patients, regardless of their condition, were seen in the office by a dermatologist after approximately 1 month Control: Usual practice referral. In most cases this involved patients visiting the outpatient clinic with a letter in which the GP described findings pertinent to the case Length of follow up: 1 month Response and/or attrition rate: Of 56 GP practices eligible for participation, 36 (53%), including 110 GPs, agreed to participate Context (from what/who to what/who): GP to dermatology |
Outcome measures: The proportion of office visits prevented by teledermatologic consultation, as determined by dermatologists |
Main results: The 85 study GPs enrolled 631 patients (46 intervention GPs, 327 patients; 39 control GPs, 304 patients). The five dermatologists considered a consultation preventable for 39.0% of patients who received teledermatologic consultation and 18.3% of 169 control patients, a difference of 20.7% (95% CI 8.5% to 32.9%) At the 1-month dermatologist visit, 20.0% of patients who received teledermatologic consultation had recovered, compared with 4.1% of control patients. No significant differences in patient satisfaction were found between groups |
Reported associations between elements for logic model: Teledermatologic consultation can reduce referrals to a dermatologist by 20.7% |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Emmerson 200340 Country: Australia Study design: Evaluation questionnaire Data collection method: Surveys Aim: To develop a psychiatric assessment and advisory service for local GPs Detail of participants (number, any reported demographics): Five psychiatrists. 200 GPs in total |
Intervention: Five full-time psychiatrists dedicated a 1-hour appointment per week in their hospital private practice clinics to assess patients referred by local GPs. The Psych Opinion clinic was advertised through the Division of General Practice Newsletter Number of hours: 1 per week Delivered by who? Psychiatrists Control: NA Length of follow-up: NA – 1-year project Response and/or attrition rate: NA Context (from what/who to what/who): GP to psychiatry |
Outcome measures: Referral to the Psych Opinion Clinic Satisfaction with and awareness of the service |
Main results: After 12 months referrals to the clinic were disappointing (n = 30, with 10 referrals from one GP) Feedback from GPs who had used the service showed high levels of satisfaction with the service (mean score 6.2/7) Feedback from GPs who had not used the service showed a strong endorsement of the concept (94%), but there was poor awareness of the service’s existence (26%) |
Reported associations between elements for logic model: This study highlights the need to market new referral interventions, both initially and on an ongoing basis |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Engers 200567 Country: the Netherlands Study design: cRCT Data collection method: GP registration forms Aim: To assess the effectiveness of tailored interventions (multifaceted implementation strategy) to implement the Dutch low-back pain guideline for GPs with regard to adherence to guideline recommendations Detail of participants (number, any reported demographics): 41 of the 67 randomised GPs reported on a total of 616 consultations for 531 patients with non-specific low-back pain |
Intervention: GPs were randomised to an intervention or a control group. The GPs in the intervention group (n = 21) received tailored interventions consisting of the Dutch low-back-pain guideline for GPs, a 2-hour educational and clinical practice workshop; two scientific articles on low-back-pain management; the guideline for occupational physicians; a tool for patient education; and a tool for reaching agreement on low back care with physical, exercise, and manual therapists The participating GPs were asked to recruit consecutive patients with a new episode of low-back pain as the main reason for consultation Number of hours: 2 Delivered by who? Psychologist-physiotherapist Control: The control group (n = 20) received no intervention Length of follow-up: NR Response and/or attrition rate: NR Context (from what/who to what/who): GP referrals for low-back pain |
Outcome measures: Advice and information, referral to other health-care providers, and prescription of medication |
Main results: 41 of the 67 randomised GPs reported on a total of 616 consultations for 531 patients with non-specific low-back pain. The advice and explanation provided by the GP, the prescription of paracetamol (33% vs. 21%) or non-steroidal anti-inflammatory drugs (54% vs. 62%), and prescription of pain medication on a time-contingent (70% vs. 69%) or a pain-contingent basis (30% vs. 31%), showed no statistically significant differences between the intervention and control groups. There were also no differences in overall referral rate [23% vs. 28%, OR 0.8 (95% CI 0.5 to 1.4)]. However, in follow-up consultations fewer patients were referred to a physical or exercise therapist by the GPs in the intervention group than in the control group [36% vs. 76%, OR 0.2 (95% CI 0.1 to 0.6)] |
Reported associations between elements for logic model: The multifaceted intervention designed to address certain barriers to the implementation of the Dutch guideline for low-back pain for GPs was found to have minimal impact with regard to patient education, referral to a therapist, and prescription of pain medication, although the GPs studied here were already found to adhere to the guidelines to a fair extent |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Evans 200921 Country: UK Study design: Evaluation Data collection method: Referral data fed back to practices (patient attendance statistics) Aim: To engage GPs and consultants in discussions as to the validity, quality and appropriateness of GP referrals to increase the quality of referrals Detail of participants (number, any reported demographics): 9 of 13 interested practices selected by competitive tender |
Intervention: A year-long scheme where GPs were funded for weekly protected time to discuss their referrals retrospectively through peer review, and to attend six weekly cluster meetings where representatives from the practices met with consultants to discuss the appropriateness of the referrals and the use of alternative, community-based services Number of hours: Weekly protected time (1 hour a week) Delivered by who? NA Control: None Length of follow-up: 1-year intervention Response and/or attrition rate: NA Context (from what/who to what/who): GP referrals to hospital specialists (emergency and orthopaedics in all practices, plus one of paediatrics, gastroenterology and cardiology) |
Outcome measures: Quality of referral (appropriateness) |
Main results: There was, on average, a significant drop in referrals between the first and fourth quarters (z = 2.25, p = 0.025) The quality of referrals ad judged by doctors’ peers improved. Referral rates in orthopaedics showed a striking reduction of up to 50%. Variability between practices decreased and referral to local services increased Alternative community-based services were explored and an understanding of the best local pathways for some common conditions was reached |
Reported associations between elements for logic model: The authors suggest that this peer-review intervention was a more sustainable and intuitive method of improving referrals than referral management centres |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Evans 201122 Country: UK Study design: Service development project Data collection method: Data collected in Microsoft® Excel (Microsoft Corporation, Redmond, WA, USA) Aim: To use peer review with consultant engagement to influence GPs to improve the quality and effectiveness of their referrals Detail of participants (number, any reported demographics): 10 GP practices and seven specialties in Gwent 21 of 53 individual GPs were female (median age 44 years for females and 48.5 years for males) |
Intervention: GPs discussed the appropriateness of referrals in selected specialties including referral information, and compatibility with local guidelines, usually on a weekly basis, and were provided with regular feedback of benchmarked referral rates. Six weekly cluster groups, involving GPs, hospital specialists and community health practitioners discussed referral pathways and appropriate management in community-based services Number of hours: Six weekly groups Delivered by who? NR Control: None (some comparison with practices not taking part in the study) Length of follow-up: Study length 2008–9 Response and/or attrition rate: 10 of 13 GP practices. 53 of 58 individual GPs at the end of year 1 Context (from what/who to what/who): GP referral to seven specialties |
Outcome measures: Referral rate |
Main results: Overall, there was a reduction in variation in individual GP referral rates (from 2.7–7.7 to 3.0–6.5 per 1000 patients per quarter), and a related reduction in overall referral rates (from 5.5 to 4.3 per 1000 patients per quarter) However, although the highest individual referrers showed a decrease, the lowest referrers may show an increase in referrals (and a significant negative correlation comparing the first month’s data with the change from first to last month: r = 0.719, p = 0.019) Both reductions appeared sustainable while the intervention continued and referral rates rose in keeping with local trends once the intervention has finished |
Reported associations between elements for logic model: The peer-review intervention was effective and sustainable while the intervention continued. Subsequently, referral rates rose again in line with local trends |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Fearn 200948 Country: UK Study design: Before-and-after Data collection method: Semistructured interviews Aim: To investigate whether or not the introduction of the QOF Depression Indicators changed the pattern of referrals from primary care to a dedicated dementia clinic Detail of participants (number, any reported demographics): All referrals |
Intervention: Examined all referrals from primary care physicians seen in the cognitive function clinic for the 18-month period immediately preceding (November 2004 to April 2006) and following (May 2006 to October 2007) introduction of the QOF in April 2006 Control: NA Length of follow-up: 18-months Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to a dedicated dementia clinic |
Outcome measures: Referral |
Main results: The percentage of all referrals to the cognitive function clinic originating from primary care was about half in both time periods and did not differ significantly between the two time periods (χ2 = 0.88, df = 1, p > 0.1; z = 0.77, p > 0.05) Of the referrals from primary care, about one-third referred in both time periods had dementia. The RR of a diagnosis of dementia in a primary care referral pre and post QOF was 0.55 (95% CI 0.40 to 0.74) and 0.66 (95% CI 0.49 to 0.89), respectively The null hypothesis tested was that the proportion of patients referred from primary care with dementia was the same in cohorts seen both before and after introduction of the QOF Depression Indicator (equivalence hypothesis). The result of the chi-squared test did not permit rejection of the null hypothesis (χ2 = 0.54, df = 1, p > 0.05), a finding corroborated by the z-test (z = 0.60, p > 0.05) |
Reported associations between elements for logic model: No indication that QOF changed referral |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Felker 2004132 Country: USA Study design: Before-and-after Data collection method: Audit Aim: To evaluate a multidisciplinary mental health care team in primary care Detail of participants (number, any reported demographics): 9656 enrolled patients. Average age 53 years. 90% male PCP included 17 internal medicine physicians, 22 nurse practitioners, 10 internal medicine fellows and a variable number of residents |
Method: A multidisciplinary mental health team was created consisting of a psychologist, a psychology intern, psychiatry residents, clinical social workers and a chaplain Control: None Length of follow-up: 1 year before compared with 1 year after inception Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to mental health |
Outcome measures: Number of referrals to specialty mental health care services |
Main results: Before implementation 543 consultations occurred over the year. Of these, 543 (38%) were subsequently referred to specialty mental health care services The following year, 560 consultations occurred, but only 81 (14%) were referred The change in referral rate was significant: χ2 = 77.85, df = 1, p < 0.001 |
Reported associations between elements for logic model: A specialist mental health team in primary care reduced referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Ferris 2001145 Country: USA Study design: Before-and-after Data collection method: Record analysis Aim: To evaluate the elimination of a gatekeeping system Detail of participants (number, any reported demographics): Patients aged over 18 years, n = 59,997 at baseline, 29,999 intervention. Mean age 41.7 years; 53% female |
Intervention: Need for referral from a primary care provider removed. Patients able to call and schedule an appointment with any specialist in the group Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 6-month study period Response and/or attrition rate: NA Context (from what/who to what/who): Patient self-referral to specialist service |
Outcome measures: Absolute and relative utilisation of specialty services |
Main results: Rates of visits to specialists were stable during baseline period and during the intervention period First visits to specialists, however, increased slightly from 0.19 to 0.22 per patient per 6-month period (p < 0.001) The average proportion of visits to eligible specialists as a percentage of all visits was 29% during the year before removal of gatekeeping and 29.6% during the year afterwards (p = 0.39) |
Reported associations between elements for logic model: Patient direct access had small impact on initial assessment rates however little evidence of substantial increase in use of specialty services |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Ferris 2002146 Country: USA Study design: Before-and-after Data collection method: Routine data Aim: To investigate the impact of removing gatekeeping on specialist utilisation Detail of participants (number, any reported demographics): 59,952 patients |
Intervention: A multispecialty primary care group discontinued a gatekeeping system on 1 April 1998. The system was previously in place for 25 years Control: NA Length of follow-up: 6 months Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist |
Outcome measures: Overall number and distribution of patient visits to primary care and specialist |
Main results: Elimination of gatekeeping was not associated with changes in the mean number of visits to specialists (0.28 visits per 6 months before and after gatekeeping was removed), or the percentage of all children visits to specialists (11.6% vs. 12.1%, 95% CI 29.4 to 31.8 vs. 11.8 to 12.4) The proportion of all specialist visits that were initial consultations increased after gate keeping was removed from 39.6% (95% CI 29.4% to 31.8%) to 34.8% (95% CI 33.6% to 36.1%) Visits to any specialist by children with chronic disease increased from 18.6% (95% CI 17.7% to 19.1%) to 19.8% (95% CI 19.0% to 20.7%) New patient visits to specialists by children with chronic condition, as a percentage of all specialist visits increased from 28.1% (95% CI 25.9% to 30.2%) to 32.2% (95% CI 30.1% to 34.5%) |
Reported associations between elements for logic model: Removal of gatekeeping resulted in only minimal changes to utilisation of specialist care overall, but visits from children with chronic conditions increased |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Ferriter 2006157 Country: UK Study design: Before-and-after Data collection method: Referral audit Aim: To identify changes in the quality of information in referrals to an old age psychiatry service before and after the introduction of the single assessment process Detail of participants (number, any reported demographics): 20 consecutive new referrals from primary care to an old age psychiatry service in north-west London for the year before the new form was introduced (April 2003 to March 2004 – from 15 different general practices) and the following year (17 practices) |
Intervention: The single assessment process, a key element of the National Service Framework for Older People, was introduced to facilitate referrals between agencies and reduce duplication for patients, carers and clinicians. All referrals between agencies are now expected to be made on designated forms. Although there is no uniform national pro forma, many localities undertook rigorous consultation and development of referral forms, the use of which became mandatory for referrals to our service in April 2004. The referral form consists of several free-text sections: identity of patient and carer, identity of referrer; reason for referral; assessment of urgency; risk factors; current services provided to patient; diagnosis and recent history; current medication; signature of referrer Two senior clinicians performed independent and masked rating of each referral, using a five-point Likert scale of ‘strongly agree’ (1) to ‘strongly disagree’ (5). The raters answered the questions ‘I am able to judge the appropriateness of the referral’, ‘I would need to seek further information before processing this referral’ and ‘overall I think this referral is useful’ Number of hours: NA Delivered by who? NA Control: Before intervention Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Referral from primary care to old age psychiatry |
Outcome measures: Referral length, legibility, information and clinical utility |
Main results: The authors report that referrals were worse in all areas of quality of referral information after implementation of the single assessment process Word count decreased from 240 (SD 120) to 129 (SD 39), p = 0.005. Time to read in seconds increased from 96 (SD 40) to 124 (SD 41), p = 0.001. Illegible sections (% of) increased from 2 (10%) to 6 (30%), p = 0.011. The number of raters who strongly agreed or agreed with the following statements before, compared with after the single assessment process are as follows: ‘I am able to judge the appropriateness of the referral’ decreased from 19 to 5, p = 0.001 ‘I would need to seek further information before processing this referral’ increased from 3 to 17, p = 0.001 ‘Overall I think the referral is useful’ decreased from 17 to 3, p = 0.001 |
Reported associations between elements for logic model: The results of this small study suggest that the introduction of this particular single assessment process has impaired clinical communication between GPs and psychiatrists Note: It is clear that the practitioners did not ‘like’ the new process and it is unclear who the senior clinicians performing the ratings were |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Gandhi 2008108 Country: NR Study design: nRCT Data collection method: Survey Aim: Implementation of an electronic referral tool to analyse its impact on communication between primary care and specialists Detail of participants (number, any reported demographics): 430 referrals |
Intervention: Studied one practice site that implemented the referral tool and one that did not and surveyed affiliated specialists, PCPs and patients about referral communication Control: No electronic referral Length of follow-up: 2 years Response and/or attrition rate: Unclear Context (from what/who to what/who): Primary care to specialists |
Outcome measures: Referral communication |
Main results: Specialists more often received information before the referral visit from intervention PCPs vs. non-intervention PCPs (62% vs. 12%, p < 0.001), a finding that persisted after adjustment (OR = 3.3, p = 0.008) Intervention PCPs more often received communication from specialists (69% vs. 50%, p = 0.08) Patients of intervention PCPs were more likely than patients of control PCPs to report that specialists had received information before their visit (70% vs. 43%, p = 0.007) |
Reported associations between elements for logic model: Electronic referral can improve communication |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Glaves 200557 Country: UK Study design: Before-and-after Data collection method: Clinical data Aim: To determine if the use of request guidelines can achieve a sustained reduction in the number of radiographic examinations of the cervical spine, lumbar spine and knee joints performed for GPs Detail of participants (number, any reported demographics): All GPs referring to the three community hospitals |
Method: GPs referring to three community hospitals and a district general hospital were circulated with referral guidelines for radiography of the cervical spine, lumbar spine and knee, and all requests for these three examinations were checked. Requests that did not fit the guidelines were returned to the GP with an explanatory letter and a further copy of the guidelines. Where applicable, a large joint replacement algorithm was also enclosed. If the GP maintained the opinion that the examination was indicated, she or he had the option of supplying further justifying information in writing or speaking to a consultant radiologist Control: None Length of follow-up: 1 year Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for radiographic examinations of the cervical spine, lumbar spine and knee joints |
Outcome measures: Referral |
Main results: Over all sites and for all three examinations, the total number of examinations fell by 68% in the first year (95% CI 67% to 69%), achieving a 79% reduction in the second year (95% CI 78% to 80%). Knee radiographs fell by 64% in the first year (95% CI 62% to 65%), achieving a 77% reduction in the second year (95% CI 75% to 79%). Lumbar spine radiographs fell by 69% in the first year (95% CI 68% to 71%), achieving a 78% reduction in the second year (95% CI 77% to 80%). Cervical spine radiographs fell by 76% in the first year (95% CI 74% to 78%), achieving an 86% reduction in the second year (95% CI 84% to 88%). The p-value for all of these reductions was 0.0001 (highly significant). The largest individual reduction was 92% for cervical spine radiographs at Whitworth Hospital. The lowest reduction was 74% for lumbar spine radiographs at Buxton Hospital. For knee radiographs, the range was 75% to 86%; for lumbar spine radiographs, the range was 74% to 89%; and for cervical spine radiographs, the range was 80% to 92% |
Reported associations between elements for logic model: The use of referral guidelines, reinforced by request checking and clinical management algorithms, can produce a dramatic and sustained reduction in referral |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Gough-Palmer 200984 Country: UK Study design: Retrospective record analysis Data collection method: Analysis of GP requests for MRI scans Aim: To evaluate 12 years of GP open access to MRI scans Detail of participants (number, any reported demographics): 1798 scans requested by 209 GPs |
Intervention: GP access to MRI scans. No protocol, guidance or formal consultant or radiologist vetting Number of hours: NA Delivered by who? NA Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to MRI scanning |
Outcome measures: Number of referrals Type of scan Severity of reported findings |
Main results: GP-requested scans as percentage of workload of department are low (around 2.6%). While workload of department increased over study period, this percentage remained stable Spine, knee and brain imaging were 86% of requests. 48% of scans requested were normal or minor degenerative changes. 26% demonstrated serious pathology warranting hospital referral Wide range of scans requested per requester; average 8.5, varied from 1 to 240 |
Reported associations between elements for logic model: Marked discrepancy between GPs, suggesting need for referral guidelines While the rate of no identified abnormality was 48%, a normal scan could be beneficial in providing rapid patient reassurance, return to work and a reduction in outpatient referrals Demand for complex areas very low |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Greiver 2005114 Country: UK Study design: cRCT Data collection method: NR Aim: To determine the effectiveness of a PDA software application to help family physicians to diagnose angina among patients with chest pain Detail of participants (number, any reported demographics): 18 family physicians belonging to the North Toronto Primary Care Research Network (Nortren) or recruited from a local hospital |
Intervention: Intervention physicians received a Palm PDA (which included the angina diagnosis software). Physicians prospectively recorded the process of care for patients aged 30 to 75 years presenting with suspected angina, over 7 months Number of hours: NA Delivered by who? NA Control: Continue conventional care Length of follow-up: 7 months Response and/or attrition rate: NR Context (from what/who to what/who): GP to cardiology |
Outcome measures: Frequency of cardiac stress test orders for suspected angina The appropriateness of referral for cardiac stress testing at presentation and for nuclear cardiology testing after cardiac stress testing Secondary outcome was referrals to cardiologists |
Main results: 14 of the 28 patients in the control arm (50%) and 30 of the 37 patients in the PDA arm (81%) were referred for cardiac stress tests (p = 0.007), an absolute difference of 31% (95% CI 8% to 58%) There was a trend towards more appropriate use of stress testing (48.6% with the PDA vs. 28.6% control), an increase of 20% (p = 0.284, 95% CI –11.54% to 51.4%). There was also a trend towards more appropriate use of nuclear cardiology following cardiac stress testing (63.0% vs. 45.5%), an absolute increase of 17.5% (p = 0.400, 95% CI –13.9% to 48.9%) Referrals to cardiologists did not increase (38.2% with the PDA vs. 40.9%, p = 0.869). A referral was more likely to have been made if the final diagnosis was angina (likelihood ratio for referral 15.455, 95% CI 2.124 to 112.431); in other words, family physicians appeared to refer appropriately |
Reported associations between elements for logic model: A PDA-based software application can lead to improved care for patients with suspected angina seen in family practices; this finding requires confirmation in a larger study |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Griffiths 200658 Country: UK Study design: cRCT Data collection method: Dermatologists completed a standardised pro forma on all patients seen in their clinic with a diagnosis of psoriasis who had been referred from primary care Aim: To assess the effectiveness of guidelines and training sessions on the management of psoriasis in reducing inappropriate referrals from primary care Detail of participants (number, any reported demographics): Patients aged 18 years or over with psoriasis (165 health centres). n = 188 |
Intervention: Health centres in the intervention arm received guidelines on the management of psoriasis in primary care, developed by local dermatologists, supplemented by the offer of a practice-based nurse-led training session; those in the control arm received neither guidelines nor training sessions Number of hours: NR Delivered by who? Training delivered by nurse Control: No intervention Length of follow-up: Response and/or attrition rate: Outcome data were available for 188 of the 196 eligible patients Context (from what/who to what/who): Referral from primary care to dermatology for psoriasis |
Outcome measures: Anonymised pro formas were assessed by three members of an expert panel, comprising a dermatologist, a GP and a dermatology specialist nurse Referral was considered appropriate if the patient fulfilled any of the following criteria: extent of disease 20% of body surface area; unstable disease; no improvement following topical treatment (as per guidelines, for 6–8 weeks); or, when following dermatological assessment, the patient was admitted to hospital, was referred to our day treatment centre or to the dermatology nursing service, or received phototherapy or systemic therapy |
Main results: 82 health centres were randomised to the intervention arm and 83 were randomised to the control arm. Outcome data were available for 188 of the 196 eligible patients referred during the study period Patients in the intervention arm (82/105) were significantly more likely to be appropriately referred in comparison with patients in the control arm 49/83) (difference = 19.1%; OR 2.47; 95% CI 1.31 to 4.68; ICC = 0) Only 25 (30%) health centres in the intervention arm took up the offer of training sessions There was no significant difference in outcome between health centres in the intervention arm that received a training session and those that did not (OR 1.28, 95% CI 0.50 to 3.29; ICC = 0) |
Reported associations between elements for logic model: Dissemination of guidelines on the management of psoriasis in primary care improved the appropriateness of referral of patients to secondary care |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Gurden 2012133 Country: UK Study design: Before-and-after Data collection method: Questionnaire Aim: To describe and evaluate a community-based musculoskeletal service Detail of participants (number, any reported demographics): n = 696. Adult patients presenting to their GP with back or neck pain; mean age 52 years; two-thirds female; just over half in paid employment |
Intervention: Patients still having pain after 4–6 weeks’ ‘usual GP care’ offered a course of manual therapy and referred to private provider of their choice. Seen within 14 days Number of hours: Six treatments over 8 weeks. Practitioners worked to agreed protocols Delivered by who? Independent providers of chiropractic, osteopathy and physiotherapy services Control: None Length of follow-up: Until discharge from service (usually 8 weeks) Response and/or attrition rate: 696 of the 2810 seen by the service Context (from what/who to what/who): Community-based musculoskeletal service |
Outcome measures: Bournemouth Questionnaire (for back and neck pain) Bothersome Scale Global Improvement Scale Patient satisfaction with treatment |
Main results: Percentage change in scores baseline to discharge – Bournemouth Questionnaire = 64.6% patients categorised as improved, Bothersome Scale = 69.9% categorised as improved, Global Improvement Scale = 67.8% improved 99.5% satisfied or very satisfied with the treatment, 3% referred back to GP with recommendation for referral to secondary care services 97% given self-management advice and recommended for discharge ‘evaluation by PCT demonstrated reduced primary care consultations, imaging and inappropriate referrals to secondary care’ |
Reported associations between elements for logic model: Referrals to spinal surgeons reduced by more than 25% |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hands 200134 Country: UK Study design: Before-and-after Data collection method: Surveys Aim: To evaluate the effectiveness of an education interaction between consultants and GPs Detail of participants (number, any reported demographics): 22 consultants; 21 GPs |
Intervention: GPs attended outpatient sessions in different clinical specialties of their choice. Completed a questionnaire immediately after the session and at 6 months Control: NA Length of follow-up: 6 months Response and/or attrition rate: 21/150 Context (from what/who to what/who): GP to specialist |
Outcome measures: Referral |
Main results: GPs reported changes in their clinical behaviour which appear to have been maintained at 6 months GPs stated that referral was discussed/taught in 83% of interactions. Immediately after the session, 25% of GPs thought that this would change their referral behaviour. After 6 months, 29% reported behaviour change in reference to referral Behaviour change was also reported with regard to diagnosis (42%), management (79%), prescribing (54%), and practical skills (58%) |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Harrington 200193 Country: USA Study design: Case series Data collection method: Review of patient records, survey of patient views Aim: To evaluate the impact of guidelines on referral and a referral management programme Detail of participants (number, any reported demographics): Patients with low-back pain, records of 581 patients reviewed over 1 year |
Intervention: Referral management programme – guidelines for referral including a flow chart (algorithm) for care, plus system for separating urgent cases from others – physician contacts surgeon or managers for advice on patients with red flag symptoms as per guidelines, receptionist takes information, information verified by nurse co-ordinator, physician manager reviews information obtained to determine care plan instigated by nurse co-ordinator Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 3 years’ data reviewed, 1 year pre, transition year, year after implementation Response and/or attrition rate: NA Context (from what/who to what/who): GP to spine orthopaedists |
Outcome measures: Patient visits for low-back pain to either a primary care or a specialist care provider |
Main results: Following introduction of the guidelines little change was documented from traditional referral patterns (no other information). Three years later in response to long waiting lists the referral management programme was put in place Shift of care from spine orthopaedists to primary physicians. Before, 28% of patient visits for low-back pain were to specialist care and 72% were to primary care. During transition year 13% of patient visits were to specialist care and 87% were to primary care Year after implementation 17% of visits were to specialist care and 83% of visits were to primary care Total patient visits for low-back pain increased 16% over the time period from 7988 to 9297. Estimated cost saving of $4000,000 per year in manpower cost. 90% of patients were satisfied with referral management process |
Reported associations between elements for logic model: Pre-appointment management can reduce specialty appointments (although shifts appointments to primary care) |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Heaney 2001159 Country: UK Study design: RCT Data collection method: Use of health services audited from patients’ general practice notes in 12 months after receipt of booklet Aim: To investigate the effect of patient information booklets on overall use of health services Detail of participants (number, any reported demographics): 20 general practices in Lothian, Scotland Random sample of patients from the community health index (n = 4878) and of those contacting out-of-hours services (n = 4530) in the previous 12 months in each of the study general practices |
Intervention: Booklets were posted to participants in intervention groups (3288 were sent ‘What Should I Do?’ and 3127 were sent ‘Health Care Manual’). Patients randomised to control group (2993) did not receive a booklet ‘What Should I Do?’ was part of a patient education programme implemented in the Netherlands in 1993. The booklet outlines 40 common health problems and provides information on when to consult a doctor and on self-care, when appropriate ‘Health Care Manual’ was developed by a GP and a practice nurse in Dunkeld, Scotland. It outlines about 50 common health problems and also provides information about keeping healthy Control: No booklet Length of follow-up: 12 months Response and/or attrition rate: The final response rate from general practices was 20/30 (67%) Context (from what/who to what/who): Patient use of GP services |
Outcome measures: Types of service use, interactions between use, deprivation category of the area in which respondents live, and age |
Main results: Receipt of either booklet had no significant effect on health service use compared with a control group Total contacts: Book – before 4.19, after 4.20 Control – before 3.95, after 3.91 Difference (95% CI) 0.14 (–0.18 to 0.45) However, 9 out of 10 matched practices allocated to receive Health Care Manual had reduced consultation rates compared with matched practices allocated to ‘What Should I Do?’ |
Reported associations between elements for logic model: Widespread distribution of information booklets about the management of minor illness is unlikely to reduce demand for health services |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hemingway 200673 Country: UK Study design: Before-and-after Data collection method: NR Aim: To evaluate a protocol-driven referral system for colorectal cancer tests Detail of participants (number, any reported demographics): eight colorectal surgeons and 10 GI physicians |
Intervention: Leicester Colorectal Test Protocol – included list of presenting symptoms, age criteria for test and appropriate diagnostic test for each symptom. Patients had investigation before seeing outpatient clinician or on the day of the clinic. Referrals processed by ‘2-week wait’ administration staff using the protocol and assessments booked by these administration staff Protection of slots within the testing suites Referrals not complying with protocol were redirected to appropriate test without referral back to GP Number of hours: NA Delivered by who? Predominantly administrators in department Control: None Length of follow-up: Up to 2-year period Response and/or attrition rate: NA Context (from what/who to what/who): GP to colorectal outpatient clinic |
Outcome measures: Time referral to diagnosis Percentage of patients referred as urgent who were seen within 31-day target timescale |
Main results: Data for intervention period were not clear; reported by year rather than before and after Baseline before protocol: Year 1 median time to diagnosis non-emergencies 35 days (interquartile range 13–80), fast-track (categorised as 2-week wait or ‘soon’) 21 days (10–48) 62% of cancers referred as either 2-week wait or ‘soon’ were diagnosed within 31 days Year 2 non-emergencies 22 (9–59) emergencies 15 (7–37) After introduction of protocol (pilot and full implementation): Year 3 non-emergencies 20 (10–59) emergencies 13 (8–29) Year 4 non-emergencies 20 (10–51) emergencies 13 (9–23) During the 2-month full implementation period during year 3 service received 256 referrals, 64% came through 2-week wait protocol office and 36% referred directly to consultants. In these referrals 70% were diagnosed with a pathology and 19 patients were diagnosed with cancer, all within 31 days Overall, during year 3 79% of patients with colorectal cancer diagnosed who were referred as 2-week wait or ‘soon’ were diagnosed within 31 days; in year 4 the figure was 82% |
Reported associations between elements for logic model: Protocol-driven intervention had a positive impact on speed of diagnosis |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hermush 2009137 Country: Israel Study design: Before-and-after Data collection method: Retrospective examination of patient data Aim: To describe and evaluate a new model used in caring for the elderly in the community Detail of participants (number, any reported demographics): n = 512 elderly patients; mean age 79 years; 66% female |
Intervention: GP refers difficult or complex cases to a geriatrician who carries out a clinic in the same primary care location Number of hours: NA Delivered by who? Geriatrician Control: None Length of follow-up: Data collected over 3 years Response and/or attrition rate: 5086 patients over 65 years treated in the time frame Context (from what/who to what/who): Large primary care clinics in a city to geriatrician |
Outcome measures: Number of referrals Type of clinical problem |
Main results: Referrals to geriatrician increased significantly from 133 at baseline to 207 2 years later (p = 0.01) Number of visits to GP decreased in the 6 months following the consultation with the geriatrician (p < 0.01) |
Reported associations between elements for logic model: Relocation of specialist service to primary care can increase referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hill 200049 Country: UK Study design: Before-and-after (audit) Data collection method: GP audit Aim: To assess how appropriate referrals were just before and after distribution of the guidelines Detail of participants (number, any reported demographics): 33 GP practices. Data on 155 patients pre distribution of guidelines and 153 patients post distribution. In the 2-year follow-up audit, a sample of 114 new patients, seen consecutively over a period of 3 weeks, was taken |
Intervention: Referral guidelines for dermatology were compiled by the dermatologist at the Royal Surrey County Hospital in consultation with local GPs. An audit was undertaken to assess how appropriate referrals were just before and after distribution of the guidelines and was repeated 2 years later to determine whether or not they had made any significant impact Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 2 years Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to dermatology |
Outcome measures: Appropriate referrals |
Main results: In the original audit a 40% increase in the numbers of referrals which were recorded by the dermatologist as appropriate immediately after the guidelines were sent (from 57% to 80%) was seen. The 2-year follow-up audit, however, demonstrated that the improvement had not been sustained, with a decline to 48% appropriate referrals Five common conditions accounted for two-thirds of inappropriate referrals before and after the guidelines were sent |
Reported associations between elements for logic model: In response to referral guidelines, appropriate referrals increased in the short term but did not persist The need for continued GP education in dermatology to reinforce referral guidelines is demonstrated |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hilty 200624 Country: USA Study design: Before-and-after Data collection method: Videoconferencing consultations Aim: To assess changes over time in the utilisation of telepsychiatric services by individual PCPs and clinics in rural areas Detail of participants (number, any reported demographics): First 200 and the subsequent 200 telepsychiatric initial consultations |
Intervention: 400 consecutive patients received an initial telepsychiatric consultation delivered from an academic medical centre to rural or suburban primary care sites from July 1996 to December 2002 The following educational strategies were implemented:
Control: NA Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to psychiatry |
Outcome measures: Patient demographics, diagnoses, reason for consultation, medication dosing and satisfaction |
Main results: Adult patients were primarily referred for mood and anxiety disorders, particularly for diagnosis and medication treatment planning. Over time, PCPs significantly improved medication dosing and asked for more treatment planning help. PCPs’ satisfaction also improved over time Among the first 200 consultations, only 47.4% of the medication doses for depressive and anxiety disorders were adequate, according to national guidelines. Among the second 200 consultations, dosing adequacy improved to 63.6% (p < 0.001) PCPs rated the quality of consultation as significantly higher over time (95% Cl 4.45 to 4.83; p < 0.001), and likewise with overall satisfaction (95% Cl 4.49 to 4.73; p < 0025) |
Reported associations between elements for logic model: Telepsychiatric consultation, in combination with specific educational interventions, appears to facilitate the enhancement of skills and knowledge of PCPs |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hockey 200491 Country: Australia Study design: Longitudinal (no control) evaluation Data collection method Aim: To examine the feasibility of a low-cost store and forward teledermatology service for GPs in regional Queensland Detail of participants (number, any reported demographics): 63 referrals |
Intervention: Digital pictures and a brief case history were transmitted by e-mail. A service co-ordinator carried out quality-control checks and then forwarded the messages to a consultant dermatologist. The co-ordinator returned the message to the GP. The aim was to provide advice to rural GPs within 1 working day Control: None Length of follow-up: None (6-month study) Response and/or attrition rate: NA Context (from what/who to what/who): GP to dermatology |
Outcome measures: Referral |
Main results: Over 6 months, 63 referrals were processed by the teledermatology service. In the majority of cases, the referring doctors were able to treat the condition after receipt of e-mail advice from the dermatologist. In 10 cases (16%) additional images or biopsy results were requested because image quality was inadequate The average time between a referral being received and clinical advice being provided was 46 hours |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hughes-Anderson 2002136 Country: Australia Study design: Before-and-after Data collection method: Prospective data collection from all patients undergoing upper and lower endoscopy procedures between January 1996 and June 2000 Aim: To assess whether or not an outreach surgical service offering open-access endoscopy to rural areas was being overutilised Detail of participants (number, any reported demographics): A total of 4400 patients were seen by the outreach programme in the 5 years 1996–2000 The mean age of patients was 50.8 years (range, 15–94 years); 45% were women |
Intervention: Indications for referral between the GPs and the visiting surgeons were reviewed in patient records and assessed for compliance with the American Society for Gastrointestinal Endoscopy (ASGE) guidelines Two groups of patients were defined: those referred directly for open-access endoscopy and those selected by the surgeons. The open-access endoscopy patients were assessed on the day, prior to the procedure by the visiting surgeon Records for all patients undergoing colonoscopy were reviewed to determine the reason and number of cancelled procedures Control: None Length of follow-up: January 1996 to June 2000 Response and/or attrition rate: NR Context (from what/who to what/who): GP referral for endoscopy |
Outcome measures: The groups were analysed for appropriateness of referrals and frequency of positive pathology investigations |
Main results: A total of 772 endoscopies were performed and 75% were booked as open-access services. The referral rate for procedures was greater for GPs (583: 75%) than for the visiting surgeons (189: 25%) The overall compliance rate for approved indications using the ASGE guidelines for both groups was 92%. There was no significant difference in pathology found between groups The appropriateness of referrals for colonoscopy indicated that 28 of the colonoscopies were outside the ASGE indications. There was no significant difference between the two groups on the basis of the guidelines Difference between GP and visiting surgeon (appropriate indications) for endoscopy is 3.2%, 95% CI –1.8% to 8.2%; p = 0.34827, not significant Difference between GP and visiting surgeon (appropriate indications) for colonoscopy is 6.8%, 95% CI –1.8% to 15.4%; p = 0.14782, not significant |
Reported associations between elements for logic model: Outreach surgical service did not induce unnecessary procedures |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Idiculla 200044 Country: UK Study design: retrospective survey Data collection method: Referral letter contents Aim: To ascertain whether or not local guidelines for diabetes management influence the content of GP referral letters to a diabetes specialist clinic Detail of participants (number, any reported demographics): 400 GP referral letters |
Intervention: Analysis of 200 GP referral letters submitted before (set 1) and 200 submitted after (set 2) local guidelines on the management of adult diabetes had been issued to local GPs The frequency with which micro- and macro-vascular complications of diabetes were documented in the GP letters was compared with frequency ascertained at the first attendance at the specialist clinic Control: None Length of follow-up: None Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for diabetes complications |
Outcome measures: Content of referral letters |
Main results: Following the distribution of the guidelines there was no significant change in the frequency with which specific conditions were documented in referral letter (set 1 vs. set 2): hypertension 72 vs. 79%, cerebrovascular disease 89 vs. 80%, etc. Many unreported complications were found in painters who had been referred after various periods of treatment in primary care However, the guidelines did appear to have encouraged the active treatment of hyperglycaemia by GPs before referral |
Reported associations between elements for logic model: Diabetes guidelines had very little effect on increasing the information provided in GP referral letters |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Imkampe 200647 Country: UK Study design: Before-and-after Data collection method: Aim: To determine whether or not GP grading of referrals into urgent and non-urgent had improved after the introduction of the 2-week rule was introduced Detail of participants (number, any reported demographics): All new GP referrals |
Intervention: A retrospective review of GP referrals over 8 months, between September 2003 and April 2004, with regard to their urgency, subsequent diagnosis and the use of pro formas (standardised referral formats) was carried out. The results were compared with the 1999 audit Control: None Length of follow-up: None Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for breast cancer |
Outcome measures: Appropriate referral |
Main results: 82 of 1178 patients referred by GP had breast cancer vs. 115 of 1176 patients referred in 1999. Sixty-eight per cent (56/82) of breast cancer patients were referred as urgent, compared with 47% (54/115) in 1999 (p = 0.005) A pro forma was used in 47% (548/1178) of GP referrals, while no pro forma was used in 1999 Sixty-five of the 82 cancer patients were referred with a pro forma and 85% (55/65) were referred as urgent |
Reported associations between elements for logic model: GP prioritisation of referrals has improved since 1999. With the use of pro formas a significant number of patients with cancer were referred urgently |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Iversen and Luras 2000151 Country: Norway Study design: Economic analysis and modelling Data collection method: NR Aim: To explore whether or not the payment system for GPs has an impact on referral Detail of participants (number, any reported demographics): 150 GPs across four municipalities |
Intervention: Change from contract system (whereby GP receives a fixed practice allowance plus charges fee per item to each patient) to a capitation system where each person registers with a particular GP and GP income based on the number of patients on their list Number of hours: NA Delivered by who? NA Control: None Length of follow-up: Study over 3 years Response and/or attrition rate: 37% of GPs who took part in the intervention provided data Context (from what/who to what/who): GP to specialist |
Outcome measures: Number of referrals to specialists |
Main results: In the capitation system where GP income is determined by number of patients on list the GP referral rates to specialists increased by 42%. It was hypothesised that it is less profitable for the GP to provide services themselves and more profitable for them to let the specialists provide the services Number of years GP practised in the area did not have a significant impact on referral rate |
Reported associations between elements for logic model: Model of GP payment and referral rate |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Jaatinen 200295 Country: Finland Study design: RCT Data collection method: Questionnaires Aim: To consider teleconsultation as a replacement for referral to an outpatient clinic Detail of participants (number, any reported demographics): 93 patients. After non-attendance, n = 54 intervention and n = 24 control |
Intervention: GPs had to decide whether to refer for electronic consultation with the hospital, or whether to refer to outpatients as usual. Electronic communication with the hospital was through a secure web-based system Control: Conventional referral letter sent to hospital outpatient clinic Length of follow-up: None – 5-month study Response and/or attrition rate: 15 non attendees Context (from what/who to what/who): GP to specialist |
Outcome measures: Referral satisfaction |
Main results: All patients treated by teleconsultation said they wanted the same procedure in the future and 63% of the control group said they would prefer a teleconsultation next time (p = 0.02), although they were nearly as satisfied as those who received a teleconference (p = 0.37) The doctors quickly learned to exploit the telecommunication model. The responsibility for treatment was maintained with the primary-care centre in 52% of cases using teleconsultation without any hospital visit required. The GPs and doctors agreed on follow-up treatment |
Reported associations between elements for logic model: Teleconsultation increased the probability of GPs maintaining responsibility for treatment |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Jiwa 200423 Country: UK Study design: nRCT Data collection method Aim: To determine if written feedback about the contents of GP referral letters mediated by local peers was acceptable to GPs and how this feedback influenced the content and variety of their referrals Detail of participants (number, any reported demographics): 26 GPs |
Intervention: In a controlled trial, 26 GPs were offered written feedback about the documented contents of their colorectal referral letters over 1 year. The feedback was designed and mediated by two nominated local GPs. The contents of referral letters were measured in the year before and 6 months after feedback. GPs were asked about the style of the feedback. The contents of referral letters and the proportion of patients with organic pathology were compared for the feedback GPs and other local GPs who could be identified as having used the same hospital for their referrals in the period before and after feedback Control: Control subjects were up to 50 practitioners who referred to the same local district general hospital Length of follow-up: 1 year before and 6 months after feedback Response and/or attrition rate: None withdrew from the project Context (from what/who to what/who): GP referral to specialist |
Outcome measures: Referral letter quality |
Main results: All GPs declared the method of feedback to be acceptable but raised concerns about their own performance, and some were upset by the experience There was a difference of 7.1 points (95% CI 1.9 to 12.2 points) in the content scores between the feedback group and the controls after adjusting for baseline differences between the groups There was a considerable improvement in the content of the referral letters from the feedback group from before to after feedback as illustrated below. There was no improvement in the scores for the control group in the same period Feedback group/control group: Mean scores before feedback 34.1/28.2 Mean scores after feedback 39.5/28.7 Mean difference and CIs 5.3 (1.5 to 9.2)/0.55 (–1.4 to 2.5); t-test df 20/36; p = 0.008/0.6 Of the GPs who referred to the same hospital before and after feedback, the feedback GPs referred more patients with organic pathology than other local colleagues |
Reported associations between elements for logic model: In some cases feedback improves the content of GP referral letters and may also impact on the type of patients referred for investigation by specialists |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Jiwa 200668 Country: UK Study design: cRCT (clustered by practice) Data collection method: Semistructured interviews were conducted to identify key themes relating to the use of the software. Questionnaire to practitioners and interview Aim: To evaluate a referral guideline intervention for lower bowel symptoms Detail of participants (number, any reported demographics): From 150 invitations, 44 practices were recruited with a total list size of 265,707. 44 practices with 180 GPs and 504 patients over 6-month period. GPs aged 30–60 years |
Intervention: Practices were offered an electronic interactive referral pro forma, an educational outreach visit by a local colorectal surgeon, both or neither 1. Pro forma: They developed and piloted an interactive electronic pro forma for processing referrals to colorectal surgeons (General Practice Referral Assessment Facilitator or G-RAF). The interactive pro forma requested information on drop-down menus for 15 clinical signs and symptoms previously identified by GPs and colorectal surgeons as those of significant colorectal disease. The interactive software offered the practitioner guidance on which cases needed urgent referral with reference to current UK Department of Health guidelines. A referral letter was automatically produced seeking an appropriate appointment at a hospital clinic 2. Education: A colorectal surgeon delivered short educational sessions. During the 45-minute educational outreach meeting, the presenter summarised the features of significant organic colorectal disease and encouraged questions. The published guidelines and the potential for the improvement to the management of patients were emphasised 3. Both interventions Number of hours: Education = 45 minutes Delivered by who? Local colorectal surgeon Control: No intervention Length of follow-up: NR Response and/or attrition rate: From 150 invitations, 44 practices were recruited Context (from what/who to what/who): GP referral to a colorectal surgeon |
Outcome measures: The main outcome measure was the proportion of cases with severe diverticular disease, cancer or precancerous lesions and inflammatory bowel disease in those referred by each group A secondary outcome was a referral letter quality score |
Main results: There were 716 consecutive referrals recorded over a 6-month period, of whom a diagnosis was available for 514 In the combined software arms 14% (37/261) had significant pathology, compared with 19% (49/253) in the non-software arms, RR 0.73 (95% CI 0.46 to 1.15) In the combined educational outreach arms 15% (38/258) had significant pathology compared with 19% (48/256) in the non-educational arms, RR 0.79 (95% CI 0.50 to 1.24) Pro forma practices documented better assessment of patients at referral Pilot work suggested proportion of patients referred with significant pathology is approximately 0.14. Only 18% of referrals in intervention one arm used the software. No significant difference in proportion of cases with significant pathology for either intervention or compared with no intervention Point estimates suggest that the interventions performed worse than no intervention. About a 4% absolute improvement; intervention arms could give as much as a 7% lower absolute percentage in referrals with significant pathology than control Themes in interviews: Concerns regarding the pro forma creating an additional task in the process – quicker to dictate a letter, and therefore poor adoption of the software The ‘don’t know if it is cancer’ option led to processing as an urgent referral, whereas most referrals were because the GP did not know for sure – potential overuse of urgent referral slots |
Reported associations between elements for logic model: There was a lack of evidence that either intervention increased the proportion of patients with organic pathology among those referred. The interactive software did improve the amount of information relayed in referral letters although we were unable to confirm if this made a significant difference to patients or their health-care providers The potential value of either intervention may have been diminished by their limited uptake Computer pro forma systems unpopular as administrative burden shifted to clinicians from administrative staff Study did not take account of how innovation was to be used in practice and impact on professional identify and established practices |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Jiwa 2012105 Country: Australia Study design: Before-and-after study Data collection method: Analysis of referral letters Aim: To explore if increasing the amount of relevant information in referral letters between GPs and hospital specialists helps in the scheduling of appointments for patients Detail of participants (number, any reported demographics): NR |
Intervention: Referral Writer Software – a software system to assist referral writing, consisting of a pro forma that selects relevant information from the electronic patient record and requests the doctor to choose one of six specialties for referral: urology, breast, gynaecology, upper GI, colorectal and respiratory. The doctors were finally prompted to enter details about the patient’s condition The amount of relevant information in the referral letters were assessed with reference to a published schedule 3 months before and 4 months after the intervention start date The letters were scored by a researcher for the amount of relevant information and independently checked by two specialists to determine if the urgency of the referral could be established, and what the most likely outcome was. This was later compared with the actual diagnosis Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 4 months after intervention start Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist |
Outcome measures: Relevant information in referral |
Main results: Each GP referred 5.6 patients on average (range 1–14) before the RW and 4.8 patients (range 0–14) after the RW. The amount of relevant information in the referrals improved substantially after the RW, mean difference 37%, 95% CI 43% to 30%; p < 0.001 For 91% of referrals after the RW, both specialists in each specialty were confident or very confident that they had enough information to decide when the patient should come to their clinic; this increased from 50% before RW, p = 0.001 There was no association between the amount of relevant information and the final diagnosis |
Reported associations between elements for logic model: Standardising and using electronic communications to refer appears to facilitate referral scheduling of specialist appointments |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Johnson 2008139 Country: UK Study design: Cross-sectional analysis of services and referral patterns Data collection method: Survey Aim: Does the provision of acupuncture in primary care reduce need for referral? Detail of participants (number, any reported demographics): Three PCTs; rural and urban mix. 109 practices; 13% offered acupuncture service |
Intervention: Acupuncture service in primary care Number of hours: NA Delivered by who? Acupuncture clinic Control: None Length of follow-up: NA Response and/or attrition rate: 57% response to first e-mail, 73% to second e-mail, remaining practices contacted by telephone Context (from what/who to what/who): Specialist clinic provided in primary care |
Outcome measures: Referral rate to orthopaedic, pain, physiotherapy, rheumatology Cost of painkillers |
Main results: ‘No evidence from the data that provision of acupuncture is associated with lower referral rates’ Note: Data presented outline mean referral rates for practices providing acupuncture clinics and ‘some’ versus ‘higher’ number of acupuncture appointments but NOT practices with no acupuncture, so this conclusion needs modification. Wide variation between different PCTs. Variation between PCTs possibly associated with local differences in referral patterns and sociodemographic characteristics |
Reported associations between elements for logic model: Variation between referral rates between providers |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Joyce 2000147 Country: USA Study design: Retrospective study Data collection method: Retrospective analysis of routine patient data Aim: To assess utilisation of ambulatory visits to primary care physicians and to specialists in two different managed care models – a closed-panel gatekeeper health maintenance organisation (HMO) and an open-panel point-of-service HMO Detail of participants (number, any reported demographics): 16,192 working-age members of the gatekeeper HMO and 36,819 working-age members of the point-of-service HMO |
Intervention: Retrospective study of patients enrolled in a single managed care organisation with two distinct product lines – a gatekeeper HMO and a point-of-service HMO. Both plans shared the same physician network Estimated the number of primary care physician and specialist visits using negative binomial regression models and predicted the number of visits per year for each person under each HMO type and copayment option Number of hours: NA Delivered by who? NA Control: Two different managed care models Length of follow-up: 2 years Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to specialists |
Outcome measures: Referral, number of visits |
Main results: There were more annual visits to primary care physicians and a greater number of total physician visits in the gatekeeper HMO than in the point-of-service plan. However, they did not observe higher rates of specialist visits in the point-of-service HMO |
Reported associations between elements for logic model: No evidence that direct patient access to specialists leads to higher rates of specialty visits in plans with modest cost-sharing arrangements |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Julian 200762 Country: UK Study design: nRCT Data collection method: Referral data and patient diaries Aim: To examine the outcomes of an integrated model that lends weight to GP-led care Detail of participants (number, any reported demographics): Large teaching hospital and GP practice; 99 Bridges, 94 one-stop menstrual clinic |
Intervention: Women attending the new Bridges pathway were compared with those attending a consultant-led one-stop menstrual clinic The Bridges pathway involved the use of shared care evidence-based guidelines for the management of patients in primary and secondary care, which determined the timings for investigations and surgical treatment. Management decisions were made by GPs in all but atypical/complex cases Control: Consultant-led one-stop clinic Length of follow-up: 8 months Response and/or attrition rate: 8/89 GPs declined Context (from what/who to what/who): GP to gynaecology |
Outcome measures: Outpatient appointments Clinical outcomes Patient views |
Main results: At 8 months there were no significant differences between the groups in terms of surgical and medical treatments of in the use of GP clinic appointments. Significantly fewer hospital outpatient appointments were made in the Bridges group than in the one-stop menstrual clinic (p < 0.001) The patient diaries demonstrated a significant improvement in the Bridges group for patient information, ease of access (p < 0.001), choice of doctor (p < 0.002), waiting time (p < 0.001) and less ‘limbo’ between primary and secondary care (p < 0.001) |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Junghans 2007109 Country: UK Study design: RCT Data collection method Aim: The effect of patient-specific ratings vs. conventional guidelines on appropriate investigation of angina Detail of participants (number, any reported demographics): n = 145 physicians |
Method: RCT of 145 physicians receiving patient-specific ratings (online prompt stating whether the specific vignette was considered appropriate or inappropriate for investigation, with access to detailed information on how the ratings were derived) and 147 physicians receiving conventional guidelines from the American Heart Association and the European Society of Cardiology. Physicians made recommendations on 12 web-based patient vignettes before and on 12 vignettes after these interventions Control: Conventional guidelines Length of follow-up: NR Response and/or attrition rate: NR Context (from what/who to what/who): GP referral for angina |
Outcome measures: Proportion of appropriate investigative decisions as defined by two independent expert panels |
Main results: Decisions for exercise electrocardiography were more appropriate with patient-specific ratings [819/1491 (55%)], compared with conventional guidelines [648/1488 (44%)] (OR 1.57; 95% CI 1.36 to 1.82). The effect was stronger for angiography [1274/1595 (80%) with patient-specific ratings, compared with 1009/1576 (64%) with conventional guidelines (OR 2.24, 95% CI 1.90 to 2.62)] Within-arm comparisons confirmed that conventional guidelines had no effect but that patient-specific ratings significantly changed physicians’ decisions towards appropriate recommendations for exercise electrocardiography (55% vs. 42%; OR 2.62, 95% CI 2.14 to 3.22) and for angiography (80% vs. 65%; OR 2.10, 95% CI 1.79 to 2.47) These effects were robust to physician specialty (cardiologists and GPs) and to vignette characteristics, including older age, female sex and non-white race/ethnicity |
Reported associations between elements for logic model: Patient-specific ratings result in more appropriate investigations of angina than conventional guidelines |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Kennedy et al. 2012106 Country: UK (Scotland) Study design: Retrospective audit Data collection method: Analysis of electronic referral system data over 1 year Aim: To evaluate an electronic referral system Detail of participants (number, any reported demographics): n = 190 patients referred with suspected squamous cell carcinoma of the head and neck; 55% female; aged 19 to 92 years; mean age 58 years |
Intervention: A fast-track electronic referral system including referral guidelines. Suspected Cancer Urgent Referral Electronically System containing specific alarm symptoms Number of hours: NA Delivered by who? Electronic system Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): From GP to head and neck cancer clinic |
Outcome measures: Appropriateness of referral: number of patients referred who were subsequently diagnosed with cancer |
Main results: 52% of the urgent referrals required no further investigation following assessment and were discharged Head and neck cancer detection rate (% of patients with confirmed diagnosis from total number of referrals) was 8%. Overall cancer detection rate 15% During the time period of system operation only 14% of the total number of head and neck cancers diagnosed were referred via the electronic system. All others had been referred by non-urgent referral channels (by the same group of practitioners) 27 different GP practices used the system to refer; however, one city-centre practice accounted for 17% of referrals Author conclusion: GP referral guidelines and fast-track clinic did not work, with 86% of patients diagnosed with cancer bypassing the system |
Reported associations between elements for logic model: Referral rate disproportionately high for one city-centre practice suggesting a lower threshold to refer Analysis of referral letters revealed disappointing level of compliance with referral guidelines with 12% not mentioning any of the alarm symptoms and many not detailing risk factors Speeding of referral via the system for some patients may have resulted in longer waiting times for other patients |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Kerry 200059 Country: UK Study design: RCT Data collection method: All doctors were sent a questionnaire about the guidelines Aim: To see if the introduction of radiological guidelines into general practices together with feedback on referral rates reduces the number of GP radiological requests over 1 year; and to explore GP attitudes to the guidelines Detail of participants (number, any reported demographics): 69 practices |
Intervention: In February 1995 a GP version of the RCR guidelines was sent to each GP in the 33 practices in the intervention group. After 9 months’ intervention, practices were sent revised guidelines with individual feedback on the number of examinations requested in the past 6 months. The total number of requests per practice was compared for the year before and the year after the introduction of the guidelines Guidelines for examination of chest, hips, knees, spine, skull and sinuses were printed verbatim on two sides of a sheet of A4 paper, which was then laminated Number of hours: NA Delivered by who? NA Control: Control practices were sent the guidelines at the end of the study Length of follow-up: 9 months Response and/or attrition rate: GP questionnaire 60% response rate Context (from what/who to what/who): GP referral to radiology for spinal examination |
Outcome measures: Referral rates Attitudes to guidelines |
Main results: A total of 43,778 radiological requests were made during the 2 years 1994–6 The number of referrals for all spinal examinations fell by 18% in the intervention group, compared with a 2% rise in the control group (p = 0.05) Taking requests for the lumbar spine alone, there was a reduction of 15% in the intervention group compared with a rise of 5% in the control group, giving a difference of 20% between the groups (95% CI 3% to 37%) Overall, an 8% reduction in total numbers of radiological requests was observed in the intervention group, compared with a 2% increase in the control group, giving a difference of 10% between the two groups, but this did not achieve statistical significance |
Reported associations between elements for logic model: Introduction of radiological guidelines together with feedback on referral rates was effective in reducing the number of requests for spinal examinations over 1 year |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Khan 200871 Country: UK Study design: Cohort Data collection method: NR Aim: Efficacy of direct GP referral to the hospital respiratory specialist team in the Hot Clinic in avoiding hospital admissions Detail of participants (number, any reported demographics): Data from 173 patients enrolled between 1 January 2007 and 30 June 2007 were studied. Ninety-seven (57%) were men and 75% were either current or ex-smokers |
Intervention: Hospital at-home schemes are popular for the management of acute exacerbations of COPD aimed at reducing demand for hospital inpatient beds and promoting a patient-centred approach through admission avoidance GPs and community nurses directly referred patients threatening an acute hospital admission, by fax, for a rapid assessment. The Hot Clinic service operates Monday to Friday, 09:00–16:00 hours. Patients are seen within 24 hours of the receipt of the referral letter. The consultation includes clinical assessment, chest radiograph, laboratory data and a decision whether to treat the patient in the community or to admit to the hospital. The GP would be informed by a returned typed faxed letter the same day Number of hours: NA Delivered by who? The Hot Clinic team is led by the respiratory consultant and assisted by the specialist registrar and respiratory nurse Control: None Length of follow-up: None Response and/or attrition rate: NA Context (from what/who to what/who): Referral from primary care to COPD clinic (respiratory) |
Outcome measures: The efficacy of this service was assessed in terms of admission avoidance and the rate of readmission within 1 week and 1 month of the consultation |
Main results: 27 patients (16%) were admitted directly from the Hot Clinic and 146 (84%) were treated in the community. Of those 146 patients, nine (5%) were later admitted within 1 week and 12 (7%) were admitted over 1 week to 1 month after the Hot Clinic appointment. Overall, 125 (72%) were thus treated successfully in the community without the need for hospitalisation It is unclear if all would have been hospitalised without the clinic |
Reported associations between elements for logic model: Suggests potential effectiveness of a direct GP referral system to the hospital respiratory team in avoiding hospital admissions |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Kim 2004155 Country: USA Study design: Evaluation Data collection method: Telephone/postal interview Aim: To examine the effect of referral management on diabetes care Detail of participants (number, any reported demographics): n = 6941; mean age 61 (SD 13) years; 54% female |
Intervention: Translating research into Action for Diabetes (TRIAD) was a multicentre study of managed care enrolees with diabetes Prospective referral management consisted of gatekeeping and mandatory authorisation from the management office. Retrospective referral management consisted of referral profiling and appropriateness reviews Control: No referral management strategy Length of follow-up: 1 year Response and/or attrition rate: NR Context (from what/who to what/who): GP to specialist |
Outcome measures: Self reported visit to specialist Difficulty in getting referrals (perceived) |
Main results: Referral management was commonly used by health plans (55%) and provider groups (52%). In adjusted analysis, there were no associations between any of the referral management strategies and any of the outcome measures |
Reported associations between elements for logic model: Referral management strategies did not affect referrals or perceptions of referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Kim 200998 Country: USA Study design: Cross-sectional Data collection method: Survey web-based Aim: To survey PCPs to assess the impact of electronic referrals on workflow and clinical care Detail of participants (number, any reported demographics): n = 298 |
Method: 18-item, web-based questionnaire to all 368 PCPs who had the option of referring to San Francisco General Hospital Asked participants to rate time spent submitting a referral, guidance of workup, wait times and change in overall clinical care compared with prior referral methods using five-point Likert scales Length of follow-up: None Response and/or attrition rate: Two hundred ninety-eight PCPs (81.0%) from 24 clinics participated Context (from what/who to what/who): Primary care to clinical care |
Outcome measures: Practitioner views Referral |
Main results: Over half (55.4%) worked at hospital-based clinics, 27.9% at county-funded community clinics and 17.1% at non-county-funded community clinics. Most (71.9%) reported that electronic referrals had improved overall clinical care. Providers from non-county-funded clinics (AOR 0.40, 95% CI 0.14 to 0.79) and those who spent ≥ 6 minutes submitting an electronic referral (AOR 0.33, 95% CI 0.18 to 0.61) were significantly less likely than other participants to report that electronic referrals had improved clinical care |
Reported associations between elements for logic model: PCPs felt that electronic referrals improved health-care access and quality; those who reported a negative impact on workflow were less likely to agree |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Kim-Hwang 2010102 Country: USA Study design: Before-and-after Data collection method: Questionnaire Aim: To determine the impact of the e-Referral, compared with paper-based referral, on specialty referral rates Detail of participants (number, any reported demographics): Specialist clinicians, n = 505 |
Intervention: The study was based on a visit-based questionnaire appended to new patient charts at randomly selected specialist clinic sessions before and after the implementation of e-Referrals (using web-based pro forma). A specialist reviewer (physician or nurse) reviews the referrals and determines whether or not it is appropriate to schedule an appointment Control: Paper-based referral Length of follow-up: 2-year study Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist |
Outcome measures: Self-reported difficulty in identifying the referral question Referral appropriateness, need for, and avoidability of follow-up visits |
Main results: It was difficult to identify the reason for referral in 19.8% of medical and 38.0% of surgical visits using paper based methods vs. 11.0% and 9.5% of those using e-Referral (p = 0.03 and p < 0.001) Of those using e-Referral, 6.4% and 9.8% of medical/surgical referrals using paper methods vs. 2.6% and 2.1% were deemed not completely appropriate (p = 0.21 and p = 0.03) Follow-up was requested for 82.4% and 76.2% of medical and surgical patients with paper referrals vs. 90.1% and 58.1% of e-Referrals (p = 0.06 and p = 0.01) Follow-up was considered avoidable for 32.4% and 44.7% of medical/surgical follow-ups with paper-based methods vs. 27.5% and 13.5% with e-Referral (p = 0.41 and p < 0.001) |
Reported associations between elements for logic model: e-Referral can improve communication and increase the appropriateness of referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| King 2001119 Country: UK Study design: Before-and-after Data collection method: Questionnaires and interviews Aim: Whether or not in practices with high referral rate, an invitation to review referrals could identify patients on the waiting list who considered their referral unnecessary, leading to a negotiated cancelling of their appointment Detail of participants (number, any reported demographics): 435 put patient referrals made in 4 months by one general practice with 6600 patients. n = 109 eligible for study |
Intervention: 4–7 weeks after referral, selected patients were sent a questionnaire and an invitation to a review appointment Exclusion criteria were symptoms which raise the possibility of significant disease; patient’s mental state precludes consent or co-operation; the referring doctors prefers the patient not to participate; such urgency that an outpatient appointment could be expected within 3 weeks Subsequently, a series of 22 semistructured interviews were undertaken to seek the review of patients on their willingness to review the referral decision Number of hours: NA Delivered by who? GP Control: NA Length of follow-up: NA Response and/or attrition rate: 109 of 435 referrals Context (from what/who to what/who): GP referral to any specialty |
Outcome measures: Outpatient appointment cancellation |
Main results: Of 435 referrals, 109 (25%) were eligible for this study. 77 (72%) responded to the questionnaire and of those, 10 (13% of responders) indicated uncertainty that referral was still needed Eight of these attended for review, but in none of these cases was the appointment subsequently cancelled Therefore, taking cancellation of hospital appointment as an end point, the effect shown is 0 out of 435 referrals and 0 out of 109 in the intervention group (95% CI for 0 out of 109 = 0% to 3%) |
Reported associations between elements for logic model: Referral review is not an effective way to detect avoidable referrals or enable negotiated cancelling of outpatient referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Knab 2001112 Country: USA Study design: Before-and-after Data collection method Aim: To determine whether or not CBDS could enhance the ability of primary care physicians to manage chronic pain Detail of participants (number, any reported demographics): 100 chronic pain patients |
Intervention: Structured summaries were generated for 50 chronic pain patients referred by primary care physicians to a pain clinic. A pain specialist used a decision support system to determine appropriate pain therapy and sent letters to the referring physicians outlining these recommendations. Separately, five board-certified primary care physicians used a CBDS system to ‘treat’ the 50 cases. A successful outcome was defined as one in which new or adjusted therapies recommended by the software were acceptable to the primary care physicians (i.e. they would have prescribed it to the patient in actual practice). Two pain specialists reviewed the primary care physicians’ outcomes and assigned medical appropriateness scores (0 totally inappropriate to 10 totally appropriate). One year later, the hospital database provided information on how the actual patients’ pain was managed and the number of patients rereferred by their primary care physician to the pain clinic Control: None Length of follow-up: 1 year Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for chronic pain |
Outcome measures: Appropriateness Rereferral |
Main results: On the basis of CBDS recommendations, the primary care physician subjects ‘prescribed’ additional pain therapy in 213 of 250 evaluations (85%), with a medical appropriateness score of 5.5 ± 0.1. Only 25% of these chronic pain patients were subsequently rereferred to the pain clinic within 1 year |
Reported associations between elements for logic model: The use of a CBDS system may improve the ability of PCPs to manage chronic pain and may also facilitate screening of consults to optimise specialist utilisation |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Knol 200690 Country: the Netherlands Study design: Before-and-after Data collection method: Referral data/interviews Aim: To reduce dermatology referrals Detail of participants (number, any reported demographics): n = 505 consultations by 29 GPs |
Intervention: One overview and two detailed digital photographs of the skin problems were taken on a digital camera and attached to an e-mail message containing standard clinical information. The e-mail was sent to a dermatologist who replied after evaluation. After a median follow-up of 548 days, GPs were interviewed about dermatology referrals Control: NA Length of follow-up: 2 years Response and/or attrition rate: follow-up data not available for 32 (6%) patients Context (from what/who to what/who): GP to dermatology |
Outcome measures: Referral rate |
Main results: Patients were split into those who GPs would have referred without the intervention (n = 306) and those who they would not have referred Using teledermatology, 163 patients were not referred, a reduction of 163/306 or 53% There was no significant difference between dermatologist for secondary referral (χ2 = 1.6, p = 0.45). Patient gender did not affect secondary referral (χ2 = 0.8, p = 0.36) When GPs had no prior intention to refer, there turned out to be a secondary consultation in 17% of cases (24/136) Older patients were more likely to be referred (χ2 = 10.6, p < 0.01) |
Reported associations between elements for logic model: The 51% referral reduction was similar to other studies of videoconferencing |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Kousgaard 200329 Country: Denmark Study design: RCT (unblinded) Data collection method: Questionnaires Aim: To investigate GP assessment of a structured oncology information pack sent to GPs when newly referred patients had visited a department of oncology for the first time, and to compare their assessment of this material with their assessment of traditional information provided by the department Detail of participants (number, any reported demographics): 248 cancer patients and their 199 GPs |
Intervention: Intervention group practitioners received a structured information pack when their patients attended the department of oncology for the first time. The patients were informed that their GP would receive this information and thus the study was unblinded. The pack included (1) a discharge letter written in accordance with specially developed guidelines and bearing the direct telephone number of a departmental contact person, (2) information about the cancer, its treatment and prognosis, (3) general information about radiotherapy and chemotherapy and treatment of nausea and sickness and (4) information that the patient had been advised to see his/her own practitioner about problems and question Control: Participating practitioners in the control group received the traditional information from the department (i.e. the discharge letter or an extract from the hospital record) Length of follow-up: NR Response and/or attrition rate: 88.3% of the 248 questionnaires were returned Context (from what/who to what/who): GP referral to oncology |
Outcome measures: Practitioner views GP assessment of the quality of the information material received for each patient |
Main results: The structured information pack improved GP knowledge of oncology; GPs found themselves better equipped to support and counsel patients during the course of their illness, and practitioner satisfaction with the department rose GP evaluations of the first discharge letter received from the department. The two groups were significantly different (p = 0.039): Intervention group practitioners gave a significantly higher score to the information value of the discharge letter than did control group practitioners. The most pronounced difference was seen for psychosocial conditions (p = 0.001) and information about what the patient had been told at the department (p = 0.001). Stratification according to sex, years as a GP and practice location revealed no differences between the groups (data not shown) |
Reported associations between elements for logic model: Intervention, though reasonably simple, inexpensive and not particularly time-consuming, improved co-operation between the specialist department and the GP |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Lam 201125 Country: UK/China Study design: Cross-sectional Data collection method: Questionnaire (postal) Aim: To examine the outcomes of a postgraduate training course in geriatrics for primary care doctors Detail of participants (number, any reported demographics): n = 98 |
Method: An evaluative study was conducted to examine the impact of the Postgraduate Diploma in Community Geriatrics, which is a 1-year part-time program for primary care doctors developed by the Family Medicine Unit of The University of Hong Kong The diploma includes the components of clinical attachment (20 sessions of clinical geriatric teaching and five sessions of rehabilitation and community health services), interactive workshops, locally developed distance-learning manual, written assignments and examination as well as a clinical examination Control: NA Length of follow-up: NA Response and/or attrition rate: Ninety-eight replies were received with a response rate of 52.4% (98/187) Context (from what/who to what/who): GP referral to geriatrics |
Outcome measures: Referral Views on training |
Main results: Most respondents felt it was more rewarding and had participated more in geriatric care, and the majority had improvement in their communication skills with elderly patients after taking the course. Moreover, the graduates are more confident in diagnosing and managing common geriatric problems, and deciding to which specialty to refer the elderly patients Of the referrals, there was a significant increase to private geriatricians and a significant reduction to other specialists. The average number of elderly patients seen per day had also increased However, little change was observed about making nursing home visits, the frequency of which remained low. Many graduates expressed difficulties in conducting nursing home visits |
Reported associations between elements for logic model: Education can affect referral and confidence |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Leggett 200485 Country: UK Study design: RCT Data collection method: NR Aim: To compare outcomes of referral for dermatology appointments between patients whose referral letters do or do not include instant photograph(s) Detail of participants (number, any reported demographics): n = 136 (20 GPs: 10 intervention and 10 control) |
Intervention: Instant photographs, taken by the GP, were included in the referral letters. The GP took photograph(s) of the skin condition and sent them with a referral letter to the dermatologist in a numbered, sealed envelope. If a diagnosis was not possible, patients were given an appointment. If diagnosis was possible, a letter was sent to the GP with advice on management: some patients were also given an appointment for further management Number of hours: GPs were trained for 15 minutes to use a camera to produce digital photos of the presenting condition Delivered by who? NA Control: Control group patients were given outpatient appointments in the usual way Length of follow-up: NA Response and/or attrition rate: NR Context (from what/who to what/who): GP to dermatology |
Outcome measures: The numbers of study group patients needing an appointment for diagnosis or management and with a changed diagnosis after face-to-face consultations were recorded Waiting time from referral to appointment or management plan was recorded for both groups |
Main results: For 63% of the study group (45/71), a diagnosis and a management plan were made without the patient requiring an appointment. This included 38% (27/71) who, after diagnosis and initial management, needed an appointment and 25% (18/71) who did not The remainder of the study group (37%; 26/71) required a face-to-face consultation The mean time for formulation of a management plan for patients without an appointment was 17 days (SD = 11); waiting times for appointments in study and control groups were similar (mean 55 days; SD = 40) |
Reported associations between elements for logic model: Instant photography is helpful in managing dermatology referrals and offers the potential to reduce numbers requiring an outpatient appointment by 25% |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Leiba 2002130 Country: Israel Study design: nRCT Data collection method Aim: Evaluation of easy-to-access to specialists on health service use, cost (time and money) and GP attitude Detail of participants (number, any reported demographics): None |
Intervention: A specialist outreach clinic was established in a home-front military primary care clinic. Patients were referred to nearly by specialists but no further referral was required for continuity of specialist care The same analysis was applied to a similar clinic employing only GPs, which refers to military specialist centres or hospital outpatient clinics Number of hours: NA Delivered by who? NA Control: No outreach clinic Length of follow-up: 6 months Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to specialist outreach clinic or usual hospital care |
Outcome measures: Health service use, cost (time and money) and GP attitude |
Main results: The incorporation of specialists did not result in a significant increase in the overall consumption of medical services (p < 0.05). It reduced the number of referrals out of the clinic to specialist centres from 1449 to 421 per month (p < 0.05). In the control clinic, referrals to distant specialist centres and outpatient clinics showed a slight and non-significant increase Loss of work days was reduced from 2891 days per month to 1938 days per month (p < 0.001) The total cost of all medical interactions and referrals did not significantly increase after the introduction of the outreach specialist clinic (p < 0.05). Primary physicians graded their satisfaction with the new clinic as 4.5 (out of 5) |
Reported associations between elements for logic model: Improving access to specialists geographically and removing the need for a referral for each specialist visit did not increase total health-care use and costs |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Lester 200939 Country: UK Study design: cRCT Data collection method: NR Aim: To assess the effect of an educational intervention for GPs on referral rates to early-intervention services and the duration of untreated psychosis for young people with first-episode psychosis Detail of participants (number, any reported demographics): A total of 110 of 135 eligible practices (81%) were recruited. 179 young people were referred: 97 from intervention and 82 from control practices |
Intervention: REDIRECT trial (BiRmingham Early Detection In untREated psyChosis Trial) Practices with access to the three early-intervention services in three inner-city PCTs in Birmingham were eligible for inclusion. Intervention practices received an educational intervention addressing GP knowledge, skills and attitudes about first-episode psychosis. The outcome of the theoretical and modelling work suggested that the educational intervention needed to impart knowledge about important symptoms and signs evident in first-episode psychosis, teach core questioning skills, and encourage more positive attitudes towards young people with the condition A 17-minute video made specifically for the study, depicting role-played primary care consultations with young people with first-episode psychosis, was shown to GPs in intervention practices. The study team then led a 15-minute question-and-answer session including referral guidelines to early-intervention services. Two refresher educational sessions were conducted Number of hours: NR Delivered by who? NR Control: no intervention Length of follow-up: Follow-up at 4 months Response and/or attrition rate: NR Context (from what/who to what/who): Referral from general practice to psychiatry |
Outcome measures: Difference in the number of referrals to early intervention services between practices Duration of untreated psychosis Time to recovery Use of the Mental Health Act, and GP consultation rate during the developing illness |
Main results: Ninety-seven people with a first episode of psychosis were referred by intervention practices, and 82 people from control practices during the study: RR of referral 1.20 (95% CI 0.74 to 1.95, p = 0.48) No effect was observed on secondary outcomes except for ‘delay in reaching early-intervention services’, which was statistically significantly shorter in patients registered in intervention practices (95% CI 83.5 to 360.5, p = 0.002) |
Reported associations between elements for logic model: GP training on first-episode psychosis is insufficient to alter referral rates to early-intervention services or reduce the duration of untreated psychosis |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Levell 2012129 Country: UK Study design: Before-and-after study Data collection method: Standard hospital systems Aim: To assess the effect of introducing dermatology integrated intermediate-care services on the numbers of dermatology referrals to secondary care Detail of participants (number, any reported demographics): None |
Method: The dermatology intermediate care service was set up in 2005, providing services in two locations by two GPwSIs in dermatology. The GPwSIs were supported by experienced dermatology nurses and in total six clinics weekly were held, seeing approximately 30 new patients weekly Control: None (before-and-after) Length of follow-up: 2004–10 Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to dermatology |
Outcome measures: Dermatology new patients |
Main results: The numbers of dermatology new patients seen, which had been stable for 5 years, showed an increase in 2007 followed by a substantial increase in 2008 and then 2009 The mean number of new patients seen in dermatology in 2004–6 was 6927 patients per year; in 2007, 7844 patients; and the mean number of new patients seen between 2008 and 2010 was 11 535 patients per year. This was an increase of 67% in the number of new patients seen. Overall, over this period, there was a 23% increase in dermatology new patients seen in secondary care dermatology in England |
Reported associations between elements for logic model: The introduction of dermatology intermediate care services was followed by a 67% increase in secondary care new patients |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Lucassen 200145 Country: UK Study design: Before-and-after Data collection method: NR Aim: To see whether or not guidelines on whom to refer to a regional genetics service could improve appropriateness of referrals Detail of participants (number, any reported demographics): NR |
Intervention: Referral guidelines for ORGS family cancer clinic were drawn up in discussion with local GPs, surgeons, radiologists, gynaecologists, public health physicians and geneticists. Evidence from national consensus was incorporated where available. The guidelines were sent to all Oxfordshire GPs and subsequent content of referral letters was analysed. A retrospective analysis of referral letters sent during the previous 6 months was also made Number of hours: NA Delivered by who? NA Control: NA Length of follow-up: 8 months Response and/or attrition rate: NA Context (from what/who to what/who): GP to regional genetics service |
Outcome measures: Appropriate referral |
Main results: Post guidelines, more referrals met the criteria than before (χ2 = 15.79, p < 0.001) Fewer lower-risk referrals were made: 34% of letters (36/103) were high risk pre guidelines, whereas 47% (46/110) were high risk post guidance (not significant: χ2 for change in proportion of low risk pre and post = 1.34, p = 0.24, and for high risk = 3.33, p = 0.07), and that the description of the risk in the GP letter improved so that a greater proportion of generic clinic risks agreed with those described in the GP letter |
Reported associations between elements for logic model: The use of referral guidelines can improve appropriateness of referrals to secondary care (regional genetic screening service) |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Lyon 2009160 Country: UK Study design: Before-and-after Data collection method: GP data Aim: To promote the early presentation and diagnosis of breast, bowl and lung cancer Detail of participants (number, any reported demographics): NR |
Intervention: Involving local people working in partnership in their communities to raise awareness of cancer symptoms and promote early presentation. The teams work with primary care, other statutory organisations and with the voluntary sector. The specific contribution of the local people was in the identification of hard-to-reach groups and the tailoring of effective health messages Number of hours: NR Delivered by who? Local people and primary care Control: NA Length of follow-up: 1 year Response and/or attrition rate: NA Context (from what/who to what/who): GP to cancer screening |
Outcome measures: Referrals |
Main results: Interim results show an increase in the number of urgent 2-week referrals and the proportion of new cancer cases diagnosed through the urgent 2-week referral route (from 43% to 51%) for all three cancers. These results were statistically significant for the bowel cancer (χ2 = 22.193, df = 1; p < 0.001) and lung cancer pathways (χ2 = 8.886, df = 1; p = 0.003). There was also an increase in the proportion with no spread at the time of diagnosis for bowel cancer (38–43%) and breast cancer (41–44.5%), but these results did not reach statistical significance |
Reported associations between elements for logic model: Community awareness raising led to an increase in 2-week cancer referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Maddison 2004154 Country: UK Study design: Before-and-after Data collection method Aim: Impact of the targeted early access to musculoskeletal services (TEAMS) programme on accessibility to musculoskeletal services Detail of participants (number, any reported demographics): No information |
Intervention: Establishing with central clinical triage a common pathway for all musculoskeletal referrals so that patients attend the appropriate department A back pain pathway led by extended scope physiotherapists was developed, and GPwSIs and extended scope physiotherapists were trained to provide services for patients with uncomplicated musculoskeletal problems in the community Control: NA Length of follow-up: 18 months Response and/or attrition rate: NA Context (from what/who to what/who): GP to musculoskeletal |
Outcome measures: Number of patients referred and seen with musculoskeletal problems, waiting times, number of duplicate referrals, and surgery conversion rates in orthopaedic clinics |
Main results: After the introduction of the targeted early access to musculoskeletal services in April 2002, there was a major increase (116%) in the total number of referrals for musculoskeletal problems. In contrast, the number of orthopaedic referrals was slightly reduced Over 18 months the total number of referrals more than doubled. Despite this, waiting times for musculoskeletal services fell; this was noticeable for rheumatology and pain management (primary data not given) Duplicate referrals were abolished. Surgery conversion rates did not, however, change The community musculoskeletal clinics were well received by GPs, and the short waiting time of 4–6 weeks put them in demand. Patients were generally seen on a one-off basis; < 10% were referred on or followed up. Patient satisfaction questionnaires showed that 88% of patients rated the service as excellent or good, and 75% were completely satisfied with the service provided |
Reported associations between elements for logic model: Community-based multidisciplinary clinics run by specially trained GPwSIs and extended scope physiotherapists are an effective way of managing patients with uncomplicated musculoskeletal problems and have been well received by patients and GPs |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Magill 2009115 Country: USA Study design: Before-and-after Data collection method: Analysis of referral rate data Aim: To evaluate a computer-based system to enhance referral for colonoscopy Detail of participants (number, any reported demographics): Patients aged 50 years or older with no record of having a colonoscopy in last 10 years. No detail of staff characteristics beyond description of outpatient practices varying in size, with smallest having two physicians and largest having 25 |
Intervention: 1. Pop-up prompt for screening colonoscopy on EMR modified 2. Also education sessions for primary care providers comprising epidemiology of colon cancer, strategies for early detection, how to use EMR and optimal clinic workflow to facilitate screening 3. Medical assistants asked to discuss screening with eligible patients before seen by physician and initiate preliminary order for test + best practice alerts, computerised documentation of referral status, individual physician feedback implemented later Number of hours: No detail of how long/many education sessions Delivered by who? NR Control: None Length of follow-up: Baseline January 2003, intervention through to July 2007 (4 years) Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to colonoscopy service |
Outcome measures: Colonoscopy referral rate |
Main results: Individual site providers experienced very different local conditions and changes during the course of the project (e.g. relocation, new services, personnel change, introduction of revenue for screening site and physician from referrals) At baseline monthly referral rates 5–7% Pop-up prompt and provider education introduced over 2-month period showed little or no immediate correlation Initiation of MA workflow change 2 months later was associated with 11% increase in referral rate. Following 29 months all had referral rates above the baseline point (p < 0.001) Small increases observed after best practice alerts and computerised documentation of referral status implemented 2.5 years after initial intervention (no details of these intervention methods). Also small increases after unblinded individual physician feedback implemented 3 years later At 4-year point referral rates remained above baseline Wide variation in performance between providers, even those practising in the same clinic. Improved performance data mostly due to performance at the two largest clinics |
Reported associations between elements for logic model: Physicians responded differently to the interventions. Of those who did respond initially many did not sustain improvement and reverted to baseline Only a few demonstrated improvement trend over longer than 9 months Different local conditions and aggregated data masking individual differences between clinicians |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Malik 200741 Country: UK Study design: Audit Data collection method: Patient records/referral letters Aim: To determine if the 2-week wait referral guidelines had been followed, and what proportion of patients referred under the guideline had malignant tumours Detail of participants (number, any reported demographics): 40 patients |
Intervention: Referral letters were evaluated to see if they met Department of Health guidelines for referral of a suspected bone or soft tissue tumour Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for cancer |
Outcome measures: Referral meets guidelines |
Main results: 40 patients were referred under the guideline between January 2004 and December 2005. Ten of these patients (25%) had malignant tumours, compared with 243 of 507 (48%) of those referred from other sources Most (31 of 40, 78%) ‘2-week’ referrals met the published referral guidelines. In 9 of the 40 cases, the patient did not meet the criteria for urgent referral. None of the nine patients had malignant tumours |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Mariotti 2008113 Country: Italy Study design: Audit Data collection method: Clinical data Aim: To evaluate a new method of prioritisation of patients suffering from significant GI disorders needing rapid access to diagnostic procedures Detail of participants (number, any reported demographics): n = 438 outpatients |
Intervention: GPs used a ranking of waiting times for different levels of clinical priority called homogenous waiting groups. Specialists also assigned a priority level for each patient as well as evaluating the appropriateness of the referral and the presence of significant endoscopic disorders. Agreement between GP and specialist was evaluated Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 7 months of intervention data Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist |
Outcome measures: Referral priority GP/specialist agreement |
Main results: Most referrals (74.4%) were deemed low priority by GPs, with no maximum waiting time assigned. The level of agreement between GPs and specialists as regards patients’ priorities was poor to moderate; for gastroscopy the kappa was 0.31, and for colonoscopy 0.44 There was an association between the proportion of significant disorders identified with endoscopy and the priority assigned to the referral (χ2 = 18.9, 1 df; p < 0.001). The overall proportion of referrals deemed inappropriate by specialists was 22.1% |
Reported associations between elements for logic model: There is value in liaison between GPs and specialists for achieving timely referrals and avoiding delayed diagnosis. High levels of agreement need to be achieved |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Matowe et al. 200250 Country: UK Study design: Before-and-after Data collection method: Admin system data Aim: To evaluate the effect of disseminating guidelines Detail of participants (number, any reported demographics): 376 GPs in 87 practices in Grampian. 117,747 referrals, mostly chest X-rays followed by limb and joint and then spine |
Intervention: Copies of Royal College of Radiology guidelines were mailed to GPs Number of hours: NA Delivered by who? NA Control: None Length of follow-up: Data for 3 years Response and/or attrition rate: NA Context (from what/who to what/who): GP to radiography |
Outcome measures: Data from two radiology departments – effect of intervention on total number of referrals – absolute change in referral, underlying trend, and change in referral trend Effect on investigations requested average more than 20 times per month |
Main results: Month of May had highest number of referrals; December had the lowest No significant effects of intervention on total number of general practice imaging requests. Total referrals decreased by 32 (95% CI –226.7 to + 291.4) in month following guideline dissemination while trend decreased by –1.82 requests per month (95% CI –11.8 to + 8.2). Referral decreased by average 1.2 per month for the entire 35-month period None of 18 examinations evaluated changed significantly after introduction of guidelines on time series analysis. Eleven of the 18 did show significant difference before and after introduction of the guidelines, however, with 10 having significant underlying trends |
Reported associations between elements for logic model: No effect of passive distribution of guidelines. Before-and-after studies may erroneously find an effect |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| McGarry 2009148 Country: Australia Study design: Evaluation survey Data collection method: NR Aim: To examine changes in patient management and referral for care following the BOiMHC initiative Detail of participants (number, any reported demographics): One hundred and thirty-three (33%) GPs responded |
Intervention: Significant government spending has resulted in substantial changes to the Australian primary mental health-care system. Initially producing the BOiMHC initiative, this has been replaced by the Better Access to Mental Health Care programme, which allows all GPs to refer patients for allied psychological health care under BOiMHC. Incentives commenced August 2002. GPs working in accredited practices who had completed accredited mental health training were able to receive service incentive payments (SIP) for providing care to patients with ICD-10-diagnosed mental illness. Trained GPs able to refer patients for psychological therapies to the Access to Allied Psychological Service (ATAPS) via divisions of general practice. GP Psych Support provides GPs with access to advice from psychiatrists via telephone, e-mail or fax BOiMHC: Commenced November 2006. All GPs regardless of training or practice location receive higher Medicare rebates to complete GP mental health plans for patients with ICD-10-diagnosed mental illness, as well as higher rebates for mental health consultations. Patients with an ICD-10 diagnosis and a GP mental health plan are eligible for Medicare rebates for psychological care, for up to 12 sessions per year (individual) and 12 sessions (group therapy) This study was a comparison of results of a 2006 postal survey of Australian GPs examining self-reported management of patients with depression with a similar survey conducted in 2001–2, prior to the BOiMHC initiative Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 4 years Response and/or attrition rate: 133 of 410 responded Context (from what/who to what/who): GP referral to mental health care |
Outcome measures: Referral to mental health care |
Main results: The main self-reported strategies for managing patients with depression were similar to the previous study: supportive counselling and medication Compared with the original study, significant differences between rates of formal training (short course, diploma, certificate, degree or work at the level of psychiatry registrar or above) were only found in Institute for Psychological Therapies (p = 0.03) and relaxation therapy (p = 0.03), with fewer responders reporting formal IPT training (current: 8.6%, 11/128; original: 17.5%, 24/137) and more reporting formal training in relaxation strategies (current: 18.6%, 24/129; original 9.3%, 13/140) in the current study. Otherwise there were no significant differences in rates of formal training Significantly higher rates of referral for psychological treatments were reported in 2006 than in 2002. Significantly higher proportions of responders in the current study reported referring half or more of their patients with mild to moderate depression for PST (p < 0.001) or cognitive–behavioural therapy (p < 0.001). In fact, significantly more responders reported higher rates of referral for most modalities than in the original study |
Reported associations between elements for logic model: While GPs’ main reported strategies for managing patients with depression were unchanged, reported referral for psychological therapies was significantly higher in 2006, possibly reflecting the impact of changes to the primary mental health-care system |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| McGowan 2008107 Country: Canada Study design: RCT Data collection method: Survey Aim: Evaluated whether or not information provided by librarians to answer clinical questions positively impacted time, decision-making, cost savings and satisfaction Detail of participants (number, any reported demographics): Physicians (93.2%; n = 82), with a small number of nurse practitioners (4.5%; n = 4), residents (1.1%; n = 1) and nurses (1.1%; n = 1) |
Intervention: The ‘just-in-time information’ librarian consultation service was designed to provide a rapid response to clinical questions during patient visit hours. The questions were submitted by the participants and each question was randomly assigned to the intervention (librarian information) or control (no librarian information) group. If the question was randomised to the control group, participants received a message within 1 minute that their question would not be answered. The librarian still answered the question, but the software blocked the response from being sent to the participant. Thus, they would need to try to answer the question themselves. The object of the randomisation was a clinical question Each participant had clinical questions randomly allocated to both intervention (librarian information) and control (no librarian information) groups. Participants were trained to send clinical questions via a hand-held device Control: No library information Length of follow up: Survey sent 24 hours after a question was submitted Response and/or attrition rate: A total of 110 individuals signed consent forms; 21 of these individuals withdrew from participation before randomisation, leaving a final group of 88 individuals who participated in the RCT Context (from what/who to what/who): GP to specialist |
Outcome measures: Impact of the information provided by the service (or not provided by the service), additional resources and time required for both groups |
Main results: The average time for ‘just-in-time information’ librarians to respond to all questions was 13.68 minutes/question (95% CI 13.38 to 13.98 minutes). The average time for participants to respond their control questions was 20.29 minutes/question (95% CI 18.72 to 21.86 minutes). Using an impact assessment scale rating cognitive impact, participants rated 62.9% of information provided to intervention group questions as having a highly positive cognitive impact. They rated 14.8% of their own answers to control question as having a highly positive cognitive impact, 44.9% as having a negative cognitive impact, and 24.8% with no cognitive impact at all In an exit survey measuring satisfaction, 86% (62/72 responses) of participants scored the service as having a positive impact on care and 72% (52/72) indicated that they would use the service frequently if it were continued |
Reported associations between elements for logic model: Providing timely information to clinical questions had a highly positive impact on decision-making and a high approval rating from participants |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| McKoy 200489 Country: USA Study design: Before-and-after Data collection method: Clinical data Aim: To evaluate the accuracy, access time, cost and acceptance by patients and physicians of an asynchronous teledermatology referral intervention in primary care Detail of participants (number, any reported demographics): n = 52 patients aged 25–89 years. 46% female |
Intervention: Primary care physicians in a multispecialty group referred patients for teledermatology consultation. Same-day history and digital images taken by a nurse were electronically sent to a dermatologist who returned a diagnosis to the referring physician Control: NA Length of follow-up: NA Response and/or attrition rate: 52 of 54 enrolled patients completed the study Context (from what/who to what/who): GP to dermatology |
Outcome measures: Diagnosis Referral |
Main results: History was adequate for diagnosis in 81% of cases; images were adequate in 75% of cases. Accuracy of the teledermatology diagnosis in cases with adequate images was 97%; accuracy for all cases was 92% A dermatology visit was recommended in 26% of cases with adequate images and in 42% of all cases Access time for a teledermatology opinion was 1.9 days, compared with 52 days for a regular dermatology appointment |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| McNally 200374 Country: UK Study design: Retrospective data analysis before and after introduction Data collection method: Examination of case notes Aim: To assess the impact of a fast-track clinic Detail of participants (number, any reported demographics): 295 cases over a 6-year period; patients with primary ovarian cancer. 109 cases prior to intervention and 133 cases after |
Intervention: Clinic appointment within 2 weeks to fast-track clinic. Clinical referral criteria. GPs informed of the clinic and referral criteria by individual letter, GP newsletter, and meetings Number of hours: NA Delivered by who? NA Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Referral for ovarian cancer |
Outcome measures: Time to diagnosis |
Main results: Median waiting time for referral to specialist was 3 days (range 0–188 days). This did not change significantly after clinic introduction (p = 0.05). The impact of fast-track clinic on referral and diagnosis time variables was not significant The fast-track clinic saw 10%, 20.1% and 10.3% of ovarian cancers diagnosed by the service during the first 3 years of operation. 13.5% of patients were referred to the fast-track clinic |
Reported associations between elements for logic model: Rapid access clinic may have some limited impact but may be underused |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Melia et al. 200851 Country: UK Study design: Before-and-after Data collection method: Request for GP data, data from pathology labs Aim: To evaluate whether or not guidelines for GPs impacted on GP referral for potential prostate cancer screening Detail of participants (number, any reported demographics): GPs referring to pathology lab in four study areas (Chichester, Sutton, Truro and York). 200 GP partners in 48 practices. Male patients aged 45–84 years, n = 1520 |
Intervention: Prostate Cancer Risk Management Programme (guidelines for GPs on age-specific prostate-specific antigen cut-off levels in asymptomatic men) Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 1–2 years pre intervention to post Response and/or attrition rate: 48 of 69 practices invited took part (70%), 79% of patients’ baseline data, 90% at intervention Context (from what/who to what/who): GP to urologists |
Outcome measures: GP demographics Awareness of receiving guidelines pack Proportion of asymptomatic men with raised antigens referred to urologists |
Main results: Awareness of pack acknowledged by 112 (56%) of GPs, 24 unaware, 64 did not know. Awareness not significantly different by area, age, gender, MRCGP registration, number of years working or number of sessions per week of GP Proportion of asymptomatic men referred who had raised antigen levels did not increase significantly from baseline to intervention (24% pre intervention, 29% post p = 0.42). No significant difference in referral rate by area (p = 0.33) |
Reported associations between elements for logic model: Influence of guidelines low |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Morrison 200164 Country: UK Study design: RCT Data collection method: GP report of management before referral, hospital record review Aim: To evaluate the effect of clinical guidelines on the management of infertility Detail of participants (number, any reported demographics): 214 GP practices in Glasgow, stratified by small, medium and large practices and by catchment area. 689 referrals; most aged over 34 years, Carstairs deprivation 6/7; 84% female only |
Intervention: Local guidelines developed. Management pack sent to intervention practices and invitation to attend a meeting to discuss. 17% of doctors attended a meeting. Individual visits also offered, taken up by two practices Number of hours: NA Delivered by who? NA Control: Delayed intervention, practices received guidelines 1 year later Length of follow-up: Referral letters screened for 1 year Response and/or attrition rate: Seven practices (4%) withdrew, 84% provided pre-referral management information, case notes available for 90% Context (from what/who to what/who): GP to outpatient infertility clinic |
Outcome measures: Time presentation to referral Investigations completed by GP Number and content of outpatient visits Time to reach management plan Cost of referral |
Main results: No difference control and intervention in regard to whether a management plan was made (OR 1.239, 95% CI 0.869 to 1.765; p = 0.236) No difference in duration between first appointment and date of management plan, no difference mean number of outpatient visits before a management plan put in place Intervention practices had a significantly higher mean of total number of investigations carried out before referral (2.81 intervention vs. 2.50 control, OR 1.32, 95% CI 1 to 1.75; p = 0.05) No significant difference in total costs to the NHS (£349.78 vs. £327.48, p > 0.05) Author conclusion: No evidence that subsequent hospital management was influenced by more investigations by GPs prior to referral |
Reported associations between elements for logic model: Simple dissemination of guidelines is unlikely to result in changed referral patterns |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Nicholson 200697 Country: Australia Study design: Survey data evaluation of intervention – Brisbane e-referral project Data collection method: Survey of GPs, patients, specialists Detail of participants (number, any reported demographics): Six patients and 19 GPs from five practices took part in the intervention. Survey given to eight who had referred patients and 11 who had been trained in using the system |
Intervention: Brisbane e-referral project. E-referral and booking appointment system from GP to a hospital outpatient department for patients with suspected cancer. Referral document was attached to the appointment booking slot Control: None Length of follow-up: None Response and/or attrition rate: Context (from what/who to what/who): GP to a hospital outpatient department for patients with suspected cancer |
Outcome measures: Likert scale responses 1 strongly disagree, 5 strongly agree, neutral 3 |
Main results: GP satisfaction high – Saved me time in referring mean 3.75 (SD 1.612) Contributed to feeling of increased trust between hospital and community mean 3.88 (0.885) Given me improved referral template 3.75 Provided useful guideline 4.19 Reduces test duplication for patient 4.13 Makes best use of professional time 3.88 Delivers best patient care 4.13 Improves the timeliness of outpatient bookings 4.62 Increased confidence referral being auctioned 4.50 Made the referral process easy for patients 4.62 Made little difference to booking appointment 1.75 Improved the quality of information sharing 3.88 Patient satisfaction – Understood what was planned for my care at all times 4.5 Believe necessary information was transferred 4.75 Needed to have tests repeated 2.67 Sometimes anxious not knowing if had appointment 3.17 |
Reported associations between elements for logic model: Trust Staff time Information sharing Test results Patient satisfaction |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Nielsen 200392 Country: Denmark Study design: RCT Data collection method: Questionnaire Aim: To determine the effect of a shared care programme on the attitudes of newly referred cancer patients towards the health-care system and their health-related quality of life and performance status, and to assess patients’ reports on contacts with their GP Detail of participants (number, any reported demographics): 248 consecutive cancer patients recently referred to the department |
Intervention: RCT in which patients completed questionnaires at three time points. The shared care programme included transfer of knowledge from the oncologist to the GP, improved communication between the parties, and active patient involvement Control: The intervention was compared with the normal procedure in the department Length of follow-up: 3 and 6 months Response and/or attrition rate: Sixteen patients refused to participate because they felt they could not manage to fill in the questionnaires and we failed to include eight patients. Two patients were excluded after randomisation because they did not meet the inclusion criteria Context (from what/who to what/who): GP referral for cancer |
Outcome measures: Patients’ attitudes towards the health-care services, their health-related quality of life, performance status and reports on contacts with their GPs |
Main results: The shared care programme had a positive effect on patient evaluation of co-operation between the primary and secondary health-care sectors. The effect was particularly significant in men and in younger patients (18–49 years) who felt that they received more care from the GP and were left less in limbo Young patients in the intervention group rated the GP’s knowledge of disease and treatment significantly higher than young patients in the control group. The number of contacts with the GP was significantly higher in the intervention group The European Organisation for Research and Treatment of Cancer quality of life questionnaire and performance status showed no significant differences between the two groups |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Patterson 2004104 Country: UK Study design: Cohort Data collection method: NR Aim: To determine whether or not an e-mail triage system between GPs and a neurologist for new outpatient referrals is feasible, acceptable, efficient, safe, and effective Detail of participants (number, any reported demographics): Seventy-six consecutive patients with neurological symptoms from nine GPs, for whom a specialist opinion was deemed necessary Seventy-six referrals were received for 75 patients in a 14-month period (27 male, 48 female, mean age 44 years, range 16–80 years) |
Intervention: A structured form was devised for GPs to refer patients. This set out the required history and examination and was either sent as an e-mail attachment or incorporated in the body text of the e-mail When the neurologist received the e-mail referral he decided if advice alone was appropriate, if investigations were needed or if a clinic visit was necessary. When the investigation results were available, either a clinic appointment was made or further advice was given Control: Conventional consultation Length of follow-up: 6 months Response and/or attrition rate: NA Context (from what/who to what/who): GP to neurology |
Outcome measures: The number of participants managed without clinic attendance and the reduction in neurologist’s time compared with conventional consultation |
Main results: Forty-three per cent of participants required a clinic appointment, 45% were managed by e-mail advice alone, and 12% by e-mail plus investigations GP satisfaction was high Forty-four per cent of the neurologist’s time was saved compared with conventional consultation: total time spent was, therefore, 1270 minutes (mean of 16.7 minutes per patient) No deaths or significant changes in diagnosis were recorded during the 6-month follow-up period |
Reported associations between elements for logic model: E-mail correspondence between a GP and a neurologist enables the majority of patients to be dealt with within 3 days of referral, enables 57% of those referred to be given advice or have investigations arranged without entering the hospital clinic system and reduces the time of the neurologist by 44% |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Pfeiffer 2011138 Country: USA Study design: Retrospective analysis of clinic data Data collection method: Analysis of hospital patient data from locations with primary mental health services and those without over a 1-year period Aim: To determine whether or not implementation of primary care mental health services is associated with differences in specialty mental health clinic use Detail of participants (number, any reported demographics): 49,957 primary care patients with new visits to specialty mental health clinics. Mean age 55.7 years, male 93%. 118 primary care facilities offering specialist mental health services and 142 without |
Intervention: Veterans Health Administration, primary care mental health services providing collocated collaborative mental health specialists and managers screening and managing common mental health conditions (depression, alcohol misuse, PTSD) Number of hours: NA Delivered by who? NA Control: Primary care facilities not having an integrated mental health service Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Primary care services to specialty mental health care |
Outcome measures: New visits to specialty mental health clinics Mental health diagnosis Illness severity |
Main results: Initiation of treatment at specialty mental health clinic did not differ between primary care with mental health facilities and those without the service (5.6% vs. 5.8%) Attendance at a primary care service for mental health was not a predictor of total number of specialist mental health clinic visits Author conclusion: provision of primary care mental health service not associated with differences in new use of specialty mental health services or diagnoses received |
Reported associations between elements for logic model: No impact on referrals from primary care mental health service to specialist service |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Potter 200746 Country: UK Study design: Prospective cohort study Data collection method: Standard data collection sheets that were completed at the time of consultation and diagnosis Aim: To investigate the long-term impact of the two week wait rule for breast cancer on referral patterns, cancer diagnoses and waiting times Detail of participants (number, any reported demographics): All patients referred to breast clinic from primary care between 1999 and 2005 Over the 7 years, the centre received 24 999 new referrals from primary care, a mean annual referral rate of 3571 (SD 182) |
Intervention: Uses routine data to consider the effect of the introduction of the 2-week wait guideline Control: None Length of follow-up: 7 years Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for breast cancer |
Outcome measures: Number, route, and outcome of referrals from primary care and waiting times for urgent and routine appointments |
Main results: The annual number of referrals increased by 9% over the seven years from 3499 in 1999 to 3821 in 2005, a significant increase of 1.6% (95% CI 1.0% to 2.2%) The number of 2-week wait referrals increased by 42% (n = 739) from 1751 in 1999 to 2490 in 2005, an estimated increase of 5.8% per year (5.0% to 6.7%, p = 0.001). By contrast, the number of routine referrals has declined over the same period by an estimated 4.3% per year (3.3% to 5.2%, p < 0.001), giving an apparent reduction of 24% (n = 417) from 1999 to 2005 The percentage of patients diagnosed with cancer in the 2-week wait group decreased from 12.8% (224/1751) in 1999 to 7.7% (191/2490) in 2005 (p < 0.001), while the number of cancers detected in the ‘routine’ group increased from 2.5% (43/1748) to 5.3% (70/1331) (p < 0.001) over the same period. About 27% (70/261) of people with cancer are currently referred in the non-urgent group. Waiting times for routine referrals have increased with time |
Reported associations between elements for logic model: The 2-week wait rule for breast cancer is failing patients. The number of cancers detected in the 2-week wait population is decreasing, and an unacceptable proportion is now being referred via the routine route |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Prades 201175 Country: Spain Study design: Mixed method, retrospective data analysis + qualitative Data collection method: Analysis of seven fast-track hospital monitoring indicators + interviews with health professionals administrators Aim: To analyse the implementation and effectiveness of a fast-track referral system for cancer Detail of participants (number, any reported demographics): Data from 56,020 patients referred to 18 fast-track breast, lung or colorectal cancer clinics over a 3-year period. 83 health professionals from 18 fast-track clinics, 38% GPs, 22% clinical lead, 16% medical director, plus clinicians, nurse case managers, secretaries |
Intervention: Cancer fast-track programme – ‘circuits’ that would foster rapid coordination of the process of referral to a rapid diagnosis unit at a teaching hospital in each region Included clinical criteria (referral guidelines) for primary care referral and patient pathway management in hospital. Described as any process having a systematic approach which seeks to synchronise clinical needs by implementing passive (e.g. slots for diagnostic tests) or active measures (e.g. case management) leading to improved co-ordination Number of hours: NA Delivered by who? Staff in hospital Control: None Length of follow-up: None Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to hospital and intrahospital referral. Referrals could come via primary care route or emergencies or other clinical departments |
Outcome measures: Waiting time from detection of suspected cancer in primary care to start of initial treatment Categorisation of waiting times – over 30 days, 30 to 45 days, over 45 days Participant views on fast-track programmes |
Main results: Increase in completeness of hospital data during intervention period – 74% to 96% indicating degree to which programme had been implemented increased. Adherence to clinical criteria for including patients in fast track system was more than 70% (no specific data). About half of all new patients were diagnosed via the fast track. Cancer rate declined during the period. Mean time to treatment from primary care was 32 days for breast, 30 days for colorectal, 37 days for lung cancer No data for patients not referred via the programme to compare. No data on time to treatment prior to the programme being implemented to compare No qualitative data included, poorly reported. Description of professionals fearing overuse of system; however, this had not happened. One hospital was reported to have been overwhelmed with overindicated colorectal referrals. Professionals reported important to have length of the process clearly indicated. Poor feedback from the clinic to the referrer. 13 clinics had instituted ‘case management’ systems where nurses were case managers and acted as gatekeepers and reference points for a patient throughout the process Top-down systems where hospital managers design programmes or clinicians lead process of change limit spread of knowledge of guidelines and intrareferral mechanisms. Participatory strategies where a leader is appointed and implementation committee set up ensured an integrated development. Also common circuit across the three types of cancer could create disparity in implementation |
Reported associations between elements for logic model: Importance of implementation system Case manager system was effective, also integrated development approaches Half of patients referred via system, half not |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Ramsay 200327 Country: Canada Study design: Follow-up to RCT Data collection method: Further analysis of trial data in form of referral record analysis Aim: To further examine the effect of educational reminders on radiology referrals – do the effects vary by time period Detail of participants (number, any reported demographics): 40 control and 41 intervention practices, 1424 control referrals and 920 intervention referrals |
Intervention: Educational reminder message based on national guidelines added to the report of every radiograph ordered Original trial also had second intervention arm – audit and feedback to individual GPs at the start of the intervention and 6 months later. Practice level information relating to number of requests at each practice compared to all practices in the study was included in the audit. This arm not included in this additional analysis Number of hours: NA Delivered by who? NA Control: No intervention Length of follow-up: 12 months’ intervention period Response and/or attrition rate: 66% of control and 67% of intervention practices who participated in original study Context (from what/who to what/who): Radiology referrals (knee and lumbar spine radiographs) |
Outcome measures: Change in monthly total number of referrals for knee and lumbar spine radiographs per GP practice (adjusted for practice size) |
Main results: Previous study had found audit and feedback (arm 2) had led to a non-significant reduction of around 1% in radiograph requests. Educational reminder messages led to a statistically significant relative reduction of about 20% in radiograph requests. These figures were means across the intervention period Current analysis – number of knee radiograph referrals in intervention group was consistently below that of control group across the study period. Absolute mean reduction in referrals per month of 1.1 in intervention group. A similar pattern was observed in the lumbar spine radiograph referrals Evidence of seasonal variation. After adjusting for this – knee radiographs RR reduction = 0.65 and RR lumbar spine radiographs = 0.64. No decay over the intervention period Mean number of referrals per practice per month: Control 2.97 (SD 3.22) knee, 2.88 (SD 3.05) spine Intervention 1.87 (2.4) knee, 1.76 (2.38) spine |
Reported associations between elements for logic model: Effect of educational reminder, no effect of audit and feedback |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Ridsdale 2008124 Country: UK Study design: Evaluation Data collection method: Questionnaire Aim: To describe the training of GPwSIs in headache and the setting up of a GPwSI clinic in general practice, and report on a comparison with the existing neurology service Detail of participants (number, any reported demographics): Five GPwSIs were trained. Each agreed to provide a service for patients in their own practices A total of 211 consecutive patients with headache were sent the postal questionnaire Fifty-six hospital attendees responded, and 61 responded from the clinic run by the GPwSI. There was no significant difference in the age (mean 41.1 years, SD 15.2 years vs. mean 38.3 years, SD 2.78 years; p = 0.2) or in the proportion of each sex (57% female vs. 53% female; χ2 = 0.54; p = 0.77) of responders vs. non-responders, respectively |
Intervention: The intervention involved training GPs as GPwSIs and setting up a GP headache service. A questionnaire survey was conducted, measuring headache impact, satisfaction, and cost estimates Control: NA Length of follow-up: NA Response and/or attrition rate: 117 patients (55%) responded Context (from what/who to what/who): GP referral to neurology |
Outcome measures: Case severity, patient satisfaction, and cost |
Main results: There was no significant difference in headache impact between hospital (mean score 61.2, SD 10.4) and GPwSI clinic attendees (mean score 64.3, SD 9.3), after adjustment for age, sex, and ethnicity, mean difference 2.7; 95% CI 1.6 to 7.0). There was no measurable difference in the frequency of severe headache pain between the groups (AOR 1.05; 95% CI 0.47 to 2.32), or in how often headaches limited their daily activities (AOR 1.57; 95% CI 0.72 to 3.41) Patients were significantly more satisfied with the GPwSI service, particularly that the service was effective in helping to relieve their symptoms (89% vs. 76%; AOR 7.7; 95% CI 2.7 to 22.4) The cost per first appointment was estimated to be £136, with £68 for subsequent contacts. These are lower than costs for neurologist contact |
Reported associations between elements for logic model: GPwSI services can satisfy the needs of patients with similar headache impact at costs that are lower than those for secondary care services |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Robling 200260 Country: UK Study design: RCT Data collection method: NR Aim: To investigate whether method of access or method of guideline dissemination effects concordance with referral guidelines for MRI Detail of participants (number, any reported demographics): 123 practices in trial 1, 232 referrals. 121 practices continued to trial 2, 182 referrals |
Intervention: Two trials ran sequentially, one examining method of access and the second the method of guideline dissemination (1) One group practice request MRI by telephone or second group requested in writing using a standard request form, third group could refer as wished (2) One group practice received guidelines via a seminar, second group received feedback via a newsletter with practice-specific data on referrals. A third group received both seminar and feedback, and a fourth group received guidelines only by post Number of hours: NA Delivered by who? Seminar facilitated by academic GP and researcher Control: Seven interventions compared with each other Length of follow-up: NA Response and/or attrition rate: 121 practices continued to trail two Context (from what/who to what/who): GP request for lumbar spine or knee MRI scan |
Outcome measures: Concordance with local guidelines for referral Cost-effectiveness |
Main results: Trail one – 65% of requests judged to be concordant with guidelines. No difference in concordance rate between the three groups. Telephone access proved unpopular among participants; trial concluded before reached target. Written access more cost-effective Trail two – 74% judged to be concordant. No association between method of dissemination of guidelines and concordance Requests made after dissemination of guidelines were more likely to be concordant, 74% vs. 65% (OR 1.62, p < 0.005). No control group receiving no guidelines, however, so may be a trend rather than impact of guidelines Requests from larger practices more likely to be concordant (OR 1.18 per 1000 registered patients, p < 0.005) Cost per seminar £224.87 (£80 per trainee) Telephone access not popular; however, some GPs valued the direct feedback |
Reported associations between elements for logic model: Dissemination of guidelines had some impact on appropriacy of referral Method of accessing referral had no impact on appropriacy |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rosen 2006128 Country: UK Study design: Observational comparative cohort design Data collection method: Hospital data and questionnaires Aim: To evaluate the impact of GPwSI services on access to specialist care, user satisfaction and costs Detail of participants (number, any reported demographics): Four sites – three with dermatology GPwSI services and one in which a GPwSI musculoskeletal service has been developed as part of a wider reorganisation of orthopaedic, rheumatology and physiotherapy services |
Intervention: Quantitative analysis of GPwSI clinic and hospital outpatient activity data Referrals were compared from GP practices that had access to GPwSI clinics and those that did not Self-completed postal questionnaires were used to assess patient experiences of GPSI clinics and to assess the views of GPs referring patients to GPwSI clinics. Costs were assessed using a template of costs incurred in setting up and running the service from the perspective of PCT or hospital trust Control: No access to GPwSIs Length of follow-up: NR Response and/or attrition rate: NR Context (from what/who to what/who): GP to specialist |
Outcome measures: Changes in activity, referral rate and waiting times over two 6-month periods before and after the introduction of GPSI services Patient experiences Cost |
Main results: The association between the introduction of GPwSI clinics and hospital referral rates was variable and unpredictable There were no significant changes in hospital referral rates following the introduction of GPwSI clinics in any of the sites studied. Overall referrals to hospital and GPwSI clinics combined increased in the three sites for which data were available The likelihood of referral, calculated as the RR, adjusted for baseline and linear time trend, did not change after the launch of the GPwSI clinics in any of the sites studied. Small changes in risks of referral from studying control practices did not reach statistical significance. In the musculoskeletal site, where all practices had access to GPwSI clinics, there was a significant (p = 0.08) 13% increase in overall referrals The association between the launch of GPwSI clinics and hospital outpatient waiting times was variable. After adjustment for secular trends, there was evidence of decreased waiting times for hospital appointments after the introduction of the GPwSI service in two sites and of increased waiting times in two sites. Interview data revealed that changes in the staffing and organisation of each clinic might also have influenced these findings There was no consistency across sites in the methods used to monitor and evaluate the costs of establishing and running GPwSI clinics. Using available data, the cost per GPwSI appointment in each site ranged from £35 to £93. Data were not available to compare the costs of hospital and GPwSI clinics |
Reported associations between elements for logic model: That a GPwSI clinic would reduce hospital referrals from practices with access to that clinic relative to control practices was not supported by the data |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rowlands 200326 Country: UK Study design: Questionnaire data collected during RCT Data collection method: Survey of GPs who took part in an educational intervention (referral meetings) Detail of participants (number, any reported demographics): 13 practices in intervention arm, practices with four or more partners, health authorities in London. 60 of the 62 doctors eligible responded |
Methods: Questionnaire data collected during RCT of GPs who took part in an educational intervention (referral meetings) Control: None Length of follow-up: None Response and/or attrition rate: NR Context (from what/who to what/who): GP to specialist |
Outcome measures: Likert scale rating of how useful GP found the components of the intervention |
Main results: Educational value of the meetings was rated as mean 6.7 (range 4.4–8) on a scale where 0 = no value and 10 is extremely valuable 48 doctors found the meetings ‘useful’, remainder ‘not sure’ 98% valued opportunity to discuss problems with partners. Comments that doctors learned from each other, found out about new referral pathways and possible alternatives to referral Several doctors commented on patient pressure to refer, feedback mixed, overall feeling that referral meetings might have a short-term effect on enabling doctors to resist pressure for unnecessary referrals but that the effect might only be short term. Less than half of doctors became involved with development of formal referral or clinical protocols. 88% noted a change in their referral practice. More than half felt they used more internal referrals between GPs. More than one-third reported an increased tolerance of clinical uncertainty and more confidence to resist pressure from patients. 39% said they were more aware of financial implications to the practice of NHS referral. 29% more aware of financial implications to NHS as a whole. 20% of doctors said they had requested pathology tests more frequently No change on referral rate in intervention group |
Reported associations between elements for logic model: Value of educational intervention Patient pressure Tolerance of uncertainty Awareness of financial implications of referral Discrepancy between reporting perception of changed practice and actual change in practice |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Salisbury 2005125 Country: UK Study design: RCT Data collection method Aim: The aim of this study was to investigate the effectiveness, cost-effectiveness, accessibility and acceptability of a PCDS in comparison with a hospital outpatient clinic for dermatology Detail of participants (number, any reported demographics): 30 practices in one PCT area; n = 768 patients eligible. 556 (72%) were randomised: 354 to PCDS and 202 to outpatients |
Intervention: A RCT comparing patients referred to the PCDS with those receiving usual care at the hospital outpatient clinic The Bristol PCDS was staffed by two GPSIs and a specialist nurse, and provided from a suburban health centre. Patients are referred by their GPs to the outpatient Dermatology Centre at the Bristol Royal Infirmary as usual. Those who appear on the basis of their referral letter to be suitable for management in the PCDS are given an appointment there rather than at the outpatient department. At the time of the trial, suitable patients were adults with non-urgent skin conditions with a provisional diagnosis made by their GP Control: Usual care at the hospital outpatient clinic Length of follow-up: 9 months Response and/or attrition rate: 422 (76%) were followed up Context (from what/who to what/who): GP to dermatology |
Outcome measures: Clinical Satisfaction Did not attend Hospital referral |
Main results: The PCDS was more accessible [the difference between means on the access scale (scored out of 100) was 14, 95% CI 11 to 19; p < 0.001] and patients had reduced waiting times by a mean of 40 days (95% CI 35 to 46 days; p < 0.001) Patients expressed slightly greater satisfaction with PCDS consultations (difference in mean CSQ, 4%; 95% CI 1% to 7%; p = 0.011) and were more likely to prefer care at PCDS, both at baseline and after 9 months Fewer PCDS patients (6%) than hospital patients (11%) failed to attend their initial appointment, but overall did-not-attend rates for new and follow-up appointments were similar in both sites (PCDS, 8%; hospital, 11%). Of those patients seen initially at PCDS, 12% were referred to the hospital for one or more follow-up appointments There were no marked differences between the PCDS and hospital care in respect of clinical outcome (median Dermatology Quality of Life Index was 1 in both arms; ratio of geometric means, 0.99; 95% CI 0.85 to 1.15; p = 0.9, adjusting for baseline and stratification) |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Sanderson 2002126 Country: UK Study design: RCT Data collection method Aim: To assess the effectiveness, accessibility, and acceptability of a GPwSI service for skin problems compared with a hospital dermatology clinic Detail of participants (number, any reported demographics): 72% (556/768) of eligible patients agreed to participate and were randomised, 354 to the GPwSI service and 202 to hospital outpatient care |
Intervention: Participants were randomised in 2 : 1 ratio to receive management by a GPwSI or usual hospital outpatient care The Bristol GPwSI dermatology service is staffed by two GPwSIs and a specialist nurse. A consultant dermatologist provides clinical support for two sessions per month Suitable patients had non-urgent skin problems and had been identified from the referral letter as suitable for management by a GPwSI Control: Usual hospital outpatient care Length of follow-up: 9 months Response and/or attrition rate: After 9 months, 422 (76%) Context (from what/who to what/who): GP to dermatology |
Outcome measures: Primary outcomes were disease-related quality of life (dermatology life quality index) and improvement in patients’ perception of access to services, assessed 9 months after randomisation. Secondary outcomes were patient satisfaction, preference for site of care, proportion of failed appointments, and waiting times to first appointment |
Main results: 49% of the participants were judged suitable for care by the GPwSI service. Of 768 patients eligible, 556 (72.4%) were randomised (354 to GPwSI, 202 to hospital outpatient care) No noticeable differences were found between the groups in clinical outcome (median dermatology life quality index score = 1 both arms, ratio of geometric means 0.99, 95% CI 0.85 to 1.15) The GPwSI service was more accessible (difference between means on access scale 14, 11 to 19) and waited a mean of 40 (35 to 46) days less. Patients expressed slightly greater satisfaction with consultations with a GPwSI (difference in mean satisfaction score 4, 1 to 7), and at baseline and after 9 months 61% said they preferred care at the service |
Reported associations between elements for logic model: The GPwSI service for dermatology was more accessible and preferred by patients than hospital outpatient care, achieving similar clinical outcomes |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Sauro 2005127 Country: Italy Study design: nRCT Data collection method: Clinical records Aim: To demonstrate the effect of using guidelines and spirometry on the management of COPD and asthma Detail of participants (number, any reported demographics): Patients of 24 GPs. n = 32,785 |
Intervention: Three study groups (not clear if randomly allocated): group 1 – spirometer and practice training in its use, including information on guidelines (n = 11,050); group 2 – only info on guidelines (no spirometer) (n = 11,040); group 3 – control (n = 1049) Control: No training and proceeded as usual Length of follow-up: NR Response and/or attrition rate: NA? Context (from what/who to what/who): GP referral to respiratory |
Outcome measures: Referral Diagnosis |
Main results: COPD was diagnosed in 5.8% of group 1, 1.5% of group 2 and 2.3% of group 3 (p < 0.001, CI 99%) Group 1 performed the test in 65.7% cases of COPD or asthma. Group 2 referred 7.8% of patients. The control group requested the test in 96.8% of the cases. There were significant differences between prescribing and/or utilising spirometry between all three groups (p < 0.001, data not given) Group 1 referred 7.5% to a specialist and diagnosis was confirmed in 91.8% of cases. Group 2 sent 7.8% to the specialist and diagnosis was confirmed in 75.8%. The control group referred 96.8%, of whom 27.2% only had a confirmed diagnosis |
Reported associations between elements for logic model: Authors suggest guidelines alone are not useful, and could be dangerous |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Schillinger 2000144 Country: USA Study design: RCT Data collection method: NR Aim: To evaluate the effect of open access vs. physician approval of referral to ED and specialist services Detail of participants (number, any reported demographics): n = 1121 patients in intervention group, 1172 in control, uninsured who had at least one visit to family physician in previous 12 months |
Intervention: Patients required prior approval from their primary care physician in order to receive specialty care at the local hospital. Computer programme blocked scheduling of unapproved appointments for these patients. Primary care physicians were required to complete a consultation form including clinical information and number of visits requested prior to unlocking of system Number of hours: NA Delivered by who? Primary physician Control: Physician approval not required prior to accessing services, self-referral or physician referral Length of follow-up: Study over 1-year period Response and/or attrition rate: NR Context (from what/who to what/who): GP to hospital specialty or emergency care |
Outcome measures: Comparison of each patient utilisation of hospital inpatient and outpatient services in study year vs. previous year |
Main results: Intervention patients decreased specialty use by 0.57 visits per year more than control patients did (p = 0.04, 95% CI –1.05 to –0.01) Intervention group increased primary care use; however, this was not significant Changes in patient satisfaction with care, perceived access to specialists and use of services were similar between the two groups |
Reported associations between elements for logic model: Gatekeeping decreases specialty use |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Schulpen 2003134 Country: the Netherlands Study design: nRCT Data collection method: NR Aim: To evaluate joint consultation sessions Detail of participants (number, any reported demographics): 17 GPs within university hospital Maastricht; mean age 48.5 years; 12% female; 47% rural practice 43 non-participants acted as controls (characteristics comparable with those in intervention group). 166 patients; mean age 53.7 years; 27% male. 87 intervention group patients, 79 control group |
Intervention: Joint consultation session between GP and consultant held 6-weekly consisting of three GPs and one visiting rheumatologist at the practice of a host GP. GPs presented each patient, consultant examined patient and formulated diagnosis and therapy policy together with GP Number of hours: 54 sessions held (for 87 patients) average of 1.4 hours a session. 47 follow-up consultations with standard duration of 15 minutes Delivered by who? Visiting consultant rheumatologist Control: No intervention Length of follow-up: 2-year intervention period Response and/or attrition rate: 65% of GPs agreed to participate Context (from what/who to what/who): GP to rheumatology |
Outcome measures: Number of referrals to outpatient clinics for internal medicine, neurology, orthopaedic surgery, rheumatology Data over 1 year compared with previous year (this was year 1 and year 2 of study not pre intervention and during/post) Hours of consultant time |
Main results: 23 of the 166 patients were referred by intervention GPs without using the intervention (16 patient declined, four GP opinion not suitable, three other reasons) Prior to intervention increasing referral rate to rheumatology department By end of study period the number of patients referred by each GP per year differed – 62% in intervention group compared with controls Average reduction in referral rate to rheumatology of –2.8 (SD 3.9) at end of second year of intervention period compared with first year in the intervention group. In the control group referral rate difference of zero (SD 2.1). Difference between intervention and control group before and after was significant (p = 0.024, Mann–Whitney U-test). Change not found for referrals to other specialisms Total 87.4 hours of rheumatology consultant time in intervention group, 83.3 hours spent by consultants seeing control patients. Per patient 1-hour intervention, 1.1-hour standard clinic consultation Based on referral rates prior to the intervention if all patients had been referred to normal outpatient clinic would have taken 307.8 hours of consultant time. If all referrals during study period had been seen via the joint clinic system would have used 166.7 hours. Authors argue, therefore, a decrease of 46% in time spent by rheumatologist consultants Joint consultations, however, lead to increased workload for GP, although providing skills for GP may reduce referrals. In this study large decrease in referrals for one condition (fibromyalgia) in intervention group may be due to GPs more confident at managing without referral |
Reported associations between elements for logic model: Joint consultation effective in reducing referrals. Intervention and decrease in consultant hours Links between skilling GPs and reduction in referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Shariff 201028 Country: UK Study design: Cohort Data collection method: NR Aim: To quantify the effect of a targeted continuing medical education module on improving GP diagnostic accuracy for skin cancer Detail of participants (number, any reported demographics): NR |
Intervention: All referrals made under the 2-week wait rule to the skin cancer services of a teaching hospital in the north-east of England, covering a population of approximately 700,000 were initially studied for a 3-month period between July and August 2006. This included referrals to both the plastic surgery and dermatology departments The 94 GPs in the Hull and East Riding area were then sent a 12-page illustrated guide to common skin lesions positively highlighting common diagnostic pitfalls in the diagnosis of cutaneous melanoma and squamous cell carcinomas This educational module was aimed at building confidence in the diagnosis of lesions not requiring an urgent referral, especially basal cell carcinomas and seborrhoeic keratoses, referred through the 2-week wait route. After 11 months, all 2-week wait referrals were prospectively studied between August and October 2007 Number of hours: NA Delivered by who? NA Control: NA Length of follow-up: 11 months Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for suspected skin cancer |
Outcome measures: The main outcome measure was the percentage of correctly referred squamous cell carcinomas and melanomas |
Main results: 237 referrals were made between July and August 2006, and 223 referrals between August and October 2007 The proportion of appropriately referred skin cancers (squamous cell carcinomas and melanomas) was 23.2% before continuing medical education, and 20.6% after continuing medical education There were no differences in pick-up rates before and after the continuing medical education among suspected squamous cell carcinomas (21.1% vs. 29.7%) or melanomas (24.6% vs. 15.1%, respectively) Referrals to plastic surgery were more likely to be confirmed histologically as melanomas or squamous cell carcinomas (23.6% and 33.7%, respectively) than those made to dermatology (17.5% and 15.3%, respectively) |
Reported associations between elements for logic model: The proportion of correctly suspected skin malignancies under the 2-week wait initiative remains low despite education A targeted continuing medical education module sent to GPs fails to improve pick-up rates |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Shaw 200677 Country: UK Study design: cRCT Data collection method Aim: To determine the effects of providing a H. pylori serology service for GPs on demand for open-access endoscopy Detail of participants (number, any reported demographics): Forty-seven general practices in Gloucestershire |
Intervention: General practices were stratified by endoscopy referral rate and randomised into two groups. The intervention group was provided with access to H. pylori serology testing and encouraged to use it in place of endoscopy for patients aged under 55 years with dyspepsia. They were sent written information promoting the use of the serology service in place of endoscopy for patients under the age of 55 years suffering from dyspepsia without alarm symptoms and were issued with a summary of the Maastricht consensus statement on the management of H. pylori. The GPs remained free to refer for open-access endoscopy as they felt necessary Number of hours: NA Delivered by who? NA Control: Endoscopy usual care Length of follow-up: NR Response and/or attrition rate: NR Context (from what/who to what/who): GP referral for open-access endoscopy |
Outcome measures: Endpoints were referral for endoscopy and serology use |
Main results: The number of endoscopy referrals fell in both groups during the study period, but fell by a greater amount in the intervention group than in the control group. During the 2-year study period, 626 referrals were received from the intervention group compared with 771 from the control group, a crude reduction of 18.8% There was a significant reduction in referrals for endoscopy in the intervention group compared with the control group: 18.8% (95% CI 5.0% to 30.6%, p = 0.009) |
Reported associations between elements for logic model: Providing GPs with H. pylori serology testing reduced demand for open-access endoscopy, but by less than previous studies had predicted |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Simpson 2003143 Country: UK Study design: Before-and-after Data collection method Aim: To investigate the effect of employing counsellors in general practice on referral rates to mental health services Detail of participants (number, any reported demographics): 85 GP practices in South Derbyshire |
Intervention: The practices employed counsellor – the counsellors were well established and practices are allocated 6–12 hours per week Control: NA Length of follow-up: 8 years (retrospective data) Response and/or attrition rate: Context (from what/who to what/who): GP to psychiatry |
Outcome measures: Referral Drug prescribing |
Main results: The findings suggest that the cost of the counsellor could be offset elsewhere The provision of counselling had no statistically significant effect on referrals or the volume and cost of prescribing |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Simpson 201078 Country: UK (Scotland) Study design: Retrospective data analysis and survey of GPs Data collection method: Patient CT findings reviewed a questionnaire to referring GP and cost-effectiveness estimate Aim: To assess the significance of pathology, impact on service and GP satisfaction with direct access to CT for chronic daily headache Detail of participants (number, any reported demographics): n = 4404 scans on patients presenting with chronic headache for more than 3 months. n = 986 GPs completed survey |
Intervention: Direct access to head CT for GPs Number of hours: NA Delivered by who? GP carried out CT Control: None Length of follow-up: NA Response and/or attrition rate: All patients referred for head CT 1999 to 2007 (8 years) 23% return rate for GP questionnaires Context (from what/who to what/who): GP to specialist service for chronic headache (usually neurology or general medicine) |
Outcome measures: Number of referrals GP satisfaction Cost estimate |
Main results: 10.5% of scans indicated abnormalities. 1.4% of these were judged to be a potentially causative factor for the symptoms GPs reported that if direct access CT had not been available then 44% would have referred to neurology and 38% to general medicine. 10% of patients would not have been referred Following scan 86% did not require further specialist referral 67% of reports issued following scan were received in 1 week, 79% within 14 days. Content of report understood and acceptable to 99% of GPs, 1% reported terminology not understood. 100% of comments were positive about the service Without direct access 90% would have been referred cost estimate of £503,428. Cost of scan and outpatient appointments for patients in study estimated £602,026.80. A specialist headache clinic where typically 29% are referred for scans would cost £131,991.68 with a further review appointment costing total £688,708.61. Cost-saving estimate £86,681.81 |
Reported associations between elements for logic model: Reduced waiting time for patients Low number of referrals identified abnormality Lower uptake of resources – 83% fewer total number of patients referred for specialist opinion. Cost-effectiveness unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Slade 2008117 Country: UK Study design: cRCT (clustered by practice), some qualitative data on views of staff Data collection method: Rating scale of severity and appropriateness, observation of team meetings, ?interviews with staff (not reported how data collected) Aim: To investigate whether or not introducing standardised assessment of severity improved referral agreement Detail of participants (number, any reported demographics): 281 GPs from practices representative of population and including high deprivation inner-city areas. Two areas – London borough of Croydon and Manchester. 1061 mental health referrals to 11 CMHTs |
Intervention: GPs complete threshold assessment grid, a one-page assessment of mental health severity and attach to referral form/letter Number of hours: NA Delivered by who? NA Control: Usual referral practice Length of follow-up: NA Response and/or attrition rate: Data available for 1055 patients: n = 514 intervention, n = 541 control Context (from what/who to what/who): GP to CMHT |
Outcome measures: Appropriateness of referral Appropriate rating of urgency of referral Identification of appropriate professional to carry out the initial assessment Time taken for CMHT to discuss the referral |
Main results: Reasons for practices approached to opt not to take part in the study – too busy, too much paperwork already, not interested, do not see benefit, too complicated, not interested unless paid Implementation was low – threshold assessment grid used with 25% of referrals. Blocks identified by GPs were forgetting to use the threshold assessment grid, threshold assessment grid was too simple and did not reflect the complexity of the patient, concern that the threshold assessment grid could be manipulated to coerce CMHTs to accept referral, fear of being used to restrict referrals. Blocks identified by CMHTs were GPs not willing to complete and threshold assessment grid scores not being discussed at meetings. One area called service development and supported by mental health trust, other with lower participation rates considered study research. No significant difference between trial arms (p = 0.05) for any of the comparisons Appropriateness of referral was 64% vs. 60% (intervention vs. control p = 0.41 adjusted), rating of urgency was 81% intervention vs. 76% control (p = 0.15), identification of appropriate professional was 89% intervention vs. 87% control (p = 0.46), time to discuss referral by MHT was 2.08 vs. 2.15 minutes (p = 0.37) |
Reported associations between elements for logic model: Barriers to implementation No effect of intervention |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Spatafora 200569 Country: Italy Study design: Before-and-after Data collection method: NR Aim: To evaluate a shared protocol Detail of participants (number, any reported demographics): 45 urological centres, 263 GPs; mean age 47 years; 18% female Patient characteristics described in another paper, 50 years or older and in general good health. Mean age 64 years. n = 856 |
Intervention: A short algorithm on procedures to be used with men with LUTS was developed by urologists and approved by a panel of experts. It was presented at a meeting with local GPs and revised in line with feedback and revised protocol was presented at each centre. The protocol was a clinical report form containing history, examination, use and outcome of tests, diagnosis Number of hours: NA Delivered by who? NA Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to urology outpatient clinics |
Outcome measures: Percentage of patients managed by GP without referral Number of patients undergoing testing and cost of testing Time from first visit to final diagnosis |
Main results: 16% of centres accepted the original protocol with no changes No significant change in referral pattern from baseline to intervention – 51.2% of patients managed entirely by GP, 44.3% referred to urologist after some diagnostic procedures and 4.5% referred without any diagnostic testing Use of digital rectal exam increased significantly from 32% to 41%, p < 0.001 This predominantly in centres which endorsed this test Overall time to diagnosis was 29 days baseline and 28 days T2. Total costs average T1 = €71.82, and T2 = €61.93 Discussion of use of different diagnostic tests |
Reported associations between elements for logic model: Lack of change in referral patterns |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Stainkey 2010118 Country: Australia Study design: Service evaluation Data collection method: Patient data collected Aim: To evaluate a specialist appointment service for long-waiting patients Detail of participants (number, any reported demographics): 872 patients waiting for orthopaedic appointments in first wave, 6885 patients waiting for orthopaedic, ENT, neurosurgery, urology and general surgery appointments in second wave |
Intervention: Letters sent to patients who had been waiting 2 years or more (triaged by the hospital as non-urgent). Patients respond if appointment still needed, and seen at specially arranged clinics Number of hours: NA Delivered by who? Administrative staff send letter, special clinics delivered by hospital Control: None Length of follow-up: NA Response and/or attrition rate: 101 of 872 patients responded in wave 1. 532 of 6885 patients responded in second wave Context (from what/who to what/who): GP to orthopaedic, ENT, neurosurgery, urology and general surgery |
Outcome measures: Number of patients updating clinical information and attending a clinic Number of surgical procedures resulting from clinic attendance |
Main results: In first wave 16 patients required procedures (of the 101 who had responded to the letter and been seen in a clinic) In the second wave 532 patients responded to the letter and were seen in a clinic. 177 had surgical procedures resulting from these appointments |
Reported associations between elements for logic model: Process effective to identify small number of people on a long wait list in need of a procedure and identify those no longer in need of an appointment |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Standing 2001122 Country: UK Study design: Before-and-after Data collection method: Patient notes Aim: To investigate whether or not ambulatory ECG monitoring in general practice could decrease unnecessary referrals and pick up unsuspected cardiac abnormalities Detail of participants (number, any reported demographics): n = 73 (26 male, 47 female) patients from 27 GPs. 52 (71%) had no previous cardiac history, 12 (16%) had documented arrhythmia and 8 (11%) had ischaemia, one patient had a myocardial infarct and two had mitral valve incompetence |
Intervention: Patients were recruited to use a novel ambulatory ECG machine designed to detect arrhythmias in general practice Patients were selected if they had signs and symptoms indicative or cardiac abnormalities including dizzy spells, fainting, palpitations or pounding chest, as well as considering their medical history, and general profile Patients made two GP visits. On the first they underwent a normal consultation and the GP recorded any diagnosis made, whether or not he would refer the patient and, if so, what test he would request. The ECG device was fitted and the patient was given a diary card and general advice about the equipment. The patients ECG signal was then analysed for 24 hours. The patient was instructed to return to the surgery the next day where the GP reviewed the reported generated by the equipment and decided whether or not to refer the patient to the cardiology clinic Number of hours: 24 hours Delivered by who? GP Control: NA Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to cardiology outpatients |
Outcome measures: Referral to cardiology outpatients |
Main results: In the open study, 68% of patients were studied after reporting palpitations, others complained of dizziness and chest pain Following GP assessment prior to using the ECG machine, GPs were intending to refer 49 (68%) to cardiology outpatients for further tests. Of these, three cases were considered to need urgent appointments The ECG data identified 22 patients with atrial ectopic beats, and 13 with ventrical ectopic beats, 13 cases of intermittent tachycardia, seven of ST depression, two with pauses and two with missed beats. In seven patients no abnormality was detected. 3 further cases gave non-diagnostic results (probably due to poor fitting). The number of patients that GPs decide to refer to cardiology outpatients reduced by 60% from 49 to 19 patients. However, the number of patients identified as urgent increased from 3 to 7. 36 (of 49) were unlikely to need cardiology referral |
Reported associations between elements for logic model: Use of ambulatory ECG in primary care refines the screening process and has the potential to reduce referrals to cardiology outpatients |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Stoves 2010103 Country: UK Study design: Before-and-after Data collection method: Aim: Comparing nephrology referrals from implementation and non-implementation practices following the introduction of electronic consultations (e-consultations) for chronic kidney disease Detail of participants (number, any reported demographics): 17 general practices and a secondary care nephrology service in Bradford, UK Data from 17 implementation and 68 non implementation practices |
Intervention: Electronic sharing of primary care electronic health records with the nephrology service was introduced to implementation practices. Participating GPs attended education workshops and received paper and e-guidance about the new service The service allows GPs to send electronic referrals and share patient electronic health records with a renal specialist after first obtaining verbal patient consent. GPs use criteria agreed in local guidelines to ‘request advice’ or ‘question the need for hospital clinic review’ The renal specialist can open the electronic health record and a decision is then made whether a patient should be referred to clinic, undergo tests or interventions in the primary care setting, or continue to be monitored and treated by the primary care team Control: NA Length of follow-up: NA Response and/or attrition rate: NR Context (from what/who to what/who): GP to nephrology |
Outcome measures: The number, appropriateness and quality of new referrals (paper and electronic) from primary care, the timeliness of responses and the satisfaction of patients and health professionals with the new service |
Main results: There was a significant reduction in paper referrals from implementation practices. E-consultation provided nephrologists with access to more clinical information. GPs reported that the service was convenient, provided timely and helpful advice, and avoided outpatient referrals. Specialist recommendations were well followed, and GPs felt more confident about managing chronic kidney disease in the community The mean age of patients referred electronically was greater than patients referred by letter (72.8 years vs. 68 years, p < 0.01) The mean (SE) interval between the GP sending an e-consultation referral and the renal specialist submitting an electronic response was 7 (0.8) days. This contrasted with a mean wait of 55.1 (1.6) days between the GP sending a paper referral and the patient attending a hospital clinic When GPs were requesting clinic review by letter, only 56% of referrals were appropriate according to local criteria (71% and 52% for implementation and non-implementation practices, respectively), but 98% of these were accepted for hospital clinic review. This might have been due to insufficient information in some referral letters and a general reluctance to cancel appointments that had been pre-booked by the GP or the patient. By contrast, 90% of e-consultations that questioned the need for clinic review were appropriate, and clinic assessment was recommended in only 27% of cases |
Reported associations between elements for logic model: E-consultation promotes effective management of patients with mild to moderate chronic kidney disease in primary care, allowing specialist resources to be directed towards supporting patients with more complex needs |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Suris 200735 Country: Spain Study design: Before-and-after Data collection method: NR Aim: Effectiveness of GP training on referral Detail of participants (number, any reported demographics): Mean number of GPs in each health-care area was 11.7. Catalonia, Spain, served by 15 health-care areas with 117 GPs. One general hospital referral centre for 10 health-care areas with a rheumatology unit |
Intervention: Consultants carry out a biweekly session with GPs in each health-care area. Mean of eight GPs attended each session. Programme lasted for 1 year Number of hours: 1-hour biweekly, 120 sessions carried out by four rheumatologists. New cases discussed, local guidelines revised and discussed Delivered by who? NA Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to rheumatology |
Outcome measures: Activity – number of sessions, number of GPs attending, number of cases discussed Percentage of cases discussed that were not referred to the unit Number of referrals Mean waiting time GP satisfaction |
Main results: Average number of cases discussed in a session was 4.6, of these a mean of 1.43 cases from each session were referred. Consultancy resolution was, thus, 69% At end of pilot year total number of GP referrals was 31% lower than in previous year (1141 vs. 1652, no significance levels reported). Total number of new visits to the unit, however, increased 3.34% during the programme GP resolution rate for musculoskeletal disorders improved significantly. Referral rate to rheumatology unit decreased significantly from 8.13 per 1000 to 5.53 per 1000 (2.59%, 95% CI 2.09 to 3.10; p < 0.001) Waiting time to be seen dropped by a mean of 15 days over the study period (95% CI 119.7 to 9.2) GP satisfaction – no table in document printed – five items evaluated by GPs showed significant improvement (p < 0.001). Most improvement was in terms of patient accessibility |
Reported associations between elements for logic model: Providing outreach consultation can reduce referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Sved-Williams 201072 Country: Australia Study design: Before-and-after Data collection method: Referral data plus GP survey Aim: Described and evaluate a service that provides a single point of entry for GPs wishing to refer their patients for one off psychiatric consultations Detail of participants (number, any reported demographics): 45 psychiatrists, 301 GPs |
Intervention: All psychiatrists in the region were invited to provide reserved appointments to an administrative officer based at the Department of General Practice. They could specify the number of appointments, and withdraw unfilled appointments at any time. Mental Health Programme Officers in all Divisions of General Practice were briefed about the service and asked to advertise it to GPs in their area To make an appointment, a GP or practice nurse phoned a dedicated number Mon-Fri 9–5. There was no paperwork; the GP was supplied with the appointment time, along with the name and contact details of the psychiatrist over the telephone. GP and practice nurses involved in the referral process were surveyed at 10 months into the service Number of hours: NA Delivered by who? Psychiatrists Control: NA Length of follow-up: 28-month study Response and/or attrition rate: 26% of GPs responded to survey Context (from what/who to what/who): GP referral to psychiatry |
Outcome measures: Service use |
Main results: 824 patients were referred by 301 GPs. From August 2005 to March 2007 84% of offered appointments were filled 45/200 psychiatrists contributed appointments during the 28-month study period Use of the service rose from six referrals to 10 per week over the course of the study. 55% of psychiatrists continued to provide regular appointments after the study period Only 26% of GPs responded to the survey. Of those, 94% had received a written management plan within the specified 14-day period, and 40% had been verbally contacted by the psychiatrist |
Reported associations between elements for logic model: A single point of access for GPs to private psychiatric consultants increased access for many patients to expert assessment and management plans |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Tadros 200996 Country: UK Study design: RCT Data collection method: NR Aim: Referral of suspect skin cancers as well as non- malignant symptomatic skin lesions using high-quality digital images transferred via a secure electronic referral system vs. conventional pathways Detail of participants (number, any reported demographics): 300 patients referred by electronic referral service |
Method: A comparison of the diagnoses made from digital images with the diagnoses confirmed on pathology reports for lesions excised is described using a random selection of patients’ images and referrals. Intraobserver analysis was also assessed. A sample group of patients with lesions deemed as benign, not requiring surgery or other treatment and therefore not seen in secondary care were revisited at a special clinic to determine the safety of the referral system Control: Conventional referral Length of follow-up: NR Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for suspected skin cancer |
Outcome measures: Speed of referral, diagnosis and subsequent treatment |
Main results: In this series of 300 patients the study concludes that digital image referral for skin malignancy and other cutaneous lesions reduced the interval between referral and diagnosis by 81% and referral to commencement of treatment in suspect lesions by 30% Diagnostic accuracy in a random sample of 30 patients was comparable to that reported for patients seen in face-to-face consultations. High levels of GP and patient satisfaction were recorded |
Reported associations between elements for logic model: Digital image referral for skin malignancy and other cutaneous lesions is a safe and cost-effective referral pathway, significantly reducing the interval between referral diagnosis and onset of treatment for skin malignancy |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Tan 2007140 Country: UK Study design: Referral pattern analysis Data collection method: Analysis of new referrals 1996–2005 Aim: To evaluate the impact of restricted-referral guidance Detail of participants (number, any reported demographics): None (document only a brief report) |
Intervention: Restricted-referral guidance issued to GPs for dermatology referrals inspired by the Oregon Health Plan, a rationing policy. A list of conditions that the service would no longer treat or treat only in exceptional circumstances. Introduced in 2000 Number of hours: NA Delivered by who? NA Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to dermatology clinic |
Outcome measures: Number of referrals |
Main results: Following introduction a reduction in rate of increase of referrals occurred. For a further 3–4 years the volume of new referrals remained static Data only in the form of a chart – looking at the chart 1996 around 450 new referrals, 2000 peak of 800 new referrals, 2001–4 peak of around 600 referrals per year |
Reported associations between elements for logic model: Effectiveness of rationing system |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Thomas 200379 Country: UK Study design: cRCT Data collection method: Hospital/GP records Aim: To evaluate the effectiveness and efficiency of a guideline-based open-access urological investigation service Detail of participants (number, any reported demographics): 66 general practices in the Grampian region of Scotland referring 959 patients |
Intervention: General practices were randomised to receive either referral guidelines and access to the investigation service for LUTS or referral guidelines and access to the investigation service for microscopic haematuria Participating GPs were offered a 2-hour educational meeting and were mailed a guideline package, including a guideline booklet, quick reference flow chart and structured referral checklists Number of hours: 2-hour training Delivered by who? NR Control: Each condition provided a control for the other Length of follow-up: 12 months Response and/or attrition rate: NR Context (from what/who to what/who): General practice to urology |
Outcome measures: Compliance with guidelines (number of recommended investigations completed), number of general practice consultations, the number and case mix of referrals, waiting time to initial hospital appointment, and the number of patients with a management decision reached at initial appointment and discharged by 12 months after referral |
Main results: GPs’ compliance with referral guidelines increased (difference in means 0.5, 95% CI 0.2 to 0.8; p = 0.001). Approximately 50% of eligible patients were referred through the new system. The number and case mix of referrals were similar. The intervention reduced the waiting time from referral to initial outpatient appointment (ratio of means 0.7; 95% CI 0.5 to 0.9, patients with LUTS only) and increased the number of patients who had a management decision reached at initial appointment (OR 5.8, 95% CI 2.9 to 11.5; p < 0.001, both conditions) Patients were more likely to be discharged within 12 months (OR 1.7, 95% CI 0.9 to 3.3; p = 0.11). There were no significant changes detected in patient outcomes. Overall, the new service was probably cost saving to the NHS |
Reported associations between elements for logic model: The guideline-based open-access investigation service streamlined the process of outpatient referral, resulting in a more efficient service with reduced outpatient waiting times, fewer outpatient and investigation appointments and release of specialist and clinic time |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Thomas 201080 Country: UK (Scotland) Study design: Analysis of referral data Data collection method: Analysis of patient data, questionnaire to GP Aim: To estimate the referral rate of patients with chronic headache to open-access CT and the effect on neurology referral rates Detail of participants (number, any reported demographics): n = 232 patients referred for brain CT. Patients aged older than 18 years with symptoms of chronic unchanging headache (duration not specified). Age range 20–85 years. 72 GP practices; 309 GPs |
Intervention: GP direct access to CT brain scans at three sites Number of hours: NA Delivered by who? GPs Control: None Length of follow-up: 1 year Response and/or attrition rate: 88% of GPs returned questionnaire 1 year follow up for 215 patients Context (from what/who to what/who): GP to specialist neurology referral |
Outcome measures | Main results: Referral rate to open-access service was 1.2% of headache consultations by GPs. Open-access scans accounted for 4% of the annual number of scans Scan not performed for 17 of the 232 due to non-attendance, cancellation or incomplete details. 215 scanned. Of these, three lesions were identified which may have caused chronic headache. 1.4% yield for significant findings and 10.2% for non-significant findings. 88.4% of scans were normal Service used by 45% of GPs from 82% of practices. Highest number from any one practice was seven, most referred one GPs reported that in 88% of cases the scan had stopped a referral to secondary care. The results of the scan were said to have caused a referral in 5% of cases. In 6% of cases a referral had been made at the same time as the scan requested One-year follow-up of the patients scanned – 14% (30) were subsequently referred to neurology clinic because of headaches; of these, 40% at the same time as CT scan request and 60% referred after their brain-scan CT. Of these later referrals 17 of the 30 had normal CT findings |
Reported associations between elements for logic model: Scanned patients had a lower referral rate to neurology immediately and in the year following the scan Referrals of scanned patients to neurology were reduced by 86% |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Tierney 2003116 Country: UK Study design: RCT Data collection method: Electronic patient data Aim: To assess the effects of computer-based cardiac care suggestions Detail of participants (number, any reported demographics): 706 outpatients with heart failure and/or ischaemic heart disease |
Intervention: Evidence-based cardiac care suggestions, approved by a panel of local cardiologists and general internists, were displayed to physicians and pharmacists as they cared for enrolled patients. Evidence-based guidelines published by the Agency for Health Care Policy and Research and national professional organisations were used to develop the cardiac care rules During the study period, physicians received a variety of patient-specific feedback about various clinical issues. For patients in the physician control group, these suggestions were withheld. For patients in the physician intervention group, the cardiac care suggestions were printed at the end of the medication list on the encounter form and displayed as ‘Suggested Orders’ on physicians’ workstations. Physicians could view the guidelines and references via the ‘help’ key. They could avoid all suggestions made for that patient that day by hitting the ‘escape’ key Number of hours: NA Delivered by who? NA Control: No intervention Length of follow-up: Each firm had an equal number of intervention and control sessions Response and/or attrition rate: NR Context (from what/who to what/who): GP referral to cardiology |
Outcome measures: Adherence with the care suggestions, generic and condition-specific quality of life, acute exacerbations of their cardiac disease, medication compliance, health-care costs, satisfaction with care, and physicians’ attitudes towards guidelines |
Main results: Subjects were followed for 1 year during which they made 3419 primary care visits and were eligible for 2609 separate cardiac care suggestions The intervention had no effect on physicians’ adherence to the care suggestions (23% for intervention patients vs. 22% for controls). There were no intervention-control differences in quality of life, medication compliance, health-care utilisation, costs, or satisfaction with care. Physicians viewed guidelines as providing helpful information but constraining their practice and not helpful in making decisions for individual patients |
Reported associations between elements for logic model: Care suggestions generated by a sophisticated EMR system failed to improve adherence to accepted practice guidelines or outcomes for patients with heart disease |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Twomey 200342 Country: UK Study design: Before-and-after Data collection method: An audit of GP referral for plain radiographs in the following areas – hip, knee, cervical spine, and lumbar spine – was undertaken for the calendar years 2000 and 2001 Aim: To decrease GP referral to radiology Detail of participants (number, any reported demographics): None |
Intervention: Established a procedure for the development of care pathways This procedure consisted of a framework for considering evidence and engaging a wide range of stakeholders which was followed for the development of a local guideline called Making the Best Use of Our Local Radiology Department. It considered the appropriate utilisation of plain radiography within a number of diagnostic areas. The rationale of this element of the guideline was to facilitate an evidence-based approach to diagnostic use of plain radiography by local GPs with the anticipated reduction in overall number of requests. The proposal was discussed at the PCG Clinical Governance and shared with rheumatology and orthopaedic consultants. The proposed guidelines were circulated to all GPs within North East Lincolnshire PCT to be utilised for future referrals within clinical areas specially identified Number of hours: NR Delivered by who? NA Control: None Length of follow-up: NR Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to radiology for radiograph (hip, knee, cervical spine and lumbar spine) |
Outcome measures: Referral to radiology |
Main results: GP referrals to radiology department for plain radiography declined: Year 2000, n = 2365 Year 2001, n = 1077 Reduction: Total 1288 % 54 The audit demonstrates a significant reduction in referrals across all diagnostic areas reviewed Total number of plain radiograph requests: Year 2000, n = 6650 Year 2001, n = 4291 Reduction: Total 2359 % 35.5 |
Reported associations between elements for logic model: The reduction in primary care referrals for plain radiography for the priority areas within the guideline is mirrored by an overall reduction in primary care plain radiography requests The audit demonstrates a very positive response by local GPs and reflects an effective and shared implementation of the guideline |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| van Bokhoven 2012120 Country: the Netherlands Study design: Cluster RCT randomised at GP practice level Data collection method: Test ordering numbers, questionnaires Aim: How feasible is watchful waiting compared to immediate blood test ordering? Detail of participants (number, any reported demographics): n = 498 patients presenting with unexplained complaints that did not cause alarm for the GP including fatigue, abdominal complaints, weight change, musculoskeletal complaints, itch. Mean age 43 years; 28% male. 57 Dutch GP practices; 63 GPs; mean age 45 years; 74% male |
Intervention: Group A: Watchful-waiting approach Group B: Watchful waiting plus ‘quality improvement strategy’ – two small group meetings including explanation of diagnostic value of tests, discussion of difficulties dealing with patients with unexplained complaints, goal setting to change GP behaviour. On practice visit Number of hours: 5 hours small group meeting plus one hour practice visit, plus homework tasks for group B Delivered by who? Not described in this paper, presumably members of the research team? Control: GPs ordered tests immediately Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to any specialty for unexplained symptoms |
Outcome measures: Percentage of patients for whom tests ordered Number of tests ordered at first consultation Number of tests ordered after 4 weeks GP communication and performance (self-reported and patient reported) |
Main results: No statistically significant difference between the two intervention groups in terms of number of patients for whom tests were ordered or GP performance (performs adequate examination, explains findings to patient) GPs in intervention groups ‘better scores’ for knowledge about seriousness of complaint and GP asks patient to return if symptoms do not resolve First consultation GPs ordered mean of seven tests in both control group and trained intervention group, six in untrained intervention group. 52 of the 498 patients returned to the GP after 2 weeks for a further consultation Lack of effect of training however this may be due to limited room for improvement Author conclusion: Watchful waiting is a feasible approach for patients presenting with unexplained symptoms. Laboratory testing rarely used later on as few patients returned |
Reported associations between elements for logic model: Lack of effect of providing GP training Watchful waiting can reduce GP use of blood testing and, therefore, potentially could reduce onward referral if tests are false positive |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Van Dijk 2011123 Country: the Netherlands Study design: Retrospective medical record analysis Data collection method:Electronic medical record data examined for patients where benign neoplasm skin/naevus, sebaceous cyst or laceration/cut and/or minor surgery was performed Aim: To examine associations between number of minor GP surgical interventions and hospital referrals Detail of participants (number, any reported demographics): Data from 48 GP practices – n = 14,202 patients, mean age 39 years, 51% female. 4440 laceration/cut, 5373 benign neoplasm skin/naevus, 2220 sebaceous cyst |
Intervention: GP providing minor surgery intervention in primary care Number of hours: NA Delivered by who? GP Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to hospital |
Outcome measures: Number of referrals Type of surgery Distance to hospital Presence of a primary care nurse GP workload |
Main results: GP practices that performed more minor surgery had a lower referral rate for patients with a laceration/cut (–0.38, 95% CI –0.6 to –0.11), and for patients with a sebaceous cyst (–0.42, 95% CI –0.63 to –0.16) but not for those with benign neoplasm skin/naevus (–0.26 95% CI –0.51 to 0.03) Minor surgery was more often performed in older patients. Presence of primary care nurse only affected referral for benign neoplasm. Women had a smaller likelihood of minor surgery and smaller likelihood of referral for sebaceous cysts Significant negative correlation between minor surgery intervention and referrals at a practice level. For laceration/cut and sebaceous cysts GP practices that perform more minor surgery interventions refer fewer patients to a medical specialist. Presence of a primary care nurse and GPs workload showed a higher negative correlation Author conclusion: the effect of minor surgery on the rate of referral varied by diagnosis. Minor surgery associated with fewer referrals for sebaceous cysts and laceration/cuts but not for benign neoplasm skin/naevus. Absolute referral rate appeared only relevant for sebaceous cysts Performing five more minor surgery interventions per 100 care episodes would result in 4.3 fewer referrals for sebaceous cyst. In the UK an increase of 5% in GP minor surgery interventions for sebaceous cyst would result in saving of about £3000 |
Reported associations between elements for logic model: Referral management differs by type of diagnosis |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Van Dijk 2010141 Country: the Netherlands Study design: Retrospective patient data analysis Data collection method: Referrals to internists, ophthalmologists, cardiologists or mental health care Aim: To assess whether or not the introduction of primary care nurses affects referral rate for diabetes-related hospital treatment Detail of participants (number, any reported demographics): 25 practices in wave 1 (2004) and 29 in wave 2 (2006) described as a representative sample of Dutch general practices. Newly diagnosed type 2 diabetes patients. n = 301 in wave 1, 450 in wave 2. 50% male, mean age 61 years, mean distance to hospital 8.6 km, 39% unrelated comorbidity |
Intervention: Introduction of primary care nurses (online file gives description of introduction process if needed) Number of hours: NA Delivered by who? Primary care nurses Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to secondary care for diabetes-related conditions |
Outcome measures: Referrals Diabetes-related contacts Primary care nurse presence Comorbidity Distance to hospital |
Main results: Referral rates tended to be low on average to internists, cardiologists and mental health care Referral rate to internists for newly diagnosed patients decreased for practices both with and without a practice nurse between the two time points (7.3% vs. 3.3%) The trend in referral patterns to internists for known diabetic patients was lower in general practices with primary care nurses than those without (OR 0.59, 95% CI 0.31 to 1.11; significant p < 0.1) The number of diabetes-related contacts did not differ between practices with and without primary care nurses |
Reported associations between elements for logic model: Referral rates changed over time potential impact on interpretation of study findings No significant difference introduction of practice nurses |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Van Dijk 2013149 Country: the Netherlands Study design: Before-and-after Data collection method: patient EMRs Aim: The effects of replacing separate remuneration systems for publicly insured patients (capitation) and privately insured patients (fee-for-service) with a combined system of capitation and fee-for service for all on the quality of care in terms of guideline adherence were investigated Detail of participants (number, any reported demographics): 21,421 to 39,828 patients from 32 to 52 general practices |
Method: A longitudinal study from 2002 to 2009 using data from patient EMRs in general practice. A multilevel (patient and practice) approach was applied to study the effect of changes in the remuneration system on guideline adherence Control: Length of follow-up: Response and/or attrition rate: Context (from what/who to what/who): GP referral for chronic and acute conditions |
Outcome measures: Sixteen guideline adherence indicators on prescriptions and referrals for acute and chronic conditions |
Main results: Guideline adherence increased between 2002 and 2008 by 7% for (formerly) publicly insured patients and 10% for (formerly) privately insured patients. In general, no significant differences in the trends for guideline adherence were found between privately and publicly insured patients, indicating the absence of an effect of the remuneration system on guideline adherence Adherence to guidelines involving more time investment in terms of follow-up contacts was affected by changes in the remuneration system. For publicly insured patients, GPs showed a higher trend for guideline adherence for guidelines involving more time investment in terms of follow-up contacts compared with privately insured patients |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Vardy 2008150 Country: Israel Study design: Evaluation Data collection method: Mixed method – attendance data, interviews with patients and PCPs Aim: To evaluate a copayment system Detail of participants (number, any reported demographics): Telephone interview data from n = 3745 patients, 10 urban clinics, who had been referred 48 PCPs interviewed, 54% female, average age 45.5 years |
Intervention: Patient payment for attending specialist consultation. Described only as ‘a relatively low fixed sum to be paid prior to the appointment’ Number of hours: NA Delivered by who? NA Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to ENT, dermatology, orthopaedics, ophthalmology, cardiology, neurology, hypertension and diabetes. Included PCP and self-referral |
Outcome measures: Reasons for not attending Sociodemographics |
Main results: Attendance at planned appointment was 85% for specialist appointments in community and 91.7% for specialist hospital appointments in the time period when copayment was in operation. No difference self referral and physician referral. Only 2% reported copayment as the reason for not attending. 19% stated that copayment was a reason for not attending an appointment in the past Physicians stated need for copayment influenced referral decision especially with elderly or lower income patients |
Reported associations between elements for logic model: Payment and service use |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Vlek et al. 2003135 Country: the Netherlands Study design: RCT Data collection method: NR Aim: To evaluate joint consultation sessions Detail of participants (number, any reported demographics): 49 GPs and 13 cardiologists. 306 patients. Mean age of patients 58 years |
Intervention: Monthly joint consultation session between GP and cardiology specialist held over 18 months in surgery of GP. 3–4 patients could be examined and discussed at each session. Average seven sessions per GP (range 2 to 13 sessions) Number of hours: NA Delivered by who? Cardiologist Control: Normal care Length of follow-up: 1 year later patient assessed by a different cardiologist Response and/or attrition rate: 23% withdrawal in both groups Context (from what/who to what/who): GP to cardiology |
Outcome measures: Number of patients referred |
Main results: Fewer patients in the intervention group were referred to a cardiologist (33% vs. 52%, p = 0.001). The difference in referral rates showed an average decrease of referrals to cardiology of 6 per 1000 patients in the GPs from the intervention group. Further diagnostic procedures were required for 7% in intervention group vs. 16% in control group (p = 0.013) |
Reported associations between elements for logic model: Referral to specialist services decreased |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Walkowski 200763 Country: USA Study design: cRCT Data collection method: NR Aim: To test the effect of different academic detailing strategies to (1) inform PCPs of the high-performing cardiac specialists in their community and (2) facilitate increased referrals to these specialists Detail of participants (number, any reported demographics): General Practice, Internal Medicine and Family Practice physicians in five markets (Denver, Dallas, San Antonio, Austin and Dayton) |
Intervention: This initiative involved sending letters to primary care physicians which requested that when the physician had a patient needing referral to a cardiac specialist or facility, that they refer them to a physician or facility that had earned the United Health Premium designation for both Quality and Efficiency of care. To facilitate those referrals, the primary care physicians were also provided with a hard-copy referral list of cardiac specialists and hospitals in their community, which could be posted at the referral desk or the receptionist desk. To determine the most effective approach to academic detailing, the pilot was divided into four test groups – (1) LRL only (n = 3537), (2) LRL plus follow-up telephone call from the local health plan (n = 252), (3) LRL plus e-mail reminder (n = 1187) and (4) LRL plus in-person follow-up visit from the local Market Medical Director (n = 65) Number of hours: NA Delivered by who? NA Control: Two control markets with similar populations and United Health Premium penetration did not receive intervention letters Length of follow-up: 3 months Response and/or attrition rate: NA Context (from what/who to what/who): General Practice referrals to cardiology |
Outcome measures: Appropriate referral to a cardiology specialist |
Main results: Initial 3-month pilot data showed an overall 6.3% increase of patients referred to a United Health Premium designated quality and efficient cardiac specialists overall compared with a baseline period of 12 months prior to the mailing. These results were based on evaluation of claims for the 3 months after the distribution of the letters. Intervention effects ranged from +17% change (letter + call) to + 22% change (letter + visit), vs. 0.3% change in the control group |
Reported associations between elements for logic model: Academic detailing to community-based PCPs can increase referrals to high-performing specialists in open-access care models. The impact varies as a function of delivery method |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Wallace 200494 Country: UK Study design: RCT Data collection method Aim: To test the hypotheses that virtual outreach would reduce offers of hospital follow-up appointments and reduce numbers of medical interventions and investigations, reduce numbers of contacts with the health-care system, have a positive impact on patient satisfaction and enablement, and lead to improvements in patient health status Detail of participants (number, any reported demographics): 134 GPs from 29 practices and 20 consultant specialists |
Method: A RCT comparing joint teleconsultations between GPs, specialists and patients with standard outpatient referral Control: Usual care Length of follow-up: The patients were followed 6 months after their index consultation Response and/or attrition rate: 3170 patients were referred, of whom 2094 consented to participate in the study and were eligible for inclusion. In all, 1051 patients were randomised to the virtual outreach group and 1043 to standard outpatient appointments Context (from what/who to what/who): GP to specialists |
Outcome measures: Hospital follow-up appointments Numbers of medical interventions and investigations Numbers of contacts with the health-care system Patient satisfaction and enablement Patient health status |
Main results: 52% of patients in the virtual outreach group were offered a follow-up appointment, compared with 41% in the standard outpatient group Per-protocol analysis: The overall proportion of patients receiving an offer of follow-up was 46% in the virtual outreach group and 42% in the standard outpatient group (OR 1.19, 95% CI 0.99 to 1.44), but significant heterogeneity remained for both site and specialty (p = 0.001 and p < 0.001, respectively) Fewer tests and investigations were ordered in the virtual outreach group, by an average of 0.79 per patient In the 6-month period following the index consultation, there were no significant differences overall in number of contacts with general practice, outpatient visits, accident and emergency contacts, inpatient stays, day surgery and inpatient procedures or prescriptions between the randomised groups Patient satisfaction was greater after a virtual outreach consultation than after a standard outpatient consultation, with no heterogeneity between specialties or sites |
Reported associations between elements for logic model: Virtual outreach increased follow-up consultation |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Watson 200132 Country: UK Study design: cRCT Data collection method: NR Aim: To investigate the effect of an in-practice educational session and information pack on GP management of familial breast/ovarian cancer cases Detail of participants (number, any reported demographics): GP principals in 170 practices in Oxfordshire and Northamptonshire |
Intervention: Practices were randomised to group A (receiving an in practice educational session plus information pack), group B (receiving an information pack alone) or group C (receiving neither an educational session nor a pack). The pack contained a laminated summary card with simple referral guidelines, a booklet with more detailed background information, and two patient leaflets The educational session lasted 1 hour and was structured around a series of overheads covering the information presented in the pack. Interaction was encouraged. Sessions were conducted by one of two researchers, both experienced in the field of cancer genetics and teaching. Each participant was asked to complete a short questionnaire, which asked for a rating of the session and assessed levels of knowledge and confidence before and after the session, using a four-point Likert scale. Session participants were told that they would be sent a follow-up questionnaire, but no details regarding the content of this questionnaire were provided Number of hours: NA Delivered by who? Experienced researchers Control: Group C above Length of follow-up: The interventions were conducted between March and November 1999 Response and/or attrition rate: Of the 688 GPs, 426 (62%) participated Context (from what/who to what/who): Primary care referral to oncology for familial breast/ovarian cancer |
Outcome measures: Proportion of GPs making the correct referral decision on at least five out of six family history vignettes. GPs’ reported confidence in managing patients with a family history of breast/ovarian cancer, measured by a score that was generated by combining responses to four questions |
Main results: There was a 40% (95% CI 30% to 50%, p < 0.001) improvement in the proportion of GPs who made the correct referral decision on at least five out of the six vignettes in group A (79%) compared with the control group (39%) and a 42% (95% CI 31% to 52%, p < 0.001) improvement in group B (81%) compared with the control group (39%). There was no significant difference between groups A and B (table 1). Groups A and B performed better than group C for each of the six vignettes. Group C had most difficulty with vignette 5 (a family history on the father’s side). Exclusion of this vignette from the analysis did not change the overall finding. There was almost complete agreement between risk assessment and referral decisions across all three groups; i.e. where risk was assessed as low, GPs indicated they would not refer and where risk was assessed as higher, GPs indicated they would make a referral. Sixty-seven per cent (95% CI 60% to 75%) of GPs in group A, 75% (95% CI 68% to 83%) of GPs in group B, and 16% (95% CI 10% to 21%) of GPs in group C reported using guidelines when answering all/some of the vignettes. The difference in proportions using guidelines was not statistically significant (p = 0.11) There was a significant trend in levels of confidence, with GPs in group A (mean confidence score = 2.3) reporting greater confidence than those in group B (mean confidence score = 2.0), who in turn reported greater confidence than those in group C (mean confidence score = 1.5, p < 0.001) |
Reported associations between elements for logic model: Providing GPs with an information pack significantly improved referral decisions regarding patients with a family history of breast/ovarian cancer |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Watson 2002152 Country: UK Study design: Audit Data collection method: retrospective case note review Aim: Evaluated the impact of introducing a multidisciplinary single point of referral system for dedicated adult learning disability health services Detail of participants (number, any reported demographics): NR |
Intervention: A retrospective case note review comparing referrals to a single-point referral system with those to the old referral system The single-point referral system used common referral criteria and a streamlined information system. A new referral form and information leaflet were developed and copies distributed to social workers, data centre managers, GPs and colleges of further education Number of hours: NA Delivered by who? Single-point referral co-ordinator Control: None Length of follow-up: NR Response and/or attrition rate: NR Context (from what/who to what/who): Social workers, data centre managers, GPs to dedicated adult learning disability health services |
Outcome measures: Waiting time for referral Inappropriate referral |
Main results: With the introduction of the single-point referral system, the mean waiting time for referral to assessment was reduced from 46 (15–67) days to 6 (2–9) days. The proportion of inappropriate referrals halved from 26% to 13%. The proportion of appropriate referrals that involved more than one dedicated learning disability health professional increased from 63% to 80% |
Reported associations between elements for logic model: The single-point referral system facilitated prompt allocation of referrals to the appropriate professionals with significant reduction in waiting times |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| West 200752 Country: UK Study design: Before-and-after Data collection method: Audit of GP records and feedback questionnaire Aim: To assess the impact of written guidelines on GP referrals to an orthopaedic outpatient department Detail of participants (number, any reported demographics): NR |
Intervention: All GP referral letters for new patients with the chosen conditions in a 13-week period were audited. Letters referring patients for six specific orthopaedic complaints, namely anterior knee pain, back pain, carpal tunnel syndrome, in-toeing in children, sciatica and tennis elbow, were selected. Paper copies of referral guidelines produced by orthopaedic consultants were then distributed to all local GPs After a period of 4 weeks for distribution, the process was repeated for a further 13 weeks. Each letter was analysed for its content of therapy or management already tried by the GP prior to referral, as suggested in the guidelines. GPs were unaware that a referral letter audit was being undertaken. A feedback questionnaire was sent to GPs in the Swindon Primary Care Trust to assess the distribution and use of the guidelines Number of hours: NA Delivered by who? NA Control: NA Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to orthopaedics |
Outcome measures: Referral quality |
Main results: In total 471 referral letters were assessed: 304 before the provision of guidelines and 167 afterwards The first 13-week period had 195 (64%) referrals that consisted of patients either who had not received the recommended management, or for whom this had not been mentioned in the referral letter. The second period had 103 (61%) There was no statistically significant difference (p = 0.49) and therefore little evidence that the implementation of guidelines had an effect on the management of patients prior to referral or the consequent timing of seeking specialist opinion |
Reported associations between elements for either: The provision of orthopaedic written referral guidelines to GPs does not affect the pre-referral management of patients, or the overall number of referrals to the specialty |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| White 2000142 Country: UK Study design: Cross-sectional survey Data collection method: Questionnaire Aim: To evaluate the impact of counsellors in primary care on referrals to mental health services Detail of participants (number, any reported demographics): NR |
Intervention: Looked at the impact of counsellors in primary care. No new intervention Number of hours: NA Delivered by who? NA Control: NA Length of follow-up: NA Response and/or attrition rate: 91.1% or practices responded Context (from what/who to what/who): general practice to mental health services |
Outcome measures: Referrals to mental health services |
Main results: A counsellor was present at 20.3% of practices A random sample of 180 referrals to CMHTs was reviewed: 76 (42.2%) from practices that employed and a counsellor, and 104 (57.8%) from practices that did not There was a significantly higher referral rate from practices that employed a counsellor (p = 0.003). There was no evidence of a difference in rates of appropriateness of referrals between practices that employed a counsellor and those that did not |
Reported associations between elements for logic model: Practices employing counsellors had significantly higher rates of referral to mental health services, with no difference in the level of appropriateness between the two |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| White 200461 Country: UK Study design: Audit Data collection method: Referral letters Aim: To improve communication between GPs and CMHTs Detail of participants (number, any reported demographics): Patients newly referred to two inner-city CMHTs |
Intervention: Following a bench-mark audit of a random sample of referrals and assessment letters, locally agreed good practice protocols were developed and shared widely accompanied by a dissemination and implementation strategy Control: None Length of follow-up: 2 years Response and/or attrition rate: NA Context (from what/who to what/who): GP to mental health |
Outcome measures: Referral letter content |
Main results: Significant improvements occurred in both the GP and the CMHT letters; these were most dramatic after 1 year but tailed off considerably in the second year despite continued efforts to implement the protocol’s standards |
Reported associations between elements for logic model: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Whited 200287 Country: USA Study design: RCT Data collection method: NR Aim: To compare usual care (text-based electronic consultation) with teledermatology (usual care plus digital images and standardised history) Detail of participants (number, any reported demographics): NR |
Intervention: Teledermatology – text-based electronic consultation plus digital images and standardised history. History includes demographic information, patient-reported medical history, dermatology history, lesion location, size and duration of presence. The consultant answered the consult by scheduling an appointment, or by relaying a diagnosis and management plan back to the GP Number of hours: NA Delivered by who? GP/dermatologist Control: Usual care (text-based electronic consultation) Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to dermatology |
Outcome measures: Time to intervention Referral to (attendance at) dermatology clinic |
Main results: Patients in the intervention arm reached time to initial defined intervention sooner than those in the usual-care arm (median 41 days vs. 127 days, p < 0.001, log-rank test) 18.5% of patients in the intervention arm avoided the need for a dermatology clinic visit compared with 0 patients in the usual-care arm (p < 0.001, z-test) |
Reported associations between elements for logic model: This teledermatology intervention resulted in significantly shorter times to initial defined intervention than did a text-based-only electronic consultation. The need to attend for a clinic visit was avoided in some cases, but the appropriateness of this decision was not further considered |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Whited 200488 Country: USA Study design: Cross-sectional Data collection method: Survey Aim: To assess satisfaction with and acceptance of a store and forward teledermatology system Detail of participants (number, any reported demographics): Patients (135 intervention, 140 control), referring physicians (60) and dermatologists (8) |
Intervention: Satisfaction survey as part of a RCT that compared the clinical and economic outcomes of a store-and-forward teledermatology intervention with a conventional referral process Control: Conventional referral process Length of follow-up: NA Response and/or attrition rate: 88% GPs, 75%/66% patient intervention/control Context (from what/who to what/who): GP to dermatology |
Outcome measures: Satisfaction Referral |
Main results: A majority of referring clinicians (92%) and dermatologists (75%) reported overall satisfaction with the teledermatology consultation process. 95% of referring clinicians reported that teledermatology resulted in more timely referral, and teledermatology patients reached a point of initial interventions than patients receiving conventional referral (41 days vs. 127 days, p < 0.001) Teledermatology patients reported satisfaction with the outcome of their consultation 82% of the time, but did not express a clear preference for one consultation method (41.5% preferred teledermatology, 36.5% preferred usual care) |
Reported associations between elements for logic model: Teledermatology resulted in more timely referral |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Whiting 2011153 Country: UK Study design: Observational (specific design unclear) Data collection method: NR Aim: To develop a Manchester-wide referral gateway for triage from general practice to specialist care Detail of participants (number, any reported demographics): Pilot with four practices for 5 months, then rolled out city-wide |
Intervention: Referrals were electronically screen at three stages using a single standard referral letter template: Stage 1 – GP referrals checked for completeness (NHS number, date of birth, etc.), and checked against local non-commissioned policy Stage 2 – If data are missing, or procedure is not commissioned an electronic advice note is sent back to the GP practice Stage 3 – Clinical triage. Three course of action: referral continues, referral is diverted to an alternative service or advice and guidance from Map of Medicine, NICE or the local commissioner can be sent back to the GP to encourage more work-up or increase management in primary care. This is done within 2 working days Number of hours: NA Delivered by who? The clinical triage work was subcontracted to the out-of-hours provider Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): From GP to eight specialties – general surgery, ophthalmology, cardiology, ENT, trauma/orthopaedics, gynaecology, urology dermatology |
Outcome measures: Use of outpatient services GP feedback |
Main results: 1.2% reduction in outpatient activity (compared with a 3.8% growth predicted before the intervention). No further data reported (PULSE article) Positive feedback from GPs |
Reported associations between elements for logic model: Referral gateway to manage triage from primary care to 8 specialties resulted in a 1.2% reduction in outpatient activity (compared with a 3.8% growth predicted before the intervention) |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Wilson 200665 Country: UK Study design: cRCT Data collection method: Questionnaires Aim: To evaluate the effectiveness of the intervention in improving GP confidence in managing patients concerned about genetic risk of breast cancer Detail of participants (number, any reported demographics): General practices in the Grampian region of Scotland Linked paper Wilson 200565 not extracted separately |
Intervention: Components of the intervention software – a list of the key patient information needed in order to use the guidelines. A risk assessment module, presented as a set of short checklists, in which the Scottish referral guidelines for breast, ovarian and colorectal cancer were embedded. Background information on cancer genetics, and explanation of the evidence underlying the guidelines, prepared by local geneticists. Printer-ready, locally customised patient information leaflets. Selected web links for professionals and patients. A contact e-mail link with the Cancer Genetics Service, with a guaranteed response time. Automated production of a draft referral letter using the regionally recommended template. All partners in intervention practices were invited to interactive workshops on cancer genetics designed to complement the software Number of hours: NR Delivered by who? Workshop delivered by specialist genetics staff. Intervention was disseminated by mailing it to all intervention practices; installing it opportunistically with routine practice IT upgrades; special visit by a research associate where requested; and distribution at the educational sessions Control: Control practices received a baseline intervention only (i.e. the Scottish referral guidelines which were mailed to all GPs) Length of follow-up: 31 May 1998 to 31 October 2000 (pre intervention) and 1 July 2001 to 31 May 2002 (post intervention) Response and/or attrition rate: 20 (35%) of the eligible GP practices. 122/140 (87%) of eligible patients (88 intervention, 34 control) Context (from what/who to what/who): GP referral for familial breast cancer |
Outcome measures: GPs’ self-reported confidence in four activities related to genetics; rates of referral of patients at elevated genetic risk; and referred patients’ understanding of cancer risk factors |
Main results: No statistically significant differences were observed between intervention and control arms in the primary or secondary outcomes. A possible effect of the intervention on the proportion of referred patients who were at elevated risk could not be discounted. Only a small proportion of intervention GPs attended the educational session, were aware or the software, or made use of it in practice In the pre-intervention period, intervention GPs were less likely than control GPs to refer patients who were eventually assessed as having elevated genetic risk, with the opposite trend observed in the post-intervention period, although these results did not reach statistical significance Pre intervention [intervention 40/88 (45), control 22/34 (65), risk ratio 0.70 (95% CI 0.50 to 0.99)] Post intervention [intervention 49/85 (58), control 14/29 (48), risk ratio 1.18 (95% CI 0.88 to 1.37)] |
Reported associations between elements for logic model: No convincing evidence of the effectiveness of the intervention was found, probably reflecting barriers to its use in routine practice |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Wolters 200536 Country: the Netherlands Study design: RCT Data collection method: Routine data Aim: To determine the effect of a distance-learning programme on general practice management of men with LUTS Detail of participants (number, any reported demographics): 142 GPs showed interest and were allocated to one of the two groups: 70 to the intervention group and 72 to the control group |
Intervention: GPs were randomised to a distance-learning programme accompanied with educational materials or to a control group only receiving mailed clinical guidelines on LUTS. Clinical management was considered as outcome The distance-learning programme comprised (1) a package for individual learning developed by the Dutch College of General Practitioners, (2) consultation supporting materials: a voiding diary, the International Prostate Symptom Score and Bother score, (3) the guideline summarised into two decision trees (one on clinical management of LUTS and one on PSA testing) and a brief explanation and (4) two information leaflets for patients (on PSA testing and on treatment for LUTS) Control: The control group of GPs received the existing national guidelines on LUTS Length of follow-up: After fulfilling the intervention (April 2001) GPs were instructed to recruit patients until June 2002 Response and/or attrition rate: The educational programme was completed by 89 of the GPs; 63 GPs (31 GPs in the intervention group and 32 GPs in the control) Context (from what/who to what/who): GP referral to urology |
Outcome measures: Number of PSA requests, medication prescribed and the referral rate to a urologist |
Main results: Sixty-three GPs registered care management of 187 patients older than 50 years attending the practice because of LUTS The intervention group showed a lower referral rate to a urologist (OR 0.08, 95% CI 0.02 to 0.40), but no effect on PSA testing or prescription of medication PSA testing tended to be requested more frequently by intervention group GPs. Secondary analysis showed patients in the intervention group received more educational materials (OR 75.6, 95% CI 13.60 to 419.90) |
Reported associations between elements for logic model: The educational programme had impact on clinical management without changing PSA testing |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Wong 200081 Country: Hong Kong Study design: Controlled before-and-after Data collection method: Routine data and questionnaire Aim: To evaluate open-access endoscopy for dyspepsia Detail of participants (number, any reported demographics): n = 367 patients with dyspepsia in intervention group and 967 control (these numbers change across the paper). Mean age of patients 74 years |
Intervention: Family physicians able to arrange upper endoscopy directly with the endoscopy unit in addition to conventional referrals. Extra session each week allocated to open-access requests to ensure waiting time not affected Number of hours: NA Delivered by who? Endoscopy clinic Control: Usual practice Length of follow-up: 2 years 10 months Response and/or attrition rate: NA Context (from what/who to what/who): GP to endoscopy clinic |
Outcome measures: Waiting time request to procedure Appropriateness of referral |
Main results: Waiting time for intervention group was mean 6 weeks, for control mean 17.5 weeks to consultation and then another 4.5 weeks to procedure = 22 weeks. During this waiting time only antiacids prescribed. Abnormal findings in 19% of patients from intervention group and 22% from consultant referral – difference not significant. Only two patients (0.2%) of those referred via open access were considered inappropriate. No significant difference intervention vs. control in peptic ulcer and cancer detection rate. Significantly more non-ulcer non-cancer abnormal findings in referrals via consultant (0.5% vs. 5%, p < 0.005). 13 physicians responded to questionnaire about service, all said it was useful and were willing to use service again, and 11 of 13 said patients satisfied. 107 patients (88%) had telephone interview; all satisfied regarding diagnosis; 76% no further consultation for at least 4 weeks after endoscopy, 12% attended GP; 12% were referred to specialist or were admitted to hospital |
Reported associations between elements for logic model: Open access reduces waiting time and subsequent referral to specialists Referral systems similar in terms of diagnostic yield |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Wright 200656 Country: UK Study design: Controlled before-and-after Data collection method: Referral audit Aim: Before-and-after evaluation of a quality improvement programme for TIA referral with controlled comparison in three PCTs Detail of participants (number, any reported demographics): Three Bradford PCTs and their populations |
Intervention: Four local consensus group meetings or relevant stakeholders (including service users and carers) to adapt national guidelines to local context and identify barriers and incentives for changing practice Guideline reminders for clinicians included laminated posters, desktop coasters and electronic referral templates Disseminating guidelines: Education meetings in each PCT Education outreach visits to 19 practices Postal dissemination to other practices not requesting a visit Control: no intervention Length of follow-up: 50 months (28 months before intervention) Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for TIA for stroke prevention (cardiology) |
Outcome measures: Referral rates Appropriate treatment |
Main results: Significant difference in the change in referral to the rapid access clinic between the practices that were part of the quality improvement programme and those that were not. There was a 41% increase in referrals from trained practices compared with control practices (RR 1.41, p = 0.018) Adherence to best practice standards was significantly higher in practices that had received the training programme than in the controls |
Reported associations between elements for logic model: Tailored guidelines can be effective |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Wylie 200118 Country: UK Study design: Audit Data collection method: Questionnaire Aim: To compare the prescribing pattern and attitude of GPs in response to a clinic returning a patient referred for erectile dysfunction to the referrer by two different methods Detail of participants (number, any reported demographics): All 796 referrals for ED on the waiting list were reviewed |
Intervention: Referrals on a waiting list for an assessment of erectile dysfunction were reviewed and a subgroup of patients with criteria enabling them to be eligible for a prescription under the NHS were identified. The GP was informed either in writing or by telephone that the clinic had written to the patient, suggesting he make direct contact with his GP. A follow-up questionnaire was sent to each GP 1 month after the initial letter to the patient and contact with the GP Control: NA Length of follow-up: NA Response and/or attrition rate: 91 questionnaires were sent; 66 (73%) completed or partially completed questionnaires were returned Context (from what/who to what/who): GP referral for erectile dysfunction |
Outcome measures: Clinical outcomes Attitude of GP Prescribing behaviour |
Main results: Of 91 questionnaires sent to GPs, 66 (73%) were completed; an additional five GPs corresponded by letter rather than completing the questionnaire. The long waiting time for assessment had led to 35% of patients having already tried sildenal, and by the time the questionnaire was completed, 57% of patients had tried sildenal. Ten times as many referrers indicated that they were happy to initiate a prescription for sildenal than not to do so, for those men eligible for a NHS prescription More GPs who had received a letter returned the completed questionnaire (80%) than those who had received a courtesy telephone call (64%). There were no differences between the groups of GPs in their attitude to our contact with their patient and no difference in prescribing pattern |
Reported associations between elements for logic model: The provision of guidelines and advice to GPs either by telephone or by letter is acceptable practice in reducing long waiting-list times for ED. Safe, simple and effective treatments are available for GPs to use under NHS guidelines |
Non-intervention papers
| Abel and Thompson 201120 Country: New Zealand Study design: Qualitative Data collection method: Interviews Aim: To examine how guidelines are used Detail of participants (number, any reported demographics): Interviews with 15 GPs and 11 specialists from three largest cities in New Zealand. Selected to reflect a diversity of socioeconomic status patient lists |
Method: Qualitative study. In-depth, semistructured interviews with 15 GPs and 11 specialists. Participants were asked for their thoughts on a population-based screening programme, the surveillance guidelines for colorectal cancer, screening, advice to patients at different levels of risk for colorectal cancer, and referrals for colonoscopic investigation. Thematic analysis of interview data Control: NA Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Use of risk assessment guidelines for colorectal cancer referral |
Outcome measures: Views |
Main results: Need for flexibility in use of guidelines due to messiness and unpredictability of individual patients Guidelines in their full form little used, sheer size of documents made comprehensive knowledge of all of them difficult for GPs Specialists perceived the guidelines as too complex for GPs to understand GPs argued that the guidelines ‘don’t quite fit’ when they have a patient in front of them. Stories of patients and clinical expertise and judgement incorporated in practitioner decision-making to develop a body of evidence for each individual patient Rigid adherence to guidelines was inappropriate when working for the benefit of the patient. Decision-making more complex than simple objective linear interpretation and implementation of evidence GPs not wanting to miss something by not referring. Emotional or subjective concerns for the patient more relevant than subjective measures of risk GPs saw the task of getting patients seen as their duty to do the best for the patient |
Reported associations between elements for logic model: Patient-related factors and referral GP role and referral Lack of adherence to guidelines Risk of missing something |
| Ache 2011277 Country: USA Study design: Cross-sectional Data collection method: Survey Aim: To compare attitudes towards hospice referral by doctor ethnicity Detail of participants (number, any reported demographics): 167 white American PCPs and 46 African American PCPs |
Method: Survey had 17 questions regarding attitudes to hospice referral Number of hours: NA Control: None Length of follow up: NA Response and/or attrition rate: 51% response rate Context (from what/who to what/who): PCP to hospice |
Outcome measures: Demographics and attitudes |
Main results: Racial differences in attitude to referral for 5 of 17 questions |
Reported associations between elements for logic model: Individual demographics of physician influences referral |
| Ahluwalia 2009214 Country: USA Study design: Cross-sectional Data collection method: Survey of PCPs Aim: To examine physician factors associated with referral to outpatient palliative care Detail of participants (number, any reported demographics): n = 145 PCPs and internal medicine physicians at two hospital-based not-for-profit health maintenance organisations. 43% 30–39 years; 58% female; 45% experience with PC; 69% referred patients in previous year |
Methods: Survey based on literature on referral patterns, included demographics, belief about ability to deliver palliative care, perceived benefits and importance of practice autonomy. Beliefs measured on 20-item scale developed for study Number of hours: NA Control: None Length of follow-up: NA Response and/or attrition rate: 170 of 345 staff asked to complete of these 85% completed Context (from what/who to what/who): Physicians to palliative care |
Outcome measures: Beliefs in benefits, beliefs regarding physician autonomy, beliefs regarding ability to provide palliative care |
Main results: 70% of respondents reported referring patients to the palliative care programme in the prior year One factor associated with referral: Working at the organisation for more than 10 years compared with less than 10 years (OR 6.29, 95% CI 1.38 to 28.6) Having personal experience with palliative care was not statistically significant (OR 2.13, 95% CI 0.95 to 4.976). None of the beliefs associated with referral |
Reported associations between elements for logic model: Working at an organisation for a longer length of time associated with referral |
| Albertson 2000270 Country: USA Study design: Cross-sectional Data collection method: Patient survey Aim: To explore patient desire for referral Detail of participants (number, any reported demographics): 12 PCPs for 822 patient visits. Patients seen at a University Medical Group Practice. Included state employees, health-care workers, physicians, nurses, university employees and their family members |
Methods: Patients given pre- and post-visit questionnaire, providers surveyed after patient visit Number of hours: NA Control: None Length of follow-up: NA Response and/or attrition rate: Not reported Context (from what/who to what/who): PCP referral. Referral for emergent or urgent care, pregnancy, cancer, mental health, acquired immunodeficiency syndrome (AIDS), annual eye examination, or continuing therapy were excluded |
Outcome measures: Perceptions of referral Rating of health, recognition of referral desire |
Main results: Provider recognition of a patient referral desire varied significantly within the practice. Between 68% recognition and 24% No relation between PCP age, gender, years in practice, clinic workload, and this variability in recognition of patient referral desire Providers more likely to recognise a referral desire in patients who were healthcare workers or family members, those with more than one referral desire and patients who had a definite desire rather than a possible desire for referral No association with patient self-rating of health, worry regarding referral desire, or self-reported lower functional status related to their referral desire, duration of patient–provider relationship and provider recognition of a referral desire Patients were significantly more likely to have initiated the referral discussion when they had seen the PCP previously and had more than one referral desire. There was a trend for patient initiation of the referral discussion when the patient had known the PCP for more than a year (p = 0.08) The frequency with which a referral was made was comparable with patient initiated and PCP initiated referral discussions PCPs felt around 14% of patient referral desires were not indicated |
Reported associations between elements for logic model: Variability amongst PCPs regarding whether or not they recognise a referral desire in a patient Continuity of care and familiarity with their PCP is associated with patients initiating a referral discussion with their PCP |
| Alexander 2008308 Country: Australia Study design: Cross-sectional Data collection method: Survey of GPs Aim: To identify needs of GPs regarding referral and treatment of patients with mental health disorders Detail of participants (number, any reported demographics): 38 GPs rural New South Wales; average age 47 years; practised 18 years |
Methods: Postal survey Number of hours: NA Control: None Length of follow-up: NA Response and/or attrition rate: 26% Context (from what/who to what/who): GP to mental health services |
Outcome measures: Rating scale of self-confidence Reported constraints on referral |
Main results: 76% reported insufficient skills to provide best patient care as main reason for referring to specialist service Appropriate specialist services being neither available nor accessible was the main factor reported as preventing them referring patients Patient reluctance to accept referral also reported Lack of service targeting children and adolescents a factor preventing referral |
Reported associations between elements for logic model: GP knowledge and referral |
| Allareddy 2007232 Country: USA Study design: Qualitative Data collection method: Focus groups Aim: To understand relationships between GPs and chiropractors Detail of participants (number, any reported demographics): Carried out two focus groups, does not provide number of participants |
Methods: Qualitative analysis of focus groups Number of hours: NA Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to chiropractice |
Outcome measures: Views |
Main results: Key to establishing positive relationships is communication. Good elements of communication include feedback, promptness and constructive criticism Telephone call or face-to-face contact was important in influencing positive relationships PCPs expressed a lack of understanding of chiropractic care and did not have any relationship with practitioners. PCPs felt that geographical constraints prevented relationships being established Concern from PCPs regarding level of training of chiropractors, whether or not licenced |
Reported associations between elements for logic model: Importance of the referral relationship, communication and confidence in specialist service |
| Angstman 2009184 Country: USA Study design: Survey Data collection method: e-mail Aim: To assess provider satisfaction with viral specialist consultations via an e-mail survey Detail of participants (number, any reported demographics): 56 family medicine providers |
Methods: E-mail survey Length of follow-up: none Response and/or attrition rate: Context (from what/who to what/who): Viral specialist consultations |
Outcome measures: Provider views |
Main results: Approximately 30% of the 56 family medicine providers had not tried the viral consultation system after it had been in place for over a year, or said they often forgot that viral consultations were an option; most of the providers surveyed (73%) felt that viral consultations provided good medical care A majority felt that viral consultations are a cost-effective and efficient tool for our department (65%). Most specialists (81%) reported that viral consultations were an efficient use of their time and 67% said that viral consultations were less disruptive than contacts by telephone or pager Only 5% felt that viral consultations do not provide good medical care Specialists providing viral consultations tended to be supportive |
Reported associations between elements for logic model: Illustrates the potential for increased efficiency from viral consultation |
| Anthony 2010208 Country: USA Study design: Qualitative and cross-sectional Data collection method: Interviews followed by a survey Aim: To investigate factors influencing decisions to refer Detail of participants (number, any reported demographics): 40 physicians, 15 general internists, 15 PCPs, 10 adult/family nurse practitioners, 27 female, 13 male, 25 Caucasian, 39 urban area, years in practice 1–35 years |
Methods: Mixed methods – cross-sectional survey and qualitative interviews Number of hours: NA Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): PCPs to depression care |
Outcome measures: Themes Depression Care Questionnaire Provider Belief Survey |
Main results: Decisions to refer centred around five main themes Clinicians’ perceived severity of symptom Clinicians’ comfort in treating depression – identified by 80% as a very important factor Clinicians’ perceived complexity of the diagnosis (comorbid diagnosis or a lack of patient response to medication) Patient preference and resources – willingness to see a mental health specialist, ability to pay Practice environment (time constraints, treatment resources, access to specialists) |
Reported associations between elements for viral consultation: Patient factors influencing referral Clinician confidence |
| Ashworth 2002303 Country: UK Study design: Cross-sectional Data collection method: Analysis of referral records Aim: To explore whether or not having an on-site mental health worker impacted on referral Detail of participants (number, any reported demographics): 29 practices in inner-city London. 79 GPs. 622 referrals made. 72% had on-site worker with mean of 1.6 hours per week per 1000 patients |
Methods: All referrals over period of 1 year and case notes of those referred searched to exclude those with psychotic illness Number of hours: NA Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to psychiatry for non-psychotic patients |
Outcome measures: Referral rate Allocation of mental health workers |
Main results: Rates varied almost 10-fold between different practices No significant correlation between psychiatric referral rates for non-psychotic illness and the allocation of mental health workers to each practice (Spearman’s p = –0.22, p = 0.25) If practices in the highest quartile or quintile of referring rates were analysed there was a possible relationship between referral and worker. These practices with higher referral rates had lower allocations of mental health workers (Mann–Whitney U-test = 30.0/38.0, p = 0.03 and 0.04) No other factors associated with referral rate such as being a single-handed practice, non-fundholding, or inability to meet health screening targets, percentage of patients unemployed or with long-term sickness |
Reported associations between elements for logic model: Presence of mental health worker and mental health referral |
| Baker 2006192 Country: UK Study design: Qualitative Data collection method: Interviews with GPs Aim: To explore variation rates in GP referral rates Detail of participants (number, any reported demographics): 29 GPs with high and low referral rates in north-east of England. 24 male, 26 urban areas, 17 more than 20 years qualified |
Method: Qualitative, semistructured interviews with 29 GPs. Thematic analysis of interview data Number of hours: NA Control: None Length of follow-up: NA Response and/or attrition rate: 53% response rate Context (from what/who to what/who): GP to lumbar spine radiography |
Outcome measures: Themes High vs. low use of radiography from radiology records |
Main results: Content of the available guidelines was perceived as appropriate and limitations of radiography acknowledged by both groups. Both high and low users were aware of the guidelines. Radiography seen by some as being used to allay fears of a serious illness such as cancer. GPs also influenced by patient social and economic circumstances in regard to whether refer urgently or watchful waiting approach High-use GPs tended to view use of radiography as a legitimate means to reassure patients with benefit outweighing risk more than lower users. Lower users described more complex impact of findings GPs in higher user group tended to be more pessimistic regarding options available and the prognosis for chronic back pain sufferers Higher emphasis on fragility of doctor–patient relationship in higher user group and use of radiography to preserve this relationship Concern regarding exposure to radiation expressed more in lower referral group. Those who referred far more were less concerned. Some report of cost concerns radiography in lower user group 9 of 14 high-user GPs perceived their use as low or were unsure of their use level. Lower use group judged their use more accurately |
Reported associations between elements for logic model: Individual beliefs of GP and referral Self-perception of referral rates |
| Balduf 2008274 Country: USA Study design: Cross-sectional Data collection method: Postal survey Aim: To assess the attitudes, knowledge, and bariatric referral practices among family and internal medicine physicians in North Carolina Detail of participants (number, any reported demographics): 47% of 611 randomly chosen PCPs returned a self-completed questionnaire |
Method: Survey of PCPs Control: NA Length of follow-up: NA Response and/or attrition rate: 47% Context (from what/who to what/who): GP to specialist |
Outcome measures: The effect of demographics, PCP attitudes, and PCP knowledge on referral practices was evaluated |
Main results: Over 85% (221) of PCPs have treated operated bariatric patients within the last year and 76% (203) have referred patients for surgical evaluation Thirty-five per cent of practitioners felt unprepared to provide good-quality long-term medical care to operated patients, and just 45% felt competent to address medical complications of bariatric surgery Compared with non-referring PCPs, referring physicians provided medical care to a greater number of severely obese (mean 9.9 vs. 7.5, p < 0.001) and post-operative (mean 4.6 and 2.3, p < 0.001) bariatric patients. Referring providers were younger (46 vs. 49, p = 0.02), had higher body mass index (25.3 vs. 23.5, p < 0.001), were more familiar with NIH guidelines (14.7% vs. 3.0%, p = 0.02), and had completed more bariatric continuing medical education (49.8% vs. 34.9%, p = 0.03). In addition, they reported better resources (71.4% vs. 35.4%, p < 0.001) and competency to provide good-quality long-term care to post-operative bariatric patients (54.2% vs. 15.4%, p < 0.001) than non-referring peers |
Reported associations between elements for logic model: Unclear |
| Barnett 2011222 Country: USA Study design: Cross-sectional Data collection method: Web-based survey Aim: To examine reasons for choosing colleagues to refer to Detail of participants (number, any reported demographics): n = 386 physicians, 64% male, 36% PCP physicians |
Method: Participants presented with roster of other physicians names who had been identified by the participant as being in their professional network. Respondents reported if they referred to these physicians and if so asked for up to two reasons why Number of hours: NA Control: None Length of follow-up: NA Response and/or attrition rate: 63% Context (from what/who to what/who): PCP and specialist physicians to specialist services |
Outcome measures: Referral destination physician and factors underpinning choice |
Main results: PCPs initiated referrals to 66% of their professional network colleagues PCPs likely to cite reasons relating to patient access or physician communication when comparing specialists. PCPs – timely availability of appointments cited as factor in 15.7% of their referral relationships. Shares my medical record system 17.9% Note: Focus of paper findings is comparison between PCPs and specialists |
Reported associations between elements for logic model: PCP referral decisions influenced by between-physician communication and patient access |
| Bederman 2010196 Country: Canada Study design: Delphi consensus Data collection method: An expert panel of primary and specialist physicians, using a Delphi process, came to a consensus on referral recommendation from clinical practice guidelines based on a series of clinical vignettes Aim: To compare the actual referral practices for patients with degenerative disease of the lumbar spine with recommendations for surgical referral based on clinical practice guidelines and family physician opinions Detail of participants (number, any reported demographics): Panel n = 10 |
Method: An expert panel of primary and specialist physicians, using a Delphi process, came to a consensus on referral recommendations from clinical practice guidelines based on a series of clinical vignettes. The vignettes were also presented to practising family physicians in Ontario, Canada, to determine their preferences for (or likelihood of) referral The panel was a 10-member multispecialty expert panel Practising family physicians were randomly sampled, stratified by county, and their patients were sampled purposefully by the family physician Context (from what/who to what/who): Primary care referral practices for patients with degenerative disease of the lumbar spine |
Outcome measures: Respondents, both panellists and family physicians, were asked to rate the appropriateness of surgical referral for a series of clinical vignettes. Patients reported their clinical symptoms and whether or not they had been referred to a surgeon. Using random-effects probit regression, redictions were compared with actual referral. Receiver operating characteristic curves were constructed and area under the curve was measured |
Main results: Consensus of the panel on recommendations for referral was achieved after two iterations (Cronbach 0.96). Based on responses from 107 patients and 61 family physicians, we found poor concordance of both predicted family physician preferences (area under the curve 0.57) and clinical practice guidelines recommendations (area under the curve 0.64) with actual referral |
Reported associations between elements for logic model: Referral practices are poorly predicted by clinical practice guideline recommendations and individual family physician opinions |
| Beel 2008226 Country: Australia Study design: Qualitative Data collection method: Semistructured interviews Aim: Investigating GPs’ views on psychologists and the factors that determine patient referral Detail of participants (number, any reported demographics): 12 Western Australian GPs from the Perth metropolitan area (eight men and four women, aged 30–79 years) |
Method: GPs in Australia typically do not refer clients to mental health-care providers. Few systematic and referenced studies of the reasons for this lack of collaboration have been identified. The current research addresses this gap by investigating GPs’ views on psychologists and the factors that determine patient referral Control: NA Length of follow-up: NA Response and/or attrition rate: Context (from what/who to what/who): GP to psychology |
Outcome measures: Practitioner views |
Main results: The main findings were that GPs were satisfied with psychologists’ capabilities but not their professional communications and were overwhelmingly dissatisfied with a mental health system that does not readily provide funding for patients in need of mental health-care services The findings suggest that GPs would like health-care policies to be more inclusive of psychological services and are interested in collaborating with psychologists for better patient outcomes |
Reported associations between elements for logic model: Influence of good communication from specialist back to referrer in referrer satisfaction |
| Bekkelund 2001259 Country: Norway Study design: Cross-sectional Data collection method: Survey Aim: To explore satisfaction with self-referral Detail of participants (number, any reported demographics): 1052 patients examined by a neurologist for headache in previous 2 years |
Method: Cross-sectional survey Control: NA Length of follow-up: NA Response and/or attrition rate: 75% Context (from what/who to what/who): Doctor to neurologist |
Outcome measures: Views |
Main results: 24% initiated the referral themselves 52% of those were dissatisfied with the specialist consultation 42% of those referred by the doctor were dissatisfied with the consultation |
Reported associations logic model: Less patient satisfaction with self-referral than with doctor referral |
| Belgamwar 2011197 Country: UK Study design: Retrospective cross-sectional Data collection method: Routine data analysis Aim: This study examines the current practice in an urban area with particular reference to primary care management of anxiety and depression. It specifically looks at the stepped care model recommended in the NICE guidelines and identifies the improvements that should be made to patients’ care in the future Detail of participants (number, any reported demographics): 204 referrals from seven GP practices. Of these, 64 GP referrals (31%) mentioned primary problems such as anxiety and depression |
Method: Examined all GP referrals to our community general adult mental health services (CMHT), covering a population of 38 000, during the period of June 2008-March 2009. Referrals were assessed for adherence to NICE guidelines, specifically monitoring standards against the stepped care model in patients suffering from anxiety and depression. They examined what care had been given to these patients in primary care prior to their referrals to secondary care mental health services The stepped care model provides a framework for appropriate patient care at different stages of their illness. If the patient does not benefit from an intervention, then one from the next step in the model should be offered (NICE) Context (from what/who to what/who): GP referral for anxiety/depression |
Outcome measures: Appropriateness of referral (stepped care model) |
Main results: Exactly half of all referrals (32/64) did not follow the stepped care model. Of these, the majority of patients (28/32, 87.5%) had not received any psychological therapy prior to referral to secondary services (or at least this was not mentioned in GP letters). Only one GP had commented on the use of self-help strategies. On the other hand, only a minority of patients (4/32, 12.5%) had not been tried on medication before being referred to secondary care mental health services |
Reported associations between elements for logic model: GPs may not be aware of NICE stepped care recommendations. Education/awareness-raising is needed |
| Berendsen 2007237 Country: the Netherlands Study design: Qualitative Data collection method: Semistructured interviews Aim: What motivates GPs to initiate and continue participating with medical specialists in new collaborative care models Detail of participants (number, any reported demographics): 21 Dutch GPs |
Method: A qualitative study design was used. Starting in 2003 and finishing in 2005. Conducted semistructured interviews with a purposive sample of 21 Dutch GPs. The sampling criteria were age, gender, type of practice and practice site. The interviews were recorded, fully transcribed, and analysed by two researchers working independently. The resulting motivational factors and preferences were grouped into categories Control: NA Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to specialists |
Outcome measures: Factors which encourage participation |
Main results: Developing personal relationships’ and ‘gaining mutual respect’ appeared to dominate when the motivational factors were considered. Besides developing personal relationships with specialists, the GPs were also interested in familiarising specialists with the competencies attached to the profession of family medicine Additionally, they were eager to increase their medical knowledge to the benefit of their patients. The GPs stated a variety of preferences with respect to the design of new models of collaboration |
Reported associations between elements for logic model: Developing personal relationships with specialists appeared to be one of the dominant motives for increased collaboration |
| Berendsen 2010262 Country: the Netherlands Study design: Cross-sectional Data collection method: Survey Aim: To explore patient preferences for transition from primary to secondary care Detail of participants (number, any reported demographics): Patients – 513 survey 1, 1404 survey 2. 61% female, 59% 35–64 years old Patients over 18 years and who had been referred to a medical specialist in the last 2 years. Patient groups defined as stressed or stable |
Methods: Two questionnaires sent, one on perceived importance of care and the other on experiences of care Number of hours: NA Control: None Length of follow-up: NA Response and/or attrition rate: 69% response rate for questionnaire 1 (Importance), 65% response rate for questionnaire 2 (Experience) Context (from what/who to what/who): Referrals to medical specialist excluding paediatrics and psychiatry |
Outcome measures: Views and perceptions |
Main results: 81% of patients thought it was important that the GP gave them advice on which hospital or specialist to go to when they referred the patient. For 67% of patients the GP gave this advice 25% of patients found it important that the GP decided for them which hospital or specialist to attend. In 33% of cases the GP had done that Older people more than younger people (65 + 38%, < 65 20%, p = 0.001) and lower educated (42%) more often than higher educated people (19%, p < 0.0005) wanted the GP to make a decision. In practice the GP tended to make the decision more for lower educated people more often No difference men and women regarding role of GP 70% of patients thought it important they chose a hospital themselves, 56% a specialist. 61% actually did choose the hospital, 30% a specialist 91% wanted to choose a hospital based on proximity. Patients aged 75 and older more often wanted to choose a hospital nearby. No difference in choice of hospital nearby based on educational level, chronic illness A small number of patients wanted to choose a hospital or specialist based on information from a newspaper, the internet or a health insurance company. Patients with a curable condition, aged between 25 and 65 years, who are highly educated and have a high score on the stable scale more often wish to use information from newspapers or the internet to make a decision |
Reported associations between elements for logic model: Patient preference for GP to make referral choice Patient demographics of those who use information to make choice Role of GP as decision-maker Demographics of patients who wished to be involved in decision Proximity of service important Own/family experiences important |
| Bertakis 2001264 Country: USA Study design: Cross-sectional Data collection method: Data analysis Aim: Examines factors predictive of patient referrals to specialists by primary care residents Detail of participants (number, any reported demographics): n = 509, 38% were men. Ethnic backgrounds of the patients included Caucasian (63%), African American (22%), Hispanic (8%), Asian (4%) and Native American (3%). Patients had a mean age of 41.3 years and a median of 12 years of education |
Method: New adult patients (n = 509) were randomly assigned to primary care residents at a university medical centre. Patient referrals to specialists were monitored for 1 year of care Self-reported patient health status, sociodemographic information, number of primary care visits, and physician practice style behaviours were incorporated into statistical analyses predicting specialty referrals Control: NA Length of follow-up: 1 year Response and/or attrition rate: 821 patients out of 956 (85%) agreed to participate, 312 (38%) were excluded for various reasons. 53% of those invited made it through to the final data set Context (from what/who to what/who): GP to specialist |
Outcome measures: Self-reported patient health status, sociodemographic information, number of primary care visits, and physician practice style behaviours |
Main results: Patients who were referred to specialty care were significantly older, had poorer physical health and saw their primary care physicians more often than patients who were not referred. Patients were most frequently referred to surgical specialty clinics The majority of specialty referrals were made by the patients’ primary care physicians. Of the 1105 specialty clinic visits, 752 (68.1%) were at the request of the patients’ primary providers, 157 (14.2%) were initiated by other providers in the same primary care clinic, and another 196 (17.7%) were instigated by providers in non-primary care clinics After controlling for physical health status, gender and age, more frequent visits to a primary care resident physician who had a technically oriented style of care was associated with a greater total number of specialty clinic referrals There were more medical specialty referrals for patients having poorer physical health (p = 0.01844), advanced age (p = 0.0002), and a larger number of primary care visits (p = 0.0001) More primary care visits (p = 0.0001) and a primary care physician practice style emphasising the technical aspects of medical care (p = 0.0410) all significantly influence the total number of specialty clinic referrals |
Reported associations between elements for logic model: Unclear |
| Blundell 2010253 Country: UK Study design: Qualitative Data collection method: Semistructured interviews Aim: To explore interpretations of ‘appropriate’ and ‘inappropriate’ elective referral from primary to secondary surgical care among senior clinical and nonclinical managers in five purposively sampled PCTs and their main associated acute hospitals in the English NHS Detail of participants (number, any reported demographics): n = 22 |
Method: Semistructured face-to-face interviews were undertaken with senior managerial staff from clinical and non-clinical backgrounds. Interviews were tape-recorded, transcribed and analysed according to the Framework approach developed at the National Centre for Social Research using N6 (NUD*IST6) qualitative data analysis software Response and/or attrition rate: 22 of 23 participated (between three and five respondents per PCT and associated acute hospital) Context (from what/who to what/who): Elective referral from primary to secondary surgical care |
Outcome measures: Appropriateness of referral |
Main results: Three attributes relating to appropriateness of referral were identified – necessity (whether a patient with given characteristics was believed suitable for referral); destination or level (where or to whom a patient should be referred); and quality (or process) (how a referral was carried out, including e.g. investigations undertaken before referral, information contained in the referral and extent of patient) Involvement in the referral decision. Attributes were hierarchical. ‘Necessity’ was viewed as the most fundamental attribute, followed by ‘destination’ and, finally, ‘quality’. In general, but not always, all three attributes were perceived as necessary for a referral to be defined as appropriate |
Reported associations between elements for logic model: Three hierarchical attributes (necessity, appropriateness of destination and quality of referral process) contributed to the overall concept of appropriateness of referral from primary to secondary surgical care |
| Blundell 2011189 Country: UK Study design: Survey Data collection method: Online, telephone, fax or post Aim: To investigate GPs’ attitudes to guidelines for elective surgical referral in England. To understand their use of guidelines, and attitudes to shared decision-making in the referral decision Detail of participants (number, any reported demographics): Stratified random sample 30% (n = 310) drawn from GP lists of 10 English health districts |
Response and/or attrition rate: Responses were representative of GPs in England, but (despite up to six contacts per non-responder) the overall response rate was 41.6% (n = 129; with the range across PCTs of 25–61%) Context (from what/who to what/who): Guidelines for referral for elective surgery |
Outcome measures: Practitioner views Support for use of guidelines |
Main results: Most responding GPs indicated support for referral guidelines but 18% reported that they had never used them. Less than 3% reported use for most or all referral decisions. The odds of using guidelines decreased with increasing age, with a ten year increase in age associated with halving odds of use (OR 0.53, 95% CI 0.29 to 0.90) Over 50% of GPs wanted good access to electronic guidelines with expert information and advice on guideline availability. Almost all (> 89%) GPs agreed with sharing referral decisions with patients Female doctors (OR 5.2, 95% CI 1.02 to 26.3) were more likely to agree with this than male GPs as were those working in larger compared with small or single-handed practices (OR 5.3, 95% CI 1.4 to 19.9) |
Reported associations between elements for logic model: This group of responding GPs was supportive of guidelines but used them in different ways. Referral guidelines should have an educational component for background reading; include key messages for internalisation and application; and incorporate mechanisms to facilitate accessibility and appropriate shared decision-making with patients |
| Bolanos-Carmona et al. 2002276 Country: Spain Study design: Cross-sectional Data collection method: Examination of patient record data Aim: To explain variability in health service by patients Detail of participants (number, any reported demographics): Data from 52,152 patients and their 38 primary care physicians. Average age of physician 40 years; 66% men |
Methods: Cross-sectional examination of patient record data Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): All specialties |
Outcome measures: Ambulatory care group and physician characteristics |
Main results: Most of variability in referral numbers was due to patient characteristics (97%) while only 3% was related to physicians – ambulatory care group and post that physicians hold in their health centre (director or not). The average number of referrals per patient per year was 0.13 times greater for health centre directors than for physicians No effect of physician age, gender, minutes devoted to patient Paper does not examine specific patient characteristics |
Reported associations between elements for logic model: Lack of association between physician characteristics and referral Predominance of patient characteristics impacting on referral variance |
| Boulware 2006304 Country: USA Study design: Cross-sectional Data collection method: Questionnaire Aim: Do PCPs and specialists agree on diagnostic and referral strategies, and identify similar barriers to caring for patients? Detail of participants (number, any reported demographics): National sample of 304 physicians 126 nephrologists, 89 family physicians, and 89 general internists |
Method: A national study of PCPs and nephrologists in the USA through a questionnaire describing a PCP caring for a patient with progressing chronic kidney disease and questions to assess recognition of kidney dysfunction and approaches to diagnostic evaluation and referral Control: NA Length of follow-up: NA Response and/or attrition rate: Nephrologists (39% response rate, family physicians 28% response rate, and general internists 28% response rate) Context (from what/who to what/who): GP to nephrology |
Outcome measures: Participant and patient characteristics independently associated with kidney disease, recognition and referral |
Main results: PCPs recognised chronic kidney disease less (adjusted percentage 59%; 95% CI 47% to 69%, family physicians; adjusted percentage 78%; 95% CI 67% to 86%, general internists; adjusted percentage 97%; 95% CI 93% to 99%, nephrologists; p < 0.01), differed from nephrologists in their recommendations for diagnostic testing, and recommended referral less (adjusted percentage 76%; 95% CI 65% to 84%, family physicians; adjusted percentage 81%; 95% CI 70% to 89%, general internists; adjusted percentage 99%; 95% CI 95% to 100%, nephrologists; p < 0.01) PCPs differed from nephrologists in their expected intensity of specialists’ involvement in care (16%, family physicians; 20%, general internists; 6%, nephrologists recommending nephrologist input monthly to every 6 months; p < 0.01). Lack of awareness of clinical practice guidelines and lack of clinical and administrative resources were identified as important barriers to care |
Reported associations between elements for logic model: Lack of awareness of clinical practice guidelines and lack of clinical and administrative resources were identified as important barriers to care |
| Bowling 2000291 Country: UK Study design: Cross-sectional Data collection method: Questionnaire Aim: To analyse the patterns and process of care for the referral of outpatients, together with the views of patients, their GPs, and specialists Detail of participants (number, any reported demographics): 842 patients |
Method: A questionnaire survey of outpatients, their hospital specialists, and GPs in randomly sampled district health authorities in the North Thames Region. The measures included items and scales measuring satisfaction and processes Control: NA Length of follow-up: NA Response and/or attrition rate: The response rates to the study were patients, 74% (982); specialists to the general questionnaire, 100% (34), and to the individual (‘new’ patients as defined by the hospital) patients’ questionnaire, 91% (184); GPs to the general questionnaire, 64% (393), and to the individual (all) patients’ questionnaires, 64% (552); and the outpatients’ managers, 61% (27) Context (from what/who to what/who): GP to specialists |
Outcome measures: Practitioner views Patient views |
Main results: Almost all (95%; 800 out of 842 responders) patients thought that their consultation with the specialist was ‘necessary’, and 89% (851 out of 857 responders) rated it as ‘worthwhile’. Consistent with this, of the 540 patients for whom GPs completed the individual patient questionnaire item, 89% (481) were rated by GPs as ‘GP not able to give the care, treatment and investigations received in the hospital’, although 10% (56) felt that they could have done; 1% (3) were uncertain Most of the GPs felt that they could not have given the study patients the care, treatment and investigations they received in hospital, and most of the sampled patients’ attendances were rated by the specialists as ‘appropriate’ However, for just over one-fifth of new patients, the specialists reported that the GP could have done more tests and examinations prior to referring the study patient. Large proportions of GPs in this survey also reported having technical equipment in their practices, as well as direct access to a range of services and hospital-based facilities |
Reported associations between elements for logic model: Differing perceptions GPs and specialists regarding level of pre-referral testing |
| Bowling 2006273 Country: UK Study design: Cross-sectional Data collection method: Semistructured interviews Aim: To investigate referral rates for cardiac interventions by clinical specialty, to document doctors’ reasons for referrals and to explore doctors’ perceptions of the factors that influenced their clinical decisions Detail of participants (number, any reported demographics): 6093 electronic patients with cardiac disease 88 doctors (GPs, care-of-the-elderly specialists, cardiologists) participated in the full study, in seven areas in southern, central and northern England. Complete interview data were analysed for 76 of these |
Method: Semistructured interviews about influences on referral decisions Control: NA Length of follow-up: NA Response and/or attrition rate: 76/88 Context (from what/who to what/who): GP referral to cardiology and care of the elderly |
Outcome measures: Factors that influences referral decisions |
Main results: Not all patients who were eligible for specific investigations or treatment received these. The extent of variations in clinical decisions differed by type of intervention Apart from the general reasons for referrals, doctors raised nine main influences on their actual decision-making. The most commonly reported influence (‘barrier’) was poor access to equipment for intervention, which increased thresholds for investigation and treatment The nine barriers were lack of equipment, or lack of direct or easy access to equipment, for interventions; insufficient time; budgetary constraints; staffing shortages; the effects of hospital policies or consultant policies/clinical guidelines on criteria for investigation and referral; long waiting lists for referrals; the personal characteristics of consultants referred to; doctors’ own characteristics and interests; and patient characteristics |
Reported associations between elements for logic model: Unclear |
| Brien 2008258 Country: UK Study design: Qualitative Data collection method: Interviews Aim: To explore GP experiences of referral Detail of participants (number, any reported demographics): 10 GPs |
Method: Qualitative study. Semistructured interviews with 10 GPs. Thematic analysis of interview data using The Framework Approach Control: NA Length of follow-up: NA Response and/or attrition rate: 30% response rate Context (from what/who to what/who): Referral to a private complementary and alternative medicine clinic |
Outcome measures: Views |
Main results: GP decisions to refer mediated by their experience, evidence and knowledge of complementary and alternative medicine. Positive reports from patients, friends and colleagues, positive past experiences were influential. Those with less knowledge tended to refer via conventional routes first and only to complementary and alternative medicine once conventional routes had been exhausted. Those with less knowledge tended to refer less Good communication with the complementary and alternative medicine practitioner was important Patient experiences of and attitudes to complementary and alternative medicine vital in the process. A match between the doctor’s attitude and treatment preferences and patient views was important |
Reported associations between elements for logic model: Patient preferences and referral GP knowledge and referral Relationship with specialist and referral |
| Bruyninckx 2009209 Country: the Netherlands Study design: Qualitative Data collection method: Interviews Aim: To identify GP reasons for referring or not referring Detail of participants (number, any reported demographics): 21 GPs reporting decision making in relation to two patients each. Two interviews poorly recorded and not used. GPs – male = 18, mean age 52 years, urban 10, solo practice 9 |
Methods: Qualitative study. Semistructured interviews with 21 GPs. Thematic analysis of interview data Control: None Length of follow-up: NA Response and/or attrition rate: 85 invited at a meeting further 320 invited by e-mail (17 recruited via group, four via e-mail) Context (from what/who to what/who): GP referral for chest pain (cardiologist or emergency department) |
Outcome measures: Description of decision-making |
Main results: Referral based on three categories of factors Background knowledge about the patient (e.g. previous history) Differences in patient behaviour – previous to actual consultation Current clinical presentation – having specific pain, frequency and duration, clinical examination, ECG, combination of signs and symptoms – gut feeling GP personal ideas – uncertainty or anxiety, certainty, younger patients referred more readily to emergency department than older, perception by GP of a negative attitude towards GP by staff patient referred to, errors in the past influenced subsequent decisions |
Reported associations between elements for logic model: Patient clinical factors together with GP personal factors influenced decision |
| Burns 2002301 Country: UK Study design: Ecological (retrospective) Data collection method: Case notes Aim: To investigate the association between deprivation and referral to CMHTs, including the contribution of general practice Detail of participants (number, any reported demographics): All community/outpatient referrals to four CMHTs over 1 year |
Method: Retrospective analysis of case notes from all adult psychiatry community and out patient referrals between April 1993 and March 1994 to four CMHTs, and all adult psychiatry in patients during the same year Control: NA Length of follow-up: NA Response and/or attrition rate: Context (from what/who to what/who): GP to CMHC |
Outcome measures: Jarmin index of deprivation Referral |
Main results: Low correlation was found between referral rates for all diagnoses and the Jarmin index of deprivation and between both the index and admission rates for all diagnoses and non-psychosis Referral from GPs varied nearly 40-fold and were not related to deprivation, fund holding status or having a practice manager or practice nurse In addition, the GP factors investigated did not predict referral (not reported further) |
Reported associations between elements for logic model: |
| Calnan et al. 2007207 Country: UK Study design: Qualitative Data collection method: Interviews Aim: To investigate referral pattern variation for GPs working in out-of-hours care Detail of participants (number, any reported demographics): 15 GPs, five high, five medium and five low referrers. 10 male; mean years since qualifying 19 |
Methods: Interview plus GPs asked to fill in short questionnaire about personal and background characteristics. Vignette of 80-year-old women with breathlessness presented with alternative scenarios used to stimulate discussion during interview Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GPs (out of hours) to hospital admission (paper refers to intermediate care not emergency) |
Outcome measures: Views |
Main results: Key element in the decision to admit or not was GP‘s perspective. Little variation in significance attached to patient clinical condition High-referring GPs tended to be cautious and describe being better to admit if in doubt. They tended to express anxiety about the consequences of a decision. Tended to hold negative attitudes towards alternatives to hospital admission. Saw hospitals as places to be avoided and their role was to prevent admission Low referrers more confident in their decisions and loss often worried afterwards. More willingness to accept a degree of uncertainty. Low referrers more positive about alternatives to hospital and described themselves as able to resist pressure from family or carers Low referrers all male, older and had longer experience |
Reported associations between elements for logic model: GP individual beliefs and views linked to referral decisions (for hospital admission) |
| Carlsen et al. 2008244 Country: Norway Study design: Cross-sectional Data collection method: Survey Aim: To explore whether or not level of congruence in attitude between patient and GP influences referral rate Detail of participants (number, any reported demographics): 56 GPs in Bergen region asked to distribute a questionnaire to 50 of their patients older than age 16. 41 GPs included. Mean age 47.2 years; 66% male; mean list size 1161; mean 13% of consultations resulted in referral range 3.25 to 5.25. 835 patients; mean age 49.3 years; 71% female; three levels of education 2.15% |
Methods: Patient questionnaire, GP questionnaire containing information on demographics and practice information Both completed Patient Practitioner Orientation Scale. Difference between patient and GP scores calculated Control: None Length of follow-up: NA Response and/or attrition rate: 46% patient survey Context (from what/who to what/who): GP to specialist |
Outcome measures: Level of congruence between patient and practitioner – Patient Practitioner Orientation Scale Demographics Reported referral rate |
Main results: Male GP score – mean 4.26, SD 0.45 Female GP score – mean 4.42, SD 0.47 Male patient score – mean 4.38, SD 0.59 Female patient score – mean 4.51, SD 0.58 GP–patient difference score – mean 0.39, SD 0.36, minimum 0 and maximum 1.19 Significant negative correlation between GP score and reported referral rate –0.46, p = 0.002, indicating GPs with a preference for patient involvement are less likely to refer No significant correlation between patient score and referral rate. Low correlation between GP and patient score r = 0.029, p = 0.07 The larger the difference between GP and patient in attitudes towards shared decision-making the higher the referral rate (p = 0.001) Patient age, GP age, GP years of experience, patient educational level, list size and number of GPs in practice did not appear to be influential Male GPs had higher referral rates; however, GP gender was not significant |
Reported associations between elements for logic model: More the doctor and patient differ in attitude towards patient involvement, the more often GP refers to specialist care GPs who prefer shared decision-making refer less Importance of doctor–patient interaction |
| Chan 2003275 Country: Canada Study design: Cross-sectional Data collection method: Physician claims database Aim: Examines the factors affecting referrals by primary care physicians to specialists Detail of participants (number, any reported demographics): |
Method: Multilevel Poisson models were used to test the impact of patient- physician- and community-level variables on the referral rate (the number of office-based specialist referrals per patient by the patient’s customary primary care physician in fiscal year 1997/98). Patients from each of 6972 PCPs with sufficient data in Ontario were examined Control: NA Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist |
Outcome measures: Referral |
Main results: The average patient had 0.56 referrals per year (range 0–61). Referrals were higher at ages 1 and 77 to 78, and among women of childbearing age Chronic disease variables were strongly correlated with referral rates. Patients in poor neighbourhoods had more referrals, because they had more chronic diseases After controlling for disease, individuals in the top 9% wealthiest neighbourhoods had 4% more referrals. Female physicians made 8% more referrals than men. Older physicians referred more because they saw older patients; after controlling for patient age, physician age had no effect Referrals were 14% higher in cities with medical schools compared with other cities and 12% lower in small towns. However, local specialist supply was unrelated to referral rates |
Reported associations between elements for logic model: |
| Chauhan 2012284 Country: UK Study design: Cross-sectional Data collection method: Admission data (2006–7 and 2007–8) Aim: To identify the characteristics of general practices and patients associated with elective admissions Detail of participants (number, any reported demographics): |
Method: Audit of referral data Control: None Length of follow-up: none Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist |
Outcome measures: Practice characteristics (list size, distance from principal hospital, QOF score and GP patient access survey data) and patient characteristics (age, ethnicity and deprivation and gender) |
Main results: Practices with a higher proportion of patients aged 65 years or older and of white ethnicity had higher rates of elective hospital admissions. Practices with more male patients and with more patients reporting being able to consult a particular GP had fewer elective hospital admissions. For 2007–8, practices with a larger list size were associated with higher elective hospital admissions. QOF performance did not predict admission numbers |
Reported associations between elements for logic model: Practice characteristics predict referral |
| Chen 2005286 Country: USA Study design: Cross-sectional Data collection method: Patient contact database analysis Aim: To examine the effect of patient characteristics on referral Detail of participants (number, any reported demographics): 251,240 patient encounters in nine clinics resulting in 23,720 referrals |
Method: Patient contacts including office visits Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): All specialties |
Outcome measures: Referral rate Patient characteristics |
Main results: Higher referral rates for male patient encounters (9.95 vs. 9.11, p = 0.02) Patients over 65 years and under 18 years less likely to be referred (10.82 vs. 6.73 and 7.24, respectively, p = 0.001) Ethnic minority patients less likely to be referred (p = 0.002) 19 significant diagnostic groups for referral including likelihood of recurrence, chronic specialty, psychosocial, major/minor symptoms |
Reported associations between elements for logic model: Patient demographic and clinical characteristics and referral |
| Chew-Graham 2008229 Country: UK Study design: Qualitative study as part of RCT Data collection method: Interviews Detail of participants (number, any reported demographics): GPs, psychiatrists and managers or clinical leads of CMHTs |
Methods: Qualitative interviews Control: None Length of follow-up: None Response and/or attrition rate: NA Context (from what/who to what/who): GP to mental health |
Outcome measures: Views |
Main results: Referral purpose was to access specialist knowledge, which was perceived to lie with a psychiatrist and therefore referral to a team did not allow this access GPs have personal threshold after which they refer to secondary care Personal threshold varies between GPs Lack of direct doctor-doctor communication was perceived to contribute to fragmentation of patient care. Strategies could be used to bypass the team and achieve doctor–doctor communication |
Reported associations between elements for logic model: GP discontent with services that operate by referral to a team rather than a specialist hospital consultant Direct doctor to doctor communication a key element of GP satisfaction Different referral threshold of GPs |
| Chung 2010305 Country: USA Study design: Cross-sectional Data collection method: Database analysis Aim: To compare referral patterns by PCPs with nurse practitioners and assistants with those without Detail of participants (number, any reported demographics): |
Methods: Data from physician surveys (self-reported data) Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): PCP referral to specialist |
Outcome measures: Number of referrals to specialists |
Main results: Practices with nurse practitioners or physician assistants were found to have a greater likelihood of treating patients with complex conditions instead of referring them to specialists Managed care variables (size of practice, percentage of patients, percentage of revenue from prepaid contracts) affected referrals but only through the increased use of assistants or nurse practitioners |
Reported associations between elements for logic model: Practice characteristics and predictors of referral |
| Clarke 2010190 Country: UK Study design: Survey Data collection method: Postal return Aim: Principal aim in carrying out a nationally representative survey of GPs in England was to gather data to inform the development of new referral tools in phase 2 of the REFER project Detail of participants (number, any reported demographics): 324 GPs |
Method: The REFER project was carried out to improve the process of referral from primary to secondary care for patients with non-urgent conditions who may benefit from surgical treatment Survey of GPs, surgeons and members of the public to establish the representativeness of the guideline development groups’ views on referral appropriateness An eight-page self-completion questionnaire was developed based on previous research and the knowledge of the research team The questionnaire was mailed to 324 GPs who were randomly selected from the list of 10 representative PCTs in England Control: None Length of follow-up: None Response and/or attrition rate: 40% responded Context (from what/who to what/who): GP referral for elective surgery |
Outcome measures: Practitioner views on referral guidelines |
Main results: Although there was overall support for referral guidelines, they were rarely used in practice Over one-fifth of respondents indicated that they were expected to use referral guidelines by their local hospital or PCT. They indicated that referral guidelines would be most useful for patients with osteoarthritis of hip and knee, prostate problems, stress incontinence, infertility, back pain and menorrhagia The notion that patients should be involved in deciding whether they should be referred or not was strongly supported |
Reported associations between elements for logic model: Role of patients in decision-making Lack of use of referral guidelines |
| Clemence 2003228 Country: UK Study design: Qualitative Data collection method: Semistructured interviews Aim: Insight into the experiences and views of patients, GPs and physiotherapists in relation to physiotherapy referral for musculoskeletal conditions Detail of participants (number, any reported demographics): 22 semistructured in-depth interviews |
Method: Interviews were undertaken with GPs in primary care, health authority and hospital locations. Interviews with physiotherapists and patients were undertaken within community and district hospital locations Control: NA Length of follow-up: NA Response and/or attrition rate: NR Context (from what/who to what/who): GP referral for musculoskeletal conditions |
Outcome measures: Practitioner views |
Main results: Three classifications of referral type were developed by the authors from the data: ‘appropriate referral’, ‘load-sharing referral’ and ‘dumping referral’. There are descriptions of influences on GP referral behaviour, physiotherapists’ response to appropriateness and expectations from the perspective of GPs and physiotherapists Communication was shown as important in determining appropriate referral, but the quality of communication was variable. GPs’ past experience of physiotherapy significantly affected referral. Patients’ expectations about physiotherapy were described as variable and sometimes unrealistic |
Reported associations between elements for logic model: The selection of appropriate referrals by GPs could be helped by improved communication and better definitions of appropriateness |
| Cohen 2013267 Country: USA Study design: Cross-sectional Data collection method: Retrospective database analysis Aim: To investigate factors influencing referral Detail of participants (number, any reported demographics): Database of patients with laryngeal disorder, 149,653 patients who saw a PCP or self-referred |
Methods: Database analysis Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Referral to otolaryngology |
Outcome measures: Patient demographics, time to referral |
Main results: Older patients more commonly had a referral Women lower hazard ratio for referral than men Greater number of PCP visits, the lower hazard ratio for referral and more days to referral Multiple comorbidities higher ratio for referral Those referred by PCP seen earlier than those self-referred |
Reported associations between elements for logic model: Patient demographics and referral |
| Cooper and Wojnarowska 2001279 Country: UK Study design: Cross-sectional Data collection method: Analysis of referral data Aim: To investigate influences on referral to a vulval clinic Detail of participants (number, any reported demographics): 200 patients attending a dermatology vulval clinic in two hospitals, average age 55.3 years. Majority referred by a GP (167) some other specialties |
Methods: Retrospective cross-sectional analysis of referral data Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP and other specialists to dermatology vulval clinic |
Outcome measures: Referrer demographics |
Main results: 63% of women were referred by female doctors, despite only 38% being registered with a female GP |
Reported associations between elements for logic model: Influence of physician gender on referral patterns |
| Cornford 2004218 Country: UK Study design: Qualitative Data collection method: Interviews Aim: To investigate views regarding the 2-week rule intervention for breast cancer referral Detail of participants (number, any reported demographics): 12 patients referred via the 2-week rule: six from each centre, aged 36–70 years. Two diagnosed with breast cancer 20 professionals – nine GPs, five surgeons, three nurses, one radiologist, one radiographer and one manager |
Methods: Qualitative study using semistructured interviews Control: None Length of follow-up: NA Response and/or attrition rate: In centre 1 16 out of 60 (27%) patients replied to invitation and six were interviewed. Data were not available for centre 2 Context (from what/who to what/who): GP to breast specialist centres |
Outcome measures: Views |
Main results: None of the patients could explain the 2-week rule or define it as a new initiative. Concern was to be seen quickly and assured they did not have cancer Professionals mentioned that women referred via 2-week rule would be assured that whatever could be done would be done speedily and those without cancer could be reassured quickly All professionals described potential or actual disadvantages of the system (particularly those working in the centres). Disadvantages more strongly stated than advantages. Disadvantages – patients not referred via this category were waiting longer, or problems could result if waiting times increased. Perception that system put increased pressure on the service in terms of reorganising clinics with longer and busier clinics and more stressed staff Patients emphasised importance of communication skills throughout the referral process. None thought their symptoms trivial (even after cancer had been excluded) and felt their concerns should be taken seriously GPs reported issues of correct diagnosis and varied in the extent to which they could accept the uncertainty about diagnosis. Increased waiting time for non-2-week categorised patients could lead to 2-week box being ticked where unsure Professionals in centres perceived increased pressure on GPs from patients. Patients were described as more informed, more aware of rights and having greater expectations Some viewed increase in breast cancer awareness as beneficial but many saw it less positively as leading to increased pressure from patients Some views from secondary professionals that GPs were not following the guidelines or misused the system Views ranged from mild irritation through sarcasm to anger. Perception of loss of control expressed by professionals. Those expressing anger at the system described loss of autonomy |
Reported associations between elements for logic model: Professional dissatisfaction with the model of management Loss of autonomy, potential impact on waiting time for some patients Patient pressure and increased pressure following awareness raising via media Difficulties in implementing the system |
| Coulston et al. 2008187 Country: UK Study design: Cross-sectional Data collection method: Survey Aim: To examine referral patterns and assess knowledge of services Detail of participants (number, any reported demographics): 86 GPs in south-east Wales |
Methods: Postal survey Control: None Length of follow-up: NA Response and/or attrition rate: 86 of 120 returned (72%) Context (from what/who to what/who): GP to surgeon for groin hernia repair |
Outcome measures: Referral destination |
Main results: 84% referred to a general surgeon. Only 17% were aware of any specialist consultant surgeons in south Wales performing laparoscopic groin hernia repair. Of those who were aware of this service, 80% had at some time referred to this service Only 11% of GPs reported that patients ever requested referral to a hernia specialist and only 5% to a laparoscopic hernia specialist |
Reported associations between elements for logic model: Link between knowledge/having information about the existence of a service and referral |
| Coyle 2011280 Country: Ireland Study design: Cross-sectional Data collection method: Survey Aim: To study referral patterns Detail of participants (number, any reported demographics): 10 GPs, four GP practices, County Donegal. Data from 3166 consultations |
Methods: Data collection instrument given to GPs to complete Control: None Length of follow-up: NA Response and/or attrition rate: 100% Context (from what/who to what/who): GP to secondary care including emergency and/or to AHPs in primary care team |
Outcome measures: Clinical measures GP characteristics Referral made or not made |
Main results: Mean referral rate of male participants to secondary care was 5.4% and for female mean referral rate 11.3% (p < 0.0001) Mean referral rate of single-handed practices slightly higher than group (7.9% vs. 6.3%) |
Reported associations between elements for logic model: Gender difference in referral rate |
| Dagneaux 2012230 Country: Belgium Study design: Qualitative Data collection method: Focus groups Aim: Focused on the relationship between GPs and hospital geriatricians Detail of participants (number, any reported demographics): |
Methods: Focus group discussions were performed to describe the collaboration between GPs and hospital geriatricians: four of these focus groups contained only GPs, two groups contained only hospital geriatricians, and one group was made up of GPs and hospital specialists. Participants were invited to speak about bad or good experiences of intercollaboration Control: NA Length of follow-up: NA Response and/or attrition rate: NR Context (from what/who to what/who): GPs and hospital geriatricians |
Outcome measures: Practitioner views |
Main results: An important regional disparity was observed: better relationships and easier collaboration were reported in those regions that benefit from a wider range of geriatric services. In areas with few geriatric services, doctors knew little of other professionals and reported suspicion and even conflicts. Positive experiences and communication favour good relationships |
Reported associations between elements for logic model: Relates to links between referral and relationship between primary care and specialists The collaboration between GPs and hospital geriatricians should be enhanced: information, exchanges, and reflection on roles and competencies are essential |
| Dale 2000224 Country: UK Study design: Cross-sectional Data collection method: Postal questionnaire Aim: To investigate factors influencing referral Detail of participants (number, any reported demographics): 50 GPs referring to Great Ormond Street Hospital (GOSH) from 25 health authorities in south-east England |
Methods: Postal questionnaire to 50 GPs who made referrals to the GOSH Neurodisability Service Control: None Length of follow-up: NA Response and/or attrition rate: 100% Context (from what/who to what/who): GP to paediatric neurodisability service GOSH |
Outcome measures: GP views |
Main results: Two-thirds of GPs did not contact a specialist prior to referral 78% of GPs reported that in making the referral they were responding to parental concerns 90% gave reason for referral as parents seeking a second opinion outside their district. Other reasons for referral were having prior knowledge of the service and having previously referred to the service |
Reported associations between elements for logic model: Patient influence on referral Knowledge of service |
| Davies 2007256 Country: UK Study design: Mixed Data collection method: Audit of endoscopy referrals assessed how often these recorded rectal examination and whether or not patients were seen within 2 weeks. Qualitative interviews with 19 patients explored their experience of referral and diagnosis. Review of 33 case records assessed other possible delays Aim: To use clinical audit, qualitative data from patients and feedback from GPs to identify possible delays in referral, and to decrease these by implementing referral guidelines Detail of participants (number, any reported demographics): Above |
Methods: Patient interviews, case record reviews Control: None Length of follow-up: None Response and/or attrition rate: 19/27 (70%) patients agreed to be interviewed Context (from what/who to what/who): GP referral for endoscopy |
Outcome measures: Patient views |
Main results: Most patients referred for endoscopy were seen within 2 weeks (67%, 119/177), but only 47% (71/151) of available referral letters mentioned rectal examination Patients perceived most delay in secondary care and case records suggested that this occurred after non-urgent referral. Patients also identified some problems with communication, information and support about the diagnosis Used the results to stimulate local acceptance of national referral guidelines and wider discussion about care |
Reported associations between elements for logic model: Content of referral letter Patient perceptions |
| Dearman 2006265 Country: UK Study design: Cross-sectional Data collection method: Analysis of patient records and referral letters + 13 referred patients matched to 13 control patients Aim: To determine which older patients are referred to psychiatric services Detail of participants (number, any reported demographics): 1089 elderly patients in one large practice in central Manchester |
Methods: Retrospective analysis of computerised records and referral letters Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to psychiatry; older patients |
Outcome measures: Referral rates Patient characteristics |
Main results: Suicidal ideation and treatment failure were the principal reasons for referral Patients referred had a greater psychiatric comorbidity and had consulted their GP more frequently in the past year |
Reported associations between elements for logic model: Patient characteristics and referral likelihood |
| Delva 2011 Country: France Study design: Cross-sectional Data collection method: Survey Aim: To describe factors influencing GP referral decisions for older patients with cancer Detail of participants (number, any reported demographics): GPs stratified sample – n = 436 75% male, mean age 50 years, average years practice 21 years, two-thirds working in urban setting |
Method: Questions on ease of referral, influencing factors plus two case vignettes Control: None Length of follow-up: NA Response and/or attrition rate: 30% Context (from what/who to what/who): GPs to oncology |
Outcome measures: Factors influencing referral Demographics |
Main results: Approximately half of GPs reported that they always refer elderly cancer patients More than 75% of GPs reported being influenced by patient elements (patients/family wishes, comorbid factors, unsuitability of invasive investigations, physical and mental autonomy), cancer-related elements (severity of symptoms, side effects) and an organisational element (whether or not the GP was used to collaborating with the oncologist) Organisational difficulties in patient management and cancer site were significantly associated with less likely decision to refer in early-stage cancer (OR 0.35, 95% CI 0.24 to 0.56; p < 0.0001; and OR = 0.58, 95% CI 0.37 to 0.92; p = 0.02) GPs’ attendance at training course was associated with being more likely to refer advance cancer (OR 1.85, 95% CI 1.01 to 3.38) No individual GP characteristics associated with referral decisions. GPs less likely to refer advanced cancer were those who reported being influenced by patient age, organisational difficulty in providing care and the stage of the disease |
Reported associations between elements for logic model: Patient-related factors influence in referral decisions |
| Dodds 2004174 Country: UK Study design: Cross-sectional Data collection method: Postal survey Aim: To investigate views regarding the 2-week wait rule Detail of participants (number, any reported demographics): 331 GPs 80% practices with four or more doctors |
Methods: Cross-sectional postal survey Control: None Length of follow-up: NA Response and/or attrition rate: 65% Context (from what/who to what/who): GP to cancer specialist |
Outcome measures: Reported referral methods and views |
Main results: 90% used urgent referral form, 38% wrote and additional letter. 61% referrals faxed, 27% faxed and posted 81% of GPs believed that patients with suspected cancer had an initial appointment faster than before the rule was introduced 50% of GPs thought information about the care pathway had improved following the rule, 41% no change. Main reason for improvement was GP now received fax from hospital with date of patient’s initial appointment. Other comments – system less time-consuming and more efficient 3% reported missing direct contact with specialist Positive comments – simplicity of form, helpful guidelines on completion, completing form rather than dictating letter was speedier. Problems – having the right form for the right specialty, different centres used different forms, interpreting and applying the guidelines, rigid and inflexible system not offering scope for GP own judgement and experience, also importance of patient level of anxiety in decision-making. Report of some tendency to overuse 2-week referral due to patient pressure and erring on the side of caution GPs reported dislike of categorising breast cancer in suspected grade and worry wrong categorising would delay referral. Also fear that patients not referred as 2 weeks would wait too long Some suggestion that non-adoption of referral form highest among single-handed practices |
Reported associations between elements for logic model: Over-referral of cases as priority in cases of uncertainty/patient pressure/patient anxiety Positive views of improved waiting time and communication Simplicity and availability of forms |
| Edwards 2002257 Country: UK Study design: Qualitative Data collection method: Practice-based, semistructured group discussions Aim: To explore the responses of primary health-care professionals to guidelines in general, and to the UK national guidelines on the management and referral of women with breast conditions in particular Detail of participants (number, any reported demographics): n = 86 primary health-care professionals |
Method: All practices were involved in a study evaluating the effect of the national breast symptom management guidelines on clinical practice. Discussion groups were conducted in the practice; 15 of the 34 study practices were audio-taped. Groups comprised the available partners and practice nurses at each practice. All members of each group were well known to one another. 1-hour educational session (approved for postgraduate education allowance), facilitated by two members of the study team (two GPs, a research psychologist and a research nurse). 15 seminars; six seminars addressed the breast lump guideline and nine addressed the breast pain guideline Control: None Length of follow-up: None Response and/or attrition rate: 75% of male partners and 90% of female partners present at the meetings. Of the practice nurses, 51% attended Context (from what/who to what/who): Primary care referral using guidelines |
Outcome measures: Practitioner views |
Main results: A number of areas for comment arose consistently in each discussion. Guidelines in general were welcomed, although there was more support for evidence-based than consensus-based guidelines. The breast lump guideline was also welcomed, participants commenting that it was concordant with current practice. The breast pain guideline was more contentious, participants reflecting that it recommended more primary care management than was usual, and fell outside the experience and confidence of doctors or the expectations of patients. Participants felt it did not incorporate psychosocial factors into recommended management, despite this being an important part of primary care management |
Reported associations between elements for logic model: There are concerns about consensus guidelines and doubts about the applicability of the breast pain guideline in this complex management area. The failure of these guidelines to address patient anxiety may limit their implementation Different types of guideline – evidence-based vs. consensus Importance of patient psychosocial factors in referral Expectations of patients Doctor confidence |
| Elhayany et al. 2000167 Country: Israel Study design: Cross-sectional Data collection method: Review of referral forms Aim: To examine variation in referral rates Detail of participants (number, any reported demographics): 67,577 patients seen by 44 primary care physicians in South Israel |
Methods: Cross-sectional review of referral forms Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): From GP to any specialism |
Outcome measures: Referral rate Characteristics of physician |
Main results: Age, gender, country of origin and year of immigration to Israel of the physician were not associated with the rate of referral GPs without any postgraduate training or specialty designation were likely to refer 2.5 times more often than primary paediatricians or family physicians Being a family medicine specialist and the number of years since graduation explained 17.7% of the variability in referral rates (specialty stronger element than time since graduation). The more years since graduation, the higher the referral rate |
Reported associations between elements for logic model: GP training and referral rates |
| Espeland 2003217 Country: Norway Study design: Qualitative Data collection method: Focus groups Aim: To identify and describe (a) factors GPs consider may affect their decisions about ordering plain radiography for back pain and (b) barriers to guideline adherence suggested by such factors Detail of participants (number, any reported demographics): Norwegian GPs, n = 14 |
Method: Focus group interviews regarding factors affecting ordering decisions were carried out on a diverse sample of Norwegian GPs and were analysed qualitatively Results of this study and two qualitative studies from the Netherlands and USA on use of spine radiography were interpreted for barriers to guideline adherence These were compared with an existing barrier classification system Control: NA Length of follow-up: NA Response and/or attrition rate: Context (from what/who to what/who): Plain radiography for back pain |
Outcome measures: Factors which predict referral |
Main results: The factors that Norwegian GPs considered might affect their decisions about ordering plain radiography for back pain concerned the following broader issues: clinical ordering criteria, patients’ wishes for radiography and the GP’s response, uncertainty, professional dignity, access to radiology services, perception of whether the patient really was ill, sense of pressure from other health-care providers/social security, and expectations about the consequences of ordering radiography The three studies suggested several attitude-related and external barriers as classified in a previously reported system described in another study. Identified barriers not listed in this system were lack of expectancy that guideline adherence will lead to desired health-care process, emotional difficulty with adherence, improper access to actual/alternative health-care services and pressure from health-care providers/organisations |
Reported associations between elements for logic model: Barriers to referral for radiography |
| Feeney 2007282 Country: UK Study design: Retrospective case series Data collection method: Patient records Aim: To determine if the gender of the GP is associated with referral rates for different eating disorders Detail of participants (number, any reported demographics): 93 patients |
Method: Associations between GP gender and clinical characteristics of the patients were determined using chi-squared tests and t-tests Control: NA Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to eating disorder clinic |
Outcome measures: GP gender and clinical characteristics |
Main results: Referral rates by male and female GPs were similar for anorexia nervosa (28.6% vs. 71.4%) and bulimia nervosa (38.7% vs. 61.3%), each of which is similar to the referral rates by males and female GPs overall (32.3% vs. 67.7% of the sample, respectively) However, female GPs were substantially more likely to refer binge eating disorder patients (92.7% by female GPs vs. 7.7% by male GPs), but they were less likely to refer other Eating Disorder Not Otherwise Specified cases (50% referred by either gender of GP) The chi-squared test showed that there was a significant association between referrer gender and diagnosis (χ2 = 6.42, df = 3, p < 0.05). When the analysis was simplified to compare the association of a diagnosis of binge eating disorder (vs. all other diagnoses) with the gender of the referrer, this was also statistically significant (χ2 = 4.17, df = 1, p < 0.025) |
Reported associations between elements for logic model: Patients with binge eating disorder were substantially more likely to be referred by female GPs, while other atypical cases were more likely to be referred by male GPs. There was no evidence that the patients referred by male GPs differed in clinical presentation from those referred by female GPs |
| Forrest 2003219 Country: USA Study design: Cross-sectional Data collection method: Primary care practice-based study of referred and none referred office visits Aim: To examine the influence of gatekeeping arrangements and capitated primary care physician payment on the specialty referral process in primary care settings Detail of participants (number, any reported demographics): The study comprised 14,709 visits made by privately insured, non-elderly patients who were seen by 139 primary care physicians in 80 practices located in 31 states |
Method: Visits were grouped by health plan type: gatekeeping with capitated PCP payment; gatekeeping with fee-for-service PCP payment; no gatekeeping Control: NA Length of follow-up: NA Response and/or attrition rate: NR Context (from what/who to what/who): Primary care to specialist |
Outcome measures: Referral rates Dependent measures included the proportion of visits referred, characteristics of referrals, and physician co-ordination activities |
Main results: The percentages of office visits resulting in a referral were similar between the two gatekeeping groups and higher than the no gatekeeping group. Patients in plans with capitated PCP payment were more likely to be referred for discretionary indications than those in non-gatekeeping plans (15.5% vs. 9.9%, p < 0.05). The frequency of referring physician co-ordination activities did not vary by health plan type. The proportion of patients in gatekeeping health plans within a practice was directly related to employing staff as referral co-ordinators, allowing nurses to refer without physician consultation, and permitting patients to request referrals by leaving recorded telephone messages |
Reported associations between elements for logic model: In response to increasing numbers of patients enrolled in managed health plans with gatekeeping arrangements, physicians appear to modify the structure of their practices to facilitate access to and co-ordination of referrals |
| Forrest 2002223 Country: USA Study design: Prospective cohort Data collection method: Survey, log of clinic visits for 15 days Aim: To examine referral decisions Detail of participants (number, any reported demographics): 141 family physicians, visits and new referrals = 34,519 + 2534 over 15 consecutive days |
Method: Prospective cohort study, survey of patient data Control: None Length of follow-up: NA Response and/or attrition rate: N Context (from what/who to what/who): Primary care physicians to specialist |
Outcome measures: Type of problem Number of referrals |
Main results: Patient request reason for 13.6% of referrals Physicians recommended a specific practitioner to the patient for 86.2% of referrals; personal knowledge of the specialist was the most important reason for selecting a specific specialist Obtaining advice the most common reason for the referral |
Reported associations between elements for logic model: Personal knowledge and referral, patient request |
| Forrest 2006283 Country: USA Study design: Cross-sectional Data collection method: Survey Aim: To examine factors influencing specialty referral decision-making Detail of participants (number, any reported demographics): 142 physicians in 83 practices; 34,069 visits. Mean age of patients 42.3 years; majority female with private insurance. 43% of visits made by patients whose health plan had a gatekeeping arrangement |
Method: Questionnaire completed and data collected on all patient visits during 15 consecutive working days Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Primary care physicians to specialty referral |
Outcome measures: Modelling of occurrence of specialty referral |
Main results: Referral made in 5.2% of visits Patient characteristics had the largest effects in any referral model. Referral decisions influenced however by a complex mix of patient, physician and health-care system structural characteristics including primary care physician with less tolerance of uncertainty, larger practice, gatekeeping health plan, level of managed care Variables not influential: physician gender, anxiety due to clinical uncertainty, anxiety regarding bad outcomes, perceived clinical expertise, specialist availability, perceived pressure to control referral costs, restricted ability to obtain invasive tests, restricted ability to obtain surgical referrals, Herfindahl Index, hours of patient care per week, visits per day, type of physician income, ownership of the practice Patient variable increased the chance of referral were aged over 17 years, male, uncommon presenting problem cared for by GP, high burden of comorbidity, patient insured, health plan had gatekeeping arrangements Physician variables associated with referral were greater reluctance to disclose uncertainty to patients, less reluctance to disclose uncertainty to other physicians Practice characteristics associated with referral were higher levels of managed care in the practice and larger group practices |
Reported associations between elements for logic model: Patient, physician and service characteristics and likelihood of referral |
| Forrest 2007246 Country: USA Study design: Survey Data collection method: Self-administered questionnaire Aim: Describes referral completion from the perspectives of patients and primary-care physicians and identifies predictors of adherence to the referral recommendation Detail of participants (number, any reported demographics): Cohort of 776 referred patients from the offices of 133 physicians in 81 practices and 30 states |
Method: Referring physicians and patients completed self-administered questionnaires at the time of the referral decision and 3 months later Length of follow-up: 3 months Response and/or attrition rate: 78% patient response rate, 97% physician response rate Context (from what/who to what/who): Primary care to specialist |
Outcome measures: Practitioner views Patient views |
Main results: Physicians reported that 79.2% of patients referred had a specialist visit, and 83.0% of patients indicated they completed the referral The most common reasons for not completing the referral were ‘lack of time’ and patient belief that the ‘health problem had resolved’ The κ-statistic for patient–physician agreement on referral completion was 0.34, indicating only fair concordance. Patients in Medicaid plans were less likely than others to complete the referral, and more likely to experience a health plan denial. A longer duration of the patient relationship with the primary care physician and physician/staff scheduling of the specialty appointment were both positive predictors of referral completion |
Reported associations between elements for logic model: Referral completion rates may be increased by assisting patients with scheduling their specialty appointments and promoting continuity of care Relationship between doctor–patient relationship and attendance |
| Franks 2000216 Country: USA Study design: Cross-sectional Data collection method: Physician survey and patient database analysis Aim: To determine which factors contribute to variation in referral rates Detail of participants (number, any reported demographics): 173 internists and family physicians in Rochester, NY area |
Methods: Claims database used for patient data, survey to physicians included several psychometric scales using Likert scales Control: None Length of follow-up: NA Response and/or attrition rate: 66% Context (from what/who to what/who): Primary care to specialty referral |
Outcome measures: Patient referral rate Predicted risk of referral measure Physician demographics/characteristics |
Main results: Patients more likely to be referred if physician female, had more years in practice, was an internist and used a narrower range of diagnoses Greater psychosocial orientation and greater malpractice fear associated with greater likelihood of referral. Other psychological measures explained little of the variance in referral When physician factors excluded from the analysis risk aversion positively associated with referral likelihood |
Reported associations between elements for logic model: Physician psychological factors little associated with referral Demographic factors associated with referral |
| Franz 2010307 Country: USA Study design: Qualitative Data collection method: interviews Aim: To describe structural barriers to mental health specialists and consequences of these barriers to care for patients with dementia and neuropsychological symptoms and their primary care physicians Detail of participants (number, any reported demographics): 40 primary care physicians in California |
Methods: Open-ended interviews lasting 30–60 minutes Length of follow-up: None Response and/or attrition rate: NR Context (from what/who to what/who): primary care to mental health |
Outcome measures: Practitioner views |
Main results: 93% of the primary care physicians described problematic access and communication with mental health services (in particular psychiatrists and neuropsychiatrists) as impediments to effective care for dementia patients Thematic analysis identified structural barriers to mental health referrals ranging from problems with managed care and reimbursement policies to lack of trained providers and poor geographical distribution of specialists. Structural barriers compromised care for patients with dementia because the barriers limited PCP treatment options and resources, impacted office staff and time with other patients, impeded and delayed care, and fostered poor communication and lack of co-ordinated care Negative consequences for PCPs included increased frustration, conflict and burnout |
Reported associations between elements for logic model: Structural barriers to mental health referrals ranging from problems with managed care and reimbursement policies to lack of trained providers and poor geographical distribution of specialists |
| Freed et al. 2003172 Country: USA Study design: Cross-sectional Data collection method: Postal survey Aim: To examine the referral preferences of physicians Detail of participants (number, any reported demographics): |
Methods: Cross-sectional postal survey Control: None Length of follow-up: NA Response and/or attrition rate: 49% for family physicians Context (from what/who to what/who): Primary-care provider referral for juvenile rheumatoid arthritis |
Outcome measures: Reported factors |
Main results: 61% of family physicians referred only to confirm the diagnosis and guide initial therapy The majority of family physicians reported feeling more confident managing the disease in adults rather than children (82%). Few respondents felt that they were up to date on the latest advances in juvenile rheumatoid arthritis treatment |
Reported associations between elements for logic model: Physician beliefs regarding adequacy of knowledge |
| Fucito 2003162 Country: Australia Study design: Cross-sectional Data collection method: Survey Aim: To determine the current practices of established GPs in managing patients with drug- and alcohol-related problems and identify gaps in training Detail of participants (number, any reported demographics): 145 GPs, 51% males and 49% females had an average age of 46.5 + 11.6 years and had worked in general practice for an average of 17.5 + 10.2 years |
Method: A random sample of GPs completed a survey assessing diagnostic skills and referral practices concerning alcohol and illicit drug use in general practices in February 1999, comprising 110 GPs registered with the Central Sydney Division of General Practice Control: NA Length of follow-up: NA Response and/or attrition rate: 110 (75.9%) Context (from what/who to what/who): GP referral for drugs/alcohol problems |
Outcome measures: Competent skills and knowledge, willingness to treat |
Main results: The majority (96%) of GPs provided clinically appropriate responses for at least one drug category, although none received this rating for all six Most GPs reported that they were unwilling to treat heroin and cocaine problems themselves but expressed willingness to refer patients appropriately GPs who stated they regularly obtained information about drug and alcohol use were more likely to refer patients (χ2 = 7.0, p < 0.01) More than one-quarter of GPs were unaware of the safe drinking levels for men and women or the appropriate treatment for patients consuming above such levels Age, years in practice, type of practice, willingness to obtain drug-use histories and post-graduate training were all significantly associated with GPs’ willingness to treat and competence in managing drug- and alcohol-related problems In this study, GPs reported low levels of skills and referrals for treatment of illicit drug use and suboptimal skills in the management of alcohol problems |
Reported associations between elements for logic model: The results suggest that a more comprehensive approach to education and training is required to bring about a change in practice behaviour |
| Gandhi 2000101 Country: USA Study design: Cross-sectional Data collection method: postal survey Aim: To examine dissatisfaction in the referral process Detail of participants (number, any reported demographics): 48 primary care providers and ‘over 400’ specialists |
Methods: Cross-sectional postal survey Control: None Length of follow-up: NA Response and/or attrition rate: 56% PCPs and 53% specialists Context (from what/who to what/who): Primary care provider to orthopaedics, cardiology and gastroenterology |
Outcome measures: Scale of satisfaction |
Main results: Primary care providers – three biggest problems with referral system were lack of timeliness of information from specialists, redundancy of the current process, and time required to create adequate referral notes Specialists – lack of timeliness of information from primary care providers, time required for medical management and lack of clarity of note content from primary care providers 28% of GPs and 11% of specialists somewhat or very satisfied with information provided. 28% of primary care providers and 43% of specialists dissatisfied with information received from the other group Specialists reported not receiving adequate information to address the problem 23% of the time |
Reported associations between elements for logic model: Content of information exchange |
| Glozier 2007254 Country: UK Study design: Cohort study Data collection method: Analysis of consecutive referral data, patient interview/questionnaires Aim: To assess the impact of patient distress and beliefs on GP urgent referral requests Detail of participants (number, any reported demographics): 188 referrals |
Methods: Patients interviewed and completed questionnaires regarding impairment, distress, personality and illness beliefs Control: None Length of follow-up: NA Response and/or attrition rate: 188/208 potential participants were included (90%) Context (from what/who to what/who): GP to orthopaedic department |
Outcome measures: Psychological distress (GHQ) Patient perceptions of symptom control (Illness Perception Questionnaire – subscales regarding personal control of symptoms, treatment effectiveness in symptom control, likely consequences) Patient characteristics |
Main results: 12% or referrals urgent and 88% routine Clinical measures associated with GP urgent request were pain, joint impairment, and disability reported by patient No sociodemographic characteristics were associated with an urgent request (patient gender, occupational class, marital status, support network, ethnicity, employment) Psychological distress of patient not associated with urgent referral (OR 1.62, 95% CI 0.64 to 4.07). Distressed in fact two times less likely to be referred urgently (OR 0.39, 95% CI 0.11 to 1.36) Adjusting for pain or disability eightfold difference in likelihood of urgent referral between those with least perceived control and most. OR of an urgent referral were 2.18 (95% CI 1.18 to 4.05) for each increasing quartile of personal symptom control Greater personal control and disability accounted for 46% of variance. No significant association age, number of comorbid conditions, number of medications, impairment, neuroticism and likelihood of urgent referral |
Reported associations between elements for logic model: Appropriate referrals linked to patient disability Patient personal control and referral – assertive patients better able to influence and control their lives more successful at obtaining an urgent referral? |
| Grace 2008292 Country: Canada Study design: Cross-sectional Data collection method: Survey Aim: To compare factors affecting cardiac rehab referral in primary care physicians versus cardiac specialists Detail of participants (number, any reported demographics):510 primary care physicians and cardiac specialists |
Method: A cross-sectional survey of a stratified random sample of 510 primary care physicians and cardiac specialists (cardiologists or cardiovascular surgeons) in Ontario identified through the Canadian Medical Directory Online was administered. One hundred four primary care physicians and 81 cardiac specialists responded to the 26-item investigator-generated survey Control: NA Length of follow-up: NA Response and/or attrition rate: One hundred and four primary care physicians and 81 cardiac specialists (i.e. 185/510 or 36% response rate) Context (from what/who to what/who): GP to cardiac rehab |
Outcome measures: Medical, demographic, attitudinal and health system factors affecting CR referral |
Main results: Primary care physicians were more likely to endorse lack of familiarity with cardiac rehab site locations (p < 0.001), lack of standardised referral forms (p < 0.001), inconvenience (p = 0.04), programme quality (p = 0.004) and lack of discharge communication from cadiac rehab (p < 0.001) as factors negatively impacting cardiac rehab referral practices than cardiac specialists Cardiac specialists were significantly more likely to perceive that their colleagues and department would regularly refer patients to cardiac rehab than primary care physicians (p < 0.001) |
Reported associations between elements for logic model: Unclear |
| Green 2008220 Country: UK Study design: Cross-sectional Data collection method: Postal questionnaire and patient vignette Aim: To examine GP individual differences in intention to refer Detail of participants (number, any reported demographics): Three PCTs, 88 GPs; 55% female, practised for a mean of 15 years, half in urban/suburban practices and half rural. Two-thirds had specialist psychiatric experience |
Method: Questionnaire developed from the theory of planned behaviour to assess attitudes, perceived norms, behavioural control and intention to refer. Responses on Likert scale. Vignette of hypothetical patient presenting for travel immunisation – in half of vignettes patient had normal weight and half underweight Control: Compared vignette of patient with normal weight with that of one underweight Length of follow-up: NA Response and/or attrition rate: 33% Context (from what/who to what/who): GP to specialist eating disorder service |
Outcome measures: Referral intent |
Main results: None of the GP or practice characteristics had a significant impact on GPs intention to refer (p = 0.05) Weight of patient did not impact on decision to refer None of the GPs beliefs, feeling of control over referral or whether or not felt had necessary referral skills were predictors of intention to refer (r = –0.05, p = 0.678 and r = –0.03 and p = 0.806) Cognitive attitudes were on the positive side towards referral (referral would be sensible/appropriate/helpful) 57%/55%/55% agreement Intention to refer was significantly related to subjective norms (referral would be recommended by colleagues/guidelines/research evidence) and cognitive attitudes (r = 0.917 and 0.0896 p < 0.001); together these predictors explained 86% of the variance in intention to refer |
Reported associations between elements for logic model: Variance in referral behaviour may be explained by cognitive attitudes and subjective norms – belief that their behaviour is in line with colleagues and guidelines when in reality it varies |
| Greenaway 2006306 Country: UK Study design: Mixed method Data collection method: Postal questionnaire with free-text responses Aim: To explore GP views of onsite vs. offsite psychology services Detail of participants (number, any reported demographics): 60 GP participants’ practices that refer to offsite psychology services and 60 participants practices with on-site psychology services |
Methods: Quantitative data and free-text comment responses Control: None Length of follow-up: NA Response and/or attrition rate: 209 questionnaires sent, 120 returned (57%) Context (from what/who to what/who): GP referral to psychology |
Outcome measures: Perceptions of service, perceptions of referral process |
Main results: No significant difference in understanding of psychological therapies between those with on-site and those with off-site services. On-site group rated own understanding of different providers of psychological services as higher (p = 0.003) 36% of GPs in off-site group indicated they had been provided with information on making referrals to their local psychology service, compared with 73.2% in the on-site group (p < 0.001) Free-text data indicated more positive perceptions of on-site service Having an on-site psychology service did not significantly affect GP perceived referral rate to psychology (p > 0.05) |
Reported associations between elements for logic model: On-site services associated in GP greater knowledge of specialist service and more positive perceptions; however, they did not change perceived referral rate |
| Greer 2011240 Country: USA Study design: Cross-sectional Data collection method: Survey Aim: To assess whether or not primary care provider use of a kidney function test impacted on timing of referral Detail of participants (number, any reported demographics): 178 family physicians plus internists and nephrologists. Primary care providers’ median years of practice 12 (3–21) majority in non-academic settings |
Method: Participants sent one of four hypothetical case scenarios featuring a white or African American female patient with or without diabetes with progressing chronic kidney disease. Test results provided for EGFR and serum creatinine. Participants asked to identify the test results at which they would refer. Visual analogue scales Control: None Length of follow-up: NA Response and/or attrition rate: Of the 959 physicians in active clinical practice, a total of 126 nephrologists and 178 primary care physicians responded to the questionnaire, i.e. 32% Context (from what/who to what/who): PCP to kidney specialist |
Outcome measures: Demographics Timing of referral |
Main results: One-third of PCPs reported aware of subspecialty referral guidelines for chronic kidney disease PCPs recommended referral earlier when using eGFR test results than when using serum creatinine to estimate kidney function. 94% recommended referral using eGFR, compared with 55% using serum creatinine results. 40% of PCPs significantly improved the timing of their referral using results from the EGFR test The improvement in timing of referrals was greater for physicians presented with a hypothetical white patient than African American |
Reported associations between elements for logic model: Enhanced use of the optimal test by PCPs could be associated with more timely referral |
| Gross 2000261 Country: Israel Study design: Cross-sectional Data collection method: Telephone survey Aim: To examine perceptions of patients regarding gatekeeping vs. self-referral models Detail of participants (number, any reported demographics): 1084 patients |
Method: National telephone survey Control: NA Length of follow-up: NA Response and/or attrition rate: 81% Context (from what/who to what/who): Primary to specialist |
Outcome measures: Views |
Main results: One-third of respondents preferred self-referral to a specialist. 40% prefer their family physician to act as a gatekeeper. 19% preferred the physician to co-ordinate care but to refer themselves to a specialist Variables associated with preferring gatekeeping model were not living in the central area of the country, having sick fund membership, low level of education, being male, fair or poor health status, having a permanent family physician, and being satisfied with the professional level of the family physician. Association between self-referral and practising self-referral |
Reported associations between elements for logic model: Variance in patient attitudes towards physicians as gatekeepers |
| Gruen 2002281 Country: Australia Study design: Cross-sectional Data collection method: Survey of patient encounters Aim: To explore a link between geographical proximity and referral Detail of participants (number, any reported demographics): 3030 GPs, each reporting 100 patient encounters |
Method: Part of BEACH programme describing GP activity Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to surgical specialist, (general, vascular, plastic, orthopaedic, ENT, obstetrics and gynaecology, ophthalmology, urology and other) |
Outcome measures: Referral to surgical specialist Presence or absence of local specialist (located in same population centre) |
Main results: Absence of local specialist did not influence proportion of general surgical referrals Absence of local specialist associated with significantly lower rate for obstetric referrals and opthalmological problems (OR 0.56, 95% CI 0.44 to 0.70 and OR 0.60, 95% CI 0.49 to 0.73). Greater likelihood of referring orthopaedic problems, though Other factors influencing lower referral rate: male GP, female or younger patient, holder of health-care card, injury-related and non-cancer problem, follow-up presentation, more than one problem managed at a consultation |
Reported associations between elements for logic model: Some link local service and referral but only some specialties |
| Guevara 2009290 Country: USA Study design: Cross-sectional Data collection method: Postal survey Aim: To determine whether or not on-site mental health provision is associated with referral Detail of participants (number, any reported demographics): 56% male; 59% under 46 years; 52% suburban locations. 17% had on-site services. Number of participants not provided in region of 300 |
Method: Physician Belief Scale, demographics Control: None Length of follow-up: NA Response and/or attrition rate: 51% Context (from what/who to what/who): Primary-care paediatricians to mental health |
Outcome measures: Likelihood of consultation or referral |
Main results: Paediatricians with on-site mental health provision were more likely than those without to consult (OR 6.58, 95% CI 3.55 to 12.18) or to refer (OR 4.25, 95% CI 2.19 to 8.22) Those with greater burden less likely to refer than those with lesser burden Physician beliefs concerning mental health treatment were not associated with consultation or referral |
Reported associations between elements for logic model: Colocation of services and increased referral |
| Harlan 2009231 Country: USA Study design: Cross-sectional Data collection method: Questionnaire Aim: To perform a needs assessment by exploring important issues around communication between paediatric hospitalists and primary care providers Detail of participants (number, any reported demographics): 10 paediatric hospitalists and 12 paediatric primary care providers |
Method: Six previously identified issues around hospitalist–primary care provider communication from the adult hospitalist literature were abstracted and incorporated into an open-ended and closed-ended questionnaire. The questionnaire was pretested, revised, and administered by telephone to 10 paediatric hospitalists and 12 paediatric primary care providers Control: NA Length of follow-up: NA Response and/or attrition rate: No primary care provider who was able to be contacted declined an interview Context (from what/who to what/who): Paediatric hospitalists and primary care providers |
Outcome measures: Practitioner views |
Main results: The six identified issues were quality of communication, barriers to communication, methods of information sharing, key data element requirements, critical timing and perceived benefits Hospitalists and primary care providers rated overall quality of communication from ‘poor’ to ‘very good’ Both groups acknowledge that significant barriers to optimal communication currently exist and yet the barriers differ for each group. Hospitalists and primary care providers agree on what information is important to transmit (diagnoses, medications, follow-up needs and pending laboratory test results) and critical times for communication during the hospitalisation (at discharge, admission and during major clinical changes) Both groups also agree that optimal communication could improve many aspects of patient care |
Reported associations between elements for logic model: Unclear |
| Harris 2011268 Country: Australia Study design: Part of a quasi-experimental study Data collection method: Questionnaires Aim: Explored the patient and practice factors associated with referral of patients with diabetes, ischaemic heart disease or hypertension to external allied health providers Detail of participants (number, any reported demographics): 26 practices. The mean age of patients was 61.6 years, ranging from 19 to 90 years; females (55%) |
Method: A multilevel analysis of data collected as part of a quasi-experimental study was conducted in 26 practices in Sydney. The frequency of patient-reported referral to AHPs 6 months post intervention was measured against patient and practice characteristics assessed by patients and practice staff questionnaires Control: None Length of follow-up: None Response and/or attrition rate: Not reported Context (from what/who to what/who): Ischaemic heart disease or hypertension |
Outcome measures: The frequency of patient-reported referral |
Main results: Seven per cent of the total variance in the referrals was due to differences between practices and 93% attributed to differences between patients. Previous referral, age over 45 years, multiple conditions, longer illness duration, poor mental and physical health were associated with the likelihood of referral to allied health providers, but not socioeconomic status, patient self-assessment of care and the intervention. Those attending practices with over three GPs were more likely to be referred After adjustment for other covariates, the likelihood of referrals was greater for ‘patients age 60 and over’ (OR for 60–70 years 3.00, 95% CI 1.08 to 8.34; and OR for over 70 years 3.37, 95% CI 1.01 to 11.17). The reference group was those aged 19–45 years; patients with longer duration (4–10 years) of disease (OR 3.89, 95% CI 1.29 to 11.71) compared with 1–3 years of duration; patients with lower mental component score SF12 (OR 0.97, 95% CI 0.94 to 0.99) and lower physical component score SF12 (OR 0.97; 95% CI 0.94 to 1.00); and the likelihood of referrals was less for: patients with only hypertension or ischaemic heart disease, i.e. one condition only (OR 0.18, 95% CI 0.07 to 0.45) |
Reported associations between elements for logic model: Unclear |
| Harvey et al. 2005242 Country: UK Study design: Cross-sectional Data collection method: Postal questionnaire Aim: To examine preferences for referral to mental health services Detail of participants (number, any reported demographics): 107 GPs Sheffield |
Methods: Two questionnaires sent one year apart (no intervention in between). Use of case vignettes Control: None Length of follow-up: NA Response and/or attrition rate: 94% Context (from what/who to what/who): GP referral to psychiatry |
Outcome measures: GP demographics Clinical care |
Main results: GPs producing higher-quality referral letters had graduated more recently No difference, however, between those writing higher-quality referral letters and the number of proposed referrals to psychiatric services and those writing lower-quality letters and the number of proposed referrals |
Reported associations between elements for logic model: Recently trained GPs and referral practices No link between quality of practitioner referral letter and their proposed number of referrals |
| Holley 2010293 Country: USA Study design: Qualitative Data collection method: Focus groups Aim: To understand the barriers facing primary care providers, including nurse practitioners and physician assistants, in the current referral-to-eye-care process and to solicit suggestions from primary care providers on how to improve the current referral system Detail of participants (number, any reported demographics): 17 primary care providers |
Methods: Four focus groups were conducted with a total of 17 primary care providers – two groups with physicians (one in a rural setting and one in an academic medical centre setting) and one group of nurse practitioners and one of physican assistants, both in an academic setting Control: NA Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to eye care providers |
Outcome measures: Provider views |
Main results: The most frequently cited referral barriers included (1) poor communication from eye care providers, (2) patients’ lack of finances/insurance coverage and (3) difficulty in scheduling an eye care appointment Suggestions made in all groups on ways to improve the current referral system included (1) implementing EMRs, (2) receiving better communication/feedback from ECPs, (3) having ophthalmologists hold clinic days in primary care facilities and (4) performing retinal scans in primary care clinics We found few differences between the opinions of physicians and those of nurse practitioners and physician assistants |
Reported associations between elements for logic model: Better communication between primary care providers and eye care providers, further implementation of EMRs, and increasing eye screening in primary care clinics were common themes Role of feedback following consultation in GP satisfaction with service |
| Hugo et al. 2000272 Country: UK Study design: Cross-sectional Data collection method: Referral data and practitioner data Aim: To explore referral rates and practitioner characteristics Detail of participants (number, any reported demographics): 434 GPs, 137 practices, 34% single-handed, largest had nine partners. 39% fundholding |
Methods: Database of referrals and details obtained from medical directories and health authority data used Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to eating-disorder clinic |
Outcome measures: GP demographics Practice characteristics |
Main results: Wide variation in referral rates All factors examined were significantly associated with higher referral rates Highest rate associated with proximity to clinic, female GPs, GPs having the MRCGP qualification, being UK qualified and offering full contraceptive services. Fundholding associated with lower rate of referral |
Reported associations between elements for logic model: Practitioner and practice characteristics associated with referral rates |
| Hyman 2001248 Country: Canada Study design: Cross-sectional Data collection method: Survey Aim: To examine physician characteristics associated with referral Detail of participants (number, any reported demographics): 64 family physicians serving the Caribbean community of Toronto; 40% female; age range 29–71 years (mean 42.16 years); 55.6% born in Canada |
Methods: Postal survey, scale for likelihood of referral relating to practice and patient factors Control: None Length of follow-up: NA Response and/or attrition rate: Context (from what/who to what/who): GP referral for mammography for Caribbean patients |
Outcome measures: Likelihood of referral |
Main results: The only variable that was significantly associated with likelihood of referral was that physicians who spent more time on patient education were more likely to refer Factors not associated with likelihood of referral: number of patients seen per week, number of regular patients, gender of physician Significant gender differences with regard to perceived barriers to referral. For male physicians intervention causes patient discomfort; patient refusals were significantly barriers to referral |
Reported associations between elements for logic model: Physician factors and predictors |
| Jiwa et al. 2008193 Country: UK and Australia Study design: Cross-sectional Data collection method: Postal survey including vignettes Aim: To understand how practitioners determine which patients warrant referral Detail of participants (number, any reported demographics): n = 260; equal gender distribution; majority older than 40 years; most in practice longer than 5 years |
Method: Vignettes of patients with colorectal symptoms, six clinical details with two possible variations; therefore, 64 possible combinations. Each respondent presented with nine vignettes. Based on features of colorectal cancer as in national guidelines. 64 vignettes high-risk cancer patients, others not Control: None Length of follow-up: NA Response and/or attrition rate: 52% Context (from what/who to what/who): GP to specialist for lower bowel symptoms |
Outcome measures: Demographics Decision to refer patient on appropriate pathway (urgent or routine) |
Main results: Appropriate referral pathway selected correctly for 56.2% of cases. Of those with high-risk symptoms 58% were referred urgently 31.3% of the variability within the urgent referral decisions was explained by clinical characteristics (age of patient, gender, rectal bleeding, change in bowel habit, anaemia, weight loss), the age of the GP and location of the practice Having weight loss as a symptom was influential on referral decision (this may be when disease is at a later stage and is not a helpful basis on which to refer patients) |
Reported associations between elements for logic model: Application of guidelines is moderate or negated by influence of characteristics of patients |
| Jiwa et al. 2009241 Country: UK Study design: Cross-sectional Data collection method: Analysis of referral letters Aim: To assess whether or not the quality of a referral letter is important in a patient’s journey Detail of participants (number, any reported demographics): 207 consecutive letters to gastroenterologists. Patients diagnosed with histological lesions and no histological lesion, patients who had a diagnosis unknown, patients who failed to attend |
Methods: Quality score given to each letter based on informational content Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Gastroenterologists |
Outcome measures: Clinical categories Ability to triage from the letter |
Main results: The cases that could be triaged from the letter were those where the letter contained more information (mean 66.38 vs. 49.86, mean difference 16.95%, 95% CI 1.3 to 31.7; p < 0.001). More information was helpful when deciding which patients to triage first |
Reported associations between elements for logic model: Referral letter content and triaging |
| Johnson 2011245 Country: Australia Study design: Cross-sectional Data collection method: Postal survey Aim: To examine factors influencing referral to specialist palliative care Detail of participants (number, any reported demographics): 469 GP; details reported as compared with national average rather than giving details of sample |
Methods: Self-report survey mailed to a stratified random sample of 1680 Australian GPs Control: None Length of follow-up: NA Response and/or attrition rate: 31% Context (from what/who to what/who): GP to palliative care |
Outcome measures: Referral rate GP characteristics Patient characteristics |
Main results: Six predictors of higher referral – Practice location (New South Wales vs. other states p = 0.010) Agree primary care provider or primary care inpatient consultative service available (p = 0.001) Agree needs of family better met (p = 0.018) Very satisfied level of satisfaction with service (p < 0.001) Reasons for not referring – only absence of symptoms significant (p = 0.025) |
Reported associations between elements for logic model: Predictors of referral primarily disease related rather than for psychological and emotional concerns |
| Johnson 2011289 Country: Australia Study design: Qualitative Data collection method: Semistructured interviews Aim: To describe doctors’ perceptions of barriers to referring patients for specialist palliative care, and to identify triggers and facilitators for referral Detail of participants (number, any reported demographics): n = 40, age mean 47 years (range 30–60 years) |
Method: Forty semistructured telephone interviews were conducted with doctors involved in the care of people with advanced cancer from a variety of settings Control: NA Length of follow-up: NA Response and/or attrition rate: 40% response rate Context (from what/who to what/who): Oncology |
Outcome measures: Practitioner views |
Main results: Six themes were identified: disease and treatment; psychosocial; communication and interpersonal issues; health services issues; timing; and health professionals’ skills. All doctors considered the presence and complexity of physical symptoms, stage of the disease and treatment orientation as important in decisions to refer for specialist palliative care. Less important were the psychosocial well-being and cultural characteristics of the person with cancer and their family. Factors reportedly affecting referral and access included health professionals’ ability to communicate openly and honestly about disease progression, availability and location of specialist palliative care resources and doctors’ expertise. Divergent views were expressed about appropriate timing for access. The predominant view that specialist palliative care is for management of physical symptoms may result in non-referral of those who have complex problems without physical symptoms |
Reported associations between elements for logic model: Patient/disease characteristics and referral Less important psychosocial factors Doctor–patient relationship and referral Doctor’s expertise Availability of resources |
| Johnson et al. 2008271 Country: USA Study design: Cross-sectional Data collection method: Postal survey Aim: To examine physician knowledge and attitude towards audiology referral for elderly patients Detail of participants (number, any reported demographics): 95 surveys completed; 53.2% male; internists (72%), family physicians (16.7%), private practice (47.8%); 51% in practice for more than 15 years |
Methods: Cross-sectional postal survey Control: None Length of follow-up: NA Response and/or attrition rate: 13.7% Context (from what/who to what/who): Primary care providers to audiology/otolaryngology |
Outcome measures: GP knowledge |
Main results: Primary care providers were generally not conducting screening for hearing or balance, were not aware of patients self-report questionnaires or likely to screen in the future Referral to audiologist/otolaryngologists mainly when patients complained of having hearing or balance difficulties Participants reported that these problems were important but that they had little time and were not reimbursed for screening |
Reported associations between elements for logic model: Primary care providers’ knowledge/use of screening tools Patient request for referral |
| Jorgensen 2001181 Country: Denmark Study design: Cross-sectional Data collection method: Referral rate data and survey Aim: To describe referral rates and investigate possible predictors Detail of participants (number, any reported demographics): 38,231 referrals from 260 practices examined. 410 GPs, 30% in single-handed practices |
Methods: Mixed methods – analysis of health insurance data, and also questionnaire Control: None Length of follow-up: NA Response and/or attrition rate: 90% returned survey Context (from what/who to what/who): GP to physiotherapy |
Outcome measures: Referral rate GP demographics Patient demographics |
Main results: Referral rates varied 1.6% to 13.2% between practices Twice as many women as men referred Referral rates increase with patient age group up to 35 years and remain stable over this age Practice location, female GP practice and GPs having frequent contact with physiotherapist explained small variation in referral rates (6.7% to 9.2%), leaving the greatest majority of variation unexplained |
Reported associations between elements for logic model: Lack of explanation of demographic factors in referral rate differences |
| Kasje 2004191 Country: the Netherlands Study design: Cross-sectional Data collection method: Structured survey Aim: To compare the willingness of hospital specialists and GPs to use joint treatment guidelines, and to determine the most relevant barriers and facilitators Detail of participants (number, any reported demographics): One hundred and ninety-seven GPs and 34 general internists and cardiologists from the north of the Netherlands |
Method: Structured survey with GPs, general internists and cardiologists Control: None Length of follow-up: None Response and/or attrition rate: More than half of all cardiologists (16 out of 24) and internists (18 out of 36), and three-quarters of all GPs (197 out of 261) in the region completed the general part of the questionnaire Context (from what/who to what/who): GP/specialist referral using guidelines |
Outcome measures: Practitioner views |
Main results: Most hospital specialists relied for their prescribing on international guidelines and agreements within their own department, whereas GPs relied more on national and regional guidelines GPs were more supportive than specialists of the initiative to develop joint treatment guidelines, although both groups had concerns regarding the development process An important barrier for specialists was that they did not perceive a need for these guidelines. As enabling factors, physicians stated that these joint guidelines can lead to harmonisation between specialists and GPs, and that they can be useful as an educational tool |
Reported associations between elements for logic model: Specialists are less ready to adopt joint treatment guidelines than GPs, indicating the need for a different approach to implement such guidelines in the two sectors Different use of guidelines GP and specialist |
| Kier 2012294 Country: UK Study design: Cross-sectional Data collection method: Survey Aim: To investigate the use and compliance with guidelines Detail of participants (number, any reported demographics): Two-thirds of respondents male; 79% over 40 years; 62% had less than 20 years in practice |
Methods: Survey Control: None Length of follow-up: NA Response and/or attrition rate: 50% Context (from what/who to what/who): GP to spinal manipulative therapy (SMT) |
Outcome measures: Demographics Referral |
Main results: 72% had referred patients for SMT; a further 13% would consider referring 21% who had never referred patients neither had nor would consider it Study concludes that GPs comply with guidelines on back pain and SMT as a care option, with a minor group that do not comply |
Reported associations between elements for logic model: Compliance with guidelines |
| Kim 200998 Country: USA Study design: Survey Data collection method: Web-based Aim: To survey primary care providers to assess the impact of electronic referrals on workflow and clinical care Detail of participants (number, any reported demographics): n = 298 |
Method: 18-item, web-based questionnaire to all 368 primary care providers who had the option of referring to San Francisco General Hospital Asked participants to rate time spent submitting a referral, guidance of work-up, wait times and change in overall clinical care compared with prior referral methods using five-point Likert scales Length of follow-up: None Response and/or attrition rate: Two hundred and ninety-eight primary care providers (81.0%) from 24 clinics participated Context (from what/who to what/who): Primary care to clinical care |
Outcome measures: Practitioner views |
Main results: Over half (55.4%) worked at hospital-based clinics, 27.9% at county-funded community clinics and 17.1% at non-county-funded community clinics. Most (71.9%) reported that electronic referrals had improved overall clinical care. Providers from non-county-funded clinics (AOR 0.40, 95% CI 0.14 to 0.79) and those who spent ≥ 6 minutes submitting an electronic referral (AOR 0.33, 95% CI 0.18 to 0.61) were significantly less likely than other participants to report that electronic referrals had improved clinical care |
Reported associations between elements for logic model: Primary care providers felt electronic referrals improved health-care access and quality; those who reported a negative impact on workflow were less likely to agree |
| Kinchen 2004238 Country: USA Study design: Cross-sectional Data collection method: Survey Aim: To determine the importance of factors in primary care physicians’ choice of specialist when referring patients and to compare importance ratings by physicians’ race and sex Detail of participants (number, any reported demographics): 1252 primary care physicians |
Method: Surveyed a stratified national sample of 1252 primary care physicians serving adults to include equal numbers of black women, white women, black men and white men. Assessed the percentage of physicians rating each of 17 items to be of major importance in choosing a specialist and compared importance ratings by physicians’ race and sex Control: NA Length of follow-up: NA Response and/or attrition rate: The response rate was 59.1% Context (from what/who to what/who): GP to specialist |
Outcome measures: Referral |
Main results: Medical skill, appointment timeliness, insurance coverage, previous experience with the specialist, quality of specialist communication, specialist efforts to return patient to primary physician for care and the likelihood of good patient–specialist rapport were of major importance to most respondents Compared with black physicians, white physicians were more likely to rate previous experience with the specialist (65% vs. 55%, p = 0.05) and board certification (41% vs. 29%, p < 0.05) to be of major importance White physicians were somewhat less likely than black physicians (17% vs. 26%, p = 0.06) to rate patient convenience to be of major importance Compared with male physicians, female physicians were more likely to rate the patient’s insurance status to be of major importance (60% vs. 44%, p < 0.01) |
Reported associations between elements for logic model: The importance of patient convenience, previous experience with the specialist, specialist board certification and insurance coverage accepted by specialist varied by physicians’ race and sex |
| Kisely 2002185 Country: Australia Study design: Cross-sectional Data collection method: Questionnaire and interviews Aim: To evaluate a primary care partnership with mental health including understanding to streamline referrals, a consultation-liaison service and a Balint group (not described) Detail of participants (number, any reported demographics): 74 GPs |
Methods: 34 (45%) of GPs returned questionnaires and 46 (62%) participated in interviews Balint group membership provides a space to think about those encounters which leave professionals drained, puzzled or stuck, and through discussion about the relationship, the possibility of finding new ways forward with the patient (see http://balint.co.uk/) Control: NA Length of follow-up: NA Response and/or attrition rate: 45% Context (from what/who to what/who): GP referral to mental health services |
Outcome measures: Practitioner views |
Main results: 26 (80%) found the duty officer useful as a point of first contact liaison contact 19 (60%) used referral forms Most (80%) were aware of the consultation-liaison service and had either used it or been in the Balint group |
Reported associations between elements for logic model: Use of referral forms only 60% |
| Knight 2003177 Country: UK Study design: Qualitative Data collection method: Interviews Aim: To identify factors considered by GPs in making referral decisions Detail of participants (number, any reported demographics): Nine GPs from two surgeries in Northamptonshire – eight male, one female |
Methods: Qualitative interviews Rating scale also used within the interviews relating to service and patient factors Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to mental health services (not specified includes/predominantly community services?) |
Outcome measures: Content of interviews (content analysis) Importance of factors |
Main results: 13 patient-related factors identified influencing referral decisions: Type of problem, severity and chronicity, patient wishes and preferences, patient progress and response to treatment, patient needs, control of symptoms, social support, exclusion of physical pathology/reassurance, patient situation, severity of distress, previous treatment/assessment, effect on family and motivation Three service-related factors: Availability/waiting lists, appropriateness for in-house counselling service, quality of the service, previous experience with service, liaison/feedback, therapist interests and training, setting being familiar to patient 12 doctor-related factors: GP time/availability, doctor–patient relationship, experience, expertise, success of GP treatment, needing assessment/advice, difficulties with a particular patient, workload/emotional involvement, safeguarding career/making mistakes, GP preference for a particular problem/competence, obligation, putting problem in another person’s lap Highest priority given to patient-related factors, all of these received at least moderately high ratings Differences between GPs apparent in terms of preferences, inclination and confidence to treat psychological problems. Different referral rates among participants, low referrer may take more responsibility for patients/more interest in treating psychological problems; however, high interest could also lead to high referral as better identification of problems |
Reported associations between elements for logic model: Patient-, service- and clinician-related factors |
| Kvaerner 2007168 Country: Norway Study design: Cross-sectional Data collection method: Questionnaire Aim: To estimate the proportion of children with otitis media referred from primary to specialist care, study variation in referral pattern and factors that influence GP behaviour Detail of participants (number, any reported demographics): All Norwegian GPs in 2004 (n = 1633) |
Methods: Questionnaire Control: NA Length of follow-up: NA Response and/or attrition rate: 48% (n = 1633) Context (from what/who to what/who): GP to otolaryngologist/ENT department |
Outcome measures: Referral routines for otitis media at first visit and at follow-up |
Main results: Mean referral for otitis media was 22%, most commonly at follow-up visit. Twenty-seven per cent of children with otitis media were sent to ENT departments and 73% to practising otolaryngologists. Variation in referral pattern among GPs was moderate. GPs with specialty in general medicine had 6% fewer referrals. Separate analysis on referral to practising otolaryngologists showed that GP work load and availability to practising specialists increased referral, whereas availability to hospital services reduced the probability |
Reported associations between elements for logic model: Completed specialty in general medicine reduces referrals GP workload and availability of specialist influence referral behaviour |
| Lakha et al. 2011179 Country: Canada Study design: Cross-sectional Data collection method: Survey Aim: To examine factors associated with family physician referral Detail of participants (number, any reported demographics): 47 family physicians; mean age 48.6 years; most worked in greater Toronto area, 37% worked outside the metropolitan area. Females 45%; time since graduation mean 22.4 years |
Methods: A questionnaire-based survey Control: None Length of follow-up: NA Response and/or attrition rate: 32% Context (from what/who to what/who): Family physician to tertiary pain clinic |
Outcome measures: Reasons for referral Physician demographics Number of patients seen |
Main results: Three most cited reasons for referral were requests for nerve blocks or other injections, desire for the expertise of the programme and concerns regarding opiods Three most prevalent barriers were long waiting lists, patient preference for other treatments and distance from clinic Trend but not significant – the more chronic-pain patients a physician saw the less he or she tended to refer them to pain clinics. Also not significant were family physician age, sex, ethnicity, length of time in practice |
Reported associations between elements for logic model: No relationship between family physician demographics and referral |
| Lambert 2001175 Country: UK Study design: Cross-sectional Data collection method: Postal survey Aim: To examine the role of GPs in patients with epilepsy Detail of participants (number, any reported demographics): 312 GPs Bristol, 10 from practices with epilepsy nurse specialists |
Methods: Questionnaire Control: None Length of follow-up: NA Response and/or attrition rate: 67% Context (from what/who to what/who): GP to specialist epilepsy services |
Outcome measures: Views and reported practice |
Main results: 49% at least sometimes treated new patients before they were referred, 12% never treated patients before referral 80% of GPs with a nurse specialist at least sometimes treated before referring 16% of GPs reported auditing their management of patients with epilepsy. 60% of those with an epilepsy nurse audited practice 64% replied that they would welcome teaching on epilepsy Only 16.4% felt that joint clinics would be a good idea and 58.6% felt that there was no need for this. However, 66.4% believed that co-operation cards would be useful. 20.8% replied that they would be unnecessary and preferred communication via letters telephone or e-mail. 20% of GPs with specialist nurse wanted joint clinics; however, 80% thought co-operation cards would be useful Note: co-operation cards are described as an overview of the patient – would contain details of epilepsy and seizure types including classification, also medical history, epilepsy history, medications, checklist of counselling regarding driving, employment, etc. |
Reported associations between elements for logic model Low number of GPs viewing joint clinics as necessary Impact of having epilepsy nurse in practice |
| Lewis 2000260 Country: USA Study design: Qualitative Data collection method: Telephone interview Aim: To investigate patient preference for care by generalists and specialists Detail of participants (number, any reported demographics): 314 patients who had visited their primary care provider in the previous 2 years from 10 sites randomly generated from list. Respondents described as generally well-educated, middle-class, white women. 40% good to excellent health yet many reported frequent visits and hospitalisations |
Method: Qualitative telephone interviews Control: None Length of follow-up: None Response and/or attrition rate: NA Context (from what/who to what/who): Primary care provider to specialist |
Outcome measures: Views |
Main results: Although participants preferred continuity of care from a primary care provider, over half noted that for a new problem they preferred to see a doctor who specialises in that problem Participants valued the freedom to choose their doctor and have unencumbered access to specialists 90% would avoid insurance plans that limited their choice of generalist or specialist Patients asked who they would prefer to see for 15 conditions. Only four for which one-fifth or more indicated a preference to receive care from a specialist – prostate, initiation of birth control, changes in a mole and ingrown toenail |
Reported associations between elements for logic model: Applicability to UK? Importance of patient choice of doctor |
| Little 2004251 Country: UK Study design: Cross-sectional as part of wider RCT Data collection method: Questionnaires Aim: To explore the impact of patient pressure on doctors’ management of patients Detail of participants (number, any reported demographics): 847 patients aged 16–80 years. Five general practices, 30 GPs. Southampton area |
Methods: Patients approached while waiting for GP appointment. Completed questionnaire before appointment and after on why appointment and what outcome they hoped for/satisfaction with outcome. Also completed HADS. GPs completed questions following appointment on action taken and whether or not these were needed and pressure they felt from patient Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for patients with depression |
Outcome measures: Perceptions of medical need Perception of patient pressure to be referred Numbers referred or not referred |
Main results: Doctors’ perception of medical need strongest factor determining behaviour during consultation Doctors reported no or only slight need for referral in 22% of those examined Doctors’ perception of moderate or definite patient pressure was a predictor of doctors’ referral behaviour. It was a stronger predictor than patient expectations/pressure Perceived slight patient pressure to be referred 19% referred 5% not referred (OR 8.99, 95% CI 4.91 to 16.46; p = 0.994), perceived moderate or definite pressure 44% referred 1% not referred (OR 125.3, 95% CI 51.3 to 306.5; p = 0.005) Patient wish to be referred slight 16% referred 8% not referred (OR 3.34, 95% CI 1.88 to 5.93, p = 0.796), patient pressure moderate or definite 28% referred 5% not referred (OR 8.51, 95% CI 4.97 to 14.6; p = 0.028) |
Reported associations between elements for logic model: Perception of medical need is strongest predictor of referral. However, perceived pressure from patient is a significant factor |
| Love 2005302 Country: New Zealand Study design: Cross-sectional Data collection method: Referral data Aim: To describe patterns of variation in referral among GPs, and to establish whether or not variability among practitioners within a geographic area is associated with high levels of utilisation in an area Detail of participants (number, any reported demographics): Claims managed by 2679 GPs |
Method: Multilevel analysis of routinely collected primary care data Control: NA Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to physiotherapy |
Outcome measures: Referral to physiotherapy, specialist assessment, radiology, and approval of earnings-related compensation |
Main results: The pattern of observed variability is not consistent for different referral activities: groups of GPs within one area may practise consistently in referral for one outcome, but be highly variable for another, while practitioners in other areas can show the reverse pattern The degree of variability among GPs within geographic areas was not significantly correlated at the 95% level with the absolute level of referral to any of the referral options |
Reported associations between elements for logic model: No consistent patterns |
| Malcolm 2008299 Country: Canada Study design: Cross-sectional Data collection method: Survey (in person, by fax and by mail) Aim: Understanding primary care physicians’ perceptions of barriers to specialist diabetes care Detail of participants (number, any reported demographics): 445 primary care physicians included. 177 primary care physicians completed the survey |
Method: The survey was developed using the following steps – (1) a multidisciplinary committee was formed to develop the initial concepts to be covered in the survey; (2) an extensive review of the literature was performed; and (3) the list of tools and barriers to be included in the survey was determined Consensus was used to develop the list of tools and barriers, as it was found that the body of literature pertaining to tools for transition was small. The scores for seven preselected barriers to specialist care were recorded Control: None Length of follow-up: None Response and/or attrition rate: 40% response rate Context (from what/who to what/who): Primary care to specialist diabetes care |
Outcome measures: Barriers to referral (predefined) |
Main results: There was a fairly high degree of spread in these scores for the seven preselected barriers (2 to 6) The results indicate that primary care physicians are conversant with guidelines and evidence but have problems implementing guidelines for three main reasons: lack of patient adherence (6), lack of access to a multidisciplinary team of care providers (5) and lack of patient finances (5) The other barriers were less important: Lack of time (4) Lack of clarity of guidelines (2) Lack of awareness of evidence (2) |
Reported associations between elements for logic model: Barriers to referral |
| Massey 2004236 Country: UK Study design: Cross-sectional Data collection method: Postal survey Aim: To examine views of access to a physiotherapy service Detail of participants (number, any reported demographics): GPs in Blackpool area; 65% of 50 practices responded |
Method: Service had been in operation for 5 years Control: NA Length of follow-up: NA Response and/or attrition rate: 65% return Context (from what/who to what/who): GP to specialist physiotherapy service for patients with incontinence/pelvic floor dysfunction (previously service accessed via GP referral to consultant) |
Outcome measures: Views of the GP direct access physiotherapy continence service |
Main results: 55% stated that they were aware of the direct access of patients in the service 72% of those aware had referred to the service 94% were satisfied with the service acting as a triage for patients prior to their referral to a relevant consultant 94% stated that specific referral criteria would be useful |
Reported associations between elements for logic model: GP lack of knowledge of service; those not aware would refer in the future |
| McBride 2010287 Country: UK Study design: Cohort study Data collection method: Patient data from database Aim: To explore variation in referral patterns Detail of participants (number, any reported demographics): 5492 patients with post-menopausal bleeding; 23,121 with hip pain; 101,212 with dyspepsia. 326 general practices |
Methods: Health improvement network database with 15 years of data. Referrals solely for non-diagnostic imaging were excluded. Patients with post-menopausal bleeding prescribed hormone-replacement therapy were excluded. Referral within 2 weeks of GP consultation counted Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist |
Outcome measures: Record of referral to a specialist |
Main results: Reduced odds of referral for post-menopausal bleeding in those patients over 75 years, increased comorbidity reduced likelihood of referral Hip pain: gender, age and level of deprivation impacted on likelihood of referral Dyspepsia: gradient in referral by level of deprivation, referral rates higher over 55 years than under 55 years. Referral least likely in oldest and youngest |
Reported associations between elements for logic model: Patient demographic factors and likelihood of referral |
| McKenna 2005225 Country: USA Study design: Cross-sectional Data collection method: Postal survey Aim: To explore the impact of knowledge and demographic variables on referral Detail of participants (number, any reported demographics): 460 medicine and family practice physicians. Average age 48 years; 69% male; 71% in private practice. 87% of their time spent in primary care |
Methods: 13 case scenarios used 10 appropriate referrals and four inappropriate. Asked to rate the likelihood of referral. Asked to identify skills of specialists from list Control: None Length of follow-up: NA Response and/or attrition rate: 46% Context (from what/who to what/who): Family practice/internal medicine physicians to physical medicine and rehabilitation |
Outcome measures: Referral intention Self-reported number of referrals Knowledge of skills of physical and rehab specialists |
Main results: Females more likely to refer than males (p = 0.003) Physicians with greater understanding of the practice of the specialists were more likely to refer (p = 0.003) |
Reported associations between elements for logic model: Gender and referral Knowledge and referral |
| Mitchell 2012186 Country: Australia Study design: Cross-sectional Data collection method: Semistructured telephone interviews and an online survey Aim: To examine Australian private practice dietitians’ relationships with GPs and practice nurses as a factor that influences dietetic referrals Detail of participants (number, any reported demographics): n = 52 (interviews) and n = 90 (survey) with GPs (n = 11) and practice nurses (n = 12) from the ‘GP Access’ Division of General Practice |
Method: Mixed-methods approach. Semistructured telephone interviews and an online survey Control: NA Length of follow-up: March to July 2007 Response and/or attrition rate: Telephone interviews were conducted with 52 private practice dietitians (22% response rate); 18 dieticians from divisions providing a high number of chronic disease management consultations (30% response rate); and 37 providing a low number (20% response rate) Context (from what/who to what/who): GP referral to dieticians |
Outcome measures: Practitioner views |
Main results: GPs’ relationships with dieticians were believed to be the primary influencing factor on referral, provided by 81% of survey dieticians and 25% of interview dieticians. The most common means of initially forming relationships with GPs were face-to-face introduction (48%) and introductory letters (37%) Patient feedback via letter, fax or e-mail was the most popular method of maintaining relationships with GPs (77%). Meeting with GPs in person was believed to be the most effective activity in building relationships GPs and increasing referral rates (42%) Referral was made easier for GPs by providing paper (37%) or electronic (19%) referral forms and contact details (19%). The majority of GPs and dietitians believed that the ‘Allied Health Services Under Medicare’ made it easier to refer to a dietitian |
Reported associations between elements for logic model: Influence of referral relationship on referral Ease of referral and paper or electronic referral forms |
| Montgomery 2006180 Country: UK Study design: Qualitative Data collection method: Interviews Aim: To examine factors influencing GP referral Detail of participants (number, any reported demographics): 51 GPs, 25 urban and 26 rural; 43 male; average 50 years old; mean practice size of 2.3 partners. Median distance 15 miles from nearest dialysis centre |
Methods: Interviews included case scenarios depicting patients in varying stages of renal failure Control: None Length of follow-up: NA Response and/or attrition rate: 65% response rate Context (from what/who to what/who): GP to nephrologist or other specialist for end-stage renal failure |
Outcome measures: Referral patterns |
Main results: Neither GP characteristics nor patient characteristics affected referral pattern No patient age difference in referral patterns (χ2 = 0.73) Referral rates varied widely between cases 0–32 after first laboratory test was available to GP Referral rates did not differ by GP sex, practice size or experience with renal patients |
Reported associations between elements for logic model: Diverse management patterns not explained by patient or GP factors |
| Moore et al. 2000205 Country: USA Study design: Cross-sectional Data collection method: Survey Aim: To examine primary care provider referral decisions Detail of participants (number, any reported demographics): 504 primary care providers in state of Ohio |
Methods: 20-item survey Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Primary care provider to neurologist |
Outcome measures: Rate of referral Self-reported confidence with seizure patients |
Main results: Two groups emerged – high referrers and low referrers. Difference between the two groups was apparent in answers to questions regarding who initiated drug therapy, and their comfort level Minority of physicians rate themselves as comfortable with seizure patients; these tend to refer less than 50% of their patients. Those less comfortable refer more than 50% of patients |
Reported associations between elements for logic model: Physician comfort dealing with patients and referral |
| Morgan 2007210 Country: UK Study design: Qualitative Data collection method: Interviews Aim: Examines influences on GPs’ referral for headache in the absence of clinical indicators Detail of participants (number, any reported demographics): Semistructured interviews with 20 GPs. Eighteen urban and suburban general practices in the South Thames area, London |
Methods: Purposive sample comprising GPs with varying numbers of referrals for headache over a 12-month period Control: None Length of follow-up: 12-month study Response and/or attrition rate: Forty GPs were approached. Two refused to be interviewed, 13 had left the practice and five were unavailable owing to maternity leave, sick leave or annual leave Context (from what/who to what/who): GP referral for headache |
Outcome measures: Practitioner views |
Main results: All GPs reported observing patient anxiety and experiencing pressure for referral. Readiness to refer in response to pressure was influenced by characteristics of the consultation, including frequent attendance, communication problems and time constraints. GPs’ accounts showed variations in individual’s willingness or ‘resistance’ to refer, reflecting differences in clinical confidence in identifying risks of brain tumour, personal tolerance of uncertainty, views of patients’ ‘right’ to referral and perceptions of the therapeutic value of referral. A further source of variation was the local availability of services, including GPs with a specialist interest and charitably funded clinics |
Reported associations between elements for logic model: Link GP expertise and referral GP confidence and referral, tolerance of risk/anxiety Link patient pressure and referral Situational factors increasing readiness to refer include the local availability of sources of referral |
| Morsi 2012200 Country: USA Study design: Survey Data collection method: Web-based survey Aim: To characterise factors influencing PCP hospital referral choice Detail of participants (number, any reported demographics): 92 PCPs affiliated to three hospitals in Massachusetts |
Method: Web survey Control: None Length of follow-up: NA Response and/or attrition rate: 47% Context (from what/who to what/who): Primary care provider to hospital |
Outcome measures: Factors reported as important in referral decisions |
Main results: Factors considered important – Familiarity with the hospital (70%) Patient preference (62%) Arrangement with hospital (62%) Publicly available quality measures ‘not at all important’ to 42% No physicians reported ever using quality information to make a referral decision or discussing it with patients No physician factors were associated with awareness of publicly reported data |
Reported associations between elements for logic model: Quality of service offered by a provider not associated with referral decisions |
| Mulvaney 2005297 Country: UK Study design: Cross-sectional Data collection method: Survey Aim: To examine the relationship between private and NHS outpatient referral rates accounting for their association with deprivation Detail of participants (number, any reported demographics): 10 general practices |
Method: A prospective survey of GP referrals to private and NHS consultant-led services between 1 January and 31 December 2001 from 10 general practices in the Trent Focus Collaborative Research Network, UK. Patient referrals were aggregated to give private and NHS referral rates for each electoral ward in each practice Control: NA Length of follow-up: NA Response and/or attrition rate: NR Context (from what/who to what/who): GP to specialist |
Outcome measures: NHS referral Private referral Patient characteristics |
Main results: Of 17,137 referrals, 90.4% (15,495) were to the NHS and 9.6% (1642) were to the private sector Private referral rates were lower in patients from the most deprived fifth of wards compared with the least deprived fifth (rate ratio 0.25, 95% CI 0.15 to 0.41; p < 0.001), whereas NHS referral rates were slightly higher in patients in the most deprived fifth of wards (rate ratio 1.18, 95% CI 0.98 to 1.42; p = 0.08) both after age standardisation and adjustment for practice The NHS referral rate was significantly higher (rate ratio 1.40, 95% CI 1.15 to 1.71; p = 0.001) in wards with private referral rates in the top fifth compared with the bottom fifth after adjustment for deprivation and practice |
Reported associations between elements for logic model: Increased private health-care activity does not reduce the demand for NHS care |
| Musila 2011255 Country: UK Study design: Cross-sectional Data collection method: Guideline development group rated appropriateness Aim: An innovative approach to develop a referral guideline for patients with chronic knee pain that explicitly incorporates patients’ preferences Detail of participants (number, any reported demographics): n = 12 |
Method: A guideline development group of 12 members including patients, GPs, orthopaedic surgeons and other health-care professionals used formal consensus development informed by systematic evidence reviews. They rated the appropriateness of referral for 108 case scenarios describing patients according to symptom severity, age, body mass, comorbidity and referral preference. Appropriateness was expressed on scale from 1 (‘strongly disagree’) to 9 (‘strongly agree’) Context (from what/who to what/who): GP to specialist for patients with chronic knee pain |
Outcome measures: Appropriate referral |
Main results: Ratings of referral appropriateness were strongly influenced by symptom severity and patients’ referral preferences. The influence of other patient characteristics was small. There was consensus that patients with severe knee symptoms who want to be referred should be referred and that patient with moderate or mild symptoms and strong preference against referral should not be referred. Referral preference had a greater impact on the ratings of referral appropriateness when symptoms were moderate or severe than when symptoms were mild | Reported associations between elements for logic model: Referral decisions for patients with osteoarthritis of the knee should be guided only by symptom severity and patients’ referral preferences |
| Naccarella et al. 2008169 Country: Australia Study design: Survey Data collection method: Survey of project officers who had carried out demand management projects Detail of participants (number, any reported demographics): Data available from 89 projects (81%) |
Methods: Questionnaire Control: None Length of follow-up: None Response and/or attrition rate: Context (from what/who to what/who): GPs to mental health care |
Outcome measures: Types of demand management used Views |
Main results: Informing and training GPs was most popular demand strategy used followed by putting in place systems and/or administrative procedures Monitoring and limiting referrals was most commonly reported as the most useful strategy (29%) followed by putting in place systems and/or administrative procedures (24%) |
Reported associations between elements for logic model: Types of demand management Views regarding which most successful – system changes and referral |
| Nandy 2001204 Country: UK Study design: Qualitative Data collection method: Interviews Aim: To analyse GP decision-making processes with patients with minor mental illness Detail of participants (number, any reported demographics): 23 GPs in London and Essex. Mean age 44 years, eight women, 20 principals |
Methods: Grounded theory analysis and framework. Recording quality too poor to analyse for two interviews Control: None Length of follow-up: NA Response and/or attrition rate: 34 approached Context (from what/who to what/who): GP to mental health specialist for minor mental health issues |
Outcome measures: Themes |
Main results: Two strategies identified – containment and conduit. Containment was adopted by GPs who saw their role as preventing burden on other agencies and thus tended not to refer. GPs adopting a conduit strategy say their role as diagnostic and then triage with patients best managed by others. Many GPs used both strategies; majority initially see patients themselves then refer sometimes. Saw themselves as doing both but leaning towards one or the other Reasons for referral – lack of progress, poor rapport with patient. Containment enhanced by having an interest in mental health and having confidence in dealing with mental health Referrals of two types – proactive ‘referrals to’ (patient desire to be referred or other staff have better skills) and reactive ‘referrals away’ (failure of GP management or lack of time). Referrals away predominated Emotive as well as rational responses informed GP decisions. Feelings of frustration or irritation. Referrals to tended to be thought through whereas referrals away could be more emotional or instinctive |
Reported associations between elements for logic model: GP strategies and emotions and referral |
| Navaneethan 2010278 Country: USA Study design: Cross-sectional Data collection method: Retrospective case review, and survey Aim: To assess referral factors and compare perceptions with factors Detail of participants (number, any reported demographics): 816 patient records reviewed; 400 primary care providers surveyed |
Methods: Retrospective case review, and survey Control: None Length of follow-up: NA Response and/or attrition rate: 25% of primary care providers completed the survey Context (from what/who to what/who): Primary care provider referral of patients with chronic kidney disease to nephrologist |
Outcome measures: Factors predicting referral |
Main results: Non-referred patients were likely to be older (over 65 years p < 0.05, OR 3.5, 95% CI 2.3 to 5.2) Non-referred patients likely to be female (OR 1.4, 95% CI 1.0 to 2.0) Non-referred patients likely to be of non-white race (OR 2.6, 96% CI 1.5 to 4.5) Patients with comorbidities also less likely to be referred 62% of primary care providers unfamiliar with referral guidelines |
Reported associations between elements for logic model: Patients demographics as predictor of referral Physician low knowledge of guidelines |
| O’Byrne et al. 2010243 Country: UK Study design: Cross-sectional Data collection method: Analysis of GP letter information Detail of participants (number, any reported demographics): Letters for 50 new patient referrals reviewed by two consultants |
Methods: Consultant read the referral letter and predicted which diagnostic investigations they would order based on the letter content. Then saw patient and compared expectations with actual Control: None Length of follow-up: None Response and/or attrition rate: Context (from what/who to what/who): GP to respiratory consultants |
Outcome measures: Number of changes in recommended diagnostic investigations |
Main results: 72% of patients required an alteration to the diagnostic investigations thought to be necessary Specific content omitted related to allergies and comorbidities The number of tests ordered on basis of referral letter was substantially fewer than those that were thought necessary at the consultation. Tests planned were less diverse and tended to lack specificity than those actually recommended by specialist Review of literature on missing content of referral letters Useful references for issues with direct access systems, standardised referrals/guidelines/electronic systems |
Reported associations between elements for logic model: Referral information is insufficient to provide a reliable basis for selection of tests prior to consultation Suggests that a straight-to-test system based on referral information would result in redundant diagnostic testing and repeated hospital attendance for correct tests to be carried out |
| O’Neill 2005170 Country: USA Study design: Cross-sectional Data collection method: Survey Aim: To explain variation in physicians’ practice patterns in terms of physician personal characteristics, practice setting, patient population, and managed care involvement Detail of participants (number, any reported demographics): 2455 primary care physicians. The average age of physicians was 47 years, with an average of 15 years in practice |
Method: Data on 2455 primary care physicians were derived from the Community Tracking Study Physician Survey (1996–7). Factor scores were determined based on responses to three clinical scenarios that represent discretionary medical decisions. These scenarios include a specialist referral for benign prostatic hyperplasia, prescription drugs for elevated cholesterol and an office visit for vaginal discharge Control: NA Length of follow-up: NA Response and/or attrition rate: The Community Tracking Survey had a response rate of 65% Context (from what/who to what/who): GP to specialist |
Outcome measures: Physician personal characteristics, practice setting, patient population, and managed care involvement |
Main results: Physician age, being a foreign medical school graduate, being a solo practitioner and having a larger proportion of Medicaid patients were all associated with higher factor scores, a greater likelihood of ordering a service Being board certified was associated with lower factor scores Managed care involvement was not a significant predictor of factor scores All of the predictor variables were significantly correlated with factor score (p < 0.01) except gender, percentage of revenue from Medicare and percentage of capitated revenue. Board certified was negatively correlated with age (–0.296) and foreign medical school graduate (–0.268). Percentage of patient revenue from managed care was negatively correlated with factor score (–0.052), age (–0.159), and solo practice (–0.237) and positively correlated with board certified (0.101), female gender (0.146), and institutional setting (0.277). Percentage of Medicaid revenue was positively associated with foreign medical school graduate (0.133) and institutional setting (0.219) and negatively associated with board certified (–0.081) |
Reported associations between elements for logic model: Unclear |
| Olson et al. 2012206 Country: Canada Study design: Cross-sectional Data collection method: Postal survey Aim: To assess the relationship between family physicians knowledge of radiotherapy and referral Detail of participants (number, any reported demographics): |
Methods: Postal survey Control: None Length of follow-up: NA Response and/or attrition rate: 33% Context (from what/who to what/who): Family physician to palliative radiotherapy |
Outcome measures: Physician self-reported knowledge |
Main results: Strong relationship between family physician referral and self-assessed or tested knowledge (p < 0.001 and p < 0.010) |
Reported associations between elements for logic model: Physician knowledge level and referral |
| Pfeiffer 2011138 Country: USA Study design: Retrospective analysis of clinic data Data collection method: Analysis of hospital patient data from locations with primary mental health services and those without over a 1-year period Aim: To determine whether or not implementation of primary care mental health services is associated with differences in specialty mental health clinic use Detail of participants (number, any reported demographics): 49,957 primary care patients with new visits to specialty mental health clinics. Mean age 55.7 years; male 93%. 118 primary care facilities offering specialist mental health services and 142 without |
Intervention: Veterans Health Administration, primary care mental health services providing collocated collaborative mental health specialists and managers screening and managing common mental health conditions (depression, alcohol misuse, post-traumatic stress disorder) Control: Primary care facilities not having an integrated mental health service Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Primary care services to specialty mental health care |
Outcome measures: New visits to specialty mental health clinics Mental health diagnosis Illness severity |
Main results: Initiation of treatment at specialty mental health clinic did not differ between primary care with mental health facilities and those without the service (5.6% vs. 5.8%) Attendance at a primary care service for mental health was not a predictor of total number of specialist mental health clinic visits Author conclusion: provision of primary care mental health service not associated with differences in new use of specialty mental health services or diagnoses received |
Reported associations between elements for logic model: No impact on referrals from primary care mental health service to specialist service |
| Philichi 2010202 Country: USA Study design: Cross-sectional Data collection method: Postal survey Aim: To assess treatment strategies and referral decisions Detail of participants (number, any reported demographics): 237 primary care providers. 81% paediatricians, 19% nurse practitioners |
Methods: Cross-sectional postal survey Control: None Length of follow-up: NA Response and/or attrition rate: 38% Context (from what/who to what/who): GP to paediatric gastroenterology |
Outcome measures: Clinical diagnosis Reasons for referral No practitioner demographics collected |
Main results: Most frequently identified reason for referral was patient unresponsiveness to treatment (71%) Second most frequent: parents want a second opinion (15%) Third: to rule out organic cause (9%) Fourth: management too time-consuming (5%) |
Reported associations between elements for logic model: Patient factors and referral |
| Pomeroy 2010176 Country: Australia Study design: Cross-sectional plus qualitative Data collection method: Interviews and postal survey Aim: To describe GP decision-making processes Detail of participants (number, any reported demographics): 30 GPs in Victoria interviewed; 14 male; 2–5 GPs in practice 15, 6–11 GPs in practice 10. 248 GPs surveyed |
Methods: Mixed methods – cross-sectional postal survey and qualitative interviews Control: None Length of follow-up: NA Response and/or attrition rate: 30% Context (from what/who to what/who): GP to dietitian |
Outcome measures: Themes |
Main results: Synthesising management information – number and complexity of health problems impacted on whether nutrition intervention was prioritised as high or delayed until later. Reasoning seemed based on doctor’s previous experiences. Presence of multiple and complex health problems influenced referral for two-thirds of GPs Patient age not viewed as a causal factor for referral. No association between doctor’s age and referral Forecasting outcomes – capacity of patient to implement changes, potential effect Planning management – patient choice of treatment, patient willingness to attend Actioning the referral – GP knowledge of local service, matching patient with service, access, transport, interpreter services Concept model developed – doctor’s clinical knowledge – patient’s nutrition history – patient’s clinical profile (risks) – doctor’s clinical experience and intuition Note: paper does not report the results of the survey |
Reported associations between elements for logic model: Complexity of issues and referral Age and referral Patient choice GP previous experience and knowledge of service |
| Pryor and Knowles 2001215 Country: Australia Study design: Cross-sectional Data collection method: Survey Aim: To explore whether GP attitudes to psychologists impacts on referral Detail of participants (number, any reported demographics): 105 GPs in Victoria, Australia; 69% female; 96% had referred to a psychologist |
Methods: 33-item questionnaire including questions regarding competence, how likely were to refer, reasons would not refer Control: None Length of follow-up: NA Response and/or attrition rate: 66% Context (from what/who to what/who): GP to psychology |
Outcome measures: Perceived competency Referral rate GP demographics |
Main results: High-referring GPs rated psychologist professional competency and value to the community higher than low-referring GPs Female GPs referred more frequently than male GPs. GP age predictor of number of referrals (36–55-year band higher) No significant difference metropolitan and regional GPs Perception of professional competency not a barrier to referral, factors of cost act as barrier |
Reported associations between elements for logic model: Younger female GPs with positive attitude to psychology referred more |
| Ramanathan et al. 2011188 Country: Australia Study design: Cross-sectional Data collection method: Postal survey Aim: To describe GP referral patterns and factors Detail of participants (number, any reported demographics): 1402 GPs – 771 metropolitan and 631 rural or remote |
Methods: Self-completed or online questionnaire including 128 case vignettes of a patient with gynaecological symptoms. Each GP received 12 vignettes, Telephone interviews had been used to add to the clinical guidelines and develop clinical variables Control: None Length of follow-up: NA Response and/or attrition rate: 45.5% Context (from what/who to what/who): GP to gynaecologist or gynaecological ontologist |
Outcome measures: Referral decision Diagnosis of probability of patient having cancer Demographic data Practice and service data |
Main results: Metropolitan GPs more likely to refer than rural GPs for endometrial cancer (68.4% vs. 61.1%) Best predictors of referral were age of patients, results of ultrasound and duration of symptoms Practitioners with more than 15 years of experience were more likely to refer vignettes of women with ovarian cancer Patient factors were better predictors of referral behaviour than the characteristics of GPs. Age of patient and duration or symptoms were important determinants of referral Greater variation in referral practice for endometrial cancer for which there are no Australian guidelines (68% of vignettes with high probability of cancer were referred compared with 83% for ovarian cancer and 80% for cervical cancer for which guidelines are available) |
Reported associations between elements for logic model: Location of GP and referral Availability of guidelines and referral practice |
| Ramchandiani 2002247 Country: UK Study design: Cross-sectional Data collection method: Postal survey Aim: To seek views of pooled waiting lists Detail of participants (number, any reported demographics): 50 GPs in Birmingham. 776 consultant opthalmologists. 85 patients (55 female, mean age 75.7 years) |
Methods: Cross-sectional postal survey Control: None Length of follow-up: NA Response and/or attrition rate: 64% Context (from what/who to what/who): GP to ophthalmology for patients with cataracts |
Outcome measures: Views |
Main results: 7.5% of consultants reported using pooled lists; 73% patients would move between consultants if wait became excessive 30% of consultants in favour of pooled lists and 67% against, 3% do not know. Pooled lists described as being suitable only for routine cases, being a loss of responsibility for care, devaluing the doctor–patient relationship and resulting in loss of consultant control. 26 complex case unsuitable, 16 different operating techniques/standards, 14 devalues operation, 13 increases disparity in workload 92% of GPs for pooled lists, 8% against. 40% of GPs referred to named consultant, 56% to a department. 92% of GPs happy for patient to be referred to another consultant if they would be seen sooner. 88% would switch if patient could be seen 1 month sooner in a wait of 7 months; 8% would want same surgeon whatever waiting time 82.4% of patients for pooled lists, 17.6% against. 82% of patients reported that they would want operation to be done sooner if performed by another surgeon of equal ability. 79% would change consultant for a 1-month reduction in waiting time if wait was 7 months. 18% would not wish to change consultant at all. 73% of patients did not know the name of their consultant |
Reported associations between elements for logic model GPs and patients more in favour of pooled waiting lists, consultants more against |
| Ridsdale 2007266 Country: UK Study design: Cohort Data collection method Aim: To describe and compare headache patients managed in primary care with those referred to neurologists Detail of participants (number, any reported demographics): Eighteen general practices in south-east England 488 eligible patients consulting GPs with primary headache over 7 weeks and 81 patients referred to neurologists over 1 year |
Method: This study examined 488 eligible patients consulting GPs with primary headache over 7 weeks and 81 patients referred to neurologists over 1 year. Headache disability was measured by the Migraine Disability Assessment Score, headache impact by the Headache Impact Test, emotional distress by the Hospital Anxiety and Depression Scale and illness perception was assessed using the Illness Perception Questionnaire Control: Non-referred patients Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to neurologist |
Outcome measures: Referral Clinical measures |
Main results: Participants were 303 patients who agreed to participate. Both groups reported severe disability and very severe impact on functioning Referred patients consulted more frequently than those not referred in the 3 months before referral (p = 0.003) There was no significant difference between GP-managed and referred groups in mean headache disability, impact, anxiety, depression or satisfaction with care The referred group were more likely to link an increased number of symptoms to their headaches (p = 0.01), to have stronger emotional representations of their headaches (p = 0.006), to worry more (p = 0.001), and were made anxious by their headache symptoms (p = 0.044) |
Reported associations between elements for logic model: Referral is not related to clinical severity of headaches, but is associated with higher consultation frequency and patients’ anxiety and concern about their headache symptoms |
| Ringard 2010164 Country: Norway Study design: Cross-sectional Data collection method: Two surveys Aim: To examine referral patterns to a local rather than further away providers Detail of participants (number, any reported demographics): Distributed to all GPs in Norway. First 1635 GPs, second 1858 GPs. Mean age 48 years |
Method: Survey carried out in 2004 and again in 2006 of referrals for hip replacement, knee surgery, back pain treatment Control: None Length of follow-up: NA Response and/or attrition rate: 48.4% first and 50.9% second Context (from what/who to what/who): GP to hospital |
Outcome measures: Proportion of patients referred to local vs. alternative or private hospital |
Main results: Increase in average referrals for all diagnoses between 2004 and 2006; however, only increase in referrals for back pain significantly different (p = 0.05) Few variables relating to GP individual characteristics or the practice characteristics were significantly related to referral pattern. Exception was the GPs level of expertise in back pain and hip surgery where GPs with a specialty within general medicine were less likely to refer patients to an alternative hospital. Practising in an area with an older population or more people with disability payments was associated with greater likelihood of referring to the local hospital. Practising in an area with a highly educated population significantly increased the likelihood of referring elsewhere for knee surgery only Long waiting times increased likelihood of referring away but only for knee and hip surgery Referral also influenced by having a formal arena for co-operation and exchange of information, frequency of GPs attending formal meetings and having a positive perception of the collaboration (all less likely to refer away) |
Reported associations between elements for logic model: Variation in referral behaviour different diagnoses Formal co-ordinative mechanisms led to a stronger referral relationship |
| Robinson and Taylor 2010288 Country: New Zealand Study design: Cross-sectional Data collection method: Referral data analysis Aim: To assess the factors associated with allocation to urgent or semiurgent appointment Detail of participants (number, any reported demographics): 128 new patients’ data; 69% female; average age 53 years |
Methods: Data extracted from medical records and GP referral letters. Time to treatment calculated Study service carries out urgent vs. semiurgent triage for new referrals Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to rheumatology |
Outcome measures: Predictors of an urgent allocation as compared with a semiurgent allocation of a referral |
Main results: The only factors which influenced whether patients were triaged by the rheumatology service as urgent or semiurgent was GP assessment of urgency (OR 13.34, 95% CI 2.20 to 81.02) and patient being of a younger age No evidence of triaging based on known prognostic indicators by service Need for interventions to improve information contained within referral from GP and/or education to improve triage at point of receipt of referral |
Reported associations between elements for logic model: Triaging of referrals at point of receipt |
| Rosemann 2005211 Country: Germany Study design: Observational Data collection method: Questionnaires completed after referral Aim: Describing the experiences of consultants, GPs and patients with referrals from primary care to medical specialist care Detail of participants (number, any reported demographics): Referrals of 26 GPs from 25 practices in Marbach, a rural region in the south of Germany, were studied |
Method: GPs, consultants and patients completed short structured forms to document factual characteristics of each referral and their experiences with the referral. GPs and patients completed forms before and after the referral was made, while the consultants completed forms after the patient had consulted them Control: NA Length of follow-up: NA Response and/or attrition rate: NR Context (from what/who to what/who): Medical specialists |
Outcome measures: Practitioner view Patient views |
Main results: Overall, consultants were very positive about appropriateness of the referral (91%). They were somewhat more critical regarding the information provided on the patients’ medical history (61%) and prescriptions (48%) In 258 referrals (63%) GPs perceived clear diagnostic benefits, while in 202 referrals (49%) they perceived clear treatment benefits. GPs’ experiences were more positive if the GP’s purpose was to reduce diagnostic uncertainty (beta = 0.318, p < 0.001) or if the purpose was to exclude serious illness (beta = 0.143, p < 0.010) Other purposes of the referral had no impact on their experiences Patients’ expectations regarding the referrals mostly referred to diagnosis, including increased diagnostic certainty (80%), detailed information about the illness (66%) and exclusion of serious illness (62%). They were overall satisfied with the referral (83%). Their experiences with the referral were more positive if the initiative for the referral came from the physician (beta = 0.365, p < 0.000) |
Reported associations between elements for logic model: Patients were most positive if the physician had initiated the referral Lack of content in referral letter Purpose of the referral to reduce diagnostic uncertainty or exclude serious illness |
| Rosen 2007252 (King’s Fund Report) Country: UK Study design: Qualitative Data collection method: Telephone interviews and focus groups Aim: To seek GP views regarding making choices in referral Detail of participants (number, any reported demographics): GPs in Suffolk, Westminster, Haringey, Somerset, Colchester and Central Cheshire |
Methods: Qualitative analysis of telephone interviews and focus groups Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Any specialist |
Outcome measures: Views |
Main results: GPs vary considerably in the extent to which they actively support patient choice Most GPs make choices on the patient’s behalf (with or without Choose and Book) unless the patient expresses a preference. They will typically only try to over-ride patient preferences in order to steer them towards excellent providers or away from providers about whom they have concerns Very few GPs had used Choose and Book to actively engage patients in discussion about choice of provider Four factors influence the degree to which GPs support patient choice: the availability and quality of formal and informal information about services; GPs’ views of their professional role; patient preferences; and local PCT policies that may restrict choice Patients and GPs seek overlapping but different characteristics when choosing a hospital. GPs’ advice to patients typically balances soft information about clinical quality with patient preferences or published data about waiting times and convenience of access The referral management centre opened by one PCT was seen to restrict choice Early experiences of Choose and Book have raised particular concerns about not being able to refer to a named consultant and about choices that are excluded by the PCT but highly desired by patients In terms of their role in supporting choice, GPs broadly fell into three groups – enthusiasts, sceptics and paternalists Views differed on the value of developing support services, such as choice advisers or health advisory centres, to support patient choice There was no consensus about the type of information GPs wanted to support patient choice or about how it should be presented The distrust of formal information was countered by a preference for soft, informal sources of information GPs recognised that in reality both formal and informal information was incomplete and unreliable. They took a pragmatic approach to combining the two in making referral decisions For all GPs, the single most important equity issue was access and the constraints of geography and transport. This was felt most acutely by GPs in rural areas, but was present even for GPs in inner cities whose populations were deprived or elderly Many GPs do not see choice as an equity issue because they do not see it as providing benefit for patients Choose and Book systems should include the option to refer to a named consultant as a default setting that is removed only as a result of a specific local decision Where referral management centres exist, there should be an explicit policy about the range of advice that can appropriately be provided and the way it is imparted to ensure that it complements advice from the GP or other referring clinicians Performance data about different providers should be available in multiple formats with the option for online information users to manipulate the data into the format of their choice. As data sets are developed, they should be piloted to test both their clarity and their credibility before they are widely disseminated |
Reported associations between elements for logic model: Influence of GPs’ informal knowledge about local services Lack of trust in formal data Preference for referring to named consultant Different levels of support for patient choice |
| Rowlands 2001182 Country: UK Study design: Qualitative Data collection method: video transcripts Aim: To investigate the effect of in practice meetings on practice referral rates Detail of participants (number, any reported demographics): |
Method: Implementation and evaluation of existing guidelines on the use of neurophysiological tests in non-acute migraine patients: a questionnaire survey of neurologists and primary care physicians. Subsequent to a RCT education intervention Control: NA Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist |
Outcome measures: Referral |
Main results: There was no alteration of practice referral rate following the education intervention. The qualitative study highlighted the complexity of decision-making in general practice and likely impact of historical background and internal and external pressures on referral |
Reported associations between elements for logic model: Unclear |
| Rushton et al. 2002183 Country: USA, Canada and Puerto Rico Study design: Cross-sectional Data collection method: Analysis of prior data from provider survey, parent survey and patient data Aim: To examine primary care provider referral patterns Detail of participants (number, any reported demographics): 4012 patients in the child behaviour study who had an identified psychosocial problem. Provider data and parents |
Methods: Cross-sectional analysis of patient data Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): PCP referral for child psychosocial services |
Outcome measures: Factors associated with referral |
Main results: Factors associated with likelihood of referral were patient factors (severity, type of problem, academic difficulties, prior mental health service use, parent agreement with plan, visit time of more than 20 minutes, comorbidity) and family factors (mental health referral of parent, low maternal education, family dysfunction) None of the provider factors were significant: gender, age, specialty, behavioural training, solo practice, rural population and geographic location Clinicians mentioned barriers to referral; however, these were rarely reported as influencing individual management decisions |
Reported associations between elements for logic model: Importance of patient-related factors, lack of association with physician factors Report of obstacles to referral but these not influencing individual referrals |
| Ruston 2004194 Country: UK Study design: Qualitative Data collection method: Interviews Aim: To explore referral decision-making by GPs Detail of participants (number, any reported demographics): 85 women newly referred to specialist breast clinic in one health authority. Health authority had higher than average incidence of breast cancer and lower than average survival rates 85 GPs of the women referred, 49 male, 36 female; most trained in London; been in practice 1–10 years; 35 from practices with five or more partners |
Methods: Interviews with patients and their GP analysed and compared Number of hours: NA Delivered by who? Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist breast clinic |
Outcome measures: Reasons for referring |
Main results: None of GPs reported using referral guidelines. Considered of theoretical rather than practical relevance and therefore did not adequately address the fact that practitioners deal with individual patients with individual characteristics Respondents perceived guidelines covered clinical aspects only and neglected other factors Defensive strategies described by GP – risk to the woman of not referring when breast cancer was a serious disease and risk of patient resorting to litigation if not referred and problem found later |
Reported associations between elements for logic model: Lack of use of guidelines Defensive practice, fear of litigation |
| Samant 2007234 Country: Canada Study design: Cross-sectional Data collection method: Survey Aim: To investigate the factors influencing radiotherapy referral among family physicians Detail of participants (number, any reported demographics): 400 physicians in eastern Ontario |
Method: A 30-item survey was developed to determine palliative radiotherapy knowledge and factors influencing referral. It was sent to 400 physicians in eastern Ontario (Canada) and the completed surveys were evaluated Control: None Length of follow-up: None Response and/or attrition rate: 50% Context (from what/who to what/who): GP referral for radiotherapy |
Outcome measures: Factors affecting referral |
Main results: The overall response rate was 50%, with almost all physicians seeing cancer patients recently (97%) and the majority (80%) providing palliative care Approximately 56% had referred patients for radiotherapy previously and 59% were aware of the regional community oncology programme Factors influencing radiotherapy referral included the following: waiting times for radiotherapy consultation and treatment, uncertainty about the benefits of radiotherapy, patient age, and perceived patient inconvenience Physicians who referred patients for radiotherapy were more than likely to provide palliative care, work outside urban centres, have hospital privileges and have sought advice from a radiation oncologist in the past A variety of factors influence the referral of cancer patients for radiotherapy by family physicians, and addressing issues such as long waiting times, lack of palliative radiotherapy knowledge and awareness of Cancer Centre services could increase the rate of appropriate radiotherapy patient referral |
Reported associations between elements for logic model: Unclear |
| Scheeres 2007163 Country: the Netherlands Study design: Cross-sectional Data collection method: Survey Aim: Investigated the impact of an informational intervention among GPs about a new treatment with cognitive–behavioural therapy for chronic fatigue syndrome (CFS) in a mental health centre Detail of participants (number, any reported demographics): 301 GPs |
Method: 301 GPs, who all had received written information about chronic fatigue syndrome four times, and who partly had also visited an informational group session, completed a short questionnaire survey on chronic fatigue syndrome knowledge and attitudes. Referral data were obtained from the mental health centre Control: NA Length of follow-up: NA Response and/or attrition rate: 67% response rate Context (from what/who to what/who): GP referral to mental health |
Outcome measures: GP knowledge and attitude |
Main results: During 16 months 22% of all GPs in the concerning region had referred at least one CFS patient Concerning knowledge and attitude, the survey results showed that 70% of the GPs had remembered the intervention’s main message, namely the new treatment possibility These informed GPs reported better knowledge and more positive attitudes towards CFS than the non-informed GPs, who had not seen and read the intervention’s information |
Reported associations between elements for logic model: This study showed that disseminating written materials can be a useful method for stimulating GPs to refer |
| Shadd 2011263 Country: Canada Study design: Retrospective data analysis Data collection method: Analysis of electronic health record database Aim: To analyse referral patterns and variance in referral rates Detail of participants (number, any reported demographics): Records of 33,998 patients from 10 primary care providers. All patients who visited their family practice 2007–8 |
Method: Data extracted from electronic health records Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to any specialist |
Outcome measures: Referrals to specialist following visit to primary care provider |
Main results: 31.3% of patients had one or more referrals. Overall rate 455 per 1000 patients per year Rates higher for females, older patients, and rural practices Referral rate from rural practices was higher than from urban (424/1000 p = 0.0001) Referral rate varied by specialty. Highest general surgery, obstetrics and gynaecology and orthopaedic surgery. Differed if analysed by gender/age band Referral rate correlated with number of primary care providers visits 92% of the variance in referral rates was attributable to the patient (rather than the practice). Patient level variance 0.574 with SE 0.022 and practice level 0.051 with SE of 0.023 |
Reported associations between elements for logic model: Variance in referral rate predominantly due to patient difference |
| Sigel 2004198 Country: UK Study design: Qualitative Data collection method: Interview Aim: How do GPs manage mental health problems Detail of participants (number, any reported demographics): 10 GPs in one locality, seven male, aged 38–60 years, years in practice 9–26 |
Methods: Qualitative. Semistructured interviews Control: None Length of follow-up: NA Response and/or attrition rate: 32% response rate Context (from what/who to what/who): GP referral for psychological problems |
Outcome measures: Perceptions of referral process |
Main results: Referral decisions made when GPs perceive that they have reached the limits of their capabilities for treating a problem, taking account of patient suitability for therapy and access to services. Some occasions GPs referred without treating first when they felt out of their depth Reasons for not referring on were when problem was perceived as self-limiting or patient was too upset to cope with therapy Referral decisions influenced by views of psychological problems and therapies and by GPs’ professional interactions with psychologists Patient suitability defined in terms of patient preference, and patient characteristics (e.g. insight and ability to articulate and willingness to engage) |
Reported associations between elements for logic model: GP individual decision-making and referral Patient characteristics and referral |
| Soerensen 2009298 Country: Denmark Study design: Cross-sectional Data collection method: Practice data Aim: To explore the association between patients’ socioeconomic status and their referral from general practice to specialised health care Detail of participants (number, any reported demographics): N/R |
Method: Multiple regression analysis was used on cross-sectional data on general practice referral rates for all Danish general practices in year 2006 Control: NA Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist |
Outcome measures: Referral Patient characteristics |
Main results: The models explained between 26% and 45% of the variation in general practice referral to specialised care Adjusting for access to specialised care (local supply of hospitals and practising specialists) reduced the association between socioeconomic factors and referral rates The results suggest that persons with high socioeconomic status are referred more to practising specialist than persons with low socioeconomic status and that the latter are referred more to hospital care than the former |
Reported associations between elements for logic model: The results indicate that the influence of socioeconomic factors may be overstated failing to control for access to specialised care |
| Stavrou 2009249 Country: UK Study design: Qualitative Data collection method: Interviews using case study patients Aim: To explore GP accounts of decisions to refer Detail of participants (number, any reported demographics): 14 GPs from inner London boroughs in practices that had counsellors and clinical psychologists. Seven male, seven female; mean age 39 years; mean 11 years in practice. Most working in large practices and with interest in mental health |
Methods: Clinical psychologist and counsellor identified the GP’s five most recent referrals aged 18–65 and referred for anxiety, depression, panic disorder, obsessive–compulsive disorder and also five comparable cases where GP had decided not to refer. In interview GP asked to describe and compare patients Control: None Length of follow-up: NA Response and/or attrition rate: 47% response rate Context (from what/who to what/who): GP referral of patients with anxiety, depression to psychological therapy or mental health services |
Outcome measures: Themes |
Main results: Three factors distinguished those referred and not referred Patient’s request and interest in referral – no GP refused if patient asked, reassured that patient thought about it and showed motivation, gave a focus to the consultation and saved time Patient’s likely benefit – how suitable a candidate, including more motivated and having more insight, psychologically minded, obsessive–compulsive disorder and panic disorder referred as viewed as being treatable. Also those with history of not attending considered not suitable. Those with intractable or chronic problems seen as not benefiting GP’s perceived capacity to help – perceived no expertise to help (e.g. childhood abuse, obsessive–compulsive disorder, post-traumatic stress disorder, feeling of needing more than they could offer, not having enough time, whether or not improving with GP treatment and GP confidence in treatment, severity of presentation) Often a combination of these factors, and some GPs placed more emphasis on some factors than another Waiting lists mentioned in consideration whether or not to refer. GPs would prioritise patients they thought would use the referral well and benefit the most |
Reported associations between elements for logic model: Views of GP and referral |
| Steele 2012203 Country: Canada Study design: Cross-sectional Data collection method: Survey Aim: To examine the referral patterns of remote/rural PCPs Detail of participants (number, any reported demographics): PCPs located in rural/remote areas. n = 847. Majority male, aged 41–60 |
Method: Postal or online survey; rating of frequency of referral often, sometimes, rarely Number of hours: NA Control: None Length of follow-up: NA Response and/or attrition rate: 24.9% Context (from what/who to what/who): Primary care providers to psychiatry |
Outcome measures: Factors in referral decisions |
Main results: Long wait times most common deterrent to referral Main reasons for referral were to obtain recommendation regarding medication and for assessment of non-responsive patients High confidence in making referrals, lower confidence in managing mental health patients, need for professional development identified |
Reported associations between elements for logic model: Referral requested with regard to medication, non-responding patients |
| Sullivan 2005285 Country: UK Study design: Retrospective cohort Data collection method: Database analysis Aim: To examine the role of age, sex and morbidity in referral Detail of participants (number, any reported demographics): 202 GP practices with patient total 1,161,892 |
Method: Data from General Practice Research Database Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GPs to specialist referral |
Outcome measures: Percentage of patients referred |
Main results: 14% of patients referred to a specialist at least once during study period The percentage of patients referred increased with age and morbidity. 7.5% aged 0–15 years, compared with 21.1% aged under 65 years. Females 17.1% vs. 12.2% males Age and sex explained only 5.3% of variability, morbidity explained 30.4% of the variability (healthy to least healthy) Most variation occurred within practices, with between-practices variation only 5%. Range in patients referred after excluding those with very low referrals was 2.4% to 24% Two-thirds (66.1%) of variation remained unexplained at patient level and 3.6% at the practice level |
Reported associations between elements for logic model: Variation in within-practice referral rates, patient characteristics and referral |
| Swarztrauber 2002171 Country: USA Study design: Cross-sectional Data collection method: Questionnaire Aim: To determine neurologists’ and primary care physicians’ preferences for specialist involvement in the management of common neurological conditions and the factors influencing these preferences Detail of participants (number, any reported demographics): 608 family physicians, 624 general internists, and 492 neurologists |
Method: A self-administered questionnaire was developed with the assistance of a multispecialty Advisory Board and sent to a stratified probability sample of 608 family physicians, 624 general internists and 492 neurologists The questionnaire contained three clinical scenarios, each followed by questions regarding respondent preferences for the primary care physician to manage alone, curbside or refer to a specialist. The questionnaire also contained knowledge questions corresponding to each scenario and physician and practice characteristics questions Control: NA Length of follow-up: NA Response and/or attrition rate: Response rate was 60% Context (from what/who to what/who): GP to neurology |
Outcome measures: Reasons for referral Knowledge |
Main results: For all three scenarios, primary care physicians’ preferences to involve a specialist differed substantially from neurologists’ preferences, with nearly all neurologists preferring involvement of a specialist: transient neurological event (48% and 39% vs. 2%), Parkinson’s disease (37% and 38% vs. 3%), and dementia (74% and 59% vs. 2%) Primary care physicians with less knowledge were more likely to prefer assistance from a neurologist Primary care physicians who preferred to manage the patient without specialty involvement had higher knowledge scores than primary care physicians who preferred to curbside or refer to a specialist (p < 0.001) Physician age and practice setting influenced the type of assistance preferred (curbside vs. referral) Utilisation management techniques and financial incentives had little influence on physician preferences regarding involving a specialist |
Reported associations between elements for logic model: Unclear |
| Taggarshe 2006233 Country: UK Study design: Cross-sectional Data collection method: focus group and questionnaire Aim: Assessed GP’s referral patterns across a metropolitan health authority, which has actively encouraged generic referrals Detail of participants (number, any reported demographics): |
Method: A focus group of GPs was used to determine the factors influencing referral patterns to secondary care for a surgical opinion A questionnaire was devised based on the factors that emerged from the focus group All GPs attending continuing-medical-education sessions across Doncaster Health authority were asked to complete this questionnaire Control: NA Length of follow-up: NA Response and/or attrition rate: 99% response rate Context (from what/who to what/who): GP referrals to specialist (generic referrals) |
Outcome measures: GP attitudes/behaviours |
Main results: Of the 79 GPs surveyed, 78 completed the questionnaire Of the 78 respondents, only 17 (22%) stated that it was their policy to make generic referrals, and 61 GPs (78%) always made specific named consultant referrals Almost four of five GPs made referrals specifically to a named surgeon. A total of 43% of the GPs who referred to a named surgeon ranked perceived clinical skills/competence as the most important factor The other factors that influenced their decision in order of importance were waiting times (19%), personal rapport with consultant (12.6%) and feedback from patients (12.6%) |
Reported associations between elements for logic model: GPs in the Doncaster area do not make generic referrals. This has to be taken into account in planning service delivery |
| Thorsen 2012239 Country: Norway Study design: Qualitative study Data collection method: Semistructured focus group interviews Aim: Identify and describe GPs’ reflections on and attitudes to the referral process and co-operation with hospital specialists Detail of participants (number, any reported demographics): 17 female and 14 male GPs aged 29 to 61 years from 21 different practices, who had practised for 3–35 years |
Method: Interviews conducted over four months from November 2010 to February 2011 Control: NA Length of follow-up: NA Response and/or attrition rate: NR Context (from what/who to what/who): GP referral to hospital specialists |
Outcome measures: Description of GPs‘ views on the referral process |
Main results: GPs wished for improved dialogue with the hospital specialists. The referral process was often considered as asymmetric and sometimes humiliating. GPs saw the benefit of using templates in the referral process, but were sceptical concerning the use of mandatory fixed formats |
Reported associations between elements for logic model: Unclear |
| Todman 2011295 Country: UK Study design: Cross-sectional Data collection method: Survey Aim: To examine the therapeutic preference of GPs Detail of participants (number, any reported demographics): 119 GPs in one area of Scottish Highlands |
Method: Three questions relating to need and current treatment Control: None Length of follow-up: NA Response and/or attrition rate: 46% Context (from what/who to what/who): GP to Older Adult Psychology services |
Outcome measures: Type of treatment Perception of efficacy |
Main results: GPs reported that a lower waiting times and more localised service would increase the likelihood of a referral being made Many patients with depression or anxiety may not be referred. Social therapeutic options seen as most effective option (mean 5.23 SD 1) followed by pharmacological interventions (mean 4.92 SD 1.1). However, most likely to prescribe pharmacological treatment (84%) rather than social therapy (40%) GPs’ low rate of referral of elderly patients |
Reported associations between elements for logic model: Need to consider GP preferences for intervention type |
| Townsley et al. 2003165 Country: Canada Study design: Cross-sectional Data collection method: Postal survey Aim: To understand why older patients may be under-referred Detail of participants (number, any reported demographics): 2089 questionnaires returned from primary care providers throughout Ontario |
Method: Questionnaire developed from literature review and Delphi. 21 questions including a clinical scenario. Demographics, attitudes, potential barriers Control: None Length of follow-up: NA Response and/or attrition rate: 24% Context (from what/who to what/who): Primary care providers to oncology |
Outcome measures: Referral rate Factors influencing referral decisions |
Main results: Factors influencing GPs decision to refer: Patient desire to be referred (69%) Type of cancer (54%) Stage of cancer (49%) Severity of symptoms (49%) Age did not influence referral decision. Barriers cited were length of waiting list, need for tissue diagnosis before referral and belief that oncologists seldom relate to primary care providers Physicians in rural or mixed areas less likely to refer late-stage patients and more likely to consider patient wishes and availability of oncology specialists Physicians with extra training in geriatrics and those in practice longer less likely to refer regardless of tumour stage |
Reported associations between elements for logic model: Patient wishes and referral GP additional training and referral |
| Trude 2003199 Country: USA Study design: Cross-sectional Data collection method: Telephone survey Aim: To examine how practice setting may impact on referral Detail of participants (number, any reported demographics): 6586 primary care physicians 40% female; 34% less than 10 years qualified; half worked in practice with nine or fewer physicians. 27% acted as gatekeeper for 70% or more of caseload |
Methods: Data from the Community Tracking Study Physician Survey Control: None Length of follow-up: NA Response and/or attrition rate: 61% Context (from what/who to what/who): Primary care to mental health services |
Outcome measures: Ability to arrange referrals Practice characteristics |
Main results: 54% of physicians reported problems obtaining psychiatric hospital admissions, 54% reported difficulty arranging mental health outpatient referrals Physicians who felt that they did not have adequate time with their patients reported problems obtaining mental health services Physicians in solo and small group practices reported having more difficulty obtaining referrals to mental health services |
Reported associations between elements for logic model: Problems obtaining referral to mental health |
| Tucker 2003198 Country: UK Study design: Cross-sectional Data collection method: Postal survey and telephone interview Aim: To assess the diagnosis and management of mild non-proteinuric hypertension in pregnancy in rural general practices against guideline recommendations Detail of participants (number, any reported demographics): All 174 designated rural general practices in Scotland. 171 GPs and 158 midwives responsible for antenatal care stratified by distance from a specialist maternity hospital |
Method: Self-complete questionnaires and one reminder were mailed. The questionnaire contained two scenarios for mild non-proteinuric hypertension taken from the Scottish Obstetric Guidelines and Audit Project on mild non-proteinuric hypertension in pregnancy This national guideline uses the methodology of the Scottish Intercollegiate Guidelines Network and was developed by a multidisciplinary group from university teaching and district general hospitals, as well as by staff from primary/community care settings. Scottish Intercollegiate Guidelines Network guidelines are widely circulated and are sent automatically to all GPs in Scotland. Professionals were asked for their diagnosis and management for scenario 1 and management for scenario 2. Replies were compared with the correct diagnosis and the recommended management according to the guideline Response and/or attrition rate: At least one professional returned a completed questionnaire for 158 (91%) of the 174 practices. The response rates were 68% (117/171) for GPs and 77% (121/158) for midwives Context (from what/who to what/who): GP referral for mild non-proteinuric hypertension in pregnancy |
Outcome measures: Accuracy of diagnosis and appropriateness of management compared with guideline |
Main results: At least one respondent replied for 91% (158/174) of rural practices. Response rates were 68% (117/170) for GPs and 77% (121/158) for midwives. Both GP and midwife replied for 46% (80/174) of practices. Most GPs (80%, 87/109) and midwives (63%, 71/113) overdiagnosed the scenario. Intended management was, therefore, most often referral or admission to specialist hospital (59%, 132/224), both courses of action beyond guideline recommendations There was an association between distance of practice from specialist maternity hospital and professionals’ report of intended referral or admission Explanatory factors from telephone interviews included a poor knowledge base, cautious risk assessment and perceived inflexibility of guidelines for remote situations |
Reported associations between elements for logic model: The results suggest that women in rural settings may experience more antenatal referrals and admissions than are clinically appropriate according to the guidelines |
| Tzaribachev 2009173 Country: Germany Study design: Retrospective cohort Data collection method: Patient data Aim: To report on referral patterns of primary physicians for children subsequently diagnosed with juvenile idiopathic arthritis and to identify predictors of delayed referral to a paediatric rheumatology centre Detail of participants (number, any reported demographics): A total of 132 patients with juvenile idiopathic arthritis were included; 83 (63%) were female |
Method: A retrospective cohort study of consecutive patients with juvenile idiopathic arthritis referred to a paediatric rheumatology centre over a 15-year period was performed. Variables included age, sex, juvenile idiopathic arthritis subtype, the physician’s subspecialty and distance to the paediatric rheumatology centre Putative predictors were evaluated by analysis of variance, resulting in regression models Control: NA Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to paediatric rheumatology |
Outcome measures: Time to first presentation to a primary physician Time to the first rheumatology visit Total time to referral |
Main results: The median age at the onset of symptoms was 4.5 years (range 1.0–15.8 years) Most frequently, children were referred by paediatricians (49.4%) or orthopaedic surgeons (34.1%) The median time to first presentation was short at 10 days (range 0–1610 days). In contrast, the median time to first rheumatology visit was 60 days (range 0.0–100.0 days), resulting in a long median total time to referral of 90 days (range 0.0–160.0 days) Statistically significant predictors for delayed referral were the primary physician’s subspecialty (p = 0.016) and the distance to the paediatric rheumatology centre (p < 0.001) Children living in remote areas or referred by orthopaedic surgeons had the longest referral times |
Reported associations between elements for logic model: Despite free access to health care in Germany, children with juvenile idiopathic arthritis are referred to paediatric rheumatology centres with significant delay |
| Van der Weijden 2002212 Country: the Netherlands Study design: Qualitative Data collection method: Interviews with GPs Detail of participants (number, any reported demographics): 21 GPs in rural and urban areas |
Methods: Qualitative interviews Control: None Length of follow-up: None Response and/or attrition rate: Context (from what/who to what/who): GP to laboratory testing for patients presenting with unexplained complaints |
Outcome measures: Reported variables influencing test ordering behaviour |
Main results: Variables related to practice and societal aspects and included attitudes, social influences and self-efficacy Practice – time pressures, availability of technology Societal – expense, claims culture, threshold for patient consultation Attitudes – cognition and beliefs of GP Social influences – social norms, pressure, support, modelling Self-efficacy – GPs’ expectation regarding capability to perform desired behaviour Cognitions – older patient, alarming symptoms Misconceptions – value of tests Uncertainty – handling uncertainty, error tolerance Wish to comfort patients Social influences from the patient – assertive, worried, relationship with patient, need for reassurance Social influences from the profession – influence of specialist, test ordering behaviour Perceived capability/skills – diagnostic capability, skills of patient negotiation, ability to reassure oneself, ability in history taking Other factors – routines, GP condition (e.g. tiredness, tactical motives for testing) |
Reported associations between elements for logic model: Determinants of test ordering are numerous with both medical and non-medical motives GPs order tests for many purposes. How these factors interact remains an individual-GP matter |
| Vinker 2007269 Country: Israel Study design: Cross-sectional Data collection method: Patient questionnaire and examination of referral letters Aim: To examine the characteristics of urgent referrals Detail of participants (number, any reported demographics): 257 referrals |
Methods: Cross-sectional analysis of patient data Control: None Length of follow-up: NA Response and/or attrition rate: Context (from what/who to what/who): Referral to ophthalmology, orthopaedics, ENT or dermatology |
Outcome measures: Characteristics of referral Judgement of urgency |
Main results: Length of time patient with GP did not affect referral Reasons given by patients for referral – the FP did not know how to treat the problem (44%); the FP wanted a second opinion (23%); patient request (20%); and treatment did not help (10%) Urgent referrals tended to be younger. Patients given urgent referrals perceived their GP to be more available. Most referrals to ophthalmology were urgent No significant patient gender difference whether referrals classed as urgent or non-urgent Low agreement between peer observers and referrer regarding whether an urgent referral should be classed as urgent. Observers viewed as urgent only 20.7% of urgent referrals. Disparity between consultants regarding classification and referrer (consultants thought that only 48.7% of urgent referral were urgent) |
Reported associations between elements for logic model: Patient views regarding reasons for referral Patient gender not a predictor of urgency of referral |
| Vulto et al. 2009250 Country: the Netherlands Study design: Cross-sectional Data collection method: Survey Aim: To examine GP knowledge of palliative radiotherapy Detail of participants (number, any reported demographics): 489 GPs south Netherlands; 65% male; 50% health centre practice; 75% full-time; 56% 30–60 minutes to radiotherapy department |
Methods: Questionnaire Control: None Length of follow-up: NA Response and/or attrition rate: 45.5% Context (from what/who to what/who): GP to palliative radiotherapy |
Outcome measures: Self-reported knowledge Referral rate |
Main results: 40% of respondents assessed their knowledge of palliative radiotherapy as modest. Time since graduation did not seem to affect knowledge Patient-related factors such as discomfort and poor condition but not age were reported to influence referral. Most GPs reported they reacted to the wish of the patient regarding referral |
Reported associations between elements for logic model: Knowledge and referral, patient wishes and referral |
| Wakefield 2012227 Country: Canada Study design: Cross-sectional Data collection method: Survey of family physicians Detail of participants (number, any reported demographics): 91 family physicians |
Methods: Cross-sectional survey Control: None Length of follow-up: None Response and/or attrition rate: 91 of 458 physicians responded (19.9%) Context (from what/who to what/who): Family physician to cardiology |
Outcome measures: Likert scales Data on referrals |
Main results: No difference in reported referral criteria used by physicians by gender, age or length of time in practice Analysis of physician referral considerations – scale 1 = unimportant, 5 = important Fast access 4.77; test results available quickly 4.69; notification to GP of test date 4.65; reports concise 4.59; confidence in physician to address problem 4.59; previous experience with facility 4.37; short distance to travel 4.25; option to consult with specialist 4.25; knowledge of physicians at the facility 4.23; location of facility 3.84; patient requests destination 3.65; 2.99 language barrier issues; 2.66 have privileges at that institution; patient in otherwise good health 2.32; patient age 2.26; patient gender 1.95 Factor analysis – factor of physician communication expectation explained 23.4% of variance; patient characteristics explained 16.1% of variance; physician access 10.6%; geographic location 8.2%; patient convenience 7.3%. Variance accounted for by these factors was 65.6% |
Reported associations between elements for logic model: Physicians choose referral destination based on a range of factors with good communication from specialist service to GP and speed of access for patient of key importance |
| Walders 2003300 Country: USA Study design: Survey Data collection method: Aim: To examine the frequency of barriers to mental health referral according to paediatric primary care physician report and to identify factors related to perceptions of referral barriers for patients with managed care coverage Detail of participants (number, any reported demographics): Data from a national sample of 319 PCPs were examined |
Method: Data from a national sample of 319 primary care physicians were examined. Comparisons were made concerning the frequency of primary care physician-reported barriers to mental health referrals for patients with fee-for-service vs. managed care coverage. The relationship between relevant factors (e.g. practice structure, interdisciplinary office staff, availability of community mental health resources) and perceived barriers was examined for managed care coverage Control: None Length of follow-up: none Response and/or attrition rate: A total of 539 clinicians agreed to participate and 431 clinicians returned the study measures, reflecting an 80% response rate Context (from what/who to what/who): Paediatric primary care referrals for mental health |
Outcome measures: Barriers to referral |
Main results: Barriers to mental health referrals were more commonly reported for those with managed care vs. fee-for-service coverage for 11 of the 12 barriers assessed (p < 0.001) For patients with managed care coverage a high availability of community mental health resources was associated with fewer perceived barriers among physicians (p < 0.001) |
Reported associations between elements for logic model: Referral barriers were more commonly reported for patients with managed care coverage |
| Wassenaar 2007178 Country: USA Study design: Cross-sectional Data collection method: Postal survey with case scenarios Aim: To examine whether or not referral patterns for lung cancer differed from breast cancer Detail of participants (number, any reported demographics): 672 physicians in Wisconsin |
Methods: Postal survey with case scenarios Control: None Length of follow-up: NA Response and/or attrition rate: 59.4% Context (from what/who to what/who): PCPs to oncology |
Outcome measures: Likelihood of referral |
Main results: GPs reported that type of cancer was not a factor in their referral decisions; however, metastatic breast cancer patients were likely to be referred for further therapy whereas metastatic lung cancer patients were likely to be referred to symptom management only Patient smoking or non-smoking not a factor in referral decision No difference in referral patterns related to those who had more or fewer patients with cancer in their practice; age of physician not associated with referral pattern; no difference associated with physician gender More physicians knew chemotherapy improved survival in advance breast cancer than in advanced lung cancer |
Reported associations between elements for logic model: Physician demographics not associated with referral decisions Differing knowledge levels different types of disease and referral |
| Watson 2001195 Country: UK Study design: Cross-sectional Data collection method: Questionnaire (quantitative) Aim: The aim of the study was to investigate the appropriateness of primary care referrals to the Oxford Regional Genetics Service on account of family history of breast and/or ovarian cancer and to explore GPs’ expectations following a referral Detail of participants (number, any reported demographics): n = 50 consecutive referring GPs |
Method: 50 consecutive GPs were sent a questionnaire post referral, and their referral letters were reviewed Response and/or attrition rate: 94% response rate Context (from what/who to what/who): GPs to Oxford Regional Genetics Service |
Outcome measures: Appropriate referral |
Main results: Many GPs did not know which patients warranted referral to the genetics service and had unrealistic expectations of what happened at the clinic 41 GP referral letters were available in the clinic notes for assessment of whether the referral met the locally agreed guideline criteria. 46% (19/41) of the letters met the referral criteria (i.e. reported family history suggestive of moderate/high risk), 29% (12/41) did not meet the criteria, and in 24% (10/41) of cases there was insufficient information in the letter to know whether or not the criteria had been met. 53% (8/15) of referrals where the GP reported using guidelines met the referral criteria, compared with 42% (11/26) where guidelines were not reported (χ2= 0.465, p = 0.495). In the nine referrals which raised the issue of family history, only 1/3 (3/9) actually met the referral criteria |
Reported associations between elements for logic model: GP education in the proper use of referral guidelines is required |
| Wilkes et al. 2009213 Country: UK Study design: Qualitative Data collection method: Interviews Aim: To explore perceptions and attitudes of patients and professionals to open-access hysterosalpingography screening Detail of participants (number, any reported demographics): 12 GPs, five fertility specialists, 13 infertile couples |
Methods: Qualitative interviews Convenience sample Control: None Length of follow-up: NA Response and/or attrition rate: NR Context (from what/who to what/who): GP referral for infertility hysterosalpingography (an investigation of infertile women not usually available to GPs) |
Outcome measures: Views and perceptions |
Main results: Three GPs had access and used it, six had access and did not use it and three did not have access. Five couples had not experienced it, eight had experienced open-access testing Most GPs felt that open-access testing would speed up the investigation process but felt uncomfortable organising it Main barriers – infrequent exposure to infertile couples leading to lack of skills, lack of confidence and low priority. GPs felt it would create more work, although minimal. View that GPs may lack interest and therefore prefer to refer directly Suggestion that it should be managed by one GP in a practice on behalf of a group Half of GPs felt that it should be in their remit and that they had the expertise. Concerned regarding pressure on time, having expertise and keeping skills up to date. Minority felt that it should be a GPwSI, majority thought that it would fall within remit of GP with an informal interest Specialists reported benefit from the investigation being done by the GP – enabled diagnosis and management to be made at first appointment. Some concern about inappropriate use by GPs but ‘rationalised with access to HSG being criteria driven’. Specialists felt that it was appropriate for GP to perform the test Patients positive as it ‘got the ball rolling’, speeded up access to specialists. Majority felt that it was/should be within the GP remit; minority preferred direct referral Most participants felt that access to the test would direct referrals more appropriately with decreased commissioning costs through a decrease in unnecessary or wrongly directed referrals Guidelines seen as enabling practitioners to work effectively and offered some indemnity All GPs, specialists and patients who had experienced the service wished it to remain in place, including some who had not experienced it. Not all GPs were keen to have access to it personally |
Reported associations between elements for logic model: GP expertise, knowledge and open-access investigation Satisfaction with model Perceived benefits of open-access testing |
| Xu 2002235 Country: USA Study design: Cross-sectional Data collection method: Database analysis Aim: To examine physician and practice characteristics associated with perceived ability to refer Detail of participants (number, any reported demographics): 2572 PCPs in solo/two physician practices and 1820 in group practices. 79% male specialty in internal medicine, family/general practice (49%), paediatrics, medical specialties |
Methods: Data from the Community Tracking Study Physician Survey, multivariate analysis Control: None Length of follow-up: NA Response and/or attrition rate: The Community Tracking Survey had a response rate of 65% Context (from what/who to what/who): Primary care referral to specialist in case of ‘medical necessity’ |
Outcome measures: Perception of ability to refer – physician, practice and financial characteristics |
Main results: The perceived ability to refer did not differ by practice size Physician experience and the effect of formal written guidelines were influential in smaller practices but not in group practices The effect of financial arrangements was more significant for physicians in solo/two-physician practices The most significant determiner of perceived ability to refer was PCP physician satisfaction in their communication with specialists |
Reported associations between elements for logic model: Financial arrangements regarding referral impacted more on perceived ability to refer in smaller practices |
| Young 2010221 Country: Australia Study design: Qualitative Data collection method: Interviews with GPs Detail of participants (number, any reported demographics): 10 GPs |
Methods: Qualitative interviews Control: None Length of follow-up: None Response and/or attrition rate: Context (from what/who to what/who): GP to consumer health organisations |
Outcome measures: Views and perceptions |
Main results: Processes of referral were influenced considerably by the degree to which GPs had taken on a broader chronic care models rather than a more traditional care approach GPs orientated towards traditional care models stated that they did not refer patients who they perceived to be unmotivated GPs reflected on decisions to refer based on severity and stage of the disease – those with more severe or life-altering conditions were referred more frequently (e.g. diabetes more than conditions such as asthma which were not seen to be as ‘complicated illness’) Difference between GPs in terms of when to refer – newly diagnosed or later so as not to overwhelm GP concern that consumer health organisation may be negative or dominated by people with a viewpoint that was not right for the patient Referral to consumer health organisation viewed as easy/quick |
Reported associations between elements for logic model: Self-perceived role of GP and referral Variation in timing of referral |
| Zielinski 2008166 Country: Lithuania Study design: Cross-sectional Data collection method: Analysis of patient records Aim: To investigate whether or not different models of health care impact on referral patterns Detail of participants (number, any reported demographics): 18 PHC practices in one region, serving 250,070 inhabitants. 70% listed with public polyclinics; one-fifth of population listed with family medicine practices (public or private). Four different models operated – rural state-owned, family medicine practices, urban privately owned practices, state owned polyclinics and privately owned polyclinics (both mostly urban) |
Methods: Data from state sickness fund database Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Different models of primary care and referral to specialised care |
Outcome measures: Patient demographics Practice characteristics Referrals |
Main results: Family medicine practices located in rural areas had lower referral rates than other three models Referral rates were higher for female patients. Highest rate of referral was in age 60–79 years. Age and gender, though, had only minor effects Patient place of residence (rural/urban) and comorbidity level were the most important factors influencing referral rate. Those with higher comorbidity were more likely to be referred Patient characteristics explained about 40% of variability; facility characteristics and whether or not a family physician explain about 10% Referral rates were higher from public and private polyclinics, which have easier access to specialists than from public practices Referral rates were higher from private practices than public practices or public polyclinics Being a specialist in family medicine, training and experience correlated with lower referral rates |
Reported associations between elements for logic model: Location of practice and referral rate Patient demographics and comorbidity and referral |
Appendix 2 Study appraisal tools and assessments
Quality appraisal: intervention studies
| Potential risk of bias | Bias present? |
|---|---|
| 1. Selection bias: Method used to generate the allocation sequence, method used to conceal the allocation sequence (where applicable), characteristics of participant group(s) | Yes/no/unclear |
| 2. Performance bias: Measures used to blind participants and personnel and outcome assessors (where applicable), presence of other potential threats to validity | Yes/no/unclear |
| 3. Attrition bias: Incomplete outcome data, high level of withdrawals from the study | Yes/no/unclear |
| 4. Detection bias: Accuracy of measurement of outcomes, length of follow-up | Yes/no/unclear |
| 5. Reporting bias: Selective reporting, accuracy of reporting | Yes/no/unclear |
Table of intervention studies
| First author and date | 1. Selection bias, Y/N | 2. Performance bias, Y/N | 3. Attrition bias, Y/N | 4. Detection bias, Y/N | 5. Reporting bias, Y/N | 6. Higher risk/lower risk/unclear | Details of concerns |
|---|---|---|---|---|---|---|---|
| Adams 201233 | N | N | Y | N | N | Lower | 132 of 271 participants (48.7%) response |
| Akbari 2012110 | N | N | N | N | N | Lower | |
| Albertson 2002158 | N | N | N | N | N | Lower | |
| Banait 200353 | N | N | N | N | N | Lower | |
| Bennett 200130 | N | N | N | N | N | Lower | |
| Bhalla 200237 | N | N | N | N | N | Lower | |
| Bridgman 200570 | N | N | N | N | N | Lower | |
| Callaway 2000121 | N | N | N | N | Y | Lower | Only single % results reported |
| Campbell 2003131 | N | N | N | N | N | Lower | |
| Chen 2010100 | N | N | N | N | Y | Lower | Only narrative results reported |
| Cooper 201219 | Y | N | N | Y | Y | Unclear | Details of participants unclear Only single % results reported |
| Cox 2013156 | N | N | N | N | N | Lower | |
| Cusack 200543 | N | N | N | N | N | Lower | |
| DAMASK trial team 200876 | N | N | N | N | N | Lower | |
| Dennison 200699 | N | N | N | N | Y | Lower | Only single % results reported |
| Dey 200466 | N | N | N | N | N | Lower | |
| Dhillon 200382 | N | N | Y | N | N | Lower | Attrition rate not reported |
| Donohoe 200031 | N | N | N | N | N | Lower | |
| Eccles 200154 | N | N | N | N | N | Lower | |
| Eley 201083 | N | N | N | N | Y | Lower | Number results only, no statistical analysis |
| Ellard 201238 | N | N | Y | N | N | Lower | Attrition rate not reported |
| Elwyn 200755 | N | N | N | N | N | Lower | |
| Emery 2007111 | N | N | N | N | N | Lower | |
| Eminovic 200986 | N | N | N | N | N | Lower | |
| Emmerson 200340 | N | N | N | N | Y | Lower | Only single % results reported |
| Engers 200567 | N | N | Y | N | N | Lower | Attrition rate/length of follow-up not reported |
| Evans 200921 | N | N | N | N | N | Lower | |
| Evans 201122 | N | N | N | N | N | Lower | |
| Fearn 200948 | Y | N | N | N | N | Lower | Patient characteristics not reported |
| Felker 2004132 | N | N | N | N | N | Lower | |
| Ferris 2002146 | N | N | Y | N | N | Lower | Attrition rate not reported |
| Ferris 2001145 | N | N | Y | N | N | Lower | Attrition rate not reported |
| Ferriter 2006157 | N | N | N | N | N | Lower | |
| Gandhi 2008108 | N | N | N | N | N | Lower | |
| Glaves 200557 | N | N | N | N | N | Lower | |
| Gough-Palmer 200984 | N | N | N | N | Y | Lower | Only single % results reported |
| Greiver 2005114 | N | N | N | N | N | Lower | |
| Griffiths 200658 | N | N | N | N | N | Lower | |
| Gurden 2012133 | N | N | N | N | Y | Lower | Only single % results reported |
| Hands 200134 | N | N | N | N | N | Lower | |
| Harrington 200193 | N | N | N | N | Y | Lower | Only single % results reported |
| Heaney 2001159 | N | N | N | N | N | Lower | |
| Hemingway 200673 | N | N | N | N | Y | Lower | Intervention period not clear; reported by year rather than before and after |
| Hermush 2009137 | N | N | N | N | N | Lower | |
| Hill 200049 | N | N | N | N | Y | Lower | Only single % results reported |
| Hilty 200624 | N | N | N | N | N | Lower | |
| Hockey 200491 | Y | N | N | Y | Y | Higher | No participant details. Only narrative results reported. Intervention problems |
| Hughes-Anderson 2002136 | N | N | N | N | N | Lower | |
| Idiculla 200044 | N | N | N | N | Y | Lower | Only narrative results reported (few percentages) |
| Imkampe 200647 | N | N | N | N | N | Lower | |
| Iversen 2000151 | N | N | Y | N | Y | Higher | Only single % results reported. Only 37% of GPs provided data |
| Jaatinen 200295 | N | N | N | N | N | Lower | |
| Jiwa 200423 | N | N | N | N | N | Lower | |
| Jiwa 200668 | N | N | N | N | N | Lower | |
| Jiwa 2012105 | N | N | N | N | N | Lower | |
| Johnson 2008a139 | N | N | N | N | Y | Lower | Only narrative results reported |
| Joyce 2000147 | N | N | N | N | Y | Lower | Only narrative results reported |
| Julian 200762 | N | N | N | N | N | Lower | |
| Junghans 2007109 | N | N | N | N | N | Lower | |
| Kennedy 2012106 | N | N | N | N | Y | Lower | Only single % results reported |
| Kerry 200059 | N | N | N | N | N | Lower | |
| Khan 200871 | N | N | N | N | Y | Lower | Only single % results reported |
| Kim 2004155 | N | N | N | N | Y | Lower | Only narrative results reported |
| Kim 200998 | N | N | N | N | N | Lower | |
| Kim-Hwang 2010102 | N | N | N | N | N | Lower | |
| King 2001119 | N | N | N | N | N | Lower | |
| Knab 2001112 | N | N | N | N | N | Lower | |
| Knol 200690 | N | N | N | Y | Y | Higher | Reports both 53% and 51% reduction. Poor categorisation |
| Kousgaard 200329 | N | N | N | N | N | Lower | |
| Lam 201125 | N | N | Y | N | Y | Higher | 52.4% response rate. Only narrative results reported. Weak outcome measure |
| Leggett 200485 | N | N | N | N | N | Lower | |
| Leiba 2002130 | N | N | N | N | N | Lower | |
| Lester 200939 | N | N | Y | N | N | Lower | Attrition rate not reported |
| Levell 2012129 | Y | N | Y | N | N | Higher | Patient characteristics not reported. Attrition rate not reported. Descriptive |
| Lucassen 200145 | N | N | Y | N | N | Lower | Attrition rate not reported |
| Lyon 2009160 | N | N | Y | N | N | Lower | |
| Maddison 2004154 | Y | N | N | Y | Y | Higher | Patient characteristics not reported Only single % results reported |
| Magill 2009115 | N | N | N | N | N | Lower | |
| Malik 200741 | Y | N | N | Y | Y | Unclear | Patient characteristics not reported Only single % results reported |
| Mariotti 2008113 | N | N | N | N | N | Lower | |
| Matowe 200250 | N | N | N | N | N | Lower | |
| McGarry 2009148 | N | N | Y | N | N | Lower | 33% response rate |
| McGowan 2008107 | N | N | N | N | N | Lower | |
| McKoy 200489 | N | N | N | N | Y | Lower | Only single % results reported |
| McNally 200374 | N | N | N | N | N | Lower | |
| Melia 200851 | N | N | N | N | N | Lower | |
| Morrison 200164 | N | N | N | N | N | Lower | |
| Nicholson 200697 | Y | N | N | Y | N | Higher | Very small sample, survey not piloted, questions strongly leading to the positive |
| Nielsen 200392 | N | N | N | N | N | Lower | |
| Patterson 2004104 | N | N | N | N | Y | Lower | Only narrative results reported |
| Pfeiffer 2011138 | N | N | N | N | N | Lower | |
| Potter 200746 | N | N | N | N | N | Lower | |
| Prades 201175 | N | N | N | Y | Y | Lower | |
| Ramsay 200327 | N | N | N | N | N | Lower | |
| Ridsdale 2008124 | N | N | N | N | N | Lower | |
| Robling 200260 | N | N | N | N | N | Lower | |
| Rosen 2006128 | N | N | N | N | N | Lower | |
| Rowlands 200326 | Y | N | N | Y | N | Higher | Low recruitment. Tool leading and not tested |
| Salisbury 2005125 | N | N | N | N | N | Lower | |
| Sanderson 2002126 | N | N | N | N | N | Lower | |
| Sauro 2005127 | Y | N | N | N | N | Lower | Not clear if allocation was random |
| Schillinger 2000144 | N | N | Y | N | N | Lower | Attrition rate not reported |
| Schulpen 2003134 | N | N | Y | N | N | Lower | Attrition rate not reported |
| Shariff 201028 | Y | N | N | N | N | Lower | Patient characteristics not reported |
| Shaw 200677 | N | N | N | N | N | Lower | |
| Simpson 2003143 | Y | N | N | N | Y | Higher | No participant details. Only narrative results reported. Weak outcome measures |
| Simpson 201078 | N | N | Y | N | N | Lower | 23% return rate for GP questionnaires. Weak outcome measures |
| Slade 2008117 | N | N | N | N | N | Lower | |
| Spatafora 200569 | N | N | N | N | N | Lower | |
| Stainkey 2010118 | N | N | N | N | Y | Lower | Only single % results reported |
| Standing 2001122 | N | N | N | N | Y | Lower | Only single % results reported |
| Stoves 2010103 | N | N | N | N | N | Lower | |
| Suris 200735 | N | N | N | N | N | Lower | |
| Sved-Williams 201072 | N | N | Y | Y | N | Higher | 26% of GPs responded to survey. Weak outcome measures |
| Tadros 200996 | N | N | N | N | Y | Lower | Only single % results reported |
| Tan 2007140 | Y | N | N | Y | Y | Higher | Patient characteristics not reported Data presented in chart form only |
| Thomas 200379 | N | N | N | N | Y | Lower | Only single % results reported |
| Thomas 201080 | N | N | N | N | N | Lower | |
| Tierney 2003116 | N | N | N | N | N | Lower | |
| Twomey 200342 | Y | N | N | N | Y | Higher | Patient characteristics not reported Only single % results reported |
| van Bokhoven 2012120 | N | N | N | N | N | Lower | |
| Van Dijk 2013149 | N | N | N | N | Y | Lower | Only narrative results reported |
| Van Dijk 2010141 | N | N | N | N | N | Lower | |
| Van Dijk 2011123 | N | N | N | N | N | Lower | |
| Vardy 2008150 | N | N | N | N | Y | Lower | Only single % results reported |
| Vlek 2003135 | N | N | N | N | N | Lower | |
| Walkowski 200763 | N | N | N | N | Y | Lower | Only single % results reported |
| Wallace 200494 | N | N | N | N | N | Lower | |
| Watson 200132 | N | N | N | N | N | Lower | |
| Watson 2002152 | N | N | N | N | Y | Lower | Only single % results reported |
| West 200752 | Y | N | N | N | N | Lower | Patient characteristics not reported |
| White 2000142 | Y | N | N | N | N | Lower | Patient characteristics not reported |
| White 200461 | N | N | N | N | Y | Lower | Only narrative results reported |
| Whited 200287 | N | N | N | N | N | Lower | |
| Whited 200488 | N | N | N | N | N | Lower | |
| Whiting 2011153 | Y | N | N | Y | Y | Higher | Patient characteristics not reported. Study design unclear. Only single % results |
| Wilson 200665 | N | N | N | N | N | Lower | |
| Wolters 200536 | N | N | N | N | N | Lower | |
| Wong 200081 | N | N | N | N | N | Lower | |
| Wright 200656 | N | N | N | N | N | Lower | |
| Wylie 200118 | N | N | N | N | Y | Lower | Only narrative results reported |
Quality appraisal: quantitative studies
| Potential risk of bias | Bias present? |
|---|---|
| 1. Selection bias: Method used to generate the allocation sequence, method used to conceal the allocation sequence (where applicable), characteristics of participant group(s) | Yes/no/unclear |
| 2. Performance bias: Measures used to blind participants and personnel and outcome assessors (where applicable), presence of other potential threats to validity | Yes/no/unclear |
| 3. Attrition bias: Incomplete outcome data, high level of withdrawals from the study | Yes/no/unclear |
| 4. Detection bias: Accuracy of measurement of outcomes, length of follow-up | Yes/no/unclear |
| 5. Reporting bias: Selective reporting, accuracy of reporting | Yes/no/unclear |
Table of quantitative studies
| First author and date | 1. Selection bias, Y/N | 2. Performance bias, Y/N | 3. Attrition bias, Y/N | 4. Detection bias, Y/N | 5. Reporting bias, Y/N | 6. Higher risk/lower risk/unclear | Details of concerns |
|---|---|---|---|---|---|---|---|
| Ache 2011277 | Y | N | N | Y | N | Higher | Practitioners working only in Mayo clinics; low number of African American participants; some questions leading |
| Ahluwalia 2009214 | Y | N | N | Y | N | Higher | Two Kaiser Permanente hospitals; administered to those who attended a meeting. Asked to recall practice during previous year |
| Albertson 2000270 | Y | N | N | N | Y | Lower | Conclusions need to more strongly highlight the very small sample of PCPs (12) compared with 856 patients. Single clinic in an academic medical centre |
| Alexander 2008308 | Y | N | Y | N | N | Higher | 26% survey response from one rural region |
| Angstman 2009184 | Y | N | N | Y | N | Higher | Survey tool not piloted; physicians in a single clinic |
| Anthony 2010208 | Y | N | N | Y | N | Higher | One area; snowball sampling; depression care questionnaire not tested; participants asked to recollect practice over previous year. Mixed methods include qualitative data |
| Ashworth 2002303 | Y | N | N | N | N | Lower | Practices in one area of London |
| Balduf 2008274 | N | N | N | N | N | Lower | |
| Barnett 2011222 | Y | N | N | N | N | Lower | Sample clinicians who were members of a particular organisation |
| Bederman 2010196 | N | N | Y | N | N | Lower | FP response rate 12% |
| Bekkelund 2001259 | N | N | N | N | N | Lower | |
| Belgamwar 2011197 | Y | N | Y | N | N | Lower | Referrals to a single centre |
| Berendsen 2010262 | N | N | N | N | N | Lower | |
| Bertakis 2001264 | Y | N | Y | N | N | Lower | Referrals to a single centre |
| Blundell 2011189 | N | N | N | N | N | Lower | Response rate 40% |
| Bolanos-Carmona 2002276 | Y | N | N | N | N | Lower | Convenience sample |
| Boulware 2006304 | N | N | Y | N | N | Lower | 28% response rate |
| Bowling 2000291 | N | N | N | N | N | Lower | |
| Burns 2002301 | N | N | N | N | N | Lower | |
| Carlsen 2008244 | Y | N | N | Y | N | Higher | GPs from one municipality; GPs distributed questionnaire to their patients; GP self-report number of referrals |
| Chan et al. 2003275 | N | N | N | N | N | Lower | |
| Chauhan et al. 2012284 | Y | N | N | N | N | Lower | Data from two PCTs |
| Chen et al. 2005286 | Y | N | N | N | N | Lower | Nine clinics run by one provider |
| Chung et al. 2010305 | N | N | N | N | N | Lower | |
| Clarke et al. 2010190 | Y | N | Y | Unsure | N | Higher | GP response rate 13%; survey not piloted? Scenario development process unclear; compares findings of a guideline development group to the survey |
| Cohen 2013267 | N | N | N | N | N | Lower | |
| Cooper 2001279 | Y | N | N | N | N | Lower | Case notes from clinic in two hospitals |
| Coulston 2008187 | N | N | N | N | Y | Lower | Limited analysis |
| Coyle 2011280 | Y | N | N | N | N | Lower | Small sample of GPs from one county |
| Dale 2000224 | Y | N | N | Y | Y | Higher | Small sample size of GPs referring to one centre. Methods unclear (e.g. ‘postal interview’). Table of demographics but limited other data |
| Dearman 2006265 | N | N | N | N | N | Lower | |
| Delva 2011161 | N | N | Y | N | N | Lower | 30% response rate |
| Dodds 2004174 | Y | N | N | N | N | Lower | Data from single PCT |
| Elhayany 2000167 | Y | N | N | N | N | Lower | Data from one district |
| Feeney 2007282 | Y | N | N | N | N | Lower | Referrals to a single centre |
| Forrest 2007246 | N | N | N | N | N | Lower | |
| Forrest 2002223 | N | N | N | N | N | Lower | |
| Forrest 2003219 | N | N | N | N | N | Lower | |
| Forrest 2006283 | N | N | N | N | N | Lower | |
| Franks 2000216 | Y | N | N | N | N | Lower | One managed care organisation. Ceiling effect on some measured items |
| Franz 2010307 | Y | N | N | N | N | Lower | |
| Freed 2003172 | Y | N | N | N | N | Lower | Low response from family physicians |
| Fucito 2003162 | Y | N | N | Y | N | Higher | GPs in Sydney; survey and vignette not piloted; self-reported knowledge/skills |
| Gandhi 2000101 | Y | N | N | N | N | Lower | Data from single academic tertiary care medical centre |
| Glozier 2007254 | Y | N | N | N | N | Lower | Referrals to clinic in one trust |
| Grace 2008292 | N | N | Y | N | N | Lower | Low response rate |
| Green 2008220 | N | N | N | Y | N | Lower | Vignette not piloted |
| Greer 2011240 | N | N | Y | N | N | Lower | Case scenarios not piloted |
| Gross 2000261 | N | N | N | N | N | Lower | |
| Gruen 2002281 | N | N | N | N | N | Lower | |
| Guevara 2009290 | N | N | N | N | N | Lower | |
| Harlan 2009231 | Y | N | N | N | N | Lower | Data from small area; telephone interviews |
| Harris 2011268 | Y | N | N | N | N | Lower | Practices in Sydney area |
| Harvey 2005242 | Y | N | N | N | N | Lower | Physicians in a single city |
| Hugo 2000272 | Y | N | N | N | N | Lower | Data from a single clinic, although referrers from wider area |
| Hyman 2001248 | N | N | N | Y | N | Lower | Limited reporting |
| Jiwa 2008193 | N | N | N | N | N | Lower | |
| Jiwa 2009241 | Y | N | N | N | N | Lower | Little detail of the specialists rating the letters apart from that they offered to take part |
| Johnson 2008271 | N | N | Y | Y | Y | Higher | Response rate 13%; no examination of non-responders. Description of development of tool but not piloted; significant use of ‘unsure’ category. Use of descriptive stats; possible overstatement of findings |
| Johnson 2011245 | N | N | N | N | N | Lower | 40% response |
| Johnson 2011289 | N | N | Y | N | N | Lower | 31% response rate, only location comparison made between responders and non responders |
| Jorgensen 2001181 | N | N | N | N | N | Lower | |
| Kasje 2004191 | Y | N | N | N | N | Lower | Single area, north Netherlands |
| Kier 2012294 | Y | N | N | N | N | Lower | Selection process of GPs contacted not clear; no details of questions asked |
| Kim 200998 | Y | N | N | N | N | Lower | Referrals to one hospital; web-based survey |
| Kinchen 2004238 | N | N | N | N | N | Lower | |
| Kisely 2002185 | Y | N | N | Y | N | Higher | This study is a partial evaluation of an intervention; however, it is predominantly a survey of practitioners. Survey tool not tested; self-reported knowledge levels |
| Kvaerner 2007168 | N | N | N | Y | N | Lower | GPs asked to retrospectively estimate referrals |
| Lakha 2011179 | Y | N | N | N | N | Lower | Data from a single clinic. Low response rate but assessed any difference between responders and non-responders |
| Lambert 2001175 | Y | N | N | N | N | Lower | Data from single city |
| Lewis 2000260 | Y | N | N | N | N | Lower | Authors highlight potential selection bias |
| Little 2004251 | N | N | N | N | N | Lower | Practices around one city; however, compared with national data |
| Love 2005302 | N | N | N | N | N | Lower | |
| Malcolm 2008299 | Y | N | N | N | N | Lower | Clinics in one city; 39% response rate |
| Massey 2004236 | Y | N | N | Y | N | Higher | Questionnaire tool has weaknesses in wording and method for design of tool not reported; sample size small; limited detail of recruitment |
| McBride 2010287 | N | N | N | N | N | Lower | |
| McKenna 2005225 | N | N | N | Y | N | Lower | Survey and vignettes not piloted; self-reported knowledge levels |
| Mitchell 2012186 | Y | N | Y | N | N | Higher | Mixed methods, but few qualitative data. Low response rate; dietitians in private practice |
| Montgomery 2006180 | N | N | N | Y | N | Lower | Case scenarios not piloted |
| Moore 2000205 | Y | N | Y | Y | Y | Higher | PCPs in one state; 11.6% response rate; self-reported estimate referral rate; some questions leading; findings overstated? |
| Morsi 2012200 | Y | N | N | N | N | Lower | Three hospitals in one area |
| Mulvaney 2005297 | N | N | N | N | N | Lower | |
| Musila 2011255 | Y | N | N | Y | N | Higher | Work carried out by members of a guideline development group; case scenarios developed from discussion and not tested prior to use |
| Naccarella 2008169 | Y | N | N | N | N | Lower | Survey of project officers taking part in a project |
| Navaneethan 2010278 | Y | N | N | N | N | Lower | 25% survey response; physicians from a single area |
| O’Byrne 2010243 | N | N | N | N | N | Lower | |
| Sullivan 2005285 | N | N | N | Y | N | Lower | GPs entered own coded data |
| O’Neill 2005170 | N | N | N | Unclear | N | Lower | Level of testing of clinical vignettes unclear (some excluded from the analysis) |
| Olson 2012206 | N | N | Y | N | N | Lower | 33% response rate; no examination of non-responders |
| Pfeiffer 2011138 | Y | N | N | N | N | Lower | Veterans Health Administration facilities only |
| Philichi 2010202 | Y | N | Y | Y | N | Higher | Convenience sample sent survey, 38% response rate; survey not piloted |
| Pryor 2001215 | Y | N | N | N | N | Lower | Purposive sampling to increase response rate |
| Ramanathan 2011188 | N | N | N | Y | N | Lower | Case vignettes not tested |
| Ramchandani 2002247 | Y | N | N | N | N | Lower | Data from single city |
| Ridsdale 2007266 | N | N | N | N | N | Lower | |
| Ringard 2010164 | N | N | N | N | N | Lower | |
| Robinson 2010288 | Y | N | N | N | N | Lower | Referrals to a single service |
| Rosemann 2005211 | Y | N | N | N | N | Lower | GPs selected from single rural region |
| Rushton 2002183 | N | N | N | N | N | Lower | |
| Samant 2007234 | N | N | N | Y | N | Lower | Survey tool not tested prior to use |
| Scheerers 2007 | Y | N | N | N | N | Lower | Referrers to a single centre |
| Shadd 2011263 | Y | N | N | N | N | Lower | PCPs in one state |
| Soerensen 2009298 | N | N | N | N | N | Lower | |
| Steele 2012203 | N | N | Y | Y | Y | Higher | Response rate 24.9%; no examination of non-responders. Tool described as exploratory, not piloted. Descriptive rather than statistical reporting of findings |
| Swarztrauber 2002171 | N | N | N | Unclear | N | Lower | Survey piloted but unsure development and testing of scenarios |
| Taggarshe 2006233 | Y | N | N | N | Unsure | Higher | Survey distributed only to those attending a meeting. Very little detail regarding findings |
| Todman 2011295 | Y | N | N | Y | N | Higher | Single area; limited pilot testing; self-reported retrospective number of referrals |
| Townsley 2003165 | Y | N | Y | N | N | Higher | Physicians in one state; low response rate |
| Trude 2003199 | N | N | N | N | N | Lower | |
| Tucker 2003198 | N | N | N | N | N | Lower | Mixed method study; rural practices across Scotland; rural key area of investigation in study, however |
| Tzaribachev 2009173 | Y | N | N | N | N | Lower | Patients referred to a single centre |
| Vinker 2007269 | Y | N | N | N | N | Lower | Data relates to a single community clinic |
| Vulto 2009250 | N | N | N | Y | N | Lower | Self-reported GP knowledge levels |
| Wakefield 2012227 | Y | N | Y | N | N | Higher | One region, 19% response rate; however, demographics compared with another study |
| Walders 2003300 | N | N | N | N | N | Lower | |
| Wassenaar 2007178 | Y | N | N | N | N | Lower | Single state |
| Watson 2001b195 | N | N | N | N | N | Lower | Referrals to a single centre |
| Xu 2002235 | N | N | N | Y | N | Lower | Use of subjective measure |
| Zielinski 2008166 | Y | N | N | N | N | Lower | Data from one city and region |
Quality appraisal: qualitative studies
| Potential risk of bias | Bias present? |
|---|---|
| 1. Was there a clear statement of the aim of the research? | Yes/no |
| 2. Is a qualitative methodology appropriate to address the aims of the research? | Yes/no |
| 3. Was the recruitment strategy appropriate to the aims of the research? | Yes/no/unclear |
| 4. Were the data collected in a way that addressed the research issue? | Yes/no/unclear |
| 5. Has the relationship between researcher and participant been adequately considered? | Yes/no |
| 6. Have ethical issues been taken into account? | Yes/no/unclear |
| 7. Was the data analysis sufficiently rigorous? | Yes/no |
| 8. Is there a clear statement of findings? | Yes/no |
Table of qualitative studies
| First author and date | 1. Was there a clear statement of the aim of the research? (Y/N) | 2. Is a qualitative methodology appropriate to address the aims of the research? (Y/N) | 3. Was the recruitment strategy appropriate to the aims of the research? (Y/N/unclear) | 4. Were the data collected in a way that addressed the research issue? (Y/N/unclear) | 5. Has the relationship between researcher and participant been adequately considered? (Y/N) | 6. Have ethical issues been taken into account? (Y/N/unclear) | 7. Was the data analysis sufficiently rigorous? (Y/N) | 8. Is there a clear statement of findings? (Y/N) | Higher/lower risk of bias; details of concerns |
|---|---|---|---|---|---|---|---|---|---|
| Abel and Thompson 201120 | N | Y | Unclear | Y | Y | Y | N | N | Higher Aims not fully clear Process for selection of participants unclear Data not clearly distinguished from report of other authors’ work |
| Allareddy 2007232 | Y | Y | N | Y | N | N | Y | Y | Lower Detail of interviewer not provided, convenience sample, lack of discussion of ethics relating to focus groups |
| Baker 2006192 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Beel 2008226 | Y | Y | Y | Y | N | Y | Y | Y | Lower |
| Berendsen 2007237 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Blundell 2012253 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Bowling 2006273 | Y | Y | Y | Y | Y | Y | Y | Y | Lower Mixed-methods study; however, qualitative data predominate |
| Brien 2008258 | Y | Y | Y | Y | N | Y | Y | Y | Lower No details of interviewer |
| Bruyninckx 2009209 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Calnan 2007207 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Chew-Graham 2008229 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Clemence 2003228 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Cornford 2004218 | Y | Y | Y | Y | N | N | Y | Y | Lower Ethical issues regarding discarding potential participant replies, patients unaware of intervention. Interviewed by professional involved in service |
| Dagneaux 2012230 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Davies 2007256 | Y | Y | Y | Y | Y | Y | Y | Y | Lower Mixed method; qualitative data predominates |
| Edwards 2002257 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Espeland 2003217 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Greenaway 2006306 | Y | Y | Y | Y | Y | Y | Y | Y | Lower Survey with free text so limited qualitative data |
| Holley 2010293 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Knight 2003177 | Y | Y | Y | Y | Y | N | Y | Y | Lower Reporting of methods potentially identifies participants |
| Morgan 2007210 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Nandy 2001204 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Pomeroy 2010176 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Rosen 2007252 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Rowlands 2001182 | Y | Y | Y | Y | Y | Y | Y | Y | Lower Data part of RCT |
| Ruston 2004194 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Sigel and Leiper 2004201 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Stavrou 2009249 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Thorsen 2012239 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Van der Weijden 2002212 | Y | Y | Y | Y | Y | Y | Y | Y | Lower |
| Wilkes 2009213 | Y | Y | Y | Y | N | Y | Y | Y | Lower Detail of interviewer not provided |
| Young 2010221 | Y | Y | N | Y | Y | Y | Y | Y | Lower GPs had already taken part in an earlier study |
Appendix 3 Data sources
MEDLINE In-Process & Other Non-Indexed Citations and MEDLINE via Ovid (1946 to present).
Cochrane Database of Systematic Reviews.
Cochrane Central Register of Controlled Trials.
Cochrane Methodology Register.
Database of Abstracts of Reviews of Effects.
Health Technology Assessment Database.
NHS Economic Evaluation Database.
All accessed via The Cochrane Library, published by John Wiley and Sons Ltd (from inception).
CINAHL via EBSCO (from inception).
EMBASE via Ovid (1974 to 13 November 2012).
PsycINFO via Ovid (1806 to week 1 November 2012).
Science Citation Index, Social Science Citation Index, Science Conference Papers Index, Social Science Conference Papers Index via Web of Science published by Thomson Reuters (from inception).
Scopus via Elsevier (from inception).
Applied Social Sciences Index and Abstracts (ASSIA) via ProQuest (from inception).
Sociological abstracts via ProQuest (from inception).
Social Policy and Practice via Ovid (1890s to October 2012).
International Bibliography of the Social Sciences (IBSS) via Proquest (from inception).
HMIC via NHS Evidence (from inception).
Health Business Elite via EBSCO (from inception).
Business Source Premier via EBSCO (from inception).
Emerald Management Reviews via www.emeraldinsight.com/products/reviews/index.htm (from inception).
EPPI Centre databases: Bibliomap, Database of Promoting Health Effectiveness Reviews (DoPHER), Trials Register of Promoting Health Interventions (TRoPHI) via http://eppi.ioe.ac.uk/ (from inception).
OpenGrey via www.opengrey.eu/ (from inception).
Opensource via www.greynet.org/greysourceindex.html (from inception).
Google Scholar via scholar.google.co.uk/ (from inception).
Appendix 4 Search strategies
Initial search
Database: Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations and Ovid MEDLINE(R).
Date searched: 1946 to present.
Date of search: November 2012.
Search strategy:
-
*Primary Health Care/(31,226)
-
(primary care or general practitioner$or gp).ti. (38,162)
-
*Family practice/or *General practitioners/(38,225)
-
1 or 2 or 3 (83,924)
-
(referral or referred or refer).ti. (10,316)
-
demand management.ti,ab. (141)
-
*’Referral and Consultation’/(17,682)
-
Specialisation/(20,898)
-
5 or 6 or 7 or 8 (43,885)
-
4 and 9 (4328)
-
limit 10 to yr = ‘2000-Current’ (1978)
Phrase search
Database: Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations and Ovid MEDLINE(R).
Date searched: 1946 to present.
Date of search: March 2013.
Search strategy:
-
‘referral management centre$’.mp. [mp = title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept, rare disease supplementary concept, unique identifier] (7)
-
limit 1 to yr = ‘2000-Current’ (7)
‘Clinical reasoning’ search
Database: Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations and Ovid MEDLINE(R).
Date searched: 1946 to present.
Date of search: April 2013.
Search strategy:
-
(evidence based adj (practice or medicine)).ti,ab. (12,476)
-
Evidence-Based Practice/or Evidence-Based Medicine/(52,802)
-
1 or 2 (57,837)
-
(gp or general practitioner).ti,ab. (36,865)
-
General Practice/or Family Practice/or General Practitioners/(62,491)
-
4 or 5 (90,428)
-
(clinical reasoning or clinical judgement or problem solving or decision making or critical thinking).ti,ab. (73,412)
-
Clinical Competence/(60,196)
-
Problem Solving/(20,285)
-
7 or 8 or 9 (145,906)
-
3 and 6 and 10 (170)
‘Systems’ search
Database: Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations and Ovid MEDLINE(R).
Date searched: 1946 to present.
Date of search: April 2013.
Search strategy:
-
referral.ti,ab. (54,436)
-
exp ‘Referral and Consultation’/(54,186)
-
1 or 2 (96,206)
-
*’Appointments and Schedules’/(3226)
-
*Computer Systems/(3932)
-
*Information Systems/(11,904)
-
*Physician Incentive Plans/(1225)
-
(Proforma$or checklist$or appointment$or direct access or IT system* or informatics or software or electronic or outreach clinic$or specialist nurse$or patient review$or gatekeep$or nominated destination or financial incentive$or payment$).ti,ab. (233,486)
-
4 or 5 or 6 or 7 or 8 (249,981)
-
3 and 9 (5345)
-
((doctor$or gp$or general practitioner$or physician$) adj3 (attitude$or behavio?r or knowledge)).ti,ab. (9324)
-
(patient$adj3 (attitude$or behavio?r or knowledge)).ti,ab. (19,104)
-
exp *attitude to health/or *health knowledge, attitudes, practice/(136,183)
-
11 or 12 or 13 (158,409)
-
10 and 14 (574)
Appendix 5 Excluded studies
| Full paper excluded | Reason |
|---|---|
| Abraham AR. General practitioner’s back pain referrals: easing the load? J Integr Care Pathways 2001;5:133–5 | Clinical care focus |
| Abu-Ramadan MA. Making better use of scarce resources: the Palestinian experience, 1995–1999. J Ambulatory Care Manage 2002;25:63 | Country |
| Agarwal A, Charles-Holmes S. Out patient waiting time for common skin conditions – do general practitioners and dermatologists have the same priorities? A questionnaire-based survey. Clin Exp Dermatol 2001;26:13–15 | Clinical outcomes |
| Agyapong VIO. Shared care between specialised psychiatric services and primary care. Int J Psychiatry Med 2011;42:295–313 | Focus on diagnosis and care |
| Allen D, O’Brien T, Popert R. The two-week-wait cancer initiative in urology: useful modernisation? J R Soc Med 2004;97:279–81 | Clinical outcomes |
| Allgar VL, Neal RD, Ali N, Leese B, Heywood P, Proctor G, et al. Urgent GP referrals for suspected lung, colorectal, prostate and ovarian cancer. Br J Gen Pract 2006;56:355–62 | Clinical outcomes |
| Andrews KV, Penny JR, King PA. Are patients referred for NHS-funded dental implant treatment being selected in accordance with national guidelines and subsequently funded by their primary care trust? Ann R Coll Surg Engl 2010;92:512–14 | Dental, prevalence data |
| Anis AH, Guh DP, Lacaille D, Marra CA, Rashidi AA, Li X, et al. When patients have to pay a share of drug costs: effects on frequency of physician visits, hospital admissions and filling of prescriptions. CMAJ 2005;173:1335–40 | Effect on prescribing not referral |
| Annells M, Allen J, Nunn R, Lang L, Petrie E, Clark E, et al. An evaluation of a mental health screening and referral pathway for community nursing care: nurses’ and general practitioners’ perspectives. J Clin Nurs 2011;20:214–26 | Nursing referral to GP rather than primary to secondary |
| Anthony D. Changing the nature of physician referral relationships in the US: the impact of managed care. Soc Sci Med 2003;56:2033–44 | Background discursive paper |
| Anthony JS, Baik SY, Bowers BJ, Tidjani B, Jacobson CJ, Susman J, et al. Conditions that influence a primary care clinician’s decision to refer patients for depression care. Rehabil Nurs 2010;35:113–22 | Background discursive paper |
| Apostolopoulos DL. Completeness of referral details to rheumatologists from general practice. Poster presented at the Austin Life Sciences Research Week, Victoria, Australia, October 2010 | Conference paper |
| Argenziano G, Puig S, Zalaudek I, Sera F, Corona R, Alsina M, et al. Dermoscopy improves accuracy of primary care physicians to triage lesions suggestive of skin cancer. J Clin Oncol 2006;24:1877–82 | New intervention in primary care. Not about referral |
| Augestad KMR. The One-stop trial: does electronic referral and booking by the general practitioner (GPs) to outpatient day case surgery reduce waiting time and costs? A randomised controlled trial protocol. BMC Surg 2008;8:14 | Looks at reducing waiting times, not referral |
| Auladell MA, Caballeria L, Pera G, Rodriguez L, Casas JD, Aznar J, et al. Adequacy and quality of abdominal echographies requested by primary care professionals. BMC Gastroenterol 2010;10:101 | Study protocol |
| Carlsen B, Norheim OF. Introduction of the patient-list system in general practice. Changes in Norwegian physicians’ perception of their gatekeeper role. Scand J PrimHealth Care 2003;21:209–13 | Practitioner awareness of gatekeeper role, not referrals |
| Bal Gl, Sellier E, Gennai Sp, Caillis M, François P, Pavese P. Infectious disease specialist telephone consultations requested by general practitioners. Int J Psychiatry Med 2011;43:912–17 | Discussion of the technology used (technical features) |
| Bal R, Mastboom F, Spiers HP, Rutten H. The product and process of referral: optimising general practitioner-medical specialist interaction through information technology. Int J Med Inf 2007;76(Suppl. 1):28–34 | Discussion of the technology used (technical features) |
| Barnes JJP. Why do general dental practitioners refer to a specific specialist endodontist in practice? Int Endod J 2011;44:21–32 | Clinical outcomes (conditions referred) |
| Bassi A, Sturgess R, Bodger K. Impact of a rapid access upper GI cancer service (RAUGICS) on direct referrals for endoscopy from primary care. Gastroenterology 2002;122:A485–6 | Audit of resource use |
| Baughan PK. Urgent suspected cancer referrals from general practice: audit of compliance with guidelines and referral outcomes. Br J Gen Pract 2011;61:e700–6 | Clinical outcomes (types of cancer referred) |
| Baumeister T, Weistenhöfer W, Drexler H, Kütting B. Prevention of work-related skin diseases: teledermatology as an alternative approach in occupational screenings. Contact Dermatitis 2009;61:224–30 | Not primary care |
| Becker A, Leonhardt C, Kochen MM, Keller S, Wegscheider K, Baum E, et al. Effects of two guideline implementation strategies on patient outcomes in primary care – a cluster randomised controlled trial. Spine (Phila Pa 1976) 2008;33:473–80 | Primary care only, not referral |
| Ben-Arye E, Frenkel M. Referring to complementary and alternative medicine – a possible tool for implementation. Complement Ther Med 2008;16:325–30 | Validation of tool for referral to CAM |
| Benjamin C, Booth K. Integrating care across traditional service boundaries; problems encountered during referral from primary care to cancer services for a family history of breast cancer. Psychooncology 2002;11:547 | Conference abstract |
| Benjamin C, Ellis I. Realisation of risk: a central process involved in initiating referral from primary care due to a family history of breast cancer. J Medical Genetics 2005;42:S124 | Conference abstract |
| Berendsen AJ, Benneker WH, Schuling J, Rijkers-Koorn N, Slaets JP, Meyboom-de JB, et al. Collaboration with general practitioners: preferences of medical specialists--a qualitative study. BMC Health Serv Res 2006;6:155 | Describes specialist views of GPs |
| Berendsen AJ, de Jong GM, Schuling J, Bosveld HE, de Waal MW, Mitchell GK, et al. Patient’s need for choice and information across the interface between primary and secondary care: a survey. Patient Educ Couns 2010;79:100–5 | Hospital choice |
| Berendsen AJ, Kuiken A, Benneker W, de Jong BM, Voorn T, Schuling J. How do general practitioners and specialists value their mutual communication? A survey. BMCHealth Serv Res 2009;9:143 | Clinical outcomes |
| Bhatt R, Rajesh A, Morgan B, Finlay D, Bhatt R, Rajesh A, et al. An audit of hip radiographs performed for general practitioners. Clin Radiol 2001;56:970–2 | Referral rates |
| Boggis AR, Cornford CS. General Practitioners with special clinical interests: a qualitative study of the views of doctors, health managers and patients. Health Policy 2007;80:172–8 | Does not consider referral |
| Borgermans L, Goderis G, Van Den Broeke C, Verbeke G, Carbonez A, Ivanova A, et al. Interdisciplinary diabetes care teams operating on the interface between primary and specialty care are associated with improved outcomes of care: findings from the Leuven Diabetes Project. BMC Health Serv Res 2009;9:179 | Intervention to improve GP clinical management of patients rather than onward referral |
| Bratton RL, Cody C. Telemedicine applications in primary care, a geriatric pilot project. Mayo Clin Proc 2000;75:365–8 | Primary care only. Not about referral |
| Brealey SD. Influence of magnetic resonance of the knee on GPs’ decisions: a randomised trial. Br J Gen Pract 2007;57:622–9 | Clinical outcomes not about improving referral |
| Brez S.Rowan. Transition from specialist to primary diabetes care: a qualitative study of perspectives of primary care physicians. BMC Fam Pract 2009;10:39 | Relates to care after the referral and specialist appointment |
| Butler R, Oyewole D, Pitt B. What is the relationship between general practitioners’ community referrals, and hospital referrals to an old age psychiatric service? Ageing Ment Health 2000;4:79–81 | Describes differences in referral rates but does not explore factors |
| Campbell KH, Dale W, Stocking CB, Hemmerich JA, Smith SG, Sachs GA. Primary care physician referral decisions for older patients with chronic kidney disease: a pilot study. J Am Geriatr Soc 2007;55:S34 | Effect of subspecialty of referrer on referral rates |
| Cape J, Parham A. Rated casemix of general practitioner referrals to practice counsellors and clinical psychologists: a retrospective survey of a year’s caseload. Br J Med Psychol 2001;74:237–46 | Clinical factors |
| Capper RC. Is there agreement among general practitioners, paediatricians and otolaryngologists about the management of children with recurrent tonsillitis? Clin Otolaryngol Allied Sci 2001;26:371–8 | Clinical factors |
| Cardiff Uni. Evaluation of Referral Management Pilots in Wales. URL: www.wales.nhs.uk/sitesplus/documents/829/Referral%20Management%20Pilots%20in%20Wales%20-%20Follow%20Up%20Review%20Final%20Report.PDF (accessed 16 January 2014) | Describes setting up of RMC, no data on impact on referral, etc. |
| Carey T, Mullan R. GP referral guidelines: is there another side to the coin? Clin Psychol Forum 2007;170:29–32 | Discussion paper |
| Carli P, De Giorgi V, Crocetti E, Caldini L, Ressel C, Giannotti B. Diagnostic and referral accuracy of family doctors in melanoma screening: effect of a short formal training. Eur J Cancer Prev 2005;14:51–5 | Improving clinical skills of doctors |
| Chan B, Proudfoot J, Zwar N, Davies GP, Harris MF. Satisfaction with referral relationships between GP and allied health professionals. Aust J Prim Health 2011;17:250–8 | Validation of tool |
| Checkland K, Coleman A, Harrison S, Hiroeh U. Practice Based Commissioning in the National Health Service: Interim Report of a Qualitative Study. 2008. URL: www.population-health.manchester.ac.uk/primarycare/npcrdc-archive/Publications/PBC_INTERIM_REPORT.pdf (accessed 16 January 2014) | List of interventions only (pp. 52–3) |
| Chen AH, Yee HF Jr. Improving primary care-specialty care communication: lessons from San Francisco’s safety net: comment on ‘Referral and consultation communication between primary care and specialist physicians’. Arch Intern Med 2011;171:65–7 | Commentary on O’Malley paper on communication systems. Useful references on electronic referral systems – have printed |
| Chenot JF, Scherer M, Becker A, Donner-Banzhoff N, Baum E, Leonhardt C, et al. Acceptance and perceived barriers of implementing a guideline for managing low back in general practice. Implement Sci 2008;3:7 | Guidlines for low-back pain in primary care – not about referral |
| Chenot J-FL. The impact of specialist care for low back pain on health service utilisation in primary care patients: a prospective cohort study. Eur J Pain 2008;12:275–83 | Clinical characteristics of patients referred |
| Chew GC, Dowrick C, Wearden A, Richardson V, Peters S. Making the diagnosis of Chronic Fatigue Syndrome/Myalgic Encephalitis in primary care: a qualitative study. BMC Fam Pract 2010;11:16 | Diagnosis and clinical care only |
| Clarkson JE, Turner S, Grimshaw JM, Ramsay CR, Johnston M, Scott A, et al. Changing clinicians’ behaviour: a randomised controlled trial of fees and education. J Dent Res 2008;87:640–4 | Impact of cost on dental procedures |
| Coady DA, Abdullah S, Mangion P, Chuck A. Missing out the middle man . . . direct access GP referrals for carpal tunnel tests. Rheumatology (Oxford) 2005;44:I59 | Conference abstract |
| Coast J, Noble S, Noble A, Horrocks S, Asim O, Peters TJ, et al. Economic evaluation of a general practitioner with special interests led dermatology service in primary care. BMJ 2005;331:1444–9 | Cost only, no referral outcomes |
| Collins K, Walters W, Bowns I. Patient satisfaction with teledermatology. J Telemed Telecare 2004;10:29–33 | Change to secondary clinical care |
| Corbett M, Foster N, Ong BN. GP attitudes and self-reported behaviour in primary care consultations for low back pain. Fam Pract 2009;26:359–64 | Guidelines relating to clinical care of back pain rather than referral |
| Corey K. An analysis of terminology used by primary care physicians to describe concerning lesions referred to an urgent dermatology clinic. Presentation at the Triennial Research Conference, Marbella, Spain, November 2012 | Exclude: conference abstract |
| Coulston JE, Williams GL, Stephenson BM, Coulston JE, Williams GL, Stephenson BM. Audit of referral patterns for hernia repair – are general practitioners aware of the changing face of herniorrhaphy? Ann R Coll Surg Engl 2008;90:140–1 | Clinical focus |
| Cressey D. Simple tool can aid GPs’ stroke referral. Pulse 2006;66:20 | Tool validation |
| Crimlisk HL, Bhatia KP, Cope H, David AS, Marsden D, Ron MA. Patterns of referral in patients with medically unexplained motor symptoms. J Psychosom Res 2000;49:217–19 | Clinical focus |
| Daggett P. Referral management centres and diabetes. Practical Diabetes International 2007;24:119–20 | Discussion article |
| DAMASK (Direct Access to Magnetic Resonance Imaging: Assessment for Suspect Knees) Trial Team. Effectiveness of GP access to magnetic resonance imaging of the knee: a randomised trial. Br J Gen Pract 2008;58:e1–8 | Clinical rather than service outcomes |
| Davenport TE, Watts HG, Kulig K, Resnik C. Current status and correlates of physicians’ referral diagnoses for physical therapy. J Orthop Sports Phys Ther 2005;35:572–9 | Information content of referral letters |
| Davey CJ, Green C, Elliott DB, Davey CJ, Green C, Elliott DB. Assessment of referrals to the hospital eye service by optometrists and GPs in Bradford and Airedale. Ophthalmic Physiol Opt 2011;31:23–8 | Clinical characteristics of referrals |
| De Bondt BA, Aartman IH, Zentner A. Referral patterns of Dutch general dental practitioners to orthodontic specialists. Eur J Orthod 2010;32:548–54 | Rates of referral only |
| Del Poggio P, Jamoletti C, Iazzetti M, Filippi A, Mattiello M, Mazzoleni M, et al. Management and referral patterns of patients with hepatitis C by primary care physicians: impact of an educational programme. J Hepatol 2000;32:178 | Conference abstract, no data |
| Della Rossa AN. Diagnosis and referral of rheumatoid arthritis by primary care physician: Results of a pilot study on the city of Pisa, Italy. Clin Rheumatol 2010;29:71–81 | Exclude: about disease prevalence/diagnosis, not referral |
| Delnoij D, Van Merode G, Paulus A, Groenewegen P. Does general practitioner gatekeeping curb health care expenditure? J Health Serv Res Policy 2000;5:22–6 | No referral outcomes |
| Devereux C, Eedy DJ. An audit of the implementation of a policy of not accepting GP referrals of benign cosmetic skin lesions. Br J Dermatol 2009;161:67 | No data, brief report (abstract) |
| Deweese D, Franklin MD. Managing referrals and referral specialists. Fam Pract Manag 2012;19:10 | Letter/comment only |
| Dorr DA, Wilcox A, McConnell KJ, Burns L, Brunker CP, Dorr DA, et al. Productivity enhancement for primary care providers using multicondition care management. Am J Manag Care 2007;13:22–8 | Efficiency of GP working practice rather than referral |
| Eccles M, Steen N, Grimshaw J, Thomas L, McNamee P, Soutter J, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet 2001;357:1406–9 | Primary care intervention, not about referral |
| Eide MJA. Web-based curriculum improves primary care providers’ skin cancer detection skills. Presentation at the Triennial Research Conference, Marbella, Spain, November 2012 | Abstract |
| Eisinger F, Pivot X, Coscas Y, Viguier J, Calazel-Benque A, Blay JY, et al. Impact of general practitioners’ sex and age on systematic recommendation for cancer screening. Eur J Cancer Prev 2011;20(Suppl. 1):39–41 | GP inviting patients for screening rather than referral |
| Enns SM, Muma RD, Lary MJ. Examining referral practices of primary care physician assistants. JAAPA 1986;13:81–4 | Not relevant to question |
| Evaluation of a GP referral service for manual treatment of back and neck pain. Clinical Chiropractic 2012;15:89–90 | Providing new GP service. Not about referral |
| Evans JW. Clinician opinions about the appropriateness and severity of general practitioner referrals to specialist mental health services: a cross-sectional survey. Primary Care Psychiatry 2002;8:91–4 | Describes issues in clinicians deciding appropriateness |
| Fauchais A, Demaziereb K, Ly K, Gondran G, Bezanahary H, Liozon E, et al. Interface between general practitioners and specialists in SSc management. Rheumatology 2012;51(Suppl. 2):ii98 | Exclude: conference abstract |
| Feldman S. The referral dance: improving the interface between primary care practitioners and specialists caring for patients with dementia. Am J Alzheimers Dis Other Demen 2009;23 | Audit of referral rates – no intervention |
| Fenton JJ, Levine MD, Mahoney LD, Heagerty PJ, Wagner EH, Fenton JJ, et al. Bringing geriatricians to the front lines: evaluation of a quality improvement intervention in primary care. J Am Board Fam Med 2006;19:331–9 | Clinical outcomes |
| Finch R. Specialist GPs ‘cut referrals by 80%’. Pulse 2004;64:9 | No data. But cites Pulse, 17 May 2004 |
| Fisher LW. Comparison of specialty referral patterns of primary care providers. J Healthc Manag 2002;47:197–204 | Acute interhospital referrals |
| Fitzgerald A, de Coster C, McMillan S, Naden R, Armstrong F, Barber A, et al. Relative urgency for referral from primary care to rheumatologists: the Priority Referral Score. Arthritis Care Res 2011;63:231–9 | Describes the development of a rating scale for referral criteria |
| Font B, Lahoz R, Casamor R, Escalada FJ, Ezjurra P, Ferrer JC, et al. Criteria for referral of type 2 diabetes patients from primary care to specialised care and vice versa in Spain. Pathways study. Value Health 2011;A483 | Exclude: conference abstract |
| Ford S, Schofield T, Hope T. Observing decision-making in the general practice consultation: who makes which decisions? Health Expect 2006;9:130–7 | Do parents believe they contribute to decision? Not linked to referral rate |
| Forrest B. Primary care gatekeeping and referrals: effective filter or failed experiment? BMJ 2003;326:692–5 | Discussion article |
| Forrest CB, Majeed A, Weiner JP, Carroll K, Bindman AB, Forrest CB, et al. Comparison of specialty referral rates in the United Kingdom and the United States: retrospective cohort analysis. BMJ 2002;325:370–1 | Describes patters of referrals only |
| Forrest CB, Majeed A, Weiner JP, Carroll K, Bindman AB. Referral of children to specialists in the US and UK. Arch Pediatr Adolesc Med 2003;157:279–85 | Comparison of referral rates, no intervention |
| Forrest CB, Reid RJ. Prevalence of health problems and primary care physicians’ specialty referral decisions. J Fam Pract 2001;50:427–32 | Types of health problem referred |
| French EM. Referral patterns of elderly persons to psychotherapy by primary care providers at a community health center. Clin Gerontol 2000;21 | Referrals within community |
| Freund T, Peters KF, Rochon J, Mahler C, Gensichen J, Erler A, et al. Primary care practice-based care management for chronically ill patients (PraCMan): study protocol for a cluster randomised controlled trial [ISRCTN56104508]. Trials 2011;12:163 | Study protocol only |
| Frost DW, Toubassi D, Detsky AS. Rethinking the consultation process: optimising collaboration between primary care physicians and specialists. Can Fam Physician 2012;58:825–8 | Discussion piece |
| Fuat A, Murphy JJ, Mehrzad AA, Johnston JI, Smellie WSA, Brennan G. Suspected heart failure in primary care – the utility of N-terminal pro b-type natriuretic peptide (NT proBNP) as a pre-screening test for secondary care referral – a real life study. Eur Heart J 2005;26:181 | Cost-effectiveness study of alternative methods of diagnosis heart failure in primary care |
| Gabel JR, Fahlman C, Kang R, Wozniak G, Kletke P, Hay JW. Where do I send thee? Does physician-ownership affect referral patterns to ambulatory surgery centers? Health Aff (Millwood) 2008;27:w165–74 | General trends in referral only |
| Gabel S. Innovations in Practice: child and adolescent psychiatrists and primary care – innovative models of consultation in the United States. Child Adolesc Ment Health 2012;17:252–5 | Treatment options |
| Garasen H, Johnsen R. The quality of communication about older patients between hospital physicians and general practitioners: a panel study assessment. BMC Health Serv Res 2007;7:133 | Exclude: quality of hospital discharge letters |
| Garrido T, Jamieson L, Zhou YU, Wiesenthal A, Liang L. Effect of electronic health records in ambulatory care. BMJ 2005;330:581 | Attendance in primary and secondary care, not about referral |
| Gask L, Dowrick C, Dixon C, Sutton C, Perry R, Torgerson D, et al. A pragmatic cluster randomised trial of an educational intervention for GPs in the assessment and management of depression. Psychol Med 2004;34:63–72 | Clinical outcomes |
| Gately S. E-referral and e-triage as mechanisms for enhancing and monitoring patient care across the primary-secondary provider interface. J Telemed Telecare 2003;9:350–2 | Letter |
| George S, Pockney P, Primrose J, Smith H, Little P, Kinley H, et al. A prospective randomised comparison of minor surgery in primary and secondary care. The MiSTIC trial. Health Technol Assess 2008;12(23) | Safety of minor operations performed in GP rather than hospital |
| Gillooly J, Tang V, Walker R. Assessing use of the two-week rule in urology. Br J Healthc Manage 2009;15:397–400 | Clinical outcomes |
| Gjessing KF. Exploring factors that affect hospital referral in rural settings: a case study from Norway. Rural Remote Health 2009;9:975 | Clinical diagnosis of patients referred |
| Gonzalez P. Gatekeeping versus direct-access when patient information matters. Health Econ 2010;19:730–54 | Gatekeeping and cost analysis |
| Goodey RD, Brickley MR, Hill CM, Shepherd JP. A controlled trial of three referral methods for patients with third molars. Br Dent J 2000;189:556–60 | Not hospital care |
| Gormley GJ, Steele WK, Gilliland A, Leggett P, Wright GD, Bell AL, et al. Can diagnostic triage by general practitioners or rheumatology nurses improve the positive predictive value of referrals to early arthritis clinics? Rheumatology (Oxford) 2003;42:763–8 | Clinical and diagnostic skills of staff |
| Burn K. GP initiative streamlines referrals. Pulse 2004;64:2–3 | Three-paragraph Pulse article. No useable data |
| Graffen M, Saligari M, Le Couteur DG, Naganathan V, McLean A, Graffen M, et al. Development of a mentorship programme for rural general practitioners by visiting specialist physicians. Rural Remote Health 2005;5:374 | Letter; no data |
| Granlund H, Thoden CJ, Carlson C, Harno K. Realtime teleconsultation versus face to face consultation in dermatology. J Telemed Telecare 2003;9:204–9 | Change to secondary clinical care |
| Grant C, Gallier L, Fahey T, Pearson N, Sarangi J, Grant C, et al. Management of menorrhagia in primary care-impact on referral and hysterectomy: data from the Somerset Morbidity Project. J Epidemiol Community Health 2000;54:709–13 | Clinical outcomes |
| Grant C, Goodenough T, Harvey I, Hine C. A randomised controlled trial and economic evaluation of a referrals facilitator between primary care and the voluntary sector. BMJ 2000;320:419–23 | Looks at the benefits of additional care on mental well-being |
| Grembowski DE, Martin D, Patrick DL, Diehr P, Katon W, Williams B, et al. Managed care, access to mental health specialists, and outcomes among primary care patients with depressive symptoms. J Gen Intern Med 2002;17:258–69 | Changed pattern of clinical care and access to mental health specialist rather than referral management, mental health outcomes |
| Grimshaw JM, Zwarenstein M, Tetroe JM, Godin G, Graham ID, Lemyre L, et al. Looking inside the black box: a theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the Ontario printed educational message, OPEM) to improve referral and prescribing practices in primary care in Ontario, Canada. Implement Sci 2007;2:38 | Study protocol only; no other papers found reporting data on Google search marked as in progress on project website |
| Gruen RL, Knox S, Britt H, Bailie RS. The Surgical Nosology in Primary-care Settings (SNIPS): a simple bridging classification for the interface between primary and specialist care. BMC Health ServRes 2001;4:8 | Development of a classification system – no evaluation of the system |
| Guarnaccia S, Lombardi A, Gaffurini A, Chiarini M, Domenighini S, D’Agata E, et al. Application and implementation of the GINA asthma guidelines by specialist and primary care physicians: a longitudinal follow-up study on 264 children. PrimCare Respir J 2007;16:357–62 | Describes clinical management of asthma rather than referral systems |
| Gucciardi E, Chan V, Fortugno M, Khan S, Horodezny S, Swartzack S. Primary care physician referral patterns to diabetes education programs in Southern Ontario, Canada. Can J Diabetes 2011;35:262–8 | Referral within primary care |
| Gulzar Z, Goff S, Njindou A, Hearty H, Rafi I, Savage R, et al. Nurse-led cancer genetics clinics in primary and secondary care in varied ethnic population areas: interaction with primary care to improve ascertainment of individuals from ethnic minorities. Fam Cancer 2007;6:205–12 | New service provision rather than demand management |
| Haggstrom DA, Phillips KA, Liang SY, Haas JS, Tye S, Kerlikowske K, et al. Variation in screening mammography and Papanicolaou smear by primary care physician specialty and gatekeeper plan (United States). Cancer Causes Control 2004;15:883–92 | Likelihood of GP doing the screening test rather than referral related |
| Hamilton W, Round A, Sharp D. Patient, hospital, and general practitioner characteristics associated with non-attendance: a cohort study. Br J Gen Pract 2002;52:317–9 | Looks at non-attendance at specialist |
| Harbison J, Davis J, Louw S, Sen B, Ford G. Comparison of stroke unit referrals by general practitioner (GP), emergency room (ER) doctors and ambulance paramedics using a rapid screening tool. Stroke 2000;31:2833 | Considers paramedics ability to diagnose stroke |
| Hartveit M, Biringer E, Vanhaeht K, Haug K, Aslaksen A. The Western Norway mental health interface study: a controlled intervention trial on referral letters between primary care and specialist mental health care. BMC Psychiatry 2011;11:177 | Study protocol only, looked for study papers presumably still in progress |
| Hirsch O, Trager S, Bosner S, Ilhan M, Becker A, Baum E, et al. Referral from primary to secondary care in Germany: Developing a taxonomy based on cluster analysis. Scand J Public Health 2012;40:571–8 | Taxonomy of referrals – types of referrals not reasons |
| Ho CKY. Improving timely access to GI specialty care using an electronic referral management system (Ereferral) that links primary care providers (PCPS) and a GI Specialist. Gastroenterology 2011:S721 | Conference abstract |
| Holbrook A, Pullenayegum E, Thabane L, Troyan S, Foster G, Keshavjee K, et al. Shared electronic vascular risk decision support in primary care. Arch Intern Med 2011;171:1736–44 | Clinical outcomes, not about referral |
| Holdsworth LK, Webster VS, McFadyen AK. The Scottish Physiotherapy Self Referral Study Group. Self-referral to physiotherapy: deprivation and geographical setting. Is there a relationship? Results of a national trial. Physiotherapy 2006;92:16–25 | Not hospital care |
| Hollingworth W, Todd CJ, King H, Males T, Dixon AK, Karia KR, et al. Primary care referrals for lumbar spine radiography: diagnostic yield and clinical guidelines. Br J Gen Pract 2002;52:475–80 | No intervention; data regarding general trends only |
| Horrocks S, Coast J. Patient choice: an exploration of primary care dermatology patient values and expectations of care. Qual Prim Care 2007;15:185–93 | Clinical care focus rather than managing demand |
| Hoyle JL, Hussey L, Agius R. Occupational asthma; referral patterns from primary care in the UK. Thorax 2010;65:A72 | Exclude: conference abstract |
| Hsu EY, Schwend RM, Julia L. How many referrals to a pediatric orthopaedic hospital specialty clinic are primary care problems? J Pediatr Orthop 2012;32:732–6 | Clinical care focus rather than managing demand |
| Isinkaye T, Gilbert S, Seddon P, Smith H. What proportion of paediatric allergy referrals could be dealt with in primary care? J Allergy Clin Immunol 2011;127:AB118 | Exclude: conference abstract |
| Jankowski RF. Implementing national guidelines at local level: changes in clinicians’ behaviour in primary care need to be reflected in secondary care. BMJ 2001;322:1258–9 | Discussion article |
| Jiwa M. Referral from primary to secondary care. BMJ 2010;341:c6175 | Editorial comment |
| Jiwa MB. GP letter writing in colorectal cancer: a qualitative study. Curr Med Res Opin 2002;18:342–6 | Quality of letter writing in critical care |
| Jiwa MH. Referral of suspected colorectal cancer: Have guidelines made a difference? Br J Gen Pract 2004;54:608–10 | Opinion piece |
| Johansen ECJ. Tympanometry for diagnosis and treatment of otitis media in general practice. Fam Pract 2000;17:317–22 | Evaluation of the treatment, not the referral process |
| Johansson B, Berglund G, Hoffman K, Glimelius B, Sjöden PO. The role of the general practitioner in cancer care and the effect of an extended information routine. Scand J Prim Health Care 2000;18:143–8 | Information exchange between GP and specialist – patient already referred |
| John SK, George S, Howell RD, Primrose JN, Fozard JB. Validation of the lower gastrointestinal electronic referral protocol. Br J Surg 2008;95:506–14 | Tool validation |
| Jones R, Rosen R, Tomlin Z, Cavanagh M, Oxley D. General practitioners with special interests: evolution and evaluation. J Health Serv Res Policy 2006;11:106–9 | Discussion paper |
| Kada S, Nygaard A, Geitung T, Mukesh N, Naik M. Quality and appropriateness of referrals for dementia patients. Qual Prim Care 2007;15:53–7 | Describes poor content of letters but does not link to referral management |
| Kendrick T, Simons L, Mynors-Wallis L, Grey A, Lathlean J, Pickering R, et al. Cost-effectiveness of referral for generic care or problem-solving treatment from community mental health nurses, compared with usual general practitioner care for common mental disorders: randomised controlled trial. Br J Psychiatry 2006;189:50–9 | Referral from one community service to another |
| King TM, Tandon SD, Macias MM, Healy JA, Duncan PM, Swigonski NL, et al. Implementing developmental screening and referrals: lessons learned from a national project. Pediatrics 2010;125:350–60 | Emphasis on improving screening rather than referral |
| Klimidis S, Minas H, Kokanovic R. Ethnic minority community patients and the better outcomes in mental health care initiative. Australas Psychiatry 2006;14:212–5 | Enabling patient access to service Queery |
| Krajcar NM. Improving primary to acute referrals: the role of the cancer clinical network in developing consistent information for general practitioners. Asia-Pacific J Clin Oncol 2011;7(Suppl. S4):117–95 | Conference abstract |
| Kourkouta S, Darbar UR. An audit of the quality and content of periodontal referrals and the effect of implementing referral criteria. Prim Dent Care 2006;13:99–106 | No actual referral outcomes |
| Krahn DD, Bartels SJ, Coakley E, Oslin DW, Chen H, McIntyre J, et al. PRISM-E: comparison of integrated care and enhanced specialty referral models in depression outcomes. Psychiatr Serv 2006;57:946–53 | Clinical outcomes |
| Kuo DZ, Cheng TL, Rowe PC, Kuo DZ, Cheng TL, Rowe PC. Successful use of a primary care practice-specialty collaboration in the care of an adolescent with chronic fatigue syndrome. Pediatrics 2007;120:e1536–9 | Case report/discussion |
| Kvamme OJ, Olesen F, Samuelsson M. Improving the interface between primary and secondary care. Qual Health Care 2001;10:33–9 | Position statement, no empirical data |
| Kyprianou I, D’Souza A, Saravanappa N, Lewis DM, Courtney-Harris R, Kyprianou I, et al. Referral patterns in paediatric orbital cellulitis. Eur J Emerg Med 2005;12:6–9 | Clinical focus |
| Laird S. Referral Management Centres Lack Evidence. 2006. URL: http://connection.ebscohost.com/c/articles/20941025/referral-management-centres-lack-evidence (accessed 16 January 2014) | News article on Welsh study (evaluation pilots) |
| Latinovic R, Gulliford M, Ridsdale L. Headache and migraine in primary care: consultation, prescription, and referral rates in a large population. J Neurol Neurosurg Psychiatry 2006;77:385–7 | Rates of referrals not reasons |
| Leamon MH. When to refer patients for substance abuse assessment and treatment. Prim Psychiatry 2006;13(6) | Discussion editorial |
| Lear SA, MacKinnon D, Farias-Godoy A, Nasmith J, Mazowita G, Ignaszewski A. Rapid access to cardiology expertise: an innovative program to provide telephone support for family physicians. Healthc Q 2010;13:56–60 | Clinical outcomes |
| Lees L. Developing a nurse led GP referral service. Emerg Nurse 2003;11:28–32 | Referral to emergency care |
| Lehnert BE, Bree RL. Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: how critical is the need for improved decision support? [Erratum published in J Am Coll Radiol 2010;7:466.] J Am Coll Radiol 2010;7:192–7 | Clinical outcomes |
| Linden M, Gothe H, Ormel J. Pathways to care an psychological problems of general practitioners in a gatekeeper and an open access health care system: a comparison of Germany and the Netherlands. Soc Psychiatry Psychiatr Epidemiol 2003;38:690–7 | Comparison of German and Dutch referral |
| Linnala A, Aromaa A, Mattila K. Specialists as consultants to GPs. Private sector services as an alternative way of organising consultant services in health care. Scand J Prim Health Care 2001;19:90–4 | Increasing access using private sector rather than managing referral |
| Love T. Quality indicators and variation in primary care: modelling GP referral patterns. Fam Pract 2004;21:160–5 | Prescribing rates by area |
| Lucas A, Smeenk F, Smeele I, Brouwer T, van Schayck O, Lucas A, et al. The validity of diagnostic support of an asthma/COPD service in primary care. Br J Gen Pract 2007;57:892–6 | Increasing skills of GPs in COPD diagnosis |
| Ludwick DA, Lortie C, Doucette J, Rao J, Samoil-Schelstraete C. Evaluation of a telehealth clinic as a means to facilitate dermatological consultation: pilot project to assess the efficiency and experience of teledermatology used in a primary care network. J Cutan Med Surg 2010;14:7–12 | Looks at speed of referral not quality |
| Maddock GR, Startup M, Carter GL. Patient characteristics associated with GP referral to the Access to Allied Psychological Services Program: a case–control study. Aust NZ J Psychiatry 2012;46:435–44 | Types of clinical problems referred |
| Mahima H, Rajdeep G, Gary LL. Referral equity and referral management: the supplier firm’s perspective. Rev Marketing Res 2010;7:93–144 | Discussion article |
| Mannion R, Goddard M. General practitioners’ assessments of hospital quality and performance. Clin Governance 2004;9:42–7 | How GPs asses quality of hospital |
| Mariñoso BG, Jelovac I. GPs’ payment contracts and their referral practice. J Health Econ 2003;22:617–35 | Economic analysis of gatekeeping as a predictor of referral |
| Martens JD, Werkhoven MJ, Severens JL, Winkens RA. Effects of a behaviour independent financial incentive on prescribing behaviour of general practitioners. J Eval Clin Pract 2007;13:369–73 | Drug prescribing, not referral |
| McGorm K, Burton C, Weller D, Murray G, Sharpe M. Patients repeatedly referred to secondary care with symptoms unexplained by organic disease: prevalence, characteristics and referral pattern. Fam Pract 2010;27:479–86 | Frequently vs. regularly referred patients |
| McGraw E, Barthel H, Arrington M. A model for demand management in a managed care environment. Mil Med 2000;165:305–8 | Referral to emergency care |
| McHugh A, Campbell M, Luker A. GP referral of patients with osteoarthritis for consideration of total joint replacement: a longitudinal study. Br J Gen Pract 2011;61:e459–68 | Clinical variation |
| McIsaac WJ, Coyte P, Croxford R, Harji S, Feldman W, McIsaac WJ, et al. Referral of children with otitis media. Do family physicians and pediatricians agree? Can Fam Physician 2000;46:1780–2 | Clinical predictors |
| McKay K. Evaluating model programs to support dissemination: an evaluation of strengthening the developmental surveillance and referral practices of child health providers. J Dev Behav Pediatr 2006;27(Suppl. 1):26–9 | Satisfaction outcomes only – development of a tool |
| McKinstry B, Walker J, Campbell C, Heaney D, Wyke S. Telephone consultations to manage requests for same-day appointments: a randomised controlled trial in two practices. Br J Gen Pract 2002;52:306–10 | Primary care only, not referral |
| Meadows GN, Harvey CA, Joubert L, Barton D, Bedi G. Best practices: the consultation-liaison in primary-care psychiatry program: a structured approach to long-term collaboration. Psychiatr Serv 2007;58:1036–8 | Discussion piece |
| Menachemi N, Perkins RM, van Durme DJ, Brooks RG. Examining the adoption of electronic health records and personal digital assistants by family physicians in Florida. Inform Prim Care 2006;14:1–9 | Use of technology and barriers to use |
| Meredith LS, Yano EM, Hickey SC, Sherman SE. Primary care provider attitudes are associated with smoking cessation counseling and referral. Med Care 2005;43:929–34 | Not referral to hospital elective |
| Mitchell A, Keenan RA. Are Grampian general practitioners good gatekeepers for access to emergency general surgical care? Scott Med J 2008;53:33–5 | Emergency |
| Mitchell GK, De Jong IC, Del Mar CB, Clavarino AM, Kennedy R. General practitioner attitudes to case conference: How can we increase participation and effectiveness? Med J Aust 2002;177:95–7 | Nothing of relevance |
| Mondry A, Zhu AL, Loh M, Vo TD, Hahn K. Active collaboration with primary care providers increases specialist referral in chronic renal disease. BMC Nephrol 2004;5:16 | Clinical outcomes |
| Montgomery JA, McGee HM, Shannon W, Donohoe J. Factors influencing general practitioner referral of patients developing end-stage renal failure: a standardised case-analysis study. BMC Health Serv Res 2006;6:114 | Clinical reasons for referral |
| Moreno-Ramirez D, Ferrandiz L, Nieto-Garcia A, Carrasco R, Moreno-Alvarez P, Galdeano R, et al. Store-and-forward teledermatology in skin cancer triage: experience and evaluation of 2009 teleconsultations. Arch Dermatol 2007;143:479–84 | Clinical measures not referral |
| Moreno-Ramirez D, Ferrandiz L, Ruiz-de-Casas A, Nieto-Garcia A, Moreno-Alvarez P, Galdeano R, et al. Economic evaluation of a store-and-forward teledermatology system for skin cancer patients. J Telemed Telecare 2009;15:40–5 | Looks at cost only, not effect on referrals |
| Mostert MC, Richardus JH, de Man RA, Mostert MC, Richardus JH, de Man RA. Referral of chronic hepatitis B patients from primary to specialist care: making a simple guideline work. J Hepatol 2004;41:1026–30 | Referral to public health department for notification of disease and then advice sent back to GP to onward refer |
| Murphy FM, James HD, Lloyd KR. Closer working with primary care is associated with a sharp increase in referrals to community mental health services. J Ment Health 2002;11:605–10 | Community referrals not hospital care |
| Murray M. Reducing waits and delays in the referral process: by formalizing your referral relationships, you can make life easier for you and your patients. Fam Pract Manag 2002;9:39–42 | Discussion editorial |
| Myfanwy D, Glyn E. Referral management centres: promising innovations or Trojan horses? BMJ 2006;332:844–8 | Discussion paper |
| New JP, Mason JM, Freemantle N, Teasdale S, Wong L, Bruce NJ, et al. Educational outreach in diabetes to encourage practice nurses to use primary care hypertension and hyperlipidaemia guidelines (EDEN): a randomized controlled trial. Diabet Med 2004;21:599–603 | Effect of guideline promotion on clinical outcomes |
| Nixon RM, Duffy SW, Fender GR, Day NE, Prevost TC. Randomization at the level of primary care practice: use of pre-intervention data and random effects models. Stat Med 2001;20:1727–38 | Pre-intervention data only |
| Nor AM, Ford GA. Development of a primary care referral stroke identification tool (PRESIT). Stroke 2006;37:671–2 | Poster abstract |
| Nutting PA, Gallagher K, Riley K, White S, Dickinson WP, Korsen N, et al. Care management for depression in primary care practice: findings from the RESPECT-Depression trial. Ann Fam Med 2008;6:30–7 | Patient care focus |
| O’Dowd A. GP consortium agrees partnership with private firm to run referral service. BMJ 2011;342:c7470 | Very short news article. No data |
| O’Brien C, Cambouropoulos P. Combating information overload: a six-month pilot evaluation of a knowledge management system in general practice. Br J Gen Pract 2000;50:489–90 | Primary care only, not referral |
| O’Malley AS, Reschovsky JD. Referral and consultation communication between primary care and specialist physicians: finding common ground. Arch Intern Med 2011;171:56–65 | Quality of care outcomes |
| Orden M, Hoffman T, Haffmans J, Spinhoven P, Hoencamp E. Collaborative mental health care versus care as usual in a primary care setting: a randomized controlled trial. Psychiatr Serv 2009;60:74–9 | Looks at referral delay |
| Oslin DW, Grantham S, Coakley E, Maxwell J, Miles K, Ware J, et al. PRISM-E: comparison of integrated care and enhanced specialty referral in managing at-risk alcohol use. [Erratum published in Psychiatr Serv 2006;57:1492. Olsen, Ed (added); Kirchner, JoAnn E (added); Levkoff, Sue (added)]. Psychiatr Serv 2006;57:954–8 | Clinical outcomes |
| Overland J, Mira M, Yue DK. Differential shared care for diabetes: does it provide the optimal partition between primary and specialist care? Diabet Med 2001;18:554–7 | Clinical outcomes |
| Pallan M, Linnane J, Ramaiah S. Evaluation of an independent, radiographer-led community diagnostic ultrasound service provided to general practitioners. J Public Health (Oxf) 2005;27:176–81 | Relocation of services, not referral |
| Park CHT. Factors influencing the periodontal referral process. J Periodontol 2011;82:1288–94 | Clinical outcomes |
| Park JC, Ross AH, Tole DM, Sparrow JM, Penny J, Mundasad MV, et al. Evaluation of a new cataract surgery referral pathway. Eye 2009;23:309–13 | Quality of referral information assessed in terms of clinical outcomes |
| Pattinson J. Primary care. Central reservations. Health Serv J 2003;113:30–1 | Community services only |
| Penson DF. Re: Are men on 5alpha-reductase inhibitors appropriately referred to urology? A survey of primary care physicians. J Urol 2012;188:114 | Clinical data |
| Piek E, van der Meer K, Penninx BWJH, Verhaak PFM, Nolen WA. Referral of patients with depression to mental health care by Dutch general practitioners: an observational study. BMC Fam Pract 2011;12:41 | GP diagnostic ability and referral |
| Pileggi C, Carbone V, Pavia M, Angelillo IF. Patient perceptions and related behaviours on role of primary care physician in Italy. Eur J Pub Health 2004;14:258–60 | Nothing of relevance; outlines why attended GP |
| Pletcher BA, Rimsza ME, Cull WL, Shipman SA, Shugerman RP, O’Connor KG, et al. Primary care pediatricians’ satisfaction with subspecialty care, perceived supply, and barriers to care. J Pediatr 1015;156:1011–15 | Relates to views of optimal clinical care rather than referral |
| Podgorski CA, Lanning BD, Casaceli CJ, Nardi AL, Cox C, Podgorski CA, et al. Dementia consults: predictors of compliance by primary care physicians. Am J Alzheimers Dis Other Demen 2002;17:44–50 | Specialist to GP compliance (not GP-specific referral) |
| Poels PJ, Schermer TR, Thoonen BP, Jacobs JE, Akkermans RP, de Vries Robbe PF, et al. Spirometry expert support in family practice: a cluster-randomised trial. Prim Care Respir J 2009;18:189–97 | Improving diagnostic skills of GP rather than referral |
| Poiraudeau S, Rannou F, Le Henanff A, Coudeyre E, Rozenberg S, Huas D, et al. Outcome of subacute low back pain: influence of patients’ and rheumatologists’ characteristics. Rheumatology (Oxford) 2006;45:718–23 | Predictors of clinical outcomes |
| Price A, Williams A. Primary care nurse practitioners and the interface with secondary care: a qualitative study of referral practice. J Interprof Care 2003;17:239–50 | Focus on professional roles for clinical care rather than referral |
| Quinn DC, Graber AL, Elasy TA, Thomas J, Wolff K, Brown A, et al. Overcoming turf battles: developing a pragmatic, collaborative model to improve glycaemic control in patients with diabetes. Jt Comm J Qual Improv 2001;27:255–64 | Patient clinical care model rather than referral |
| Qureshi N, Modell B, Modell M. Timeline – raising the profile of genetics in primary care. Nature Reviews Genetics 2004;5:783–90 | Exclude country (Saudi Arabia) |
| Rao JK, Kroenke K, Mihaliak KA, Eckert GJ, Weinberger M, Rao JK, et al. Can guidelines impact the ordering of magnetic resonance imaging studies by primary care providers for low back pain? Am J Manag Care 2002;8:27–35 | MRI in primary care, not referral |
| Raymont AM. New Zealand general practitioners’ non-urgent referrals to surgeons: who and why? N Z Med J 2008;121:57–64 | General referral data |
| Redfern J, Bowling A. Efficiency of care at the primary–secondary interface: variations with GP fundholding. Health Place 2000;6:15–23 | Clinical outcomes |
| Ree MH, Timmerman MF, Wesselink PR. Factors influencing referral for specialist endodontic treatment amongst a group of Dutch general practitioners. Int Endod J 2003;36:129–34 | Dental service |
| Reinders ME, Blankenstein AH, van Marwijk HW, Schleypen H, Schoonheim PL, Stalman WA, et al. Development and feasibility of a patient feedback programme to improve consultation skills in general practice training. Patient Educ Couns 2008;72:12–19 | Improvement of GP trainee skills |
| Reiss-Brennan B, Briot P, Cannon W, James B. Mental health integration: rethinking practitioner roles in the treatment of depression: the specialist, primary care physicians, and the practice nurse. Ethn Dis 2006;16(2 Suppl. 3):S3–43 | Clinical outcomes of revised case management system |
| Richards DA, Meakins J, Godfrey L, Tawfik J, Dutton E. Survey of the impact of nurse telephone triage on general practitioner activity. Br J Gen Pract 2004;54:207–10 | Referral within community (nurse to GP) |
| Ricketts TS. Evaluating the development, implementation and impact of protocols between primary care and specialist mental health services. J Ment Health 2003;12:369–83 | Development of a referral protocol only rather than testing of it |
| Ridsdale L, Doherty J, McCrone P, Seed P, Clarke L, Das R, et al. A new GP with special interest headache service: observational study (Structured abstract). Br J Gen Pract 2008;58:478–83 | Clinical/cost outcomes |
| Robinson PC, Taylor WJ. Time to treatment in rheumatoid arthritis: factors associated with time to treatment initiation and urgent triage assessment of general practitioner referrals. J Clin Rheumatol 2010;16:267–73 | Patients already referred to secondary care |
| Roddy E, Zwierska I, Dawes P, Hider SL, Jordan KP, Packham J, et al. The Staffordshire arthritis, musculoskeletal, and back assessment (SAMBA) study: a prospective observational study of patient outcome following referral to a primary-secondary care musculoskeletal interface service. BMC Musculoskelet Disord 2010;11:67 | Study protocol. No further papers found in Google search |
| Rossi PS. Implementation and evaluation of existing guidelines on the use of neurophysiological tests in non-acute migraine patients: a questionnaire survey of neurologists and primary care physicians. Eur J Neurol 2009;16:937–42 | Appropriateness of treatment recommendation GP vs. specialist |
| Rucci P, Piazza A, Menchetti M, Berardi D, Fioritti A, Mimmi S, et al. Integration between primary care and mental health services in Italy: determinants of referral and stepped care. Int J Fam Med 2012;2012:507464 | Care-based outcomes rather than referral outcomes |
| Russell V, McCauley M, MacMahon J, Casey S, McCullagh H, Begley J. Liaison psychiatry in rural general practice. Ir J Psychol Med 2003;20:65–8 | Discussion paper |
| Ryan T, Hatfield B, Sharma I. Outcomes of referrals over a six-month period to a mental health gateway team. J Psychiatr Ment Health Nurs 2007;14:527–34 | Analysis of clinical characteristics of referrals that were made to team |
| Salvador CH, Gonzalez MA, Munoz A, Pascual M. Teleradiology from primary care: comparison of user activity in two different scenarios. J Telemed Telecare 2002;8:178–82 | Quality of teleradiology system, e.g. bandwidth and diagnostics |
| Sarvet B, Gold J, Straus JH. Bridging the divide between child psychiatry and primary care: the use of telephone consultation within a population-based collaborative system. Child Adolesc Psychiatr Clin N Am 2011;20:41–53 | Description of service rather than empirical data |
| Saxon AJ, Malte CA, Sloan KL, Baer JS, Calsyn DA, Nichol P, et al. Randomized trial of onsite versus referral primary medical care for veterans in addictions treatment (provisional abstract). Med Care 2006;44:334–42 | Treatment options |
| Scott K. The Swansea electronic referrals project. J Telemed Telecare 2009;15:156–8 | Focuses on ease of use of specific e-referral system. No referral outcomes |
| Sequist TD. Is referral for colonoscopy overutilized by primary care physicians? Ann Intern Med 2006;145:654–9 | Opinion |
| Shaw I, Smith KM, Middleton H, Woodward L. A letter of consequence: referral letters from general practitioners to secondary mental health services. Qual Health Res 2005;15:116–28 | Narrative discussion |
| Sheldon T. Dutch GPs agree to refer fewer patients to hospital and prescribe more generic drugs. BMJ 2012;344:e4510 | General description of service change |
| Shepherd M. Involving psychologists, counsellors and referrers in the systematic development of consensus-based referral criteria for a primary care psychology and counselling service. Clin Psychol 2003;21:39–43 | Referring within primary care |
| Shershneva MB, Carnes M, Bakken LL, Shershneva MB, Carnes M, Bakken LL. A model of teaching-learning transactions in generalist-specialist consultations. J Contin Educ Health Prof 2006;26:222–9 | GP training only, not relevant |
| Short D, Frischer M, Bashford J. Barriers to the adoption of computerised decision support systems in general practice consultations: a qualitative study of GPs’ perspectives. Int J Med Inform 2004;73:357–62 | Primary care only, not referral |
| Shreibati JB, Baker LC. The relationship between low back magnetic resonance imaging, surgery, and spending: impact of physician self-referral status. Health Serv Res 2011;46:1362–81 | Relationship between MRI and surgery |
| Sifri R, Wender R, Lieberman D, Potter M, Peterson K, Weber TK, et al. Developing a quality screening colonoscopy referral system in primary care oractice: a report from the national colorectal cancer roundtable. CA Cancer J Clin 2010;60:40–9 | Discussion/report. References need fully checking |
| Singh H, Esquivel A, Sittig DF, Murphy D, Kadiyala H, Schiesser R, et al. Follow-up actions on electronic referral communication in a multispecialty outpatient setting. J Gen Intern Med 2011;26:64–9 | Audit of discontinued referrals. No intervention |
| Singh H, Petersen LA, Daci K, Collins C, Khan M, El-Serag HB. Reducing referral delays in colorectal cancer diagnosis: is it about how you ask? Qual Saf Health Care 2010;19:e27 | Clinical factors |
| Souster V. Learning for primary care practitioners via clinical attachments: the Learning Packages Project. Work Based Learn Prim Care 2006;4:141–51 | Discussion paper |
| Speed CA, Crisp AJ. Referrals to hospital-based rheumatology and orthopaedic services: seeking direction. Rheumatology (Oxford) 2004;44:469–71 | No intervention, description of content of referral letters |
| St George I, Cullen M, Gardiner L, Karabatsos G. Universal telenursing triage in Australia and New Zealand – a new primary health service. Aust Fam Physician 2008;37:476–9 | Summary discussion paper |
| Stanton MR, Atherton WL, Toriello PJ, Hodgson JL. Implementation of a ‘learner-driven’ curriculum: an screening, brief intervention, and referral to treatment (SBIRT) interdisciplinary primary care model. Subst Abuse 2012;33:312–15 | Commentary. No data |
| Steele M, Shapiro J, Davidson B, Floyd G, Johnston J, Stretch N, et al. Survey comparing criteria used by rural and urban primary care physicians for referrals to child and adolescent psychiatrists and children’s mental health agencies in Ontario. J Can Acad Child Adolesc Psychiatry 2010;19:284–9 | Awareness of alternative mental health treatments |
| Stevens SL, Kinsman SB, Pailler ME, Latif S, Levy SA, Diamond GS. Providers’ experiences with identification, management and referral of adolescents with depression. J Adolesc Health Volume 2007;40(Suppl.):42–3 | Diagnosis and clinical care |
| Stille CJ, McLaughlin TJ, Primack WA, Mazor KM, Wasserman RC. Determinants and impact of generalist-specialist communication about pediatric outpatient referrals. Pediatrics 2006;118:1341–9 | Emphasis on communication for optimal clinical care |
| Stille CJ, Primack WA, McLaughlin TJ, Wasserman RC. Parents as information intermediaries between primary care and specialty physicians. Pediatrics 2007;120:1238–46 | Communication relating to ongoing care |
| Street RL Jr. Specialist-primary care provider-patient communication in telemedical consultations. Telemed J 2000;6:45–54 | Looks at the content of telemedical consultations. Not consequences for referral |
| Sved Williams A, Dodding J, Wilson I, Fuller J, Wade V. Consultation-liaison to general practitioners coming of age: the South Australian psychiatrists’ experience. Australas 2006;14:206–11 | |
| Syed MA, Schofield JK, Kanji A. The challenge of demand management and dermatology referrals: the general practitioner’s view. Br J Dermatol 2012;167:41 | Exclude: conference abstract |
| Tanielian TL. Referrals to psychiatrists: assessing the communication interface between psychiatry and primary care. Psychosomatics 2000;41:245–52 | Describes the referral process rather than managing the process |
| Taylor KM, Narbey A, McNair A, Foxton M. Screening, referral practice and management of hepatitis B and C in primary care: a survey of general practitioners. Gut 2010;59:A78 | Exclude: conference abstract |
| Thomas H. Monitoring referrals to mental health services. Nurs Older People 2010;22:16–22 | Epidemiology data and discussion piece |
| Thong JF, Mok P, Loke D. A quality assurance survey to improve communication between ENT specialists and general practitioners. Singapore Med J 2010;51:796–9 | Letters between hospital ENT and GP rather than primary to secondary |
| Thornton JD, Chandriani K, Thonton JG, Farooq S, Moallem M, Krishnan V, et al. Assessing the prioritization of primary care referrals for polysomnograms. Sleep 2010;33:1255–60 | Compares referral rates for different investigations with emphasis on clinical identification using different investigations |
| Tjerbo T. Does competition among general practitioners increase or decrease the consumption of specialist health care? Health Econ Policy Law 2010;5:53–70 | Data on spending on specialist care costs rather than referral |
| Tucker JL. Comparison of specialty referral patterns of primary care providers – practitioner response. J Healthc Manag 2002;47:205 | Clinical outcomes |
| Turley A, Roberts AP, Kunadian B, Davies A, Rowell N, De Belder MA, et al. The impact of the introduction of NT-proBNP into primary care on secondary care referral rates. Eur Heart J 2005;26:744 | Conference abstract |
| Ung YC, Del Giudice L, Young S, Vella E, Ash M, Bansal P, et al. Referral of suspected lung cancer by family physicians and other primary care providers: a clinical practice guideline. Int J Radiat Oncol Biol Phys 2012;84:S603–4 | Exclude: conference abstract |
| van den Boogaard TM, Verhaak PF, Van Dyck R, Spinhoven P. The impact of causal attributions on diagnosis and successful referral of depressed patients in primary care. Soc Sci Med 2011;73:1733–40 | Factors affecting depression diagnosis |
| van der Feltz-Cornelis CM, van Oppen P, Adèr HJ, van Dyck R. Randomised controlled trial of a collaborative care model with psychiatric consultation for persistent medically unexplained symptoms in general practice. Psychother Psychosom 2006;75:282–9 | Clinical outcomes |
| van der Straten LM, van Stel HF, Spee FJ, Vreeburg ME, Schrijvers AJ, Sturms LM. Safety and efficiency of triaging low urgent self-referred patients to a general practitioner at an acute care post: an observational study. Emerg Med J 2012;29:877–81 | Triaging of emergency department patients |
| van Dam L, van der Togt-van Leeuwen AC, Hol L, Vos CJ, Joung IM, van Ballegooijen M, et al. Experiences of general practitioners regarding their role in the referral process for colonoscopy after a positive colorectal cancer screening test. Gastroenterology 2010;138:S191 | Conference abstract |
| van Dijk CE, Verheij RA, Spreeuwenberg P, Groenewegen PP, de Bakker DH. Minor surgery in general practice and effects on referrals to hospital care: observational study. BMC Health Serv Res 2011;11:2 | Clinical outcomes |
| Van Schaik P, Flynn D, Van Wersch A, Douglass A, Cann P. The acceptance of a computerised decision-support system: a preliminary investigation. Behav Inform Technol 2004;23:321–6 | Focus on the technology rather than referral process |
| van Zanten SV. A self help guidebook reduced primary care consultations in irritable bowel syndrome. Evid Based Med 2006;11:179 | Patient education to reduce primary care consultations |
| Van K, Liu S, Conn L, Hoadley A, Ho S. Improving the effectiveness of fecal occult blood testing in a primary care clinic by direct colonoscopy referral for positive tests. J Healthc Qual 2010;32:62–9 | Effectiveness of test conducted in primary care |
| Vanden Bussche P, Desmyter F, Duchesnes C, Massart V, Giet D, Petermans J, et al. Geriatric day hospital: opportunity or threat? A qualitative exploratory study of the referral behaviour of Belgian general practitioners. BMC Health Serv Res 2010;10:202 | Audit of referral rates – no intervention |
| Vause J. A recommendation from suspected cancer in primary care: guidance for referral and reducing disparities from the NZ Guidelines Group. J Prim Health Care 2009;1:144 | Short opinion piece |
| Verstappen WH, van der Weijden T, Dubois WI, Smeele I, Hermsen J, Tan FE, et al. Improving test ordering in primary care: the added value of a small-group quality improvement strategy compared with classic feedback only. Ann Fam Med 2004;2:569–75 | Test ordering (blood tests, etc.) not referral |
| Walker JN, Rourke D, Allen K, Karavitaki N, Levy J, Wass JA, et al. An e-mail GP advisory service: a more efficient way of dealing with clinical enquiries. Br J Hosp Med (Lond) 2009;70:532–3 | Clinical queries rather than referral |
| Wallace P, Haines A, Harrison R, Barber J, Thompson S, Jacklin P, et al. Joint teleconsultations (virtual outreach) versus standard outpatient appointments for patients referred by their general practitioner for a specialist opinion: a randomised trial. Lancet 2002;359:1961–8. | Protocol paper |
| Walshe C, Chew-Graham C, Todd C, Caress A, Walshe C, Chew-Graham C, et al. What influences referrals within community palliative care services? A qualitative case study. Soc Sci Med 2008;67:137–46 | Within-community referrals |
| Warren J Gu Y, Day K, White S, Pollock M. Electronic referrals: what matters to the users. Stud Health Technol Informs 2012;178:235–41 | Focus on the technology rather than referral process |
| Watson JM, McDonnell V, Bhaumik S. Valuing people: evaluating referral systems. Br J Dev Disabil 2005;51:155–70 | Exclude: community referrals |
| Wee S-LK. Improving access to outpatient cardiac care at the National Heart Centre – a partnership between specialists and primary care. Ann Acad Med Singapore 2008;37:151–7 | None OECD country |
| White DB, Bonham VL, Jenkins J, Stevens N, McBride CM. Too many referrals of low-risk women for BRCA1/2 genetic services by family physicians. Cancer Epidemiol Biomarkers Prev 2008;17:2980–6 | Clinical appropriateness of referral |
| White P, Singleton A, Jones R. Copying referral letters to patients, views of patients, representatives and doctors. Patient Educ Couns 2004;55:94–8 | Not referral |
| Wiley-Exley E, Domino ME, Maxwell J, Levkoff SE, Wiley-Exley E, Domino ME, et al. Cost-effectiveness of integrated care for elderly depressed patients in the PRISM-E study. J Ment Health Policy Econ 2009;12:205–13 | Compares clinical outcomes for integrated case management to specialty referral |
| Wilkens JR, Belgrade MJ. Do pain specialists meet the needs of the referring physician? A survey of primary care providers. J Opioid Manag 2008;4:13–20 | Views of drug management |
| Wilkes S, Rubin G, Crosland A, Hall N, Murdoch A. Patient and professional views of open access hysterosalpingography for the initial management of infertility in primary care. Br J Gen Pract 2009;59:336–42 | Views of GPs towards infertility treatment; pre-testing as part of referral process rather than managing demand |
| Wilson BJ, Torrance N, Mollison J, Watson MS, Douglas A, Miedzybrodzka Z, et al. Cluster randomized trial of a multifaceted primary care decision-support intervention for inherited breast cancer risk. Fam Pract 2006;23:537–44 | Primary care only |
| Wong WM, Beeler J, Risner-Adler S, Habib S, Bautista J, Fass R. Attitudes and referral patterns of primary care physicians when evaluating subjects with noncardiac chest pain – a national survey. Dig Dis Sci 2005;50:656–61 | Diagnosis and clinical care emphasis |
| Worrall SF. An audit of general dental practitioners’ referral practice following the distribution of third molar guidelines. Ann R Coll Surg Engl 2001;83:61–4 | Not hospital care |
| Xibillé-Friedmann D, Mondragón-Flores V, Horcasitas de la Rosa C. Criteria used by primary care physicians for the diagnosis and referral to a rheumatologist of patients with rheumatoid arthritis. Reumatologia Clinica 2006;2:235–8 | Clinical outcomes |
| Yafi FA, Aprikian AG, Tanguay S, Kassouf W. Patients with microscopic and gross hematuria: practice and referral patterns among primary care physicians in a universal health care system. Can Urol Assoc J 2011;5:97–101 | Clinical knowledge of physicians |
| Yardley L, Kirby S, Barker F, Little P, Raftery J, King D, et al. An evaluation of the cost-effectiveness of booklet-based self-management of dizziness in primary care, with and without expert telephone support. BMC Ear Nose Throat Dis 2009;9:13 | Study protocol only Outcomes paper has clinical data only, nothing on referrals: www.ncbi.nlm.nih.gov/pubmed/22674920?dopt=Abstract |
| Zanjani F, Miller B, Turiano N, Ross J, Oslin DE-MA, Zanjani F. Effectiveness of telephone-based referral care management, a brief intervention to improve psychiatric treatment engagement. Psychiatr Serv 2008;59:776–81 | Looks at alternative treatment, not referral |
Appendix 6 Systematic review papers
Alvarez MP, Agra Y. Systematic review of educational interventions in palliative care for primary care physicians. Palliat Med 2006;20:673–83.
Bazian Ltd. Specialist outreach into primary care: is it better than standard care? Evid-Based Healthc Public Health 2005;9:294–301.
Bower P, Sibbald B. Systematic review of the effect of on-site mental health professionals on the clinical behaviour of general practitioners. BMJ 2000;320:614–17.
Bower P, Sibbald B. Do consultation-liaison services change the behaviour of primary care providers? A review. General Hospital Psychiatry 2000;22:84–96.
Brocklehurst PR, Baker SR, Speight PM. Primary care clinicians and the detection and referral of potentially malignant disorders in the mouth: a summary of the current evidence. Prim Dent Care 2010;17:65–71.
Cardiff University, CRG Research Ltd. Evaluation of Referral Management Pilots in Wales. NHS Wales and National Leadership and Innovation Agency for Healthcare; 2006.
CRG Research Ltd, Cardiff University. Referral Management Pilots in Wales – Follow Up Review. National Leadership and Innovation Agency for Healthcare; 2007.
Clarke A, Blundell N, Forde I, Musila N, Spitzer D, Naqvi S, et al. Can guidelines improve referral to elective surgical specialties for adults: a systematic review. Qual Saf Health Care 2008;19:187–94.
Delva F, Soubeyran P, Rainfray M, Mathoulin-Pelissier S. Referral of elderly cancer patients to specialists: action proposals for general practitioners. Cancer Treat Rev 2012;38:935–41.
Foot C, Naylor C, Imison C. The Quality of GP Diagnosis and Referral. London: The King’s Fund; 2010.
Forrest CB. Primary care gatekeeping and referrals: effective filter of failed experiment? BMJ 2003;326:692–5.
Gruen RL, Weeramanthri TS, Knight SS, Bailie RS. Specialist outreach clinics in primary care and rural hospital settings. Cochrane Database Syst Rev 2003;4:CD003798.
Harkness EF, Bower PJ. On site mental health workers delivering psychological therapy and psychosocial interventions to patients in primary care: effects on the professional practice of primary care providers. Cochrane Database Syst Rev 2009;1:CD000532.
Herrington P, Baker R, Gibson SL, Golden S. GP referrals for counselling: a review and model. J Interprof Care 2003;17:263–71.
Jiwa M, Dadich A. Referral letter content: can it affect patient outcomes? Br J Health Care Manage 2013;19:140–7.
Lin CY. Improving care coordination in the specialty referral process between primary and specialty care. NC Med J 2012;73:61–2.
Martin A, Macleod C, Naqui SAR. Effectiveness and Cost-Effectiveness of Targeted Interventions to Reduce Unnecessary Referrals and Improve the Quality of Referrals from Primary Care to Secondary Care. NHS Evidence Adoption Centre East of England; 2010.
Mead N, Bower P. Patient centred consultations and outcomes in primary care: a review of the literature. Patient Educ Couns 2002;48:51–61.
Mehrota A, Forrest CB, Lin CY. Dropping the baton: specialty referrals in the United States. Millbank Q 2011;89:39–68.
Navaneethan SD, Aloudat S, Singh S. A systematic review of patient and health system characteristics associated with late referral in chronic kidney disease. BMC Nephrology 2008;9.
O’Donnell CA. Variation in GP referral rates: what can we learn from the literature? Fam Pract 2000;17:462–71.
Piterman L, Koritsas S. Part II General practitioner-specialist referral process. Intern Med J 2005;35:491–6.
Powell J. Systematic review of outreach clinics in primary care in the UK. J Health Serv Res Policy 2007;7:177–83.
Qureshi NA, van der Molen HT, Schmidt HG, Al-Habeeb TA, Magzoub MEM. Criteria for a good referral system for psychiatric patients: the view from Saudi Arabia. East Mediterr Health J 2009;15:1580–95.
Roland M, McDonald R, Sibbald B, Boyd A, Fotaki M, Gravelle H, et al. Outpatient Services and Primary Care: A Scoping Review of Research into Strategies for Improving Outpatient Effectiveness and Efficiency. National Primary Care Research and Development Centre, and Centre for Public Policy and Management of the University of Manchester; 2006.
Review studies quality assessment
Assessment checklist:
1. The study has a clear research question and defined inclusion/exclusion criteria.
2. There is evidence of a substantial effort to identify all relevant research across several sources.
3. Appropriate methods were used to minimise reviewer error or bias in study selection, extraction and quality appraisal.
4. Validity of included studies was adequately assessed.
5. Sufficient detail for individual studies was provided.
6. The studies were summarised appropriately.
7. The authors’ conclusion was an accurate reflection of the evidence presented.
Lower risk of bias: all or nearly all of the checklist criteria have been fulfilled.
Higher risk of bias: some of the checklist criteria have been fulfilled.
Not clear: unable to make an assessment due to lack of detail in the paper.
Quality assessment of review papers
| Study | Checklist item | Rating | ||||||
|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | ||
| Akbari et al. 20081 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Lower risk of bias |
| Alvarez and Agra 2006312 | ✓ | ✓ | Unclear | Unclear | ✓ | ✓ | ✓ | Lower risk of bias |
| Bazian Ltd 2005313 | ✓ | ✗ | Unclear | Unclear | ✓ | ✓ | ✓ | Lower risk of bias |
| Bower and Sibbald 2000314 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Lower risk of bias |
| Bower and Sibbald 2000315 | ✓ | ✓ | Unclear | Unclear | ✓ | ✓ | ✓ | Lower risk of bias |
| Brocklehurst et al. 2010316 | ✗ | ✗ | Unclear | Unclear | ✗ | ✗ | ✓ | Higher risk of bias |
| CRG Research and Cardiff University 2006317 | ✗ | ✗ | Unclear | Unclear | ✗ | ✗ | Unclear | Higher risk of bias |
| CRG Research and Cardiff University 2007318 | ✗ | ✗ | Unclear | Unclear | ✗ | ✗ | ✓ | Higher risk of bias |
| Clarke et al. 2010319 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Lower risk of bias |
| Delva et al. 2012320 | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | ✓ | Lower risk of bias |
| Dunst and Gorman 20063 | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ | Lower risk of bias |
| Faulkner et al. 20032 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Lower risk of bias |
| Foot et al. 2010321 | ✓ | ✓ | Unclear | Unclear | ✗ | ✗ | ✓ | Higher risk of bias |
| Forrest 2003322 | ✗ | ✗ | Unclear | Unclear | ✗ | ✗ | Unclear | Higher risk of bias |
| Grimshaw et al. 20054 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Lower risk of bias |
| Gruen et al. 2003323 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Lower risk of bias |
| Harkness and Bower 2009324 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Lower risk of bias |
| Herrington et al. 2003325 | ✗ | ✓ | Unclear | Unclear | ✗ | ✗ | ✓ | Higher risk of bias |
| Imison and Naylor 20105 | ✓ | ✓ | Unclear | Unclear | ✗ | ✗ | ✓ | Higher risk of bias |
| Jiwa and Dadich 2013326 | ✗ | ✗ | Unclear | Unclear | ✗ | ✗ | ✓ | Higher risk of bias |
| Lin 2012327 | ✗ | ✗ | Unclear | Unclear | ✗ | ✗ | Unclear | Higher risk of bias |
| Martin et al. 2010328 | ✓ | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ | Lower risk of bias |
| Mead and Bower 2002329 | ✓ | ✓ | Unclear | ✓ | ✓ | ✓ | ✓ | Lower risk of bias |
| Mehrota et al. 2011330 | ✗ | ✗ | Unclear | Unclear | ✗ | ✗ | ✓ | Higher risk of bias |
| Navaneethan et al. 2008331 | ✓ | ✓ | ✓ | Unclear | ✓ | ✓ | ✓ | Lower risk of bias |
| O’Donnell 2000332 | ✗ | ✗ | Unclear | Unclear | ✗ | ✓ | ✓ | Higher risk of bias |
| Piterman and Koritsas 2005333 | ✗ | ✗ | Unclear | Unclear | ✗ | ✗ | ✓ | Higher risk of bias |
| Powell 2002334 | ✓ | ✓ | Unclear | Unclear | ✓ | ✓ | ✓ | Lower risk of bias |
| Qureshi et al. 2009335 | ✗ | ✗ | Unclear | Unclear | ✗ | ✗ | Unclear | Higher risk of bias |
| Roland et al. 2006336 | ✓ | ✓ | ✓ | Unclear | ✓ | ✓ | ✓ | Lower risk of bias |
List of abbreviations
- CBDS
- computer-based decision support
- CI
- confidence interval
- CINAHL
- Cumulative Index to Nursing and Allied Health Literature
- CMHT
- community mental health team
- COPD
- chronic obstructive pulmonary disease
- CT
- computerised tomography
- df
- degree of freedom
- DRAC
- direct-referral audiology clinic
- DXA
- dual-energy X-ray absorptiometry
- ECG
- electrocardiogram
- eGFR
- estimated glomerular filtration rate
- EMR
- electronic medical record
- ENT
- ear, nose and throat
- GI
- gastrointestinal
- GP
- general practitioner
- GPwSI
- GP with special interest
- GRAIDS
- Genetic Risk Assessment on the Internet with Decision Support software
- ICC
- intracluster correlation coefficient
- ICD-10
- International Classification of Diseases, Tenth Edition
- LEEP
- loop electrical excision procedure
- LRL
- letter and referral list
- LUTS
- lower urinary tract symptoms
- MRI
- magnetic resonance imaging
- NICE
- National Institute for Health and Care Excellence
- nRCT
- non-randomised controlled trial
- OR
- odds ratio
- PCDS
- primary care dermatology service
- PCT
- primary care trust
- PDA
- personal digital assistant
- PSA
- prostate-specific antigen
- QALY
- quality-adjusted life-year
- QOF
- Quality and Outcomes Framework
- RCGP
- Royal College of General Practitioners
- RCT
- randomised controlled trial
- RR
- relative risk
- SD
- standard deviation
- SE
- standard error
- SPR
- single point of referral
- TIA
- transient ischaemic attack