Health Technology Assessment

Initial assessment and management of adults with suspected acute respiratory infection: a rapid evidence synthesis of reviews and cost-effectiveness studies

Crossmark status check

Toolkit

View Award

Background

This work was undertaken to inform a National Institute for Health and Care Excellence guideline on the initial assessment of adults with suspected acute respiratory infection.

Objective

To undertake a rapid evidence synthesis of systematic reviews and cost-effectiveness studies of signs, symptoms and early warning scores for the initial assessment of adults with suspected acute respiratory infection.

Methods

MEDLINE, EMBASE and Cochrane Database of Systematic Reviews were searched for systematic reviews and MEDLINE, EMBASE, EconLit and National Health Service Economic Evaluation Database were searched for cost-effectiveness studies in May 2023. References of relevant studies were checked. Clinical outcomes of interest included escalation of care, antibiotic/antiviral use, time to resolution of symptoms, mortality and health-related quality of life. Risk of bias was assessed using the Risk of Bias in Systematic Reviews tool or the National Institute for Health and Care Excellence economic evaluations checklist. Results were summarised using narrative synthesis.

Results

Nine systematic reviews and one cost-effectiveness study met eligibility criteria.

Seven reviews assessed several early warning scores for patients with community- acquired pneumonia, one assessed early warning scores for nursing home-acquired pneumonia and one assessed individual signs/symptoms and the Centor score for patients with sore throat symptoms; all in face-to-face settings. Two good-quality reviews concluded that further research is needed to validate the CRB-65 in primary care/community settings. One also concluded that further research is needed on the Pneumonia Severity Index in community settings; however, the Pneumonia Severity Index requires data from tests not routinely conducted in community settings. One good-quality review concluded that National Early Warning Score appears to be useful in an emergency department/acute medical setting. One review (unclear quality) concluded that the Pneumonia Severity Index and CURB-65 appear useful in an emergency department setting. Two poor-quality reviews concluded that early warning scores can support clinical judgement and one poor-quality review found numerous problems with using early warning scores in a nursing home setting. A good-quality review concluded that individual signs and symptoms have a modest ability to diagnose streptococcal pharyngitis, and that the Centor score can enhance appropriate prescribing of antibiotics.

The cost-effectiveness study assessed clinical scores and rapid antigen detection tests for sore throat, compared to delayed antibiotic prescribing. The study concluded that the clinical score is a cost-effective approach when compared to delayed prescribing and rapid antigen testing.

Conclusions

Several early warning scores have been evaluated in adults with suspected acute respiratory infection, mainly the CRB-65, CURB-65 and Pneumonia Severity Index in patients with community-acquired pneumonia. The evidence was insufficient to determine what triage strategies avoid serious illness. Some early warning scores (CURB-65, Pneumonia Severity Index and National Early Warning Score) appear to be useful in an emergency department/acute medical setting; however, further research is required to validate the CRB-65 and Pneumonia Severity Index in primary care/community settings. The economic evidence indicated that clinical scores may be a cost-effective approach to triage patients compared with delayed prescribing.

Future work and limitations

Only systematic reviews were eligible for inclusion in the synthesis of clinical evidence. There was a great deal of overlap in the primary studies included in the reviews, many of which had significant limitations. No studies were undertaken in remote settings (e.g. NHS 111). Only one cost-effectiveness study was identified, with limited applicability to the review question.

Funding

This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme as award number NIHR159945.

Notes

Article history

The contractual start date for this research was in April 2023. This article began editorial review in June 2023 and was accepted for publication in March 2024. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The Health Technology Assessment editors and publisher have tried to ensure the accuracy of the authors’ article and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this article.

Permissions

Copyright statement

Copyright © 2024 Wade et al. This work was produced by Wade et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaption in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

2024 Wade et al.

Background

Before the COVID-19 pandemic, people with suspected acute respiratory infection (ARI) either presented to NHS 111 or primary care for assessment and management, with more severe cases referred for hospital assessment, or they presented directly to an emergency department or to the ambulance service if their symptoms were more serious. Since the pandemic, the levels of ARI (particularly pneumonia caused by COVID-19 infection) have increased. In response to this, the NHS has set up a number of ARI hubs and ARI virtual wards to relieve pressure on other parts of the local healthcare system.

For people aged 16 and over with suspected ARI, initial consultations with the health system may occur remotely [e.g. through online apps, e-mail exchange or text message, via telephone through NHS 111 or with a general practitioner (GP), via video call, or direct to 999 emergency call centres] or face to face (e.g. in the person’s home or care home, in primary care including community pharmacy or ARI hubs, in NHS walk-in centres and in emergency departments). Those with suspected ARI can be advised to remain at home for self-monitoring (with or without being prescribed antibiotics or antivirals), referred to ARI virtual wards for further monitoring, or referred to, and/or admitted to, a hospital.

The National Institute for Health and Care Excellence (NICE) has been asked to produce a number of related products to inform the development of NICE Guideline 10376 – acute respiratory infection in over 16s: initial assessment and management and to support the expansion of virtual ward provision and other intermediate care areas. This guideline is intended to aid healthcare professionals in deciding whether to refer people aged 16 and over with suspected ARI, including referrals to virtual wards and ARI hubs. The York Evidence Synthesis Group was commissioned by NICE to undertake a rapid review focused on the early assessment of people aged 16 and over with suspected ARI, in both remote and face-to-face settings. Evidence on the use of signs, symptoms and early warning scores (EWS), either individually or in combination, to identify serious cases or predict potential to deteriorate (requiring a different level of monitoring and healthcare) was identified and summarised. This rapid evidence synthesis was undertaken as part of the NICE guideline process and was designed to align with the guideline development schedule timetable.

Aim and objectives

The review scope and questions were provided by NICE to meet the requirements of the guideline development process.

The aim of this rapid evidence synthesis was to assess the value and usefulness of, and clinical decision rules based on, different symptoms, signs and EWS (individually or in combination) for guiding management in patients with suspected ARI.

Review questions

In people aged 16 years or over with suspected ARI:

  1. What are the signs, symptoms and EWS that have been evaluated?

  2. What are the strategies for the triage of patients (e.g. applying clinical prediction rules using signs, symptoms, EWS thresholds) to avoid serious illness?

Clinical review methods

The evidence review was conducted following the methods and process described in Developing NICE guidelines: the manual. 1

Inclusion criteria

Population

People aged 16 years or over with suspected ARI [including bronchitis, common cold, glandular fever, influenza, laryngitis, sore throat (pharyngitis and tonsillitis), pneumonia and severe acute respiratory syndrome (SARS)].

Exclusion criteria: People aged 16 or over with a confirmed COVID-19 diagnosis, hospital inpatients (including those with hospital acquired respiratory infections), people who have a respiratory infection during end-of-life care, those with aspiration pneumonia, bronchiectasis, cystic fibrosis (CF) or known immunosuppression and children and young people under 16 years.

Phenomenon of interest

Signs, symptoms and externally validated EWS for the assessment of suspected ARI, including: cough, coughing up blood, purulent sputum, malaise, coryza, temperature/signs of fever, sore throat, hoarse voice, breathlessness and/or increased respiratory rate, wheeze/chest tightness, cyanosis, loss of appetite, lethargy, agitation, confusion, delirium, drowsiness, headache, rigors, chest pain, monitoring parameters based on digital technologies where available (e.g. pulse oximetry, peak flow), sudden deterioration in any of the above, EWS [including National Early Warning Score (NEWS/NEWS2), CRB65/CURB65, Centor criteria] and any combination of the above.

Setting

Remote settings (via telephone, video call, online app, e-mail or text message, e.g. NHS 111, 999 call centres or calls from GP practices) and face-to-face settings [e.g. the person’s home, a care home, primary care (including community pharmacy or ARI hubs), NHS walk-in centres, emergency departments].

Exclusion criteria: Hospital inpatient settings.

Outcomes

The outcomes of interest, assessed within 4 weeks of consultation:

  • hospital admission

  • escalation of care to any setting including:

    • face-to-face consultation

    • re-consultation/appointment

    • virtual ward

    • referral to ARI hub

    • emergency department visit

    • unplanned hospital admission

  • hospital length of stay

  • follow-up consultation/ongoing monitoring

  • antibiotic/antiviral use

  • time to clinical cure/resolution of symptoms

  • mortality.

The 4-week time period was chosen to ensure outcomes relevant solely to the assessment of signs, symptoms and EWS were identified.

Secondary outcomes were:

  • patient acceptability

  • patient preference

  • health-related quality of life (HRQoL) (using a validated scale).

Study design

Systematic reviews. No restrictions were applied based on the study designs included in the systematic reviews or on review date (as it is unlikely that symptoms and signs of suspected ARI have changed significantly over time).

Systematic reviews were identified by the use of all of the following:

  • clear and unambiguous eligibility criteria

  • comprehensive search (either stated as their aim or implied by use of two or more bibliographic databases)

  • details of included studies separately identifiable (e.g. with a table of characteristics and references for all included studies).

If no relevant systematic reviews were identified, primary studies would have been eligible for inclusion; prospective cohorts would have been the preferred cohort study type, but retrospective cohorts would have been considered. In some cases, comparative studies, including randomised controlled trials (RCTs), would have been relevant.

Search strategy for identification of systematic reviews

A systematic search of bibliographic databases was undertaken to identify systematic reviews relating to the assessment of signs, symptoms and EWS or strategies for triage of people with suspected ARI. The search strategy was developed in Ovid MEDLINE by an Information Specialist (MH) in consultation with the review team. The strategy was comprised of terms for respiratory infections combined (using the Boolean operator AND) with terms for the assessment of signs and symptoms, EWS or triage strategies. Text word searches in the title and abstract fields of records were included in the strategy along with relevant subject headings. The MEDLINE search strategy was checked by a second information specialist using aspects of the PRESS checklist. 2 The final MEDLINE strategy was adapted for use in all databases searched.

The following databases were searched on 15 May 2023:

  • MEDLINE ALL via Ovid

  • EMBASE via Ovid

  • Cochrane Database of Systematic Reviews via Wiley.

Searches were limited to systematic reviews published in English. Reference lists of relevant systematic reviews were screened to identify additional relevant reviews. Search results were imported into EndNote 20 (Clarivate Analytics, Philadelphia, PA, USA) for deduplication. All search strategies are presented in full in Appendix 1.

Study selection and data extraction

Studies were initially assessed for relevance using titles and abstracts. The study selection process was piloted on 2% (73) of the references to check consistency in screening decisions between reviewers. A single reviewer screened each identified title/abstract and 10% of records were checked by another reviewer, with discrepancies resolved through discussion. Full-text articles were independently screened by two reviewers, with discrepancies resolved through discussion and, where necessary, consultation with a third reviewer.

A data extraction form was developed using Microsoft Word® (Microsoft Corporation, Redmond, WA, USA), piloted and refined. Data on review characteristics (e.g. search strategy, inclusion/exclusion criteria, quality assessment methods, intervention and outcomes assessed), primary study characteristics (e.g. study location, setting, sample size, patient characteristics, quality), results and authors’ conclusions were extracted by one reviewer (RW or CU-C) and independently checked by a second reviewer (AE or RW). Any discrepancies were resolved through discussion.

Critical appraisal

Risk of bias was assessed using the Risk of Bias in Systematic Reviews (ROBIS) tool. 3 Risk of bias assessment was undertaken by one reviewer (RW or CU-C) and independently checked by a second reviewer (AE or RW). Any disagreements were resolved through discussion.

Clinical review results

Studies included in the review

The electronic searches identified a total of 3621 records after deduplication between databases. No additional records were identified from screening reference lists of relevant systematic reviews.

The full texts of 127 reviews were ordered for closer inspection; 118 were excluded at full paper stage and are listed in Appendix 2, along with the reasons for their exclusion. Nine studies met the review inclusion criteria. Figure 1 presents the flow of studies through the study selection process.

FIGURE 1.

Flow diagram of the study selection process.

Characteristics of the included reviews

Table 1 summarises the nine included reviews. Seven reviews included patients with community-acquired pneumonia (CAP), one included patients with nursing home-acquired pneumonia (NHAP) and one included patients with sore throat symptoms. While we only included reviews of patients in a community setting (i.e. not hospitalised patients), the setting of most studies included the emergency department, walk-in medical centre and/or acute medical unit, rather than exclusively primary care. No reviews included studies of remote settings. Reviews were published between 2005 and 2021 and the studies included in the reviews were published between 1975 and 2018. Where reported, most included studies were conducted in the USA, Canada, Europe and the UK.

Study details Population Setting Prognostic factors/prognostic model(s) Outcomes Risk of bias
Individual signs/symptoms and Centor score for adults presenting with sore throat symptoms
Aalbers (2011)4

Systematic review including 21 studies		 Adults (≥ 15 years of age) presenting with sore throat symptoms Primary care and the emergency department (USA, Canada, Europe, New Zealand, Thailand, Israel) Individual signs and symptoms (absence of cough, fever, anterior cervical adenopathy, tender anterior cervical adenopathy, any exudates) and Centor score Usefulness of individual signs and symptoms in assessing the risk of streptococcal pharyngitis and diagnostic accuracy of the Centor score as a decision rule for antibiotic treatment Low
EWS for patients with CAP
Akram (2011)5

Systematic review including 13 studies		 Outpatients with CAP Outpatients; either exclusively managed in the community or discharged from an emergency department < 24 hours after admission (USA, Canada, Netherlands, Germany, Spain, France, UK) CRB65, CURB65 and PSI Outpatient mortality and diagnostic accuracy Low
Chalmers (2011)6

Systematic review including six studies		 Outpatients with CAP Emergency department and walk-in medical centre (USA, Canada, Spain, France) PSI and other criteria for assessing severity/requirement for inpatient care Proportion of patients treated as outpatients, mortality, hospital re-admissions, HRQoL, return to usual activities and patient satisfaction with care Low
Ebell (2019)7

Systematic review including 29 studies; 15 were in emergency department or primary care settings (update of McNally 2010)		 Patients with CAP The review included hospitalised patients, ambulatory patients and both; the 15 studies that included patients in emergency department or primary care settings are relevant to this review (most studies from Europe) CRB-65 Prediction of mortality High
McNally (2010)8

Systematic review including 14 studies; 4 included community-based patients		 Adults (≥ 16 years of age) with a primary diagnosis of CAP The review included hospitalised patients, primary care patients and patients treated as outpatients; the four studies that included primary care patients and patients treated as outpatients are relevant to this review (study location not reported) CRB-65 30-day mortality Low
Metlay (2019)9

Systematic review including seven studies relating to the question of interest		 Adults diagnosed with CAP Inpatient vs. outpatient treatment location (study location not reported) PSI and CURB-65 Initial site of treatment High
Nannan Panday (2017)10

Systematic review including 42 studies; 4 included patients with CAP or respiratory distress		 Adults (≥ 16 years of age) at the emergency department or acute medical unit Emergency department and acute medical unit (Denmark, Netherlands, Norway, Germany, Hong Kong, Ireland, Israel, Italy, Singapore, South Africa, South Korea, Sri Lanka, Sweden, Switzerland, Turkey, UK, USA and Vietnam) Twenty-five different types of EWS. For the four studies relevant to our question, the scores assessed were CREWS, CRB-65, CURB-65, NEWS,a PSI, SIRS, SEWS and S-NEWS Prediction of mortality and/or ICU admission Low
Smith (2021)11

Systematic review including 38 studies relating to the question of interest		 Adult emergency department patients diagnosed with CAP Emergency department (USA, Spain, Switzerland, Australia, Canada, China, France, Japan, Korea, Turkey, UK and Europe, where reported) PSI and CURB-65 for predicting mortality. Five clinical decision aids for predicting the need for ICU admission: ATS 2001, IDSA/ATS 2007, SCAP (SCAP/CURXO-80), SMART-COP, REA-ICU Prediction of mortality (PSI and CURB-65) and prediction of need for ICU admission (ATS 2001, IDSA/ATS 2007, SCAP/CURXO-80, SMART-COP and REA-ICU) Unclear
EWS for patients with NHAP
Dosa (2005)12

Systematic review including three studies relating to the question of interest		 Nursing home residents with NHAP Nursing homes (USA) PSI, a 5-point scale developed by Naughton and Mylotte and an 8-variable model developed by Mehr et al. Prediction of mortality High

ATS, American Thoracic Society; CREWS, Chronic Respiratory Early Warning Score; ICU, intensive care unit; IDSA/ATS, Infectious Diseases Society of America/American Thoracic Society; MEDS, Mortality in Emergency Department Sepsis score; MEWS, Modified Early Warning Score; NEWS, National Early Warning Score; PSI, Pneumonia Severity Index; REA-ICU, Risk of Early Admission to the ICU; REMS, Rapid Emergency Medicine Score; SCAP, severe CAP; SEWS, Standardised Early Warning Score; SIRS, systemic inflammatory response syndrome; S-NEWS, Salford National Early Warning Score.

NEWS was updated to NEWS2 in December 2017, after the Nannan Panday review was published.

Quality and applicability of the included reviews

Risk of bias was assessed using the ROBIS tool. 3 Five of the included reviews had a low overall risk of bias. Three reviews had a high overall risk of bias; two had a high risk of bias for every domain assessed,9,12 while one had a low risk of bias for most domains, but a high risk of bias owing to a very limited search strategy. 7 One review had an unclear risk of bias due to very limited reporting of review methods. Table 2 presents the risk of bias assessment results.

Review Phase 2 risk of bias Phase 3
1. Study eligibility criteria 2. Identification and selection of studies 3. Data collection and study appraisal 4. Synthesis and findings Risk of bias in the review
Aalbers (2011)4 Low Low Low Low Low
Akram (2011)5 Low Low Low Low Low
Chalmers (2011)6 Low Low Low Low Low
Dosa (2005)12 High High High High High
Ebell (2019)7 Low High Low Low High
McNally (2010)8 Low Low Low Low Low
Metlay (2019)9 High High High High High
Nannan Panday (2017)10 Low Low Unclear Low Low
Smith (2021)11 Unclear Low Unclear Unclear Unclear
Total High: 2 High: 3 High: 2 High: 2 High: 3
Unclear: 1 Unclear: 0 Unclear: 2 Unclear: 1 Unclear: 1
Low: 6 Low: 6 Low: 5 Low: 6 Low: 5

In addition to risk of bias, the applicability of the included reviews to the research question was assessed. Five reviews had good applicability to the research question. 4,5,8,11,12 Four reviews had acceptable applicability; details are presented in Table 3.

Review Applicability Details
Aalbers (2011)4 Good
Akram (2011)5 Good
Chalmers (2011)6 Acceptable Scoring system to identify low-risk patients was only one component of the interventions assessed
Dosa (2005)12 Good
Ebell (2019)7 Acceptable The population included both hospitalised and ambulatory patients, despite the setting being the emergency department or primary care
McNally (2010)8 Good
Metlay (2019)9 Acceptable Review undertaken to inform a guideline assessing multiple questions, the question on use of a clinical prediction rule plus clinical judgement vs. clinical judgement alone was relevant
Nannan Panday (2017)10 Acceptable Review addressed a much broader question; results are presented for the subgroup of studies relevant to our review question (patients with suspected CAP or respiratory distress)
Smith (2021)11 Good

Results of the included reviews

A summary of the results of the included reviews is presented below. Detailed tables of the characteristics and results of the reviews are presented in Appendix 3. Appendix 4 provides details of the components and score range of the EWS assessed in the included reviews.

Individual signs/symptoms and the Centor score for adults presenting with sore throat symptoms

One systematic review assessed the usefulness of individual signs and symptoms in assessing the risk of streptococcal pharyngitis and the diagnostic accuracy of the Centor score as a decision rule for antibiotic treatment in adults (≥ 15 years) presenting to primary care (19 studies) or the emergency department (2 studies) with symptoms of sore throat. 4 The review, published in 2011, included 21 diagnostic accuracy studies from the USA, Canada, Europe, New Zealand, Thailand and Israel that were published between 1975 and 2008; the overall quality of the included studies was considered to be good. The prevalence of Group A β-haemolytic streptococcal (GABHS) pharyngitis varied widely between studies, ranging from 4.7% to 37.6%. All 21 studies (n = 4839 patients) reported data on signs and symptoms and 15 studies (n = 2900 patients) reported data on the Centor score. Individual signs and symptoms assessed were absence of cough, fever, anterior cervical adenopathy, tender anterior cervical adenopathy and any exudates (tonsillar exudate, pharyngeal exudate or any exudate). The reference standard was throat culture. Summary diagnostic accuracy results (sensitivity, specificity, positive and negative likelihood ratios) are presented in Appendix 3.

The authors concluded that individual symptoms and signs have only a modest ability to rule in or out a diagnosis of GABHS pharyngitis. They concluded that the Centor score (cut-off score of ≥ 3) has reasonably good specificity and can enhance the appropriate prescribing of antibiotics but should be used with caution in settings with a low prevalence of GABHS pharyngitis, such as primary care. This review had a low risk of bias and the conclusions appear to be appropriate.

Early warning scores for patients with community-acquired pneumonia

Seven systematic reviews assessed EWS for patients with CAP,5–11 primarily for the prediction of mortality and/or to determine the site of treatment [inpatient vs. outpatient care or requirement for intensive care unit (ICU) admission]. Full details are presented in Appendix 3. The most commonly assessed EWS were the Pneumonia Severity Index (PSI; four reviews),5,6,9,11 CRB-65 (three reviews)5,7,8 and CURB-65 (three reviews). 5,9,11 One review assessed a range of EWS; those assessed in the subgroup of studies of patients with CAP or respiratory distress were the Chronic Respiratory Early Warning Score (CREWS), CRB-65, CURB-65, National Early Warning Score (NEWS), PSI, systemic inflammatory response syndrome (SIRS), Standardised Early Warning Score (SEWS) and Salford National Early Warning Score (S-NEWS). 10 None of the reviews assessed NEWS2; NEWS was updated to NEWS2 in December 2017, after the Nannan Panday review was published. The setting of the included studies encompassed primary care, walk-in medical centre, emergency department and acute medical, unit and most of the included studies were from the USA, Canada and Europe, where stated, and they were published between 1997 and 2018. Study quality was assessed using a range of different tools with variable results; however, many of the included studies were considered to have significant limitations/a moderate to high risk of bias. One review7 was an update of another of the included reviews. 8 There was a great deal of overlap in included primary studies between the reviews; Table 4 shows the 11 studies that were included in more than one of the reviews.

Included studies Akram, 20115 Chalmers, 20116 Ebell, 20197 McNally, 20108 Metlay, 20199 Nannan Panday, 201710 Smith, 202111
Atlas, 1998 ✔ ✔ ✔ ✔
Bauer, 2006 ✔ ✔ ✔
Bont, 2008 ✔ ✔ ✔
Capelastegui, 2006 ✔ ✔ ✔ ✔
Carratala, 2005 ✔ ✔ ✔
Fine, 1997 ✔ ✔
Julian-Jiminez, 2013 ✔ ✔
Kruger, 2008 ✔ ✔
Marrie, 2000 ✔ ✔ ✔
Renaud, 2007 ✔ ✔ ✔
Yealy, 2005 ✔ ✔ ✔ ✔

Two systematic reviews had a low risk of bias and good applicability to the review question. 5,8 Two had a low risk of bias, but poorer applicability as the risk scoring system was only one component of the interventions assessed,6 or the population also included patients with suspected exacerbation of chronic obstructive pulmonary disease (COPD). 10 One review had an unclear risk of bias as there was limited methodological detail reported but good applicability. 11 Two reviews had a high risk of bias, owing to a limited search strategy and/or poor reporting with limited details of the included studies. 7,9 The reviews judged to be at low risk of bias, assessed using the ROBIS tool,3 were considered to be good quality.

A good-quality systematic review, published in 2011, concluded that patients in low-risk PSI and CRB-65 classes were found to be at low risk of death when managed as outpatients, but that further studies are needed in outpatient cohorts; this review included studies of patients managed exclusively in the community or discharged from an emergency department within 24 hours. 5 Another good-quality review, published in 2010, concluded that the CRB-65 has not been validated sufficiently in primary care settings and preliminary findings suggest over-prediction, so its value as a prognostic indicator in the community remains unclear. 8

A good-quality review published in 2017 concluded that NEWS generally had favourable results in the emergency department or acute medical unit setting for all end points; for mortality prediction, NEWS was the most accurate score in those with respiratory distress. 10 ICU admission was best predicted with NEWS. The authors stated that future studies should concentrate on a simple and easy-to-use prognostic score such as NEWS with the aim of introducing this throughout the (pre-hospital and hospital) acute care chain.

The final good-quality systematic review, with poorer applicability due to the risk of scoring system being only one component of the interventions assessed, concluded that strategies to increase the proportion of patients treated in the community are safe, effective and acceptable to patients. 6

A review with an unclear risk of bias, published in 2021, including patients in an emergency department setting, concluded that the PSI and CURB-65 are both well-validated clinical decision aids that can predict short-term mortality in patients with CAP and can be used to identify low-risk patients for whom outpatient management may be considered. 11 The authors stated that both aids are appropriate for this purpose in the emergency care setting; the PSI appears to be slightly better at identifying low-risk patients, but requires data from a greater number of tests, including some not routinely conducted in the emergency department. They further stated that for decisions regarding ICU admission, clinical decision aids designed for this purpose (such as the IDSA/ATS 2007) should be considered superior to the PSI and CURB-65.

One of the reviews with a high risk of bias, which included patients in emergency department and primary care settings, concluded that the CRB-65 can be used by physicians to estimate mortality risk and can serve as a useful check on physician judgement; patients in the low-risk group with a score of 0 have a very low mortality risk and can, in most cases, safely be treated as outpatients, while most patients in the moderate- and high-risk groups should be hospitalised (although other considerations may alter these decisions regarding treatment setting). 7 The other review with a high risk of bias recommended that clinicians use a validated clinical prediction rule for prognosis, in addition to clinical judgement, to determine the need for hospitalisation; preferentially the PSI over the CURB-65. 9

In summary, it appears that further research is needed to validate the PSI and CRB-65 in primary care/community settings. However, the PSI requires data from a large number of tests, some of which are not routinely conducted in primary care/community settings. The PSI and CURB-65 appear to be useful for predicting short-term mortality and identifying low-risk patients who may be considered for outpatient management when used in an emergency department setting, although some tests required for the PSI may not be routinely conducted in an emergency department setting (such as arterial blood gases). NEWS appears to be useful in an emergency department or acute medical unit setting for predicting mortality and was useful for predicting need for ICU admission. The ATS 2001 and IDSA/ATS 2007 appear to be superior to the PSI and CURB-65 for decisions regarding ICU admission.

Early warning scores for patients with nursing home-acquired pneumonia

One systematic review with a high risk of bias assessed the PSI, a 5-point scale developed by Naughton and Mylotte, and an eight-variable model, developed by Mehr et al. , for predicting mortality in nursing home residents with NHAP. 12 Three studies, conducted between 1998 and 2001 in USA nursing homes, related to the question of interest; one study assessed each EWS. The review does not appear to have assessed the quality of the included studies. The authors concluded that there are numerous problems with using current models in clinical practice, such as the fact that mortality prediction models are generally age-driven and, therefore, as nursing home residents are generally very old, this eliminates one of the most discriminating features of the probability model. Prediction models do not incorporate the resident’s end-of-life wishes or overall goals of care. Current models for predicting mortality require data collection that is often not readily available at the time that triage decisions need to be made. While the issues discussed appear to be relevant considerations when assessing the use of EWS in a nursing home setting, the review was poorly conducted and reported, and it is unclear whether relevant studies were missed and whether the included studies were valid.

Review of economic studies

The economic evidence review was conducted following the methods and process described in Developing NICE guidelines: the manual. 1

Inclusion criteria

Population

People aged 16 years or over with suspected ARI [including bronchitis, common cold, glandular fever, influenza, laryngitis, sore throat (pharyngitis and tonsillitis), pneumonia and SARS].

Exclusion criteria: People aged 16 or over with a confirmed COVID-19 diagnosis, hospital inpatients (including those with hospital acquired respiratory infections), people who have a respiratory infection during end-of-life care and those with aspiration pneumonia, bronchiectasis, CF or known immunosuppression and children and young people under 16 years.

Phenomenon of interest

Signs, symptoms and externally validated EWS for the assessment of suspected ARI, including: cough, coughing up blood, purulent sputum, malaise, coryza, temperature/signs of fever, sore throat, hoarse voice, breathlessness and/or increased respiratory rate, wheeze/chest tightness, cyanosis, loss of appetite, lethargy, agitation, confusion, delirium, drowsiness, headache, rigors, chest pain, monitoring parameters based on digital technologies where available (e.g. pulse oximetry, peak flow), sudden deterioration in any of the above, EWS (including NEWS/NEWS2, CRB65/CURB65, Centor criteria) and any combination of the above.

Setting

Remote settings (via telephone, video call, online app, e-mail or text message, e.g. NHS 111, 999 call centres or calls from GP practices) and face-to-face settings [e.g. the person’s home, a care home, primary care (including community pharmacy or ARI hubs), NHS walk-in centres, emergency departments].

Exclusion criteria: Hospital inpatient settings.

Outcomes

No explicit criteria were applied in the cost-effectiveness review; however, outcomes reported in the relevant study designs were considered. These included:

  • costs

  • life years

  • quality-adjusted life-years (QALYs)

  • incremental costs and QALYs

  • incremental cost-effectiveness ratio (ICER).

Study design

Full economic evaluations comparing two or more alternatives in terms of both costs and consequences. Only cost-minimisation, cost-effectiveness, cost–utility and cost–benefit analyses were considered for inclusion.

Search strategy for identification of economic evaluations

The aim of the search was to identify economic evaluations relating to the assessment of signs and symptoms, EWS or strategies for triage in people with suspected ARI. The search strategy designed in Ovid MEDLINE by an Information Specialist (MH) for the identification of systematic reviews (as documented in Inclusion criteria) was adapted for use in the databases and searched by another Information Specialist (HF). The strategy was comprised of terms for respiratory infections combined (using the Boolean operator AND) with terms for the assessment of signs and symptoms, EWS or triage strategies. Text word searches in the title and abstract fields of records were included in the strategy along with relevant subject headings.

The following databases were searched on 15 May 2023:

  • MEDLINE ALL via Ovid

  • EMBASE via Ovid

  • EconLit via Ovid

  • NHS Economic Evaluation Database via CRD.

Searches were limited to economic evaluations published in English. Search results were imported into EndNote 20 (Clarivate Analytics, Philadelphia, PA, USA) for deduplication. All search strategies are presented in full in Appendix 1.

Study selection and data extraction

Studies were initially assessed for relevance using titles and abstracts. The study selection process was initially piloted on 10% (263) of total references for consistency between reviewers, with the remaining references independently screened by two reviewers and any disagreements resolved by consensus. Full-text articles were independently screened by two reviewers, with discrepancies resolved through discussion.

A data extraction form was developed using Microsoft Excel® (Microsoft Corporation, Redmond, WA, USA). Data on review characteristics (e.g. study design, perspective, intervention and outcomes assessed), study characteristics (e.g. study location, setting, sample size, patient characteristics, costs, time horizon), results and authors’ conclusions were extracted by one reviewer (NJD) and independently checked by a second reviewer (RH). Any discrepancies were resolved through discussion.

Quality assessment

Quality was assessed using the NICE economic evaluations checklist. 1 The quality assessment was undertaken by one reviewer (NJD) and independently checked by a second reviewer (RH). Any disagreements were resolved by consensus.

Cost-effectiveness review results

Studies included in the review

A total of 2622 records were identified through economic searches after deduplication between databases. The full texts of 13 reviews were ordered for closer inspection; 12 were excluded at full paper stage and are listed in Appendix 2, along with the reasons for their exclusion. Only one study met the economic review inclusion criteria. Figure 2 presents the flow of studies through the study selection process.

FIGURE 2.

Flow diagram of the economic study selection process.

Characteristics of the included study

Only one study, a trial-based economic evaluation, was included in the economic review; study characteristics are summarised in Table 5. The aim of the study was to assess the resource use and health impact associated with different methods of targeting antibiotics for the treatment of streptococci in patients attending primary care with an acute sore throat. The interventions assessed were a clinical score or a rapid antigen detection test (RADT) compared with delayed antibiotic prescription.

Study details Setting and location Study design Study population Sample size Intervention Comparator
Little et al. 201413 UK primary care Trial-based economic analysis Population: patients aged ≥ 3 years and had acute sore throat 613 participants (delayed group, n = 207; clinical score, n = 211; rapid test, n = 213)

  1. Clinical score (FeverPAIN)

  2. RADT

  1. Delayed antibiotic prescribing

Quality assessment

A quality assessment of the included study was conducted using the NICE economic evaluations checklist presented in Appendix 5. This study is only partially relevant to the review question as it involved a diagnostic strategy in addition to examining a clinical score and included children as well as adults; it had minor limitations as it assessed a short-term ARI. The study, however, highlights the possible impact of using symptoms to assess short-term ARI conditions. The assessment suggested no significant methodological concerns.

Results of the included economic evaluation

The economic evaluation methods conducted in the included study were a cost–utility analysis and a cost-effectiveness analysis (further details in Appendix 6). The identified study, Little et al. (2014),13 utilised outcomes from the PRImary care Streptococcal Management (PRISM) RCT which evaluated the clinical and cost-effectiveness of a clinical score and RADT for sore throats, compared to delayed (antibiotic) prescribing. The study adopted a NHS and Personal Social Services (PSS) perspective and had a time horizon of 28 days. The outcome measures assessed were clinical symptom score (based on the mean rating of sore throat and difficulty of swallowing for days 2–4) and EuroQol-5 Dimensions (EQ-5D)-3L scores (measured on day 14). These outcomes were respectively used in the reported cost-effectiveness and cost–utility analysis. Costs and resource use captured included those needed to directly provide the interventions (practitioner time and cost of test) as well as subsequent care costs. The latter included subsequent antibiotic acquisition administration costs, accident and emergency visits and inpatient hospitalisation costs. There was no discounting of costs or outcomes due to the short time horizon (28 days).

Mean severity scores were lower in the clinical score group compared to the delayed prescribing group: −0.33 (95% CI −0.64 to −0.02). A similar reduction was also observed in the RADT group: −0.30 (95% CI −0.61 to 0.004) compared to delayed prescribing. The authors commented that this is equivalent to one in three patients rating sore throat severity as slight rather than a moderately bad problem. The study found no statistically significant differences, with wide confidence intervals (CIs), in QALYs gained among the three participant groups. This uncertainty may stem from the fact that the EQ-5D scores were obtained from a smaller data set, which was not powered to reflect small differences in quality of life. Furthermore, QALYs were estimated from EQ-5D scores captured on day 14. The authors noted that there is a possibility that a significant number of individuals could have already recovered before the day 14 assessment, resulting in their health returning to normal. As a result, the EQ-5D scores at 14 days, and consequently the difference in QALYs, may not strongly correlate with changes in symptom scores. The authors also considered that EQ-5D may not accurately capture changes in HRQoL due to its potential lack of sensitivity.

Differences in mean costs between the three groups were largely attributed to the first recruitment visit and duration of that visit. The duration of contact reported by GPs was comparable between the delayed and clinical score groups, but slightly longer in the RADT group. As a result of this disparity and the cost associated with the diagnostic test, RADT was associated with higher implementation costs compared to both the delayed prescribing and clinical symptom score groups. The clinical score and RADT groups were also associated with lower antibiotic prescription compared to the delayed group, resulting in cost savings relative to delayed prescribing.

The findings of this study indicated that, from a NHS perspective, the clinical score was likely to be the most cost-effective strategy compared to both RADT and delayed (antibiotic) prescribing.

The cost-effectiveness analysis found that the clinical score was more clinically effective and less costly than RADT. However, the difference in point estimates for symptom severity scores between clinical score (2.83, 95% CI 2.61 to 3.05) and RADT (2.84, 95% CI 2.62 to 3.07) were marginal with overlapping CIs. Both the clinical score and RADT were found to dominate delayed prescribing, generating greater benefits at lower cost.

Although the cost–utility analysis demonstrated considerable uncertainty around the QALY estimates, the results suggested that the clinical score was the most likely to be cost-effective, particularly at lower willingness-to-pay thresholds. RADT was the most effective intervention in the cost–utility analysis, yielding marginally higher QALY gains than the clinical score group. Resulting pairwise ICERs for RADT compared with the clinical score were £74,286 and £24,528 per QALY at 14 and 28 days’ follow-up, respectively. As per the cost-effectiveness analysis, both the clinical score and RADT were found to dominate delayed prescribing, generating greater benefits at lower cost.

Discussion

Summary of findings

The aim of this rapid evidence synthesis was to assess the value and usefulness of, and clinical decision rules based on, different symptoms, signs and EWS (individually or in combination) for guiding management in patients with suspected ARI. A summary of the findings relating to both review questions is presented below.

Review question 1: In people aged 16 years or over with suspected ARI, what are the signs, symptoms and EWS that have been evaluated?

Only one systematic review assessed the usefulness of individual signs and symptoms, in assessing the risk of GABHS pharyngitis in adults (aged 15 years or over) presenting to primary care or the emergency department with sore throat. Individual signs and symptoms (absence of cough, fever, anterior cervical adenopathy, tender anterior cervical adenopathy and any exudates) were found to have only a modest ability to rule in or out a diagnosis of GABHS pharyngitis.

Several EWS have been evaluated in people aged 16 years or over with suspected ARI: Centor, CRB-65, CURB-65, PSI, CREWS, NEWS, SIRS, SEWS, S-NEWS, ATS 2001, IDSA/ATS 2007, SCAP/CURXO-80, SMART-COP and REA-ICU. Nine systematic reviews addressed this research question – all assessed patients presenting in face-to-face settings (primary care, walk-in medical centre, emergency department, acute medical unit or nursing home) rather than remote settings. The most commonly assessed EWS were the PSI, CRB-65 and CURB-65.

Review question 2: In people aged 16 years or over with suspected ARI, what are the strategies for the triage of patients (e.g. applying clinical prediction rules using signs, symptoms, EWS thresholds) to avoid serious illness?

The evidence was insufficient to definitively answer this question.

Seven systematic reviews assessed EWS for predicting mortality and/or to determine the treatment setting for patients with CAP. There was a great deal of overlap in the primary studies included in the reviews and many of the primary studies were considered to have significant limitations.

Two reviews that assessed the CRB-65 (both good quality) concluded that further research is needed in community settings. One of these reviews also assessed the PSI; however, the PSI requires data from a large number of tests, some of which are not routinely conducted in community settings. One review (also good quality) concluded that NEWS appears to provide the most accurate score for predicting mortality and the need for ICU admission in patients with respiratory distress in an emergency department or acute medical unit setting.

One review (good quality) concluded that individual symptoms and signs (absence of cough, fever, anterior cervical adenopathy, tender anterior cervical adenopathy, any exudates) have only a modest ability to rule in or out a diagnosis of streptococcal pharyngitis in adults presenting to primary care or the emergency department with sore throat. The review concluded that the Centor score (cut-off ≥ 3) has reasonably good specificity and can enhance the appropriate prescribing of antibiotics for streptococcal pharyngitis, but that it should be used with caution in low-prevalence settings, such as primary care.

Only one review (poor quality) assessed the use of EWS (PSI and two other scores) for predicting mortality in nursing home residents with NHAP; the review concluded that there are numerous problems with using the scores in clinical practice.

The economic evidence review identified a single study indicating that clinical scores may be a cost-effective approach to triage patients compared with delayed prescribing. The study also offers insight into the cost-effectiveness of diagnostic testing in ARI scenarios. In this particular case, the findings indicated that there is no apparent advantage in incorporating diagnostic testing alongside clinical scores compared to using clinical scores alone. The cost-effectiveness analysis also found that the clinical score group and RADT group were associated with lower antibiotic use compared to delayed (antibiotic) prescribing. This may represent a positive externality not formally captured by the economic analysis.

Strengths and limitations

This rapid evidence synthesis was undertaken using systematic methods, reducing the potential for errors and bias; inclusion and exclusion criteria were clearly defined in advance, the validity and applicability of the included studies were assessed using relevant tools, data extraction and validity assessment were independently checked and studies were synthesised using appropriate methods.

The review was designed to align with the NICE guideline development schedule; the clinical evidence review was thereby limited to systematic reviews in the first instance, rather than synthesising evidence from primary studies. There was a great deal of duplication in the primary studies, often with identified limitations, that were included in the reviews of EWS for CAP, potentially reinforcing review conclusions based on the same low-quality evidence. The review was also restricted to studies of suspected ARI; reviews relating to more general symptom assessment were not eligible but could potentially provide valuable information. Owing to the requirements of the NICE guideline development schedule, the searches were restricted to English language literature and only a small number of bibliographic databases was searched, along with screening reference lists. Therefore, it is possible that relevant systematic reviews and economic evaluations were not included. Clinician and patient perspectives on the review findings were provided during deliberations at the NICE Guideline Committee stage.

No reviews were identified that considered the use of signs, symptoms and EWS in remote settings; reviews reported only studies undertaken in face-to-face settings (primarily the emergency department and/or primary care) and none compared face-to-face versus remote settings. No reviews reported data on several of the outcomes of interest, including ongoing monitoring, resolution of symptoms, HRQoL and patient preference.

Limited relevant cost-effectiveness evidence was identified with only one study included in the cost-effectiveness review. The study only partially met the criteria concerning the intervention because it involved evaluating a diagnostic strategy in addition to examining a clinical symptom score; nonetheless, by examining a clinical score in conjunction with standard care, the study might offer insights into the potential cost-effectiveness of implementing a clinical score-based approach for the triage of ARIs.

There was uncertainty in study results due to small differences in QALYs gained across the three intervention groups. This may have resulted from QALYs being estimated from EQ-5D scores at baseline and day 14, whereas values were carried forward from daily visual analogue scores where symptoms resolved before day 14. As a result, the differences in QALYs may not be strongly correlated with changes in symptom scores and may not appropriately capture changes in quality of life. Furthermore, although there is substantial evidence from the analysis regarding the clinical benefits of clinical scores, the evidence also shows that these scores represent a low-cost intervention; thus conducting a cost-effectiveness analysis may not be worthwhile. It is unclear whether these results are generalisable to the broader assessment of other ARI conditions. Differences in severity, duration of disease and probability of escalation or complications, however, likely limit inferences to the indication considered.

Implications for future research

A comprehensive systematic review of primary studies, informed by a range of expert perspectives, and assessing signs, symptoms and EWS in adults with symptoms suggestive of ARI (including non-ARI conditions) summarising available data on important outcomes (including ongoing monitoring, resolution of symptoms, HRQoL and patient preference), could inform and guide management of patients with suspected ARI, helping determine which triage strategies avoid serious illness. Where possible, studies of patients seen in face-to-face settings should be assessed separately to those in virtual settings (e.g. NHS 111, 999 call centres, calls from GP practices and ARI hubs). Subgroups of interest include patients with chronic comorbidity (e.g. COPD) and different patient ages; several EWS include components relating to age.

Two good-quality reviews identified concluded that further research is required to validate the CRB-65 and PSI in primary care/community settings; current evidence suggests overprediction, owing to low mortality rates in these settings. However, the applicability of the PSI in community settings remains unclear, since it requires data from a large number of tests, some of which are not routinely conducted in community settings.

Critical to all future research in this area is proper consideration of the context in which consultation, assessment, treatment and triage decisions are being taken, as well as how patients access and experience these. Patient characteristics have considerable implications for the effectiveness and cost-effectiveness of different strategies; in making decisions, clinicians often need to take account of general physical health and frailty, as well as patient knowledge, experience and understanding. Applicability of future research must also be considered; the variety of available settings and care pathways, as well as the introduction of new resources and technologies to inform decision-making, will have implications for the interpretation and implementation of findings.

While there is limited existing economic evidence, the single study identified may help inform the design of future studies. The acute nature of ARIs lends them to trial-based rather than model-based evaluations due to the dynamic nature, in terms of urgency and rapid onset, of ARIs; it also means an economic evaluation need only consider a short time horizon permitting the evaluation of all differences in costs and benefits within a trial setting. Future trials of triage strategies for ARIs should include an economic evaluation wherever possible to assess the cost-effectiveness of specific triage strategies.

The design of future trial-based economic evaluations should consider that the incremental costs and benefits for alternative triage strategies may be small and therefore future trials should be adequately powered to detect differences between groups. In line with best practice, future economic evaluations (either trial or model based) should not only appropriately consider uncertainty in results but should also consider extending probabilistic analysis to evaluate the value of information. This will help better inform the value of future clinical and economic evaluations.

Cost–utility analysis is likely to be the preferred approach as it conforms to decision-making standards in the UK. However, collecting appropriate quality-of-life data, such as EQ-5D, might be challenging in the context of acute infections with short durations. For this reason, delayed data collection should be avoided to maintain statistical power and detect QALY differences. Where data collection is problematic, conducting supplementary cost-effectiveness analysis using relevant clinical outcomes may be helpful. However, interpreting the results of such analysis can be difficult except in the limiting case where one technology clearly dominates others.

Patient and public involvement

Patient and public involvement (PPI) routinely forms part of the NICE guideline development process. To align with the NICE guideline development schedule, PPI in this study took place indirectly. Feedback on the report compiled to inform discussion at the NICE Guideline Committee stage was received from NICE and considered in the development of this manuscript. Any relevant comments from PPI stakeholders have been incorporated. Only one point of clarification was identified; this is reflected in the text in the Discussion (see Summary of findings) relating to Research Question 1.

Equality, diversity and inclusion

The applicability and generalisability of the available systematic review evidence, and clear gaps in the evidence base (particularly in terms of settings and patient groups), were considered in the characteristics of the included reviews (see Characteristics of the included reviews), the quality of the included reviews (see Quality and applicability of the included reviews), the results of the included reviews (see Results of the included reviews), as well as the discussion (see Summary of findings).

The assessment and management of signs, symptoms and EWS in important patient subgroups identified by NICE (including patients with comorbidities and those in different age groups) were considered throughout the project.

In writing this report, as far as possible, we have tried to ensure use of accessible language and terminology, including provision of definitions as required.

Conclusions

Several EWS have been evaluated in people aged 16 years or over with suspected ARI in face-to-face settings; the most commonly assessed EWS were the PSI, CRB-65 and CURB-65. No reviews assessed the use of EWS in remote settings. Most of the included reviews assessed the ability of EWS to predict short-term mortality and/or determine the site of treatment for patients with CAP. Some EWS (NEWS, CURB-65 and PSI) appear to be useful in an emergency department/acute medical setting; however, further research is needed to validate the CRB-65 and PSI in primary care/community settings (although PSI requires data from a large number of tests, some of which are not routinely conducted in community settings). While individual symptoms and signs have only a modest ability to rule in or out a diagnosis of streptococcal pharyngitis, the Centor score (cut-off score of three) may enhance the appropriate prescribing of antibiotics but should be used with caution in low-prevalence settings, such as primary care. There appear to be numerous problems with using EWS (e.g. PSI) in a nursing home setting.

There is a paucity of cost-effectiveness evidence for the use of signs, symptoms and EWS in guiding the management of most ARIs with only one study identified in sore throat. The cost-effectiveness evidence obtained suggested that clinical scores are likely to be cost-effective compared to both RADT and delayed prescribing. Results were, however, uncertain due to the small differences in costs and benefits, making it difficult to draw firm conclusions.

Overall, the information available from existing systematic reviews to guide decision-making is limited, with clear implications for future research.

Additional information

CRediT contribution statement

Ros Wade (https://orcid.org/0000-0002-8666-8110): investigation (equal), writing – original draft (lead), writing – reviewing and editing (lead).

Nyanar Jasmine Deng (https://orcid.org/0009-0006-5901-1101): investigation (equal), writing – original draft (equal).

Chinyereugo Umemneku-Chikere (https://orcid.org/0000-0003-4114-2227): investigation (equal), writing – original draft (supporting).

Melissa Harden (https://orcid.org/0000-0003-2338-6869): investigation (equal), writing – original draft (equal).

Helen Fulbright (https://orcid.org/0000-0002-1073-1099): investigation (equal), writing – original draft (equal).

Robert Hodgson (https://orcid.org/0000-0001-6962-2893): investigation (equal), writing – original draft (equal), writing – reviewing and editing (equal).

Alison Eastwood (https://orcid.org/0000-0003-1079-7781): investigation (equal), writing – original draft (equal).

Rachel Churchill (https://orcid.org/0000-0002-1751-0512): investigation (supporting), writing – original draft (equal), writing – reviewing and editing (equal), funding acquisition (lead), project administration (lead).

Disclosure of interests

Full disclosure of interests: Completed ICMJE forms for all authors, including all related interests, are available in the toolkit on the NIHR Journals Library report publication page at https://doi.org/10.3310/GRPL6978.

Primary conflicts of interest: Rachel Churchill – Evidence Synthesis Programme Advisory Group (2016–20).

Data-sharing statement

All available data can be obtained by contacting the corresponding author.

Ethics statement

This project did not require ethical approval, as the study design was a rapid evidence synthesis of systematic reviews and cost-effectiveness studies.

Information governance statement

This project did not handle any personal information.

Department of Health and Social Care disclaimer

This publication presents independent research commissioned by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, MRC, NIHR Coordinating Centre, the Health Technology Assessment programme or the Department of Health and Social Care.

This article was published based on current knowledge at the time and date of publication. NIHR is committed to being inclusive and will continually monitor best practice and guidance in relation to terminology and language to ensure that we remain relevant to our stakeholders.

Funding

This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme as award number NIHR159945.

This article reports on one component of the research award Initial assessment and management of adults with suspected acute respiratory infection: a rapid evidence synthesis of reviews and costeffectiveness studies. For more information about this research please view the award page [https://www.fundingawards.nihr.ac.uk/award/159945].

About this article

The contractual start date for this research was in April 2023. This article began editorial review in June 2023 and was accepted for publication in March 2024. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The Health Technology Assessment editors and publisher have tried to ensure the accuracy of the authors’ article and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this article.

Copyright

Copyright © 2024 Wade et al. This work was produced by Wade et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaption in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

List of abbreviations

ARI
acute respiratory infection
ATS
American Thoracic Society
CAP
community-acquired pneumonia
CF
cystic fibrosis
COPD
chronic obstructive pulmonary disease
CREWS
Chronic Respiratory Early Warning Score
EQ-5D
EuroQol-5 Dimensions
EQ-5D-3L
EuroQol-5 Dimensions, three-level version
EWS
early warning scores
GABHS
Group A β-haemolytic streptococcal
GP
general practitioner
HRQoL
health-related quality of life
ICER
incremental cost-effectiveness ratio
ICU
intensive care unit
IDSA
Infectious Diseases Society of America
MEDS
Mortality in Emergency Department Sepsis score
MEWS
Modified Early Warning Score
NEWS
National Early Warning Score
NHAP
nursing home-acquired pneumonia
NHS EED
NHS Economic Evaluations Database
NICE
National Institute for Health and Care Excellence
PPI
patient and public involvement
PSI
Pneumonia Severity Index
PSS
Personal Social Services
QALY
quality-adjusted life-year
RADT
rapid antigen detection tests
RCT
randomised controlled trial
REA-ICU
Risk of Early Admission to the Intensive Care Unit
REMS
Rapid Emergency Medicine Score
ROBIS
Risk of Bias in Systematic Reviews
SARS
severe acute respiratory syndrome
SCAP
severe community-acquired pneumonia
SEWS
Standardised Early Warning Score
SIRS
systemic inflammatory response syndrome
S-NEWS
Salford National Early Warning Score

References

  1. National Institute for Health and Care Excellence . Developing NICE Guidelines: The Manual 2022. www.nice.org.uk/process/pmg20/chapter/introduction (accessed 13 July 2023).
  2. McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol 2016;75:40-6. https://doi.org/10.1016/j.jclinepi.2016.01.021.
  3. Whiting P, Savović J, Higgins JP, Caldwell DM, Reeves BC, Shea B, et al. ROBIS group . ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 2016;69:225-34. https://doi.org/10.1016/j.jclinepi.2015.06.005.
  4. Aalbers J, O’Brien KK, Chan WS, Falk GA, Teljeur C, Dimitrov BD, et al. Predicting streptococcal pharyngitis in adults in primary care: a systematic review of the diagnostic accuracy of symptoms and signs and validation of the Centor score. BMC Med 2011;9. https://doi.org/10.1186/1741-7015-9-67.
  5. Akram AR, Chalmers JD, Hill AT. Predicting mortality with severity assessment tools in out-patients with community-acquired pneumonia. QJM 2011;104:871-9. https://doi.org/10.1093/qjmed/hcr088.
  6. Chalmers JD, Akram AR, Hill AT. Increasing outpatient treatment of mild community-acquired pneumonia: systematic review and meta-analysis. Eur Respir J 2011;37:858-64. https://doi.org/10.1183/09031936.00065610.
  7. Ebell MH, Walsh ME, Fahey T, Kearney M, Marchello C. Meta-analysis of calibration, discrimination, and stratum-specific likelihood ratios for the CRB-65 score. J Gen Intern Med 2019;34:1304-13. https://doi.org/10.1007/s11606-019-04869-z.
  8. McNally M, Curtain J, O’Brien KK, Dimitrov BD, Fahey T. Validity of British Thoracic Society guidance (the CRB-65 rule) for predicting the severity of pneumonia in general practice: systematic review and meta-analysis. Br J Gen Pract 2010;60:e423-33. https://doi.org/10.3399/bjgp10X532422.
  9. Metlay JP, Waterer GW, Long AC, Anzueto A, Brozek J, Crothers K, et al. Diagnosis and treatment of adults with community-acquired pneumonia. An official clinical practice guideline of the American Thoracic Society and Infectious Diseases Society of America. Am J Respir Crit Care Med 2019;200:e45-67. https://doi.org/10.1164/rccm.201908-1581ST.
  10. Nannan Panday RS, Minderhoud TC, Alam N, Nanayakkara PWB. Prognostic value of early warning scores in the emergency department (ED) and acute medical unit (AMU): a narrative review. Eur J Intern Med 2017;45:20-31. https://doi.org/10.1016/j.ejim.2017.09.027.
  11. Smith MD, Fee C, Mace SE, Maughan B, Perkins JC, Kaji A, et al. American College of Emergency Physicians Clinical Policies Subcommittee (Writing Committee) on Community-Acquired Pneumonia . Clinical policy: critical issues in the management of adult patients presenting to the emergency department with community-acquired pneumonia. Ann Emerg Med 2021;77:e1-57. https://doi.org/10.1016/j.annemergmed.2020.10.024.
  12. Dosa D. Should I hospitalize my resident with nursing home-acquired pneumonia?. J Am Med Dir Assoc 2005;6:327-33.
  13. Little P, Hobbs FD, Moore M, Mant D, Williamson I, McNulty C, et al. PRImary care Streptococcal Management (PRISM) study: in vitro study, diagnostic cohorts and a pragmatic adaptive randomised controlled trial with nested qualitative study and cost-effectiveness study. Health Technol Assess 2014;18:1-101. https://doi.org/10.3310/hta18060.

Appendix 1 Search strategies

Search strategies for identification of systematic reviews

MEDLINE ALL

via Ovid http://ovidsp.ovid.com/

Date range: 1946 to May 11, 2023

Date searched: 15 May 2023

Records retrieved: 2659

The following search strategy contains a section to limit retrieval to systematic reviews (lines 50–59). The terms used are based on those from a previous NICE guideline on pneumonia.{National Clinical Guidelines Centre, 2014 #5639}

  1. exp Respiratory Tract Infections/ (605,237)

  2. ((airway$ or bronchopulmonar$ or broncho-pulmonar$ or tracheobronch$ or tracheo-bronch$ or pulmonar$ tract or pulmonary or respirat$ tract or respiratory or chest or lung? or lobar or pleura?) adj3 (infect$ or coinfect$ or inflam$ or swollen or swelling$ or abscess$)).ti,ab. (153,445)

  3. (bronchit$ or bronchiolit$ or allergic bronchopulmon$ or bronchopneumon$ or common cold$ or coryza or croup or empyem$ or epipharyngit$ or epiglottit$ or epiglotit$ or flu or influenza or laryngit$ or laryngotracheobronchit$ or laryngo tracheo bronchit$ or laryngo tracheobronchit$ or laryngotracheit$ or nasopharyngit$ or otitis media or parainfluenza or pharyngit$ or pleurisy or pneumoni$ or pleuropneumoni$ or rhinit$ or rhinopharyngit$ or rhinosinusit$ or severe acute respiratory syndrome or SARS or sinusit$ or sore throat$ or throat infection$ or supraglottit$ or supraglotit$ or tonsillit$ or tonsilit$ or tracheit$ or whooping cough or pertussis or pertusis).mp. (821,333)

  4. (ARTI or RTI or LRTI or URTI or ALRI or AURI or SARI).ti,ab. (7276)

  5. Infectious Mononucleosis/ (7318)

  6. (glandular fever or Infectious Mononucleosis or Epstein-Barr).ti,ab. (40,792)

  7. ((strep$ adj3 (throat$ or pharyn$ or tonsil$)) or (strep$ and (airway$ or pulmonary or brochopulmonar$ or brocho-pulmonar$ or respiratory$))).mp. (22,155)

  8. ((acute$ or exacerbate$ or flare$) adj3 (copd or coad or chronic obstructive pulmonary disease or chronic obstructive airway$ disease or chronic obstructive lung disease)).mp. (10,290)

  9. ((acute$ or subacute$ or exacerbat$ or prolonged) adj3 cough$).mp. (1546)

  10. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 (1,131,600)

  11. early warning score/ (380)

  12. ‘Severity of Illness Index’/ (270,315)

  13. (early warning$ or red flag$ or (flag$ adj2 early)).ti,ab. (12,990)

  14. (severity adj3 (score$ or scoring or scale$ or tool$ or instrument$ or index$ or indice$ or calculat$ or algorithm$ or metric$ or measur$ or criteri$ or code$)).ti,ab. (79,034)

  15. (severity adj3 (assess$ or estimat$ or evaluat$ or classif$ or rate? or rating? or value? or quantif$ or grade$ or chart$ or equation$ or table$ or model$ or framework$ or predict$)).ti,ab. (70,990)

  16. 11 or 12 or 13 or 14 or 15 (386,863)

  17. (curb65 or crb65 or curb-65 or crb-65 or news2 or enews or pnews).ti,ab. (1132)

  18. ((curb or news) adj3 (criteri$ or rule$ or scor$ or predict$ or tool$)).ti,ab. (1172)

  19. CENTOR.ti,ab. (135)

  20. (PMEWS or eMEWS).ti,ab. (20)

  21. (McIsaac adj (score$ or scoring or criteri$)).ti,ab. (37)

  22. (sino-nasal outcome test$ or SNOT-22 or SNOT22).ti,ab. (1372)

  23. (pneumonia severity index or PSI or (PORT adj (Score$ or scoring))).ti,ab. (20,696)

  24. 17 or 18 or 19 or 20 or 21 or 22 or 23 (23,631)

  25. 16 or 24 (408,300)

  26. 10 and 25 (30,022)

  27. Triage/ (14,830)

  28. (triage$ or triaging).ti,ab. (27,182)

  29. ((stratif$ or priorit$ or classif$) adj3 (patient$ or outpatient$)).ti,ab. (110,619)

  30. ((stratif$ or priorit$ or classif$) adj3 (symptom$ or sign? or illness$ or disease$ or disorder$ or severity or risk$)).ti,ab. (122,512)

  31. 27 or 28 or 29 or 30 (243,129)

  32. 10 and 31 (14,211)

  33. Symptom Assessment/ (7065)

  34. Patient Acuity/ (2591)

  35. ((initial or first or primary or point of care) adj3 (assess$ or evaluat$ or examin$ or screen$) adj3 (patient$ or outpatient$ or sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (13,243)

  36. ((sign? or symptom$) adj2 (score$ or scoring)).ti,ab. (31,415)

  37. ((assess$ or evaluat$ or determin$ or detect$ or analys$ or screen$) adj5 (severe$ or severity or serious$) adj5 (sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (28,501)

  38. ((patient$ or sign? or symptom$ or illness$ or disease$ or disorder$ or infection$) adj3 acuity).ti,ab. (7682)

  39. 33 or 34 or 35 or 36 or 37 or 38 (88,339)

  40. 10 and 39 (10,530)

  41. Clinical Decision Rules/ (911)

  42. (clinical$ adj5 (decision$ or predicti$) adj5 (aid? or algorithm? or characteristic? or criteri$ or evaluation? or index or indices or marker? or method$ or model$ or panel? or parameter? or rule or rules or score? or scoring or screen$ or signs or symptoms or system? or technique? or test$ or tool? or value? or variable$)).mp. (44,013)

  43. (clinical$ adj (predicti$ or predictor$)).ti,ab. (11,212)

  44. (rule in or ruled in or rule out or ruled out).ti,ab. (60,226)

  45. (predict$ adj5 (severe$ or severity or serious$) adj5 (sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (9210)

  46. ((predict$ or prognos$ or cluster$) adj3 (sign? or symptom$)).ti,ab. (28,230)

  47. 41 or 42 or 43 or 44 or 45 or 46 (145,502)

  48. 10 and 47 (8781)

  49. 26 or 32 or 40 or 48 (55,802)

  50. ‘systematic review’.pt. (228,202)

  51. meta analysis.pt. (180,733)

  52. (meta analy$ or metanaly$ or metaanaly$).ti,ab. (268,778)

  53. ((systematic$ or evidence$) adj3 (review$ or overview$)).ti,ab. (359,433)

  54. (reference list$ or bibliograph$ or hand search$ or manual search$ or relevant journals).ab. (54,013)

  55. (search strategy or search criteria or systematic search or study selection or data extraction).ab. (80,940)

  56. (search$ adj4 literature).ab. (96,383)

  57. (medline or pubmed or cochrane or embase or psychlit or psyclit or psychinfo or psycinfo or cinahl or science citation index or bids or cancerlit).ab. (356,783)

  58. cochrane.jw. (16,330)

  59. ((diagnos$ or prognos$) adj2 review$).ti,ab. (11,734)

  60. 50 or 51 or 52 or 53 or 54 or 55 or 56 or 57 or 59 (686,228)

  61. 49 and 60 (2766)

  62. exp animals/ not humans.sh. (5,120,552)

  63. 61 not 62 (2761)

  64. limit 63 to english language (2704)

  65. (comment or editorial or letter or news).pt. (2,359,631)

  66. 64 not 65 (2659)

Key:

/ = subject heading (MeSH heading)

sh = subject heading (MeSH heading)

exp = exploded subject heading (MeSH heading)

$ = truncation

? = optional wildcard – one or no characters

ti,ab = terms in title or abstract fields

mp = multi-purpose field search – terms in title, original title, abstract, name of substance word, or subject heading word

pt = publication type

jw = journal word

adj3 = terms within three words of each other (any order)

adj = terms next to each other in order specified

EMBASE

via Ovid http://ovidsp.ovid.com/

Date range: 1974 to 2023 May 12

Date searched: 15 May 2023

Records retrieved: 2632

The following search strategy contains a section to limit retrieval to systematic reviews (lines 50–59). The terms used are based on those from a previous NICE guideline on pneumonia.{National Clinical Guidelines Centre, 2014 #5639}

  1. exp respiratory tract infection/ (486,791)

  2. ((airway$ or bronchopulmonar$ or broncho-pulmonar$ or tracheobronch$ or tracheo-bronch$ or pulmonar$ tract or pulmonary or respirat$ tract or respiratory or chest or lung? or lobar or pleura?) adj3 (infect$ or coinfect$ or inflam$ or swollen or swelling$ or abscess$)).ti,ab. (227,122)

  3. (bronchit$ or bronchiolit$ or allergic bronchopulmon$ or bronchopneumon$ or common cold$ or coryza or croup or empyem$ or epipharyngit$ or epiglottit$ or epiglotit$ or flu or influenza or laryngit$ or laryngotracheobronchit$ or laryngo tracheo bronchit$ or laryngo tracheobronchit$ or laryngotracheit$ or nasopharyngit$ or otitis media or parainfluenza or pharyngit$ or pleurisy or pneumoni$ or pleuropneumoni$ or rhinit$ or rhinopharyngit$ or rhinosinusit$ or severe acute respiratory syndrome or SARS or sinusit$ or sore throat$ or throat infection$ or supraglottit$ or supraglotit$ or tonsillit$ or tonsilit$ or tracheit$ or whooping cough or pertussis or pertusis).mp. (1,187,643)

  4. (ARTI or RTI or LRTI or URTI or ALRI or AURI or SARI).ti,ab. (11,236)

  5. mononucleosis/ (2883)

  6. (glandular fever or infectious mononucleosis or Epstein-Barr).ti,ab. (47,931)

  7. streptococcal pharyngitis/ (1777)

  8. ((strep$ adj3 (throat$ or pharyn$ or tonsil$)) or (strep$ and (airway$ or pulmonary or brochopulmonar$ or brocho-pulmonar$ or respiratory$))).mp. (42,535)

  9. ((acute$ or exacerbat$ or flare$) adj3 (copd or coad or chronic obstructive pulmonary disease or chronic obstructive airway$ disease or chronic obstructive lung disease)).mp. (19,296)

  10. ((acute$ or subacute$ or exacerbat$ or prolonged) adj3 cough$).mp. (2474)

  11. or/1-10 (1,509,554)

  12. exp early warning score/ (1794)

  13. disease severity assessment/ (9886)

  14. ‘severity of illness index’/ (20,395)

  15. (early warning$ or red flag$ or (flag$ adj2 early)).ti,ab. (17,967)

  16. (severity adj3 (score$ or scoring or scale$ or tool$ or instrument$ or index$ or indice$ or calculat$ or algorithm$ or metric$ or measur$ or criteri$ or code$)).ti,ab. (129,233)

  17. (severity adj3 (assess$ or estimat$ or evaluat$ or classif$ or rate? or rating? or value? or quantif$ or grade$ or chart$ or equation$ or table$ or model$ or framework$ or predict$)).ti,ab. (115,235)

  18. 12 or 13 or 14 or 15 or 16 or 17 (261,868)

  19. (curb65 or crb65 or curb-65 or crb-65 or news2 or enews or pnews).ti,ab. (2054)

  20. ((curb or news) adj3 (criteri$ or rule$ or scor$ or predict$ or tool$)).ti,ab. (1970)

  21. CENTOR.ti,ab. (185)

  22. (PMEWS or eMEWS).ti,ab. (26)

  23. (McIsaac adj (score$ or scoring or criteri$)).ti,ab. (49)

  24. (sino-nasal outcome test$ or SNOT-22 or SNOT22).ti,ab. (2010)

  25. (pneumonia severity index or PSI or (PORT adj (score$ or scoring))).ti,ab. (21,566)

  26. 19 or 20 or 21 or 22 or 23 or 24 or 25 (26,187)

  27. 18 or 26 (284,907)

  28. 11 and 27 (24,815)

  29. patient triage/ (3244)

  30. (triage$ or triaging).ti,ab. (43,825)

  31. ((stratif$ or priorit$ or classif$) adj3 (patient$ or outpatient$)).ti,ab. (201,540)

  32. ((stratif$ or priorit$ or classif$) adj3 (symptom$ or sign? or illness$ or disease$ or disorder$ or severity or risk$)).ti,ab. (202,687)

  33. 29 or 30 or 31 or 32 (406,394)

  34. 11 and 33 (22,210)

  35. symptom assessment/ (11,857)

  36. patient acuity/ (1293)

  37. ((initial or first or primary or point of care) adj3 (assess$ or evaluat$ or examin$ or screen$) adj3 (patient$ or outpatient$ or sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (22,489)

  38. ((sign? or symptom$) adj2 (score$ or scoring)).ti,ab. (51,668)

  39. ((assess$ or evaluat$ or determin$ or detect$ or analys$ or screen$) adj5 (severe$ or severity or serious$) adj5 (sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (46,809)

  40. ((patient$ or sign? or symptom$ or illness$ or disease$ or disorder$ or infection$) adj3 acuity).ti,ab. (11,416)

  41. 35 or 36 or 37 or 38 or 39 or 40 (140,927)

  42. 11 and 41 (15,434)

  43. clinical decision rule/ (684)

  44. (clinical$ adj5 (decision$ or predicti$) adj5 (aid? Or algorithm? Or characteristic? Or criteri$ or evaluation? Or index or indices or marker? Or method$ or model$ or panel? Or parameter? Or rule or rules or score? Or scoring or screen$ or signs or symptoms or system? Or technique? Or test$ or tool? Or value? Or variable$)).mp. (62,551)

  45. (clinical$ adj (predicti$ or predictor$)).ti,ab. (18,367)

  46. (rule in or ruled in or rule out or ruled out).ti,ab. (93,769)

  47. (predict$ adj5 (severe$ or severity or serious$) adj5 (sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (14,169)

  48. ((predict$ or prognos$ or cluster$) adj3 (sign? or symptom$)).ti,ab. (39,509)

  49. 43 or 44 or 45 or 46 or 47 or 48 (217,048)

  50. 11 and 49 (15,032)

  51. 28 or 34 or 42 or 50 (68,399)

  52. ‘systematic review’/ (434,122)

  53. exp meta analysis/ (293,135)

  54. (meta analy$ or metanaly$ or metaanaly$).ti,ab. (356,347)

  55. ((systematic or evidence) adj2 (review$ or overview$)).ti,ab. (412,624)

  56. (reference list$ or bibliograph$ or hand search$ or manual search$ or relevant journals).ab. (67,522)

  57. (search strategy or search criteria or systematic search or study selection or data extraction).ab. (100,509)

  58. (search$ adj4 literature).ab. (125,065)

  59. (medline or pubmed or cochrane or embase or psychlit or psyclit or psychinfo or psycinfo or cinahl or science citation index or bids or cancerlit).ab. (451,666)

  60. ((pool$ or combined) adj2 (data or trials or studies or results)).ab. (92,673)

  61. cochrane.jw. (24,683)

  62. ((diagnos$ or prognos$) adj2 review$).ti,ab. (17,027)

  63. 52 or 53 or 54 or 55 or 56 or 57 or 58 or 59 or 60 or 61 or 62 (980,485)

  64. 51 and 63 (3452)

  65. (animal/ or animal experiment/ or animal model/ or animal tissue/ or nonhuman/) not exp human/ (6,800,393)

  66. 64 not 65 (3426)

  67. (editorial or letter or note).pt. (3,015,508)

  68. 66 not 67 (3396)

  69. (conference abstract$ or conference review or conference paper or conference proceeding).db,pt,su. (5,535,870)

  70. 68 not 69 (2716)

  71. preprint.pt. (65,307)

  72. 70 not 71 (2694)

  73. limit 72 to english language (2632)

Key:

/ = subject heading (Emtree heading)

exp = exploded subject heading (Emtree heading)

$ = truncation

? = optional wildcard – one or no characters

ti,ab = terms in title or abstract fields

mp = multi-purpose field search – terms in title, original title, abstract, name of substance word, or subject heading word

pt = publication type

jw = journal word

db = database

su = source type

adj3 = terms within three words of each other (any order)

adj = terms next to each other in order specified

Cochrane Database of Systematic Reviews

via Wiley http://onlinelibrary.wiley.com/

Issue: Issue 5 of 12, May 2023

Date searched: 15 May 2022

Records retrieved: 203

  1. MeSH descriptor: [Respiratory Tract Infections] explode all trees (23,846)

  2. ((airway* or bronchopulmonar* or broncho-pulmonar* or tracheobronch* or tracheo-bronch* or pulmonar* tract or pulmonary or (respirat*next tract) or respiratory or chest or lung? or lobar or pleura?) near/3 (infect* or coinfect* or inflam* or swollen or swelling* or abscess*)):ti,ab,kw (30,789)

  3. (bronchit* or bronchiolit* or (allergic next bronchopulmon*) or bronchopneumon* or (common next cold*) or coryza or croup or empyem* or epipharyngit* or epiglottit* or epiglotit* or flu or influenza or laryngit* or laryngotracheobronchit* or (laryngo next trachea next bronchit*) or (laryngo next tracheobronchit*) or laryngotracheit* or nasopharyngit* or ‘otitis media’ or parainfluenza or pharyngit* or pleurisy or pneumoni* or pleuropneumoni* or rhinit* or rhinopharyngit* or rhinosinusit* or ‘severe acute respiratory syndrome’ or SARS or sinusit* or (sore next throat*) or (throat next infection*) or supraglottit* or supraglotit* or tonsillit* or tonsilit* or tracheit* or ‘whooping cough’ or pertussis or pertusis):ti,ab,kw (69,533)

  4. MeSH descriptor: [Otitis Media] explode all trees (1392)

  5. (ARTI or RTI or LRTI or URTI or ALRI or AURI or SARI):ti,ab,kw (1608)

  6. MeSH descriptor: [Infectious Mononucleosis] this term only (62)

  7. (‘glandular fever’ or ‘Infectious Mononucleosis’ or Epstein-Barr):ti,ab,kw 599

  8. ((strep* near/3 (throat* or pharyn* or tonsil*)) or (strep* and (airway* or pulmonary or brochopulmonar* or brocho-pulmonar* or respiratory*))):ti,ab,kw (1729)

  9. ((acute* or exacerbat* or flare*) near/3 (copd or coad or ‘chronic obstructive pulmonary disease’ or (‘chronic obstructive’ next airway* next disease) or ‘chronic obstructive lung disease’)):ti,ab,kw (4040)

  10. ((acute* or subacute* or exacerbate* or prolonged) near/3 cough*):ti,ab,kw (525)

  11. #1 or #2 or #3 or #4 or #5 or #6 or #7 or #8 or #9 or #10 (97,500)

  12. MeSH descriptor: [Early Warning Score] this term only (11)

  13. MeSH descriptor: [Severity of Illness Index] this term only (22,685)

  14. ((early next warning*) or (red next flag*) or (flag* near/2 early)):ti,ab,kw (675)

  15. (severity near/3 (score* or scoring or scale* or tool* or instrument* or index* or indice* or calculat* or algorithm* or metric* or measur* or criteri* or code*)):ti,ab,kw (47,560)

  16. (severity near/3 (assess* or estimat* or evaluat* or classif* or rate? or rating? or value? or quantif* or grade* or chart* or equation* or table* or model* or framework* or predict*)):ti,ab,kw (15,000)

  17. #12 or #13 or #14 or #15 or #16 (57,740)

  18. (curb65 or crb65 or curb-65 or crb-65 or news2 or enews or pnews):ti,ab,kw (163)

  19. ((curb or news) near/3 (criteri* or rule* or scor* or predict* or tool*)):ti,ab,kw (196)

  20. CENTOR:ti,ab,kw (33)

  21. (PMEWS or eMEWS):ti,ab,kw (2)

  22. (McIsaac next (score* or scoring or criteri*)):ti,ab,kw (5)

  23. ((‘sino-nasal outcome’ next test*) or SNOT-22 or SNOT22):ti,ab,kw (630)

  24. (‘pneumonia severity index’ or PSI or (PORT next (score* or scoring))):ti,ab,kw (1055)

  25. #18 or #19 or #20 or #21 or #22 or #23 or #24 (1995)

  26. #17 or #25 (59,302)

  27. #11 and #26 in Cochrane Reviews, Cochrane Protocols (50)

  28. MeSH descriptor: [Triage] this term only (400)

  29. (triage* or triaging):ti,ab,kw (2255)

  30. ((stratif* or priorit* or classif*) near/3 (patient* or outpatient*)):ti,ab,kw (21,550)

  31. ((stratif* or priorit* or classif*) near/3 (symptom* or sign? or illness* or disease* or disorder* or severity or risk*)):ti,ab,kw (16,858)

  32. #28 or #29 or #30 or #31 (38,181)

  33. #11 and #32 in Cochrane Reviews, Cochrane Protocols (22)

  34. MeSH descriptor: [Symptom Assessment] this term only (502)

  35. MeSH descriptor: [Patient Acuity] this term only (182)

  36. ((initial or first or primary or point of care) near/3 (assess* or evaluat* or examin* or screen*) near/3 (patient* or outpatient* or sign? or symptom* or illness* or disease* or disorder* or infection*)):ti,ab,kw (57,714)

  37. ((sign? or symptom*) near/2 (score* or scoring)):ti,ab,kw (18,921)

  38. ((assess* or evaluat* or determin* or detect* or analys* or screen*) near/5 (severe* or severity or serious*) near/5 (sign? or symptom* or illness* or disease* or disorder* or infection*)):ti,ab,kw (7534)

  39. ((patient* or sign? or symptom* or illness* or disease* or disorder* or infection*) near/3 acuity):ti,ab,kw (1326)

  40. #34 or #35 or #36 or #37 or #38 or #39 (81,543)

  41. #11 and #40 in Cochrane Reviews, Cochrane Protocols (130)

  42. MeSH descriptor: [Clinical Decision Rules] this term only (43)

  43. (clinical* near/5 (decision* or predicti*) near/5 (aid? or algorithm? or characteristic? or criteri* or evaluation? or index or indices or marker? or method* or model* or panel? or parameter? or rule or rules or score? or scoring or screen* or signs or symptoms or system? or technique? or test* or tool? or value? or variable*)):ti,ab,kw (5920)

  44. (clinical* next (predicti* or predictor*)):ti,ab,kw (984)

  45. (rule in or ruled in or rule out or ruled out):ti,ab,kw (5641)

  46. (predict* near/5 (severe* or severity or serious*) near/5 (sign? or symptom* or illness* or disease* or disorder* or infection*)):ti,ab,kw (599)

  47. ((predict* or prognos* or cluster*) near/3 (sign? Or symptom*)):ti,ab,kw (2592)

  48. #42 or #43 or #44 or #45 or #46 or #47 (14,792)

  49. #11 and #48 in Cochrane Reviews, Cochrane Protocols (43)

  50. #27 or #33 or #41 or #49 in Cochrane Reviews, Cochrane Protocols (203)

Key:

MeSH descriptor = subject heading (MeSH heading)

* = truncation

? = wildcard – zero or one characters

ti,ab,kw = terms in title, abstract or keyword fields

near/3 = terms within three words of each other (any order)

next = terms are next to each other

Search strategies for identification of economic evaluations

MEDLINE ALL

via Ovid http://ovidsp.ovid.com/

Date range searched: 1946 to May 11, 2023

Date searched: 15 May 2023

Records retrieved: 1778

  1. exp Respiratory Tract Infections/ (605,237)

  2. ((airway$ or bronchopulmonar$ or broncho-pulmonar$ or tracheobronch$ or tracheo-bronch$ or pulmonar$ tract or pulmonary or respirat$ tract or respiratory or chest or lung? or lobar or pleura?) adj3 (infect$ or coinfect$ or inflam$ or swollen or swelling$ or abscess$)).ti,ab. (153,445)

  3. (bronchit$ or bronchiolit$ or allergic bronchopulmon$ or bronchopneumon$ or common cold$ or coryza or croup or empyem$ or epipharyngit$ or epiglottit$ or epiglotit$ or flu or influenza or laryngit$ or laryngotracheobronchit$ or laryngo tracheo bronchit$ or laryngo tracheobronchit$ or laryngotracheit$ or nasopharyngit$ or otitis media or parainfluenza or pharyngit$ or pleurisy or pneumoni$ or pleuropneumoni$ or rhinit$ or rhinopharyngit$ or rhinosinusit$ or severe acute respiratory syndrome or SARS or sinusit$ or sore throat$ or throat infection$ or supraglottit$ or supraglotit$ or tonsillit$ or tonsilit$ or tracheit$ or whooping cough or pertussis or pertusis).mp. (821,333)

  4. (ARTI or RTI or LRTI or URTI or ALRI or AURI or SARI).ti,ab. (7276)

  5. Infectious Mononucleosis/ (7318)

  6. (glandular fever or Infectious Mononucleosis or Epstein-Barr).ti,ab. (40,792)

  7. ((strep$ adj3 (throat$ or pharyn$ or tonsil$)) or (strep$ and (airway$ or pulmonary or brochopulmonar$ or brocho-pulmonar$ or respiratory$))).mp. (22,155)

  8. ((acute$ or exacerbat$ or flare$) adj3 (copd or coad or chronic obstructive pulmonary disease or chronic obstructive airway$ disease or chronic obstructive lung disease)).mp. (10,290)

  9. ((acute$ or subacute$ or exacerbat$ or prolonged) adj3 cough$).mp. (1546)

  10. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 (1,131,600)

  11. early warning score/ (380)

  12. ‘Severity of Illness Index’/ (270,315)

  13. (early warning$ or red flag$ or (flag$ adj2 early)).ti,ab. (12,990)

  14. (severity adj3 (score$ or scoring or scale$ or tool$ or instrument$ or index$ or indice$ or calculat$ or algorithm$ or metric$ or measur$ or criteri$ or code$)).ti,ab. (79,034)

  15. (severity adj3 (assess$ or estimat$ or evaluat$ or classif$ or rate? or rating? or value? or quantif$ or grade$ or chart$ or equation$ or table$ or model$ or framework$ or predict$)).ti,ab. (70,990)

  16. 11 or 12 or 13 or 14 or 15 (386,863)

  17. (curb65 or crb65 or curb-65 or crb-65 or news2 or enews or pnews).ti,ab. (1132)

  18. ((curb or news) adj3 (criteri$ or rule$ or scor$ or predict$ or tool$)).ti,ab. (1172)

  19. CENTOR.ti,ab. (135)

  20. (PMEWS or eMEWS).ti,ab. (20)

  21. (McIsaac adj (score$ or scoring or criteri$)).ti,ab. (37)

  22. (sino-nasal outcome test$ or SNOT-22 or SNOT22).ti,ab. (1372)

  23. (pneumonia severity index or PSI or (PORT adj (Score$ or scoring))).ti,ab. (20,696)

  24. 17 or 18 or 19 or 20 or 21 or 22 or 23 (23,631)

  25. 16 or 24 (408,300)

  26. 10 and 25 (30,022)

  27. Triage/ (14,830)

  28. (triage$ or triaging).ti,ab. (27,182)

  29. ((stratif$ or priorit$ or classif$) adj3 (patient$ or outpatient$)).ti,ab. (110,619)

  30. ((stratif$ or priorit$ or classif$) adj3 (symptom$ or sign? or illness$ or disease$ or disorder$ or severity or risk$)).ti,ab. (122,512)

  31. 27 or 28 or 29 or 30 (243,129)

  32. 10 and 31 (14,211)

  33. Symptom Assessment/ (7065)

  34. Patient Acuity/ (2591)

  35. ((initial or first or primary or point of care) adj3 (assess$ or evaluat$ or examin$ or screen$) adj3 (patient$ or outpatient$ or sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (13,243)

  36. ((sign? or symptom$) adj2 (score$ or scoring)).ti,ab. (31,415)

  37. ((assess$ or evaluat$ or determin$ or detect$ or analys$ or screen$) adj5 (severe$ or severity or serious$) adj5 (sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (28,501)

  38. ((patient$ or sign? or symptom$ or illness$ or disease$ or disorder$ or infection$) adj3 acuity).ti,ab. (7682)

  39. 33 or 34 or 35 or 36 or 37 or 38 (88,339)

  40. 10 and 39 (10,530)

  41. Clinical Decision Rules/ (911)

  42. (clinical$ adj5 (decision$ or predicti$) adj5 (aid? or algorithm? or characteristic? or criteri$ or evaluation? or index or indices or marker? or method$ or model$ or panel? or parameter? or rule or rules or score? or scoring or screen$ or signs or symptoms or system? or technique? or test$ or tool? or value? or variable$)).mp. (44,013)

  43. (clinical$ adj (predicti$ or predictor$)).ti,ab. (11,212)

  44. (rule in or ruled in or rule out or ruled out).ti,ab. (60,226)

  45. (predict$ adj5 (severe$ or severity or serious$) adj5 (sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (9210)

  46. ((predict$ or prognos$ or cluster$) adj3 (sign? or symptom$)).ti,ab. (28,230)

  47. 41 or 42 or 43 or 44 or 45 or 46 (145,502)

  48. 10 and 47 (8781)

  49. 26 or 32 or 40 or 48 (55,802)

  50. Economics/ (27,500)

  51. exp ‘costs and cost analysis’/ (264,277)

  52. Economics, Dental/ (1921)

  53. exp economics, hospital/ (25,710)

  54. Economics, Medical/ (9245)

  55. Economics, Nursing/ (4013)

  56. Economics, Pharmaceutical/ (3103)

  57. (economic$ or cost or costs or costly or costing or price or prices or pricing or pharmacoeconomic$).ti,ab. (1,030,924)

  58. (expenditure$ not energy).ti,ab. (36,561)

  59. value for money.ti,ab. (2105)

  60. budget$.ti,ab. (35,216)

  61. or/50-60 (1,195,231)

  62. ((energy or oxygen) adj cost).ti,ab. (4741)

  63. (metabolic adj cost).ti,ab. (1698)

  64. ((energy or oxygen) adj expenditure).ti,ab. (28,877)

  65. or/62-64 (34,259)

  66. 61 not 65 (1,187,317)

  67. 49 and 66 (2910)

  68. exp animals/not humans/ (5,120,552)

  69. 67 not 68 (2866)

  70. limit 69 to english language (2727)

  71. (comment or editorial or letter or news).pt. (2,359,631)

  72. 70 not 71 (2699)

  73. limit 72 to yr=‘2014 -Current’ (1783)

  74. remove duplicates from 73 (1778)

Key:

/ = subject heading (MeSH heading)

exp = exploded subject heading (MeSH heading)

$ = truncation

? = optional wildcard – one or no characters

ti,ab = terms in title or abstract fields

mp = multi-purpose field search – terms in title, original title, abstract, name of substance word, or subject heading word

adj3 = terms within three words of each other (any order)

adj = terms next to each other in order specified

EMBASE

via Ovid http://ovidsp.ovid.com/

Date range searched: 1974 to 2023 May 12

Date searched: 15 May 2023

Records retrieved: 1705

  1. exp respiratory tract infection/ (486,791)

  2. ((airway$ or bronchopulmonar$ or broncho-pulmonar$ or tracheobronch$ or tracheo-bronch$ or pulmonar$ tract or pulmonary or respirat$ tract or respiratory or chest or lung? or lobar or pleura?) adj3 (infect$ or coinfect$ or inflam$ or swollen or swelling$ or abscess$)).ti,ab. (227,122)

  3. (bronchit$ or bronchiolit$ or allergic bronchopulmon$ or bronchopneumon$ or common cold$ or coryza or croup or empyem$ or epipharyngit$ or epiglottit$ or epiglotit$ or flu or influenza or laryngit$ or laryngotracheobronchit$ or laryngo tracheo bronchit$ or laryngo tracheobronchit$ or laryngotracheit$ or nasopharyngit$ or otitis media or parainfluenza or pharyngit$ or pleurisy or pneumoni$ or pleuropneumoni$ or rhinit$ or rhinopharyngit$ or rhinosinusit$ or severe acute respiratory syndrome or SARS or sinusit$ or sore throat$ or throat infection$ or supraglottit$ or supraglotit$ or tonsillit$ or tonsilit$ or tracheit$ or whooping cough or pertussis or pertusis).mp. (1,187,643)

  4. (ARTI or RTI or LRTI or URTI or ALRI or AURI or SARI).ti,ab. (11,236)

  5. mononucleosis/ (2883)

  6. (glandular fever or infectious mononucleosis or Epstein-Barr).ti,ab. (47,931)

  7. streptococcal pharyngitis/ (1777)

  8. ((strep$ adj3 (throat$ or pharyn$ or tonsil$)) or (strep$ and (airway$ or pulmonary or brochopulmonar$ or brocho-pulmonar$ or respiratory$))).mp. (42,535)

  9. ((acute$ or exacerbat$ or flare$) adj3 (copd or coad or chronic obstructive pulmonary disease or chronic obstructive airway$ disease or chronic obstructive lung disease)).mp. (19,296)

  10. ((acute$ or subacute$ or exacerbat$ or prolonged) adj3 cough$).mp. (2474)

  11. or/1-10 (1,509,554)

  12. exp early warning score/ (1794)

  13. disease severity assessment/ (9886)

  14. ‘severity of illness index’/ (20,395)

  15. (early warning$ or red flag$ or (flag$ adj2 early)).ti,ab. (17,967)

  16. (severity adj3 (score$ or scoring or scale$ or tool$ or instrument$ or index$ or indice$ or calculat$ or algorithm$ or metric$ or measur$ or criteri$ or code$)).ti,ab. (129,233)

  17. (severity adj3 (assess$ or estimat$ or evaluat$ or classif$ or rate? or rating? or value? or quantif$ or grade$ or chart$ or equation$ or table$ or model$ or framework$ or predict$)).ti,ab. (115,235)

  18. 12 or 13 or 14 or 15 or 16 or 17 (261,868)

  19. (curb65 or crb65 or curb-65 or crb-65 or news2 or enews or pnews).ti,ab. (2054)

  20. ((curb or news) adj3 (criteri$ or rule$ or scor$ or predict$ or tool$)).ti,ab. (1970)

  21. CENTOR.ti,ab. (185)

  22. (PMEWS or eMEWS).ti,ab. (26)

  23. (McIsaac adj (score$ or scoring or criteri$)).ti,ab. (49)

  24. (sino-nasal outcome test$ or SNOT-22 or SNOT22).ti,ab. (2010)

  25. (pneumonia severity index or PSI or (PORT adj (score$ or scoring))).ti,ab. (21,566)

  26. 19 or 20 or 21 or 22 or 23 or 24 or 25 (26,187)

  27. 18 or 26 (284,907)

  28. 11 and 27 (24,815)

  29. patient triage/ (3244)

  30. (triage$ or triaging).ti,ab. (43,825)

  31. ((stratif$ or priorit$ or classif$) adj3 (patient$ or outpatient$)).ti,ab. (201,540)

  32. ((stratif$ or priorit$ or classif$) adj3 (symptom$ or sign? or illness$ or disease$ or disorder$ or severity or risk$)).ti,ab. (202,687)

  33. 29 or 30 or 31 or 32 (406,394)

  34. 11 and 33 (22,210)

  35. symptom assessment/ (11,857)

  36. patient acuity/ (1293)

  37. ((initial or first or primary or point of care) adj3 (assess$ or evaluat$ or examin$ or screen$) adj3 (patient$ or outpatient$ or sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (22,489)

  38. ((sign? or symptom$) adj2 (score$ or scoring)).ti,ab. (51,668)

  39. ((assess$ or evaluat$ or determin$ or detect$ or analys$ or screen$) adj5 (severe$ or severity or serious$) adj5 (sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (46,809)

  40. ((patient$ or sign? or symptom$ or illness$ or disease$ or disorder$ or infection$) adj3 acuity).ti,ab. (11,416)

  41. 35 or 36 or 37 or 38 or 39 or 40 (140,927)

  42. 11 and 41 (15,434)

  43. clinical decision rule/ (684)

  44. (clinical$ adj5 (decision$ or predicti$) adj5 (aid? or algorithm? or characteristic? or criteri$ or evaluation? or index or indices or marker? or method$ or model$ or panel? or parameter? or rule or rules or score? or scoring or screen$ or signs or symptoms or system? or technique? or test$ or tool? or value? or variable$)).mp. (62,551)

  45. (clinical$ adj (predicti$ or predictor$)).ti,ab. (18,367)

  46. (rule in or ruled in or rule out or ruled out).ti,ab. (93,769)

  47. (predict$ adj5 (severe$ or severity or serious$) adj5 (sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (14,169)

  48. ((predict$ or prognos$ or cluster$) adj3 (sign? or symptom$)).ti,ab. (39,509)

  49. 43 or 44 or 45 or 46 or 47 or 48 (217,048)

  50. 11 and 49 (15,032)

  51. 28 or 34 or 42 or 50 (68,399)

  52. Health Economics/ (35,574)

  53. exp Economic Evaluation/ (352,561)

  54. exp Health Care Cost/ (336,376)

  55. pharmacoeconomics/ (9169)

  56. (econom$ or cost or costs or costly or costing or price or prices or pricing or pharmacoeconomic$).ti,ab. (1,380,284)

  57. (expenditure$ not energy).ti,ab. (50,208)

  58. (value adj2 money).ti,ab. (2978)

  59. budget$.ti,ab. (46,855)

  60. or/52-59 (1,669,816)

  61. (metabolic adj cost).ti,ab. (1858)

  62. ((energy or oxygen) adj cost).ti,ab. (5046)

  63. ((energy or oxygen) adj expenditure).ti,ab. (37,278)

  64. 60 not 63 (1,666,739)

  65. 51 and 64 (4185)

  66. (animal/or animal experiment/or animal model/or animal tissue/or nonhuman/) not exp human/ (6,800,393)

  67. 65 not 66 (4080)

  68. limit 67 to english language (3933)

  69. (editorial or letter or note).pt. (3,015,508)

  70. preprint.pt. (65,307)

  71. (conference abstract* or conference review or conference paper or conference proceeding).db,pt,su. (5,535,870)

  72. or/69-71 (8,616,644)

  73. 68 not 72 (2705)

  74. limit 73 to yr=‘2014 -Current’ (1795)

  75. remove duplicates from 74 (1705)

Key:

/ = subject heading (Emtree heading)

exp = exploded subject heading (Emtree heading)

$ = truncation

? = optional wildcard – one or no characters

ti,ab = terms in title or abstract fields

mp = multi-purpose field search – terms in title, original title, abstract, name of substance word, or subject heading word

pt = publication type

jw = journal word

db = database

su = source type

adj3 = terms within three words of each other (any order)

adj = terms next to each other in order specified

EconLit

via Ovid http://ovidsp.ovid.com/

Date range searched: 1886 to April 27, 2023

Date searched: 15 May 2023

Records retrieved: 24

  1. ((airway$ or bronchopulmonar$ or broncho-pulmonar$ or tracheobronch$ or tracheo-bronch$ or pulmonar$ tract or pulmonary or respirat$ tract or respiratory or chest or lung? or lobar or pleura?) adj3 (infect$ or coinfect$ or inflam$ or swollen or swelling$ or abscess$)).ti,ab. (107)

  2. (bronchit$ or bronchiolit$ or allergic bronchopulmon$ or bronchopneumon$ or common cold$ or coryza or croup or empyem$ or epipharyngit$ or epiglottit$ or epiglotit$ or flu or influenza or laryngit$ or laryngotracheobronchit$ or laryngo tracheo bronchit$ or laryngo tracheobronchit$ or laryngotracheit$ or nasopharyngit$ or otitis media or parainfluenza or pharyngit$ or pleurisy or pneumoni$ or pleuropneumoni$ or rhinit$ or rhinopharyngit$ or rhinosinusit$ or severe acute respiratory syndrome or SARS or sinusit$ or sore throat$ or throat infection$ or supraglottit$ or supraglotit$ or tonsillit$ or tonsilit$ or tracheit$ or whooping cough or pertussis or pertusis).mp. (1282)

  3. (ARTI or RTI or LRTI or URTI or ALRI or AURI or SARI).ti,ab. (67)

  4. (glandular fever or Infectious Mononucleosis or Epstein-Barr).ti,ab. (0)

  5. ((strep$ adj3 (throat$ or pharyn$ or tonsil$)) or (strep$ and (airway$ or pulmonary or brochopulmonar$ or brocho-pulmonar$ or respiratory$))).mp. (1)

  6. ((acute$ or exacerbat$ or flare$) adj3 (copd or coad or chronic obstructive pulmonary disease or chronic obstructive airway$ disease or chronic obstructive lung disease)).mp. (6)

  7. ((acute$ or subacute$ or exacerbat$ or prolonged) adj3 cough$).mp. (2)

  8. or/1-7 (1433)

  9. (early warning$ or red flag$ or (flag$ adj2 early)).ti,ab. (1206)

  10. (severity adj3 (score$ or scoring or scale$ or tool$ or instrument$ or index$ or indice$ or calculat$ or algorithm$ or metric$ or measur$ or criteri$ or code$)).ti,ab. (216)

  11. (severity adj3 (assess$ or estimat$ or evaluat$ or classif$ or rate? or rating? or value? or quantif$ or grade$ or chart$ or equation$ or table$ or model$ or framework$ or predict$)).ti,ab. (280)

  12. or/9-11 (1680)

  13. (curb65 or crb65 or curb-65 or crb-65 or news2 or enews or pnews).ti,ab. (0)

  14. ((curb or news) adj3 (criteri$ or rule$ or scor$ or predict$ or tool$)).ti,ab. (146)

  15. CENTOR.ti,ab. (0)

  16. (PMEWS or eMEWS).ti,ab. (0)

  17. (McIsaac adj (score$ or scoring or criteri$)).ti,ab. (0)

  18. (sino-nasal outcome test$ or SNOT-22 or SNOT22).ti,ab. (0)

  19. (pneumonia severity index or PSI or (PORT adj (Score$ or scoring))).ti,ab. (165)

  20. or/13-19 (311)

  21. 12 or 20 (1989)

  22. 8 and 21 (12)

  23. (triage$ or triaging).ti,ab. (126)

  24. ((stratif$ or priorit$ or classif$) adj3 (patient$ or outpatient$)).ti,ab. (145)

  25. ((stratif$ or priorit$ or classif$) adj3 (symptom$ or sign? or illness$ or disease$ or disorder$ or severity or risk$)).ti,ab. (510)

  26. or/23-25 (750)

  27. 8 and 26 (9)

  28. ((initial or first or primary or point of care) adj3 (assess$ or evaluat$ or examin$ or screen$) adj3 (patient$ or outpatient$ or sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (18)

  29. ((sign? or symptom$) adj2 (score$ or scoring)).ti,ab. (11)

  30. ((assess$ or evaluat$ or determin$ or detect$ or analys$ or screen$) adj5 (severe$ or severity or serious$) adj5 (sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (25)

  31. ((patient$ or sign? or symptom$ or illness$ or disease$ or disorder$ or infection$) adj3 acuity).ti,ab. (15)

  32. or/28-31 (69)

  33. 8 and 32 (3)

  34. (clinical$ adj5 (decision$ or predicti$) adj5 (aid? or algorithm? or characteristic? or criteri$ or evaluation? or index or indices or marker? or method$ or model$ or panel? or parameter? or rule or rules or score? or scoring or screen$ or signs or symptoms or system? or technique? or test$ or tool? or value? or variable$)).mp. (45)

  35. (clinical$ adj (predicti$ or predictor$)).ti,ab. (3)

  36. (rule in or ruled in or rule out or ruled out).ti,ab. (3585)

  37. (predict$ adj5 (severe$ or severity or serious$) adj5 (sign? or symptom$ or illness$ or disease$ or disorder$ or infection$)).ti,ab. (13)

  38. ((predict$ or prognos$ or cluster$) adj3 (sign? or symptom$)).ti,ab. (158)

  39. or/34-38 (3801)

  40. 8 and 39 (4)

  41. 22 or 27 or 33 or 40 (24)

  42. remove duplicates from 41 (24)

Key:

$ = truncation

? = optional wildcard – one or no characters

ti,ab = terms in title or abstract fields

mp = multi-purpose field search – terms in title, original title, abstract, name of substance word, or subject heading word

adj3 = terms within three words of each other (any order)

adj = terms next to each other in order specified

NHS Economic Evaluation Database (NHS EED)

via CRD www.crd.york.ac.uk/CRDWeb/HomePage.asp

Date range searched: Inception to 31 March 2015

Date searched: 15 May 2023

Records retrieved: 126

  1. MeSH DESCRIPTOR Respiratory Tract Infections EXPLODE ALL TREES IN NHSEED (582)

  2. ((airway* or bronchopulmonar* or broncho-pulmonar* or tracheobronch* or tracheo-bronch* or pulmonar* tract or pulmonary or respirat* tract or respiratory or chest or lung* or lobar or pleura*) NEAR4 (infect* or coinfect* or inflam* or swollen or swelling* or abscess*)) IN NHSEED (178)

  3. (bronchit* or bronchiolit* or allergic bronchopulmon* or bronchopneumon* or common cold* or coryza or croup or empyem* or epipharyngit* or epiglottit* or epiglotit* or flu or influenza or laryngit* or laryngotracheobronchit* or laryngo tracheo bronchit* or laryngo tracheobronchit* or laryngotracheit* or nasopharyngit* or otitis media or parainfluenza or pharyngit* or pleurisy or pneumoni* or pleuropneumoni* or rhinit* or rhinopharyngit* or rhinosinusit* or severe acute respiratory syndrome or SARS or sinusit* or sore throat* or throat infection* or supraglottit* or supraglotit* or tonsillit* or tonsilit* or tracheit* or whooping cough or pertussis or pertusis) IN NHSEED (826)

  4. (ARTI or RTI or LRTI or URTI or ALRI or AURI or SARI) IN NHSEED (29)

  5. MeSH DESCRIPTOR Infectious Mononucleosis IN NHSEED (0)

  6. (glandular fever or Infectious Mononucleosis or Epstein-Barr) IN NHSEED (3)

  7. ((strep* NEAR4 (throat* or pharyn* or tonsil*)) or (strep* and (airway* or pulmonary or brochopulmonar* or brocho-pulmonar* or respiratory*))) IN NHSEED (22)

  8. ((acute* or exacerbat* or flare*) NEAR4 (copd or coad or chronic obstructive pulmonary disease or chronic obstructive airway* disease or chronic obstructive lung disease)) IN NHSEED (27)

  9. ((acute* or subacute* or exacerbat* or prolonged) NEAR4 cough*) IN NHSEED (3)

  10. #1 or #2 or #3 or #4 or #5 or #6 or #7 or #8 or #9 (1057)

  11. MeSH DESCRIPTOR early warning score IN NHSEED (0)

  12. MeSH DESCRIPTOR ‘Severity of Illness Index’ IN NHSEED (0)

  13. (early warning* or red flag* or (flag* NEAR3 early)) IN NHSEED (5)

  14. (severity NEAR4 (score* or scoring or scale* or tool* or instrument* or index* or indice* or calculat* or algorithm* or metric* or measur* or criteri* or code*)) IN NHSEED (660)

  15. (severity NEAR4 (assess* or estimat* or evaluat* or classif* or rate* or rating* or value* or quantif* or grade* or chart* or equation* or table* or model* or framework* or predict*)) IN NHSEED (88)

  16. #11 or #12 or #13 or #14 or #15 (709)

  17. (curb65 or crb65 or curb-65 or crb-65 or news2 or enews or pnews) IN NHSEED (0)

  18. ((curb or news) NEAR4 (criteri* or rule* or scor* or predict* or tool*)) IN NHSEED (0)

  19. CENTOR IN NHSEED (5)

  20. (PMEWS or eMEWS) IN NHSEED (0)

  21. (McIsaac NEAR1 (score* or scoring or criteri*)) IN NHSEED (0)

  22. (sino-nasal outcome test* or SNOT-22 or SNOT22) IN NHSEED (0)

  23. (pneumonia severity index or PSI or (PORT NEAR1 (Score* or scoring))) IN NHSEED (9)

  24. #17 or #18 or #19 or #20 or #21 or #22 or #23 (14)

  25. #16 or #24 (719)

  26. #10 and #25 (55)

  27. MeSH DESCRIPTOR Triage IN NHSEED (47)

  28. (triage* or triaging) IN NHSEED (111)

  29. ((stratif* or priorit* or classif*) NEAR4 (patient* or outpatient*)) IN NHSEED (107)

  30. ((stratif* or priorit* or classif*) NEAR4 (symptom* or sign* or illness* or disease* or disorder* or severity or risk*)) IN NHSEED (179)

  31. #27 or #28 or #29 or #30 (368)

  32. #10 and #31 (24)

  33. MeSH DESCRIPTOR Symptom Assessment IN NHSEED (0)

  34. MeSH DESCRIPTOR Patient Acuity IN NHSEED (5)

  35. ((initial or first or primary or point of care) NEAR4 (assess* or evaluat* or examin* or screen*) NEAR4 (patient* or outpatient* or sign* or symptom* or illness* or disease* or disorder* or infection*)) IN NHSEED (65)

  36. ((sign* or symptom*) NEAR3 (score* or scoring)) IN NHSEED (153)

  37. ((assess* or evaluat* or determin* or detect* or analys* or screen*) NEAR6 (severe* or severity or serious*) NEAR6 (sign* or symptom* or illness* or disease* or disorder* or infection*)) IN NHSEED (109)

  38. ((patient* or sign* or symptom* or illness* or disease* or disorder* or infection*) NEAR4 acuity) IN NHSEED (27)

  39. #33 or #34 or #35 or #36 or #37 or #38 (346)

  40. #10 and #39 (27)

  41. MeSH DESCRIPTOR Clinical Decision Rules IN NHSEED (0)

  42. (clinical* NEAR6 (decision* or predicti*) NEAR6 (aid* or algorithm* or characteristic* or criteri* or evaluation* or index or indices or marker* or method* or model* or panel* or parameter* or rule or rules or score* or scoring or screen* or signs or symptoms or system* or technique* or test* or tool* or value* or variable*)) IN NHSEED (199)

  43. (clinical* NEAR1 (predicti* or predictor*)) IN NHSEED (12)

  44. (rule in or ruled in or rule out or ruled out) IN NHSEED (174)

  45. (predict* NEAR6 (severe* or severity or serious*) NEAR6 (sign* or symptom* or illness* or disease* or disorder* or infection*)) IN NHSEED (4)

  46. ((predict* or prognos* or cluster*) NEAR4 (sign* or symptom*)) IN NHSEED (23)

  47. #41 or #42 or #43 or #44 or #45 or #46 (401)

  48. #10 and #47 (41)

  49. #26 or #32 or #40 or #48 (126)

Key:

MeSH DESCRIPTOR = subject heading (MeSH heading)

EXPLODE ALL TREES = exploded subject heading (MeSH heading)

NEAR4 = terms within four words of each other (specified order only)

* = truncation

Appendix 2 Excluded studies

Study Reason for exclusion
Al Hussain SK, Kurdi A, Abutheraa N, et al. Validity of pneumonia severity assessment scores in Africa and South Asia: a systematic review and meta-analysis. Healthcare 2021;9:11 Population (includes hospitalised patients)
Anevlavis S, Bouros D. Community acquired bacterial pneumonia. Expert Opin Pharmacother 2010;11:361–74 Study design (not a systematic review)
Anonymous. Age-sex differences in the global burden of lower respiratory infections and risk factors, 1990–2019: results from the Global Burden of Disease Study 2019. Lancet Infect Dis 2022;22:1626–47 Study design (not a systematic review)
Asrar Khan W, Woodhead M. Major advances in managing community-acquired pneumonia. F1000Prime Rep 2013;5:43 Study design (not a systematic review)
Barbagelata E, Cilloniz C, Dominedo C, et al. Gender differences in community-acquired pneumonia. Minerva Med 2020;111:153–65 Population (includes children and hospitalised patients)
Bergmann M, Haasenritter J, Beidatsch D, et al. Prevalence, aetiologies and prognosis of the symptom cough in primary care: a systematic review and meta-analysis. BMC Fam Pract 2021;22:151 Intervention (assesses prevalence, aetiologies and prognosis, not symptoms, signs and EWS)
Berti E, Galli L, de Martino M, Chiappini E. International guidelines on tackling community-acquired pneumonia show major discrepancies between developed and developing countries. Acta Paediatr 2013;102:4–16 Population (includes children)
Bird JH, Biggs TC, King EV. Controversies in the management of acute tonsillitis: an evidence-based review. Clin Otolaryngol 2014;39:368–74 Study design (not a systematic review)
Boulet LP. Future directions in the clinical management of cough: ACCP evidence-based clinical practice guidelines. Chest 2006;129:287S–92 Study design (not a systematic review)
Braman SS. Chronic cough due to acute bronchitis: ACCP evidence-based clinical practice guidelines. Chest 2006;129:95S–103 Study design (not a systematic review)
Bryan C, Boren SA. The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Inform Prim Care 2008;16:79–91 Population (not specific to ARI)
Cabanas AM, Fuentes-Guajardo M, Latorre K, et al. Skin pigmentation influence on pulse oximetry accuracy: a systematic review and bibliometric analysis. Sensors(Basel) 2022;22:29 Population (includes ICU patients, healthy adults, children and COVID patients)
Caini S, Kroneman M, Wiegers T, et al. Clinical characteristics and severity of influenza infections by virus type, subtype, and lineage: a systematic literature review. Influenza Other Respir Viruses 2018;12:780–92 Population (includes children and hospitalised patients)
Campbell SG, Patrick W, Urquhart DG, et al. Patients with community acquired pneumonia discharged from the emergency department according to a clinical practice guideline. Emerg Med J 2004;21:667–9 Study design (not a systematic review)
Carvalho É, Estrela M, Zapata-Cachafeiro M, et al. E-health tools to improve antibiotic use and resistances: a systematic review. Antibiotics (Basel) 2020;9:12 Population (includes children and hospitalised patients)
Chalmers JD, Singanayagam A, Akram AR, et al. Severity assessment tools for predicting mortality in hospitalised patients with community-acquired pneumonia: systematic review and meta-analysis. Thorax 2010;65:878–83 Population (includes hospitalised patients)
Chalmers JD, Mandal P, Singanayagam A, et al. Severity assessment tools to guide ICU admission in community-acquired pneumonia: systematic review and meta-analysis. Intensive Care Med 2011;37:1409–20 Population (includes hospitalised patients)
Chalmers JD, Rutherford J. Can we use severity assessment tools to increase outpatient management of community-acquired pneumonia?. Eur J Intern Med 2012;23(5):398–406 Study design (not a systematic review)
Chen G, Xu K, Sun F, et al. Risk factors of multidrug-resistant bacteria in lower respiratory tract infections: a systematic review and meta-analysis. Can J Infect Dis Med Microbiol 2020;2020:7268519 Population (includes children and hospitalised patients)
Chiappini E, Regoli M, Bonsignori F, et al. Analysis of different recommendations from international guidelines for the management of acute pharyngitis in adults and children. Clin Ther 2011;33:48–58 Outcomes (compares international guidelines on the management of pharyngitis, does not report relevant outcomes)
Cho I, Bates DW. Behavioral economics interventions in clinical decision support systems. Yearb Med Inform 2018;27:114–21 Intervention (background paper on clinical decision support systems, not signs, symptoms and EWS)
Cohen JF, Pauchard JY, Hjelm N, et al. Efficacy and safety of rapid tests to guide antibiotic prescriptions for sore throat. Cochrane Database Syst Rev 2020;6:CD012431 Population (includes children)
Corrales-Medina VF, Suh KN, Rose G, et al. Cardiac complications in patients with community-acquired pneumonia: a systematic review and meta-analysis of observational studies. PLOS Med 2011;8:e1001048 Population (includes hospitalised patients)
Corrêa RA, Costa AN, Lundgren F, et al. 2018 recommendations for the management of community acquired pneumonia. J Bras Pneumol 2018;44:405–23 Study design (not a systematic review)
Coutinho G, Duerden M, Sessa A, et al. Worldwide comparison of treatment guidelines for sore throat. Int J Clin Pract 2021;75:no pagination Outcomes (comparison of guidelines, no outcomes of interest)
Dale AP, Marchello C, Ebell MH. Clinical gestalt to diagnose pneumonia, sinusitis, and pharyngitis: a meta-analysis. Br J Gen Pract 2019;69:e444–53 Intervention (assessment of clinical gestalt rather than signs and symptoms)
DeLaney M, Khoury C. Community-acquired pneumonia in the emergency department. Emerg Med Pract 2021;23:1–24 Study design (not a systematic review)
Demirdal T, Sen P, Emir B. Predictors of mortality in invasive pneumococcal disease: a meta-analysis. Expert Rev Anti Infect Ther 2021;19:927–44 Population (includes children and non-ARI patients)
Derber CJ, Troy SB. Head and neck emergencies: bacterial meningitis, encephalitis, brain abscess, upper airway obstruction, and jugular septic thrombophlebitis. Med Clin North Am 2012;96:1107–26 Study design (not a systematic review)
Dhawan N, Pandya N, Khalili M, et al. Predictors of mortality for nursing home-acquired pneumonia: a systematic review. BioMed Res Int 2015;2015:285983 Outcomes (unclear whether relevant outcomes are assessed within 4 weeks of consultation; outcomes/results are discussed, rather than clearly reported)
Dobler CC, Sanchez M, Gionfriddo MR, et al. Impact of decision aids used during clinical encounters on clinician outcomes and consultation length: a systematic review. BMJ Qual Saf 2019;28:499–510 Intervention (clinical decision rules for a range of conditions, not just ARI)
Dosa D. Should I hospitalize my resident with nursing home-acquired pneumonia?. J Am Med Dir Assoc 2006;7:S74–80, 73 Duplicate report
Durand C, Alfandari S, Béraud G, et al. Clinical decision support systems for antibiotic prescribing: an inventory of current French language tools. Antibiotics (Basel) 2022;11:14 Population (includes children and non-ARI conditions)
Ebell MH, Smith MA, Barry HC, et al. The rational clinical examination. Does this patient have strep throat?. JAMA 2000;284:2912–8 Study design (not a systematic review)
Ebell MH, White LL, Casault T. A systematic review of the history and physical examination to diagnose influenza. J Am Board Fam Pract 2004;17:1–5 Outcomes (outcome was confirmed diagnosis of influenza, no outcomes relating to severity of disease, etc.)
Ebell MH, Afonso A. A systematic review of clinical decision rules for the diagnosis of influenza. Ann Fam Med 2011;9:69–77 Outcomes (outcome was confirmed diagnosis of influenza, no outcomes relating to severity of disease, etc.)
Ebell MH, Grad R. Top 20 research studies of 2014 for primary care physicians. Am Fam Physician 2015;92:377–83 Intervention
Ebell MH, Marchello C, Callahan M. Clinical diagnosis of Bordetella pertussis infection: a systematic review. J Am Board Fam Med 2017;30:308–19 Outcomes (outcome was confirmed diagnosis of Bordetella pertussis infection, no outcomes relating to severity of disease, etc.)
Ebell MH, McKay B, Dale A, et al. Accuracy of signs and symptoms for the diagnosis of acute rhinosinusitis and acute bacterial rhinosinusitis. Ann Fam Med 2019;17:164–72 Outcomes (outcome was confirmed diagnosis of acute rhinosinusitis and acute bacterial rhinosinusitis, no outcomes relating to severity of disease, etc.)
Ebell MH, Rahmatullah I, Cai X, et al. A systematic review of clinical prediction rules for the diagnosis of influenza. J Am Board Fam Med 2021;34:1123–40 Population (includes children)
El-Gohary M, Hay AD, Coventry P, et al. Corticosteroids for acute and subacute cough following respiratory tract infection: a systematic review. Fam Pract 2013;30:492–500 Intervention (treatment, not assessment of severity)
Elmenawi KA, Anil V, Gosal H, et al. The importance of measuring troponin in chronic obstructive pulmonary disease exacerbations: a systematic review. Cureus 2021;13:e17451 Population (exacerbation of COPD, not suspected ARI patients)
Exarchos K, Aggelopoulou A, Oikonomou A, et al. Review of artificial intelligence techniques in chronic obstructive lung disease. IEEE J Biomed Health Inform 2022;26:2331–8 Population (COPD, not suspected ARI patients)
Fall A, Kenmoe S, Ebogo-Belobo JT, et al. Global prevalence and case fatality rate of Enterovirus D68 infections, a systematic review and meta-analysis. PLOS Negl Trop Dis 2022;16:e0010073 Intervention (prevalence and case fatality rate, not assessment of signs and symptoms)
Fendrick AM, Saint S, Brook I, et al. Diagnosis and treatment of upper respiratory tract infections in the primary care setting. Clin Ther 2001;23:1683–706 Study design (not a systematic review)
Ferdinands JM, Thompson MG, Blanton L, et al. Does influenza vaccination attenuate the severity of breakthrough infections? A narrative review and recommendations for further research. Vaccine 2021;39:3678–95 Population (includes hospitalised patients and children)
Fischer C, Knüsli J, Lhopitallier L, et al. Pulse oximetry as an aid to rule out pneumonia among patients with a lower respiratory tract infection in primary care. Antibiotics (Basel) 2023;12:2 Study design (not a systematic review)
Franciosi LG, Page CP, Celli BR, et al. Markers of exacerbation severity in chronic obstructive pulmonary disease. Respir Res 2006;7:74 Population (COPD not ARI)
Froom J, Culpepper L, Green LA, et al. A cross-national study of acute otitis media: risk factors, severity, and treatment at initial visit. Report from the International Primary Care Network (IPCN) and the Ambulatory Sentinel Practice Network (ASPN). J Am Board Fam Pract 2001;14:406–17 Population (includes children)
Garten S, Falkner RV. Continual smoking of mentholated cigarettes may mask the early warning symptoms of respiratory disease. Prev Med 2003;37:291–6 Study design (not a systematic review)
Gleeson LL, Clyne B, Barlow JW, et al. Medication safety incidents associated with the remote delivery of primary care: a rapid review. Int J Pharm Pract 2022;30:495–506 Intervention (not related to ARI)
Goka EA, Vallely PJ, Mutton KJ, Klapper PE. Single and multiple respiratory virus infections and severity of respiratory disease: a systematic review. Paediatr Respir Rev 2014;15:363–70 Population (includes hospitalised patients and children)
Graffelman AW, le Cessie S, Knuistingh Neven A, et al. Can history and exam alone reliably predict pneumonia?. J Fam Pract 2007;56:465–70 Study design (not a systematic review)
Haimi M, Gesser-Edelsburg A. Application and implementation of telehealth services designed for the elderly population during the COVID-19 pandemic: a systematic review. Health Informatics J 2022;28:14604582221075561 Intervention (telemedicine services, not assessment of ARI)
Hirner S, Pigoga JL, Naidoo AV, et al. Potential solutions for screening, triage, and severity scoring of suspected COVID-19 positive patients in low-resource settings: a scoping review. BMJ Open 2021;11:e046130 Intervention (focused on patients suspected or confirmed COVID, not ARI)
Htun TP, Sun Y, Chua HL, Pang J. Clinical features for diagnosis of pneumonia among adults in primary care setting: a systematic and meta-review. Sci Rep 2019;9:7600 Outcome (outcome is diagnosis of pneumonia, not escalation of care, antibiotic use, severity, mortality, etc.)
Huntley AL, Davies B, Jones N, et al. Determining when a hospital admission of an older person can be avoided in a subacute setting: a systematic review and concept analysis. J Health Serv Res Policy 2020;25:252–64 Intervention (not assessment of scoring methods or procedures to assess patients with ARI)
Justicia-Grande AJ, Pardo Seco J, Rivero Calle I, Martinón-Torres F. Clinical respiratory scales: which one should we use?. Expert Rev Respir Med 2017;11:925–43 Population (includes children and non-ARI)
Kerdemelidis M, Lennon D, Arroll B, Peat B. Guidelines for sore throat management in New Zealand. N Z Med J 2009;122:10–8 Population (includes children)
Kolditz M, Ewig S. Community-acquired pneumonia in adults. Dtsch Arztebl Int 2017;114:838–48 Study design (not a systematic review)
Krüger K, Töpfner N, Berner R, et al. Clinical practice guideline: sore throat. Dtsch Arztebl Int 2021;118:188–94 Population (includes children)
Krüger K, Holzinger F, Trauth J, et al. Chronic cough. Dtsch Arztebl Int 2022;119:59–65 Population (chronic cough, not ARI)
Kulik E, Stuart B, Willcox M. Predictors of rheumatic fever in sore throat patients: a systematic review and meta-analysis. Trans R Soc Trop Med Hyg 2022;116:286–97 Population (includes children)
Kwok CS, Loke YK, Woo K, Myint PK. Risk prediction models for mortality in community-acquired pneumonia: a systematic review. BioMed Res Int 2013;2013:504136. Population (includes hospitalised patients)
Launders N, Ryan D, Winchester CC, et al. Management of community-acquired pneumonia: an observational study in UK primary care. Pragmat Obs Res 2019;10:53–65 Study design (not a systematic review)
Li J, Zhou K, Duan H, et al. Value of D-dimer in predicting various clinical outcomes following community-acquired pneumonia: a network meta-analysis. PLOS ONE 2022;17:e0263215 Intervention (assessment of D-dimer, not signs, symptoms and EWS)
Liapikou A, Torres A. Current treatment of community-acquired pneumonia. Expert Opin Pharmacother 2013;14:1319–32 Intervention (therapies for patients with CAP, not severity or outcomes)
Little P, Rumsby K, Kelly J, et al. Information leaflet and antibiotic prescribing strategies for acute lower respiratory tract infection: a randomized controlled trial. JAMA 2005;293:3029–35 Study design (not a systematic review)
Little P, Williamson I. Sore throat management in general practice. Fam Pract 1996;13:317–21 Intervention (treatment and management, not assessment of symptoms and outcomes)
Loeb M. Community-acquired pneumonia. BMJ Clin Evid 2010;18:18 Intervention (therapies for patients with CAP, not assessment of severity)
Loke YK, Kwok CS, Niruban A, Myint PK. Value of severity scales in predicting mortality from community-acquired pneumonia: systematic review and meta-analysis. Thorax 2010;65:884–90 Population (includes hospitalised patients)
Long B, Long D, Koyfman A. Emergency medicine evaluation of community-acquired pneumonia: history, examination, imaging and laboratory assessment, and risk scores. J Emerg Med 2017;53:642–52 Study design (not a systematic review)
Ma HM, Ip M, Woo J. Effect of age and residential status on the predictive performance of CURB-65 score. Intern Med J 2015;45:300–4 Study design (not a systematic review)
Magaziner J, Tenney JH, DeForge B, et al. Prevalence and characteristics of nursing home-acquired infections in the aged. J Am Geriatr Soc 1991;39:1071–8 Study design (not a systematic review)
Malosh RE, Martin ET, Ortiz JR, Monto AS. The risk of lower respiratory tract infection following influenza virus infection: a systematic and narrative review. Vaccine 2018;36:141–7 Population (includes children)
Marchello CS, Ebell MH, Dale AP, et al. Signs and symptoms that rule out community-acquired pneumonia in outpatient adults: a systematic review and meta-analysis. J Am Board Fam Med 2019;32:234–47 Outcomes (outcome is diagnosis of CAP, not escalation of care, antibiotic use, severity, mortality, etc.)
Marti C, Garin N, Grosgurin O, et al. Prediction of severe community-acquired pneumonia: a systematic review and meta-analysis. Crit Care 2012;16:R141 Population (includes hospitalised patients)
Martinez FJ. Acute exacerbations of chronic bronchitis: diagnosis and therapy.

J Clin Outcomes Manag 2004;11(10):659–73		 Study design (not a systematic review)
Matthys H, Kamin W. Positioning of the Bronchitis Severity Score (BSS) for standardised use in clinical studies. Curr Med Res Opin 2013;29:1383–90 Population (includes children)
Maxwell DJ, Easton KL. Community-acquired pneumonia. J Pharm Pract Res 2004;34:212–7 Study design (not a systematic review)
McDonagh MS, Peterson K, Winthrop K, et al. Interventions to reduce inappropriate prescribing of antibiotics for acute respiratory tract infections: summary and update of a systematic review. J Int Med Res 2018;46:3337–57 Intervention (interventions to reduce prescribing, not EWS or signs and symptoms)
Memon RA, Rashid MA, Avva S, et al. The use of the SMART-COP score in predicting severity outcomes among patients with community-acquired pneumonia: a meta-analysis. Cureus 2022;14:e27248 Population (includes hospitalised patients)
Mertz D, Lo CK, Lytvyn L, et al. Pregnancy as a risk factor for severe influenza infection: an individual participant data meta-analysis. BMC Infect Dis 2019;19:683 Population (pregnant women, also includes hospitalised patients)
Modi AR, Kovacs CS. Community-acquired pneumonia: strategies for triage and treatment. Cleve Clin J Med 2020;87:145–51 Study design (not a systematic review)
Moore A, Ashdown HF, Shinkins B, et al. Clinical characteristics of pertussis-associated cough in adults and children: a diagnostic systematic review and meta-analysis. Chest 2017;152:353–67 Population (includes children and hospitalised patients)
Moore A, Harnden A, Grant CC, et al. Clinically diagnosing pertussis-associated cough in adults and children: CHEST guideline and expert panel report. Chest 2019;155:147–54 Study design (not a systematic review)
Morice AH. A new way to look at acute cough in the pharmacy. Clin Pharm 2017;9 Study design (not a systematic review)
Moriyama M, Hugentobler WJ, Iwasaki A. Seasonality of respiratory viral infections. Annu Rev Virol 2020;7:83–101 Study design (not a systematic review)
Mosby LG, Rasmussen SA, Jamieson DJ. 2009 pandemic influenza A (H1N1) in pregnancy: a systematic review of the literature. Am J Obstet Gynecol 2011;205:10–8 Intervention (impact of pandemic H1N1 influenza in pregnancy, not assessment of symptoms, signs and EWS in ARI)
Myint PK, Kwok CS, Majumdar SR, et al. The International Community-Acquired Pneumonia (CAP) Collaboration Cohort (ICCC) study: rationale, design and description of study cohorts and patients. BMJ Open 2012;2 Population (includes hospitalised patients)
Nabovati E, Jeddi FR, Farrahi R, Anvari S. Information technology interventions to improve antibiotic prescribing for patients with acute respiratory infection: a systematic review. Clin Microbiol Infect 2021;27:838–45 Intervention (includes children and hospitalised patients)
Neuner JM, Hamel MB, Phillips RS, et al. Diagnosis and management of adults with pharyngitis. A cost-effectiveness analysis. Ann Intern Med 2003;139:113–22 Study design (not a systematic review)
Noguchi S, Yatera K, Kawanami T, et al. Pneumonia severity assessment tools for predicting mortality in patients with healthcare-associated pneumonia: a systematic review and meta-analysis. Respiration 2017;93:441–450 Population (includes hospitalised patients)
Obisesan O. The evaluation of upper respiratory tract infection symptoms to show the significance of developing a quality-of-life evaluation instrument for upper respiratory tract infections to assess respiratory disorder-related disability. Am J Ther 2005;12:142–50 Study design (not a systematic review)
Petrozzino JJ, Smith C, Atkinson MJ. Rapid diagnostic testing for seasonal influenza: an evidence-based review and comparison with unaided clinical diagnosis. J Emerg Med 2010;39:476–90.e1 Population (includes children)
Phua J, Dean NC, Guo Q, et al. Severe community-acquired pneumonia: timely management measures in the first 24 hours. Crit Care 2016;20:237 Population (includes hospitalised patients)
Ponnapalli A, Khare Y, Dominic C, et al. Remote risk-stratification of dyspnoea in acute respiratory disorders: a systematic review of the literature. J R Coll Physicians Edinb 2021;51:221–9 Population (includes children, hospitalised patients and COVID patients)
Pratter MR. Overview of common causes of chronic cough: ACCP evidence-based clinical practice guidelines. Chest 2006;129:59S–62 Population (chronic cough, not ARI)
Renaud B, Santin A, Coma E, et al. Association between timing of intensive care unit admission and outcomes for emergency department patients with community-acquired pneumonia. Crit Care Med 2009;37:2867–74 Study design (not a systematic review)
Rodríguez-Acelas AL, Reich R, de Abreu Almeida M, et al. Nursing outcome ‘Severity of infection’: conceptual definitions for indicators related to respiratory problems. Invest Educ Enferm 2016;34:38–45 Intervention (not an assessment of symptoms, signs and EWS for the assessment of ARI)
Rombauts A, Abelenda-Alonso G, Cuervo G, et al. Role of the inflammatory response in community-acquired pneumonia: clinical implications. Expert Rev Anti Infect Ther 2022;20:1261–74 Study design (not a systematic review)
Rottman SJ, Shoaf KI, Schlesinger J, et al. Pandemic influenza triage in the clinical setting. Prehosp Disaster Med 2010;25:99–104 Study design (not a systematic review)
Schmit KM, Coeytaux RR, Goode AP, et al. Evaluating cough assessment tools: a systematic review. Chest 2013;144:1819–26 Population (includes tools for lung cancer, lung transplant, etc., not just ARI)
Schofield C, Colombo RE, Richard SA, et al. Comparable disease severity by influenza virus subtype in the acute respiratory infection consortium natural history study. Mil Med 2020;185:e1008–15 Study design (not a systematic review)
Schuetz P, Koller M, Christ-Crain M, et al. Predicting mortality with pneumonia severity scores: importance of model recalibration to local settings. Epidemiol Infect 2008;136(12):1628–37 Study design (not a systematic review)
Simpson SH, Marrie TJ, Majumdar SR. Do guidelines guide pneumonia practice? A systematic review of interventions and barriers to best practice in the management of community-acquired pneumonia. Respir Care Clin N Am 2005;11:1–13 Intervention (adherence to guidelines, not assessment of symptoms, signs and EWS)
Solari L, Acuna-Villaorduna C, Soto A, van der Stuyft P. Evaluation of clinical prediction rules for respiratory isolation of inpatients with suspected pulmonary tuberculosis. Clin Infect Dis 2011;52:595–603 Population (patients with pulmonary tuberculosis, not ARI)
Solari L, Soto A, Van der Stuyft P. Performance of clinical prediction rules for diagnosis of pleural tuberculosis in a high-incidence setting. Trop Med Int Health 2017;22:1283–92 Population (patients with pleural tuberculosis, not ARI)
Song WJ, Kim HJ, Shim JS, et al. Diagnostic accuracy of fractional exhaled nitric oxide measurement in predicting cough-variant asthma and eosinophilic bronchitis in adults with chronic cough: a systematic review and meta-analysis. J Allergy Clin Immunol 2017;140:701–9 Population (chronic cough not ARI)
Sunjaya AP, Ansari S, Jenkins CR. A systematic review on the effectiveness and impact of clinical decision support systems for breathlessness. NPJ Prim Care Respir Med 2022;32(1):no pagination Population (not ARI)
Thai TN, Dale AP, Ebell MH. Signs and symptoms of Group A versus Non-Group A strep throat: a meta-analysis. Fam Pract 2018;35:231–8 Population (includes children)
Torres A, Chalmers JD, Dela Cruz CS, et al. Challenges in severe community-acquired pneumonia: a point-of-view review. Intensive Care Med 2019;45:159–71 Study design (not a systematic review)
Vines C, Dean NC. Technology implementation impacting the outcomes of patients with CAP. Semin Respir Crit Care Med 2012;33:292–7 Intervention (assessment of technology implementation, not symptoms, signs and EWS)
Wallace E, Uijen MJ, Clyne B, et al. Impact analysis studies of clinical prediction rules relevant to primary care: a systematic review. BMJ Open 2016;6:e009957 Population (includes children)
Willis BH, Coomar D, Baragilly M. Comparison of Centor and McIsaac scores in primary care: a meta-analysis over multiple thresholds. Br J Gen Pract 2020;70:e245–54 Population (includes children)
Womack J, Kropa J. Community-acquired pneumonia in adults: rapid evidence review. Am Fam Physician 2022;105:625–30 Study design (not a systematic review)
Woolley SL, Bernstein JM, Davidson JA, Smith DR. Sore throat in adults – does the introduction of a clinical scoring system improve the management of these patients in a secondary care setting?. J Laryngol Otol 2005;119:550–5 Study design (not a systematic review)
Xie CX, Chen Q, Hincapié CA, et al. Effectiveness of clinical dashboards as audit and feedback or clinical decision support tools on medication use and test ordering: a systematic review of randomized controlled trials. J Am Med Inform Assoc 2022;29:1773–85 Population (any health condition, not specifically ARI)
Study Exclusion reason(s)
Bartenschlager CC, et al. A simulation-based cost-effectiveness analysis of severe acute respiratory syndrome coronavirus 2 infection prevention strategies for visitors of healthcare institutions. Value Health 2022;25(11):1846–52 Intervention (assessment of infection prevention strategies)
Bashir S, et al. Economic analysis of different throughput scenarios and implementation strategies of computer-aided detection software as a screening and triage test for pulmonary TB. PLOS ONE 2022;17(12):e0277393 Intervention (assessment of diagnostic strategies)
Bastos HN, et al. A prediction rule to stratify mortality risk of patients with pulmonary tuberculosis. PLOS ONE 2016;11(9):e016279 Study design (not an economic evaluation)
Chew R, et al. Modelling the cost-effectiveness of pulse oximetry in primary care management of acute respiratory infection in rural northern Thailand. Trop Med Int Health 2022;27(10):881–90 Population (includes children)
Chouaid C, et al. Cost-analysis of four diagnostic strategies for Pneumocystis carinii pneumonia in HIV-infected subjects. Eur Respir J 1995;8(9):1554–58 Intervention (assessment of diagnostic strategies)
Fan L, et al. Semiquantitative cough strength score and associated outcomes in noninvasive positive pressure ventilation patients with acute exacerbation of chronic obstructive pulmonary disease. Respir Med 2014;108(12):1801–7 Intervention (hospital inpatient setting)
Huijskens EGW, et al. The value of signs and symptoms in differentiating between bacterial, viral and mixed aetiology in patients with community-acquired pneumonia. J Med Microbiol 2014;63(Pt 3):441–52 Intervention (assessment of diagnostic strategies)

Study design (not an economic evaluation)
Melhuish A, et al. Cost evaluation of point-of-care testing for community-acquired influenza in adults presenting to the emergency department. J Clin Virol 2020;129:104533 Intervention (assessment of diagnostic strategies)
Nsengiyumva NP, et al. Triage of persons with tuberculosis symptoms using artificial intelligence-based chest radiograph interpretation: a cost-effectiveness analysis. Open Forum Infect Dis 2021;8(12):ofab567 Intervention (assessment of diagnostic strategies)
Spaeth B, et al. Impact of point-of-care testing for white blood cell count on triage of patients with infection in the remote Northern Territory of Australia. Pathology 2019;51(5):512–7 Intervention (assessment of diagnostic strategies)
van de Maat J, et al. Cost study of a cluster randomized trial on a clinical decision rule guiding antibiotic treatment in children with suspected lower respiratory tract infections in the emergency department. Pediatr Infect Dis J 2020;39(11):1026–31 Population (includes children)
Webb BJ, et al. Antibiotic use and outcomes after implementation of the drug resistance in pneumonia score in ED patients with community-onset pneumonia. Chest 2019;156(5):843–51 Study design (not an economic evaluation)

Appendix 3 Characteristics and results of the included reviews

Aalbers, 20114

Bibliographic reference Aalbers J, O’Brien KK, Chan WS, et al. Predicting streptococcal pharyngitis in adults in primary care: a systematic review of the diagnostic accuracy of symptoms and signs and validation of the Centor score. BMC Med 2011;9:67

Study details

Study type Systematic review
Study location Included studies were from USA, Canada, Europe, New Zealand, Thailand, Israel
Study setting Primary care (19 studies) and the emergency department (2 studies)
Study dates PubMed and EMBASE were searched to 26 July 2010; included studies were published between 1975 and 2008
Sources of funding Health Research Board of Ireland through the HRB Centre for Primary Care Research
Review question To analyse the current evidence on the usefulness of individual signs and symptoms in assessing the risk of streptococcal pharyngitis in adults, to assess the diagnostic accuracy of the Centor score as a decision rule for antibiotic treatment (discrimination analysis) and to perform a meta-analysis on validation studies of the Centor score (calibration analysis)
Inclusion criteria Studies were included if participants were recruited upon first presentation from an ambulatory care setting, had a sore throat as their main presenting complaint, and were ≥ 15 years of age. Both prospective and retrospective studies were included. Each included study assessed the diagnostic accuracy of signs and symptoms and/or validated the Centor score
Exclusion criteria Not reported
Study design of included studies Diagnostic accuracy studies
Sample size Twenty-one included studies, comprising 4839 patients (range 70–693), reported data on signs and symptoms. Fifteen included studies, comprising 2900 patients (range 70–453), reported data on the Centor score
Quality of included studies The overall quality of the included studies was good, assessed using a modified version of the QUADAS tool. The spectrum of patients was generally appropriate and representative, selection criteria were stated and the signs and symptoms were generally clearly described. Test and diagnostic review bias items scored well. Observer variation in assessing signs and symptoms was poorly reported
Target condition/outcome GABHS pharyngitis
Patient characteristics Mean age: range 23.9–35.6 years (where reported)

Sex: range 16.7–63.6% male (where reported)

Prevalence of GABHS pharyngitis: range 4.7–37.6%
Signs, symptoms and EWS Individual signs and symptoms:

  • Absence of cough

  • Fever

  • Anterior cervical adenopathy

  • Tender anterior cervical adenopathy

  • Any exudates (either tonsillar exudate or pharyngeal exudate or any exudate)



Centor score
Comparator/reference standard Throat culture
Results

Absence of cough (19 studies, 4653 patients)

  1. Sensitivity (95% CI): 0.74 (0.68 to 0.79)

  2. Specificity (95% CI): 0.49 (0.40 to 0.58)

  3. Positive likelihood ratio (95% CI): 1.46 (1.28 to 1.66)

  4. Negative likelihood ratio (95% CI): 0.53 (0.46 to 0.61)

Fever (21 studies, 4635 patients; the most widely used cut-off to indicate fever was 38.0 °C)

  1. Sensitivity (95% CI): 0.50 (0.39 to 0.62)

  2. Specificity (95% CI): 0.70 (0.58 to 0.79)

  3. Positive likelihood ratio (95% CI): 1.65 (1.40 to 1.95)

  4. Negative likelihood ratio (95% CI): 0.71 (0.64 to 0.80)

Anterior cervical adenopathy (9 studies, 2101 patients)

  1. Sensitivity (95% CI): 0.65 (0.55 to 0.74)

  2. Specificity (95% CI): 0.55 (0.45 to 0.64)

  3. Positive likelihood ratio (95% CI): 1.45 (1.25 to 1.67)

  4. Negative likelihood ratio (95% CI): 0.63 (0.52 to 0.76)

Tender anterior cervical adenopathy (16 studies, 4144 patients)

  1. Sensitivity (95% CI): 0.67 (0.52 to 0.79)

  2. Specificity (95% CI): 0.59 (0.49 to 0.69)

  3. Positive likelihood ratio (95% CI): 1.65 (1.41 to 1.92)

  4. Negative likelihood ratio (95% CI): 0.56 (0.41 to 0.76)

Any exudates (21 studies, 4839 patients)

  1. Sensitivity (95% CI): 0.57 (0.44 to 0.70)

  2. Specificity (95% CI): 0.74 (0.63 to 0.82)

  3. Positive likelihood ratio (95% CI): 2.20 (1.76 to 2.74)

  4. Negative likelihood ratio (95% CI): 0.58 (0.47 to 0.72)

Centor score ≥ 1 (11 studies)

  1. Sensitivity (95% CI): 0.95 (0.91 to 0.97)

  2. Specificity (95% CI): 0.18 (0.12 to 0.26)

  3. Positive likelihood ratio (95% CI): 1.16 (1.08 to 1.25)

  4. Negative likelihood ratio (95% CI): 0.27 (0.16 to 0.46)

Centor score ≥ 2 (12 studies)

  1. Sensitivity (95% CI): 0.79 (0.71 to 0.86)

  2. Specificity (95% CI): 0.55 (0.45 to 0.65)

  3. Positive likelihood ratio (95% CI): 1.76 (1.51 to 2.07)

  4. Negative likelihood ratio (95% CI): 0.37 (0.29 to 0.48)

Centor score ≥ 3 (the recommended cut-off point for empirical antibiotic treatment according to the ACP/ASIM guidelines) (11 studies)

  1. Sensitivity (95% CI): 0.49 (0.38 to 0.60)

  2. Specificity (95% CI): 0.82 (0.72 to 0.88)

  3. Positive likelihood ratio (95% CI): 2.68 (1.92 to 3.75)

  4. Negative likelihood ratio (95% CI): 0.62 (0.52 to 0.74)

Centor score 4 (11 studies)

  1. Sensitivity (95% CI): 0.18 (0.12 to 0.27)

  2. Specificity (95% CI): 0.95 (0.92 to 0.97)

  3. Positive likelihood ratio (95% CI): 3.85 (2.05 to 7.24)

  4. Negative likelihood ratio (95% CI): 0.86 (0.78 to 0.93)
Post-test probability of GABHS pharyngitis for a range of pre-test probabilities
Points Likelihood ratio Pretest probability of GABHS pharyngitis (%)
5 10 15 20 25 30 35 40
≥ 1 1.16 6 11 17 22 28 33 38 44
≥ 2 1.76 8 16 24 31 37 43 49 54
≥ 3 2.68 12 23 32 40 47 53 59 64
4 3.85 17 30 40 49 56 62 67 72
Authors’ conclusion Individual symptoms and signs have only a modest ability to rule in or out a diagnosis of GABHS pharyngitis. The Centor score uses a combination of signs and symptoms to predict the risk of GABHS pharyngitis; the score is well calibrated across a variety of countries and settings. It has reasonably good specificity, and can enhance the appropriate prescribing of antibiotics, but should be used with caution in low-prevalence settings of GABHS pharyngitis such as primary care
Limitations Prevalence of GABHS pharyngitis varied widely among the included studies (range 4.7–37.6%); however, the authors undertook a subgroup analysis based on prevalence for each score category of the Centor score. While not explicitly stated, the conclusion relating to the reasonably high specificity of the Centor score relates to the cut-off score of ≥ 3, which is the recommended cut-off point for empirical antibiotic treatment according to the ACP/ASIM guidelines
Comments There was a low risk of bias for each ROBIS domain. The conclusions of the review appear to be appropriate, noting the authors’ caution relating to the use of the Centor score when used as a decision aid for antibiotic prescribing

ACP/ASIM, American College of Physicians-American Society of Internal Medicine.

Reproduced with permission from Aalbers et al. 4 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The text includes additions and formatting changes to the original text.

Critical appraisal – ROBIS tool

Overall risk of bias Low
Applicability as a source of data Good

Akram, 20115

Bibliographic reference Akram AR, Chalmers JD, Hill AT. Predicting mortality with severity assessment tools in out-patients with community-acquired pneumonia. QJM 2011;104:871–9

Study details

Study type Systematic review
Study location Included studies were from USA, Canada, Netherlands, Germany, Spain, France, UK
Study setting Outpatients (either exclusively managed in the community or discharged from an emergency department < 24 hours after admission)
Study dates MEDLINE and EMBASE were searched between 1981 and 2010; included studies were published between 1997 and 2008
Sources of funding One of the authors was supported by a Clinical Research Training Fellowship from the Medical Research Council
Review question To systematically review the published literature in relation to pneumonia scoring systems [such as the Pneumonia Severity Index (PSI) and CURB65/CRB65] for predicting mortality in patients managed in outpatient settings
Inclusion criteria Studies were included if they reported data (calculation of severity score based on admission data) on at least 20 unselected outpatients with CAP. There were no inclusion/exclusion criteria relating to study design
Exclusion criteria Non-CAP diagnoses (e.g. non-pneumonic exacerbation of COPD)
Study design of included studies Nine prospective cohort studies, one retrospective case review and three RCTs
Sample size Thirteen included studies, comprising 5444 patients (range 48–1061)
Quality of included studies Overall, six studies were rated as good, five as moderate and two as suboptimal, using criteria relating to inclusion criteria, follow-up, measurement of severity score and potential confounding
Target condition/outcome 30-day mortality
Patient characteristics Mean age: range 46.8–77.3 (where reported)

Sex: not reported

Mortality rate: range 0–3.5%
Signs, symptoms and EWS PSI (10 studies)

CRB65 (4 studies)

CURB65 (2 studies)
Comparator/reference standard Not applicable
Results PSI (10 studies, 39–72 patients)

  1. PSI I–III (low risk): 0.2% mortality (8 of 3655 patients)

  2. PSI IV–V (high risk): 10.1% mortality (32 of 317 patients)
Comparing low against high risk (6 studies): Pooled sensitivity = 92% (64–100%), pooled specificity = 90% (89–91%). Negative likelihood ratio = 0.21 (0.08–0.59). Area under the sROC = 0.92 (standard error 0.03). The risk of death in low-risk patients (PSI I–III) was compared to the preset 1% predicted level of mortality, PSI had a relative risk of 0.35 (0.17–0.72) with no significant heterogeneity
CRB65 (4 studies, 1648 patients)

  1. CRB65 = 0: 0% mortality (879 patients)

  2. CRB65 = 1: 0.5% mortality (615 patients)

  3. CRB65 = 2: 6.3% mortality (126 patients)

  4. CRB65 = 3: 13.2% mortality (28 patients)

  5. CRB65 = 4: No patients in this category
Requirement for hospitalisation: Using the recommended cut-off of CRB65 > 0, pooled sensitivity = 100% (48–100%), pooled specificity = 65% (62–68%), with no significant heterogeneity (three studies). Using CRB65 > 1, pooled sensitivity = 81% (54–96%), pooled specificity = 91% (90–93%). Area under the sROC = 0.91 (standard error 0.05). Pooled diagnostic odds ratio for a CRB65 score ≥ 2 = 16.47 (4.9–55.4) with no significant heterogeneity. Estimates were limited by low event rate. Comparing the performance of CRB65 in patients with CRB65 0–1 (low-risk patients) to the preset 1% level of mortality, CRB65 was associated with a relative risk of 0.35 (0.10–1.16) with no significant heterogeneity

CURB65 (two studies; therefore, meta-analysis not feasible)

One study reported data in 676 outpatients and 1 study reported data in 176 outpatients; each study had 1 death in the outpatient group and both with CURB65 ≥ 2
Authors’ conclusion Patients in the low-risk CRB65 and PSI classes are at low risk of death when managed as outpatients, but further studies are needed in outpatient cohorts
Limitations The majority of the data presented were derived from patients initially assessed in hospital and discharged within 24 hours; the authors acknowledge that this is a significant limitation of the analysis and further studies in exclusively outpatient populations are required. The authors also comment on other potential confounders relating to patient factors which may have resulted in more high-risk patients being managed as outpatients
Comments There was a low risk of bias for each ROBIS domain. The conclusions of the review appear to be appropriate, noting the authors’ caution relating to the need for further studies in exclusively outpatient cohorts (as opposed to patients initially assessed in hospital and discharged within 24 hours)

sROC, summary receiver operator characteristic curve.

Critical appraisal – ROBIS tool

Overall risk of bias Low
Applicability as a source of data Good

Chalmers, 20116

Bibliographic reference Chalmers JD, Akram AR, Hill AT. Increasing outpatient treatment of mild community-acquired pneumonia: systematic review and meta-analysis. Eur Respir J 2011;37:858–64

Study details

Study type Systematic review
Study location Included studies were from USA, Canada, Spain and France
Study setting Emergency departments (five studies) and walk-in medical centres (one study)
Study dates PubMed and EMBASE were searched between January 1981 and April 2010; included studies were published between 1998 and 2007
Sources of funding One of the authors was supported by a Clinical Research Training Fellowship from the Medical Research Council (UK)
Review question To identify, synthesise and interpret the evidence relating to strategies to increase the proportion of low-risk patients with CAP treated in the community
Inclusion criteria Studies were included if they described an intervention aimed to increase the proportion of patients treated in the community, included a control group in which the intervention was withheld and included data reporting the safety of the intervention
Exclusion criteria Studies reporting outpatient care but without control data were not included
Study design of included studies RCTs; implementation studies with either a prospective or retrospective control group; prospective observational study with control
Sample size Six included studies, comprising 5092 patients (range 223–1901)
Quality of included studies The authors state that quality was assessed using standardised criteria and reference the Cochrane Handbook. They state that all of the included studies had significant limitations. Two studies used a retrospective control cohort design, which is associated with a significant risk of bias. In one study, the centres were not randomised, but decided independently to implement the PSI or not, with no way of knowing what other aspects of CAP management differed between centres. Two cluster RCTs were more robust; however, randomisation at the hospital level cannot ensure that PSI was not used at the individual–physician level in the control hospitals. The final study was more robust but was underpowered to detect mortality
Target condition/outcome Proportion of patients treated as outpatients, mortality, hospital re-admissions, patient satisfaction with care, HRQoL and return to work or usual activities
Patient characteristics Not reported
Signs, symptoms and EWS The interventions were generally complex, but all included a scoring system to identify low-risk patients; in five studies, the PSI was used to help determine where patients should be treated, in one study the authors derived their own criteria for inpatient care
Comparator/reference standard Usual care (prospective or retrospective control group) or low-intensity guideline implementation (vs. moderate or high intensity)
Results Five studies (4869 patients) were included in the meta-analysis for outpatient care (the other study randomised patients to out- or inpatient care, rather than implementing a guideline to increase the proportion of patients treated in the community); 64.6% of patients in the intervention groups were treated in the community compared with 48.7% of patients in the control groups. The interventions were associated with a significant increase in outpatient-managed patients (OR 2.31, 95% CI 2.03 to 2.63), and there was no significant heterogeneity.

Mortality was not increased in the intervention groups (OR 0.83, 95% CI 0.59 to 1.17; six studies). There was no increase in hospital re-admissions (OR 1.08, 95% CI 0.82 to 1.42; six studies). There was no difference in patient satisfaction with care between intervention and control groups (OR 1.21, 95% CI 0.97 to 1.49; three studies). There was no significant heterogeneity in these analyses.

There were insufficient data to pool studies of return to usual activities or quality of life. One study reported no significant difference between intervention and control groups in return to usual activities, or in patients reporting excellent or very good general health at 4 weeks. Two studies assessed quality of life using Short-Form 36 and reported no significant difference between intervention and control groups. One study reported no significant difference in return to work and usual activities at day 30 between groups
Authors’ conclusion Current evidence suggests that strategies to increase the proportion of patients treated in the community are safe, effective and acceptable to patients
Limitations Each study included in the review had significant methodological limitations. The interventions included in the studies were generally complex, the scoring system to identify low-risk patients was only one component and, as acknowledged by the authors, evaluating which components of the intervention were responsible for the effects seen is not straightforward
Comments There was a low risk of bias for each ROBIS domain. The conclusions of the review appear to be appropriate. However, the scoring system to identify low-risk patients was only one component of the interventions assessed

Critical appraisal – ROBIS tool

Overall risk of bias Low
Applicability as a source of data Acceptable (scoring system to identify low-risk patients was only one component of the interventions assessed)

Dosa, 200512

Bibliographic reference Dosa D. Should I hospitalize my resident with nursing home-acquired pneumonia? J Am Med Dir Assoc 2005;6:327–33

Study details

Study type Systematic review
Study location Included studies were from USA
Study setting Nursing home
Study dates MEDLINE was searched between 1966 and ‘present day’; included studies were published between 1998 and 2001
Sources of funding Not reported
Review question Are there prediction tools that can help determine when treating a resident in the nursing home is safe?
Inclusion criteria The author performed a structured search relating to the diagnosis, treatment and triage of residents with NHAP. There were no inclusion/exclusion criteria relating to study design
Exclusion criteria Not reported
Study design of included studies One prospective cohort study and two retrospective studies (relating to the question of interest)
Sample size Three included studies, comprising 1942 cases/episodes (range 158–1406)
Quality of included studies Not reported (studies do not appear to have been assessed for quality)
Target condition/outcome Thirty-day mortality
Patient characteristics Not reported
Signs, symptoms and EWS PSI

Five-point scale developed by Naughton and Mylotte

Eight-variable model developed by Mehr et al.
Comparator/reference standard Not applicable
Results PSI (1 study, 158 episodes)

  1. Similar reliability to that in patients with community-acquired pneumonia. However, 85% of nursing home residents were classified as high risk (class IV or V) requiring hospitalisation, making the PSI a poor discriminatory tool in the nursing home environment. Additionally, the difficulty in obtaining arterial blood gas measurements in the nursing home setting has severely limited its use
Five-point scale developed by Naughton and Mylotte (1 study, 378 cases)

  1. Analysis of a retrospective chart review revealed four predictors of mortality developed into a 5-point scale relating to respiratory rate, pulse rate, change in mental status and presence of dementia. Applying this model in the derivation cohort revealed increasing mortality with increasing score. However, the model does not appear to have been prospectively validated
Eight-variable model developed by Mehr et al. (1 study, 1406 episodes among 1044 residents)

  1. A model was developed using a prospective cohort, based on levels of serum urea nitrogen (BUN), white blood count, absolute lymphocyte count, heart rate, sex, body mass index, activities of daily living and mood deterioration in last 90 days. While the model was developed in a large study conducted in typical community nursing homes, it has not been independently validated
Author’s conclusion There are several problems with using prediction models in clinical practice. While they may predict mortality risk, they cannot determine whether a nursing home resident’s care would be better or worse in a hospital setting, and they do not account for the end-of-life wishes of nursing home residents. Prediction models often require data that are not readily available at the time that triage decisions need to be made, and they are often age-driven; nursing home residents are generally very old
Limitations This was a poorly conducted and reported systematic review, addressing multiple questions including the one of interest here. It is unclear whether all relevant studies were identified, the quality of the studies was not systematically assessed and limited details of the included studies were presented
Comments There was a high risk of bias for each ROBIS domain. The author’s conclusions appear appropriate based on the included studies; however, in view of the considerable risk of bias, they may not be reliable

Critical appraisal – ROBIS tool

Overall risk of bias High (poorly conducted and reported review, it is unclear whether all relevant studies were identified, the quality of included studies was not assessed and limited details of included studies were presented)
Applicability as a source of data Good

Ebell, 20197

Bibliographic reference Ebell MH, Walsh ME, Fahey T, et al. Meta-analysis of calibration, discrimination, and stratum-specific likelihood ratios for the CRB-65 score. J Gen Intern Med 2019;34:1304–13

Study details

Study type Systematic review/meta-analysis update of McNally et al., 2010
Study location Not fully reported. All but 3 studies were set in Europe, including 10 in Germany and 6 in Spain; none were set in the USA or Canada
Study setting Hospitalised patients, ambulatory patients and both; the 15 studies that included ambulatory patients in emergency department or primary care settings are relevant to this review
Study dates PubMed was searched from January 2009 to update a previous systematic review that searched up to June 2009; included studies were published between 2006 and 2015
Sources of funding One of the authors was supported by a 2018/9 Fulbright Teaching/Research award
Review question To perform an updated meta-analysis of the accuracy of the CRB-65 for mortality prediction
Inclusion criteria Studies reporting the accuracy of the CRB-65 score among patients with CAP. Studies had to provide sufficient data to calculate mortality for low-risk, moderate-risk and high-risk groups. Both prospective and retrospective cohort studies were included
Exclusion criteria Studies in children, studies in special populations (such as immunocompromised patients or those characterised by a comorbidity such as asthma, cancer or diabetes), and studies of patients with sepsis, hospital-acquired or ventilator-acquired pneumonia were excluded. Studies performed in countries classified as low income or lower middle income, and case control studies
Study design of included studies Nine studies gathered data retrospectively, while the remainder gathered data prospectively, often as part of the CAPNETZ disease registry
Sample size Twenty-nine included studies, comprising 1,089,419 patients (range 105–669,594). Thirteen studies where the rule was applied in both hospitalised and ambulatory settings included 20,282 patients (range 152–6142). Two studies in ambulatory settings included 956 patients (range 314–642)
Quality of included studies Overall, 12 studies were judged to be at low risk of bias and 17 studies were judged to be at high risk of bias, using an adaptation of the TRIPOD and PROBAST criteria. Of the 15 studies where the rule was applied in emergency department or primary care settings, 7 were judged to be at low risk of bias and 8 were judged to be at a high risk of bias
Target condition/outcome Thirty-day mortality
Patient characteristics Mean or median age: range 36.5–78.3

Sex: not reported

Mortality rate: range 0.5–18.0%
Signs, symptoms and EWS CRB-65
Comparator/reference standard Not applicable
Results Subgroup analysis of studies where the rule was applied in emergency department or primary care settings and patients could be treated as either outpatients or inpatients

  1. Summary estimate of Observed/Expected (O : E) ratio: 1.05 (95% CI 0.87 to 1.27); 15 studies (n = 20,667 patients), I2 = 91.3%

  2. Area under the receiver characteristic curve (AUC): 0.75 (95% CI 0.71 to 0.78); 13 studies (n = 14,373 patients), I2 = 85.1%

  3. Stratum-specific likelihood ratios:

  4. CRB-65 = 0 (low risk): 0.12 (95% CI 0.07 to 0.19; 11 studies, I2: 34.6%)

  5. CRB-65 = 1–2 (moderate risk): 1.10 (95% CI 0.96 to 1.25; 15 studies, I2: 93.8%)

  6. CRB-65 = 3–4 (high risk): 5.59 (95% CI 4.25 to 7.34; 15 studies, I2: 75.6%)
Subgroup analysis of studies at low risk of bias where the rule was applied in emergency department or primary care settings and patients could be treated as either outpatients or inpatients

  1. Summary estimate of Observed/Expected (O : E) ratio: 0.88 (95% CI 0.69 to 1.13); 8 studies (n = 17,248 patients), I2 = 92.7%

  2. Area under the receiver characteristic curve (AUC): 0.76 (95% CI 0.70 to 0.81); 17 studies (n = 11,106 patients), I2 = 91.0%

  3. Stratum-specific likelihood ratios:

  4. CRB-65 = 0 (low risk): 0.13 (95% CI 0.08 to 0.21; 8 studies, I2: 40.0%)

  5. CRB-65 = 1–2 (moderate risk): 1.30 (95% CI 1.17 to 1.44; 8 studies, I2: 84.7%)

  6. CRB-65 = 3–4 (high risk): 5.61 (95% CI 3.71 to 8.47; 8 studies, I2: 85.6%)
Authors’ conclusion CRB-65 can be used to estimate mortality risk, providing a useful check on physician judgement. Patients with a score of 0 (low-risk group) have a very low mortality risk and can be safely treated as outpatients in most cases, whereas most patients in the moderate- and high-risk groups should be hospitalised. However, other factors may need to be considered when making decisions regarding treatment setting
Limitations The majority of studies included in the subgroup analyses of studies where the rule was applied in emergency department or primary care settings included both hospitalised and ambulatory patients, only two studies included only ambulatory patients. There was significant heterogeneity between studies
Comments There was a low risk of bias for most ROBIS domains, although the domain relating to the identification and selection of studies had a high risk of bias, as the authors only searched PubMed and the first 100 articles on Google Scholar (along with reference lists of included articles). The authors’ conclusions appear to be appropriate, although, as acknowledged by the authors, there was significant heterogeneity for the higher-risk subgroups

Critical appraisal – ROBIS tool

Overall risk of bias High (limited search strategy; it is unclear whether all relevant studies were identified)
Applicability as a source of data Acceptable (most studies where the rule was applied in emergency department or primary care settings included both hospitalised and ambulatory patients)

McNally, 20108

Bibliographic reference McNally M, Curtain J, O’Brien KK, et al. Validity of British Thoracic Society guidance (the CRB-65 rule) for predicting the severity of pneumonia in general practice: systematic review and meta-analysis. Br J Gen Pract 2010;60:e423–33

Study details

Study type Systematic review (this review has been updated by Ebell et al., 2019)
Study location Not reported
Study setting Hospitalised patients, emergency department, primary care patients and patients treated as outpatients; the four studies that included primary care patients and patients treated as outpatients are relevant to this review
Study dates PubMed (from 1966 to June 2009), MEDLINE, EMBASE and the Cochrane Library were searched; included studies were published between 2006 and 2009
Sources of funding One of the authors was supported by a RCSI Research Studentship, two authors were supported by the HRB Centre for Primary Care Research
Review question To determine the accuracy of CRB-65 in predicting 30-day mortality and assess how well it performs in community and hospital settings
Inclusion criteria Cohort studies of community-based or hospital-based adults (≥ 16 years) with a primary diagnosis of CAP, in which CRB-65 score was calculated, and death within 30 days was reported, were eligible
Exclusion criteria Not reported
Study design of included studies Eight prospective cohort studies, three retrospective analyses of prospectively collected data, one retrospective cohort study, one longitudinal cohort study and one study reporting pooled data from two RCTs. Three of the four studies relevant to this review were prospective cohort studies and one was a retrospective analysis of a prospective consecutive cohort
Sample size Fourteen included studies, comprising 397,875 patients (range 105–388,406). The 4 studies which included primary care patients and patients treated as outpatients included 1817 community-based patients (range 314–676)
Quality of included studies Quality was assessed following the methodological standard of McGinn for validation studies of clinical prediction rules. In 11 studies, patients were chosen in an unbiased fashion, but in 2 studies they were not and in 1 study it was unclear. Patients represented a wide spectrum of disease in six studies, but not in eight studies. Only 2 studies reported blinded assessment of the rule criteria for all patients; this was unclear in 12 studies. There was an explicit and accurate interpretation of the predictor variables and the actual rule without knowledge of the outcome in all studies. There was 100% follow-up in three studies, but not in seven studies and this was unclear in four studies
Target condition/outcome Thirty-day mortality
Patient characteristics Mean/median age: range 60.4–77.3 (where reported)

Sex: proportion male was not reported

Mortality rate: not reported
Signs, symptoms and EWS CRB-65
Comparator/reference standard The initial derivation study of the CRB-65 rule was used as the predictive model to which all validation studies were compared
Results Among community-based patients, 54.4% of patients (n = 1025) were in the low-risk category and there were 0 mortality events (risk ratio 9.41, 95% CI 1.75 to 50.66; three studies, I 2 = 0%); 43.6% of patients (n = 765) were in the intermediate-risk group, with 1.6% mortality events (risk ratio 4.84, 95% CI 2.61 to 8.96; four studies, I 2 = 0%); 1.9% of patients (n = 27) were in the high-risk group, with 18.5% mortality events (risk ratio 1.58, 95% CI 0.59 to 4.19; three studies, I 2 = 0%)
Authors’ conclusion CRB-65 has not been validated sufficiently in primary care settings and preliminary findings suggest overprediction, so its value as a prognostic indicator in the community remains uncertain
Limitations The authors acknowledge that low event rates make precise estimates about CRB-65 performance less certain
Comments There was a low risk of bias for each ROBIS domain. The conclusions of the review appear to be appropriate

Critical appraisal – ROBIS tool

Overall risk of bias Low
Applicability as a source of data Good

Metlay, 20199

Bibliographic reference Metlay JP, Waterer GW, Long AC, et al. Diagnosis and treatment of adults with community-acquired pneumonia. An official clinical practice guideline of the American Thoracic Society and Infectious Diseases Society of America. Am J Respir Crit Care Med 2019;200:e45–67

Study details

Study type Systematic review
Study location Not reported
Study setting Not reported; however, studies assessed initial site of treatment and requirement for hospitalisation
Study dates PubMed was searched on a monthly basis between 2015 and 2017; included studies were published between 1998 and 2015
Sources of funding Supported by the American Thoracic Society and Infectious Diseases Society of America
Review question Should a clinical prediction rule for prognosis plus clinical judgement vs. clinical judgement alone be used to determine inpatient vs. outpatient treatment location for adults with CAP? This was 1 of 16 questions addressed in the article; it was the only one relevant to the current review
Inclusion criteria Not reported (although focus was on studies that used radiographic criteria for the definition of CAP, US adult patients without immunocompromising conditions such as inherited or acquired immune deficiency or drug-induced neutropenia)
Exclusion criteria Not reported
Study design of included studies Two RCTs and five observational studies
Sample size Seven included studies, the number of included patients was not reported
Quality of included studies The quality of the evidence for each outcome of interest was assessed using the GRADE approach, categorised into four levels: high, moderate, low and very low. For the two RCTs, the level of certainty was low to moderate. For the five observational studies, the quality of evidence was very low for all outcomes
Target condition/outcome Thirty-day mortality, outpatient treatment, subsequent hospitalisation/hospital re-admission, ICU admission, hospital length of stay
Patient characteristics Not reported
Signs, symptoms and EWS PSI and CURB-65
Comparator/reference standard Not applicable
Results Two multicentre, cluster-randomised trials demonstrated that use of the PSI safely increases the proportion of patients who can be treated in the outpatient setting. These trials support the safety of using the PSI to guide the initial site of treatment of patients without worsening mortality or other clinically relevant outcomes. Consistent evidence from three pre–post-intervention studies and one prospective controlled observational study support the effectiveness and safety of using the PSI to guide the initial site of treatment. Evidence for the CURB-65 is less convincing
Authors’ conclusion A validated clinical prediction rule is recommended (in addition to clinical judgement) for determining the need for hospitalisation in adults diagnosed with CAP; preferentially, the PSI (strong recommendation, moderate quality of evidence) over the CURB-65 (conditional recommendation, low quality of evidence)
Limitations This was a poorly reported systematic review, addressing multiple questions including the one of interest here. It is unclear whether all relevant studies were identified and limited details of the included studies were presented
Comments There was a high risk of bias for each ROBIS domain. The authors’ conclusions appear appropriate based on the studies described; however, in view of the considerable risk of bias, they may not be reliable

Critical appraisal – ROBIS tool

Overall risk of bias High (poorly reported review, it is unclear whether all relevant studies were identified and limited details of included studies were presented)
Applicability as a source of data Acceptable (guideline assessing multiple questions, the question on use of a clinical prediction rule plus clinical judgement vs. clinical judgement alone was relevant to this review)

Nannan Panday, 201710

Bibliographic reference Nannan Panday RS, Minderhoud TC, Alam N, Nanayakkara PWB. Prognostic value of early warning scores in the emergency department (ED) and acute medical unit (AMU): a narrative review. Eur J Intern Med 2017;45:20–31

Study details

Study type Systematic review
Study location Included studies were from Denmark, Netherlands, Norway, Germany, Hong Kong, Ireland, Israel, Italy, Singapore, South Africa, South Korea, Sri Lanka, Sweden, Switzerland, Turkey, UK, USA and Vietnam
Study setting Emergency department (ED) and Acute Medical Unit (AMU)
Study dates PubMed and EMBASE were searched from inception to April 2017; included studies were published between 2003 and 2017
Sources of funding Not reported
Review question To provide an overview of studies conducted on the value of EWS on predicting intensive care (ICU) admission and mortality in the ED and AMU
Inclusion criteria Retrospective or prospective observational studies including patients (16 years and older) at the ED or AMU that used the predictive value of EWS as a primary or secondary outcome, and the predictive value of the EWS was studied for mortality, intensive care admission or a composite outcome of these
Exclusion criteria Studies conducted exclusively on patients from disciplines other than internal medicine, where it was unclear when the first assessment of EWS was performed or when the first assessment of EWS was done after the ED or AMU was excluded. Studies where the aim of the study was to determine whether implementation of an EWS led to an improvement in patient mortality and/or ICU admission were also excluded
Study design of included studies Twenty-four prospective studies and 18 retrospective studies were included; 4 studies were relevant to this review, 1 prospective study and 3 retrospective studies
Sample size Forty-two included studies, comprising 166,344 patients (range 125–39,992).

The 4 studies of relevance to this review comprised of 3951 patients (range 246–2361)
Quality of included studies Study quality was assessed with the Quality in Prognostic Studies (QUIPS) tool: 18 studies were found to have a low risk of bias, 22 studies had a moderate risk of bias and 2 studies had a high risk of bias. Inadequate or incomplete reporting resulted in potential bias relating to attrition and possible confounders. Of the four studies of relevance to this review, three had a low risk of bias and one had a moderate risk
Target condition/outcome Mortality, ICU admission or a composite of these
Patient characteristics Where reported, mean/median age ranged from 43 to 75.

For the four studies relevant to our question, median age ranged from 70.5 to 74
Signs, symptoms and EWS A total of 25 different types of EWS were identified. The most frequently used prognostic scores were the Modified Early Warning Score (MEWS), which was applied in 19 studies, and the NEWS, which was used in 12 studies. Nine studies used the Rapid Emergency Medicine Score (REMS) and seven studied Mortality in the Emergency Department Sepsis score (MEDS). Several variations of the EWS were used, with slight modifications such as adding age, adding laboratory values or different cut-off values

For the four studies relevant to our question, the scores assessed were CREWS, CRB-65, CURB-65, NEWS, PSI, SIRS, SEWS and S-NEWS
Comparator/reference standard Not applicable
Results Due to the heterogeneity of the included studies, results were presented in three groups: studies that included the general ED population, studies that only included patients with a possible infection or sepsis and studies that specifically included patients who had either CAP or respiratory distress. The final group is the one of relevance to this review.
Four studies were conducted in the subgroup of patients with CAP or respiratory distress, presenting results as area under the receiver operator characteristic (AUROC)
Thirty-day mortality

  1. One study, 419 ED patients with suspected CAP:

  2. AUROC CURB-65: 0.78

  3. AUROC CRB-65: 0.73

  4. AUROC SIRS: 0.68

  5. AUROC SEWS 0.64

  6. One study, 925 ED patients with suspected CAP:

  7. AUROC NEWS: 0.65

  8. AUROC PSI: 0.80

  9. AUROC CURB-65: 0.72
In-hospital mortality

  1. One study, 2361 ED patients with suspected exacerbation of COPD:

  2. AUROC NEWS: 0.74

  3. AUROC CREWS: 0.62

  4. AUROC S-NEWS: 0.62
Ninety-day mortality

  1. One study, 246 ED patients with respiratory distress:

  2. AUROC NEWS: 0.809
ICU admission

  1. One study, 925 ED patients with suspected CAP:

  2. AUROC NEWS: 0.73

  3. AUROC PSI: 0.64

  4. AUROC CURB-65: 0.64
Authors’ conclusion MEWS and NEWS generally had favourable results in the ED and AMU for all end points. For mortality prediction, NEWS was the most accurate score in those with respiratory distress. ICU admission was best predicted with NEWS. Many studies have been performed on ED and AMU populations using heterogeneous prognostic scores. However, future studies should concentrate on a simple and easy-to-use prognostic score such as NEWS with the aim of introducing this throughout the (pre-hospital and hospital) acute care chain
Limitations Patients’ characteristics (with the exception of age) were not reported and individual study details included in the review were limited, so it is not clear how directly relevant the populations of included studies were
Comments There was a low risk of bias for three ROBIS domains (the other was unclear). The conclusions of the review appear to be appropriate

Critical appraisal – ROBIS tool

Overall risk of bias Low
Applicability as a source of data Acceptable [only a subset of studies was relevant to our review question (patients with suspected CAP or respiratory distress); however one of these studies included patients with suspected exacerbation of COPD]

Smith, 202111

Bibliographic reference American College of Emergency Physicians Clinical Policies Subcommittee (Writing Committee) on Community-Acquired Pneumonia; Smith MD, Fee C, Mace SE, Maughan B, Perkins JC Jr, Kaji A, Wolf SJ. Clinical policy: critical issues in the management of adult patients presenting to the emergency department with community-acquired pneumonia. Ann Emerg Med 2021;77(1):e1–57

Study details

Study type Systematic review
Study location Where reported, included studies were from USA, Spain, Switzerland, Australia, Canada, China, France, Japan, Korea, Turkey, UK and Europe
Study setting Emergency department (ED)
Study dates MEDLINE, MEDLINE InProcess, Scopus, EMBASE, Web of Science and the Cochrane database were searched between January 2007 and 30 August 2017. Included studies were published between 1997 and 2018
Sources of funding American College of Emergency Physicians
Review question The systematic review addressed a number of questions to inform a revision of the American College of Emergency Physicians Clinical Policy for the management of adult patients presenting to the ED with CAP. The question of relevance to this review is:

In the adult ED patient diagnosed with CAP, what clinical decision aids can inform the determination of patient disposition?
Inclusion criteria No inclusion criteria were listed for the review question; the guideline inclusion criteria were adult ED patients with CAP
Exclusion criteria No exclusion criteria were listed for the review question; the guideline exclusion criteria were paediatric or pregnant patients
Study design of included studies Randomised and non-randomised trials, systematic review and meta-analysis, cohort studies (retrospective and prospective, single and multicentre), observational studies
Sample size Thirty-eight studies were included, sample sizes are not reported in the text, but patient numbers are provided in the tables for some studies
Quality of included studies Each article was assessed, graded and assigned a Class of evidence [Class I, Class II, Class III or Class X (for fatal flaws)] using a predetermined process combining the study’s design, methodological quality and applicability to the critical question.

Out of the 38 articles included to answer research question 1, 2 were graded as Class II and 36 were graded as Class III
Target condition/outcome Mortality and ICU admission
Patient characteristics Not reported
Signs, symptoms and EWS Seven clinical decision aids were identified.

Two clinical decision aids to predict mortality in patients with CAP: PSI and CURB-65.

Five clinical decision aids to predict the need for ICU admission: Criteria from the American Thoracic Society (ATS) 2001 CAP guidelines; criteria from the 2007 Infectious Diseases Society of America (IDSA)/ATS 2007 CAP guidelines; Severe CAP (SCAP) aid also known as CURXO-80; SMART-COP scale; and Risk of early admission to the ICU (REA-ICU)
Comparator/reference standard Not applicable
Results The authors summarise the findings of the included studies and provide recommendations based on their findings:
Thirty-day mortality

  1. PSI (seven patient cohorts from five class III studies):

  2. Risk class I: 0–0.4%

  3. Risk class II: 0.4–1.0%

  4. Risk class III: 0.9–3.8%

  5. Risk class IV: 6.0–11.4%

  6. Risk class V: 16.8–38.3%
CURB-65 (five patient cohorts from four class III studies)

  1. Score of 0: 0–0.7%

  2. Score of 1: 0–3%

  3. Score of 2: 5.9–9.2%

  4. Score of 3: 13–21.4%

  5. Score of 4: 17–41.9%

  6. Score of 5: 14–60%

  7. The PSI and CURB-65 have been compared for predicting CAP mortality in ED patients; the PSI appears to have slightly greater predictive value for identifying low-risk patients, but requires data from a greater number of tests and takes longer to complete. Both tools are appropriate for predicting CAP mortality in an ED setting
ICU admission

  1. ICU-specific aids (such as the 2007 IDSA/ATS minor criteria) appear to be superior to PSI and CURB-65 for predicting the requirement for ICU admission. However, no studies have prospectively examined the effectiveness or safety of using these ICU admission decision aids to guide patient management
Author’s conclusion The PSI and CURB-65 are both well-validated clinical decision aids for predicting short-term mortality in CAP patients in an emergency care setting and for identifying low-risk CAP patients for whom outpatient management may be considered. The PSI appears slightly better for identifying low-risk patients, but requires more data, including from some tests not routinely conducted in the emergency department (i.e. arterial blood gases). ICU-specific clinical decision aids (such as the IDSA/ATS minor criteria) should be considered superior to the PSI and CURB-65 for decisions regarding ICU admission
Limitations Patient characteristics were not reported and differences between the studies were not explored. The authors acknowledge the lack of evidence in some areas requiring consensus recommendations
Comments Risk of bias was low or unclear for each ROBIS domain (as insufficient methodological detail is reported in the article). However, the conclusions of the review appear to be appropriate, although it should be noted that some of the authors’ conclusions include consensus recommendations as part of the guideline which are not based on the included evidence

Critical appraisal – ROBIS tool

Overall risk of bias Unclear
Applicability as a source of data Good

Appendix 4 Early warning scores assessed in the systematic reviews

Abbreviation/EWS name Data required Range
Centor

Cough, Exudate, Nodes Temperature, young OR old modifier		 History of fever, tonsillar exudate, anterior cervical lymphadenopathy, absence of cough, age −1 to 5
CRB-65

Confusion, Respiratory rate, Blood pressure, Age ≥ 65		 Mental status, respiratory rate, blood pressure, age ≥ 65 0–4
CREWS

Chronic Respiratory Early Warning Score		 Pulse, respiratory rate, temperature, blood pressure, SpO2, oxygen supplemental, AVPU 0–20
CURB-65

Confusion, Urea, Respiratory Rate, Blood pressure, Age ≥ 65		 Mental status, urea, respiratory rate, blood pressure, age ≥ 65 0–5
IDSA/ATS 2007

Infectious Diseases Society of America/American Thoracic Society 2007 guidelines		 Minor criteria include: respiratory rate, PaO2/FiO2 ratio, multilobar infiltrates, confusion/disorientation, uraemia, leucopenia, thrombocytopenia, hypothermia, hypotension.

Major criteria include: septic shock with need for vasopressors, respiratory failure requiring mechanical ventilation		 Either one major criterion or three or more of the minor criteria
MEDS

Mortality in Emergency Department Sepsis		 Functional status, vital parameters, lab values 0–27
MEWS

Modified Early Warning Score		 Pulse, respiratory rate, temperature, urinary output, blood pressure, AVPU 0–17
NEWS

National Early Warning Score		 Pulse, respiratory rate, temperature, blood pressure, SpO2, oxygen supplemental, AVPU 0–20
PSI

Pneumonia Severity Index		 Age, type of residence, laboratory values, vital parameters 0–395
REA-ICU

Risk of Early Admission to the ICU		 Male gender, age < 80, comorbid conditions, respiratory rate, heart rate, multilobar infiltrate or pleural effusion, white blood cell count, hypoxaemia, blood urea nitrogen, arterial pH, sodium 0–17
REMS

Rapid Emergency Medicine Score		 Age, blood pressure, heart rate, respiratory rate, SpO2, GCS 0–26
SCAP

Severe CAP

Also known as CURXO-80

Confusion, Urea, Respiratory rate, X-ray multilobar bilateral, Oxygenation, age ≥ 80		 Minor criteria include: confusion, urea, respiratory rate, multilobar involvement, oxygenation, age ≥ 80

Major criteria include: arterial pH, systolic blood pressure		 Either one major criterion or two or more minor criteria
SEWS

Standardised Early Warning Score		 Pulse, respiratory rate, temperature, blood pressure, SpO2, AVPU 0–18
SIRS

Systemic inflammatory response syndrome		 Vital parameters + lab values 0–4
SMART-COP

Systolic blood pressure, Multilobar chest radiography involvement, Albumin level, Respiratory rate, Tachycardia, Confusion, Oxygenation and arterial pH		 Blood pressure, multilobar involvement, albumin level, respiratory rate, tachycardia, confusion, oxygenation, arterial pH 0–11
S-NEWS

Salford National Early Warning Score		 Pulse, respiratory rate, temperature, blood pressure, SpO2, oxygen supplemental, AVPU 0–20

AVPU, Alert, Verbally responsive, Painfully responsive, Unresponsive; GCS, Glasgow Coma Scale; PaO2/FiO2 ratio, ratio of arterial oxygen partial pressure to fractional inspired oxygen; SpO2, oxygen saturation.

Note

None of the reviews assessed NEWS2; NEWS was updated to NEWS2 in December 2017.

Appendix 5 Quality assessment of the economic study

Study identification

Little P, Hobbs FD, Moore M, Mant D, Williamson I, McNulty C, et al. PRImary care Streptococcal Management (PRISM) study: in vitro study, diagnostic cohorts and a pragmatic adaptive randomised controlled trial with nested qualitative study and cost-effectiveness study. Health Technol Assess 2014;18(6):1–101
Category Rating Comments
Applicability
1.1 Is the study population appropriate for the review question? No Not directly applicable to the review question; however, this study met the inclusion criteria
1.2 Are the interventions appropriate for the review question? Partly Clinical symptom scores are assessed
1.3 Is the system in which the study was conducted sufficiently similar to the current UK context? Yes
1.4 Is the perspective for costs appropriate for the review question? Yes NHS and PSS perspective
1.5 Is the perspective for outcomes appropriate for the review question? Yes
1.6 Are all future costs and outcomes discounted appropriately? N/A Due to short time horizon. The analysis covered a 28-day follow-up period
1.7 Are QALYs, derived using NICE’s preferred methods, or an appropriate social care-related equivalent used as an outcome? If not, describe rationale and outcomes used in line with analytical perspectives taken (item 1.5 above). Yes QALYs were derived from EQ-5D scores
1.8 OVERALL JUDGEMENT PARTIALLY APPLICABLE
Other comments:
Study limitations
2.1 Does the model structure adequately reflect the nature of the topic under evaluation? N/A Trial-based analysis
2.2 Is the time horizon sufficiently long to reflect all important differences in costs and outcomes? Yes
2.3 Are all important and relevant outcomes included? Yes
2.4 Are the estimates of baseline outcomes from the best available source? Yes
2.5 Are the estimates of relative intervention effects from the best available source? Yes
2.6 Are all important and relevant costs included? Yes
2.7 Are the estimates of resource use from the best available source? Yes
2.8 Are the unit costs of resources from the best available source? Yes
2.9 Is an appropriate incremental analysis presented or can it be calculated from the data? Yes
2.10 Are all important parameters whose values are uncertain subjected to appropriate sensitivity analysis? Yes
2.11 Has no potential financial conflict of interest been declared? Yes
2.12 OVERALL ASSESSMENT MINOR LIMITATIONS

N/A, not applicable.

Appendix 6 Economic evaluation methods and results of the included economic study

Study Economic evaluation Perspective Time horizon Costs and resource use Utility measure Effects (QALYs) ICER Uncertainty Author’s conclusion
Little et al. 2014 13

  • Cost–utility analysis

  • Cost-effectiveness analysis

 NHS and PSS 1 month

Total costs at 14 and 28 days (95% CI):

  • DP: £49.70 (43.30 to 56.00)

  • FeverPAIN: £45.90 (41.50 to 50.20)

  • RADT: £48.50 (45.00 to 52.00)

 EQ-5D completed at baseline and at 14 days.

The last EQ-5D score obtained was carried forward to estimate QALYs gained for 28 days		 Cost–utility analysis (outcome measure: QALYs)

Cost–utility analysis

  • DP is dominated (more costly and less clinically effective) by FeverPAIN and RADT.

  • Compared to FeverPain, RADT generates an ICER of £74,286 and £24,528 at 14 and 28 days, respectively.

Cost-effectiveness acceptability curves indicated considerable uncertainty, particularly around the QALY estimate.

At a threshold of £30,000 per QALY, the probabilities that delayed prescribing, clinical score and RADT are the most cost-effective option were 25%, 40% and 35%, respectively, for the 14-day period, and 28%, 38% and 35%, respectively, for the 28-day period.		 Targeting antibiotics for acute sore throat based on a clinical score demonstrated a more efficient utilisation of healthcare resources compared to the other two groups, based on changes in symptoms.
14-day period (95% CI):

  • DP: 0.0057 (0.0044 to 0.007)

  • FeverPAIN: 0.0058 (0.0045 to 0.0071)

  • RADT: 0.00584 (0.0046 to 0.0071)
Costs included: GP/NP visit, testing costs, prescribing fees and community care contacts from illness or treatment complications.

No discounting applied due to the short time horizon.		 28-day period (95% CI):

  • DP: 0.0171 (0.0131 to 0.0211)

  • FeverPAIN: 0.01741 (0.0135 to 0.0213)

  • RADT: 0.01752 (0.0138 to 0.0212

 Cost-effectiveness analysis

  • DP is dominated (more costly and less clinically effective) by FeverPAIN and RADT.

  • RADT is dominated (more costly and less clinically effective) by FeverPAIN
Cost-effectiveness analysis (outcome measure: symptom score)

  • DP: 3.15 (2.93 to 3.37)

  • FeverPAIN: 2.83 (2.61 to 3.05)

  • RADT: 2.84 (2.62 to 3.07)

DP, delayed antibiotic prescribing.

Acute respiratory infections, such as cold and flu, are common and can be caused by viruses or bacteria. People with symptoms of acute respiratory infection often go to their general practitioner, who may advise them to stay at home (with or without antibiotics or antivirals) or might refer them to hospital if the infection is serious. Doctors assess the patient’s symptoms or may use a tool called an ‘early warning score’ to judge whether the infection is serious.

A systematic review is a research method where all relevant studies assessing a specific question are found and summarised. We aimed to summarise all systematic reviews and cost-effectiveness studies that assessed signs, symptoms and ‘early warning scores’ in adults with suspected acute respiratory infections in the community (i.e. not hospitalised patients). We found nine systematic reviews and one cost-effectiveness study. Several different early warning scores for acute respiratory infection have been assessed in systematic reviews. Seven of the reviews assessed early warning scores in patients with community-acquired pneumonia. Good-quality reviews concluded that further research is needed to see how useful the ‘CRB-65’ and ‘Pneumonia Severity Index’ early warning scores are for assessing pneumonia severity in the community. Another good-quality review concluded that the ‘National Early Warning Score’ early warning score appears to be useful in an emergency department setting. A good-quality review found that individual symptoms are not very reliable for diagnosing pharyngitis caused by streptococcal bacteria in patients with sore throat; the review also found that the ‘Centor score’ can help doctors decide whether to prescribe antibiotics for pharyngitis.

The cost-effectiveness study assessed clinical scores and rapid antigen detection tests (which test for substances that increase in our blood when we have certain infections) in patients with sore throat, and found that clinical scores may be cost-effective compared to delaying prescribing antibiotics.

We are currently working on improvements to this feature. Please check back soon for updates