Rapid antigen detection and molecular tests for group A streptococcal infections for acute sore throat: systematic reviews and economic evaluation

Patient selection domain

Two further signalling questions were added to this domain. The first was ‘were selection criteria clearly described?’. It is important that the correct patient groups were included in the studies. Patients aged < 5 years follow a different NICE clinical pathway27 because they are more likely to present with a sore throat and less likely to be able to articulate their symptoms, and it is less likely that a throat swab can be obtained. Likewise, a clinical score (such as Centor or FeverPAIN) should be reported, with patients included only if they have a score of > 3 points on Centor or > 4 points on FeverPAIN. Those with lower scores may be systematically different and, therefore, test accuracy may also differ, introducing bias. Including patients aged < 5 years and with a low clinical score also raises applicability concerns.

The second signalling question that was added was ‘were patients seen in an ambulatory care setting?’. Patients seen as inpatients may vary in severity and have comorbidities affecting their diagnosis.

Index test domain

Two questions were added within this domain. The first was ‘was a separate swab undertaken for the index test?’. This question was added as manufacturers’ specifications require separate swabs to be taken for index and reference standard tests. Using one swab for multiple purposes may reduce the quantity of the sample for testing and, thus, affect the accuracy of the test. The second question was ‘is the test reading objective?’. Some of the tests require a subjective reading of whether or not a line, indicating a positive result, has appeared. Owing to this, there is always a high level of bias in any rapid test that requires a determination of the result by the human reader. Tests with automated readings have been shown to have improved specificity and reduce operator errors, especially in unclear results. 28

Comparator domain

One additional signalling question was added in this domain: ‘was a separate swab taken for throat culture testing?’. Using one swab for multiple purposes may reduce the quantity of the sample for testing and, thus, affect the accuracy of the test. Under this domain, the directions for taking a throat culture specimen were clarified based on the PHE guidelines on UK Standards for Microbiology Investigations. 29

Flow and timing

Two further signalling questions were added to the flow and timing domain. The first was ‘were both index test(s) and reference standard (and comparator where included) all carried out at the same appointment?’. The swabs for a rapid test and culture should be obtained at the same appointment. The levels of strep A are likely to vary by day, so taking a later sample could introduce systematic bias.

The additional signalling question was ‘were both index test(s) and reference standard (and comparator where included) all carried out prior to commencement of antibiotics?’. Patients should not have been treated with antibiotics prior to testing, as antibiotics are likely to have reduced the amount of strep A present.

Quality appraisal strategy for studies of prescribing behaviour and clinical outcomes

Quality appraisal for studies of prescribing behaviour and clinical outcomes used two different tools: the Cochrane risk-of-bias tool for randomised controlled trials (RCTs)30 and the Joanna Briggs Institute (JBI) Critical Appraisal Checklist for analytical cross-sectional studies. 31 Methodological quality was assessed by a single reviewer and findings were checked by a second reviewer. Disagreements were resolved by consensus or use of a third reviewer.

Study characteristics

Characteristics of the 38 studies included in the clinical effectiveness review are described in Figures 3 and 4 and Table 6. Of the 29 studies (full texts and abstracts)6,20,23,24,34–58 identified by the search, 26 of the studies reported test accuracy data. 20,23,24,34–52,54,57,58,63 Three of the identified studies6,53,55 reported only other outcomes (such as antibiotic-prescribing rates) and did not report test accuracy. In addition, five studies were sent by manufacturers in response to a request for information by NICE and four FDA documents were retrieved. 59–62

FIGURE 3.

Diagram of studies by test type, setting and population for included test accuracy studies with extractable 2 × 2 data. Note that lines between tests indicate head-to-head (direct) comparisons. M, molecular test; NR, not reported; R, rapid test.

FIGURE 4.

Diagram of studies by test type, setting and population for included test accuracy studies with extractable 2 × 2 data. M, molecular test; NR, not reported; R, rapid test.

The tests, their settings, the populations they cover and the head-to-head studies are illustrated in Figures 3 and 4.

Population

The 38 included studies comprised ≈ 14,000 symptomatic participants. Prevalence of strep A ranged from 15% to 49%, with no clear demographic or clinical patterns accounting for this variation. 39,56 Similarly, prevalence estimates of strep A were no more or less likely to be higher in secondary or primary care settings. The study population comprised adults and children; however, the exact proportions are unknown as they were not reported in about half of the included studies. In most of the included studies, participants aged < 18 years were identified as children. In fact, only two studies met the age criterion for children (ages 5 to 14 years) as defined in the protocol and scope. 50,51 Hence, studies that included children aged < 5 years as well as ≥ 5 years were included in the present review. More so, only two studies met the age criterion for adults (age ≥ 15 years) as defined in the protocol and scope and, therefore, the findings of the review may be applicable to only a mixed population. 39,52

All 38 studies included patients with a sore throat; however, other clinical characteristics were insufficiently reported across most of the included studies. For instance, sore throat clinical scores (e.g. Centor/McIsaac/FeverPAIN scores) were reported in 16 studies,6,23,35–39,44–46,50,52,53,55,57 of which two exclusively included patients with high clinical scores (i.e. Centor ≥ 3 points, FeverPAIN ≥ 4 points). 6,53 Both of these studies were on prescribing behaviours. However, there were two test accuracy studies that included patients with lower clinical scores (i.e. Centor scores of < 3 points) but reported test accuracy results separately by Centor score. 39,45

Recent antibiotic use prior to enrolment was considered in eight included studies, and patients without any recent use of antibiotics prior to recruitment were eligible for inclusion in these studies. 24,35,36,42,45,47,48,50

Index tests

There were more studies evaluating RADTs (76%, 29/38) than were evaluating molecular tests (18%, 7/38) or studies comparing both rapid tests and molecular tests (5%, 2/38). For instance, the Alere™ TestPack +Plus Strep A (Abbott Laboratories) was the most common antigen detection test, which was evaluated in 13 studies (excluding unpublished studies conducted by the manufacturers). 6,23,38–43,46–48,51,58 Conversely, the only molecular test evaluated in a peer-reviewed journal article was the PCR-based cobas Liat Strep A Assay (Roche Diagnostics). 24

As shown in Table 6, there were four studies providing head-to-head comparisons of index tests: BD Veritor System (Becton Dickinson) and QuikRead Go (Orion Diagnostica);34 Alere i Strep A (Abbott Laboratories) and BD Veritor System (Becton Dickinson);20 Alere i Strep A (Abbott Laboratories) and Sofia Strep A fluorescent immunoassay (FIA) (Quidel);23 and Alere i Strep A (Abbott Laboratories) and OSOM Strep A. 54 Essentially, each index test was compared with throat culture as the reference standard in order to obtain test accuracy.

The search strategy revealed test accuracy studies of OSOM Ultra Strep A (Sanofi Genzyme and Sekisui Diagnostics). 64,65 However, these studies were subsequently excluded because the EAG could not confirm whether it is the same as the OSOM Strep A test (Genzyme and Sekisui Diagnostics), which is listed among the scoped rapid tests. Similarly, it was unclear if Sofia Strep A+ Plus FIA (Quidel)66 and OSOM Strep A (Sekisui Diagnostics)67 were identical to Sofia Strep A FIA (Quidel) and OSOM Strep A (Sekisui Diagnostics), respectively; hence, studies of the former were excluded.

Comparator and reference standard

Index tests were compared with Centor, McIsaac or FeverPAIN scoring tools in 12 studies. 6,35,36,38,45,46,50,52,53,55,57,68 However, only six of these studies directly compared test accuracy between clinical scoring tools and point-of-care tests. 39,44–46,52 Only two reported test accuracy in patients with high clinical scores (Centor ≥ 3 points, FeverPAIN ≥ 4 points). 39,45

The culture medium used for the reference standard (blood agar or streptococcal selective agar) was reported in all but five studies. 24,40,56,57 Neither the manufacturers’ submissions (submitted directly to NICE in response to a request for information) nor the FDA studies provided information on index tests compared with clinical scoring tools.

Outcomes

Thirty-eight studies were included across all outcomes. Twenty-six published articles (full texts and abstracts) reported test accuracy data (of which seven also reported on antibiotic-prescribing rates and five reported on test failure rate); there were an additional five submissions from manufacturers and four FDA documents.

Five studies had insufficient data to construct 2 × 2 contingency tables to ascertain the accuracy of index tests with microbiological throat culture as the reference standard. 6,35,47,53,55 These studies were further excluded from the assessment of test accuracy in Point-of-care/index tests. An attempt to verify at least some of the discrepant results between rapid tests and microbiological culture was undertaken in only five studies. 20,23,24,37,43 Antibiotic-prescribing behaviour was reported in 12 studies. 6,20,35,36,39,40,45,46,50,52,53,55 None of the other outcomes in the scope or protocol was reported in any of the included studies.

Setting

Participants in the included studies were recruited from GP/primary care clinics/family practices,6,40,41,43–47 community pharmacies,53 paediatric clinics,42,49,54,56 paediatric emergency departments,23,48,57 hospital outpatient departments20,51 and emergency departments. 35,50 There were two multicentre studies with mixed populations from primary and secondary care settings: Cohen et al. 37 sampled patients from the emergency department (secondary care) and urgent care clinics (primary care); and Wang et al. 24 sampled patients from paediatric clinics (secondary care) and family practices (primary care).

Only one unpublished study supplied by the manufacturers confirmed the study setting (Orion Diagnostica, primary care). The remaining unpublished studies conducted by manufacturers may have included mixed populations from primary and secondary care settings; however, this is purely speculative as study settings were not reported in these studies. However, these studies provide no evidence to suggest any recruitment of inpatients.

Study design

The 26 published studies on test accuracy comprised one RCT45 and 25 cohort studies. 20,23,24,34–44,46,48–52,55–58,69 It was unclear what study design had been undertaken in any of the unpublished studies provided by the manufacturers or the FDA.

The 12 studies that provided data on antibiotic-prescribing rates comprised three RCTs,6,45,55 one before-and-after cohort study20 and eight one-armed cohort studies. 35,36,39,40,46,50,52,53

Risk of bias for test accuracy studies

In general, the methodological and reporting quality of the included studies was poor, with risk of bias considered to be high in two or more domains for 13 studies (50%). 20,34–36,38,40–43,46,49–51 No study was considered to be at a low risk of bias in all four domains.

In 65.4% of studies (17/26),20,23,24,35,37,42,43,46,47,49,51,52,54,56–58,70 it was not clear whether patients were consecutively included or a convenience sample had been chosen, and only 15.4% (4/26 studies)39,44,45,48 were rated as having a low risk of bias in the patient selection domain (domain 1: patient selection). The selection process in the remaining 19.2% of studies (5/26)34,36,41,50 was rated as being at a high risk of bias, with studies clearly reporting convenience samples, having case–control designs or having made inappropriate exclusions from the eligible screening population.

The key risks of bias were surrounding how the index test was undertaken (22/30 domains were rated as being at high risk, 73.3%, 22/26 studies). 20,23,35,36,38–46,48–51,55–58,69 Although all of the included studies were on predeveloped tests that had in-built thresholds, in many cases use of the index test required a subjective reading by a clinician (domain 2: index tests). There were further concerns that studies often used the same swab intended for the index test to first streak the agar for biological culture, rather than taking an additional swab sample. Using one swab for multiple purposes may reduce the amount of the sample and underestimate the accuracy of the test.

Unclear or incomplete reporting was common in the reference standard domain (domain 3: reference standard). In all studies, time taken to process the biological culture exceeded that of the rapid test, with biological cultures generally reported 48 hours following sample collection. However, many studies did not state that laboratory staff were blinded to the results of the index test or reference standard (domain 3: reference standard, 13/27 studies, 48.1%). 24,34,35,39,40,44,45,47,48,51,52,56,57 There was a high risk of bias in 22.2% of the studies (6/27)20,38,41–43,49 because the methods of biological culture testing did not match current UK guidelines. 29

The flow of patients through the studies was rated as being at a high risk of bias in 31% of studies (8/26, domain 4: flow and timing). 34–36,38,40,41,46,51 The majority of these (62.5%, 5/8 studies)34–36,41,46,51 had incomplete testing and made exclusions from the analysis. However, in two of these studies only some patients received the reference standard (partial verification bias). In one study,38 only patients with negative rapid test results received the reference standard; in the other,40 only those with positive rapid test results were given the reference standard. The use of antibiotics was a further concern, with one study directly reporting 61 patients taking antibiotics at the time of testing,34 and 90% (9/10) of unclear ratings were linked to prior/current antibiotic use not being reported. 20,37,42,45,49,52,54,56–58

Applicability of study findings for test accuracy studies

The applicability of study findings was assessed with regard to three domains: patient selection, index test (rapid or molecular test) and reference standard (biological culture). There were significant concerns regarding the applicability of the studies to UK practice for patient selection in 22 of the 26 studies (85%, domain 1: patient selection). 20,23,24,34–37,40–44,48–52,55–58,69 In the UK, the test would be given only following an assessment using a clinical scoring tool, such as Centor or FeverPAIN. The rapid test would be given only to people with Centor scores of ≥ 3 points and FeverPAIN scores of ≥ 4 points. In all 22 studies, either a clinical scoring tool was not used or, if used, patients were included with scores lower than UK cut-off points and test accuracy data were not reported separately by score. In addition, 17 of the 22 studies (77%)20,23,24,35,37,40,42,48–52,55–58,69 included children aged < 5 years. Children aged < 5 years follow a different clinical pathway owing to differences in the presentation of symptoms and difficulties around communication and sample collection. 27 Concerns regarding the applicability of the index test were rated as being low for the majority of the studies (21/30 domains, 70%, 18 studies),20,23,24,34,36,37,39,41–45,48–52,69 with studies reporting that the tests were carried out in accordance with manufacturer’s guidelines. The eight remaining studies35,38,40,46,54,56–58 were rated as being unclear as this was not specified (domain 2: index test). Only four studies (4/27, 14.8%)38,40,42,43 were rated as having high concern for the applicability with respect to the reference standard (owing to deviations from UK guidelines on the undertaking of appropriate culture methods with respect to agar type, incubation period or atmosphere; domain 3: reference standard).

Randomised controlled trials

Risk of bias of the included trials is shown in Figure 6 and Table 8. The domains regarding blinding were removed, as we were interested in test–treat trials measuring prescribing decisions with and without rapid tests. Therefore, clinicians could not be blinded to test results, and we considered blinding to which exact test was used to be unnecessary in this context. In general, the methodological quality of the RCTs was rated as being fair, with all studies having at least one domain rated as unclear. There was unclear risk of bias in four domains across the three studies (random sequence generation, allocation concealment, incomplete outcome data and selective outcome reporting). This was owing to insufficient information presented on which to make an assessment. The remaining applicable domains were judged to be at a low risk of bias.

FIGURE 6.

Concerns regarding the risk of bias of included RCTs.

Cohort studies

Risk of bias in the included cohort studies is shown in Figure 7 and Table 9. No study was rated as having had high methodological quality across all areas. There was low methodological quality regarding criteria for inclusion in 83% of studies (five out of six) and details regarding the study subjects in 33% of studies two out of six). 20,35,40,50,52 These studies reported the details of the patients, but provided no information on the details of those who are making the prescribing decisions. The outcome of interest in these studies was prescribing behaviour. The measurement of prescribing behaviour considered to be valid and reliable was recording in medical records; only 33% of the studies clearly reported this. 20,36 A confounder in the studies was current antibiotic use; 33% (two out of six) of studies did not clearly specify current or recent antibiotic use as an excluding factor.

FIGURE 7.

Methodological quality of included analytical cross-sectional studies. NA, not applicable.

Accuracy of clinical scoring tools with culture as the reference standard

Accuracy statistics for Centor39,44,45,52 and McIsaac scores46,52,57 with microbiological culture as the reference standard are presented in Table 10. The results show wide variations in the test accuracy of sore throat clinical scoring tools. Specificity point estimates were reported between 0.172 and 0.648, and sensitivity point estimates were reported between 0.735 and 0.972. This suggests that these tools might be better at identifying people who do have Streptococcus than they are at identifying people who do not.

Rosenberg et al. 50 and Johansson et al. 40 also reported accuracy statistics for sore throat symptoms with culture as the reference standard. However, these studies provided insufficient data to construct 2 × 2 contingency tables using the recommended clinical scoring threshold (see Data extraction strategy). The use of different clinical scoring tools, age selection criteria, clinical score inclusion criteria and settings across the seven contributing studies precluded any pooling.

Accuracy of clinical scoring tools split by age group

Two46,52 of the six studies included a mixed population of adults and children. In the study by McIsaac et al. ,46 a threshold of > 2 points (Modified Centor/McIsaac score) produced a sensitivity estimate of 0.884 (95% CI 0.820 to 0.928) and a specificity estimate of 0.234 (95% CI 0.188 to 0.287) in children aged 3–17 years, and a sensitivity estimate of 0.767 (95% CI 0.651 to 0.858) and a specificity estimate of 0.439 (95% CI 0.378 to 0.501) in adults aged ≥ 18 years.

In the study by Stefaniuk et al. ,52 a threshold of > 2 points (Modified Centor/McIsaac score) produced a sensitivity estimate of 1.00 (95% CI 0.80 to 1.00) and a specificity estimate of 0.083 (95% CI 0.015 to 0.285) in children aged 1–14 years, and a sensitivity estimate of 0.739 (95% CI 0.513 to 0.889) and a specificity estimate of 0.414 (95% CI 0.241 to 0.609) in participants aged ≥ 15 years. As previously discussed (see Population), this overlap across age groups potentially limits subgroup analysis. However, none of the other six studies included patients aged < 14 years.

Accuracy of clinical scoring tools split by primary/secondary care setting

Patients were recruited from primary care settings in five39,44–46,52 of the six studies. Details of the primary care setting studies are outlined in Table 10. In brief, these studies provided point estimates of sensitivity of 0.74–0.86 and of specificity of 0.25–0.65. The single study from a secondary care setting57 reported a higher point estimate for sensitivity (0.972, 95% CI 0.893 to 0.995) and a lower point estimate for specificity (0.172, 95% CI 0.112 to 0.253), albeit with overlapping CIs with some of the primary care setting studies, than the other five studies. This may have been a result of the setting or other sources of heterogeneity between studies.

Accuracy of clinical scoring tools using polymerase chain reaction to resolve discordant cases

No analysis of discordant results between sore throat clinical scores and microbiological culture was undertaken in any of the included studies.

Accuracy of point-of-care tests with culture as the reference standard

The systematic review identified 35 pieces of literature that provided evidence comparing the performance of 18 of the named index tests with culture. These were 23 peer-reviewed papers, three abstracts, five manufacturer responses (submitted directly to NICE in response to a request for information) and four FDA reports. Two studies reported results that were inconsistent, which prevented the construction of a reliable 2 × 2 contingency table, and were excluded during the data extraction. 35,47 A summary of the final 33 pieces of literature can be found in Table 11. The sources provided by the manufacturers were not peer reviewed, and neither were three abstracts. The sources identified from FDA reports received some scrutiny from the FDA. The remaining 21 studies were published in peer-reviewed journals. All sensitivity and specificity estimates are presented alongside their 95% CI. Meta-analyses were performed where appropriate, a summary of which can be found in Figure 8.

FIGURE 8.

Summary of meta analyses carried out on tests with multiple studies excluding manufacturer responses and FDA reports.

Summary

Figures 9 and 10 present the sensitivity and specificity for all studies that had complete 2 × 2 data. Data were available for only 18 tests, and just seven tests were used in more than one independent study. Ignoring manufacturer and FDA sources of data, this reduces to 10 tests with published data and five tests with more than one independent study.

FIGURE 9.

Study-level data for the studies included in the meta-analysis of test accuracy: sensitivity.

FIGURE 10.

Study-level data for the studies included in the meta-analysis of test accuracy: specificity.

Note that where studies provided performance data by subgroups, these were incorporated into the relevant analyses when producing estimates to feed into the cost-effectiveness modelling. It is clear that there is a large degree of heterogeneity between the studies, and it is difficult to attribute any observed differences in test performance to the tests themselves. The CIs in Figures 8–10 may differ slightly to those in Table 11, owing to differences in their method of calculation.

It is apparent that the data sourced from the manufacturer responses (submitted directly to NICE in response to a request for information) and the FDA submissions consistently provided higher estimates of sensitivity and specificity than the peer-reviewed studies. This supports the view that the manufacturer data may be judged as being at high risk of bias, and any cost-effectiveness analyses incorporating them may be unreliable.

Head-to-head (direct) comparison between tests

Initially, we sought to identify whether or not there was evidence to support the hypothesis that the tests might have different levels of accuracy. Owing to the large degree of interstudy variability, the most informative studies were those that conducted multiple tests on the same patient population, of which there were four.

Azrad et al. 34 compared both the BD Veritor System (Becton Dickinson) and the QuikRead Go Strep A Kit (Orion Diagnostica) with culture for 100 patients. The BD Veritor System had a sensitivity of 0.80 (95% CI 0.59 to 0.92) and a specificity of 0.79 (95% CI 0.67 to 0.87). The QuikRead Go test had an identical sensitivity, of 0.80 (95% CI 0.59 to 0.92), and a slightly lower point estimate for specificity, of 0.73, with overlapping CIs (95% CI 0.62 to 0.83).

Berry et al. 20 compared both the BD Veritor System and the Alere i Strep A tests with culture. The tests performed differently, with the BD Veritor System having a sensitivity of 0.76 (95% CI 0.60 to 0.87) and a specificity of 0.94 (95% CI 0.89 to 0.97) and Alere i Strep A having a sensitivity of 1.00 (95% CI 0.90 to 1.00) and a specificity of 0.91 (95% CI 0.86 to 0.95).

Lacroix et al. 23 investigated both the Alere TestPack Plus and the Sofia Strep A FIA tests. Again, the tests performed differently, with the Alere TestPack Plus having lower detection rates, with a sensitivity of 0.76 (95% CI 0.71 to 0.80) and a specificity of 0.97 (95% CI 0.95 to 0.98). Meanwhile, the Sofia Strep A FIA had a sensitivity of 0.85 (95% CI 0.81 to 0.89) and a specificity of 0.95 (95% CI 0.93 to 0.97).

Finally, Weinzierl et al. 54 assessed the Alere i Strep A and the OSOM Strep A Strip tests. The OSOM Strep A Strip had a sensitivity of 0.89 (95% CI 0.77 to 0.95) and a specificity of 0.91 (95% CI 0.83 to 0.96), whereas the Alere i Strep A test had a sensitivity of 0.98 (95% CI 0.90 to 1.00) and a specificity of 1.00 (95% CI 0.95 to 1.00).

Conclusion

There is insufficient evidence to conduct a meaningful comparison of the rapid tests or to establish any reliable hierarchy of test performance. Although some tests may perform similarly, the existing evidence does not allow identification of any clear groups of tests, and it is likely that there is some variation in accuracy of the 21 tests. There is considerable heterogeneity, potentially caused by the differences in study design and population.

Accuracy of point-of-care tests in the population at high risk of group A Streptococcus infection as defined by sore throat clinical scores

The primary population of interest in this review is patients with high clinical scores (Centor score of ≥ 3 points, FeverPAIN score of ≥ 4 points). We report test accuracy data in that population, and for other thresholds of clinical measuring tools, such as Centor or McIsaac, as were reported by published studies. The majority of studies either did not place or did not report placing a restriction of the clinical scoring tool on their patient populations.

Eight studies present results based on some restriction of Centor or McIsaac (either ≥ 1 point or 2 or 3 points), which informed test accuracy data for four tests. A summary of evidence is provided in Table 12.

Only two studies presented data for populations that matched the NICE scope, that is having either a Centor or McIsaac score of ≥ 3 points or a FeverPAIN score of ≥ 4 points. 39,45 We dichotomised the data from these studies into (1) patients meeting the scope based on throat score and (2) patients not meeting the scope.

Humair et al. 39 investigated the Alere TestPack Plus test in adults presenting in a primary care setting, with a Centor score of ≥ 2 points. In the Centor score = 2 points and Centor score of > 2 points subgroups, the sensitivities were 0.80 (95% CI 0.63 to 0.92) and 0.95 (95% CI 0.89 to 0.98), and the specificities were 0.96 (95% CI 0.91 to 0.99) and 0.94 (95% CI 0.88 to 0.98), respectively. The subgroups had 148 and 224 patients, respectively.

Llor et al. 45 investigated adult patients in a primary care setting with a Centor score of ≥ 1 point when assessing the performance of the OSOM Strep A Strip. In the population with a Centor score of 1 or 2 points, consisting of 160 patients, the OSOM Strep A Strip had a sensitivity of 0.85 (95% CI 0.55 to 0.98) and a specificity of 0.93 (95% CI 0.87 to 0.96). In the population with a Centor score of > 2 points, with 116 patients, the test had a sensitivity of 0.92 (95% CI 0.76 to 0.98) and a specificity of 0.96 (95% CI 0.89 to 0.99).

The remaining data for studies that restricted their populations by throat score are presented in the following sections.

Accuracy of point-of-care tests split by age group

We sought to identify whether or not there was evidence to support the hypothesis that the tests might have different performance characteristics based on the age group on which the test is being used. No studies were categorised into the age groups as detailed in the NICE scope, and so we classified them into child and adult populations where possible, or a combination of children and adults. No studies presented results specific to a ≥ 60-year-old population, although patients in this category may have been included within an ‘adult’ population. Seven studies concentrated on exclusively adult populations, providing accuracy data for two tests. 36,38,39,41,43–45 Ten studies looked exclusively at children, providing data for nine tests. 20,23,24,42,48,49,51,54,56,57 Three studies considered both adults and children, and presented accuracy data for them separately, allowing a within-trial comparison to be made. 37,46,52 Each of these three studies investigated a different test.

Cohen et al. 37 examined both adults and children when investigating the accuracy of the Alere i Strep A test. In children, the test had a sensitivity of 0.96 (95% CI 0.91 to 0.99) and a specificity of 0.93 (95% CI 0.89 to 0.96). In adults, the sensitivity was 0.95 (95% CI 0.74 to 1.00) and the specificity was 0.97 (95% CI 0.92 to 0.99).

McIsaac et al. 46 examined the Alere TestPack Plus test in child and adult populations, presenting the results by age category. In children, the sensitivity was 0.86 (95% CI 0.79 to 0.91) and the specificity was 0.99 (95% CI 0.97 to 1.00). In adults, the sensitivity was 0.77 (95% CI 0.65 to 0.86) and the specificity was 0.99 (95% CI 0.97 to 1.00).

Stefaniuk et al. 52 used the QuikRead Go Strep A Kit test in both adults and children. In children, a sensitivity of 0.80 (95% CI 0.56 to 0.94) and a specificity of 0.91 (95% CI 0.72 to 0.99) were estimated. In adults, the test sensitivity was 1.00 (95% CI 0.86 to 0.95) and specificity was 0.79 (95% CI 0.60 to 0.92).

Further age-specific results are presented below and in Table 13.

Accuracy of point-of-care tests split by primary/secondary care setting

We sought to identify whether or not there was evidence to support the hypothesis that the tests might have different performance characteristics based on the setting in which the test is being used. No studies provided a breakdown of results comparing test accuracy between primary and secondary settings. Fourteen studies considered patients in a secondary care setting, which provided data for nine tests. Ten studies looked at patients in primary care settings, which covered four tests. A summary of care-setting-related data can be found in Table 14.

Estimates of test accuracy for cost-effectiveness modelling

Having established that a number of factors may influence test accuracy, we sought to provide estimates for each test to be used in the cost-effectiveness modelling. This is consistent with the findings of Leeflang et al. 72 Ideally, estimates would have come from a meta-analysis of several studies specific to the scope population, by age group and setting. However, the evidence base was not sufficient to do this. In total, there were 21 tests × 3 age groups × 3 settings = 189 pairs of sensitivity and specificity estimates required. However, no data were available that were specific to the elderly or the pharmacy setting, or for three of the tests, meaning that there were just 18 × 2 × 2 = 72 potential pairs of estimates. Each estimate came from a combination of five studies or fewer. Factoring in the observed variation in test accuracy between studies alongside the scant evidence base, there is a significant likelihood that the final estimates may not be representative of the tests’ true accuracy. There is a significant risk that a test with a larger evidence base published in peer-reviewed journal articles may be disadvantaged in comparison with a test in which there is only unpublished manufacturer information at high risk of bias.

We prioritised information from published studies [i.e. not those in manufacturer (submitted directly to NICE in response to a request for information) and FDA documents] in which data were available for patients restricted by throat score as per the scope. This provided accuracy data for one pair of estimates and relaxing the age group restriction provided another pair of estimates. It was necessary to relax the throat score restriction to obtain further estimates. An additional 13 pairs of estimates were obtained from studies that matched the age and care setting of the test. One further pair of estimates was obtained by using estimates from a mixed age population for an adult population. Relaxing the care setting and age restrictions allowed estimation of 24 pairs of test accuracy estimates for child and adult populations. Where there were multiple options for considering relaxing either age group or setting differences between studies and target population, factors such as sample size and number of studies were also considered. Studies in manufacturer responses to NICE and in FDA documents were included only if no other evidence was available for a specific test. Where these data are used, we consider the analysis to be at extremely high risk of bias and we do not recommend that these are sufficient to underpin any clinical decisions. The data from neither of these two sources matched a subgroup of interest or were restricted by throat score, but relaxing the ages and care settings provided estimates for a further 32 pairs.

A summary of the studies providing evidence for each estimate can be found in Table 15.

Accuracy of point-of-care tests using polymerase chain reaction to resolve discordant cases

Discordant results between point-of-care tests and culture were resolved using PCR in four studies. 20,24,37,73 All discrepant results between a point-of-care test and culture (point of care positive, culture negative and vice versa) were analysed in two of these studies. 20,24

All of the 20 samples that were cobas Liat Strep A positive but culture negative were confirmed as positive by PCR and bidirectional sequencing. All three samples that were cobas Liat Strep A negative and reference culture positive were confirmed as positive by PCR and bidirectional sequencing. 24 Wang et al. 24 also examined the discrepancies between the TestPack Plus Strep A test and culture. All discordant results, that is the 20 cases positive by the test and negative by culture and the three cases that were test negative and culture positive, were positive according to PCR.

In evaluating the accuracy of the BD Veritor system, Berry et al. 20 also identified 21 discordant results with throat culture, including 11 positive on the index test but not culture, and 10 positive on culture but not the index test. PCR detected strep A in 6 of the 11 results that were positive by the BD Veritor System but negative by culture. PCR detected strep A in all of the 11 samples that were negative by the BD Veritor system but positive by culture. In the same population, Berry et al. 20 found that 14 of the 15 results that were positive for the Alere i Strep A test and negative for culture were found to be positive according to PCR. There were no reported occasions when the Alere i Strep A test gave a negative result when culture gave a positive result.

Similarly, Cohen et al. 37 and Lacroix et al. 23 analysed only some of the discrepancies between a point-of-care test and culture. Cohen et al. 37 identified a total of 24 discordant results between the Alere i Strep A test and culture. There were 18 positive samples on the Alere i Strep A test and not on culture, 13 of which were confirmed as positive by PCR, whereas the other five results were PCR negative. Four of the six cases that were positive on culture but not on Alere i Strep A were confirmed as negative by PCR.

Lacroix et al. 23 found 84 discordant results between Sofia Strep A FIA and culture (31 false positives and 53 false negatives). Eleven of the 31 false-positive samples were missing; hence, PCR assays could not be conducted for these samples. Eleven of those with samples present were confirmed as positive by PCR and nine were negative by PCR. Lacroix et al. 23 also found 21 results that were positive by TestPack Plus Strep A but negative by culture, nine of which were confirmed as positive by PCR. Eight were confirmed as PCR negative, leaving four missing samples, which precluded additional PCR assays. Lacroix et al. 23 did not provide test-specific results for the cases that were negative by rapid test and positive by culture.

Table 16 summarises the key findings from these analyses.

Interestingly, Lindbæk et al. 43 used a second culture medium (a liquid medium/broth) to resolve discrepant results between the TestPack Plus Strep A test (Abbott Laboratories) and microbiological culture (streptococcal selective agar). In this study, the second culture medium [colistin and oxolinic acid (COBA) + tryptic soy agar (TSA) sheep + sulfamethoxazole and trimethoprim (SXT) + Lim broth + first culture medium] detected strep A in 17 out of 27 (63%) patients who previously tested positive by the Alere TestPack +Plus Strep A test but negative by the first culture medium (Columbia agar + horse blood + COBA).

Direct comparison of point-of-care test accuracy with clinical scores

Six studies directly compared levels of test accuracy between point-of-care tests and clinical scores. 39,44–46,52,57 The results are summarised in Table 17. Sensitivity point estimates for clinical scores were higher for point-of-care tests than for rapid tests in two studies. 46,57 However, point estimates for sensitivity and particularly specificity of rapid tests (including TestPack Plus Strep A, OSOM Strep A and QuikRead Go Strep A tests) were generally higher. Sensitivity (0.829 to 0.946) and specificity (0.849 to 0.991) point estimates of point-of-care tests were consistently high when compared with point estimates for clinical scores (sensitivity 0.735 to 0.972; specificity 0.172 to 0.648).

Test failure rate

Five studies reported on test failure rate. 23,37,38,43,54 These five studies reported on three different point-of-care tests (Alere i, Testpack Strep A Plus and Sofia FIA Strep A). For the Alere i test, the test failure rate ranged from 0.0%54 to 2.8%. 37 The TestPack Strep A Plus test failure rate ranged from 0.3%38 to 1.3%. 43 The Sofia FIA strep test failure rate was reported as 4.7%. 23 Differences could be a result of environmental factors, such as staff training, as opposed to issues with the tests.

Test accuracy of combined clinical score and point-of-care test with culture as reference standard

None of the included studies evaluated the accuracy of a combined strategy of a sore throat clinical score (at the recommended NICE cut-off points of Centor/McIsaac score of ≥ 3 points or FeverPAIN score of ≥ 4 points) with a point-of-care test. This would require the combination of the two methods into a single procedure, in which positive results are produced by individuals with both a high clinical score and a positive point-of-care test, and negative results are given either by patients with a low clinical score or by patients with a high score but a negative point-of-care test. As shown in Table 18, Rosenberg et al. 50 provide the only available evidence that attempts to match the proposed pathway, but not at the recommended Centor cut-off point.

Antibiotic-prescribing behaviours: randomised controlled trial evidence

There were three RCTs that reported on antibiotic use. All three trials found higher antibiotic prescription rates or use in control arms with no point-of-care test than in those with a point-of-care test.

In the UK RCT in primary care by Little et al. ,6 patients (mean ages 29 and 31 years across arms, no age range provided) in a primary health-care setting were randomly assigned to a delayed antibiotics control arm, a clinical score arm or a RADT arm (IMI TestPack, later known as Alere i Strep A test). In the delayed antibiotics control arm, depending on the severity of their presentation, patients were given antibiotics, given no antibiotics or given a delayed prescription of antibiotics to collect after 3–5 days if symptoms did not improve or worsened. This control group was there to represent current UK practice at the time. In the clinical score arm, patients were assessed using the FeverPAIN clinical scoring tool. Patients with scores of 0 or 1 points were not offered antibiotics. Immediate antibiotics were offered for patients with scores of ≥ 4 points; for patients with scores of 2 or 3 points, delayed antibiotics were offered. In the RADT group, all patients also received the clinical scoring tool. Those with scores of 0 or 1 points were not offered antibiotics or RADT, those with a score of 2 points were offered delayed antibiotics and those with scores of ≥ 3 points were given a RADT. All those with negative RADT results were not offered antibiotics. There were 207 patients in the delayed prescribing arm, of whom 79% (164/207) received a delayed prescription, 10% (21/207) received no antibiotics and 10% (21/207) received immediate antibiotics. In the clinical score arm, 41% (87/211) received a delayed prescription, 41% (87/211) received no antibiotics and 16% (33/211) received immediate antibiotics. In the RADT arm, there were fewer delayed prescription decisions, with only 23% (48/213) of patients receiving a delayed prescription; 59% (126/213) of patients were offered no antibiotics and 18% (38/213) were given immediate antibiotics. Patients reported antibiotic use of 46% (75/164) in the delayed prescription arm, 37% (60/161) in the clinical score arm and 35% (58/164) in the clinical score plus RADT arm. The total numbers in each arm were considerably lower for antibiotic use, indicating significant loss to follow-up, so these numbers should be interpreted with caution. Likewise, symptom severity was worse in the control arm, so effect sizes may be overestimates. This was a UK-based trial based in a primary health-care setting. For this reason, it is likely to be generalisable to the UK population.

The second trial was by Llor et al. 45 They included patients aged > 14 years (mean age 31.7 years) visiting primary health-care centres across Spain. This was a cluster RCT with the centre as the unit of randomisation. This form of randomisation can be prone to imbalancing baseline characteristics of patients; however, the authors reported no significant differences in baseline characteristics (such as gender, mean age and by clinical symptoms) between the participants across the intervention and control arms. Patients were randomised to either a control arm, in which patients were assessed using only clinical criteria (Centor), or an intervention arm, in which patients were assessed with both a Centor score and a RADT (OSOM Strep A test). In total, 54% (291/543) of patients were prescribed antibiotics. Antibiotics were more likely to be prescribed in the clinical score only arm, with GPs prescribing antibiotics in 64% (168/262) of patients, compared with 44% (123/281) in the RADT arm. There was a correlation between Centor score and antibiotic prescription rates across both groups, with more antibiotics prescribed to those with higher scores [score of 4 points – 80% antibiotics (37/46) in intervention arm and 96% (43/35) in the control arm compared with 16% (4/70) in intervention arm and 33.% (20/61) in control arm]. In the subgroup of interest to the UK population (those with Centor scores of ≥ 3 points), 74% (90/122) were given antibiotics in the intervention arm, compared with 85% (100/119) in the control arm. Antibiotic appropriateness is also discussed in the trial. Ninety-eight per cent (59/60) of patients with a positive RADT result were given antibiotics and 31% (69/225) of those with a negative test result received antibiotics. The authors determined that treatment was inappropriate (based on culture results) in 43% of patients (226/526), with 210 unnecessary prescriptions and 16 untreated cases. A total of 153 of these cases occurred in the control arm and 73 were in the RADT arm; however, the category of inappropriate decision (overprescribing or underprescribing) is not reported by trial arm.

The third trial was a four-armed cluster randomised trial in Canada by Worrall et al. 55 The trial included 40 physicians who were asked to consecutively recruit adult patients (aged ≥ 19 years, no further details reported). There was a control arm using usual clinical practice, an intervention arm using sore throat decision rules (STDR) (modified Centor), an intervention arm using a rapid test (RADT) and an intervention arm using both STDR and RADT. In the STDR group, for clinical scores of ≤ 1 point no antibiotics were recommended, for scores of 3 or 4 points antibiotics were recommended and for scores of 2 points the prescribing decision lay with clinicians. In the combined STDR and RADT group, RADT was used only for patients with scores of 2. It is implied, although not explicitly stated, that all those in the RADT arm received a RADT. The authors found that 47% (247/533) of patients received antibiotics. By arm, 58% (82/141) of patients received antibiotics in the usual practice arm, compared with 55% (94/170) with Centor score alone, 27% (32/120) with rapid antigen testing alone and 38% (39/102) with combined rapid antigen testing and Centor score. As this was a cluster randomised trial and each arm included only 8–10 doctors, differences could be a result of differences between doctors rather than between strategies. In addition, they may be owing to differences in patients across arms. The study reports on the characteristics of the physicians only; we have no baseline patient data. Finally, the Canadian medical system differs to the UK system, so the results may not be generalisable.

There was no RCT evidence on molecular technologies and antibiotic-prescribing rates.

Antibiotic-prescribing behaviours: before-and-after studies

There was one study that was a before-and-after study assessing antibiotic-prescribing rates. The study by Bird et al. 35 analysed children (aged 6 months to 16 years) presenting to a UK paediatric emergency department with a sore throat. The study compared baseline data from October and November 2014 with prescribing rates in the following 2 years (August to November 2015 and September to November 2016) following the implementation of using both a McIsaac score and a RADT. Baseline data were collected retrospectively from a departmental audit, when it is implied that the method of diagnosis was just clinician examination, with the aim to assess the impact of a clinical scoring system and rapid test on prescribing rates. A rapid test could be requested only if there was a McIsaac score of ≥ 3 points. Following implementation, antibiotic-prescribing rates fell steeply, from 79% (166/210) at baseline to 24% (51/214) in year 1 and 28% (51/181) for the second year. However, seasonality may be a confounding factor, with higher prescribing rates over the later autumn months (October and November) than in late summer (August and September). Likewise, there may be some regression to the mean, as the high initial prescribing rates may have prompted the study but may be subject to fluctuations.

There were no two-armed cohort studies analysing molecular technologies and antibiotic-prescribing rates.

Antibiotic-prescribing behaviours: other study designs

There were an additional eight studies that reported antibiotic-prescribing behaviours in single-arm cohorts. 20,36,39,40,46,50,52,53 No comparative data were possible within these study designs, only hypothetical comparisons, so all of the results in this section should be interpreted with caution and considered less informative than the RCT results. In these cohort studies, patients received the same intervention; however, authors also determined hypothetical management scenarios and compared how this would have affected antibiotic-prescribing rates. Of the eight studies, three provided hypothetical rules, which does not help inform us on real-world behaviour. These three studies39,46,53 have been included and briefly summarised but were not quality appraised. Five studies20,36,40,50,52 reported on either what happened in the real world or what clinicians reported they would do. These studies suggested that using a rapid test would decrease antibiotic use by as little as 9% up to 74%.

Two of the five single-arm cohorts reported on real-world behaviour. The first study, by Stefaniuk et al. ,52 examined children and adults in a primary care setting in Poland. Forty-six per cent (44/96) of the study group were children aged 3–14 years and 25% (24/96) were adults aged 31–35 years (overall mean age was not provided). Ninety-eight per cent (46/47) of patients with a positive QuikRead Go Strep A test result received antibiotics and 24% (12/49) of patients with both a negative rapid test and a negative culture were treated with antibiotics.

The second study reporting on real-world behaviour, by Berry et al. ,20 compared BD Veritor testing with Alere i testing and a chart review to determine hypothetical impact of results on antibiotic use. The study took place in paediatric outpatient clinics (mean age not reported) in the USA. Prescribing decisions were made with knowledge of the BD Veritor test results, but not of the Alere i test or culture. The authors found that 34% (73/215) of patients were prescribed antibiotics; of these, 25 patients were prescribed antibiotics at a clinic visit and antibiotics were later deemed to be inappropriate treatment (on the basis of culture results). Of these, 20 out of 25 (80%) patients were negative on BD Veritor, Alere i and culture, and five were positive with the BD Veritor only. Of the 215 who did not receive antibiotics, 13 BD Veritor-negative cases were identified by the authors as potential missed cases on the basis of PCR and Alere i positive results, of which six received antibiotics within 6 days of the original appointment. These analyses provide descriptive behaviour data using the BD Veritor test, but cannot be used to compare Alere and BD Veritor for appropriateness of prescribing behaviour as decisions were made using the BD Veritor and not the Alere i. This study using Alere i was the only study to use a molecular technology, and no prescribing behaviours were based on it; hence, there is no evidence on molecular technologies and antibiotic-prescribing rates.

Three of the single-arm cohort studies reported on hypothetical scenarios based around clinicians’ decisions. 36,40,50 Bura et al. 36 examined a cohort of adults (median age 26 years, range 18–44 years) in primary care in Poland with Centor scores of > 2 points (this was a case–control design for test accuracy outcomes, but cohort for prescribing behaviour). All patients and controls were given a rapid test and culture. GPs could then choose whether or not to give antibiotic therapy. It is stated that this choice was not influenced by the research team; however, we cannot be certain of this as they were aware of the rapid test result. Clinicians were aware of the Centor score at the time of antibiotic prescribing. They found that 58% (59/101) of patients received an antibiotic. All RADT-positive patients received treatment, including two who were culture negative. In addition, 46% (35/77) of test-negative patients received antibiotics. They determined that 40% (23/59) of cases received an unnecessary antibiotic prescription. Unnecessary has been defined here as being culture negative. The authors also gave hypothetical management scenarios based on different Centor scores and scenarios. Antibiotics would be prescribed to 29% (11/38) of patients with a Centor score of 2 points, 62% (23/37) of patients with a Centor score of 3 points and 96% (25/26) of patients with a score of 4 points. They surmised that 23% (23/101) would have been treated using positive culture results alone and 24% (24/101) would have been treated using a rapid test, meaning that one person was mistakenly given antibiotics. However, 54% of those given antibiotics were treated for non-strep A. From the control group, one person would have been treated with antibiotics; additionally, other forms of streptococci were identified in 13 people from this group.

The study by Rosenberg et al. 50 was a one-armed prospective observational cohort in which all patients were given a clinical score (Centor), rapid test and culture. The study included patients older than 3 years [47% (59/126) aged 3–14 years, 50% (63/126) aged 15–44 years and 3% (4/126) aged ≥ 45 years] presenting to an emergency department in Canada. The authors also report physicians’ clinical impressions and their hypothetical management. Authors report on score alone, physician examination alone, rapid test alone or rapid test for clinical scores of > 3 points. They found that physicians prescribed antibiotics to 37% (46/126) of patients, after obtaining the results of the rapid test; of these, 18 had negative culture results. They hypothesised that 20% (25/126) would have received antibiotics in the rapid test group, compared with 29% (37/126) in the clinical score group.

The last study, by Johansson et al. ,40 was a prospective observational single-armed cohort in which all patients received both a rapid test and culture and these results were compared with hypothetical management suggestions made by physicians. It included adult patients (aged 25–44 years, mean age not reported) reporting to primary health-care centres in Sweden. Physicians also clinically assessed patients, and gave hypothetical management suggestions based on their level of certainty for strep A (absolutely positive, positive, possibly positive, possibly negative, negative and absolutely negative). No results are clearly provided; however, 26% (24/94) of patients with a negative rapid test received treatment, but it is unclear how many of these were culture positive.

There were three additional studies39,46,53 that reported on hypothetical prescribing decisions based on assumptions about doctors’ behaviour; however, no real-world decisions were reported and doctors were not asked about behaviour.

Search strategy

A comprehensive search of the literature for published economic evaluations (including any existing models), cost studies and quality-of-life (utility) studies was carried out. The systematic search included searching the following electronic databases during January 2019 (on 22, 29 and 30 January 2019), and an updated search was conducted on all databases during March 2019 (on 7 and 13 March 2019):

• MEDLINE and Epub Ahead of Print, In-Process & Other Non-Indexed Citations, Daily and Versions (via OvidSP)

• Excerpta Medica database (EMBASE) (via OvidSP)

• NHS Economic Evaluation Database (NHS EED) and HTA database (via CRD)

• Science Citation Index and Conference Proceedings Citation Index – Science (via the Web of Science)

• Cost-Effectiveness Analysis (CEA) registry

• EconPapers [Research Papers in Economics (RePEc)]

• School of Health and Related Research Health Utilities Database (ScHARRHUD).

The search terms included economic and quality-of-life terms combined with either ‘sore throat’ or ‘strep A’. No date limits were applied and databases were searched from inception. The search strategy was developed by an experienced information specialist, based on the clinical effectiveness review and with input from a health economist. Details of the full search strategies are provided in Appendix 6. In addition to these searches, any relevant cost-effectiveness studies identified during the clinical effectiveness review were brought to the attention of the reviewers and assessed for eligibility alongside the results of this review.

Assessment of eligibility

Citations and abstracts from the electronic online databases were exported into a citation software package (EndNote X7) and duplicate records were identified and removed. Two reviewers independently reviewed titles and abstracts to identify potentially relevant papers for inclusion. Any discrepancies were resolved by discussion.

Inclusion criteria

Only studies meeting the following inclusion criteria were included in the review:

• study type – fully published economic evaluations (including any economic models)

• population – people aged ≥ 5 years presenting to health-care providers in a primary care (GP surgeries, community pharmacies and walk-in centres) or secondary care (urgent care/walk-in centres and emergency departments) setting with symptoms of an acute sore throat

• intervention – 17 RADTs or four molecular tests (as described in Chapter 1, Comparative technical overview of the point-of-care tests for group A Streptococcus)

• comparator – antibiotic-prescribing decisions using clinical judgement and a clinical scoring tool such as FeverPAIN or Centor

• outcomes – cost–benefit or cost–consequences or cost-effectiveness or cost–utility studies reporting outcomes as cost–consequence measures or clinical effectiveness measures or utility measures [utility, EuroQol-5 Dimensions (EQ-5D) or Short Form questionnaire-6 Dimensions score or quality-adjusted life-years (QALYs)].

Exclusion criteria

Studies meeting the following exclusion criteria were excluded from the review:

• non-English-language publications

• studies not in humans

• studies not in strep A or sore throat

• studies with the wrong test or no specified test

• studies that were not full economic evaluations (incremental costs and incremental benefits).

Studies that provided useful information for the economic model, such as resource use, costs, utilities and probabilities, were retained but were not included in this review.

Data extraction

Data extraction was carried out by one reviewer using standardised data extraction sheets and was then checked by a second reviewer. Data extracted included the following information:

• study details – study title, author names, source of publication, language and publication type

• baseline characteristics – population (and subgroups), intervention, comparators, outcomes, study design, setting and location and type of economic evaluation

• methods – study perspective, time horizon, discount rate, measurement of effectiveness, measurement and valuation of preference-based outcomes, resource use and costs, currency, price date and conversion, model type, assumptions and analytical methods

• results – study parameters, incremental costs and outcomes and reporting of uncertainty

• discussion – study findings, limitations, generalisability and conclusions

• other – sources of funding, conflicts of interest and any comments.

Data synthesis

Information extracted from the included studies was summarised and tabulated. Findings from individual studies were compared narratively.

Quality assessment

The quality of full economic evaluation studies that were identified was assessed using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist by one reviewer and cross-checked by a second reviewer. The CHEERS checklist comprises six dimensions (title and abstract, introduction, methods, results, discussion and other); under these dimensions, a series of questions check whether or not the criteria have been clearly reported. 74 If the studies included any model-based economic evaluations, they were further critically appraised using the framework on quality assessment for economic modelling developed by Phillips et al. 75 The framework assesses models under the dimensions of structure, data and consistency and whether or not the criteria have been clearly reported.

Search results

The literature search identified 6980 records through electronic database searches and other sources. After removing duplicates, 2756 records were screened for inclusion. One article was found via our clinical effectiveness search. Based on title and abstract sift only, 2737 records were excluded. The remaining 19 records were included for full-text screening. A further 16 articles were excluded at the full-text stage, as these studies did not contain a full economic evaluation or specify the right test (see Appendix 7 for further comments).

The literature search identified three studies that had evidence pertaining to incremental costs and outcomes: Bura et al. ,36 Humair et al. 39 and Little et al. 76 (Figure 11).

FIGURE 11.

The PRISMA flow diagram for economic evaluation studies.

The economic information from the first two studies has been summarised below, as there was not enough information for a full data extraction (see Appendix 8). These two studies did not explicitly state the following: study perspective, time horizon, type of economic evaluation, measurement of effectiveness or analytical methods. Bura et al. 36 was a prospective case–control study consisting of 101 adults (aged 18–44 years) who went to GP clinics in Poland because of sore throat lasting no longer than 7 days. Control participants (n = 101) were volunteers from the same area, who were matched to cases according to their age and sex. The study was conducted over 1 year. The OSOM Strep A test (Sekisui Diagnostics) in conjunction with throat culture was compared with Centor and throat culture to confirm presence of strep A. The costs of diagnosing and treating strep A included symptomatic treatment, test cost (€1.39), a single culture to identify strep A, antibiotic therapy and antimicrobial medications. Economic analysis of five strategies were compared for treating patients with strep A in terms of cost per patient with appropriate strep A treatment ranged from €2.89 (for treat only RADT positive cases) to €6.93 [for treat only strep A + (culture-positive) cases]. The authors concluded that the use of the rapid test significantly increases the number of people with strep A-related pharyngitis to be treated with antibiotics.

Humair et al. 39 was a prospective cohort study consisting of 372 adults (aged 15–65 years) who were treated at a GP clinic in Switzerland. The Alere TestPack +Plus Strep A (Abbott Laboratories) was compared with throat culture. A decision tree model was used to compare antibiotic prescription for five strategies. Information used in the decision model included antibiotic rate for appropriate use, overuse in patients without strep A, underuse in patients with strep A, appropriate treatment for patients with strep A and without treatment in patients without strep A. The model did not consider quality of life, complications or adverse drug effects. Costs were in US$ in 2002 prices. Costs included a 10-day course of penicillin, a test cost of $5.00 and $18.00 for throat culture. The authors found that systematic throat culture had the highest rates of appropriate treatment, whereas empirical treatment in patients with clinical scores of 3 or 4 resulted in the most antibiotic overuse. The cost per case appropriately treated ranged from $15.30 (systematic rapid test) to $32.40 (systematic throat culture). Sensitivity analyses were conducted to check the robustness of results. The authors concluded that the rapid test is a valid test for diagnosis of strep A.

Little et al. 76 conducted an economic analysis alongside a RCT in the UK, which included both adults and children with acute sore throat who were seen in primary care clinics (see Appendix 8). They compared randomised patients with targeted antibiotic use according to (1) delayed antibiotics (control group), (2) clinical score using FeverPAIN or (3) RADT – Alere TestPack +Plus Strep A (Abbott Laboratories) used according to clinical score. The analysis was from an NHS perspective and the time horizon was short (14 and 28 days); hence, long-term effects were not captured. Health-related quality of life was evaluated using the EQ-5D. QALYs were adjusted for baseline differences and were calculated using mean EQ-5D scores obtained from the 14-day diary records. It was assumed that the health-related quality of life changes linearly over time. The analysis included a cost-effectiveness analysis (cost per change in symptom severity) and a cost–utility analysis (cost per QALY). Cost-effectiveness acceptability curves (CEACs) were generated using bootstrapping with 5000 samples.

The mean symptom scores were adjusted for baseline differences. For the cost-effectiveness analysis, the clinical score group dominated both the delayed antibiotic group and the RADT group, as it was more clinically effective (lower symptom score) and less costly. However, the point estimate of symptom score and the corresponding 95% CIs for clinical score and RADT groups were quite close. The CEAC showed that if the value of a 1-point change in the symptom score was varied between £0 and £500, and it was found that over the entire range the clinical score group was most likely to be cost-effective. In the cost–utility analysis, the delayed group was dominated by the clinical score group for both time frames. The incremental cost-effectiveness ratio (ICER) for the RADT group compared with clinical score group was £74,286 for the 14-day time frame and £24,528 for the 28-day time frame.

Quality assessment

The quality of the reporting of the economic analysis of the three studies was assessed using the 25-point CHEERS checklist74 and is provided in Table 19. The Little et al. 76 article was comprehensively reported: 22 of the 25 statements (88.0%) were a ‘yes’, one statement (4.0%) was not completed and two statements (8.0%) did not apply.

Accuracy of clinical scoring algorithms (all models)

Accuracy in the usual-care arm was based on estimates of sensitivity and specificity of the Centor score taken from a published meta-analysis of 12 studies by Aalbers et al. 84 Table 20 summarises the reported estimates of sensitivity and specificity of the Centor score at cut-off points of ≥ 1, ≥ 2, ≥ 3 and 4 points for positive strep A infection. The base-case model used the estimates at a cut-off point of ≥ 3 points for a positive result and < 3 points for a negative result. At this threshold, the Centor score has a sensitivity of 0.49 (95% CI 0.38 to 0.60) and a specificity of 0.82 (95% CI 0.72 to 0.88). Alternative thresholds on the Centor score were explored in sensitivity analyses. 84 However, we were unable to evaluate the FeverPAIN clinical score owing to a lack of accuracy estimates in a format suitable for the economic model (i.e. sensitivity and specificity of the FeverPAIN at a cut-off point of ≥ 4 points).

Accuracy of point-of-care tests

Estimates of test accuracy for the point-of-care tests obtained from our systematic review with and without meta-analyses are summarised in Table 21 by test, clinical setting (primary care) and patient population (adults and children). When no studies reporting accuracy data were identified in our systematic review, we obtained the estimates from either the manufacturer website or the manufacturer submissions (submitted directly to NICE in response to a request for information). Test accuracy data were available for six (28.6%) of the 21 tests from published sources identified in the clinical effectiveness review; a further four tests (19%) had accuracy data from both published sources and manufacturer submissions, six tests (27.6%) had only manufacturer data and two tests (9.5%) had FDA data. Test accuracy data were not available for the three (14.3%) remaining tests (Biopanda Reagents’ Strep A Rapid Test strip, bioNexia Strep A cassette and bioNexia Strep A plus cassette). Two of the three tests (bioNexia Strep A cassette and bioNexia Strep A plus cassette) were excluded from the economic modelling of individual tests owing to a lack of test accuracy data. The accuracy of Biopanda Reagents’ Strep A Rapid Test strip was assumed to be equal to that of the cassette version of this test, for which accuracy estimates were available. In general, estimates of sensitivity and specificity obtained from the published sources tended to be variable and lower than those provided by the manufacturer. For example, sensitivity of point-of-care testing in adults based on the published sources ranged from 0.68, for Abbott Laboratories’ Clearview Exact Strep A cassette, to 1.00, for the QuikRead Go Strep A test kit, whereas estimates provided in the manufacturer submission ranged from 0.95, for Biopanda Reagents’ Strep A Rapid Test cassette, to 0.98, for nal von minden GmbH’s NADAL Strep A test. A similar trend in specificity is observed, with the manufacturers’ estimates being generally much higher than estimates based on published data. Thus, the source of test accuracy data is likely to be an important driver of cost-effectiveness. The economic models presented here, which are based solely on manufacturers’ test accuracy data with no peer-reviewed published data, are likely to overestimate test accuracy and, therefore, the results of these models cannot be reliably interpreted.

Cost of tests

Table 25 presents the unit cost for each point-of-care test and estimates of resource use in terms of the additional GP time required to administer and process the test results. Cost data were available for 14 (66.7%) of the 21 tests considered in the NICE scope. The majority of the costs were provided by the manufacturers (submitted directly to NICE in response to a request for information) and ranged from £0.64 per test for Biopanda Reagents’ Strep A Rapid Test strip to £64.63 (2017/18 prices) for the cobas Liat Strep A Assay supplied by Roche Diagnostics. Unit costs for Abbott Laboratories’ Clearview Exact Strep A tests were obtained from the NHS supply chain catalogue at £1.92 per test for the Clearview Strep A dipstick – test strip and £2.72 for the cassette version. 88 The duration of additional GP time for processing test results was estimated based on information provided in the manufacturer submission and ranged from 5 to 12 minutes. Costs associated with additional GP time for processing test results are included in the base-case analysis. The costs of confirmatory swab culture following a negative test result are calculated as part of the costs associated with modelled pathways in the intervention arm except for the Alere TestPack +Plus Strep A – cassette (Abbott Laboratories), Alere i Strep A 2 (Abbott Laboratories), cobas Liat Strep A Assay (Roche Diagnostics) and all five NADAL tests supplied by nal von minden GmbH. Details of costing methods are given in the next section.

Treatment costs

Unit costs of health-care service use associated with modelled care pathways are summarised in Table 26. As described in Model structure, the base-case models incorporate three treatment options for patients presenting with suspected strep A infection in primary care and secondary care settings: immediate antibiotics (option 1), delayed antibiotics with reported use (option 2) and delayed antibiotics that have not been used or no antibiotics offered (option 3). All three options account for repeat GP consultations at 14.2%6 over the modelled time horizon, with a typical GP consultation lasting 9.22 minutes at an average cost of £4.02 per minute90 and pain relief (500 mg of paracetamol) costing £0.74 per 32-tablet pack,91 but the options differed in the way that antibiotics are prescribed.

Under options 1 and 2, patients incur a cost of antibiotics at £0.91 per treatment course (phenoxymethylpenicillin 250 mg, 28-tablet pack)91 and costs associated with managing adverse effects of penicillin: penicillin-induced rash [assumed to be seen by GP at additional expense (£4.02 per minute) and switched to erythromycin 500 mg at £10 per treatment course91 weighted by 0.02, the probability of a rash] and penicillin-induced anaphylaxis [estimated at £1744 based on data reported in a 2017 cost of sepsis study93 (see Table 26) weighted by 0.0001, the probability of sepsis]. 78 No costs associated with antibiotic use are included under option 3 (delayed antibiotics prescription given but not used); however, we assume that 14.2% of patients attended a repeat consultation86 and will use the delayed antibiotics prescription under this option.

Confirmatory swab culture costing £892 was added to options 2 and 3 for patients with a negative test result (intervention only) but not to option 1, as patients with a positive test result receive immediate antibiotics. On average, the estimated treatment costs based on these assumptions and a repeat consultation rate of 14.2% were £44.89 (option 1: intervention and usual-care arms; option 2: usual-care arm), £52.89 (option 2: intervention arm including confirmatory culture costs), £43.78 (option 3: usual-care arm) and £51.77 (option 3: intervention arm including confirmatory culture costs) (see Table 26).

The cost of sepsis was estimated to be £1744 based on data reported in a study,93 which estimated that 93,973 adults would need treatment for sepsis in UK hospitals, at an annual total cost of £163,949,055 (see Table 26). The cost of treating strep A-related abscess was estimated at £1571 based on the NHS reference cost for a tonsillectomy in adults aged ≥ 19 years with a Healthcare Resource Group code of CA60A. 92 The cost of treating acute rheumatic fever was estimated at £1772.44 based on the NHS reference cost for other acquired cardiac conditions with a CC (complications and comorbidities) score of 6–8 and a Healthcare Resource Group code EB14C. 92