Modelling of hypothetical SARS-CoV-2 point-of-care tests on admission to hospital from A&E: rapid cost-effectiveness analysis

Attendance at accident and emergency department

This element simulates the arrival of a patient at hospital. The next step is dependent on the SARS-CoV-2 testing strategy that is in place, as the model has the functionality to explore testing all patients who attend A&E or testing only those patients who are to be admitted to hospital.

If the strategy of testing all patients who attend A&E is selected, then the patient would have a SARS-CoV-2 test performed (see Testing strategies for patients arriving at accident and emergency) before moving to ‘waiting for a decision on admission’. When tests are performed only on those admitted to hospital, the person moves directly to ‘waiting for a decision on admission’. For this report, only results for when testing is used for those admitted to hospital are presented.

Waiting for a decision on admission

This element simulates the period during which a patient is waiting in a hospital to see a clinician who will make a definitive decision about whether or not the patient requires hospital admission. The time required for this decision is not modelled explicitly based on the number of patients and number of staff but uses estimated times that differ by time of day. Further details are provided in Interarrival times and time required to make a decision on whether or not hospital admission is needed.

During this period, the person could possibly be infected by another person with SARS-CoV-2 who is waiting for a decision about hospital admission, or this person, if infected with SARS-CoV-2, could potentially infect other people who do not currently have SARS-CoV-2. Further details are provided in Simulation of the spread of SARS-CoV-2 infection.

Following the definitive clinical decision, patients who do not need to be admitted are discharged. Those who require hospital admission receive a SARS-CoV-2 test, if they did not receive this test on arrival at A&E, and are moved to the ‘waiting for a decision on bed allocation’ element. It is assumed that all patients can be accommodated within the ‘waiting for a decision on bed allocation’ area (see The configuration of the hospitals used in the example).

Waiting for a decision on bed allocation

This element is divided into two sub-elements: those who have symptoms indicative of potential COVID-19 infection and those who do not. Currently, the EAG has heard that hospitals will try to separate patients into these groups while a SARS-CoV-2 test is performed to aid the decision about allocating the patient to a designated SARS-CoV-2-infected bay or to a designated non-SARS-CoV-2-infected bay to reduce the risk of spreading or contracting infection while in hospital. Hence, if a patient receives a test more quickly, then this is manifested in a shorter time waiting for a decision about the appropriate bed and less capacity pressure on the waiting bays. It is assumed that all patients can be accommodated in the SARS-CoV-2-infected bays and the non-SARS-CoV-2-infected bays (see The configuration of the hospitals used in the example).

In the base-case model, all patients who have clinical symptoms suggestive of COVID-19 would have a second test if their initial test result was negative, in case that result was a false negative. The allocation of the patient would be made on the basis of the second SARS-CoV-2 test unless the patient had chest imaging results indicative of COVID-19, in which case both tests would be over-ridden and the patient would be sent to a SARS-CoV-2-infected bay. The model has the functionality to alter the proportion of instances in which two negative tests were over-ridden. Sensitivity analyses have been undertaken assuming that the proportion of patients over-ridden was zero.

Patients who have clinical symptoms suggestive of COVID-19 and a positive SARS-CoV-2 test are moved to a SARS-CoV-2-infected bay. Patients without clinical symptoms have only one test, with a positive SARS-CoV-2 test resulting in them being placed in a SARS-CoV-2-infected bay and a negative SARS-CoV-2 test resulting in the them being placed in a non-SARS-CoV-2-infected bay.

For all patients there is a possibility that the SARS-CoV-2 test fails. In this instance the patient remains in the waiting bay while the test is reconducted. The probability of SARS-CoV-2 test failure is detailed in Characteristics of the point-of-care tests and laboratory-based tests.

The model has the functionality to alter the base case with these amendments explored in scenario analyses. Exceptions to allocating based purely on the base-case model are:

1. Allocation of a patient to an intermediate bay (henceforth labelled a ‘pink bay’). Where the hospital has sufficient capacity, pink bays may be created for patients who have no clinical symptoms of COVID-19 and no evidence from chest imaging that would indicate SARS-CoV-2 infection. In this instance a clinician may suspect that the test was a false positive and not wish to confine the patient to a SARS-CoV-2-infected bay. When pink bays were used, a new tranche of staff working in pink bays was activated.

2. Allowing patients with clinical symptoms of COVID-19 and two negative SARS-CoV-2 tests to remain in a waiting bay until discharge, transfer to the intensive care unit (ICU) or death. This option is explored in scenario analyses and may be feasible for only some hospitals.

3. Removing the requirement to retest patients who have clinical symptoms suggestive of COVID-19 but a negative SARS-CoV-2 test result. In this scenario, people with a negative SARS-CoV-2 test result and no symptoms of COVID-19 on chest imaging would be allocated to a non-SARS-CoV-2-infected bay. Those with chest imaging indicative of COVID-19 would have their destination (SARS-CoV-2-infected bay or non-SARS-CoV-2-infected bay) chosen depending on the assumption used for over-riding the SARS-CoV-2 test.

The configuration of the bays in the area reserved for patients who require hospital admission and are waiting for the result of a SARS-CoV-2 test is user-definable. The assumptions used in the modelling are discussed in The configuration of the hospitals used in the example.

Allocated to an appropriate bed in a SARS-CoV-2-infected bay or a non-SARS-CoV-2-infected bay

As detailed in Waiting for a decision on bed allocation, for a patient admitted to hospital there are a number of options regarding the allocated bed.

Scenario analyses expand the choices of destination by allowing the patient to move to a pink bay (see Waiting for a decision on bed allocation) or to remain in the waiting bay until discharge or until ICU care is required (see Retesting strategies for patients admitted to hospital).

Based on the patient’s characteristics, he or she is assumed to leave the SARS-CoV-2-infected bay or non-SARS-CoV-2-infected bay by one of three routes: to discharge, following recovery; to the ICU, following a worsening of the condition; or as a result of death while in the bay. The probability of these three options, and the length of stay before leaving the bay, is discussed in Simulation of the spread of SARS-CoV-2 infection.

When the general bay (SARS-CoV-2-infected or non-SARS-CoV-2-infected) has been determined, patients can infect, and by infected by, other patients in the bay and by staff employed in that bay. Staff attend patients in more than one bay and, thus, it is possible for staff to pass infection between bays. See Simulation of the spread of SARS-CoV-2 infection for more details.

Patients in non-SARS-CoV-2-infected bays can develop clinical symptoms of COVID-19 if they have been infected while in hospital (see Simulation of the spread of SARS-CoV-2 infection) and can also have a positive SARS-CoV-2 test when they are routinely tested in hospital (see Retesting strategies for patients admitted to hospital). In both instances, the patient would be moved to a SARS-CoV-2-infected bay. The model assumes that clinical symptoms develop only following SARS-CoV-2 infection and not as a result of other diseases such as pneumonia.

Admission to intensive care unit

The ICU is subdivided into ICU bays reserved for those who are suspected of having SARS-CoV-2 infection, or have a positive SARS-CoV-2 test, and ICU bays for the remaining patients. Typically, this will be decided depending on whether or not the patient had been in a bay for patients with SARS-CoV-2 infection immediately before their admission to ICU. An exception to this would be a patient requiring ICU care who had been in a non-SARS-CoV-2-infected bay in which another patient had developed clinical symptoms suggestive of COVID-19 or had a positive SARS-CoV-2 test during routine retesting. This patient would go to a SARS-CoV-2-infected ICU bay, rather than to a non-SARS-CoV-2-infected ICU bay.

Patients in non-SARS-CoV-2-infected ICU bays can develop clinical symptoms of COVID-19 if they have been infected while in hospital and can also have a positive SARS-CoV-2 test when routinely tested in hospital (see Retesting strategies for patients admitted to hospital) as a result of infection in hospital, a false-negative result to testing on admission to hospital or a false-positive result on the repeat test. In both of these instances the patient would be moved to a SARS-CoV-2-infected ICU bay.

Once admitted to ICU, the patient can leave in one of two ways: recovery and removal to a non-ICU bay, conditional on whether or not the patient had previously been allocated to a SARS-CoV-2-infected bay; or death. The probability of these events happening is based on the patient’s characteristics and is detailed in The characteristics of patients attending accident and emergency. For simplicity, patients who are moved from the ICU to a general bay on their recovery are assumed to no longer be infectious with SARS-CoV-2 because of the likely number of days since clinical onset, although these patients would still be assumed to be grouped in SARS-CoV-2-infected bays. It is assumed that patients cannot be reinfected during the same hospital stay and will be discharged following a period in the general bay.

During a patient’s stay in the ICU, they can infect, and be infected by, other patients in the ICU bay and by staff employed in that ICU bay.

The configurations of SARS-CoV-2-infected ICU bays and of non-SARS-CoV-2-infected ICU bays are user-definable. The assumptions used in the modelling are discussed in The configuration of the hospitals used in the example.

The model does not currently have the facility to allow patients to enter the ICU directly.

Discharge

Patients can be discharged from either A&E or a general bay. The earliest point of discharge is after medical assessment of whether or not the patient requires hospital admission (see Waiting for a decision on admission). If this patient has a SARS-CoV-2 test, then they will be informed of the result on discharge if this is known, or at a later date otherwise. Patients can also be discharged from a general bay if they do not require ICU care. Those patients who have received ICU care will be transferred as detailed in Allocated to an appropriate bed in a SARS-CoV-2-infected bay or a non-SARS-CoV-2-infected bay. In a scenario analysis in which patients who have clinical symptoms suggestive of COVID-19 but have two negative tests are kept in waiting bays, discharge can occur from the waiting bay (see Retesting strategies for patients admitted to hospital).

There is the possibility that, on discharge, the patient could infect a member of staff or be infected by a member of staff. Further details on this are provided in Effective contacts and attack rates.

Death in hospital

Patients can die in the hospital. This is detailed in The mortality rates for patients admitted to hospital.

The characteristics of patients attending accident and emergency

The characteristics of staff members working in the hospital

At the start of the model, it is assumed that no staff members have been infected by SARS-CoV-2 and that none exhibits clinical symptoms suggestive of COVID-19. The age distributions, ethnicity and comorbidity status of staff members were not required as the model does not explicitly take into consideration the consequences of infection for staff beyond infecting other staff members and patients.

To simplify the model, it was assumed that each area bullet-pointed in The conceptual model: staff used 20 staff. These staff members work across multiple bays in the distinct areas and so infection can be passed between bays. Thus, staff members assigned to SARS-CoV-2 infection bays would not work in non-SARS-CoV-2 infection bays or at discharge. Staff members were randomly chosen when contacting patients and other staff members (as detailed in Effective contacts and attack rates).

It is assumed that staff members can be infected with SARS-CoV-2 outside the hospital setting. The risk of this per day is assumed to be 0.6 per 10,000 people, based on data from the Office for National Statistics. 20

The configuration of the hospitals used in the example

The model has the functionality to assess different levels of patient attendance per day and different numbers of bays. In this report we focus on an example of a large hospital. It is acknowledged that these numbers will not reflect a specific hospital, but they are there, along with the sensitivity analysis, to provide indicative results. For each of the waiting bays, the admitted bays, the ICU bays and the waiting for decision on admission bays, the last bay is set to an infinite capacity to signify the use of non-designated areas when necessary or to take into account transfers to alternative hospitals or facilities if required. The number of people in each infinite bay across time has been presented for selected strategies.

The configuration of the example hospital is provided in Table 6. In sensitivity analyses, three four-bed SARS-CoV-2-infected bays were removed and replaced with three four-bed pink bays and one infinite pink bay.

The routing into all bays assumes that patients will fill the bays in numerical order, so if there were six four-bed bays then the first four patients would be put in to the first bay, with the fifth patient being allocated to the second bay, unless one of the patients in the first bay was no longer occupying a bed. The EAG notes that if hospitals were prepared to open multiple bays and fill these sequentially, this may reduce the number of infections.

Interarrival times and time required to make a decision on whether or not hospital admission is needed

The model does not explicitly simulate the interaction between staff and patients to work out queueing times. A simplified approach has been taken, whereby patients arrive at constant rates but the time before a decision is made whether or not to admit the person is altered depending on the time of day to simulate variable waiting times before the decision is made to admit to hospital or to discharge.

The number of people attending A&E per day was assumed to be 250 following discussion with clinical experts. The waiting time before a decision to admit or not to admit was assumed depending on the time of day the patient arrived, with the assumption that the 250 patients would be spaced equally throughout the day. The time required for the admission decision was assumed to be 2 hours for patients arriving between midnight and 6 a.m., 3 hours for patients arriving between 6 a.m. and 6 p.m. and 4 hours for patients arriving between 6 p.m. and midnight. For simplicity, it was assumed that the time until the decision on admission was independent of the severity of a person’s injury.

Simulation of the spread of SARS-CoV-2 infection

The model simulates infections in the following four categories: patient to patient, patient to staff, staff to patient and staff to staff. It has been assumed that infection can be passed only to patients and staff who did not enter A&E with SARS-CoV-2 infection and have not been infected previously during the hospital stay.

Testing strategies for patients arriving at accident and emergency

As described in The conceptual model: patients, the model has the functionality to employ two testing strategies: to test all people arriving at A&E or to test only those about whom a decision has been made to admit to hospital. Only the scenario of testing patients on admission to hospital has been evaluated in this report.

Characteristics of the point-of-care tests and laboratory-based tests

The POCTs for SARS-CoV-2 are hypothetical. The assumed diagnostic accuracies of the POCTs and laboratory-based tests, the times for the result to become known by the treating physician, and the failure rates used in the base-case model are shown in Table 9. Sensitivity analyses using these parameters have been undertaken. It is assumed that neither the laboratory-based test nor the POCT could detect SARS-CoV-2 until 0.5 days after infection, based on Jarvis et al. 26 The model uses a simplifying assumption that, until this point is reached, SARS-CoV-2 cannot be detected, although false-positive tests could still occur in this situation. Beyond this time point the sensitivity values reported in Table 9 are considered appropriate. All patients arriving at A&E with SARS-CoV-2 infection are assumed to be at, or near, the peak of their viral levels.

The time required for the treating physician to receive the test result has an impact on the capacity utilisation of the waiting bays. A shorter time for a test result would mean quicker allocation to either a SARS-CoV-2-infected or a non-SARS-CoV-2-infected bay. These times are longer than the actual processing times of POCTs and should take into account all relevant steps in the process. Sensitivity analyses have been conducted using alternative times to obtain the result. The failure rates of all tests have been set to that in the TPP.

In the base case for SARS-CoV-2 laboratory-based tests, a time of 35.9 hours was used, as this was the median time reported by Collier et al. 27 However, some hospitals may have access to more rapid turnaround times, and sensitivity analyses have been run at 24 hours and at 6 hours. 28 The sensitivity of laboratory-based tests was assumed to be 89%, based on a meta-analysis conducted by Kim et al. ,29 with a specificity of 100% following Grassly et al. 22 It was believed that the sensitivity of SARS-CoV-2 tests might be reduced in asymptomatic patients. Based on data provided in Clifford et al. ,10 the ratio of sensitivities was assumed to be 82.6%, as values of 75% sensitivity in symptomatic patients and 62% sensitivity in asymptomatic patients were reported.

For the SARS-CoV-2 POCTs, the TPPs published by the Medicines and Healthcare products Regulatory Agency3 were used. These were divided into desired and acceptable characteristics. In addition, a sensitivity analysis using data from currently available SARS-CoV-2 POCTs was used. In this scenario, the time to obtain the result was assumed to be 3.8 hours, based on data presented in Collier et al. ,27 with sensitivity and specificity taken from a Cochrane review by Dinnes et al. 30 but adjusted to take into consideration that these were compared with laboratory-based tests, which were assumed to have imperfect sensitivity (0.89; see Table 9). The sensitivity reported in Dinnes et al. was, therefore, multiplied by 0.89.

The cost of the hypothetical POCT is assumed to be £27.76,31 based on the 5.8 million DnaNudge test the UK government has for £161M. The deal is likely to include 5000 platforms for running the tests which overestimate the cost. The cost of a urine and plasma laboratory-based test (BioPorto, Hellerup, Denmark) used to assess acute kidney failure in people considered for admission to critical care was used as a proxy for the cost of a laboratory-based test, and was assumed to be £21.90. 32 We have assumed that the staff costs associated with POCTs and laboratory-based tests are equivalent. The costs of tests are explored in sensitivity analyses. All costs used in the analyses are assumed to be in Great British pounds in 2020.

Retesting strategies for patients admitted to hospital

Retesting strategies fall into three categories: retesting of those admitted to hospital who had a failed test; routine retesting of those allocated to a non-SARS-CoV-2-infected bay following negative SARS-CoV-2 tests; and, depending on the strategy chosen, retesting of patients with clinical symptoms suggestive of COVID-19 who had a negative SARS-CoV-2 test.

Testing strategies for asymptomatic staff

The model has the functionality for staff to be tested at user-defined intervals, or for a strategy of no staff testing to be employed. In both the testing and the no-testing strategies, it has been assumed that any staff member with clinical symptoms indicative of COVID-19, due to infection either in the community or in the hospital, would be removed from duty.

Staff testing can be undertaken with either POCTs or laboratory-based tests. The characteristics of the tests are the same as those for patients. There is a delay in the staff test result becoming available in line with the SARS-CoV-2 test used (POCT or laboratory based). Those staff members with a positive SARS-CoV-2 test result would be removed from duty. When testing of asymptomatic staff is employed, the duration between tests is user-definable. It is assumed that if a staff member has a test result that is a failure they will wait until the next test period to receive a second test, assuming that they do not develop clinical symptoms suggestive of COVID-19 in the interim.

Length of stay in the hospital

For patients with COVID-19, the distribution of length of stay conditional on being discharged alive or dead is depicted in the appendix of Docherty et al. 9 It was assumed that for patients who died in hospital the distribution could be represented by a log-normal distribution with a mean of 2.00 and a standard deviation of 0.79, which give values resembling the mode and the percentage of people alive at 14 days. For patients alive at discharge, a distinction was made between those aged < 70 years and those aged ≥ 70 years. For the younger group a log-normal distribution with a mean of 1.95 and a standard deviation of 0.59 was assumed; and for the older group a log-normal distribution with a mean of 2.29 and a standard deviation of 0.59 was assumed.

For patients admitted to hospital without SARS-CoV-2 infection the average number of nights spent in hospital was 7.5. 8 For simplicity, it was assumed that this could be represented by a triangular distribution (1, 5, 12) for patients aged ≤ 69 years and a triangular distribution (6, 12, 18) for patients aged ≥ 70 years. For simplicity, it was assumed that these outcomes were independent of other outcomes (discharge, ICU care or death).

For patients admitted to ICU it was assumed that the length of stay was 2 days regardless of age, SARS-CoV-2 infection status, or outcome based on data presented to the NICE diagnostic committee. 32 The time in post-ICU beds was set at 4 days also independent of age and SARS-CoV-2 infection status.

Quality-adjusted life-years lost through death in the hospital

The model estimates the additional discounted quality-adjusted life-years (QALYs) lost assuming a discount rate of 3.5% per annum as recommended by NICE,4 and life-years lost through SARS-CoV-2 infections acquired within the hospital for admitted patients. These values have been estimated using 2016–18 data from Office for National Statistics life tables34 and age- and sex-adjusted general population utility values reported by Ara and Brazier,35 and take into account the death rates at each age by sex, the utility by age and sex, and the NICE recommended discount rate. Patients who are younger will therefore have greater expected QALYs than older patients as they are expected to both live longer and have a higher utility at the start of the model.

Example values of the estimated QALY losses and life-years lost for selected ages are provided in Table 10.

For patients who were infected with SARS-CoV-2 in hospital but were discharged alive, a multiplier can be applied to take into consideration that the QALYs gained would be lower than that of the average age- and sex-matched patients due to reduced life expectancy and/or reduced quality of life. No data were found to populate these multipliers; an arbitrary value of 0.8 was applied for patients who needed ICU care as a result of a SARS-CoV-2 infection acquired in hospital. For patients who did not require ICU care the multiplier was set to 1 to signify no loss in expected future QALYs.

The strategies modelled

Initially, 10 strategies were modelled, as shown in Table 11. All of these used the base-case values for the laboratory-based test and the POCT, as shown in Table 9. Further scenario analyses were run using the variations in Table 9, which included reduced times for the laboratory based-tests, using the acceptable TPP for POCTs and data from currently available POCTs. 27,30 It was assumed that there were sufficient testing kits, reagents and consumables for all SARS-CoV-2 tests. We did not assess any strategies that used confirmatory laboratory-based SARS-CoV-2 tests after a SARS-CoV-2 POCT.

Sensitivity analyses were undertaken on selected strategies to show the sensitivity of the model to changes in key assumptions. These are detailed in Sensitivity analyses applied to strategy 22, strategy 24 and strategy 25.

Further strategies were run that expanded on those outlined in Table 11. These, along with a description, are provided in Table 12; the analyses are not exhaustive but it is believed that inferences can be made for any strategies not explicitly run.

Calibration of the model

It is acknowledged that many parameter estimates are uncertain or have no data (e.g. effective contacts and attack rates). To attempt to provide meaningful results, the product of effective contacts and attack rates was calibrated. Carter et al. 36 reported that approximately 8.7% of the total patients diagnosed (i.e. those diagnosed at admission plus those diagnosed after admission) had nosocomial infections. We therefore set a target for the number of patients moved from non-SARS-CoV-2-infected bays to SARS-CoV-2-infected bays as approximately 8.7% of the sum of those patients identified as SARS-CoV-2 infected at admission and the numbers moved to SARS-CoV-2-infected bays in strategy 1, using 50 random number streams.

In the absence of data, the product of the number of effective contacts and attack rates per infectious person (θ₁) was assumed to be the same in each bay, with the exception that this was assumed to be 31% lower for symptomatic patients (see Effective contacts and attack rates). θ₁ was assumed applicable for patient-to-patient contact and staff-to-staff contact while waiting for a decision on admission, and for staff-to-staff contact for those working within the distinct areas of testing, discharge and informing patients of the admission decision. For the staff and patient contact during testing, discharge and informing the patient of the decision to admit to hospital or discharge, it was assumed that the product of effective contacts and attack rate was θ₁/100.

In the calibration analysis it was assumed that the base-case laboratory-based SARS-CoV-2 test was used and that there was no retesting following negative SARS-CoV-2 tests, but that all patients with clinical symptoms of COVID-19 and chest imaging suggestive of COVID-19 would be allocated to a SARS-CoV-2-infected bay. Following calibration, θ₁ was assumed to be 0.096 which estimated from 100 simulation runs that of the total number of identified COVID-19 cases 8.7% were identified after admission to a bay.

The number of different random number streams used per analysis

As with all individual-patient-based models, there is the possibility of ‘first-order’ uncertainty, which is the phenomenon that identical patients do not get identical outcomes because of the random numbers drawn to simulate whether or not a patient has an event of interest. The results for a given strategy will therefore change depending on the random number stream seed selected, as will the comparative results with alternative strategies/scenarios even when the same random number seed is used, as the allocation of random numbers will differ between the strategies/scenarios. To reduce the impact of random number streams, sufficient simulations, using different random number seeds, should be undertaken.

Analyses were undertaken to assess the variability in the mean results as the number of simulations were increased using a near-final version of the calibration model. Figure 2 depicts the number of patients infected in the hospital across each simulation and the average across all prior simulations and the current one.

FIGURE 2.

Assessment of how many simulations are required to stabilise the number of patients infected in hospital.

There is a trade-off between decreased model computation time and improved accuracy as more simulations with different random number streams are undertaken. Considering both aspects, we assumed that 50 random number streams would be sufficient to provide indicative results for number of infections in the hospital, which was believed to be an important outcome measure. For more rare events, such as critical events caused by SARS-CoV-2 infections, the presence of Monte Carlo sampling error (the differences in results purely due to the random numbers used) is significantly larger. To present meaningful results, regressions were undertaken of estimated QALYs lost through death in hospital as a result of SARS-CoV-2 infection and of estimated QALYs gained after COVID-19-related ICU stays caused by SARS-CoV-2 infection in hospital. The results of the regression were used in preference to the actual simulated results.

The use of net monetary benefit

As ratios of incremental costs divided by incremental QALYs, incremental cost-effectiveness ratios (ICERs) can be misleading when incremental QALY gains are low or when the estimated values are very close between two interventions and one is stated to be dominating the other. Incremental net monetary benefit, which is defined as the cost-per-QALY threshold multiplied by the incremental QALY gain minus the incremental cost,37 removes these limitations. For brevity, incremental net monetary benefit will be shortened to net monetary benefit (NMB) throughout this report. Within this framework, the largest NMB is associated with the most cost-effective strategy at the stated cost-per-QALY distribution, and multiple strategies can be compared simultaneously, as the absolute difference in strategies in terms of cost, having monetarised health differences, can be easily determined. These analyses have been undertaken at assumed cost-per-QALY thresholds of £20,000, £30,000 and £50,000, which are used in NICE technology appraisals,4 with the highest value used for treatments that meet end-of-life criteria.

The estimated number of infections per strategy

Figure 3 provides the modelled number of infections for each of the 30 strategies listed in Tables 11 and 12 for patients infected after a decision to admit to hospital. SARS-CoV-2 infections that occur in people not admitted to hospital have been excluded, as these are similar for all strategies and differ only as a result of Monte Carlo sampling error. The excluded infections total approximately 27 per simulation.

FIGURE 3.

The estimated number of infections per strategy.

Strategy 1, which was believed to have been used in many hospitals during the early response to SARS-CoV-2 infection, is seen to result in the largest number of infections. As expected, SARS-CoV-2 POCTs with the desirable TPP result in fewer infections than SARS-CoV-2 POCTs with the acceptable TPP, and tests with a quicker turnaround time result in fewer infections.

The estimated occupancy level in the infinite bays

As described in The configuration of the hospitals used in the example, the example hospital had infinite bays to acknowledge that patients would be treated somewhere in the hospital or transferred to an alternative facility. One possible advantage of SARS-CoV-2 POCTs is the potential for allocating patients more quickly to a SARS-CoV-2-infected bay or a non-SARS-CoV-2-infected bay, thus reducing occupancy pressure on the waiting bays. Figures have been produced that indicate the level of occupancy in the infinite bays; these are in Figure 4 for strategy 1 and in Figures 18 and 19 (see Appendix 1) for the remaining strategies. There are two things to note for these analyses. First, Simul8 does not have the capability to produce such figures using multiple random number streams and so a particular random number stream has been selected. This was the random number stream that produced the estimated amount of time in the waiting bays closest to the mean value across the 50 streams. Second, Simul8 provides the average occupancy across the full Simul8-modelled time horizon of 200 days, although patients stop entering the model after 90 days; the default occupancy level is, therefore, approximately half of the actual value, and an inspection of the underlying data is more informative.

Figure 4 shows the occupancy levels of the infinite bays among people waiting for a decision about allocation to a SARS-CoV-2-infected bay or a non-SARS-CoV-2-infected bay. Among SARS-CoV-2-infected patients, non-SARS-CoV-2-infected patients, SARS-CoV-2-infected patients post ICU and non-SARS-CoV-2-infected patients post ICU, none entered the infinitely sized ICU, so figures for these are not presented. For information, SARS-CoV-2-infected patients entered a fifth ICU bay, non-SARS-CoV-2-infected patients entered a fourth ICU bay, SARS-CoV-2-infected patients post ICU entered a sixth bay and non-SARS-CoV-2-infected patients post ICU entered a fourth bay. Occupancy levels in areas where people waited for the decision to be made about whether or not they will be admitted to hospital have not been presented as these levels would be unaffected by which test is used.

FIGURE 4.

The occupancy level in the infinite bay. (a) For patients with clinical symptoms suggestive of COVID-19 while waiting for a decision on whether or not to allocate to a SARS-CoV-2-infected hospital bay; (b) for patients with clinical symptoms not suggestive of COVID-19 while waiting for a decision on whether or not to allocate to a SARS-CoV-2-infected hospital bay; (c) for patients allocated to a SARS-CoV-2-infected bay; and (d) for patients allocated to a non-SARS-CoV-2-infected bay.

As the impact of the SARS-CoV-2 tests is likely to lead to differences in the occupancy of waiting bays before patients are allocated to a SARS-CoV-2-infected bay or a non-SARS-CoV-2-infected bay, only the occupancy levels in these two infinite bays are presented in Appendix 1 (see Figures 18 and 19). As anticipated, the tests with the shorter turnaround times resulted in lower occupancy levels in the waiting bays, with patients spending correspondingly more time in SARS-CoV-2-infected or non-SARS-CoV-2-infected bays. The desired TPP for SARS-CoV-2 POCTs (strategy 3) resulted in three single-bed bays being required for patients with clinical symptoms suggestive of COVID-19 and one single-bed bay being required for those without clinical symptoms suggestive of COVID-19. By contrast, strategy 1 required the infinite bays for both those with and those without clinical symptoms suggestive of COVID-19. As shown in Figure 4, among patients with clinical symptoms of COVID-19 the occupancy rate (labelled ‘work’ in the Simul8 output) in the infinite bay reached a maximum of 33, and among patients without clinical symptoms of COVID-19 the occupancy rate in the infinite bay reached a maximum of 7. To aid interpretation, the label of ‘time (time units)’ in the Simul8 output denotes the time in days.

Time spent in bays waiting to be allocated to a SARS-CoV-2-infected bay or a non-SARS-CoV-2-infected bay

An impact of different turnaround times for SARS-CoV-2 tests is differences in the time patients spend in waiting bays before being allocated to a SARS-CoV-2-infected bay or a non-SARS-CoV-2-infected bay, with times reduced in general bays on the assumption that the waiting time does not affect a patient’s overall time in hospital. The time spent in relevant bays is provided in Figure 5. Monte Carlo sampling error means that the total time in the bays is not identical for all strategies, although it is relatively similar, with the range between the highest and lowest summed value representing < 0.4% of the average total time.

FIGURE 5.

The days spent in bays awaiting a decision to be allocated to a SARS-CoV-2-infected bay or a non-SARS-CoV-2-infected bay and the days spent in a general bay.

Estimating the discounted quality-adjusted life-years lost due to death from SARS-CoV-2 infection

The model has the functionality to estimate the number of SARS-CoV-2 infections in hospital that result in a critical outcome (divided separately by the requirement for ICU care or by death). However, for rare events the Monte-Carlo sampling error is much greater and it was seen that 50 random number streams were not sufficient to remove this. To produce meaningful results a linear regression was undertaken to provide a general relationship between the number of infections and the number of deaths, as it was believed that the SARS-CoV-2 test employed would not influence the outcomes. A plot of infections and estimated QALY losses is provided in Figure 6, which uses as data points the results from the 30 strategies and the results from the sensitivity analyses detailed in Sensitivity analyses applied to strategy 22, strategy 24 and strategy 25, and in Appendix 2, which evaluates strategy 25 as well as strategy 22 and strategy 24. It is estimated that every SARS-CoV-2 infection in a hospitalised patient would result in 0.0604 QALY losses.

FIGURE 6.

Relationship between number of infections and simulated QALY losses due to SARS-CoV-2 infection among hospitalised patients.

It has been recently reported that if patients become co-infected with both SARS-CoV-2 and influenza then the odds of mortality increase by 2.27 (95% confidence interval 1.23 to 4.19) compared with SARS-CoV-2 infection alone. 38 These figures are presumed to refer to SARS-CoV-2 infection that becomes COVID-19. The model does not include the potential for increased risks following co-infection, which may represent a limitation.

The estimated QALY losses associated with each strategy are shown in Figure 7.

FIGURE 7.

Estimated QALY losses per strategy due to SARS-CoV-2 infection in hospitalised patients.

Estimating the discounted quality-adjusted life-years lost owing to requirement of intensive care unit care for SARS-CoV-2 infection

The model has the functionality to estimate the number of QALYs that are accrued for patients after they are discharged alive from ICU. It is anticipated that, following ICU care, the length and/or quality of life of these patients may be impaired compared with that of patients who have not had ICU care. An arbitrary QALY multiplier of 0.9 has been incorporated to take into account the loss of QALYs following ICU care, loosely based on the impact observed by Cuthbertson et al. 39 at 5 years for patients who had a severe sepsis episode. There was noticeable Monte Carlo sampling error for these values and, as with QALYs lost from death, a relationship was estimated using linear regression. The regression model fitted is shown in Figure 8 and allows the number of QALYs gained after ICU to be calculated from the estimated number of SARS-CoV-2 infections.

FIGURE 8.

Relationship between number of infections and simulated QALY losses due to SARS-CoV-2 infection in hospitalised patients.

The estimated QALYs gained post ICU for each strategy have been multiplied by 0.1 to obtain an estimate of the number of QALY losses caused by additional requirement for ICU care. These data are presented in Figure 9.

FIGURE 9.

Estimated number of QALYs lost due to the requirement of additional ICU care in patients infected with SARS-CoV-2 in hospital.

Estimating the number of tests performed, by strategy

The numbers of tests undertaken for each strategy are shown in Figure 10. These are divided into the number of tests undertaken before allocation to a SARS-CoV-2-infected bay or a non-SARS-CoV-2-infected bay and the number of routine retests in non-SARS-CoV-2-infected bays or in staff, as some strategies use laboratory-based SARS-CoV-2 tests for one group and SARS-CoV-2 POCTs for the other group.

FIGURE 10.

The estimated number of tests performed, by strategy.

Estimating the costs of tests performed, by strategy

The estimated costs of tests undertaken for each strategy are shown in Figure 11. These are divided into the costs of tests prior to allocation to a SARS-CoV-2-infected bay or a non-SARS-CoV-2-infected bay and the costs of routine retests in non-SARS-CoV-2-infected bays or in staff, as some strategies use laboratory-based SARS-CoV-2 tests for one group and SARS-CoV-2 POCTs for the other.

FIGURE 11.

The estimated costs of tests performed, by strategy.

Estimating the costs of additional intensive care unit requirement

The model calculated the time spent in ICU with each strategy, as some patients infected with SARS-CoV-2 may require ICU care when they did not previously. Analyses were performed looking at the relationship between the number of infections in hospitals and the increased time associated with ICU care. The relationship was positive but very weak (R² < 0.001) and so these costs have been omitted, which may be unfavourable to more accurate SARS-CoV-2 tests.

Estimating the cost-effectiveness of the strategies

The model calculated the estimated QALY losses for each strategy. These need to be transformed into QALY gains to be interpreted using the usual method. As a result, the QALY losses have been subtracted from a constant number (50); this does not affect the differences between strategies and hence the ICER and NMB values are not affected. The costs of testing and the transformed QALYs associated with each strategy are shown in Table 13.

A full incremental analysis has been performed providing ICERs (in terms of the incremental cost per QALY gained) for those strategies on the efficiency frontier. The efficiency frontier includes only those points that are not dominated (dominated: providing equal or fewer QALYs at higher cost, or fewer QALYs at an equal or higher cost) by a single strategy, or that are not extendedly dominated. A strategy is extendedly dominated when a combination of two other strategies results in combined cost and QALY values that would dominate the initial strategy.

However, the full incremental analyses have little meaning, as the desirable TPP for a POCT will always dominate a POCT with an acceptable TPP and yet these may not be choices available to the decision-maker. The use of a NMB approach may be preferable, as is detailed.

As ratios of incremental costs divided by incremental QALYs, ICERs can be misleading when incremental QALY gains are low or when the estimated values are very close between two interventions and one is stated to be dominating the other. NMB is helpful for removing these limitations, although it requires a threshold for the cost-per-QALY ratio to be stated, which is a decision made by the NICE committee. NMB values are presented here using threshold values of £20,000, £30,000 and £50,000 per QALY gained for all strategies compared with strategy 1. These supplementary data are presented in Figure 12. The NMB values are those associated with all 22,500 patients entering the hypothetical hospital during the 90-day period.

FIGURE 12.

The NMB associated with each strategy at stated cost-per-QALY thresholds compared with strategy 1.

It is cautioned that the cost-effectiveness analyses should not be taken in isolation. In addition to the uncertainty within the model, the characteristics of the POCT or laboratory-based SARS-CoV-2 tests may not exist or may not be available, and some strategies may not be feasible because of the configuration of the hospital. The data provided in The estimated occupancy level in the infinite bays, Time spent in bays waiting for allocation to a SARS-CoV-2-infected bay or a non-SARS-CoV-2-infected bay and Appendix 1 should be considered alongside the cost-effectiveness output.

In addition, for each of the 30 strategies evaluated in the main report, data were extracted for patients discharged after admission to hospital on the estimated numbers of people in three groups: ‘correct isolation’, ‘missed isolation’ and ‘unnecessary isolation’. Respectively, these groups comprised patients who had SARS-CoV-2 infection and were correctly told this information; had SARS-CoV-2 infection and were not told this information, because they had either a negative test result or a failed test result that was not repeated; and did not have SARS-CoV-2 infection but were told that they did.

The missed isolation group represents people who should be self-isolating but were told that this was not required, whereas the unnecessary isolation group represents people who were told that they should self-isolate even though this was not required. The estimated results are presented in Figure 13. These values include the results of SARS-CoV-2 tests that become known after the patient has been discharged, with the results conveyed to the patient when they are at home. It is seen that strategies that use the acceptable TPP for SARS-CoV-2 POCTs (strategies 13, 16, 24 and 27) are associated with increased numbers of unnecessary isolations as a result of the lower specificity associated with this TPP. This may have negative impacts if there are other pressures on isolation bays.

FIGURE 13.

Data on the number of correct isolations, missed isolations and unnecessary isolations by strategy.

Sensitivity analyses undertaken

There is considerable uncertainty relating to the parameters within the model. This has been explored in sensitivity analyses, although these were constrained by the deadlines for the report. Where additional runs of the model were necessary, three strategies that may be plausible were selected: strategy 22, in which patients and staff are tested using a laboratory-based SARS-CoV-2 test that takes 24 hours, was compared with strategy 24, which differs from strategy 22 as a SARS-CoV-2 POCT with an acceptable TPP was used rather than the 24-hour laboratory test, and strategy 25, in which the laboratory-based test was replaced with data from currently available SARS-CoV-2 POCTs. The change in the difference in key outcomes between the strategies may be informative to the committee.

A further set of sensitivity analyses were run, removing the testing of asymptomatic staff using strategy 2 and strategy 14 to provide indicative results. Strategy 2 assumes a laboratory-based SARS-CoV-2 test with a turnaround time of 35.9 hours, whereas strategy 14 uses data from currently available SARS-CoV-2 POCTs. The change in the difference in key outcomes between the strategies may be informative to the committee. As turnaround time was a key element of the decision problem, additional sensitivity analyses were undertaken that varied the turnaround time of the laboratory-based SARS-CoV-2 test in strategy 2 and of the SARS-CoV-2 POCT in strategy 14.

A final set of sensitivity analyses were run that explored the estimated number of infections if SARS-CoV-2 testing were not possible.

When interpreting the results, it should be noted that the model was not recalibrated for all sensitivity analyses described here. However, the model was recalibrated for three sensitivity analyses: (1) the relative product of attack rate and effective contacts for asymptomatic patients to that of symptomatic patients, (2) where the time before a patient became non-infectious was increased and (3) where it was assumed that the laboratory-based SARS-CoV-2 test had a turnaround time of 21.3 hours. Recalibration was necessary because (1) these were not believed to be changes that have occurred since the early stages of the pandemic but would have been the values throughout the simulation and (2) these were shown to significantly affect the results when the model was not recalibrated. For the reduction in asymptomatic product of attack rates and effective contacts, θ₁ was 0.1285; this value was 0.0759 when the time to becoming non-infectious was increased and 0.0885 when the turnaround time of the laboratory-based SARS-CoV-2 test was assumed to be 21.3 hours. θ₁ increasing when the turnaround time of a laboratory-based test was decreased was unexpected. On inspection, this could be explained as the model being calibrated based on those who received a positive SARS-CoV-2 test while in hospital. When the test took 35.9 hours, more patients had been discharged from hospital at the time of the result being known than if the SARS-CoV-2 test took 24 hours. Only patients in hospital at the time of the test result becoming available contributed to the calibration.

Sensitivity analyses applied to all strategies

The following scenario analyses could be applied after the model runs: setting the cost of laboratory tests equal to the costs of POCTs; increasing the multiplier for QALYs gained post ICU to 1; decreasing the multiplier for QALYs gained post ICU to 0.8; and reducing the proportion of SARS-CoV-2 infections that resulted in critical outcomes by 90%.

Setting the costs of laboratory-based SARS-CoV-2 tests equal to the costs of SARS-CoV-2 point-of-care tests

Characteristics of the point-of-care tests and laboratory-based tests details that the costs assumed per test were £27.76 for a POCT and £21.90 for a laboratory-based test. Sensitivity analyses have been performed setting both costs equal to £27.76. As anticipated, the results are more favourable to POCTs.

The NMB analyses in this scenario are shown in Figure 14. The efficiency frontier becomes strategy 1, strategy 3 (£3520) and strategy 8 (£49,702).

FIGURE 14.

The NMB compared with strategy 1 when the costs of SARS-CoV-2 laboratory testing are set equal to the costs of SARS-CoV-2 POCTs.

Amending the multiplier associated with quality-adjusted life-years gained post intensive care unit care caused by a SARS-CoV-2 infection

The multiplier used post ICU was 0.9, although this was arbitrary (see Estimating the discounted quality-adjusted life-years lost owing to requirement of intensive care unit care for SARS-CoV-2 infection). Analyses have been run assuming that this increased to 1 (indicating that ICU had no effect on length or quality of life after discharge) and decreased to 0.8.

When the multiplier was 1, the strategies on the efficiency frontier remained strategy 1, strategy 12 (£10,247), strategy 23 (£35,171), strategy 9 (£62,300) and strategy 8 (£5,984,563). The NMB for this sensitivity analysis is provided in Figure 20 (see Appendix 2).

When the multiplier was 0.8, the strategies on the efficiency frontier remained strategy 1, strategy 12 (£8028), strategy 23 (£27,333), strategy 9 (£48,805) and strategy 8 (£4,688,296). The NMB for this sensitivity analysis is provided in Figure 21 (see Appendix 2).

Reducing the proportion of patients infected with SARS-CoV-2 who have a critical outcome

The chance of a patient being infected with SARS-CoV-2 in hospital and now requiring ICU care or dying was detailed in The probability of patients infected with SARS-CoV-2 in hospital having a worse prognosis, with a value of 8.9% used in the base case. This value was based on many assumptions, including the proportion of SARS-CoV-2 infections that are symptomatic, which may be overestimated if there are a large number of unreported symptomatic cases, or if fatality rates have improved due to treatment enhancement since SARS-CoV-2 was identified. A sensitivity analysis was performed that reduced the risk of a critical outcome by 90%. It is seen (Figure 15) that this markedly changed the results, with strategy 1 having the highest NMB as it was the cheapest in terms of testing costs. The elements on the efficiency frontier were the same, but the ICERs were 10 times higher: strategy 1, strategy 12 (£90,025), strategy 23 (£308,993), strategy 9 (£547,329) and strategy 8 (£52,577,110).

FIGURE 15.

The NMB compared with strategy 1 when the risks of critical outcomes are reduced by 90%.

Sensitivity analyses on the turnaround time of SARS-CoV-2 tests applied to strategy 2 and strategy 14

Additional sensitivity analyses were undertaken specifically changing the time to test result for strategies 2 and 14. These represent laboratory-based SARS-CoV-2 tests and SARS-CoV-2 POCTs using data on diagnostic accuracy from currently available SARS-CoV-2 POCTs. The turnaround times evaluated, along with the estimated number of infections associated with each time, are shown in Table 16. The rationale for the analysis is that the time to get a laboratory-based SARS-CoV-2 test result will differ markedly across hospitals; POCTs are likely to have uncertain queueing times.

TABLE 16 - Number of infections associated with different turnaround times for strategy 2 and strategy 14

Laboratory-based SARS-CoV-2 test		SARS-CoV-2 POCT
Turnaround time (hours)	Number of infections	Turnaround time (hours)	Number of infections
4	175	0.5	177
6 (strategy 12)	188	1	175
12	209	2	182
16	221	3.8 (strategy 14)	192
24 (strategy 11)	227	6	197
35.9 (strategy 2)	230	8 (strategy 15)	207

A slight Monte Carlo sampling error is seen in the reduction in the estimated number of infections after hospital admittance when the turnaround time of the SARS-CoV-2 POCT was increased from 30 minutes to 1 hour.

The number of infections was used to estimate QALY losses, with the costs of testing coming from the model. It should be noted that these costs are lower than for strategies 22, 24 and 25 because of the lack of testing of asymptomatic patients.

A plot of the estimated testing costs of each strategy and the QALY losses subtracted from 50 for different turnaround times is provided in Figure 16. Test costs are lower in scenarios when the turnaround time is longer, as a higher proportion of patients would be discharged (or would have died) prior to retesting in non-SARS-CoV-2-infected bays.

FIGURE 16.

The estimated testing costs and QALY gains of each strategy under different assumptions related to turnaround times. Legend represents type of test (L, laboratory based; P, POCT) and time in hours to get the result.

The ICER for each pairwise comparison of laboratory-based SARS-CoV-2 tests and SARS-CoV-2 POCTs is provided in Table 17. It can be seen that the cost-effectiveness of SARS-CoV-2 POCTs reduces as the turnaround time between a laboratory-based SARS-CoV-2 test and a SARS-CoV-2 POCT decreases.

TABLE 17 - Pairwise comparison ICERs for laboratory-based SARS-CoV-2 tests and SARS-CoV-2 POCTs

ICERs are for SARS-CoV-2 POCTs compared with laboratory-based SARS-CoV-2 tests.

		Turnaround time for SARS-CoV-2 POCT (hours)
		0.5	1	2	3.8	6	8
Turnaround time for laboratory-based SARS-CoV-2 test (hours)	4	Dominated	Dominated	Dominated	Dominated	Dominated	Dominated
	6	£58,257	£48,545	£96,601	Dominated	Dominated	Dominated
	12	£21,331	£19,869	£24,288	£38,441	£55,315	£291,517
	16	£16,152	£15,306	£17,616	£23,499	£28,601	£45,541
	24	£15,365	£14,643	£16,565	£21,166	£24,874	£35,127
	35.9	£15,750	£15,057	£16,905	£21,202	£24,544	£33,201

Sensitivity analyses exploring the impact of no testing

Sensitivity analyses were undertaken exploring the results in the situation that it was not possible to undertake SARS-CoV-2 testing. Two strategies were tested, denoted strategy 31 and strategy 32. In strategy 31 patients were assumed to be allocated to bays based on clinical symptoms alone, whereas in strategy 32 patients were allocated to SARS-CoV-2-infected bays if they had clinical symptoms and chest imaging results suggestive of COVID-19, with all other patients going to non-SARS-CoV-2-infected bays. These strategies have more infections than those detailed in Tables 11 and 12, with 284 infections for strategy 31 and 293 infections for strategy 32. However, these strategies would not incur the costs of SARS-CoV-2 testing and would also not necessitate waiting before test results were known. Figure 17 provides the NMB at different thresholds of willingness to pay per QALY for strategies 31 and 32, and other relevant strategies: strategy 3 (a SARS-CoV-2 POCT with the desirable TPP), strategy 11 (a laboratory-based SARS-CoV-2 test with a turnaround time of 24 hours), strategy 13 (a SARS-CoV-2 POCT with the acceptable TPP) and strategy 14 (using data from currently available SARS-CoV-2 POCTs). All strategies are compared with strategy 1, the strategy assumed to be in place at the start of the pandemic.

FIGURE 17.

The estimated NMB of selected strategies assuming two no-testing strategies.

It is seen that, at low willingness-to-pay values (≤ £30,000 per QALY), no testing appears to be one of the options with the highest NMB values, owing to the high costs associated with testing. As the willingness-to-pay threshold increases to £50,000 per QALY, these strategies compare unfavourably with other strategies. This indicates that the cost of SARS-CoV-2 tests is, as expected, a key driver of the cost-effectiveness of SARS-CoV-2 testing.