The Role of Ultrasound Compared to Biopsy of Temporal Arteries in the Diagnosis and Treatment of Giant Cell Arteritis (TABUL): a diagnostic accuracy and cost-effectiveness study

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 08/64/01. The contractual start date was in January 2010. The draft report began editorial review in June 2015 and was accepted for publication in July 2016. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Raashid Luqmani received honoraria from GlaxoSmithKline (GSK), Nordic and Chemocentryx for training in the use of the Birmingham Vasculitis Activity Score and Vasculitis Damage Index, and personal fees from Roche outside the submitted work. Raashid Luqmani received grants from Fundação para a Ciência e Tecnologia (Portugal), Canadian Institute of Health Research, Arthritis Research UK, Patient Centered Outcomes Research Institute, Oxford University Hospitals NHS Trust Innovation Challenge Competition and Vasculitis UK. Raashid Luqmani has patents pending for a mechanical arm to automate acquisition of ultrasound images and analysis for reviewing ultrasound images. Bhaskar Dasgupta received personal fees from GSK, Servier, Roche, Merck, and Mundipharma and grants from Napp outside the submitted work. Andrew Hutchings was funded by a Medical Research Council special training fellowship in health services research during the development of the study. Jennifer Piper has a patent pending for an ultrasound arm.

Copyright statement

© Queen’s Printer and Controller of HMSO 2016. This work was produced by Luqmani et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

General introduction to giant cell arteritis

Giant cell arteritis (GCA), also known as temporal arteritis, is a common form of vasculitis that affects people typically aged > 50 years. 1 GCA often progresses rapidly and, if left untreated, leads to severe pain, permanent visual loss, stroke and, in some cases, death. The incidence is approximately 220 per million per year in the UK in people aged ≥ 40 years. 2 Elsewhere, the incidence varies across the world, with published figures ranging from 150 to 250 new patients per million per year. It is more common in northern European countries, particularly in Scandinavia (313 per million per year in people aged > 70 years)3 and in Minnesota, USA, which has a large Scandinavian-origin population (198 per million per year),4 and it is much less common in other parts of the world such as Japan, China and Australia.

Rapid diagnosis and glucocorticoid treatment are recommended,5 but both are problematic. Glucocorticoid treatment is usually started before a formal diagnosis is made, meaning that a proportion of patients are treated unnecessarily and are thereby exposed to side effects including weight gain, altered body habitus, hypertension, infection, osteoporosis, cataract, mood swings and thin skin. Glucocorticoid treatment also affects the accuracy of the diagnosis. The heterogeneous nature of GCA means that its diagnosis is not straightforward, but is usually based primarily on temporal artery biopsy (TAB) and supported by presenting symptoms. Glucocorticoids, by their nature, impact on inflammation; if there is a large time difference between commencement of glucocorticoids and biopsy, this reduces diagnostic accuracy. Although a positive biopsy usually (although not always) confirms GCA, the sensitivity of TAB has been estimated to vary from 39% to 91%,6,7 resulting in a large number of false negatives in the screened population. This has led to high-dose glucocorticoid therapy being continued as a precaution (in case the patients actually have GCA), even in the absence of a positive biopsy.

Ultrasound and other forms of imaging compared with the traditional role of biopsy

An alternative to biopsy has been the development of ultrasound and other imaging techniques for the diagnosis of GCA. Imaging first emerged in the 1990s as a potential means by which to provide evidence to support a diagnosis of GCA. 8–15 High-resolution magnetic resonance imaging (MRI) of temporal arteries offers a non-invasive technique for investigating suspected GCA, but it is limited by availability and cost. Ultrasound is the most practical and widely used modality. Three meta-analyses have supported the role of ultrasound in the diagnosis of GCA. 16–18 The presence of bilateral ultrasound abnormalities (both temporal arteries involved) provides high specificity (100%) for the diagnosis of GCA, but its sensitivity was 43%. 17 Two of the meta-analyses reported concerns with the quality of the included studies16,18 and the third did not assess the methodological quality of the included studies. 17 Currently, the use of ultrasound as a diagnostic tool for GCA is relatively limited, perhaps as a result of practical reasons relating to training to use ultrasound or equipment availability to facilitate rapid access and evaluation of patients with suspected GCA.

Ultrasound examination of temporal arteries is non-invasive and there is no ionising radiation involved. Furthermore, it can provide information about the entire length of both temporal arteries. Additional examination of the axillary arteries improves the sensitivity of ultrasound19 because some individuals with GCA (especially those without headaches) will have isolated abnormalities in the axillary arteries but not in the temporal arteries. This may be because of the longer persistence of scan abnormalities in larger vessels than in temporal arteries, despite the use of steroids. The chief abnormality on ultrasound that suggests the diagnosis of GCA is a halo, which is defined as a dark hypoechoic area around the vessel lumen and is thought to represent inflammatory change and oedema present in the wall and surrounding tissues of the affected blood vessel.

The role of temporal artery biopsy in the diagnosis of giant cell arteritis

A recent study of biopsy-proven GCA disease in South Australia suggested an incidence in people aged > 50 years of only 32 per million per year,20 although the relatively low incidence may be a result of the inclusion of biopsy-confirmed cases only. Biopsy for the diagnosis of GCA has a relatively low yield. 16 The difficulty in diagnosis of GCA, which forms the main underlying question of this project, is the lack of a high-quality gold standard test. Although biopsy is reported to be the current gold standard test for diagnosis, the majority of patients in whom a diagnosis of GCA is suspected do not actually have a positive result. This may reflect the fact that there is a lower index of suspicion for diagnosis, and, therefore, more people with headaches are being evaluated for GCA; equally, it may reflect the relatively poor association between the true multivessel disease of GCA and the TAB findings to support a diagnosis of GCA.

The spectrum of different forms of giant cell arteritis

In about 50% of patients with GCA, branches of the aorta, and even the aorta itself, may be involved, suggesting that there is probably much more widespread inflammation of blood vessels (vasculitis) than previously considered. 21 In a study of 120 patients with large vessel vasculitis and 212 with more conventional cranial symptoms of GCA, but without the evidence of large vessel disease, patients with large vessel disease were significantly younger, by about 7 years, and had longer duration of symptoms prior to diagnosis (3.5 months compared with 2.2 months). There was a strong association with pre-existing polymyalgia rheumatica (PMR) in 26% of patients, compared with 15% of patients with cranial GCA, and fewer cranial symptoms (41% of patients, compared with 83% of patients with cranial GCA). Visual loss was also much less likely in large vessel GCA (4% compared with 11%). The risk of relapse of GCA features was higher in patients with large vessel disease than in patients with cranial manifestations only (4.9/10 person-years, compared with 3/10 person-years) and these patients were likely to require higher doses of steroids for longer periods of time.

Clinical presentation of giant cell arteritis

Recognising new features of GCA can be very straightforward in a patient with no previous history of headache who suddenly develops unaccustomed discomfort on the side of the head with swelling or tenderness of the temporal arteries, general systemic onset and scalp tenderness. Although symptomatic headache is very troublesome, the most feared complication of GCA is neuroischaemic damage, which can result in inflammation and occlusion of small branches of the cranial arteries (including the posterior ciliary artery and the ophthalmic artery) and ultimately in permanent visual loss. A warning symptom of ischaemia is the presence of jaw or tongue claudication, typically reported by around 50% of patients with GCA at presentation. 22 Jaw and tongue claudication refers to discomfort in the patient’s masseter muscles or tongue which stops them from eating or talking. When they stop to rest their jaw or tongue, the pain resolves because it is a result of claudication of those muscles as a result of narrowing of the blood vessel supply (e.g. the facial artery and its branches). Patients who have tongue or jaw claudication are at risk of blindness because the disease can involve the posterior ciliary arteries, which supply the retina and cause unilateral, and occasionally bilateral, permanent sight loss. Sometimes the visual loss starts on one side and subsequently becomes bilateral. In an early series of 90 cases from neurology and ophthalmology clinics, up to 60% of patients presented with permanent loss of vision attributable to either ischaemic papillopathy or retinal artery occlusion;23 other cases presenting primarily to physicians demonstrated a much lower risk of sight loss of 7.4% to 19.1%. 24–27 The risk of sight loss associated with GCA has fallen in recent decades, from 15% of cases with ischaemic optic neuropathy between 1950 and 1979 to only 6% between 1980 and 2004. 28 There is a small but significant risk of stroke, highlighted as possibly being between 2.8% and 6% in two recent studies. 29,30 Therefore, an early diagnosis and initiation of immunosuppressive treatment with high doses of steroids is required, making the condition a medical emergency.

It is likely that the mechanisms driving the neuroischaemic complications are different from those driving the systemic inflammatory response. There is a suggestion that interleukin (IL)-12 and interferon-gamma are the main cytokines responsible for myointimal proliferation leading to vessel occlusion; by contrast, the mechanisms driving the systemic inflammatory response are likely to be IL-6 and IL-17. 31 The underlying pathological changes involve invading macrophages and lymphocytes which gain access to the blood vessels via the vasa vasorum. They generate a local inflammatory response in the blood vessel wall, starting in the adventitia, migrating through to the media and intima, with proliferation of the internal elastic lamina, intimal proliferation and swelling, and eventually resulting in vessel narrowing and complete occlusion in some cases.

Although intimal proliferation with intimal thickness and internal elastic lamina reduplication are typical features, the hallmark histological finding is the presence of multinucleated giant cells, hence the term GCA. These pathological mechanisms are the basis for the histological diagnosis of the condition, which was first recognised by Horton et al. 1 in 1932 who described two patients who were initially thought to have a fungal infection (actinomycosis) of the temporal arteries.

Giant cell arteritis typically affects people aged over 50 years; it is two to three times more common in women than in men. PMR is a related clinical syndrome characterised by generalised muscles aches and pains. PMR is common in patients over the age of 50 years and presents with widespread aches and pains, particularly involving proximal muscles. Criteria for classifying PMR are based on the presence of bilateral limb girdle discomfort, early-morning stiffness and an elevated inflammatory response. 32 Additional ultrasound evidence of bursitis around the hips improves the specificity of the criteria from 78% to 81% and maintains sensitivity of between 66% and 68%. PMR can be present in up to 50% of individuals with GCA and it may occur either before, during or after the manifestations of GCA appear, suggesting significant overlap between these two disease processes. 33 Therefore, our interpretation of any individual patient’s diagnosis of GCA would be influenced by either pre-existing or concomitant diagnosis of PMR or might be validated by subsequent development of PMR.

Diagnosis and classification of giant cell arteritis

The 1990 American College of Rheumatology (ACR) classification criteria for GCA are based on the following:

• aged at least 50 years

• new onset of headache

• temporal artery abnormality on physical examination

• elevated erythrocyte sedimentation rate (ESR) typically ≥ 50 mm/hour

• abnormal TAB showing features of vasculitis.

Classification of a patient as having GCA requires at least three of these criteria to be present. 34 The classification criteria are not diagnostic tests and are limited by the technology available at the time when the criteria were being developed. As technology has improved, there are more sophisticated methods available for evaluating the temporal artery with ultrasound, MRI and computerised tomography (CT). Furthermore, it is possible to image the whole arterial tree more effectively for evidence of widespread vascular abnormality using CT angiography, magnetic resonance angiography and ¹⁸F-fluorodeoxyglucose positron emission tomography CT. These techniques have revealed that some cases of GCA have much more extensive vessel involvement than previously suspected. 21 Imaging has demonstrated that GCA can present without headaches but with other features such as constitutional symptoms and polymyalgia, which is also termed polymyalgia arteritica. 35

The awareness of GCA has probably increased and it is likely that the concern regarding the threat of visual loss may affect a clinician’s decision to pre-emptively treat any patient who might have the condition as soon as possible, in order to prevent these complications from occurring. As a result, it is likely that we are starting to see a change in the level of suspicion of symptoms at which a clinician is confident in starting treatment on the basis of a presumed diagnosis of GCA. Tests used for diagnosing GCA would now be performed in different circumstances than existed previously. We may be dealing with milder cases of the disease and/or more patients with a GCA-like symptom complex who do not actually have GCA. If these patients are given steroids, the standard test result from a TAB may be significantly influenced by the fact that the biopsy was performed on mild disease that had already been partly treated and/or was performed in patients who do not actually have GCA. Kisza et al. 36 assessed over 700 cases of GCA from 1994 to 2011, with 215 biopsy-positive cases, observing a peak incidence in 1996. Machado et al. 37 observed a reduction in the frequency of patients presenting with classical features, but no change in the likelihood of a positive biopsy from 1950 to 1985. In fact, Gonzalez-Gay et al. 38 found that the incidence of biopsy-proven GCA actually increased from 1981 to 2005. A more recent study3 of 840 biopsy-positive cases of GCA in Sweden reported a reduction in incidence between 1997 and 2010, from 15.9/100,000 to 13.3/100,000, although this contrasts with an earlier Swedish study39 that reported an increased incidence from 1976 to 1995, especially in women. In the UK, there was no evidence to suggest a change in incidence between 1990 and 2001. 2 This suggests that, although there may have been some changes to the epidemiology of GCA over time, with possibly a rise in incidence in women, there has been no significant change in the overall incidence of GCA. There is evidence of a diagnostic shift in other diseases too; hypothyroidism is now recognised as significant and is associated with increased comorbidity at lower levels of thyroid-stimulating hormone than before. 40

Difficulty with diagnosis of giant cell arteritis based on the gold standard temporal artery biopsy

For clinicians managing patients who may have GCA, untreated disease can result in permanent visual loss (as discussed in González-Gay et al. 24) and the condition is therefore considered to be a medical emergency. However, there are far more people with headache (it is an almost universal experience) than there are patients with GCA.

Toxicity of treatment versus need for urgent treatment

The other main consideration is that treatment for GCA, which involves high doses of glucocorticoids such as prednisolone over a prolonged period and which will result in rapid control of the inflammatory process and reduce the risk of ischaemic manifestations, is very toxic and results in side effects in over 80% of patients. 42 The most common side effects reported in the study by Proven et al. 42 included cataracts in 41% of patients, fractures in 38% of patients, infection in 31% of patients, hypertension in 22% of patients, diabetes mellitus in 9% of patients and gastrointestinal bleeding in 4% of patients.

With modern therapy, such as the prophylactic use of calcium, vitamin D and bisphosphonates to prevent fractures, some of these complications can be avoided. Further measures to reduce risk of treatment-related toxicity include better control of hypertension and diabetes mellitus, as well as prophylactic use of proton pump inhibitors to prevent gastrointestinal bleeding (which could relate to previous use of high doses of non-steroidal anti-inflammatory drugs combined with high doses of prednisolone). The risk of serious infections remains significant and has been estimated to be 55% higher than in age- and sex-matched controls. 43

Therefore, the balance of risk versus benefit in a patient with suspected GCA rests heavily on our ability to be confident that the diagnosis is correct. A patient who is incorrectly diagnosed with GCA will be subjected to significant risk of steroid toxicity without experiencing any advantage. However, if the patient does have GCA but the diagnosis was not made and the patient was not established on high doses of steroids, then there is a significant risk of ischaemic complications, including permanent visual loss or stroke, which are the most important complications of the disease and which makes sight loss (and other acute ischaemic complications) from GCA a preventable medical emergency.

Diagnosis of giant cell arteritis relying on a gold standard of temporal artery biopsy

Since 1932 the conventional gold standard investigation for GCA has been a TAB. 1 The characteristic finding of histiocytes, epithelioid and giant cells (large multinucleated cells present in the arterial wall) at the intimal–medial junction is useful in diagnosis,41 but not always present (e.g. giant cells were found in 75% of positive biopsies in a recent series). Other pathological features include transmural inflammation, adventitial infiltrates or localised infiltrates of inflammatory cells, especially lymphocytes in the media or intima. Reduplication of the internal elastic lamina and fragmentation of the internal elastic lamina are also described. Intimal cellularity and increased thickness can occur and, in a number of cases, the vessel lumen is narrowed to occlusion with associated thrombus formation.

Most patients have headache, which on closer questioning is localised around the temporal artery and is usually worse on one side than the other. The most symptomatic artery is usually selected for biopsy and is most likely to show evidence of pathological findings. In some centres, it has previously been a routine procedure to sample both temporal arteries in suspected cases, but the value of bilateral testing is relatively low,44 with only one of 91 bilateral biopsies showing discordance. In a recent study of 132 cases undergoing bilateral biopsies, the diagnostic yield increased by 12.7%45 as a result of the second simultaneous biopsy (38 patients had bilateral findings of GCA, compared with an additional 13 patients whose biopsies showed abnormalities confined to one side only).

The purpose of high doses of glucocorticoid therapy is to resolve inflammation. Therefore, the characteristic findings of cellular infiltration of the vessel wall with lymphocytes and giant cells may have disappeared by the time the biopsy is performed if there is a significant delay between starting treatment and obtaining the biopsy. Because of the ‘clock ticking’ as a result of glucocorticoids being administered as a precautionary measure (in case the patient really does have GCA), it is not usually helpful to perform a second biopsy of the opposite (and possibly asymptomatic) artery if the first biopsy is negative for patients in whom there is a suspicion of GCA. A biopsy from the opposite artery is feasible but is less likely to have a positive result, unless the patient has active symptoms of GCA in the artery to be biopsied. Cellular infiltration is the most important histological finding but can potentially resolve within 7–10 days of commencing high-dose glucocorticoid therapy. 46 Therefore, in some patients, biopsy evidence for GCA is inadequate. Many of the changes seen in the intima and internal elastic lamina can also be found in older people who do not have any features of GCA. 47 A recent surgical series of 237 patients undergoing TABs reported positive findings of GCA in only 36 (15.1%) cases48 and the result of the biopsy did not significantly contribute to the diagnosis. Changes suggestive of GCA are not consistently present throughout the course of the vessel.

The biopsy may not actually contain any arterial tissue. Nerves or veins were sampled in error in 14 of 567 consecutive biopsies (2.5%). 49 Biopsies of temporal arteries are typically sectioned transversely to provide an overall assessment of the artery. If the pathological abnormalities are present in the areas of artery that have not been cut, it is possible to miss the relevant findings. If the biopsy length is small, the characteristic histological features, which may occur sporadically along the length of the tissue obtained (skip lesions), may be missed. The biopsy is typically sectioned in cross-section and it is possible that, if the material obtained is quite small, only a few cross-sections will be available to view. If the abnormalities to be detected are not seen in these cross-sections, the interpretation would be that the biopsy was normal. However, it is possible that if a longer specimen had been obtained and more cross-sections had been viewed then the pathological changes might have been evident. Obtaining specimens that have been subjected to more sections increases the diagnostic yield slightly but leads to significantly more work and expense for the pathology laboratory. 50

Biopsy length (after fixation) varies in different studies. Shrinkage is well recognised, with a recent study of 62 biopsies showing an average of 4.6 mm of shrinkage from the time of surgical excision to fixation. 51 A study of 966 biopsies from six different hospitals suggested that a length of at least 0.7 cm increased the diagnostic yield from 12.9% to 24.8% positive results. 52 By contrast, another study of 151 biopsies from 149 patients yielded 20 positive biopsies (13.3%), and there was no difference in the length of positive (mean 0.7 cm) compared with negative (mean 0.65 cm) biopsies. 53 The British Society for Rheumatology (BSR) guidelines recommend between a 1- and 2-cm length of artery to provide an adequate specimen, usually from only the symptomatic or most symptomatic side. 5

The presence of inflammatory infiltrates in the vasa vasorum was reported in 6.5% of 354 biopsies considered positive in one large study of patients with clinical features of GCA. 54 However, it remains controversial whether or not these findings, as well as some of the other ‘characteristic findings’ suggesting GCA, may in fact occur in patients with other forms of vasculitis such as antineutrophil cytoplasm antibody (ANCA)-associated vasculitis. 55–58

It has been suggested that TAB may be a useful test to diagnose other forms of vasculitis, which could mimic GCA. 59

There is an inevitable tension between obtaining enough material to make a diagnosis and initiating therapy before disease-related complications set in. In practice, it is common for patients to start on treatment as soon as a physician suspects the diagnosis, typically based on symptoms suggesting the diagnosis of GCA and possible laboratory investigations such as an elevated C-reactive protein (CRP) level or ESR. Treatment is commonly initiated in primary care and the primary care physician would typically contact secondary care services to request confirmation of the diagnosis with a biopsy. However, the acute phase response markers are not reliable tests to diagnose GCA, although if they are elevated, the ESR and CRP level are supportive of the diagnosis but cannot be used on their own because of their lack of specificity.

Standards for diagnosis of giant cell arteritis

The BSR guidelines on managing GCA recommend that biopsy should be considered if a diagnosis of GCA is suspected and state that an early biopsy is desirable in patients with suspected cranial GCA, preferably within 7 days of initiating high-dose steroid therapy. 5 The biopsy should be carried out by experienced surgeons to give the highest yield of positive results. Similar recommendations were made by the European League Against Rheumatism in their guidelines for the management of large vessel vasculitis. 60 Unfortunately for the NHS in England and for other health-care systems, there may be difficulty in accessing a surgical list promptly. This can result in significant delay in a biopsy being performed. Furthermore, the procedure is often performed by a relatively junior and inexperienced member of the team. The overall impact of these factors could be a reduction in the sensitivity of biopsy as a test for GCA.

Accuracy of temporal artery biopsy versus ultrasound or other imaging modalities

A meta-analysis of the use of ultrasound in GCA16 examined 23 studies and involved 2036 patients. The weighted sensitivity and specificity of the halo sign was 69% [95% confidence interval (CI) 57% to 79%] and 82% (95% CI 75% to 87%), respectively, compared with biopsy, and 55% (95% CI 36% to 73%) and 94% (95% CI 82% to 98%), respectively, compared with ACR criteria. A study of 55 patients who underwent colour Doppler ultrasound for suspected GCA61 reported a sensitivity and specificity of 82% and 91%, respectively, suggesting that an ultrasound scan could be a good alternative to biopsy in many patients.

However, ultrasonography of the temporal and axillary arteries is highly operator dependent and it is important to develop and maintain expertise in the technique before it can be applied. Therefore, any ultrasound study requires quality assurance of the adequate training of sonographers prior to the evaluation of patients with suspected GCA. By contrast, MRI is much less operator dependent. In a recent multicentre study, the diagnostic accuracy of MRI was investigated in 185 patients referred for suspected GCA, of whom 53% underwent TAB. The sensitivity and specificity of MRI for diagnosing GCA was 78.4% and 90.4%, respectively, and for TAB (in those patients who had biopsy), the sensitivity and specificity were 88.7% and 75%, respectively. 13 The accuracy of the imaging was high if the patients had received either no glucocorticoids or glucocorticoids for no more than 5 days, but more than 5 days of therapy resulted in a significant fall in diagnostic accuracy. A combined approach that used ultrasound to try to identify the most appropriate site for biopsy had no effect on the sensitivity of detecting histological evidence of GCA. 62

Summary

In summary, the management of GCA requires a balance between ensuring that patients with GCA are diagnosed and treated promptly (to avoid complications such as sight loss) and avoiding the burden of unnecessary steroid treatment in people without GCA. TAB is useful in assisting with diagnosis but lacks sensitivity. Research since the 1990s on the accuracy of ultrasound suggests that ultrasound has a role as an alternative to, or in addition to, biopsy. However, within the UK, the routine use of ultrasound for GCA is restricted to only a few centres; TAB remains the standard test for the majority of patients suspected of having GCA.

Aims and objectives

The main aim of the TABUL study was to assess the relative merits of TAB and ultrasound in contributing to the diagnosis of GCA. The objectives of the TABUL study are based on two assumptions about diagnosing and treating GCA.

First, patients with suspected GCA are treated with steroids as soon as the diagnosis is suspected (in order to reduce the risk of serious vascular complications) and before any biopsy results might be available. Therefore, the potential benefit of an ultrasound examination instead of, or in addition to, biopsy is the ability to either continue or withdraw high-dose glucocorticoid treatment appropriately owing to greater certainty of diagnosis.

Second, TAB is itself very problematic as a reference standard, because up to half or more patients with true GCA may have a negative biopsy. 6,7 This may be for a number of reasons including biopsy size, delay between onset of symptoms followed by early use of high-dose glucocorticoid therapy before biopsy and obtaining the biopsy specimen within 7–10 days of therapy commencing; furthermore, the processing and interpretation of biopsy itself can influence the outcome. A positive biopsy does confirm the diagnosis in most patients suspected of having GCA, with specificity approaching 100%. There are some exceptions because other forms of vasculitis may produce exactly the same biopsy appearances as seen in GCA. The difference for other forms of vasculitis is that patients experience clinical features in other organ systems that support that diagnosis, such as the involvement of airways or kidneys in patient with granulomatosis with polyangiitis (GPA) or eosinophilic GPA (EGPA). Ultrasound is not going to be able to achieve greater specificity than biopsy but may achieve better sensitivity if used either instead of, or in addition to, biopsy.

The first primary objective of the study was to evaluate the diagnostic performance (sensitivity and specificity) of ultrasound as an alternative to biopsy for diagnosing GCA in patients who are referred with suspected GCA and in whom a biopsy was going to be carried out.

The second primary objective was to perform a cost-effectiveness analysis to compare ultrasound as an alternative to biopsy for diagnosing GCA.

The secondary objectives in the study were to evaluate:

• interobserver agreement in the assessment of ultrasound and biopsy

• the performance (sensitivity and specificity) of alternative strategies involving ultrasound and biopsy for diagnosing GCA

• the cost-effectiveness of alternative strategies involving ultrasound and biopsy for diagnosing GCA.

Summary of study design

The study used a prospective cohort design and recruited patients with suspected GCA who were undergoing a TAB, the standard diagnostic test, as part of their routine care in order to assist with establishing the diagnosis. Patients were recruited following referral from their primary care physician or a secondary care physician and consented to have an additional diagnostic test, namely an ultrasound investigation of their temporal and axillary arteries, before having their biopsy. The clinician treating the patient, as well as the patient, was blinded to the results of the ultrasound. Patients were assessed at presentation, at 2 weeks and after 6 months. The performances of TAB and ultrasound were evaluated against a reference diagnosis derived from the clinician’s final diagnosis, which included any changes to the diagnosis during the follow-up period, such as the emergence of any GCA-related complications. The reference diagnosis confirmed the clinician’s final diagnosis using an algorithm based on the ACR classification criteria; any unconfirmed cases (and all cases in which the ultrasound result was unblinded and seen by the clinician) were independently reviewed by a panel of experts.

Agreement between sonographers and between pathologists in their interpretation of videos and images was assessed in an inter-rater agreement exercise for a sample of recruited patients. Clinical vignettes for these patients were constructed and assessed by clinicians to see what decisions about diagnosis and treatment might have been made if ultrasound results were provided instead of biopsy results. The cost-effectiveness of the different tests and combinations of tests was assessed in an economic evaluation.

Patient and public involvement

Advice on study design was sought and obtained from patients through the registered charity Polymyalgia Rheumatica & Giant Cell Arteritis UK. Patient representatives on the Trial Steering Committee and the Data Monitoring Committee provided valuable advice and input during the study (see Acknowledgements).

Recruitment of sites

Sites were eligible to take part in the study if they were responsible for seeing patients with suspected GCA and used TAB as a routine test for its diagnosis. Sites were not eligible if they used ultrasound for diagnosing GCA as part of their routine practice.

Prior to study commencement, 19 hospitals in England indicated their interest in becoming study sites for potential recruitment. Sites were eligible to take part if a site principal investigator, typically a clinician (e.g. a rheumatologist or ophthalmologist) involved in the management of patients with GCA, could be identified who would have overall responsibility for the site’s involvement in the study. Sites also needed to be able to identify the minimum of one pathologist who would have responsibility for assessing TABs and one sonographer with responsibility for performing and assessing ultrasound. Study sonographers needed to have some previous experience in the use of ultrasound but did not need to have specific experience in ultrasound of the temporal or axillary arteries for GCA. Sonographers could come from a variety of clinical disciplines and included rheumatologists, radiologists and radiographers. Sites also needed to provide assurance that, for any individual patient, the roles of the sonographer and the clinician managing the patient were separate. This was to prevent the managing clinician from knowing the results of the ultrasound scan, except when specifically allowed in the study protocol. It did not preclude a clinician (e.g. a rheumatologist who carries out ultrasound) from performing either role in different patients provided that the separation of responsibilities was maintained for each participant.

All sites needed to obtain the relevant local approvals before training could be commenced. Site participation required sonographers to successfully complete a training package in ultrasound for GCA. No training was provided to the site surgeons, who were asked to perform the biopsies as part of routine care, or to pathologists, given that TAB specimen assessment is part of standard care. At some sites, additional clinicians were involved in the management of study patients and this was a requirement if the site’s principal investigator was designated as the study sonographer to ensure that the ultrasound result was blinded for all patients. Research nurses at each site were responsible for co-ordinating recruitment and arranging tests to ensure that both ultrasound and biopsy procedures could be performed within 7 days of commencing high-dose glucocorticoid therapy. All these site personnel comprised the local TABUL team with responsibility for co-ordinating the study locally and completing the clinical, pathology and ultrasound data collection. The process for ultrasound training is described in the next section.

Each site was provided with study training during an initiation visit from the central TABUL study team which consisted of advice on data collection (including completion of study forms) and the process for submitting data. Specific training was provided on the completion of two measures used to assess patients: the Birmingham Vasculitis Activity Score (BVAS) and the Vasculitis Damage Index (VDI). Clinicians and research nurses were required to achieve test scores of 85% for the BVAS and 75% for the VDI (and at least 50% of all individual cases had to be correct) before they were approved for scoring the two measures. Monitoring visits were conducted as per the study standard operating procedures to ensure that the correct procedures were being followed.

Training in ultrasound for giant cell arteritis

Ultrasound assessment of temporal arteries is an established technique for the diagnosis of GCA but there is no standardised protocol in widespread use. We therefore developed a training package for performing and analysing ultrasound scans for the TABUL study. The purpose of the training package was to provide assurance that the sonographers in the study had achieved competence in scanning the temporal and axillary arteries and interpreting the results before recruiting patients to the study.

The training package included a standardised protocol for performing ultrasound in the TABUL study and an accompanying presentation. Sonographers’ competence in ultrasound for GCA was assessed in three ways: (1) undertaking ultrasound assessment of 10 patients or volunteers without GCA; (2) passing an examination that tested each sonographer’s competence in interpreting ultrasound videos; and (3) successfully completing a ‘hot case’ ultrasound assessment of a patient with active GCA. Sonographers were encouraged to attend the TABUL training day for sonographers in Oxford and/or participate in site visits from the TABUL study team. After successful completion of training, sonographers were required to submit recorded scans of recruited patients for ongoing assessment of competency in scanning and interpretation.

Sonographers were required to complete all components of the training before they were deemed eligible to assess patients recruited to the main study. An exception was made for sonographers who were already performing routine assessment for GCA; these sonographers were required to undergo part of the training protocol by scanning 10 control cases and completing their online assessment. These sonographers were exempt from completing the ‘hot case’ assessment on the merit of their curriculum vitae, which was assessed by the ultrasound experts for the study.

Ultrasound training programme

Although the biopsy of temporal arteries has been an established test in widespread use all over the world for decades, the use of ultrasound as a diagnostic test is much more limited. Very few of the sites involved in the study had sufficient expertise to undertake proficient vascular ultrasound scanning for GCA. We therefore developed a pragmatic training programme consisting of attendance at a training day or a site visit with hands-on training. Competence in ultrasound was assessed using a video examination to correctly identify normal or abnormal scan appearances, evidence of successfully performed scans of 10 healthy control subjects, and evidence of a successfully performed scan of at least one patient with scan findings of active GCA. Sonographers were allowed to take part in the study only once all elements had been successfully completed. In addition, we required sonographers to submit recordings of scans from all patients recruited into the study for ongoing quality control.

Ultrasound protocol training was provided during a training day in Oxford at the start of the study or at site visits by the TABUL study team. The protocol and training emphasised the importance of keeping the ultrasound result blinded from the clinician treating the patient. Sonographers were also provided with a presentation on how to scan temporal and axillary arteries to look for evidence of GCA and how to document the site and nature of the findings using standardised abbreviations (see Table 1). The presentation was developed with the supervision of one of the authors (WAS) who had extensive expertise in GCA ultrasound. The presentation provided information on recommended techniques and described the minimum equipment required to perform optimal scanning.

Study population, recruitment and sampling

The study aimed to recruit all eligible patients who were undergoing a TAB for suspected GCA. Patients were eligible if there was a clinical suspicion of a new diagnosis of GCA and the treating clinician had decided that the patient required an urgent TAB to help determine whether or not the diagnosis was GCA. No particular symptoms were specified, although it was expected that patients would have typical symptoms of GCA such as a new onset of headache, scalp tenderness, elevated CRP level or ESR, jaw or tongue claudication or visual loss. Patients had to be at least 18 years of age and be willing to attend for an ultrasound scan of their temporal and axillary arteries.

Patients were not eligible for the study if they had had a previous diagnosis of GCA or if it was not possible to arrange for their ultrasound and biopsy to be performed within 7 days of starting higher doses of glucocorticoids (defined as > 20 mg of oral prednisolone or equivalent daily). Patients were also ineligible if they had prolonged use (> 1 month) of higher dose glucocorticoids (> 20 mg of prednisolone or equivalent per day at any time) within the previous 3 months for any condition other than PMR. A current or previous diagnosis of PMR or presenting symptoms of PMR were not exclusion criteria, because this group of patients would be likely to require investigations for possible associated GCA, if they presented with new features suggesting the diagnosis. No other selection criteria were used for the recruitment of patients.

All patients were required to give written informed consent. Additional consent was required to allow serum, plasma and deoxyribonucleic acid samples to be taken at the first assessment and serum and plasma to be taken at the second and third assessments for future, currently undefined studies. Patients were also invited to consent to allow their remaining tissue biopsy samples (not required for diagnosis) to be stored centrally in the Oxford Musculoskeletal Biobank for further, future currently undefined studies. All slides that were originally required for diagnostic purposes were stored in the Oxford Musculoskeletal Biobank or returned to the site pathologists, after they had been photographed. All screened patients were allocated a unique screening number and a screening case record form (CRF) was completed for each case (see Appendix 3). All eligible patients who consented were allocated a unique study identification number.

It was expected that the majority of patients would be recruited from referrals from general practice to secondary care (either to rheumatology and/or ophthalmology on-call teams). The clinician responsible for the patient’s care obtained verbal consent from the potential patient and passed on their contact details to the local TABUL team. Following an initial telephone call the TABUL team provided the potential patients with the study invitation letter and participant/patient information sheet (see Appendix 4) and discussed the study with them. Alternatively, if a patient was attending the hospital, the study documents were given directly to them by the clinician or study team. The potential patient would then have sufficient time to read and understand the information and to ask any questions before providing written informed consent (see Appendix 5).

Study recruitment at sites was encouraged by providing study information flyers in non-patients areas of sites as an aide-memoire for research teams and clinicians. Awareness of the study was raised with rheumatologists at local, regional, national and international meetings such as the BSR, local meetings with GPs, ophthalmologists, vascular surgeons, rheumatologists and clinicians treating other forms of vasculitis. Guidance on recruitment was provided to all sites (see Appendix 6).

Sample size calculation

The sample size of 402 patients was calculated to provide 90% power at a 5% type I error rate to test the joint hypotheses that:

1. ultrasound has greater sensitivity than TAB (based on an assumed sensitivity of 76% for TAB and 87% for ultrasound)

2. the specificity of ultrasound is no less than 83% using the reference diagnosis.

The postulated sensitivity and specificity figures were based on a previous meta-analysis. 16 The sample size would allow estimation of a one-sided rectangular confidence region for ultrasound false- and true-positive fractions, assuming 80% prevalence of GCA in patients having a biopsy for suspected GCA, with the sample size inflated (gamma 0.1) because of uncertainty in the ratio of cases to controls in a cohort design. 64

In order to allow for losses to follow-up (failure to have either test done, lack of a follow-up assessment or patient withdrawal) the plan was to recruit 430 participants to the study. After monitoring actual recruitment and withdrawals during the course of the study, the target recruitment was increased to the range 435–445.

Clinical data collection

Patients who were referred with suspected GCA were screened to check their eligibility for recruitment into the study. Patients who were eligible and gave informed consent had a full clinical assessment at presentation. Appointments for ultrasound scans and then biopsy were arranged and patients returned for a follow-up clinical assessment after 2 weeks (Figure 1). After the 2-week assessment and after seeing the biopsy report, the clinician (who remained blinded to the ultrasound results) decided whether or not the patient had features consistent with a diagnosis of GCA.

FIGURE 1.

Flow of patients in the study. US, ultrasound; V, visit.

The result of the ultrasound was unblinded only if the clinician concluded that the patient did not have features consistent with GCA and was therefore planning to withdraw steroid therapy rapidly. The procedure for doing so is described below (see Ultrasound test results: procedure for revealing test results). Clinicians were allowed to alter their decision to withdraw steroids rapidly following unblinding of the ultrasound result. Patients attended a final clinical assessment after 6 months.

The standard test: temporal artery biopsy

The standard test for GCA is TAB. This normally involves a minor surgical procedure to remove a small sample of temporal artery (the BSR recommends a minimum length of 1 cm⁵) which is examined for abnormalities by a pathologist. Guidance on the collection, processing and storage of biopsy samples is shown in Appendix 11. Sites followed their usual practice for obtaining and processing TABs. The only changes to routine practice required by TABUL were that sites were instructed to send the actual pathological slides used to make their diagnosis to the TABUL office and that, in addition to their standard reporting of biopsy results, pathologists were required to complete a study-specific CRF (see Appendices 12 and 13) to report their pathological findings. We did not require any specific information from any of the surgeons undertaking the biopsy but they were all informed that the patient had been recruited to the TABUL study.

The pathologist was required to report which side or sides the biopsy had been taken from as well as the length of the biopsy (after freezing or fixation), and a note was made of whether or not it was bifurcated. They were able to add other comments on the macroscopic appearance of the sample. For each biopsy, the staining protocol was reported. The macroscopic appearance was described and a note was made of whether or not the biopsy was from the temporal artery and which sections were cut. The presence of abnormalities in the intima (arteriosclerosis or intimal hyperplasia) and the internal elastic lamina (fragmentation or reduplication) were reported. Pathologists were required to indicate if there was an inflammatory infiltrate in the sample (and the predominant site of any inflammation) and indicate if any of the following features were present: normal areas, giant cells, calcification or any other unusual features. Data were also captured on presence and causes of complete occlusion of the vessel or presence of thrombus or evidence of recanalisation in at least one section of the vessel.

The pathological diagnosis was reported as either normal or any the following: compatible with a diagnosis of GCA, compatible with another vasculitis, compatible with arteriosclerosis and compatible with any other diagnosis as specified by the pathologist. The actual pathological slides were sent to the TABUL office for image acquisition. Digital image acquisition was achieved using an Aperio Scanscope Turbo AT (Leica Biosystems, Buffalo Grove, IL, USA). Slides were loaded onto the machine’s autoloader and pre-snapped to obtain a macroscopic image before proceeding with digital scanning. The macroscopic image was used to set the tissue area, focal plane, focus points, white balance, scan/slide settings and labelling description. Once the settings had been optimised the slides were scanned in fragments and digitally stitched together to form one high-resolution virtual representation of the pathology slide. These virtual slides were stored on an external physical server and a web-based database (Aperio eSlideManager V1.0, Leica Biosystems) was used to archive and store the eSlides. Slides could be viewed remotely using Aperio’s web-based viewing systems (Leica Biosystems).

The biopsy result, which was the primary standard test, was defined as positive by the pathologist if the pathological diagnosis was compatible with a diagnosis of GCA. This included patients whose biopsy samples did not contain temporal artery (e.g. vein, fat, muscle or other tissue) or for whom no sample was obtained from surgery. An alternative standard test result was defined as the clinician’s interpretation of the biopsy result as reported on the clinical CRF at the 2-week assessment. This was reported because we expected that the clinician might reach a different conclusion from the pathologist, based on the biopsy report.

The main analyses included patients who had no sample from surgery or a biopsy sample that did not include temporal artery; these were categorised as not compatible with a diagnosis of GCA. Additional analyses excluded the indeterminate biopsy results.

The index test: ultrasound of the temporal and axillary arteries

The index test, an ultrasound of both temporal and both axillary arteries, was performed following the protocol described earlier and is available on the NIHR Evaluation, Trials and Studies Coordinating Centre website (www.nets.nihr.ac.uk) and was subject to ongoing monitoring for quality assurance. The presence of ultrasound abnormalities (halo, occlusion, stenosis and arteriosclerosis) in different segments of the temporal arteries and in the axillary arteries (as defined in Table 1) was captured in the ultrasound case report form (see Appendix 1). The primary test result for ultrasound was defined as positive and was used for the main analyses if the sonographer responded ‘yes’ to the question ‘In your opinion are the results consistent with a diagnosis of GCA?’. Additional analyses used alternative definitions of a positive result based on the presence or absence of a bilateral halo and on the interpretation of the ultrasound images from the expert review.

Ultrasound test results: procedure for revealing test results

The clinician treating the patient was provided with the biopsy result but did not have access to the results of ultrasound at the 2-week assessment. Study sonographers were required to keep the results of each patient’s scans blinded from the managing clinician for the duration of the study. The only exception was if the managing clinician had completed the 2-week assessment and was planning to withdraw steroid treatment rapidly. In these circumstances the clinician was required to contact the TABUL office and was provided with the scan results as reported by the sonographer. The clinician then had an opportunity to reconsider their decision to withdraw steroids and alter their diagnosis. Thus, the 2-week assessment included a report of the clinician’s original assessment of the diagnosis and any revision following the revealing of the ultrasound result.

The reference diagnosis

The ideal reference diagnosis for evaluating diagnostic tests is one that is independent of the tests being evaluated. No such reference diagnosis exists for GCA for evaluating the performance of biopsy and ultrasound. Criteria for classifying GCA and usual clinical practice for reaching a GCA diagnosis incorporate the results of the biopsy; therefore, they cannot be truly independent methods for defining a reference diagnosis. Furthermore, the ACR classification criteria were not intended to be used as diagnostic criteria. 34 For the purposes of the study, a partially independent approach was used, which combined elements of a clinician’s final diagnosis, the ACR classification criteria (incorporating the biopsy result), the emergence of complications consistent with GCA during follow-up, the emergence of alternative vasculitis diagnoses during follow-up and expert review to determine the reference diagnosis. The process started with the clinician’s final diagnosis for the patient as reported on the 6-month (or in its absence, 2-week) assessment. An algorithm was devised to determine if evidence from the biopsy and the presence or absence of symptoms and emerging complications and diagnoses on follow-up supported the clinician’s diagnosis or if expert review was required to determine the reference diagnosis.

If the clinician’s final diagnosis was GCA, then a reference diagnosis of GCA was given if any of the following criteria were met:

1. a stricter version of the ACR classification criteria using either the standard or tree method was met based on the patient’s symptoms and physical examination from their baseline assessment (Table 2)

2. the emergence of PMR during follow-up in patients with no previous history of PMR and no symptoms of PMR at presentation

3. the emergence of new or worsening jaw claudication, tongue claudication, abnormal anterior optic neuropathy, abnormal posterior optic neuropathy, or relative afferent pupillary defect during follow-up.

If the clinician’s final diagnosis was not GCA, then a reference diagnosis of ‘not GCA’ was given. If an alternative vasculitis diagnosis was made, these included Takayasu’s arteritis, large vessel vasculitis, polyarteritis nodosa, GPA, microscopic polyangiitis, EGPA, cryoglobulinemic vasculitis, IgA vasculitis (Henoch–Schönlein purpura), or any other vasculitis to be specified. A reference diagnosis of ‘not GCA’ was also given if all of the following criteria were met.

1. The patient failed to meet the ACR classification criteria using either the standard or tree methods (see Table 2).

2. No new-onset PMR occurred during follow-up.

3. No new or worsening jaw claudication, tongue claudication, abnormal anterior optic neuropathy, abnormal posterior optic neuropathy or relative afferent pupillary defect occurred during follow-up.

4. No symptom of reduced or lost vision in either eye occurred or worsened during follow-up.

5. No evidence of abnormal III/IV/VI nerve palsy or stroke on clinical examination was observed at 2 weeks or 6 months.

6. No sudden visual loss, cerebrovascular accident or cranial nerve palsy reported on the BVAS occurred during follow-up.

7. No retinal change, optic atrophy, visual impairment/diplopia, blindness in one eye, blindness in the second eyes or cerebrovascular accident reported on the VDI occurred during follow-up.

Any patient who was not given a confirmed reference diagnosis based on the above was referred for expert review. Furthermore, any patient who had their diagnosis altered during follow-up (typically for a diagnosis altered to GCA from not GCA following unblinding of the ultrasound report) was automatically referred for expert review regardless of any confirmed reference diagnosis given above.

The expert review group comprised five rheumatologists involved in the study. Each case requiring expert review was independently assessed by three of the five rheumatologists, and no rheumatologist could review cases from their own site. A summary report for each patient was extracted from the clinical data and included information on symptoms, GCA-related complications, items from the ACR classification criteria and the clinician’s diagnosis. Access to the clinical database was also given so that expert reviewers could examine all data collected as part of the study with the exception of the ultrasound results. Each expert reviewer independently reported their agreement or disagreement with the clinician’s final diagnosis. The clinician’s final diagnosis was supported if at least two of the experts agreed with the diagnosis. The clinician’s diagnosis was altered if all three experts disagreed with the diagnosis. If two experts disagreed with the clinician’s diagnosis then the patient was discussed by the relevant experts during a moderated teleconference until the three experts reached a consensus.

Inter-rater agreement data collection and analysis

The aim of the inter-rater agreement component of the study was to assess the extent of agreement between trained sonographers in their interpretation of ultrasound videos, and between experienced pathologists in their interpretation of biopsy images, for a sample of cases using data, videos and images from patients recruited to TABUL. Sonographers and pathologists assessed the same cases using a web-based exercise. Intrarater agreement was also assessed by repeating cases in the exercise. The impact of providing additional information about the patient was examined by including a brief vignette.

All pathologists and sonographers who assessed patients in the main TABUL study were asked to complete a web-based exercise. The exceptions were pathologists and sonographers who were involved in the management of TABUL or in the expert review of ultrasound for quality control (two pathologists and four sonographers.) Pathologists and sonographers who agreed were sent instructions for completing the exercise and a password to access the exercise.

The overall design involved a web exercise with 44 cases. Each case represented a patient recruited to TABUL and comprised ultrasound videos of both temporal arteries, scanned images of the biopsy slide and a brief clinical vignette describing the patient. The first five cases were defined as training/practice cases that allowed raters to familiarise themselves with the exercise. The remaining cases, the rating cases, consisted of 30 unique cases, six repeats of unique cases (for intrarater assessment) and three reserve cases. The reserve cases were available to replace any of the 30 cases that were subsequently found to be ineligible once the exercise had started. The overall number of cases was chosen to keep the task manageable, and the aim was to have at least 10 pathologists and 10 sonographers complete the exercise. This was to allow results to be generalised to the wider populations of pathologists and sonographers.

The criteria for including a patient’s videos and biopsy images as rating cases in the exercise were ultrasound videos of adequate quality of the right and left temporal arteries, biopsy slides received and scanned, inclusion of the patient in the main TABUL analyses and patient consent for the use of the images. Cases were ineligible if the biopsy specimen did not consist of artery or if the ultrasound was abnormal owing to axillary artery involvement without temporal artery abnormalities. Cases were also ineligible if the biopsy images or ultrasound videos included information that identified the patient or clinician involved or included markings indicating abnormality and this information could not be removed or hidden. Finally, cases were excluded if the quality of the ultrasound images was judged to be poor by expert review during quality control. Disagreement with the original sonographer’s interpretation by expert review or difficulty in interpreting the ultrasound by expert review despite adequate quality videos were not criteria for exclusion.

Identification of cases was performed in three stages before the start of the exercise because the main TABUL database and the ultrasound and biopsy databases had not been locked at the time of initial selection and because of the work involved in ensuring that videos and images were eligible. The first stage involved identifying potentially eligible cases from the list of patients recruited to the main study who had had their ultrasound videos uploaded. This list of potentially eligible cases was ordered using random numbers generated using Stata version 13 (StataCorp LP, College Station, TX, USA). The second stage involved populating the 33 rating cases from the top of the list. Any case found to be ineligible was replaced with the next available case from the list. This process was repeated until all 33 rating cases were deemed eligible. The third stage involved pilot testing of the exercise and review of all videos and images by two pathologists (BM, KW) and two sonographers (WAS, JP) to ensure that the criteria relating to the videos and images were met. The five training cases were selected purposively starting at the bottom of the ordered list. These were selected to ensure that there were at least two abnormal and two normal cases for the biopsy images and for the ultrasound videos. A final post-exercise stage involved a further eligibility check of the rating cases against the locked database. Any of the 30 rating cases subsequently found to be ineligible were replaced with one of the three reserve rating cases for inclusion in the analyses.

A web-based exercise was designed to allow remote access to the videos and images and to capture data from the assessments made by the sonographers and pathologists. Each case in the exercise began by giving access to two video images showing left and right temporal arteries (for sonographers) or one biopsy slide image for each stain available (for pathologists). Videos could be replayed as often as required and biopsy images allowed zooming for magnification at the equivalent of up to 40 times in high resolution. Raters were asked to answer yes or no to the question ‘In your opinion, do the ultrasound (or pathology) images show features of GCA?’ and to answer certain or uncertain in response to the question ‘How certain are you?’. They then gave their answers and confirmed that they were confident to submit their answers.

All cases were rated before and after seeing a brief clinical vignette describing the patient. The vignette was added to reflect a more realistic scenario for interpreting the videos or images. For example, the sonographer would see the patient in front of them when conducting temporal and axillary artery ultrasound. The pathologist might receive a brief description of the patient on the biopsy request form. The vignettes provided basic information on age, sex, glucocorticoid treatment, comorbidity, presenting symptoms and laboratory test results, for example ‘79 year old male started glucocorticoid therapy 2 days ago for suspected GCA. Patient has hypertension. Presented with new localised pain in head, jaw claudication and reduced vision. Elevated ESR and CRP’. The vignette was identical for the ultrasound and biopsy versions of each case except for the duration of glucocorticoid therapy (which varied depending on when the test was done). For repeat cases, the core information was identical to the original case but the order of wording was altered.

Cases had to be completed in order and rating cases could not be started until all five training cases had been completed. Once a rating case had been completed it was not possible to return to that case to view the videos or images or to look at the answers given. This was because six of the cases were repeated. It was possible to return to the training cases for reference. The locations of repeated cases in the 36 rating cases were assigned before the random ordering of eligible cases. Repeated cases all made their first appearance in the first 18 cases and all made their second appearance in the final 18 cases. For each of the six repeated cases there was a minimum gap of 16 cases between its first and second appearances.

Clinical vignettes data collection and analysis

The aim of the assessment of the clinical vignettes was to determine what decision about a patient’s diagnosis and treatment would have been made if there was no biopsy performed, leaving the clinician to rely on the results of the ultrasound. Two overlapping samples of cases were selected from patients recruited to the study. The first sample was the same random sample used for the assessment of inter-rater agreement. The second sample comprised all patients in the main study who had a positive biopsy and a negative ultrasound.

Clinical vignettes were structured to provide data on the patients at the times when two key decisions are made. The first is on initial presentation, when the possibility of a diagnosis of GCA is considered and a decision is taken to recommend a TAB. The second is after 2 weeks, when a decision to continue or withdraw high-dose steroids for GCA is made. Vignettes were populated with data collected during the study. Information provided at presentation comprised the patient’s age, sex, relevant current conditions and medical history, symptoms, symptom onset and any laboratory test results (ESR, CRP level or ANCA) prior to starting steroids, duration and dose of steroids, new symptoms and symptoms still present at presentation, results of the physical examination at presentation and any laboratory test results (ESR, CRP level or ANCA) at presentation. Clinicians were then asked to give their indication of the likelihood of the patient having GCA (definite, probable, possible or not GCA) and indicate whether or not, in the absence of alternative tests such as ultrasound, they would recommend this patient for a TAB.

The information at 2 weeks was presented once responses to the questions had been confirmed. Information on the vignettes comprised the results of the ultrasound test and information about the patient’s health after 2 weeks. The ultrasound test was reported as either consistent or not consistent with a diagnosis of GCA and included additional information on any abnormalities identified on ultrasound, for example ‘consistent with GCA; halo on right temporal artery; normal left temporal artery; normal axillary arteries; no occlusion or stenosis’. Other information comprised symptoms present at 2 weeks (categorised by new, worse, no change, better and resolved), results of the physical examination at 2 weeks, results from laboratory tests and any changes in current conditions. Clinicians were then asked to give their indication of the likelihood of the patient having GCA (definite, probable, possible or not GCA) and to indicate the appropriateness of continuing to treat the patient with high-dose steroids for GCA on a nine-point scale (1, extremely inappropriate; 5, uncertain; 9, extremely appropriate).

Data on the appropriateness of continuing treatment with high-dose steroids were categorised as appropriate, inappropriate or uncertain using the method outlined in The Rand/UCLA Appropriateness Method User’s Manual. 71 A panel median of 7–9 without disagreement is considered appropriate, a panel median of 4–6 or any median with disagreement is categorised as uncertain, and a panel median of 1–3 is categorised as inappropriate. Disagreement was determined using the interpercentile range adjusted for symmetry and the common approach of rounding up medians of 3.5 and 6.5 was applied. 71

Statistical analysis

The statistical analyses of the diagnostic accuracy of TAB and ultrasound were specified in the statistical analysis plan (see Appendix 14). Sensitivities and specificities were calculated for TAB and ultrasound in comparison with the gold standard reference diagnosis. The kappa statistic was used to assess agreement between TAB and ultrasound, and McNemar’s test was used to detect systematic discordance.

The inter-rater agreement between sonographers and between pathologists was evaluated using a two-way random-effects analysis of variance to estimate the intraclass correlation coefficients for agreement with 95% CIs. Both cases and raters were treated as random effects in order to generalise findings to all cases (from the sample selected) and to the potential population of trained sonographers (from the sample of sonographers doing the exercise). Intrarater agreement was evaluated by estimating kappa statistics for agreement and by examining agreement for the individual repeated cases and raters.

Statistical analysis was performed in Stata versions 12 and 13.

Pre-test probability of giant cell arteritis: definition of risk categories

The availability of data from the DCVAS study provided an opportunity to define categories of pre-test risk of a GCA diagnosis from an independent sample of patients and was used in preference to obtaining expert opinion elicited from clinical vignettes. 41 Data on 585 patients recruited to centres not participating in TABUL, and who had had a TAB, were used to derive definitions for high-, medium- and low-risk groups. The high-risk group was defined as patients with (1) claudication of the jaw or tongue and (2) elevated ESR or CRP level (ESR of at least 60 mm/hour or CRP level of at least 40 mg/l) either at pre-steroids or at presentation assessments. The low-risk group was defined as patients (1) without jaw or tongue claudication and (2) without elevated ESR or CRP level at both the pre-steroids and presentation assessments of symptoms and laboratory tests. The remaining patients were categorised as medium risk.

Changes to the study protocol

There were two substantial amendments to the study protocol. The first amendment was made in February 2011 and comprised the following key changes.

1. To alter the decision always to offer each potential participant 24 hours to consider their participation in the study. This amendment was made because there were some circumstances in which treatment may be delayed while waiting for consent, for example in an emergency (to minimise delay in normal care such as performing the biopsy) or when sites are able to provide a fast turnaround time for performing the biopsy. In these circumstances we offered the opportunity for participants to provide full written informed consent in < 24 hours from receiving information about the study.

2. To provide further clarification on the collection of additional blood and biopsy samples during the course of the study.

The second amendment was made in February 2013 and comprised the following key changes.

1. To increase the target sample size for recruitment from 430 to 435–445 (with 402 completing the primary end point).

2. To extend the recruitment period by 12 months.

3. To clarify the recruitment strategy (including the production of a poster summarising the study for use in non-patient areas).

4. To clarify the process for the managing clinician to contact the TABUL office in order to be given the results of the ultrasound result (unblind the ultrasound result).

5. To allow inclusion of patients in whom the biopsies were performed more than 7 days after starting high-dose glucocorticoids because of safety concerns about when the biopsy could be performed, for example to allow discontinuation of warfarin so that it was safe to perform the biopsy. This would be part of standard care for any patient who required a biopsy but was receiving warfarin.

Site recruitment

The study aimed to recruit sites that routinely performed biopsy of the temporal artery as part of the care pathway in the diagnosis of GCA. Forty-four sites expressed an interest in recruiting patients to the study. Two sites already made some use of temporal artery ultrasound to assist in diagnosing GCA on a non-routine basis but were in equipoise and accepted the study requirement to keep the ultrasound result hidden from the clinician managing the patient. One site made occasional use of ultrasound to mark the area of disease for surgeons to biopsy. For the purposes of the study the site agreed to suspend this practice.

Before the study began there were 19 sites in England that had indicated an interest in taking part. Recruitment of sites in the UK began in November 2009 and the first training case, as part of the ultrasound training, was recruited in March 2010. The first approval of a site to recruit patients to the main study was in June 2010. The process of gaining the relevant research approvals [e.g. NHS research and development (R&D) approval] for sites, the availability of suitable sonography staff and staff time at each site, and the process of training sonographers delayed recruitment of sites to the study. By December 2011, six sites were approved to recruit patients (four of these had begun recruiting patients) and eight sites had started ultrasound training. The study was opened up to sites in Europe following international interest in the study to increase site recruitment, with four sites (in Germany, Ireland, Portugal and Norway) beginning the process to gain approval to recruit patients.

A total of 44 sites expressed an interest in taking part in the study, although eight sites were unable to obtain R&D approval and therefore did not progress to the ultrasound training stage. One site (the ophthalmology department at the John Radcliffe Hospital, Oxford) obtained the relevant R&D approvals but acted as a referral site for the Nuffield Orthopaedic Centre, Oxford, so did not require ultrasound training. Thus, 35 sites began the process of ultrasound training to become eligible to recruit patients to the study. Two of the sites (Stoke Mandeville Hospital, Aylesbury, and Royal Berkshire Hospital, Reading) did not have their own sonographer and instead relied on the trained sonographers from the Nuffield Orthopaedic Centre for ultrasound scanning at their sites. The two sites did not require ultrasound training but did need to complete other study requirements for site approval. Twenty of the 35 sites achieved approval to recruit patients to the study and the progress of 18 of these sites in obtaining approval is shown in Figure 2.

FIGURE 2.

Time from starting ultrasound training to approval for patient recruitment for 18 sites (two sites not requiring ultrasound training are not shown; they received approval to recruit patients in 2011).

Ultrasound training

The key factor that limited progress to full-site approval to recruit patients to the study was the ultrasound training for site sonographers. There were 49 sonographers representing 35 sites who started ultrasound training and 26 sonographers representing 22 sites who passed their training. Two sites had a sonographer who passed training but the sites did not go on to recruit patients to the study. In the first site the sonographer moved to a different hospital so the site lost its approval to recruit to the study because it had no trained sonographer. The second site did not complete another component of site training (completion of BVAS and VDI training) and could not provide appropriate research nurse support to achieve approval to start the study before recruitment to the study had actually completed. Thus, there were 24 sonographers who passed training who were all located at sites that were approved and that were able to recruit patients to the study.

The main reason for not successfully passing the ultrasound training was the requirement to perform a ‘hot case’ assessment. Twenty-one sonographers did not provide or pass a ‘hot case’ assessment and 28 (58%) either passed or were exempt (Table 3). For the video examination component, 39 (80%) sonographers passed [although 21 (42%) needed more than one attempt at the examination] or were exempt and two experienced sonographers were exempt because they were study investigators involved in setting and marking the examination. For the training case component a total of 450 healthy volunteers or patients without any suspicion of GCA were scanned. Forty (82%) sonographers passed the component, although five were required to scan additional training cases before passing.

The 24 sonographers who scanned patients in the study were made up of 10 clinicians, six radiologists and eight professional sonographers. Four had previous experience in ultrasound for GCA and were exempt from the ‘hot case’ assessment and two sonographers were exempt from the video examination component as well. All 24 sonographers satisfactorily completed the training case component, as assessed by one of the study expert sonographers (WAS) not involved in scanning patients, although two sonographers needed to scan additional cases before passing. Half of the sonographers taking the video examination achieved the 75% pass mark at their first attempt with the remaining sonographers achieving this at their second or third attempt.

Seven of the 24 sonographers passed all components without requiring further attempts at any component: three professional sonographers, two radiologists and two clinicians. The 13 sonographers who did need a further attempt at one or more components comprised five professional sonographers, two radiologists and six clinicians. The four experienced sonographers (two radiologists and two clinicians) all passed those components that they took at the first attempt.

Ultrasound monitoring during the study

Once a sonographer had completed training and started scanning patients recruited to the study, all their recorded scans and completed case report forms for these patients were monitored. The team of expert reviewers assessed all submitted scans and forms to monitor the quality of video and still images being recorded, as well as the sonographers’ record of the scan findings, in order to identify any concerns with either the performance of the scans or the interpretation of the results. Details of the results of the expert review are reported in Chapter 5. The results of the expert review are reported in Chapter 4.

One site was suspended from scanning any further training cases because of the poor quality of scanning; the original sonographer at the site subsequently took no further part in the study. A new sonographer was identified who successfully passed the training requirements. Another site was suspended from recruiting patients to the study after the findings from the expert review disagreed with the ultrasound of their second enrolled patient; the site was required to submit and pass an additional ‘hot case’ assessment before being allowed to resume recruitment. In another site, patient recruitment was suspended after expert reviewers disagreed with scans reported as ultrasound positive and recommended retraining for the sonographer. Recruitment was resumed following successful retraining.

Summary of test results and the reference diagnosis

All 381 patients underwent ultrasound examination in accordance with the study protocol, and all patients had a TAB performed as part of the normal standard of care for investigations of patients with suspected GCA. In total, 101 patients (27%) had an abnormal biopsy that was consistent with a diagnosis of GCA. In a total of 28 patients biopsies failed; in four cases no samples were obtained and in 24 cases the biopsy sample did not contain arterial tissue. These patients are defined in the main analyses as having a diagnosis that is not consistent with GCA (i.e. they are analysed on the assumption that they do not have the disease). In total, 162 patients (43%) had an abnormal ultrasound that was compatible with a diagnosis of GCA. After expert review, the reference diagnosis of GCA was given to 257 patients, a prevalence of 66% in the study cohort. The diagnosis of GCA conventionally rests on the clinical pattern at presentation, combined with the results of laboratory tests, including ESR or CRP level, the response to steroid therapy and the TAB result. For many patients, not all of these aspects (clinical findings and symptoms, serological abnormalities or biopsy results) are entirely consistent with the diagnosis, which leads to a degree of variability in interpreting the findings. For example, patients who have symptoms suggestive of GCA, such as new-onset headache and jaw claudication, may actually have a normal or low ESR and or CRP level; furthermore, the biopsy result may be negative, especially if the test was performed after the patient had been treated with high doses of glucocorticoid therapy for > 7 days and/or the biopsy was small (less than 1 cm of artery). Under these circumstances it might be difficult to be absolutely certain of the diagnosis and by the time the biopsy result is provided to the clinician, it is usually too late to go back to recheck any of the tests again because, in the meantime, the patient has continued to receive high-dose glucocorticoid treatment, which is likely to significantly suppress any evidence of inflammation.

The clinical diagnosis of GCA requires the clinician to use their expertise in interpreting these different pieces of information. We therefore included an expert review in our study, so that all cases in which there was any doubt about the diagnosis of GCA were subject to expert review of the clinical and serological findings. The end result was to produce a ‘reference diagnosis’ of GCA based on the clinician’s interpretation of all of this information. Figure 6 summarises the different combinations of biopsy and ultrasound test results (GCA, not GCA or, in the case of biopsy, unsuccessful) and the final reference diagnosis for the 381 patients in the primary analysis group.

FIGURE 6.

Simplified diagnosis flow for the primary analysis group (n = 381). US, ultrasound.

In 187 patients the clinician’s interpretation was submitted for expert review, and in 23 patients the interpretation was altered (from an interpretation of GCA to a reference diagnosis of not GCA in 14 patients and from an interpretation of not GCA to a reference diagnosis of GCA in nine patients). Figure 7 illustrates these interpretations and reference diagnoses with respect to the clinicians’ initial assessment of GCA at presentation.

FIGURE 7.

Simplified flow of clinician and reference diagnoses for the primary analysis group (n = 381). a, In one patient, data on the certainty of initial diagnosis were missing and, therefore, this patient has been included in the possible group.

Participant characteristics

Ultrasound results

Ultrasound examination was performed on all 381 patients. Abnormalities consistent with GCA were found in 162 (43%) of the ultrasound scans (Table 11). Table 11 shows that the majority of patients with abnormal scans had changes in the temporal arteries (35%) and that 11.5% had abnormalities in the axillary and temporal arteries. Abnormalities were more likely to be bilateral than unilateral (29% vs. 20%). Halo was the most commonly cited reason for reaching a diagnosis of GCA (42.5%). Stenosis (12%) or occlusions (11%) were seen less commonly, but there was an overlap with patients also showing halo. The maximum length of halo in those patients in whom a halo was present was 20 mm (median) in the axillary arteries and 9 mm in the temporal arteries. It was, however, sometimes extremely difficult (especially in temporal arteries) to measure the length as a result of vessel tortuosity. In some patients the halo extended the entire length of the scanned artery. The maximum reported median thickness of halo was 1.1 mm (IQR 0.6–1.4 mm) in the axillary arteries and 0.6 mm (IQR 0.4–0.9 mm) in the temporal arteries.

Table 12 details the artery on which the halo was identified. In total, at least one halo on ultrasound was reported in 162 patients, in the majority of whom (n = 118) haloes were seen only on the temporal artery (bilateral, n = 60; unilateral, n = 58). By contrast, just nine patients had a halo on the axillary artery only, with no halos seen in the temporal arteries. In the remaining 35 patients halos were observed on both temporal and axillary arteries.

In 24 patients, ultrasound showed abnormalities but the sonographer’s diagnosis was not GCA. Table 13 describes the characteristics of these patients. The majority of the abnormalities were found in the temporal arteries (18 patients) but eight patients had axillary artery abnormalities. The abnormal findings were unilateral in 14 patients and bilateral in 10 patients and halo was detected in 10 patients, stenosis in nine and occlusion in four. Of these 24 assessments, 10 were in agreemeent with the ultrasound expert reviewers, seven were in disagreement and seven were unclear. In 23 of these 24 patients, the scan findings were attributed to atherosclerosis; one abnormal case was attributed to the use of radiotherapy for breast cancer.

Having completed training (which included 10 ultrasound test cases and at least one hot case), 23 sonographers performed ultrasound scans in the TABUL study. Around half of the scans in the study were undertaken by two sonographers, who performed more than 80 scans each. Figure 8 shows the number of ultrasound assessments undertaken by the 23 sonographers.

FIGURE 8.

Number of ultrasound study assessments performed by the sonographers (n = 23). US, ultrasound.

It is possible that the reliability of sonographers who completed fewer than 10 scans was lower than the reliability of sonographers recruiting more than 10 patients. We examined the evidence for this, which demonstrated an effect on sensitivity but not specificity (see Chapter 5). Table 14 compares the sonographers’ diagnoses with the ultrasound expert review. The expert reviewers agreed with the sonographers’ findings in 260 out of 381 patients for whom the images were clear. In 61 patients (16%), there was a disagreement about interpretation of the scan findings, but in a further 60 patients the main reason for disagreement was on the basis of an unclear or unreviewed scan result, suggesting that technical ability to perform the scan rather than interpretation of the scan result was the main problem. The limitation of technical proficiency at scanning is an important problem to address and highlights the need to consider more training if this is the main issue for the sonographer. It is also possible that the problem is a result of patient factors; for example, the presence of very tortuous temporal arteries can make it more difficult for less experienced sonographers to adequately visualise the whole of the artery.

Figure 9 shows the time interval from starting steroids to undertaking the ultrasound scan or the biopsy as well as the number of days between performing each test. Scans were performed more quickly than biopsies (as part of the protocol, the scan had to be performed before the biopsy because if the biopsy was carried out first, then that section of the artery would no longer be available for scanning). In general, the scan was easier to obtain at very short notice, typically within a few days of starting the steroid treatment, whereas the biopsy was sometimes not possible to schedule until later in the week after commencing steroids (or even later for 10 patients). However, despite these potential difficulties, for 215 out of 391 patients (55%), the tests were performed within 2 days of each other, including 52 patients (13%) for whom the scan and biopsy were performed on the same day.

FIGURE 9.

Days between starting steroids and performing ultrasound or TAB, and number of days between performing ultrasound and TAB for the 391 patients included in the secondary analysis. US, ultrasound. (a) Days from starting steroids to ultrasound scan; (b) days from starting steroids to biopsy; and (c) days between ultrasound and biopsy.

We attempted to measure the time taken to perform each scan by asking each sonographer to report the time of starting and finishing each scan. Complete data were available for 371 patients. The median time to complete a scan was 30 minutes (IQR 20–35 minutes). Looking across the centres, however, there was considerable variation. Some centres had much longer scanning times (median of 45 minutes); by contrast, the shortest scanning time was only 8.5 minutes, as shown in Table 15.

We could not see any relationship between the duration of the scan and when during the course of the study the scan was performed (Figure 10). It is likely that the scan times recorded were an estimate of the actual time taken. Ultrasound positive scans appeared to take longer than negative scans; the median time taken for positive scans was 33.5 minutes (IQR 30–40 minutes), compared with 25 minutes (IQR 20–30 minutes) for negative scans, as shown in Figure 11. This indicates that it takes longer to scan and document and record areas if there are abnormalities than if normal areas only are found. There was a wider range of times taken to complete scans towards the end of the study, which might be explained by the inclusion of a larger number of sites.

FIGURE 10.

Range of times taken to complete scans during the course of the study (n = 371). US, ultrasound.

FIGURE 11.

Time taken to perform ultrasound scans, comparing positive with negative scans. US, ultrasound.

Biopsy results

As part of the study protocol, all patients were scheduled to undergo a TAB within 7 days of starting high-dose glucocorticoid therapy. Table 16 shows that a significant minority (n = 28, 7%) of biopsy procedures resulted in no useful tissue. The most common reason for a failed biopsy was that the surgeon took a sample that contained vein instead of artery (n = 13, 3.4%). Although this could reflect the difficulty in obtaining material from tortuous vessels, it could also reflect the relative inexperience of the surgeon given the task of obtaining the biopsy. The BSR guidelines recommend that a surgical biopsy with a minimum of 1 cm of temporal artery is obtained for each patient with suspected GCA; the procedure should be performed by a trained surgeon with experience in the technique. We did not mandate this, given that the study was comparing current standard practice in the NHS with the new technique of ultrasound. It seems likely that some of the biopsies were performed by less experienced surgeons, resulting in relatively poor diagnostic yield with no artery at all in 13% of patients. In addition, the length of temporal artery obtained in 43% of patients was below the BSR-recommended length of 1 cm (Table 17). These factors could have contributed to the relatively poor performance of biopsy as a diagnostic test in GCA. Table 15 shows that giant cells were seen in 19% of biopsies overall, representing 71% of patients with GCA (72/101). Occlusion was reported in 25 biopsies. Of the 161 biopsies with abnormal pathology, four (1%) were compatible with another vasculitis and 35 (9%) were compatible with arteriosclerosis. Table 16 highlights some potential issues in interpreting the biopsy results. Of the biopsy-negative patients who were ultimately diagnosed as not having GCA according to the reference diagnosis, 19% had intimal hyperplasia and 35% showed fragmentation or reduplication of the internal elastic lamina. The frequency of these changes was lower than those seen in patients with a positive biopsy, but almost identical to those seen in patients who were diagnosed as having GCA but who had a ‘negative’ biopsy. These findings raise further concerns about the validity of interpreting the TAB in the absence of cellular changes.

The rheumatologists interpreted the biopsy findings at 2 weeks, as well as evaluating the patient’s clinical condition. In 11 patients the rheumatologist over-ruled or ignored the pathologist’s conclusions, switching the diagnosis from not being consistent with GCA to being consistent with GCA (Table 18). There were no patients in whom the pathologists’ diagnosis of GCA was over-ruled by rheumatologists, and it seems most likely that the rheumatologists would also have agreed with the pathologists’ diagnosis in the case of two patients in whom information was missing.

Table 19 shows that there is no clear association between biopsy findings in those patients with a positive biopsy and presenting symptoms. Different histological features were present in patients with all three types of symptoms at baseline. We have not included a comparison between histological features and the presence of headache because this was a very common symptom at presentation.

Clinical and reference diagnoses

Any treatment decisions were independent of the study itself. The only eligibility criterion was that the clinician suspected a diagnosis of GCA and was intending to arrange a TAB to establish the likely diagnosis. High-dose steroid treatment was not an exclusion criterion, as long as the patient had not been given high-dose steroid treatment for > 7 days prior to obtaining the scan and biopsy. The physician was allowed to use any treatment, which could include methotrexate or another immunosuppressive therapy. The duration of use of therapies (apart from high-dose steroids) did not influence eligibility for the study.

The clinicians recorded the baseline and pre-steroid clinical features for all patients with suspected GCA. They were also given access to any available blood tests results and could request any investigation apart from a temporal artery and axillary ultrasound scan. In 10% of cases no baseline ESR result was available, and in 27% of cases no pre-steroid ESR result was available; in 8% of cases there was no baseline CRP level was available and in 27% of cases no pre-steroid CRP value was available.

Table 20 shows the initial diagnosis and treatment recommended for the patients in the study. In 21% of patients the clinicians reported definite GCA, in 54% they reported probable GCA and in 25% they reported possible GCA. The level of certainty of diagnosis of GCA is potentially biased in the data set because all patients were required to have at least the possibility of GCA in order to be eligible for inclusion in the study. It is conceivable that although the GP who referred the patient might have thought it possible that the patient had GCA, the study clinician reviewing the patient may have thought otherwise. However, given the constraints of options available to the study clinician, they could define the patient only as having definite, probable or possible GCA. Therefore, the category of ‘possible’ GCA might actually contain a mixture of patients whom the study clinician might have considered did not have GCA and patients with a low likelihood of GCA. The majority of the patients were already in receipt of high doses of oral glucocorticoids (89%) at the time of the baseline visit. Very few patients (5%) were taking an immunosuppressive agent (only 18 out of the 381 patients) and, in all cases, these drugs were being given for other comorbid medical conditions rather than for suspected GCA (100% of the 18 patients).

Table 21 describes the clinical diagnosis made at 2 weeks and 6 months and shows that the majority of patients had a clinical diagnosis of GCA at both 2 weeks (67%) and 6 months (70%). In other words, most patients who were initially diagnosed as having GCA did not have any change made to their clinical diagnosis. However, in 19 patients the diagnosis was changed from not GCA to GCA following unblinding of the ultrasound results (after the 2-week visit the diagnosis has been reported). In a further 25 patients the diagnosis was changed at 6 months (this constitutes 6% of patients with available data); in 17 of these patients the diagnosis was switched from GCA to another diagnosis and in three cases the patients were reported as having a reference diagnosis of GCA. Twenty-one patients had their diagnosis changed following expert review of all the clinical data. In 13 of these patients the diagnosis was changed from GCA to another diagnosis, and in eight patients the diagnosis was changed from not GCA to GCA.

There were fewer data available at 6 months than at 2 weeks (46 fewer patients available at 6 months). Three patients were diagnosed with other forms of vasculitis at the 2-week visit and five patients had a diagnosis of another form of vasculitis at 6 months. These data highlight the potential overlapping presentation between different forms of vasculitis. In patients who did not have GCA or any form of vasculitis, non-specific headache was the most common diagnosis made (14% at 2 weeks and 12% at 6 months).

Table 22 shows the features present at 2 weeks and 6 months that were reported as influencing the clinician in making a diagnosis of GCA. There was consistent influence from the clinical symptoms (98%), signs (70%) and blood abnormalities (65%) at the 2-week visit: biopsy results influenced findings in 40% of cases. For the three cases diagnosed as GCA at 6 months but not 2 weeks, it is difficult to comment on the pattern of influence, but it looks similar to the findings at baseline.

Characteristics and outcomes over time

The prevalence of diabetes mellitus increased from 14% at baseline to 18% at 6 months. By contrast, other conditions appeared to be unchanged in frequency across the visits (Table 23). Twenty-four participants had new-onset hypertension during the follow-up period and five participants who had documented hypertension at baseline no longer had it reported as an active condition during the follow-up period. Four fractures occurred during the 6-month follow-up. The fracture that occurred at 2 weeks was of the spine/vertebrae.

Physical examination findings at each study visit are shown in Table 24. The number of abnormalities decreased at each study visit. The prevalence of thickened temporal artery fell from 50% at baseline to 13% at 2 weeks which would be in keeping with the expected clinical resolution of the physical findings of the disease as a result of treatment.

Primary analysis

The primary outcome was the performance of ultrasound and biopsy in relation to the reference diagnosis of GCA. The reference diagnosis (defined in Chapter 2) for each patient was based on the 2-week and 6-month clinical diagnosis, as well as on the opinion of an expert review panel that assessed patient data (without the ultrasound result).

Main results: robustness to variations in sample, biopsy diagnosis and reference diagnosis

Variations in ultrasound

We reviewed the variations in the interpretation of the ultrasound findings in the context of a diagnosis of GCA in order to investigate whether or not the sensitivity and specificity of ultrasound for diagnosis of GCA could be improved.

Two-week diagnosis and test findings

Ultrasound: learning effect

Ultrasonography of the temporal arteries is operator dependent; therefore, as part of the study protocol, training was given to sonographers at the beginning of the study to ensure proficiency with the technique before applying it to study participants. Some sonographers with sufficient experience were deemed exempt from the full training. The results of the training attempts are shown in Chapter 3. In addition to providing the training, as part of the protocol, all scans performed by each site’s sonographer were recorded and the images were sent to the TABUL office in Oxford, so that they could be uploaded onto a server for assessment by the expert reviewers. Scans were reviewed during the course of the study as part of the quality control and some sonographers were retrained if necessary.

The ultrasound scans are split into two groups:

1. First 10 – this included the first 10 scans (post training) within the TABUL cohort for each sonographer who received full training, or all scans before retraining for those sonographers who subsequently received further training.

2. After 10 – included all patients after the first 10 scans for the sonographers who received full training. It also includes all scans from sonographers exempt from full training. If a sonographer was retrained during the study, it included all scans after the date of retraining.

The specificity of ultrasound was almost the same for the first 10 scans (82%) and the later scans (81%), but the sensitivity was higher for the later scans (increasing from 45% to 62%), as shown in Table 34. This strongly suggests that there is a learning effect, as the sonographers become more experienced at performing the scan, and that this will predominantly influence sensitivity of the test result.

We repeated the analysis, excluding the sonographers deemed to be experts and exempt from full training (leaving n = 110 patients), splitting the data based on the first five scans versus the rest of the scans. There appears to be an improvement in sensitivity in the scans assessed between the first five patients scanned and subsequent patients scanned (rising from 42% to 59%, respectively), with similar levels of specificity (80% vs. 82%).

Timing effect

Sequential and combined test analyses

Performing both ultrasound and biopsy may not be necessary in all patients. We would speculate that ultrasound could be useful as a ‘rule-in’ test to support the diagnosis of GCA. If the ultrasound result was consistent with GCA and the clinical features supported that diagnosis, the diagnosis of GCA could be made without any further testing required. If, however, ultrasound was not consistent with GCA, patients would be recommended to have a biopsy in order to help to decide whether or not they had GCA. This test strategy can be explored in the TABUL data set and lends itself to a full economic evaluation, which is provided in Chapter 7.

The following steps describe a potential algorithm for investigating patients with suspected GCA:

• Patients present with the clinical or laboratory features suggesting a diagnosis of GCA.

• An ultrasound scan is performed and, if the results show evidence supporting a diagnosis of GCA, a diagnosis of GCA is made.

• If the ultrasound scan does not show features consistent with GCA, the patient is scheduled to have a TAB.

• If the TAB is supportive of a diagnosis of GCA, the patient is diagnosed with GCA.

• If both TAB and ultrasound are negative, the conclusion is that the patient does not have ultrasound or histological evidence to support the diagnosis of GCA.

Table 38 illustrates the effects of applying this sequential strategy on the 381 patients in the TABUL study. Overall, 162 patients would have been diagnosed with GCA based on the ultrasound scan alone, the majority (139, 86%) correctly. The remaining 219 ultrasound-negative patients would then have a biopsy. Twenty-seven of these ultrasound-negative patients had a positive biopsy and would also have been diagnosed with GCA. The 192 patients who were both scan and biopsy negative would not have received a diagnosis of GCA despite the fact that in almost half of these patients the reference diagnosis was GCA.

Table 38 shows the number of patients who would have a positive or negative result on the test compared with the eventual reference diagnosis via not GCA or GCA.

Table 39 shows the accuracy of applying a sequential strategy to the TABUL cohort. The effect of applying a second test, the biopsy, to patients who are ultrasound negative is to improve on the sensitivity of the ultrasound-only strategy (from 54% to 65%) while maintaining its specificity at 81% (although specificity is lower than the 100% obtained for a biopsy-only strategy). If this strategy was used for the cohort, 162 (43%) patients would have avoided having a TAB. However, on the basis of this strategy, without a clinician over-riding (ignoring) the test results, 91 true cases of GCA as defined by the reference diagnosis would have been missed and 23 patients would be wrongly diagnosed as having GCA according to the reference diagnosis. It is a difficult dilemma because no single test or evaluation can be used to rule out the diagnosis, whereas any single test or evaluation could be used to rule it in, over-riding a negative result.

Diagnostic strategies

We considered the implications of introducing a test strategy dependent on the pre-test probability of a patient having or not having a diagnosis of GCA. As the prevalence of GCA is very high in the high-risk group (93%), one strategy could be not to perform either ultrasound or biopsy in this group and simply diagnose the patients as having GCA without any further testing (we have defined these patients as H0). Although this would be most economic, by avoiding either test, in clinical practice both clinicians and patients would find it difficult to accept the diagnosis without at least some attempt to support the diagnosis with further investigation (biopsy or scan). We would therefore also consider a strategy of performing an initial ultrasound in the high-risk group and then performing a biopsy if the scan is negative (i.e. the scan is not consistent with a diagnosis of GCA). We would define a positive result on ultrasound as consistent with a diagnosis of GCA using four possible criteria as follows.

1. The sonographer’s opinion is that the ultrasound scan is consistent with a diagnosis of GCA (defined as H1).

2. Bilateral halo is present (in either the temporal or axillary arteries) (H2).

3. Either the sonographer’s opinion is that the ultrasound is consistent with a diagnosis of GCA or there are abnormalities in the axillary arteries (regardless of the overall sonographer opinion) (H3).

4. Bilateral halo or any axillary involvement is present (H4).

In the medium-risk groups we considered the above four strategies in which ultrasound is performed first, followed by biopsy (M1 to M4 would be equivalent to H1 to H4).

In the low-risk groups, we considered the same four strategies as well as two further strategies.

1. Using a negative ultrasound result as a ‘rule-out’ test for GCA. If ultrasound is positive, then perform a biopsy and take the diagnosis from the biopsy result (L5).

2. Use the absence of any abnormal finding on the ultrasound as a ‘rule-out’ test for GCA. If there are any abnormalities, perform a biopsy and take the diagnosis from the biopsy result (L6).

The accuracy of the diagnostic test strategies for each subgroup is shown in Table 42. These strategies are combined and the accuracy of all possible combinations displayed in Figure 12 (for full results, see Appendix 15) alongside point estimates for biopsy and ultrasound alone. It is apparent from the graph that TAB alone provides relatively poor performance in helping to diagnose GCA; by contrast, many of the combined strategies have better sensitivity and specificity than ultrasound alone. The two combined strategies that give the highest sensitivity are H0-M1-L1 and H0-M1-L3.These combine no test in the high-risk group, testing with ultrasound first and following with biopsy if the ultrasound result is negative in the medium- and low-risk groups, or following with biopsy if the ultrasound is negative and there is no axillary involvement in the low-risk group.

FIGURE 12.

Sensitivity and specificity for the diagnostic strategy combinations. US, ultrasound.

Appendix 15 contains an extensive list of combinations of different strategies that could be applied to improve accuracy in diagnosing GCA, dependent on the initial pre-test probability of the diagnosis being high, medium or low.

Exploratory findings

Participation in the agreement and vignette exercises

Twenty sonographers from 16 sites and 26 pathologists from 19 sites were asked if they wished to form the TABUL sonographers and TABUL pathologists groups that were responsible for the inter-rater agreement exercises. Twelve sonographers from 10 centres and 14 pathologists from 13 centres joined the groups and completed the exercise. Some centres had no eligible sonographers or pathologists to invite because of their involvement as expert reviewers or designers of the exercise.

Twenty clinicians with experience in managing GCA and involved in, or associated with, the TABUL study were invited to review vignettes for the clinical vignette exercise. Sixteen indicated that they were able to do the exercise and 14 completed assessments of all 30 clinical vignettes.

Selection of patients

A total of 255 initially eligible patients were identified at the first stage of selection; the first 33 patients from the randomly ordered list were selected for further screening. Twelve (36%) of these patients did not meet the inclusion criteria and were replaced with the next 12 eligible patients. Following piloting of the exercise and checking of the videos and images in the third stage, a further two patients were replaced because the ultrasound images were considered to be of inadequate quality. Three further patients were highlighted because of concerns about the ultrasound videos; two were retained because it was deemed that their videos were difficult to interpret rather than of inadequate quality; and one case was modified to include an alternative video for the same patient that better supported the original sonographer’s interpretation. Finally, one of the rating cases was replaced post exercise with one of the three rated reserve cases because the patient did not complete a follow-up assessment and was excluded from the main analyses. Therefore, there were 30 unique cases, six of which were repeated, for evaluation by 12 sonographers and 14 pathologists.

Inter-rater agreement between sonographers and pathologists

Ratings based on images alone

Sonographers and pathologists rated each case as either consistent with GCA or not consistent with GCA; they were also asked to report the confidence they had in their findings, using four categories to indicate the level of certainty in their decision. The rating was done before and after seeing a brief clinical vignette describing a few key characteristics of the patient.

The distribution of the results (consistent with GCA or not consistent with GCA) by the sonographers for the 30 original cases assessed before being shown the clinical vignette is shown in Figure 13. The 12 sonographers unanimously agreed in 10 of the 30 cases: four as GCA positive and six as GCA negative. In half the cases there was no unanimous agreement, but no more than two of the sonographers differed from the majority view. In five cases there was greater disagreement, with three or four sonographers differing from the majority.

FIGURE 13.

Frequency of evaluations consistent with or not consistent with GCA by 12 sonographers rating 30 cases.

All 14 pathologists agreed unanimously on 11 cases, six of which were consistent with GCA and five of which were not consistent with GCA (Figure 14). There were 13 cases in which no more than two pathologists differed from the majority view. In six cases, there was greater disagreement, in one of which opinion was evenly divided, with equal numbers of pathologists defining the patient as having or not having GCA.

FIGURE 14.

Frequency of evaluations consistent with or not consistent with GCA by 14 pathologists rating 30 cases.

Eight of the 30 cases involved patients who had been assessed as biopsy positive by the original reporting pathologist. All eight cases reported evidence of giant cells and these were the eight cases in the exercise that all (six cases), or all except two (two cases), of the pathologists judged to be consistent with GCA. A ninth patient was reported as biopsy negative by the original reporting pathologist but interpreted as biopsy positive by the clinician based on the abnormalities described in the biopsy report (intimal hyperplasia, fragmentation and reduplication consistent with previous GCA but no active inflammation). Two of the pathologists judged the case to be GCA positive, but most concurred with the original biopsy report and judged the case to be GCA negative.

For each GCA-positive or GCA-negative assessment, the sonographers and pathologists were asked to indicate if they were certain or uncertain in their assessments. Analysis of differences in the certainty of assessments indicated that sonographers judged fewer ratings as certain than did pathologists. For GCA-positive ratings, 69.0% were judged as certain by sonographers, whereas 79.8% were judged as certain by pathologists. For GCA-negative ratings the sonographers were certain for 54.5%, whereas the pathologists were certain for 71.0%. However, a comparison between the 14 pathologists and the 12 sonographers in the proportion of cases judged as certain did not provide strong evidence for a difference (Wilcoxon rank-sum test, p = 0.13). The distribution of these ratings is shown in Figures 15 and 16.

FIGURE 15.

Frequency of certain and uncertain positive and negative assessments by 12 sonographers rating 30 cases.

FIGURE 16.

Frequency of certain and uncertain positive and negative assessments by 14 pathologists rating 30 cases.

Intrarater agreement for sonographers and pathologists

Intrarater agreement was evaluated by including repeats of six of the cases during the exercise. Overall, there were 10 instances of inconsistent assessments between the original and repeated cases made by the 12 sonographers and seven made by the 14 pathologists (Table 52). Overall, raw agreement was 86.1% for the sonographers and 91.7% for the pathologists.

Analysis of the consistency of individual sonographers and pathologists is shown in Table 53. No individual assessor was inconsistent for more than two of the cases. Over half (57%) of the pathologists and one-third of the sonographers were completely consistent in their assessment of the six repeated cases. However, no statistically significant difference was observed between the 12 sonographers and the 14 pathologists in the number of cases that were inconsistent (Wilcoxon rank-sum test, p = 0.21).

Kappa statistics were used to estimate ‘chance-corrected’ intrarater agreement for the sonographers and pathologists. For the six of the cases that were repeated, the 14 pathologists achieved raw agreement of 91.7% for categorising cases as GCA or not GCA and an overall kappa statistic of 0.83 (Table 54). The 12 sonographers achieved raw agreement of 86.1% and an overall kappa statistic of 0.69 for the same cases. Once ratings are reported using four categories to allow for certainty in the assessments by the sonographers and pathologists, the weighted kappa statistics for agreement are similar: 0.85 for the pathologists and 0.81 for the sonographers.

Analysis of clinical vignettes with ultrasound in the absence of biopsy

Fourteen clinicians reviewed the 30 clinical vignettes and provided responses for clinical decisions at two stages: (1) the likelihood of a diagnosis of GCA and whether or not to perform a biopsy after seeing information from a patient’s initial presentation; and (2) the likelihood of a diagnosis of GCA and whether or not to continue with high-dose steroids after seeing a brief written summary of the patient’s ultrasound results and clinical information after 2 weeks.

In 21 (70%) of the vignettes, the majority of the panel considered the likelihood of GCA to be probable or definite, and for two vignettes the panel was split evenly (Table 55). In the remaining seven vignettes, which were considered least likely to be GCA, the majority of the panel would perform a biopsy; the one exception was for vignette case 8, for which only five of the 14 panel members would recommend a biopsy.

There was some evidence of an association between certainty of GCA and recommendation for biopsy. Members of the panel were generally consistent in not recommending a biopsy for patients whom they considered not to have GCA; biopsy was indicated for 10% of the time in these cases. Biopsy was most frequently recommended (94% of the time) by panellists for vignettes judged as probable GCA. The percentage of biopsy recommendations was lower for vignettes judged as definite GCA (78% recommended for biopsy) and those judged as possible GCA (80% recommended for biopsy). These findings suggest some reluctance to recommend biopsies in patients considered to have little chance of having GCA based on vignettes describing their symptoms and results of physical examinations and blood tests. The findings also suggest some reluctance to recommend biopsies in patients who were regarded as having very clear-cut evidence of GCA, based on their clinical presentation and the results of blood tests. However, in patients who are diagnosed as having GCA without undergoing a biopsy, there may be a concern that there is no irrefutable evidence of GCA if the diagnosis is subsequently questioned; by contrast, the clinician’s interpretation is perceived as always open to reconsideration.

Table 56 describes the panellists’ assessment of the diagnosis of GCA, once the ultrasound test result was revealed and information about the symptoms, blood tests and physical examination had been provided. Panel recommendations for continuing treatment for GCA with high-dose steroids were categorised as agree, disagree or uncertain. The uncertain category was used when the median rating of the 14 panellists lay in the mid-range or if there was wide variation (indicated as disagreement) in the recommendations of the panellists regardless of the median. In 11 of the cases, the ultrasound test results were reported as consistent with GCA, and panel members’ views were weighted strongly towards a definite diagnosis of GCA for these vignettes. The panel members were also in agreement that high-dose steroids for GCA should be continued for all 11 cases.

There were 19 ultrasound-negative vignettes and there was a reluctance to classify any of these vignettes as definite GCA. In only 4 of the 19 vignettes did a majority of the panel categorise the patient as having probable or definite GCA; in three of these four cases, the panel was in agreement that it was appropriate to continue with high-dose steroids and in the fourth case the panel was uncertain, owing to disagreement. Of the remaining 15 vignettes, the panel agreed that it was inappropriate to continue with high-dose steroids in five of the case vignettes and was uncertain in the others. There was one vignette, number 20, for which the biopsy was positive but the ultrasound was reported as negative. Despite the fact that the panel members were not aware of the positive biopsy result, they were still in agreement that it was appropriate to continue with high-dose steroids.

Introduction

The economic evaluation of the two tests needs to consider any differences in the diagnostic accuracy between them, as well as the costs and impact of the tests in terms of the development of GCA-related complications, treatments and related side effects.

The starting point for the modelling is the statistical output showing the sensitivity and specificity of the two individual tests and any diagnostic strategies which incorporate them (see Chapter 5). Sensitivity is the proportion of patients with true GCA who are detected by the test or strategy; the remaining proportion is made up of ‘false negatives’, that is, patients who test negative despite having GCA. Specificity is the proportion of patients without GCA who are classified as negative by the test or strategy; the remaining proportion is made up of ‘false positives’, that is, patients who test positive but who do not have GCA. A problem with false-negative and false-positive results is that patients falling into these categories may initially be managed in a different way, with potentially adverse consequences, compared with how they would have been managed had their true disease status been known earlier. The economic analysis estimates the relative cost-effectiveness of the alternative tests and strategies by quantifying and trading off the following.

• The different costs of the tests or strategies.

• The different proportions of false negatives and false positives.

• The cost and health-related quality-of-life impact of a false negative, that is, when a patient remains undetected with GCA for up to around 2 months, with the attendant risk of developing complications such as vision loss.

• The cost and health-related quality-of-life impact of a false positive, that is, initiating or continuing treatment with high-dose steroids in a patient without GCA for many months and the impact that any unnecessary treatment has on the risk of AEs such as fractures, diabetes mellitus and weight gain.

The primary objective of the economic analysis is to estimate the cost-effectiveness of ultrasound instead of biopsy for the diagnosis of GCA. The secondary objective is to estimate the cost-effectiveness of performing a biopsy following ultrasound as an alternative to TAB alone in the diagnosis of GCA. In addition, alternative diagnostic strategies have been evaluated using estimates of sensitivity and specificity from statistical modelling (as described in Chapter 5).

Biopsy and ultrasound are also evaluated when used in conjunction with clinical judgement, that is, the clinician’s decision on the diagnosis at 2 weeks based on knowledge of the patient’s symptoms, signs and available test results such as blood tests and the biopsy. This more closely reflects current clinical practice of using biopsy results to aid the clinical diagnosis rather than to define the diagnosis.

Methods

In this section, the model structure is described, followed by details of the evidence sources used for the various parameter values in the model. These include the performance of the diagnostic testing strategies; risks of GCA-related complications and glucocorticoid-related AEs; and associated costs and health-related quality-of-life effects. The costs of the tests and medications are also covered.

The development of the economic model structure was informed by evidence from published research on GCA in order to understand the main complications of the disease and steroid-related side effects. This was supplemented with evidence from previous economic and decision-analytic studies of GCA78,79 and an analysis of outcomes and cost-effectiveness of a fast-track service for GCA. 80

Model structure

The model structure takes the form of a combination of three submodels: first, a decision tree for the initial diagnostic testing; second, a risk submodel of the incidence of GCA-related complications and steroid-related AEs over 2 or 3 years; and third, a submodel of the lifetime effects of these incident complications and AEs. The model structure is shown in Figure 17.

FIGURE 17.

Economic evaluation model structure.

Risks of complications of giant cell arteritis

Visual complications

Visual complications represent the greatest burden of complications of GCA, with about 25% of cases resulting in sight loss if left untreated. 85 The major presenting symptoms are amaurosis fugax (a transient shade, dimming, fogging, blurring or monocular blindness), transient diplopia (double vision) or unilateral or bilateral partial or complete vision loss.

For the economic model, we needed to identify the risk of onset of visual complications after patients had presented to their GP, because an estimated 92% of visual complications arise prior to the initiation of high-dose steroid treatment and therefore would not be affected by the diagnostic strategies considered in TABUL. To do this, we created a submodel of visual complications, combining and modelling data from various sources, as shown in Figure 18.

FIGURE 18.

Logic modelling of evidence to obtain incidence rates of new onset of visual complications. a, relative proportion of TPs : FNs assumed to be 90 : 10 after initial diagnostic test; so, of the 1000 GCA cases, 900 would be TPs and 100 would be FNs. FN, false negative; TP, true positive.

Blindness in both eyes is rare in GCA86 because steroid treatment is usually started when sight loss occurs in one eye and should reduce the risk of sight loss occurring in the other eye. It is therefore assumed that there will be no cases of bilateral sight loss and that steroids will have been started in all cases of unilateral sight loss. The stages during the diagnostic and treatment pathway during which visual loss arises are illustrated in Figure 17, based on 30% of patients experiencing visual complications,87 15% experiencing permanent visual loss87 and 92% of visual complications arising before treatment as initiated. 88 Of this 92%, one-fifth of complications are assumed to be attributable to an initial false-negative diagnosis. Eight per cent are estimated to arise in true positives after steroid treatment has started. The required estimates of incidence rates of new visual loss among true positives and false negatives are shown by the solid arrows.

In order to assign costs of treatment and the quality-of-life impact of visual complications, we required an assessment of severity, based on a previously reported analysis89 (Table 61).

TABLE 61 - Analysis of the severity of visual loss by initial visual acuity in one eye

Rates calculated from data in Hayreh et al. 89

Visual acuity	Oral therapy or intravenous therapy, n (%)
20/50–20/70	12 (13)
20/80–20/100	4 (4)
20/200–20/400	5 (6)
Counting fingers	19 (21)
Hand motion	15 (17)
Light perception	13 (15)
No light perception	21 (24)

Although visual acuity is the primary criterion for determining vision loss, other types of vision loss (e.g. peripheral vision loss or contrast sensitivity loss) are recognised as disabilities even if central visual acuity is 20/20. Partial sight loss in the centre of vision is different to partial sight loss in the periphery, but we have no information on the nature of GCA-induced visual loss.

Stroke

For the incidence of GCA-related stroke, the models assume that 2.64% of cases of GCA result in a stroke, as per Amiri et al. ,90 and further assume that strokes arise after presentation to the patient’s GP. It is also assumed that stroke occurring as a result of GCA has the same severity and likelihood of fatality as stroke unrelated to GCA. Sixty per cent of strokes were assumed to be minor; case fatality in major strokes was assumed to be 50%.

Mortality from giant cell arteritis

There have been numerous studies reporting an increased risk of mortality in the years following a diagnosis of GCA. However, we decided that it was not necessary to include this in the model because there is no evidence to suggest that a delay in the diagnosis of several weeks (as a result of an initial false-negative test result) has an impact on this mortality risk. Hence, it is unlikely to have any impact on the relative cost-effectiveness of different test strategies.

Results

In this section, results are presented for the base case, then for the various sensitivity analyses, and the budget impact, all based around the diagnostic reference standard applicable in the TABUL study. We then investigate how varying the reference standard changes the results.

The two main measures of cost-effectiveness are the incremental cost-effectiveness ratio (ICER) and net monetary benefit (NMB). Both of these are all-encompassing measures that trade off additional costs of diagnosis, medication and treatment of complications against benefits in terms of improved life expectancy and quality of life (e.g. through reduced incidence of blindness or reduced incidence of fractures). Central to these measures is:

1. The QALY; for example, 2 years spent with a utility of 0.6 gives 1.2 QALYs.

2. The value placed on 1 QALY gained [often referred to as the willingness-to-pay (WTP) threshold], which, in the UK, is stated by NICE to be typically in the range £20,000 to £30,000 per QALY. We will use a threshold of £20,000 per QALY for our analysis because this is more usual for groups that are not disadvantaged.

The preferred measure is the ICER. This shows how cost-effective one strategy is compared with another by dividing the incremental costs by the incremental QALYs, but this can become complex to present when there are many strategies. We shall therefore report ICERs to compare a small number of strategies, but we shall use the NMB to compare the cost-effectiveness across all strategies. The NMB is the overall monetary value of a screening/treatment strategy taking account of both costs and health benefits, with the health benefits valued at £20,000 per QALY. The higher the NMB, the more cost-effective a strategy is; this allows easy comparison across multiple strategies.

To calculate the NMB of a strategy, the steps are:

1. Calculate the total costs incurred (including the cost of the tests, medications and treatment of complications and AEs).

2. Calculate the total QALYS over the model time horizon, in this case 40 years.

3. Multiply the total QALYs by the WTP threshold of £20,000 per QALY.

4. Deduct the costs calculated in (1) from the value in (3) to obtain the NMB.

Base-case results

In Table 71, results are shown for various alternative diagnostic strategies, all assuming base-case model parameters.

TABLE 71 - Results for alternative screening strategies

US, ultrasound.

Versus ‘biopsy only’ strategy, measured per patient (£).

Rank in order of cost-effectiveness.

Defined as < 20/200 and reported as percentage of true GCA cases.

H0, in high-risk group, assume GCA positive (no diagnostic test performed); M1, in medium risk group, GCA if either ultrasound or biopsy positive; M5, GCA if ultrasound positive; L1, in low risk group, GCA if either ultrasound or biopsy positive; L3, GCA if either ultrasound positive, ultrasound axillary involvement or TAB positive; L7, GCA if ultrasound positive.

Strategy	Sensitivity (%)	Specificity (%)	Having US (%)	Having TAB (%)	Total costs per patient (£)	Total QALYs per patient	NMB at £20,000/QALY per patient (£)	Incremental NMB per patient (£)a	Cost-effectiveness rankb	New onset irreversible vision loss (%)c	Fractures over 2 years among cohort (%)
Technology-only strategies
Biopsy only (all patients)	39	100	0	100	1965	7.5958	149,950	–	13	0.60	5.17
US only (all patients)	54	81	100	0	1371	7.6036	150,701	751	10	0.51	5.49
Biopsy and US (both in all patients)	65	81	100	100	1757	7.6162	150,567	617	11	0.43	5.49
US followed by biopsy if US negative	65	81	100	57	1538	7.6162	150,786	836	8	0.43	5.49
Technology followed by risk factors
US and biopsy with additional prognostic baseline factors	67	81	100	57	1512	7.6185	150,857	907	7	0.42	5.49
Biopsy and age and claudication with 81% specificity	68	81	0	100	1664	7.6196	150,728	778	9	0.41	5.49
Biopsy and age and claudication with 90% specificity	59	90	0	100	1757	7.6132	150,507	557	12	0.47	5.34
Pre-test probabilities used to filter who needs a testd
Composite pre-test strategy H0M1L1	72	77	77	46	1389	7.6225	151,060	1110	5	0.38	5.56
Composite pre-test strategy H0M1L3	72	75	77	46	1392	7.6216	151,041	1091	6	0.38	5.59
Composite pre-test strategy H0M5L7	68	77	77	0	1200	7.6179	151,159	1209	4	0.41	5.56
Technology and clinical judgement (proportion continue with steroids)
Two-week decision: biopsy and judgement	91	81	0	100	1396	7.6459	151,523	1573	3	0.26	5.49
Two-week decision: US and judgement	93	77	100	0	921	7.6464	152,008	2058	1	0.24	5.57
Two-week decision: biopsy and US and judgement	96	73	100	100	1406	7.6482	151,558	1608	2	0.22	5.63

Columns 2 and 3 show the performance of each screening strategy. Columns 4 and 5 show the proportion of patients who would undergo each test. Columns 6–10 are the economic outcomes. Column 8 is the NMB measure of cost-effectiveness. The NMB figures for each strategy appear to be of roughly the same magnitude, and, although this might suggest that they are all almost the same, this would be an incorrect interpretation. The higher the incremental net benefit in column 9 of a given strategy compared with the biopsy-only strategy, the more cost-effective that strategy is. It should be remembered that these monetary differences are per patient. The budgetary impact of selected strategies is explored later. Column 10 shows the ranking of each diagnostic strategy in terms of cost-effectiveness (based on the NMB); the lower the ranking the more cost-effective the strategy. The last two columns show two clinical outcome measures.

It may be easier to understand how the results compare visually on a cost-effectiveness plane, as shown in Figure 19. The most cost-effective strategy is indicated by bold font, that is, ‘2-week decision: ultrasound and judgement’. The green dotted line is known as a cost-effectiveness threshold, and it represents a line along which any point would have the same cost-effectiveness (any point has a cost-effectiveness ratio of £20,000 per QALY relative to this strategy). Any points below the line have a more favourable ratio of additional costs to additional benefits and would be a more cost-effective option (if there were any). Any points above the line are not cost-effective. In the case of the strategy ‘2-week decision: combined biopsy and ultrasound and judgement’ connected by a blue dashed line, the gradient is clearly much steeper than the green dotted line, indicating that the marginally higher QALY gains are not achieved in a cost-effective way. Numerically, the additional 0.0018 (7.6482 – 7.6464) QALYs cost an extra £485 (£1406 – £921), giving an ICER of £271,864, which far exceeds the acceptable threshold of £20,000 per QALY. For all other strategies, both the costs and QALYs are inferior (higher cost and fewer QALYs) compared with the optimal ‘Two-week decision: ultrasound and judgement’ strategy, which is thereby said to dominate these strategies (including ‘Biopsy and judgement’).

FIGURE 19.

Cost-effectiveness plane showing the results for each strategy. Bold indicates the most cost-effective strategy.

In light of these results, we undertook some further refinement of the ultrasound and clinical judgement strategy as shown in Table 72.

TABLE 72 - Further exploratory analyses of the ultrasound plus judgement strategy

US, ultrasound.

Rank in descending order of cost-effectiveness.

Defined as < 20/200 and reported as percentage of true GCA cases.

Strategy	Sensitivity (%)	Specificity (%)	Having US (%)	Having TAB (%)	Total costs per patient (£)	Total QALYs per patient	NMB at £20,000/QALY per patient (£)	Cost-effectiveness ranka	New onset irreversible vision loss (%)b	Fractures over 2 years among cohort (%)
Further exploratory analyses: when US and judgement decision is not GCA refer for biopsy in some cases
Refer for biopsy if high risk	94	77	100	2	920	7.6478	152,035	1	0.24	5.57
Refer for biopsy if medium or high risk	95	77	100	13	965	7.6487	152,009	2	0.23	5.57

The results lead to the following findings:

1. The most cost-effective strategies are those that include an element of clinical judgement.

2. Ultrasound and clinical judgement is the most cost-effective strategy, with the highest incremental NMB. This is largely because of the difference in the cost of the tests (Table 73).

3. For the strategy in (2) above, the estimated cost saving is £475 patient and there is a very small QALY gain of 0.0005 compared with biopsy and judgement. Rather than calculating an ICER, ultrasound is said to dominate biopsy in this case as ultrasound results in both cost savings and QALY gains.

4. Ultrasound alone is more cost-effective than biopsy alone.

5. The three sequential diagnostic strategies that incorporate pre-test probabilities (those ranked 4, 5 and 6) offer a level of cost-effectiveness between those involving clinical judgement and those (ranked 7 to 13) that include neither clinical judgement nor pre-test probabilities.

TABLE 73 - Differences in costs and QALYs

US, ultrasound.

Lower or higher stated for ultrasound relative to biopsy.

Ultrasound minus biopsy.

The £485 reconciles to the difference between the incremental NMB figures of £1680 and £2173 in Table 71.

Cost or QALY elementa	Differenceb
Lower test cost of US	–£456
Lower cost of treating complications of GCA in false negatives	–£27
Higher cost of steroids and treating AEs in (mainly because of difference in false positives)	£8
Total cost difference	–£475
Lower QALY loss from GCA complications in false negatives	0.0023
Greater QALY loss from overtreatment of false positives	–0.0017
Other difference	–0.0001
Total QALY difference	0.0005
Monetary value of QALY difference at £20,000 per QALY	£10
Incremental NMB (–475 to 10)c	–£485

A further finding from the additional analyses in Table 72 is that the ultrasound and judgement strategy may be improved slightly by undertaking a biopsy in cases in which the pre-test risk is high and the ultrasound and judgement decision would be not to treat. It should be noted that only 2% of individuals in TABUL were referred for biopsy under such a strategy, so there is some uncertainty around the benefit of a biopsy in such circumstances. It would also require the timing of the decision to perform a biopsy to be made after the outcome of the ultrasound plus judgement strategy is known. This is likely to mean that the biopsy is delayed (so may be less accurate than in our model because of the change in histology since patient presentation). Alternatively, an earlier biopsy would be possible if an ultrasound plus judgement outcome was obtained before 2 weeks. However, this would mean that there is less information available to the clinician on the patient’s symptoms and response to steroid treatment which, in turn, may lead to a less accurate outcome as a result of a more rapid ultrasound and clinical judgement strategy.

Results based around an alternative reference standard

All of the results presented so far have been based on the reference diagnosis defined for the TABUL study. In this section, we show the impact of alternative reference diagnoses that involve fewer true GCA cases by removing some cases that rely solely on clinical judgement.

The results and their interpretation are best shown graphically (Figure 20). The x-axis shows four reference standards: the one used in the TABUL study and then the three alternatives described earlier, with increasing proportions of cases for which results might be considered more borderline. The y-axis shows the incremental NMB of the four selected alternative diagnostic strategies compared with biopsy alone.

FIGURE 20.

Incremental NMB for four diagnostic strategies compared with biopsy alone according to reference standard adopted. H0M5L7, treat all high-risk patients, diagnosis for low-/medium-risk patients as per ultrasound result; US, ultrasound.

The results show that, for all alternative reference standards tested, ultrasound plus clinical judgement remains the most cost-effective strategy. It is only by adopting a reference standard with a significant reduction in cases of GCA (16% fewer GCA cases than in the TABUL cohort) that a diagnostic strategy based on pre-test risks and ultrasound might potentially become as cost-effective as ultrasound combined with clinical judgement.

Discussion

Main findings

We conducted a prospective multicentre study to compare the relative value of ultrasound assessment of both temporal and both axillary arteries with TAB in 381 patients with newly suspected GCA. In order to ensure proficiency in performing ultrasound scans, we created an extensive training programme, which was then compared with the established standard procedure of TAB, usually from the most symptomatic side, in this patient population. No training was provided for performance of the biopsy within the study. All patients in the study underwent both tests in sequence (ultrasound first followed by biopsy) and our analysis included those who underwent both tests within 10 days of commencing high doses of glucocorticoids. Usual care was given to the patients by their clinicians. The ultrasound result was not revealed to the clinician caring for the patient, unless they specifically requested the result (because they were planning rapidly to reduce and withdraw glucocorticoid therapy) after they made their clinical diagnosis at 2 weeks’ follow-up, which was the main primary outcome in the study. A final follow-up assessment was conducted at 6 months in case the diagnosis had changed.

The main objectives of the study were to compare the diagnostic performance (sensitivity and specificity) of ultrasound as an alternative to biopsy for diagnosing GCA in patients who are referred with suspected GCA and in whom a biopsy was going to be carried out and to perform a cost-effectiveness analysis to compare different potential investigation strategies for diagnosing GCA, incorporating either or both ultrasound and biopsy. The original hypothesis was that ultrasound would be a more sensitive test than biopsy and would have a specificity of at least 83%.

Early studies suggested that biopsy had 95% sensitivity and 100% specificity for GCA;114 later studies reported somewhat lower results of around 68–69% sensitivity but very high specificity. 11,115 Patients who had a positive biopsy but who did not have GCA were reported to have other forms of vasculitis. 59

We wanted to compare the performance of ultrasound, which we predicted would provide 87% sensitivity and 83% specificity or higher. Among 381 patients who had ultrasound and TAB for suspected GCA, 101 (27%) had a TAB consistent with GCA and 162 (43%) had an ultrasound result compatible with GCA. The sensitivity of biopsy for diagnosis GCA was 39%, much lower than previously published; the specificity was 100%. By contrast, the sensitivity of ultrasound was 54% with a specificity of 81%. Therefore, we failed to find evidence to support our primary hypothesis because, although ultrasound was more sensitive than biopsy, it did not achieve specificity greater than 83%. Nevertheless, we demonstrated that the current sensitivity of biopsy is much lower than previously published and that, in comparison, the sensitivity of ultrasound is superior (14% higher). The specificity of ultrasound was 81%, which is lower than expected from our original hypothesis. We cannot conclude that ultrasound can replace biopsy, based on these findings. However, the data support a significant challenge to the role of biopsy as a ‘gold standard’ test for diagnosing GCA. A combination strategy using both tests in sequence, with all patients undergoing an ultrasound scan, but only scan-negative cases undergoing a biopsy, has 65% sensitivity and 81% specificity for the reference standard diagnosis of GCA. The addition of risk stratification based on initial clinical features and measures of ESR or CRP levels can further increases the sensitivity to 77.1% and specificity to 91.2%.

The cost-effectiveness analysis indicates that ultrasound alone is more cost-effective than biopsy alone largely because of its much lower cost (£58 vs. £514) and higher sensitivity (54% vs. 39%). The use of ultrasound combined with clinical judgement is not only more cost-effective than biopsy plus clinical judgement but is estimated to result in both cost savings (largely owing to the lower cost of ultrasound) and a very small QALY gain.

Patient details

A total of 730 patients were screened for the study: 430 participants were recruited from 20 sites in five countries (England, Ireland, Norway, Germany and Portugal); and 300 patients either did not meet the inclusion criteria or declined to participate. From the 430 patients included, there were 39 withdrawals prior to the primary analysis being performed (at the 2-week assessment) and a further 49 withdrawals after the primary analysis was performed. Of the remaining 391 patients, 10 were excluded from the primary analysis; hence, the primary analysis was performed on 381 patients in total. The average age of the cohort was 71.1 years and 72% of patients were female. The majority of patients (80%) were of white British ethnicity, and the remainder were from either a white Irish or other white background or a non-white background (3%). The majority of patients (88%) had significant new headache at presentation: fatigue was reported in 65%, generalised scalp tenderness in 59% and 51% had pain over one or both temporal arteries. PMR was present in 7%. Visual symptoms were frequently present at baseline (reduced or lost vision reported in 133, amaurosis fugax in 14 and double vision in 31 patients). By 2 weeks, three, five and zero patients experienced new loss of vision, amaurosis fugax and double vision, respectively; by 6 months, new reports of these visual features occurred in seven, two and six patients, respectively. Anterior ischaemic optic neuropathy was reported in 27 patients (7%) at baseline; posterior ischaemic optic neuropathy was reported in seven patients (2%); by 2 weeks these findings were present in 4% and 0.5% of patients, and by 6 months in 4% and 1% of patients, respectively. The results of inflammatory markers (ESR and CRP level) were not always available (10% of patients had no baseline ESR result and 8% had no baseline CRP level result). The median ESR at baseline was 43 (IQR 60–70) and the median CRP level was 46 (IQR 90–91). Many patients had hypertension at baseline (52%); 7% had angina and 14% had a previous history of cancer. There was a small increase in the occurrence of diabetes mellitus during the course of the study from 14% at baseline to 18% at 6 months. The main physical findings were of tenderness (50%) or thickening (27%) of one or both of the temporal arteries, which were less likely to be detected if the patients had received even a few days of steroid treatment.

Use of the reference diagnosis

There was no absolute gold standard that we could apply in this study to decide whether or not the patient definitely had GCA. Use of the ACR classification criteria for GCA34 included using the result of the biopsy; this would bias the interpretation of the clinician’s opinion in favour of stronger agreement with the biopsy test when it was positive, and perhaps bias it against that diagnosis if it was negative. We attempted to address this by including additional aspects of the patient’s condition that would be compatible with the clinical diagnosis of GCA, such as the presence or development of visual loss attributed to GCA, the presence of stroke or PMR. Other features such as jaw or tongue claudication or significant elevation of the ESR or CRP level (above 60 and above 40, respectively) would have contributed to the clinician’s assessment and the likelihood of diagnosing GCA. However, the interpretation of all the clinical features, laboratory findings and results from the specific investigations would have to be considered individually on a case-by-case basis. This would mean that the clinician could over-ride/ignore any individual results in favour of or against the diagnosis of GCA. This is a clear limitation of the current study. However, including the ability to adjust the reference standard diagnosis in light of the development of changes to the clinical state in the 6 months following initial assessment (e.g. the development of features consistent with the diagnosis of GCA or, equally, the development of features consistent with another diagnosis) strengthens the argument for using this reference standard diagnosis as the gold standard, albeit a less than perfect one.

The use of presenting features to predict the likelihood of a diagnosis was suggested by Gabriel et al. 116 In a review of > 500 patients, the likelihood of a negative TAB was increased substantially in the absence of claudication and the absence of significant elevation of the ESR. In the current study we used the reference diagnosis rather than the biopsy as the standard for the model, but with similar findings. A limitation of this study is the lack of a robust unequivocal standard for diagnosis against which each test could be compared. In the absence of this, diagnostic criteria are being developed in the DCVAS study which might provide a better surrogate gold standard than currently exists. The clinical evaluation of the patient at baseline and after 2 weeks has the strongest influence on the diagnosis at 6 months. Part of the difficulty is the concern of clinicians that if any of the clinical features, combined with measurement of the acute phase response, are suggestive of GCA, despite negative further testing (biopsy or imaging), there is a clearly demonstrated unwillingness to dismiss the diagnosis. Rather, the tests (biopsy or imaging) are being used to provide further enhancement of the clinical opinion.

Ultrasound training

Ultrasound has not yet superseded TAB as a diagnostic test. This may reflect the poor consistency of the scanning technique as a result of the lack of a standardised scanning protocol. We developed a standardised protocol that was implemented in 439 healthy controls and subsequently in patients with suspected GCA. We assessed each patient for evidence of typical ultrasound features of GCA: the presence of a halo surrounding the vessel wall, stenosis or occlusion of the vessel. A detailed scanning protocol was developed for all patients and controls. We reported the presence or absence of ultrasound features of GCA in each segment of each temporal artery (common, parietal, frontal proximal and frontal distal) and both axillary arteries. Sonographers were asked to acquire video and static images for each patient to ensure accuracy of findings. The sonographer measured and documented halo diameter (based on a normal range of up to 0.5 mm for the temporal artery and up to 1.0 mm in the axillary artery) and length; pulse Doppler measurements prior to and within a stenosis (confirmed if the highest maximum systolic velocity was over twice the lowest maximum systolic velocity); and arterial occlusion. Each study site sonographer was required to be proficient in the protocol by scanning at least 10 healthy controls, passing an online test showing normal and abnormal scans (pass mark > 75%) and scanning a patient with ultrasound evidence of active GCA. The scanning protocol was started by 33 sites, with only 22 sites completing the training in 6.7 months (range 0.2–16.4 months). A total of 439 controls were scanned across 31 sites (one sonographer covered three sites). The online test was passed by 39 sonographers (multiple sonographers at some sites) with an average of two attempts (range 1–4); 22 sonographers successfully scanned an active GCA patient, as validated by the expert panel. The longest delay in completing the training was a result of difficulty in recruiting a patient with active GCA, which was necessary prior to commencement of the main study. Common issues encountered were a lack of time away from clinical duties and locating a new suspected GCA case for the hot case assessment. We have created a bank of 857 sets of consistently recorded images of temporal and axillary arteries from patients with suspected GCA and from healthy controls. Expert review of the scans confirmed that the overall rate of disagreement was 16%. Quality and accuracy are imperative for the clinical use of ultrasound data in diagnosis. We have developed an effective protocol, including training, which ensures consistency and proficiency in scanning. The methodology can be adapted and extended to allow for additional artery assessment, including carotid, vertebral and subclavian, extending the value of a structured approach. We recommend the current study scanning protocol as the standard approach for diagnosis of GCA using ultrasound.

How could we improve on the ultrasound training programme in practice?

We developed a novel training programme as part of the current study. The programme was based on published evidence of performing ultrasound examination of GCA; most of the publications were from experts within the study investigator group. The basic elements of the training programme consisted of (1) a tutorial/lecture [which could be provided as a recording or annotated Microsoft PowerPoint^® (version 97–2003; Microsoft Corporation, Redmond, WA, USA) presentation], (2) hands-on training for novice sonographers (which would not be required by more experienced sonographers), (3) evidence of recorded images to show proficiency in performing scans on healthy individuals to demonstrate non-diseased temporal and axillary arteries (primarily done remotely) and (4) evidence provided by sonographers of recorded images to show their proficiency at performing scans in at least one individual with active GCA, to demonstrate diseased temporal or axillary arteries (primarily done remotely). We implemented the training requirements for the purpose of this study, which was deliberately based in non-academic as well as academic centres, in order to test the practicality of establishing this new technique of ultrasound in large numbers of local hospitals, where resources might be limited. We discovered significant variation in the uptake of the training, primarily driven by local factors such as the availability of sonographers and ultrasound machines; as a result, only half of the centres that originally attempted the training programme actually preceded with the study. Given the nature of the condition (i.e. presentation with acute-onset symptoms and the need to undertake scanning within a short time of starting steroid therapy), there are minimum basic requirements in any individual centre to ensure that the technique is performed to the correct standard and can be undertaken in a timely fashion. Furthermore, a minimum number of cases scanned per annum would be advisable to ensure ongoing quality control; we found that scanning reliability was higher for sonographers who had scanned at least five cases during the study compared with those who had scanned fewer than this number. In practice, therefore, we may need to explore other ways in which to deliver the training material and to develop a programme to maintain proficiency in training. We speculate that some of the training elements could be provided as courses, whereas other elements are bespoke to individual centres and would require clear demonstration of the sonographers’ abilities to scan and to be able to clearly distinguish cases from non-cases. The nature of the training programme itself could be adapted depending on the expertise of the sonographer, for example, shortening it for more expert centres, while still maintaining minimum standards. Targeted training would need to be more intense for novice sonographers (similar to the full training programme in this study), and less intense for more expert centres (requiring the sonographers to provide evidence that they are competent at performing the scans, by providing evidence that they have been regularly scanning cases, as well as being able to submit the scans of an active case as proof that they can adequately recognise an abnormal case. For centres with some experience, but that have not been performing scans regularly, we could ask their sonographers to undertake the online quiz, to make sure that they can recognise normal and abnormal scans, as well as to provide scans from an active case that they have recently seen. Implementation of training programmes would be facilitated by their certification through Royal Colleges or national bodies, such as the BSR. This would encourage accredited training and it would be feasible to apply for this training activity to be recognised as continuing professional development.

Ultrasound findings

Ultrasound abnormalities consistent with GCA were found in 162 patients, predominantly in the temporal arteries, but in 31% of patients, the axillary arteries were also involved and in a small number of patients (2.4%) they were exclusively involved. The predominant abnormalities on ultrasound that were considered to be consistent with GCA were the presence of a halo in 162 patients, stenosis in 45 patients and occlusion in 41 patients. The median halo size was 0.6 mm (range 0.4–0.9 mm) as measured in temporal arteries. In patients with abnormal ultrasound scans, the median number of segments of artery involved was 2.5 (range 1–6).

We measured differences in the size of the halo around the arteries depending on the duration of steroid therapy prior to scanning; we correlated halo size with ischaemic symptoms of GCA. We analysed data from 301 out of 415 patients with clinically defined definite or probable GCA at baseline using linear and logistic regression models to determine the relationship between halo size and days of steroid treatment and also with ischaemic symptoms of GCA (jaw and tongue claudication, amaurosis fugax and reduced, lost or double vision). Fifty per cent of patients were scanned on or before receiving 2 days of high-dose steroid treatment. Forty-three per cent (131) of patients had a halo in one or more temporal segments, 49% (146) of patients had bilateral temporal artery halos and 13% (38) of patients had axillary involvement. The linear regression model showed a consistently smaller halo size in temporal arteries during the 7 days of steroid treatment (p < 0.005). The likelihood of finding a halo diminished with time, until day 4 of steroid treatment (p < 0.005). Jaw claudication occurred more frequently in patients with a halo (p < 0.05). Temporal artery symptoms correlated with ipsilateral ultrasound findings (p < 0.05). The findings suggest that, in newly diagnosed GCA, ultrasound halo size decreases rapidly with steroid treatment and correlates with the presence of ischaemic symptoms, supporting its early use as a diagnostic and potentially prognostic marker.

Biopsy findings

Only 353 out of 381 biopsies performed actually contained a sample of temporal artery; the remainder either consisted of another tissue (such as vein or nerve) or no sample was obtained at all. The median length of artery biopsied was 10 mm (range 7–15 mm). In 161 patients the TABs were defined as abnormal and in 101 patients (27% overall) this was compatible with the diagnosis of GCA. In four patients the biopsy was compatible with another form of vasculitis. In a further 35 patients, arteriosclerosis was the dominant finding; 27 patients had a variety of other diagnoses (not GCA or vasculitis). Fragmentation in the internal elastic lamina was reported in 156 biopsies, and reduplication in 82 patients. Thirty-nine per cent of patients had intimal hyperplasia and 10% had arteriosclerosis in the intima. Of the 101 biopsies consistent with GCA (27% overall and 39% of the patients diagnosed with GCA), giant cells were present in 72 biopsies (representing 19% of the overall cohort, but 71% of biopsies of patients with GCA). In 99% of biopsy-positive cases, inflammatory infiltrates were present, which were transmural in 42% and adventitial in 18% as the predominant sites of inflammation. Furthermore, seven patients had evidence of recanalisation in at least one section of the biopsy.

Histological features in biopsy-positive patients with GCA were not confined to one form of inflammation. The most common finding was transmural inflammation. The relatively low number of positive biopsies may reflect the low index of suspicion of GCA in the cohort, technical difficulties in obtaining an adequate sample, skip lesions or the effects of glucocorticoid therapy in changing the biopsy result. These findings highlight the need for a better diagnostic strategy for patients with suspected temporal arteritis.

Change in diagnosis after expert review

Following expert review of the clinical cases, 21 patients had a change in diagnosis: in 13 of these, the diagnosis changed from GCA to not GCA; and in eight patients the diagnosis changed from not GCA to GCA. The diagnoses were predominantly made on the basis of symptoms and signs, blood abnormalities and, to a lesser extent, the biopsy report. The most common diagnosis in patients who did not have GCA was non-specific headache, myofascial pain, migraine, temporomandibular dysfunction and sinusitis. Five patients in total were diagnosed with another form of vasculitis including Takayasu’s arteritis, EGPA, GPA and other undefined forms of vasculitis.

The confidence in the clinical diagnosis of GCA at the baseline was > 75% in favour of probable or definite GCA; 86% of the patients, regardless of the confidence in diagnosis, were being treated with high doses of steroids at baseline. Most patients did not have any change in their clinical diagnosis by the observing clinicians from the 2-week assessment to the 6-month assessment. However, 19 patients had their diagnosis changed from not GCA to GCA at the 2-week assessment after unblinding of the ultrasound result. In 25 patients the diagnosis was changed at 6 months (6% of all patients); in 17 of these patients the diagnosis was changed from GCA to not GCA and in three patients a diagnosis of GCA was made. In the remaining five patients the diagnosis changed (but not from or to GCA).

Ultrasound compared with biopsy results

There was a significant association between the biopsy and ultrasound results (κ = 0.35), but more scans were positive than biopsies, so that ultrasound was more likely to be used to diagnose GCA than biopsy (162 positive ultrasound cases compared with 101 biopsy cases). Eighty-eight patients who had ultrasound evidence consistent with GCA had a negative biopsy and 27 patients with biopsy evidence of GCA had a negative ultrasound. There was a small number of patients (23) to whom steroids were given for longer than 7 days prior to the biopsy being performed. If we excluded those patients from the analysis, the agreement between ultrasound and biopsy increases slightly with a kappa of 0.37. The finding of a halo appeared to be the most useful aspect of the ultrasound result to support a diagnosis of GCA. Combining halo assessment with other aspects of ultrasound, namely stenosis or occlusion, did not increase the overall interpretation of the ultrasound scan as being positive or negative. Ninety-three patients had bilateral halo and a clinical diagnosis of GCA. Axillary involvement on ultrasound was present in 53 patients, nine of whom did not have temporal artery involvement; three were biopsy positive and, in total, seven were given a reference of diagnosis of GCA, suggesting that ultrasound of the axillary arteries can provide further support for the diagnosis of GCA in the absence of either temporal artery ultrasound or biopsy evidence to suggest GCA.

The effect of training and expert review of scan results on diagnosis

Expert review of the ultrasound images was part of the protocol and was undertaken for ongoing quality control purposes during the study. In 16% of scans the expert reviewers’ interpretation of the scans differed from the sonographer’s interpretation; 14 patients were interpreted as GCA by the reviewers but not GCA by the sonographer, and a further 47 were interpreted as not GCA by the reviewers but as GCA by the sonographer. The overall impact of using the reviewers’ interpretation in place of the sonographer’s interpretation was to increase the specificity of ultrasound from 81% to 87% but to reduce sensitivity from 54% to 44%. One potential explanation for the lower sensitivity using the reviewers’ interpretations is that the recorded ultrasound images and videos that the reviewers saw did not capture the abnormalities seen by the sonographer during a patient’s scan. A second potential explanation is that a sonographer’s interpretation may have been influenced by seeing the patient, for example, by observing a tender or thickened artery during the scan, something the reviewers would not be aware of. For the majority of discordant interpretations (those interpreted as not GCA by the reviewers) it is unclear if the difference indicates problems with the sonographer’s interpretation or, as the reduction in sensitivity might suggest, merely difficulties in capturing abnormalities in scan recordings. The discordant interpretations that the reviewers interpreted as GCA were fewer in number, but may indicate issues with a sonographer’s interpretation of the scans. For two sonographers in the study, retraining was required before they resumed scanning patients.

In 19 patients, the 2-week diagnosis based on the clinical findings and biopsy was not of GCA, but the unblinding of the ultrasound result suggested that there were findings compatible with GCA. Unblinding improved the sensitivity but reduced the specificity of the 2-week assessment compared with the reference diagnosis (sensitivity of 0.96 and specificity of 0.77). We observed a training effect among the sonographers. There was no significant change in the specificity of ultrasound for GCA by sonographers when comparing their early (first 10) scans with their subsequent scans, but the sensitivity improved from 45% to 62%, strongly suggesting an improvement in the ability to detect the presence of halo. Such an effect suggests that it is possible to achieve improved accuracy with ultrasound as sonographers gain experience in scanning. It also raises the question of whether or not more extensive training and/or supervision should be provided in addition to the training protocol developed for this study.

The effect of delay in testing and the effect of steroids

The accuracy of biopsy was likely to be greatest if performed within 3 days of starting steroids (sensitivity of 48% at this stage, compared with 33% for biopsies performed from ≥ 7 days after the commencement of steroid treatment). For ultrasound, the accuracy was highest for patients seen on no more than one dose of steroids, but was still maintained up to 7 days. The effect of delay between the scan and the biopsy being performed did not appear to influence the probable agreement between the tests.

Combination strategies and pre-test probability of having giant cell arteritis

We derived a risk of having GCA based on data obtained from an independent cohort of patients (based on the DCVAS study). We divided patients with GCA in the DCVAS cohort into three risk groups: those with an ESR > 60 mm/hour or a CRP level > 40 mg/l combined with the presence of jaw or tongue claudication were in the highest-risk group for having GCA; the lowest-risk patients had none of these features; medium-risk patients had only either an elevated ESR or CRP level or symptoms of jaw or tongue claudication. There was a significant relationship between the assignment of patients to one of these risk groups and the certainty of diagnosis of GCA reported at baseline in the TABUL study cohort, the reference diagnosis given and the biopsy findings. Although there was a trend for the ultrasound result, it was not as consistent. In other words, the patients in the lowest-risk group still had a 31% likelihood of a positive ultrasound compared with only 7% having a positive biopsy. Within the highest-risk group, the sensitivity of biopsy was 63% and for ultrasound it was 57%, with specificities of 100% and 80%, respectively. However, in the medium-risk group, biopsy had only 33% sensitivity, with a specificity of 100%, whereas ultrasound had 57% sensitivity and 91% specificity. Furthermore, in the low-risk group, biopsy had the least sensitivity of 17%, with a specificity of 100%, whereas ultrasound had a sensitivity of 44% and a specificity of 77%. One potential option is not to do either test (biopsy or ultrasound) if patients are in the high-risk group, because there is 93% prevalence of likelihood of diagnosis GCA according to the reference diagnosis in these patients. Looking at the potential combination of strategies, the risk group (high, medium or low) would affect the sensitivity and specificity of diagnosing GCA by performing either ultrasound and/or TAB (depending on the results of the ultrasound). In every instance, combination strategies produced better receiver operating characteristic curves than biopsy alone, supporting the role of ultrasound in supplementing or, in some patients, replacing biopsy as the diagnostic test for GCA.

Assessment using vasculitis activity and damage scores and quality of life

We used standardised generic scores of vasculitis activity and damage (the BVAS and the VDI score) in this cohort of patients, primarily to screen for the possibility that some patients had a more widespread form of a different type of systemic vasculitis (and this was actually true in five patients). As an exploratory outcome, we found that in 257 patients with GCA, disease activity scores were not significantly different from patients who did not have GCA. This shows that the disease activity score is not discriminatory between GCA and non-GCA (it was never designed for this purpose). However, the BVAS was more likely to be lower at 6 months than at 2 weeks. We have to bear in mind that it is likely that the scores were under-reporting disease activity at 2 weeks, because a significant number reported no abnormalities in the GCA group. There did not appear to be any discriminatory effect of measuring the VDI score at 2 weeks or at 6 months between patients and controls (the VDI was not designed to discriminate), but there was an increase in the number of patients and controls with at least one item of damage reported after 6 months compared with the 2-week assessment. Quality of life, as measured by the EQ-5D, did not differ significantly between patients and controls and neither did it change significantly after 6 months.

Adverse events

In total, 1229 AEs were reported during the study; every patient suffered at least one event, the majority of which were related to steroid therapy. When looking at events related to the study tests, 63 patients had an AE definitely related to biopsy, 10 had events possibly related to biopsy and two had events definitely related to the scan. There were 104 serious AEs among 55 participants, but none of them was related to the study test. The serious events included 16 deaths and 74 hospitalisations; all of these characteristics would be expected in a population of patients with suspected GCA and in whom high doses of steroids have been used. 42

Inter-rater agreement

We undertook inter-rater testing to evaluate agreement between pathologists and between sonographers in their assessment of images biopsy and ultrasound images. We selected the ultrasound scan recordings and histology slides from 33 patients in the study (a mixed group chosen at random, some of whom had a reference diagnosis of GCA and some of whom did not). We performed an inter-rater exercise separately for 14 pathologists and 12 sonographers. Agreement among 14 pathologists based on ICC was 0.62; among the sonographers it was 0.61. This would suggest that the level of certainty for interpretation of either test is variable, and it is perhaps more variable for pathologists than previously appreciated. The agreements between observers for both tests were similar.

Strengths and weaknesses of the study

We recruited a large cohort of patients mostly from primary care practices in the UK to a large number of centres, including academic and non-academic centres, to establish the generalisability of our findings. We developed an ultrasound training module as part of the study to ensure proficiency of testing. We did not offer any training in biopsy techniques or in biopsy processing and interpretation, as these are standard and, as such, should not be required by participating sites. We were able to compare the effects of ultrasound and biopsy independently on the diagnosis; however, the classification criteria for GCA include the results of biopsy, introducing an inherent bias in the diagnosis of GCA which would be likely to be given more or less weight depending on whether or not the biopsy was positive or negative. Despite this bias we were still able to demonstrate that ultrasound was an effective strategy for diagnosis in a significant proportion of patients. Nevertheless, neither of the tests is perfect, and we do not have a true gold standard to compare the effectiveness of each test. Unblinding of the ultrasound result at 2 weeks could have biased the results, but, in fact, a sensitivity analysis suggested that it had only a marginal effect on the outcome of the study.

Evolution in the presentation and suspicion of giant cell arteritis

Greater awareness of GCA may prompt primary care physicians to initiate treatment at a very early stage, which might affect the likelihood of obtaining a positive test result. Studies of pathological specimens obtained in other forms of vasculitis suggests that, whereas previously a biopsy showed clear evidence of abnormality, if awareness of the disease and clinical suspicion of the diagnosis lead to earlier investigation and treatment, we might actually be changing the natural history of the disease such that we do not see the characteristic features of the disease as previously described on biopsy. For example, nasal tissue biopsies have been reported to provide diagnostic appearances in 24–53% in patients with GPA. 117,118 It is possible that the level of suspicion for the diagnosis of GCA may have changed in line with the suspicion of the diagnosis of the other form of vasculitis. 119

Generalisability of current findings

One of the potential criticisms of the project is that we were introducing a specialist form of ultrasound imaging to NHS hospitals and comparing this with established practice. The specialist techniques of ultrasound imaging of temporal and axillary arteries might be perceived as being feasible to implement only in specialised centres where more time and resources might be available to perform these scans and that it might require more specialised equipment. However, we deliberately chose to recruit participants from non-academic centres, as well as academic centres, in order to test whether or not our technique was generalisable and could be applied, with suitable training in ultrasound performance and interpretation.

Although there is a difference between sonographers in terms of experience, as demonstrated by our evaluation of performance for centres recruiting fewer than 10 patients or more than 10 patients, this in itself is not an issue of whether the centre is an expert academic centre or a non-academic centre. This is to do with the volume of patients evaluated. Given the relative frequency of GCA in the general population and the likelihood that patients who have suspected GCA are referred to hospital for assessment, there is an opportunity for all centres to increase the number of patients evaluated to improve the sensitivity and specificity of ultrasound as a diagnostic test for GCA.

What are the implications of the study findings?

Our data suggest that TAB is less effective as a diagnostic test for GCA than was previously appreciated. Although it retains a high specificity, the sensitivity is only 39%. This could be because patients who are being evaluated with this test have low pre-test probability of the diagnosis. However, patients were selected for inclusion in the study on the basis that they have at least a possibility of GCA; in 53.5% of patients there was probable diagnosis of GCA and 21% of patients were reported as having a definite diagnosis of GCA at presentation as reported by the clinician. In comparison to other cohorts of patients undergoing TAB, the biopsy yield was actually higher than the 15.1% reported previously. 48 Difficulty in interpreting the biopsy result is undoubtedly made worse by not obtaining any arterial tissue at all, which occurred in 28 patients in the cohort. In a previous cohort of 567 consecutive biopsies, 2.5% had no arterial tissue,49 suggesting limitations to the technique. The biopsy length obtained was an average of 1 cm in the current study, which is the minimum recommended by the BSR guidelines. 5 However, other studies have suggested that 0.7 cm is an adequate length;52 in fact, even smaller biopsies might be adequate, with no evidence of a difference in positive biopsies for samples < 0.65 cm compared with those longer than 0.7 cm. 53 The biopsy length referred to in the current study is the measurement taken by the pathologists once a specimen arrives in the laboratory. It is known that shrinkage occurs once the specimen has been excised; we did not measure the length of the specimen obtained by surgeons at the time of sampling.

Pathologists are usually expected to provide an opinion on the diagnosis based on the interpretation of the biopsies. Our data suggest that the variation in agreement between observers can be considerable, especially for less clear-cut cases. If the specimen did not contain characteristic features of GCA, the interpretation of changes consistent with GCA, such as reduplication of the internal elastic lamina or intimal thickening or proliferation, could be that of early features of GCA, or of healing GCA, but, equally, these findings can occur in patients who have arteriosclerosis or age-related changes in their temporal arteries specimen and do not have any features of GCA at all. 47 We should give consideration to encouraging pathologists to report on the uncertainty of interpreting the findings rather than forcing them to make a clear-cut distinction between GCA and not GCA on the basis of the histology alone if there is insufficient information to make such a distinction with confidence.

We did not provide any training specifically to either the surgeons undertaking the biopsy or to the pathologists preparing and interpreting the sample results. We did not provide any reference standards to compare abnormal results or require any evidence of proficiency by the pathologists in the interpreting biopsies. The effect of training or use of reference standards may have improved our biopsy results.

We have developed an ultrasound training protocol that was effective in allowing 20 different sites with variable experience of use of vascular ultrasound (in some cases none at all) to undertake and interpret images of the temporal and axillary arteries to a standard acceptable by an expert panel in over 90% of patients undergoing a scan. Using this methodology, we have demonstrated that we can improve on sensitivity of biopsy by using ultrasound. However, there is a lower specificity and neither technique alone provides a high rate of confidence in the diagnosis of GCA, without the interpretation of the clinical features. We have shown that ultrasound is cost-effective compared with biopsy.

However, in a significant number of patients, both tests will be negative and yet the clinician will still diagnose GCA because the patient has clinical features that strongly suggest the diagnosis (such as jaw or tongue claudication or the development of ischaemic events compatible with the clinical syndrome of GCA). Until we have a more robust measure as a diagnostic test of GCA, these two tests (biopsy and ultrasound) could be used in combination to improve early diagnosis and treatment of GCA. It is feasible that other imaging techniques could have a higher yield than ultrasound (e.g. MRI). In 64 patients who underwent MRI (and a proportion who also underwent TAB), the sensitivity and specificity of MRI was reported as 80.6% and 97.0%, respectively, compared with histology, which had 77.8% sensitivity and 100% specificity. 120 A comparison study between ultrasound and magnetic resonance showed almost identical positive and negative predictive values. 121 Unfortunately, magnetic resonance changes resolve within a few days of starting glucocorticoid therapy and access to magnetic resonance is likely to be a limiting factor, whereas access to ultrasound is much more rapid. 13 The effects of steroids on image appearances for both magnetic resonance and ultrasound have been compared in 59 patients undergoing both tests, as well as in a proportion undergoing TAB. Whereas the sensitivity of ultrasound and magnetic resonance were 92% and 90%, respectively, up to 1 day following steroid therapy, this is reduced to 50% and 80% with > 4 days of steroid therapy. 122

It is conceivable that clinicians may feel some discomfort over having to rely on a clinical diagnosis of GCA supported by an imaging test such as ultrasound, but not confirmed by histological examination of the artery on biopsy. The concern would be that they are potentially overtreating a patient, who does not have a true diagnosis of GCA. However, the current study demonstrates that videos and images can be stored and reviewed later. Expert reviews of stored imaging tests were as reliable as expert reviews of stored biopsy specimens. There is increasing use of ultrasound as a diagnostic test in GCA in some centres for which confidence in the technical proficiency is high123 as more scans are performed. The methods in this study will enable naive centres to gain proficiency and improve sensitivity and specificity of the tests. We have shown that it is practical and achievable to become proficient at vascular ultrasound, but that it does require specific training. Trained sonographers could initially perform scans in suspected cases that also undergo biopsies, until adequate sensitivity and specificity for ultrasound are achieved (in the current study there was an improvement in specificity after 10 scans). A recent retrospective review of 43 patients diagnosed with GCA based on ultrasound findings allows further characterisation of patients into those who have isolated cranial vessel involvement and those who have extracranial features. 124 Patients with extracranial disease on axillary or subclavian artery ultrasound have a lower risk of permanent blindness, but a slightly higher risk of relapse and greater steroid requirement. 19,21,36,124

Problems with interpreting tests for giant cell arteritis

The inter-rater analysis for both tests (ultrasound and biopsy) revealed that the agreement between assessors is more variable than perhaps appreciated. The variability is significantly influenced by the degree of abnormality, as is to be expected with any test result. Borderline findings are likely to be subject to more dispute by different assessors than results showing either clear-cut abnormal appearances or clear-cut highly abnormal appearances. As demonstrated in the graphs of the inter-rater agreement (Figures 13–16), this problem appears to be present for both interpretations of ultrasound images as well as the evaluation of histological samples.

Biopsy has been regarded as a gold standard in diagnostic testing for many conditions including vasculitis, but when there is more uncertainty about the test results, our expectations of the pathologist or sonographer should perhaps be lowered. When we originally designed the study we were expecting a positive or negative outcome from each of the tests, so that we could compare the differences. What we have discovered is that in up to one-third of cases there is insufficient information available in the sample to determine confidently whether the diagnosis should be ruled in or ruled out. For some conditions, such as thyroid cancer, pathologists recognised that indeterminate histology was a significant problem in around 10% of cases discussed in a recent analysis of 14 studies comprising > 60,000 samples. 125 In these patients, a repeat sample was obtained and in 57% of patients the repeat biopsy was sufficient to make a definitive diagnosis. However, interestingly, in 42% of patients, a second opinion from an independent pathology review of the original sample resulted in a definitive diagnosis. Histological analysis of other conditions such as ulcerative colitis can be challenging in the presence of atypical histological features, which can lead to variations in the interpretation of diagnosis or severity of the condition. 126

Issues with the choice of reference diagnosis for giant cell arteritis

Evaluations of diagnostic tests rely on a ‘gold standard’ reference diagnosis in order to determine the accuracy of the test(s) being evaluated. A reference diagnosis should ideally be independent of the test(s) being evaluated and the timing of its measurement should coincide with the timing of the test(s). No reference diagnosis exists for GCA that meets these standards. ACR classification criteria exist but these are not diagnostic criteria and they use the results of biopsy. The design of the study therefore sought an approach to determining the reference diagnosis that balanced these different limitations; neither a clinical diagnosis nor the ACR classification criteria alone were considered suitable.

We used an algorithm that took the clinician’s diagnosis at 2 weeks as the starting point and this decision inevitably took account of the clinician’s knowledge of the result of the biopsy. This allowed clinicians to use their judgement based on their knowledge of the patient and many biopsy-negative patients were judged to have GCA. As expected, all biopsy-positive patients were judged to have GCA. The clinician’s diagnosis was confirmed as the reference diagnosis depending on consistency with the ACR classification criteria for GCA and the presence or absence of specific GCA-related symptoms or complications during follow-up. In around half of patients, a reference diagnosis was not confirmed this way. We used expert review of these patients to determine the reference diagnosis and for 23 (6%) patients the expert review confirmed a reference diagnosis that differed from the clinician’s diagnosis.

Our finding that interobserver agreement in interpreting biopsy images is moderate undermines the assumption that a positive biopsy should be regarded as confirming a GCA diagnosis. The reference diagnosis is not independent of the biopsy result because it is incorporated in the clinician’s judgement and is part of the ACR classification criteria used to confirm the reference diagnosis. One implication is that the 100% specificity (and also the sensitivity) of biopsy may be overestimated and that false-positive biopsy results have not been identified. A second implication is that the performance of biopsy compared with ultrasound may also be overestimated in favour of biopsy.

The lack of independence of the reference diagnosis is also an issue for interpreting testing strategies that combine test results with clinical judgement. Clinical judgement, that is, the clinician’s diagnosis at 2 weeks, is part of the test strategy, but is also the starting point for determining the reference diagnosis. Clinical judgement may also draw on patients’ symptoms at presentation that also feature in the ACR classification criteria and that, in turn, may confirm the reference diagnosis. This lack of independence may therefore overestimate the performance of strategies incorporating clinical judgement. The use of expert review and GCA-related symptoms and complications during longer-term follow-up (albeit only 6 months) for confirming reference diagnoses provides some protection against this lack of independence. However, the use of emerging symptoms or complications and expert review raises the issue of timing and the possibility that the reference diagnosis is capturing newly incident GCA that was not present at the times at which biopsy and ultrasound were done. Both this timing effect and the potential for expert review to incorrectly classify a patient’s true diagnosis may mean that the performance of tests and testing strategies is underestimated.

The economic modelling included additional analyses based around alternative reference standards constructed for the purpose of testing whether or not the findings could be sensitive to the reference standard criteria. Under the alternative reference standards evaluated, ultrasound in combination with clinical judgement remained a more cost-effective strategy than biopsy plus clinical judgement.

Could the results of the study be used to improve the existing service for diagnosis of suspected giant cell arteritis?

The low sensitivity of biopsy for diagnosis of GCA was one of the most surprising findings from the study. There are likely to have been several factors leading to this outcome. We could speculate on how the sensitivity of biopsy could be improved. The selection of patients could be based on a higher pre-test probability of having GCA, with careful clinical evaluation of each individual case. Patients would need to be seen promptly, either before or very shortly after commencing high-dose glucocorticoid therapy, which would require a fast-track service for these patients. The biopsy procedure should be performed by senior surgeons with expertise in the procedure. The samples should be processed and evaluated by experienced pathologists with the potential for central review of the histology. The interpretation of the biopsy should include the possibility that the result is non-diagnostic or non-specific, in order to provide more detailed results to enable the clinician to weigh up the likelihood of diagnosis in the presence of intermediate or indeterminate results. This could be achieved without recourse to ultrasound, in order to enhance the current service provision for patients with suspected GCA.

Fast-track service in giant cell arteritis

The potential window of opportunity to diagnose GCA is small once the patient has been commenced on high doses of glucocorticoid therapy. To optimise either test (ultrasound or biopsy) the important first step is to develop a rapid-access service for patients with suspected GCA. The current study provides evidence for the rapid decline in diagnostic performance with time and the economic analysis supports the introduction of ultrasound as a cost-effective means of achieving the diagnosis more effectively at much lower cost than the existing standard of care. Furthermore, fast-track services for GCA, which incorporate the use of ultrasound, have been shown to reduce the incidence of sight loss in this population, further justifying their role in the management of suspected GCA. 111,127

Summary of findings

Giant cell arteritis or temporal arteritis remains a diagnostic and therapeutic challenge. Unfortunately, the treatment options available are relatively limited and patients usually require a very high dose of steroids for prolonged periods of time, which results in significant toxicity in > 80% of patients. If, however, the diagnosis is missed and the patient is not treated with a high dose of glucocorticoid therapy, there is a significant risk of permanent visual loss or other ischaemic complications.

The current study was performed in an attempt to explore the value of ultrasound as a diagnostic tool in assisting the management of patients with suspected GCA. Ultrasound is a readily accessible investigation in most hospitals, whereas obtaining a TAB remains problematic in the NHS. Furthermore, the diagnostic value of TAB has been questioned owing to some studies reporting low sensitivity.

Ultrasound examination of temporal arteries is a relatively specialist procedure; we wanted to explore the generalisability of diagnostic testing in GCA within a NHS setting. We therefore had to design a training programme that was effective enough and applicable enough to be generalised to clinicians and sonographers working in a variety of centres throughout the UK. We deliberately chose a mixture of district general hospitals and teaching hospitals to explore this generalisability. We trained sonographers in the technique of ultrasound examination of temporal and axillary arteries by developing a training programme based on established expertise.

The effect of the training programme was tested thoroughly by an expert review panel established specifically to view all images obtained from the main study for quality control. This ensured that the images acquired and interpreted by site sonographers were of a sufficiently high standard to be comparable to those that would have been obtained by experts.

For the main study, we needed to test the value of ultrasound as a diagnostic tool without interfering with the normal diagnostic process. We therefore designed the study so that patients underwent the normal diagnostic process if they were suspected of having GCA. This meant that they underwent a clinical assessment followed by a TAB in every case. We undertook a blinded ultrasound test before biopsy was performed, but the results of the ultrasound tests were not given to the clinician managing the patient. However, the results of the biopsy test were given to the clinician as would occur in normal practice. The results of the biopsy test, together with the clinical condition of the patient when re-evaluated 2 weeks after initial assessment, were used by the clinician to make a diagnosis. If the clinician had made a diagnosis that was not GCA and was planning to bring the patient off high doses of glucocorticoid therapy or was not planning to start high doses of steroids, we built in a safety mechanism whereby the clinicians were asked to contact the TABUL office to be given the results of the ultrasound scans just in case there was a disparity between the scan result and the clinical decision. It was then up to the clinician managing the patients to decide whether or not to alter their diagnosis and management plan, but this decision was not used as the basis for the primary outcome, although it was reported.

We asked for a 6-month follow-up visit to determine whether or not any new features consistent with the diagnosis of GCA had emerged or, indeed, whether or not any features consistent with other diagnoses had emerged and whether or not the clinician had an opportunity to change the diagnosis in 6 months. We felt that this study design was realistic and represented usual practice, but with the addition of the ultrasound scan.

Our results showed that the sensitivity of biopsy was only 39%, which was lower than in previously published studies. The sensitivity of ultrasound was 54%. The specificity of biopsy was always going to be high and in this study was 100% compared with 81% for ultrasound. As both of the tests had been performed in all patients, we were able to hypothesise on a potential sequence of tests that could have been performed to try to improve sensitivity and specificity and also to look at the cost implications of these strategies. We therefore analysed the data according to a potential strategy of performing both tests in different combinations. However, the tests were never performed in isolation from the clinical evaluation of patients; we therefore introduced two methods to define the likelihood of the patient having or not having GCA, based on clinical features and blood test results, before looking at the results of either test. One method was simply to ask the clinicians to state their opinion of the likelihood of GCA (definite, probable or possible); the other method was to use an external data set obtained from DCVAS to try to define patients as being at high, medium or low risk of having GCA. We used the presence of jaw or tongue claudication and elevated inflammatory response (ESR > 60 mm/hour or CRP level > 40 mg/l) as parameters that would define a patient as being at a high likelihood of having GCA. Patients defined as having a low likelihood of GCA did not have any of these parameters. Patients were defined as being tat intermediate risk of having GCA if they had either claudication (of jaw or tongue) or an elevated acute-phase response (ESR or CRP level) but not both of these. Using this strategy would make clinical sense. The clinicians would normally assess the patient and decide whether or not it was worthwhile to investigate a patient further for the possibility of GCA, and therefore defining who should or should not have a test such as TAB or ultrasound.

Using this strategy-based approach we demonstrated that without a clinical evaluation, a combined approach of scanning all patients and performing a biopsy only on those for whom the scan was negative would achieve a sensitivity of 65% and specificity of 81%. If we took a risk strategy approach, by only investigating those patients with the high likelihood of GCA based on clinical presentation, the sensitivity and specificity increases to 77% and 91%, respectively. However, for patients at moderate or low risk of having GCA based on the clinical presentation, the sensitivity and specificity were lower and this would inevitably result in a cohort of patients for whom there was still a clinical suspicion of diagnosis of GCA, but for whom both ultrasound and biopsy were negative. We have demonstrated that in the TABUL study there is a significant cohort of such patients (about one-quarter of all patients defined as having GCA).

In terms of cost-effectiveness, biopsy is more expensive than ultrasound (an almost ninefold difference) and this higher cost was the key factor in the greater cost-effectiveness of strategies using ultrasound. The similarity in the diagnostic performances of the two tests (when combined with clinical judgement), the estimated impact of GCA-related complications from false-negative results, and the estimated impact of steroid toxicity from false-positive results was insufficient to alter the results.

In a parallel study, we used data from the TABUL project to measure the reliability of the interpretation of ultrasound or biopsy findings. In order to do this we produced a series of 30 patients from the TABUL cohort (containing a mixture of patients with positive and negative ultrasound and biopsy results). We prepared the ultrasound scans and the histological slides of those patients and showed a brief clinical vignette together with either the scans or histology slides to a group of sonographers and pathologists, respectively, to determine inter-rater reliability of these two tests. We found that there were similar levels of agreement, with kappa values of 0.61 for sonographers and 0.62 for pathologists. The areas of disagreement among the pathologists occurred when the histological results were less clear cut, particularly when no giant cells were found. These findings suggest that the pathologist’s interpretation of biopsy material should be qualified according to the level of severity of the findings. If there are very obvious features of GCA (such as transmural inflammation or giant cells), this should be stated by pathologists, but if there are much less obvious features that might be consistent with GCA, but that equally could be consistent with normal ageing, then it is important that the pathologists are able to express this diversity of possible diagnosis rather than having to state that the biopsy is consistent with GCA but not declare that the biopsy is also consistent with normal ageing findings. The clinicians managing patients may feel less comfortable with the fact that the pathologists are not giving a clear-cut interpretation of the biopsy, but this should improve the management of patients if we avoid making conclusions based on insufficient evidence. Although no clear pattern emerged from an evaluation of cases disputed by sonographers, similar remarks would apply in interpreting ultrasound findings. Until we have a more effective diagnostic tool, the clinical evaluation of the patient remains paramount in the decision-making process.

We have challenged the place of TAB as the gold standard for the diagnosis of GCA. We have demonstrated that in 381 patients with newly suspected GCA, the application of clinical risk stratification (based on the presence of ischaemic symptoms of tongue or jaw claudication and/or an elevated acute phase response), combined with either ultrasound of temporal and axillary arteries or biopsy, will result in a high sensitivity and specificity in the diagnosis of GCA. In order to achieve this, we have created a training programme to ensure the proficiency of sonographers in performing the scans. We compared the results obtained from these scans with the traditional factors used in making a diagnosis of GCA, namely the application of clinical judgement (strongly influenced by ACR classification criteria for GCA). Despite the inherent bias of using a reference diagnosis that incorporates the results of the biopsy, ultrasound examination was more sensitive but less specific than biopsy as a diagnostic test. We tested the reliability of both tests, by asking a number of pathologists and sonographers to respectively review biopsy and scan findings from an anonymous sample of patients drawn from the cohort. We showed that the reliability of both techniques was similar (ICC of 0.61–0.62), revealing that both tests have some fallibility. Further analyses of the diagnostic strategies to combine clinical risk stratification with one or both tests in appropriate cases can be used effectively to significantly improve the diagnostic accuracy of patients with newly suspected GCA. The economic evaluation of the test strategies used in this cohort of patients has shown that an ultrasound-based approach is more cost-effective than a biopsy-based strategy if used in conjunction with clinical risk assessment.

Conclusions