OBJECTIVES Trials suggesting that carotid endarterectomy in individual subjects with asymptomatic carotid stenosis reduces stroke risk have led to calls for screening. This study aimed to determine which groups might be harmed and which might benefit from a screening programme, and also to identify which individual subjects identified as positive for severe asymptomatic stenosis by carotid ultrasound are appropriate to put forward for further tests or procedures.
METHODS A probability model was used to estimate the outcomes of three screening strategies: carotid ultrasound followed by catheter angiography, or by magnetic resonance angiography (MRA), or ultrasound alone, followed by carotid endarterectomy if severe stenosis is detected. Information from the current literature was used to estimate sensitivity and specificity of ultrasound and MRA, risks of angiography and endarterectomy, and risk reduction after surgery for severe stenosis. For each strategy over a range of possible prevalences of severe asymptomatic stenosis, overall benefit to harm ratio was calculated, and number of strokes or deaths prevented or caused per 10 000 subjects screened.
RESULTS At the prevalence of carotid stenosis found in the general population (<1%) screening will cause more strokes than it prevents, even using the most optimistic published figures. Only at prevalences of over 20% are significant benefits seen, and then only in centres with high test sensitivity and specificity and very low angiographic and surgical risk. Groups with such a high prevalence have not yet been reliably identified. Screening individual subjects from high prevalence groups would have limited public health impact, with at best about 100 strokes prevented for every 10 000 screened at 20% prevalence.
CONCLUSIONS Investigating asymptomatic individual subjects for carotid stenosis may be harmful except in high prevalence groups. There is insufficient information about which these groups are, and at present screening cannot be recommended. Acting on a positive carotid ultrasound test in individual subjects without well defined risk factors for severe stenosis is unjustified and potentially dangerous.
- carotid endarterectomy
Statistics from Altmetric.com
Large randomised controlled trials have shown that carotid endarterectomy is beneficial in patients with a recently symptomatic internal carotid artery stenosis of greater than 70%.1 2Some recent trials have examined the role of this operation in those with asymptomatic carotid stenosis,3-5 and one (the Asymptomatic Carotid Atherosclerosis Study: ACAS) showed a significant reduction in the risk of ipsilateral stroke and perioperative stroke or death in those with greater than 60% stenosis.6 A European trial is still under way.7
Unfortunately, we are still left with unanswered questions about the management of asymptomatic carotid stenosis. The first revolves around screening. There have already been calls for screening for asymptomatic carotid stenosis.7-9 Screening a group of asymptomatic subjects will always pick up some false positives (test positive but disease free). How many depends far more on the prevalence of the condition in the population screened than on the sensitivity and specificity of the screening test. Patients screening positive for severe carotid stenosis with carotid ultrasound, usually colour coded duplex sonography, may then be referred for carotid angiography or endarterectomy. Screening a population with a low prevalence of carotid stenosis might lead to more false than true positive cases being exposed to angiographic or surgical risk, for no long term benefit.
The second (increasingly common) practical question is how to manage an individual patient who comes to a clinic with a positive screening test for severe asymptomatic carotid stenosis. The subsequent management of such a patient should depend on the chance of that patient benefiting from further investigation or treatment.
Using current data it is possible to estimate which asymptomatic subjects are more likely to be harmed than helped by being referred for further tests or surgery as a result of a positive screening test. This balance depends on: (1) the predictive power of the test; (2) the risks of further procedures (including surgery); and (3) the potential benefit to the subject if appropriate surgery is carried out. For asymptomatic carotid stenosis and its treatment all these data are now available. This paper sets out a simple and robust model to identify those who might benefit, and those who are likely to be put at risk inappropriately, if action is taken on a positive ultrasound of an asymptomatic carotid artery.
The model follows the clinical course of a patient presenting for screening. The stages take the form of a simple probability calculation. Two different examples are worked in this paper. One, used in the description below, takes the most optimistic reading of current published data (the highest sensitivities and specificities and lowest operative risk). The other example uses assumptions which are probably closer to normal practice.
STAGE 1: THE SUBJECT PRESENTS FOR ULTRASOUND SCREENING OF ASYMPTOMATIC CAROTID ARTERIES
The probability of a true or false positive test result depends on the pretest probability of having severe stenosis and the sensitivity and specificity of the screening ultrasound test. The definition of severe stenosis for this paper is that used in ACAS: a greater than 60% reduction in luminal diameter of the internal carotid artery, excluding occlusions.
Carotid ultrasound is very user dependent, and a range of sensitivities and specificities have been reported in different studies. For the identification of severe stenoses, sensitivities ranging from 72% to 97%, and specificities from 83% to 95% are quoted. For the optimistic model we have assumed a sensitivity and specificity of 93% each for identifying greater than 60% stenosis. These are at the high end of values from a recent meta-analysis for ultrasound identification of severe stenoses.10
The pretest probability of a subject having any condition is the same as the prevalence in the population from which they are drawn (for example, if the prevalence of severe asymptomatic stenosis is 10% the pretest probability is 0.1). The figure shows the influence of prevalence on the probability of a positive test being a true positive result, taking as an example a sensitivity and specificity each of 93%. It can be seen from this that at a prevalence of 1% the ratio of true positives to false positives is 93:693 (about one to seven). At 20% prevalence the ratio of true to false positives is 1860:560 (about three to one). Therefore, a subject screened positive from a low prevalence population is far more likely to be a false positive than genuinely to have disease, even with high test sensitivity and specificity, whereas from an intermediate prevalence population most test positives, although by no means all, are true positives. Even at 100% sensitivity and 99% specificity, most positives will be false positives if prevalence is under 1%; prevalence has a much greater impact on predictive value than diagnostic excellence.
STAGE 2: SUBJECTS WITH A POSITIVE TEST (THAT IS, ALL TRUE AND FALSE POSITIVES FROM STAGE 1) ARE REFERRED ON
Three possible onward routes for the patient are currently advocated by different centres. A slightly different version of the model is used for each.
Approach A: refer for carotid angiography
If this confirms stenosis proceed to operation, if not no further action is taken. This approach was used in most trials.3 6 The largest published series in asymptomatic patients quotes a risk of stroke or death of 1.2% for angiography.6 The optimistic model therefore applies a probability of 0.012 of having a bad outcome to all of these patients. Carotid angiography is currently the gold standard test of severe stenosis, and we have assigned it a theoretical 100% sensitivity and specificity. This will not be strictly true but, as trials use positive angiography as their criterion for surgery the test error rate of angiography is built into the published trial results.
Approach B: refer for MRA
If this is also positive proceed to surgery, if negative take no further action. This is advocated by some as a way of avoiding the risk of carotid angiography.11 Published series suggest a sensitivity of 85 to 100% and a specificity of 90 to 93% for severe stenosis, and meta-analysis shows sensitivity and specificity similar to ultrasound.10 We have taken a generous 95% for both sensitivity and specificity of MRA to anticipate technical advances.
Approach C: refer direct for surgery, and therefore straight to stage 3
This avoids the risk of angiography.12
STAGE 3: SURGERY
We use the perioperative risk of stroke or death from ACAS. For patients in the surgical group who actually received surgery this was 1.5%. This was recalculated directly from the patient numbers quoted (11 perioperative events in 724 operated patients), excluding angiographic stroke or death.6
STAGE 4: POSTOPERATIVE PERIOD
It is in the postoperative period that the benefits of appropriate surgery are seen. The model uses the estimated five year actuarial absolute risk reduction from ACAS,6 excluding perioperative and angiographic complications as they have been accounted for in stages 2 and 3. This produces a reduction from 0.11 probability of ipsilateral stroke in the non-surgical control group to about 0.03 in the surgical group, a 0.08 absolute risk reduction. We have considered ipsilateral stroke because the risk reduction in ACAS for ipsilateral stroke and any perioperative stroke or death was the main outcome reported by the ACAS investigators and reached conventional levels of significance. However, the absolute risk reductions in any stroke or death, or in any major stroke or death, are almost identical—that is, about 0.08 (or 8%) after excluding angiographic and surgical events—and so the results of the model would not be materially affected if these were considered instead.
Three outcomes are used:
The ratio of benefit to harm—strokes prevented to perioperative and angiographic strokes or deaths caused (in stages 2 and 3). High ratios are favourable; ratios of less than one mean that any subject is more likely to be harmed than helped.
The number of strokes prevented per 10 000 people screened—that is, the sum of strokes prevented less strokes caused. This is a measure of public health impact. A negative figure indicates net harm.
The number of strokes or deaths caused in patients with false positive screening tests who subsequently undergo inappropriate angiography or endarterectomy. These figures are incorporated in the measures above, but they are important in making ethical decisions if the principle of “first do no harm” is applied.
Table 1 shows the results of the optimistic model using 93% sensitivity and specificity for ultrasound, a 1.2% risk from angiography, 1.5% perioperative risk, and a reduction in ipsilateral stroke from 11% to 3% over five years. A generous interpretation of ACAS data has been used to predict actuarial reduction in risk of stroke. In practice, even published figures are generally less good both for test sensitivity and specificity and for operative risks with perioperative complication rates of up to 11% quoted.10-14 Published series tend to come from centres of excellence, so the results are likely to reflect a best case scenario. Table 2 shows a more realistic set of outcomes based on 85% sensitivity and specificity for both ultrasound and MRA, an angiographic risk of 2%, and a perioperative risk of 3% (based on a recent systematic review).13
The advantages of screening are seen to depend critically on the prevalence of severe carotid stenosis in the screened population, and so at an individual level on the pretest probability of stenosis. Exact outcomes will depend on the particular conditions in a centre, and the model is easily adapted to reflect local variations in sensitivity, specificity, surgical or angiographic risk, and prevalence.
Tables 1 and 2 show the relative merits of the various screening strategies. At low prevalences ultrasound-MRA-surgery is clearly safer than the alternatives, both in terms of better ratios of benefit to harm, and for the low number of strokes or deaths in false positive subjects. At higher prevalences of severe stenosis ultrasound-angiography-surgery becomes the least satisfactory route, with little to choose between ultrasound-surgery and ultrasound-MRA-surgery in terms of net benefit.
Who should be investigated?
To make any meaningful use of outcomes derived from the model, we need to know the range of prevalences of severe asymptomatic carotid stenosis. Information is available from some reasonably large community based studies,15-21 but trying to correlate these with the clinical trials of carotid endarterectomy has its problems. Most community prevalence studies defined severe stenosis as greater than 50%, whereas the ACAS trial used a 60% stenosis cut off point. In the largest community based study, the prevalence of 50%-99% stenosis was five times higher than the prevalence of 75%-99% stenosis in elderly men.17 This means that prevalence studies based on 50%-99% stenosis will overestimate the prevalence of carotid stenosis that may be appropriate for operation on ACAS criteria, maybe by several times. Some studies also included cases of (inoperable) carotid occlusion in the highest grade stenosis group. In addition, all studies were ultrasound based, and in low prevalence populations will have picked up more false positives than true ones, and therefore inevitably have overestimated the prevalence. Studies will therefore suggest prevalences which are higher, in some cases significantly higher, than the true prevalence of more than 60% stenosis that ACAS suggests is appropriate for surgery.
Despite this tendency to overestimate, studies suggest that in the under 60 year old general population the prevalence is at most 1%. At this prevalence, even with the best case scenario, more harm than good will be done by proceeding from a positive ultrasound test to either angiography or surgery, and even if MRA is used the benefits are almost imperceptible (table 1). Using the more realistic figures (table 2), all possible screening routes lead to net harm. Screening this population is clearly inappropriate. Furthermore, a patient from the same general population with no additional risk factors presenting to a clinic with a positive test should not be referred on for further tests or surgery.
In the elderly (over 65 years) population, community based studies suggest a prevalence of severe carotid stenosis lying in the range 2%-7%. The impact of screening this population is small, possibly even harmful with the assumptions used in table 2. Furthermore, the risks of surgery in elderly people are likely to be higher than in the general population.22 It is therefore not appropriate to recommend screening on the basis of current evidence. Even in a centre of excellence, the potential benefits to the individual subject are so marginal that ignoring a positive ultrasound from a member of this population seems the most reasonable clinical policy.
Much higher prevalence groups have been identified in some studies. Elderly patients with hypertension, a smoking habit, peripheral vascular disease, a carotid bruit, or atrial fibrillation are those for whom there is most published evidence.21 23-26 The quoted prevalences for these groups are 14% to 28%. Most of these studies were both very small and hospital based, limiting the accuracy and generalisability of the results. In addition, they will have overestimated prevalence for the reasons outlined above. In the absence of reliable community based data it is not clear whether any group can be identified as having a high enough prevalence to make screening appropriate. Even at 20% prevalence, the number of strokes prevented per 10 000 people screened is only 112 at best (table 1) and this would involve performing around 1800 operations, equivalent to 16 operations for every stroke prevented. It seems unlikely that this would be defensible as a public health measure.27
If an asymptomatic subject from a high prevalence population presents to a clinic with a positive screening test there are potential benefits, albeit modest, from proceeding further with investigation or treatment. The clinical decision is then whether to refer for angiography, MRA, or operation directly. At high prevalences referring for angiography has the lowest net benefit. There is surprisingly little to choose between proceeding from ultrasound direct to operation or via MRA in overall outcome. As MRA leads to fewer strokes in false positive subjects (those who do not stand to gain from the operation) it is ethically more easily defended.
Despite the results of ACAS, and whatever position is taken on the public health impact of carotid endarterectomy for asymptomatic stenosis, screening asymptomatic subjects for severe carotid stenosis cannot be recommended at present, except perhaps in populations known to have a prevalence of at least 20%. The prevalence figures currently available for such high risk populations are too unreliable to form the basis of a screening programme. Even with the most optimistic interpretation of current published data, those presenting with positive screening tests who do not come from high prevalence groups are unlikely to benefit from further investigation or treatment. In most cases they are more likely to be harmed than helped by further action. Definitive answers on who should be screened or investigated are only likely to come from more reliable prevalence data or randomised controlled trials focusing specifically on high prevalence groups. In the meantime we should suppress our curiosity about the state of the carotid arteries in either populations or individual subjects with no cerebrovascular symptoms.
We are most grateful to Carl Counsell for his helpful comments on an earlier version of the manuscript. CJMW was supported by the Medical Research Council. CLMS has a Wellcome Trust training fellowship in clinical epidemiology.
FOR ALL APPROACHES. STAGE 1: ULTRASOUND
Probability of true positive=sensitivity ultrasound×prevalence=T
Probability of false positive = (1−specificity ultrasound)×(1−prevalence)=F
APPROACH 1: ULTRASOUND-CAROTID ANGIOGRAM-SURGERY
Stage 2-probability of angiographic stroke or death ifAR is the angiographic risk
=(T + F)×AR =AS
Stage 3-probability of surgical stroke or death if SR is the surgical risk
Stage 4-probability screened patient helped by surgery if RRis the risk reduction=T×(1-AR)× (1-SR)×RR = α1
Total probability of stroke or death=SS+AS=β1
Total probability of stroke or death if false positive=AR×F=γ 1
APPROACH 2: ULTRASOUND-MRA-SURGERY
Stage 2-Probablility of true positive=sensitivity MRA×T=Tt
Probability of false positive=(1−specificity MRA)×F=Ft
Stage 3 - probability of surgical stroke or death if SR is the surgical risk
Stage 4-probability screened patient helped by surgery if RRis the risk reduction=Tt×(1-SR)×RR=α2
Total probability of stroke or death if false positive=SR×Ft=γ 2
APPROACH 3: ULTRASOUND-SURGERY
Stage 2-not relevant
Stage 3-probability of surgical stroke or death if SR is the surgical risk
Stage 4-probability screened patient helped by surgery if RRis the risk reduction=T×(1-SR)×RR=α3
Total probability of stroke or death if false positive=SR×F=γ3
Ratio of strokes prevented to those caused by screening=α:β
Net number of strokes prevented per 10 000 screened=(α−β)×10 000.
Actual number of strokes or perioperative/angiographic deaths caused per
10 000 screened in false positives=γ×10 000
(The model uses probabilities throughout. Multiply by 100 to get %.)