Objective A substantial part of non-traumatic intracerebral haemorrhages (ICH) arises from a macrovascular cause, but there is little guidance on selection of patients for additional diagnostic work-up. We aimed to develop and externally validate a model for predicting the probability of a macrovascular cause in patients with non-traumatic ICH.
Methods The DIagnostic AngioGRAphy to find vascular Malformations (DIAGRAM) study (n=298; 69 macrovascular cause; 23%) is a prospective, multicentre study assessing yield and accuracy of CT angiography (CTA), MRI/ magnetic resonance angiography (MRA) and intra-arterial catheter angiography in diagnosing macrovascular causes in patients with non-traumatic ICH. We considered prespecified patient and ICH characteristics in multivariable logistic regression analyses as predictors for a macrovascular cause. We combined independent predictors in a model, which we validated in an external cohort of 173 patients with ICH (78 macrovascular cause, 45%).
Results Independent predictors were younger age, lobar or posterior fossa (vs deep) location of ICH, and absence of small vessel disease (SVD). A model that combined these predictors showed good performance in the development data (c-statistic 0.83; 95% CI 0.78 to 0.88) and moderate performance in external validation (c-statistic 0.66; 95% CI 0.58 to 0.74). When CTA results were added, the c-statistic was excellent (0.91; 95% CI 0.88 to 0.94) and good after external validation (0.88; 95% CI 0.83 to 0.94). Predicted probabilities varied from 1% in patients aged 51–70 years with deep ICH and SVD, to more than 50% in patients aged 18–50 years with lobar or posterior fossa ICH without SVD.
Conclusion The DIAGRAM scores help to predict the probability of a macrovascular cause in patients with non-traumatic ICH based on age, ICH location, SVD and CTA.
- intracerebral haemorrhage
- CT angiography
- digital subtraction angiography
- arteriovenous malformation
Statistics from Altmetric.com
Intracerebral haemorrhage (ICH) accounts for 15%–20% of all strokes and is the most devastating stroke subtype.1 2 Around 15%–25% of ICHs are caused by an underlying macrovascular cause, such as an arteriovenous malformation (AVM), aneurysm, dural arteriovenous fistula (dAVF), cavernoma and cerebral venous sinus thrombosis.3–5 Among young adults, macrovascular causes are the leading cause of ICH.6
Early diagnosis of underlying macrovascular lesions can influence clinical management and prognosis, as timely intervention might prevent recurrent haemorrhage.7 8 Intra-arterial digital subtraction angiography (IADSA) is the gold standard for detection of macrovascular abnormalities, but is an invasive procedure associated with some risk of complications.9 MRI/ magnetic resonance angiography (MRA)is less invasive, but has lower diagnostic accuracy for macrovascular causes than IADSA.
Currently, there is little guidance on which patients to select for (invasive) angiographic imaging and clinical practice thus varies widely.10 Several factors have been associated with a higher likelihood of finding a macrovascular cause, including younger age, lobar location and absence of hypertension.11 Early risk stratification of patients with ICH might help physicians to make swift, well-informed decisions about who to select for further angiographic imaging.
We aimed to develop and externally validate a prediction model to estimate the probability of finding a macrovascular cause in patients with non-traumatic ICH, based on patient characteristics, haemorrhage characteristics and, optionally, CT angiography (CTA).
We used data from the DIagnostic AngioGRAphy to find vascular Malformations (DIAGRAM) study, a prospective, multicentre cohort study that assessed yield and diagnostic accuracy of angiographic imaging (CTA, MRA, IADSA) in patients with non-traumatic ICH.12 Between 2008 and 2014, 298 patients aged 18–70 years were included in 22 participating centres across the Netherlands. Patients over 45 years of age with hypertension and ICH in the basal ganglia, thalamus or posterior fossa were excluded, because of the low probability of finding an underlying macrovascular cause.13 Also, patients with a known macrovascular abnormality, brain tumour, or patients who used oral anticoagulants and had an international normalised ratio of >2.5 at the time of ICH were excluded. All patients underwent CTA within 7 days of the ICH, followed by MRI/MRA within 4–8 weeks if the CTA was negative. Patients underwent subsequent IADSA if the results of CTA or MRI/MRA were inconclusive or negative. CTA or MRI/MRA was considered inconclusive if a macrovascular cause was suspected but a definite diagnosis could not yet be established. Scans were read both locally and centrally. In case of a new diagnosis, local centres were informed. One additional arteriovenous fistula was detected at central reading.
Two hundred and ninety-one patients had a CTA of sufficient quality for assessment (98%). MRI/MRA was performed in 255 patients (86%), of whom 214 patients with a negative or inconclusive CTA, and IADSA in 154 patients (52%) of whom 106 patients with a negative or inconclusive CTA (online supplementary figure 1). Quality of IADSA was insufficient for assessment in three patients. One hundred and twenty-six patients had a negative or inconclusive CTA, but did not undergo subsequent IADSA. The main reason for not performing IADSA in patients with a negative CTA was an alternative diagnosis on MRI/MRA, or reluctance of either patients or their treating physicians. Four patients with a negative CTA died before MRI/MRA could be performed. The outcome was presence of a macrovascular cause (AVM, aneurysm, dAVF, cavernoma, cerebral venous sinus thrombosis and developmental venous anomaly) as cause of the haemorrhage, and was based on best available evidence from all findings (CTA, MRA, IADSA) during 1-year follow-up. All participants gave written informed consent.
Supplementary file 1
Candidate predictors were preselected based on the literature and included age, hypertension (defined as a history of hypertension, use of antihypertensive drugs before ICH or evidence of left ventricular hypertrophy on admission ECG), smoking, high alcohol intake (defined as four or more units per day), location of ICH (lobar, deep or posterior fossa), presence of small vessel disease (SVD) on non-contrast CT (NCCT) (defined as presence of white matter lesions, or a lacunar infarct in basal ganglia, thalamus or posterior fossa, irrespective of whether it had been symptomatic or was an asymptomatic finding (see online supplementary methods for a detailed description of SVD assessment and online supplementary figure 2)), and CTA. We developed two models: one model based on patient characteristics and NCCT (DIAGRAM score) and another model based on patient characteristics, NCCT and results from CTA imaging for use in higher resource settings (DIAGRAM+ score), which may help to estimate the probability of a macrovascular cause given that CTA is negative. For the current analysis, inconclusive CTAs were joined with positive results, because a CTA suggesting a macrovascular cause, yet inconclusive, will always trigger further diagnostic tests. Given the one in ten rule with one predictive variable for every ten outcome events, we could study a maximum of seven predictors.14 15
Missing values for alcohol consumption (1%), smoking (1%) and CTA (2%) in the development cohort were imputed with single imputation. We used restricted cubic spline functions and graphs to assess whether age could be analysed as linear term or needed transformation. We performed multivariable logistic regression analysis to study the association between candidate predictors and the presence of a macrovascular cause. The full model containing all candidate predictors was simplified by performing backward selection based on Akaike’s information criterion. We internally validated the model by performing bootstrapping. A shrinkage factor was estimated from the bootstrap procedure and regression coefficients were multiplied by this shrinkage factor to correct for overfitting. Model performance was assessed with discrimination and calibration. Discrimination refers to the ability of the model to distinguish between someone with and without a macrovascular cause and was assessed with the c-statistic. Calibration assesses the correspondence between observed and predicted risk and was studied with a calibration plot. As a sensitivity analysis, we examined the performance of the models in a subset of patients (n=171), excluding those who did not undergo IADSA following a negative or inconclusive CTA. We generated prediction charts with predicted probabilities of finding a macrovascular abnormality for each combination of risk factors. Additionally, we created two prediction scores based on regression coefficients of the final multivariable regression models. For the prediction charts and scores, age was dichotomised at a value close to the mean.
For external validation, we used a cohort of 173 patients with non-traumatic ICH.16 Consecutive patients who underwent IADSA at the National Hospital for Neurology and Neurosurgery in London between 2010 and 2014 were retrospectively reviewed. Patients with non-traumatic ICH with available NCCT and CTA were included. NCCT and CTA were routinely performed in all patients with acute ICH presenting to the hyperacute stroke unit, unless there were contraindications. The necessity of IADSA performance was judged in a weekly neuroradiological meeting, and was based on age, ICH location and medical history. MRI was performed according to clinical care, but was not systematically undertaken in all patients. The reference standard in the validation cohort was IADSA. All CTAs were reviewed blinded to IADSA result.
We applied the original regression equation to the validation data and calculated the predicted probability of finding a macrovascular cause for each patient. We assessed model performance with the c-statistic and calibration plots. As calibration is known to be strongly influenced by the incidence of the outcome in the validation population, we recalibrated the prediction models. Recalibration was performed by logistic regression analysis in the validation data with the linear predictor (the combination of regression coefficients with covariate values) as offset in the model. The resulting intercept was combined with the original regression coefficients to obtain predicted probabilities for the validation population. We present calibration of the models after recalibration, as in practice it is also advised to recalibrate a model before putting it to use. Calibration results before recalibration are provided in the online supplementary materials. Analyses were performed with R Version 3.3.2. Results are reported in accordance with the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis statement.17
Table 1 shows the baseline characteristics of the development and validation cohorts.
Among298 patients included in the development cohort, 69 (23%) had an underlying macrovascular cause (for listing of all causes, see online supplementary table 1). In the validation cohort a macrovascular cause was found in 78 of 173 patients (45%). Patients in the development cohort were slightly older (mean age 53 years, SD 11.5 vs 50 years, SD 15.0 in the validation cohort). The frequency of underlying vascular aetiologies in each cohort is presented in table 2.
In multivariable analysis younger age, location of ICH, absence of signs of SVD and a positive or inconclusive CTA were independent predictors for presence of an underlying macrovascular cause (table 3).
A simple model based on age, location of ICH and signs of SVD had a c-statistic of 0.83 (95% CI 0.78 to 0.88) in the development cohort after shrinkage. The predictive performance of the model increased if CTA was included as predictor (c-statistic 0.91; 95% CI 0.88 to 0.94). Calibration of both models was accurate, as shown by the calibration plots (figure 1). The original regression equations are provided in online supplementary table 2. When we excluded patients in whom IADSA was not performed following a negative or inconclusive CTA, c-statistics were similar to those of the full cohort analysis. Calibration plots and c-statistics are presented in online supplementary figure 3.
Figure 2 shows risk charts with estimated probabilities of finding a macrovascular cause according to age, ICH location, presence of SVD and for the same predictors combined with CTA. The probability of finding a macrovascular cause ranged from 1% in patients aged 51–70 years with deep ICH and signs of SVD, up to more than 50% in patients aged 18–50 years with lobar or posterior fossa ICH and no signs of SVD. Two simple risk scores are presented in online supplementary table 3, which can be used in combination with online supplementary figure 4 to obtain predicted probabilities for individual patients.
External validation of the models showed a c-statistic of 0.66 (95% CI 0.58 to 0.74) for the model based on patient characteristics and NCCT, and a c-statistic of 0.88 (95% CI 0.83 to 0.94) for the model with additional CTA. The calibration plots show that the likelihood of finding a macrovascular cause increased along the range of predicted probabilities, with moderate calibration for the model with patient characteristics and NCCT (figure 1A) and good calibration for the model with additional CTA (figure 1B). Before recalibration, both models systematically underestimated the probability of finding a macrovascular cause (online supplementary figure 5).
Our study shows that younger age, lobar or posterior fossa location of ICH, absence of signs of SVD, and a positive or inconclusive CTA are independent predictors for presence of a macrovascular cause in patients with non-traumatic ICH. We combined predictors in two practical prediction charts, which we externally validated. Estimated risks vary from 1% in patient aged 51–70 with deep ICH and signs of SVD, to more than 50% in patients aged 18–50 with lobar or posterior fossa ICH and no signs of SVD. Both models showed good discriminatory ability and calibration in the development cohort, whereas performance in external validation was moderate for the model with NCCT and good for the model including CTA.
Previously, two other prediction models have been described to predict the probability of a macrovascular cause in patients with non-traumatic ICH (online supplementary table 4). The simple ICH score was developed in a retrospective cohort of 160 patients with non-traumatic ICH in which the presence of a macrovascular cause was determined with IADSA.18 Performance of the risk score was moderate in both the development and external validation cohort. This model was derived from a high-risk population, as represented by the relatively young age (mean age 41 years) and high proportion of patients with a macrovascular cause (51%). The results may therefore not be generalisable to all patients with ICH suspected of having a vascular malformation, and the prediction model will likely overestimate the probability of finding a macrovascular cause. The secondary ICH score was developed in a retrospective cohort of 623 patients with ICH in the USA.11 Presence of a macrovascular cause was determined with CTA. The model was based on patient characteristics and NCCT characteristics, which included enlarged vessels or calcifications along ICH margins and hyperattenuation within a dural venous sinus or cortical vein. Independent validation in the USA showed good performance of the model,19 performance was moderate in an external validation study in the Netherlands.3 NCCT categorisation was a strong predictor for macrovascular causes, but characteristics were not always easy to recognise on NCCT,3 which may limit easy application of the model in clinical practice. The DIAGRAM prediction score is the first model developed in a prospective cohort, excluding patients in whom yield of angiographic imaging has been shown to be very low (patients older than 45 years with a history of hypertension and a deep or posterior fossa bleed).13 Next to known predictors for a vascular malformation, we were able to add signs of SVD as important predictors of absence of a macrovascular cause. To our knowledge, this is the first prediction model that also incorporated results from CTA imaging. This can be useful in healthcare settings where CTA is often or routinely used, and clinicians have to decide whether or not to perform MRI/MRA and/or IADSA after a negative CTA. The DIAGRAM prediction score may help to weigh the probability of finding a macrovascular cause against the risk of complications of IADSA.
Performance of the model based on patient characteristics and NCCT diminished in the external validation cohort. This is likely due to the differences between the development and validation cohorts in terms of patient selection and choice of reference standard. Selection of patients influences prevalence of macrovascular causes and may affect predictor outcome associations, which in turn affect model performance. By selection of patients who underwent IADSA in the validation cohort, the prior probability of finding a macrovascular abnormality in this cohort was higher, which resulted in a systematically underestimated risk of finding a macrovascular cause by the prediction models. Simple recalibration improved correspondence between observed and predicted risks, supporting the hypothesis that differences in outcome incidence were an important source of miscalibration. Selection of more high-risk patients may also have altered predictor–outcome associations. As a consequence, the discriminatory ability of the model may have decreased. Given differences between development and validation cohorts, validation of the DIAGRAM prediction model in a prospective cohort is necessary to further establish the robustness of the model.
Strengths of our study include the prospective nature of the development cohort and the standardised radiological work-up. Another strength is the external validation in a setting outside of the Dutch healthcare system. Our study also has limitations. First, the models were developed in a preselected group of patients with a relatively high likelihood of finding a macrovascular cause, excluding those older than 70 years of age, and patients over the age of 45 years with hypertension and deep ICH or ICH in the posterior fossa. This preselected group represents patients in whom the diagnostic dilemma is most pressing in clinical practice. Generalisability to older patients with non-traumatic ICH remains to be established. In the elderly, diagnostic tests to search for macrovascular causes of ICH are often performed in only a small proportion of patients.20 Second, not all patients in the development cohort underwent IADSA. As a consequence, small AVMs or dAVFs may have been missed. However, patients were followed up for 1 year to assess occurrence of rebleeds and register possible causes of ICH identified during follow-up. Third, the association between CTA and presence of a macrovascular cause may have been overestimated, as CTA was also part of the reference standard. However, when we restricted our analyses in the development cohort to the patients who underwent IADSA, the discriminatory performance of the model remained similar. Fourth, MRI/MRA was not systematically performed in the validation cohort, which may have led to underestimation of the number of patients in whom a cavernoma was the cause of ICH.
The current models may facilitate selection of patients for further diagnostic work-up. The results of the model based on patient characteristics and NCCT suggest that in the absence of SVD, some form of angiographic imaging (CTA/MRA/IADSA) should be performed in all patients under 70 years of age, regardless of ICH location. If signs of SVD are seen on NCCT, CTA should still be considered in young patients (18–50 years old) with lobar and posterior fossa ICH, and in elderly patients (51–70 years old) with posterior fossa ICH. In settings where it is feasible to perform CTA in all patients shortly after ICH, the DIAGRAM+ score is particularly useful in patients in whom CTA was negative to guide the decision to perform these additional tests. Following a negative CTA, there is still a substantial chance of finding a macrovascular cause in patients without signs of SVD, both in young and in older patients. In these patients, performance of MRI/MRA and IADSA deserves consideration, especially in patients with lobar and posterior fossa ICH. It should be noted that also in patients with a deep ICH who do not have SVD nor hypertension (as defined by the inclusion criteria), there is around 9% (in those 18–50 years) and 3% (in those 51–70 years) chance of finding a macrovascular cause of the ICH after a negative CTA. Whether or not in these patients further imaging will be performed should be determined as part of a shared decision-making process between the patient and the team responsible for their care. Because the AVMs or dAVFs that are sought for with IADSA after a negative CTA will be small, IADSA should be performed in centres with ample experience in detecting such lesions. Although the prediction charts can provide guidance in decision-making, it should be noted that there is a degree of uncertainty around the presented estimates, as shown by the CIs in online supplementary figure 4.
In conclusion, the DIAGRAM prediction charts can help to predict the probability of finding a macrovascular cause in both low-resource and high-resource settings. External validation of the models in other prospective cohorts and in elderly patients is needed to gain further insight in the robustness of the models.
NAH, CJJA and DJW contributed equally.
Contributors CJMK, BKV, GJER and AA designed the study. BKV, GAPdK, TDW and HRJ assessed brain images. CJJvA, KMvN, FEdL, WJS, PLMdK, DWJD, TR, JH, MJHW, HK, KJ, IMB, MJMR, HPB, RJGMW, CJMK, DJW and DW collected data. NAH, CJJvA and JPG conducted the statistical analyses. NAH drafted the paper and all authors reviewed and commented on the report. CJMK is the guarantor.
Funding This study was supported by a Dutch Heart Foundation grant (no 2007B048 to CJMK). CJMK is also supported by a clinical established investigator grant from the Dutch Heart Foundation (no 2012T077), and an Aspasia grant from the Netherlands Organisation for Health Research and Development, ZonMw (015008048). JPG and NAH are supported by a grant from the Dutch Heart Foundation (no 2013T128 to JPG).
Competing interests None declared.
Ethics approval The DIAGRAM study was approved by the Medical Ethics Committee of the University Medical Center Utrecht, the Netherlands, and local approval was obtained from all participating hospitals. Data collection for the validation cohort was approved by the Clinical Governance Committee of the National Hospital and the UCL Institute of Neurology and National Hospital Joint Research Ethics Committee.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Requests for data sharing should be sent to the senior author (Karin.firstname.lastname@example.org) and will be considered on approval by the DIAGRAM investigators.
Collaborators FE de Leeuw, HB Boogaarts and EJ van Dijk (Radboud University Nijmegen Medical Center, Nijmegen; 27 patients enrolled); WJ Schonewille, WMJ Pellikaan and C Puppels-de Waard (St Antonius Hospital, Nieuwegein; 22); PLM de Kort, JP Peluso and JH van Tuijl (St Elisabeth Hospital Tilburg; 20); J Hofmeijer, FBM Joosten (Rijnstate Hospital, Arnhem; 16); DW Dippel, L Khajeh (Erasmus MC University Medical Center, Rotterdam; 16); TWM Raaijmakers (Meander Medical Center, Amersfoort; 16); MJ Wermer and MA van Walderveen (Leiden University Medical Center, Leiden; 14); H Kerkhoff, E Zock (Albert Schweitzer Hospital, Dordrecht; 14); K Jellema, GJ Lycklama à Nijeholt (Medical Center Haaglanden, The Hague; 12); IM Bronner (Flevo Hospital, Almere; 12); MJM Remmers (Amphia Hospital, Breda; 9); RJGM Witjes (Tergooi Hospital, Blaricum; 8); HP Bienfait, KE Droogh-Greve (Gelre Hospital, Apeldoorn; 8); RCJM Donders (Diakonessen Hospital, Utrecht; 6); VIH Kwa (now: Onze Lieve Vrouwe Gasthuis, Slotervaart Hospital, Amsterdam; 4); TH Schreuder and CL Franke (Atrium Medisch Centrum, Heerlen; 4); JS Straver (Hofpoort Hospital, Woerden; 2); C Jansen (Gelderse Vallei Hospital, Ede; 1); SLM Bakker and CC Pleiter (Sint Franciscus Gasthuis, Rotterdam; 1); MC Visser (Free University Medical Center, Amsterdam; 1); and CJJ van Asch, BK Velthuis, GJE Rinkel, KM van Nieuwenhuizen, CJM Klijn (University Medical Center Utrecht; 90).
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.