Cerebral small vessel disease (SVD) is a common cause of vascular cognitive impairment. A number of disease features can be assessed on MRI including lacunar infarcts, T2 lesion volume, brain atrophy, and cerebral microbleeds. In addition, diffusion tensor imaging (DTI) is sensitive to disruption of white matter ultrastructure, and recently it has been suggested that additional information on the pattern of damage may be obtained from axial diffusivity, a proposed marker of axonal damage, and radial diffusivity, an indicator of demyelination. We determined the contribution of these whole brain MRI markers to cognitive impairment in SVD. Consecutive patients with lacunar stroke and confluent leukoaraiosis were recruited into the ongoing SCANS study of cognitive impairment in SVD (n = 115), and underwent neuropsychological assessment and multimodal MRI. SVD subjects displayed poor performance on tests of executive function and processing speed. In the SVD group brain volume was lower, white matter hyperintensity volume higher and all diffusion characteristics differed significantly from control subjects (n = 50). On multi-predictor analysis independent predictors of executive function in SVD were lacunar infarct count and diffusivity of normal appearing white matter on DTI. Independent predictors of processing speed were lacunar infarct count and brain atrophy. Radial diffusivity was a stronger DTI predictor than axial diffusivity, suggesting ischaemic demyelination, seen neuropathologically in SVD, may be an important predictor of cognitive impairment in SVD. Our study provides information on the mechanism of cognitive impairment in SVD.