White matter hyperintensities (WMH), perivascular spaces (PVS) and lacunes are common MRI features of small vessel disease (SVD). However, no shared underlying pathological mechanism has been identified. We investigated whether SVD burden, in terms of WMH, PVS and lacune status, was related to changes in the cerebral arterial wall by applying global cerebral pulse wave velocity (gcPWV) measurements, a newly described marker of cerebral vascular stiffness. In a population-based cohort of 190 individuals, 66–85 years old, SVD features were estimated from T1-weighted and FLAIR images while gcPWV was estimated from 4D flow MRI data. Additionally, the gcPWV’s stability to variations in field-of-view was analyzed. The gcPWV was 10.82 (3.94) m/s and displayed a significant correlation to WMH and white matter PVS volume (r = 0.29, p < 0.001; r = 0.21, p = 0.004 respectively from nonparametric tests) that persisted after adjusting for age, blood pressure variables, body mass index, ApoB/A1 ratio, smoking as well as cerebral pulsatility index, a previously suggested early marker of SVD. The gcPWV displayed satisfactory stability to field-of-view variations. Our results suggest that SVD is accompanied by changes in the cerebral arterial wall that can be captured by considering the velocity of the pulse wave transmission through the cerebral arterial network.