Leukocyte telomere length (LTL) is a proposed biomarker for aging-related disorders, including cognitive decline and dementia. Long-term longitudinal studies measuring intra-individual changes in both LTL and cognitive outcomes are scarce, precluding strong conclusions about a potential aging-related relationship between LTL shortening and cognitive decline. This study investigated associations between baseline levels and longitudinal changes in LTL and memory performance across an up to 20-year follow-up in 880 dementia-free participants from a population-based study (mean baseline age: 56.8 years, range: 40–80; 52% female). Shorter baseline LTL significantly predicted subsequent memory decline (r = .34, 95% confidence interval: 0.06, 0.82), controlling for age, sex, and other relevant covariates. No significant associations were however observed between intra-individual changes in LTL and memory, neither concurrently nor with a 5-year time-lag between LTL shortening and memory decline. These results support the notion of short LTL as a predictive factor for aging-related memory decline, but suggest that LTL dynamics in adulthood and older age may be less informative of cognitive outcomes in aging. Furthermore, the results highlight the importance of long-term longitudinal evaluation of outcomes in biomarker research.