Atrial fibrillation (AF) is the most common arrhythmia in the elderly. Arterial stiffness may predict the risk of AF, but this relationship has not been fully evaluated. We assessed the association between arterial stiffness and prevalent AF. All subjects who had electrocardiography performed and a cardio-ankle vascular index (CAVI) calculated during a screening examination between 2010 and 2019 were enrolled. To evaluate the association between increased arterial stiffness and AF, we divided the population according to their Framingham risk score (FRS) into low-, intermediate-, and high-risk groups. A total of 8048 subjects were evaluated. The multivariate analysis revealed that increased arterial stiffness was significantly associated with AF prevalence, even after adjusting cardiovascular risk factors [odds ratio (OR) 1.685, 95% confidence interval (CI) 1.908–2.588, p = 0.017]. When we subcategorized the subjects according to their FRS, increased arterial stiffness was significantly associated with AF in the intermediate- and high-risk groups (OR 3.062, 95% CI 1.39-6.740 and OR3.877, 95% CI 1.142-13.167, respectively, BMI adjusted. High arterial stiffness shows a significant association with AF in those with intermediate or high cardiovascular risk, and can be used for further risk stratification of patients.