The influence of body fat on arterial stiffness remains controversial. This study was performed to investigate the associations between four different types of body fat parameters and brachial-ankle pulse wave velocity (baPWV). A total of 3758 subjects (mean age, 53.4 ± 8.8 years; females, 36.3%) who underwent health check-up were retrospectively analyzed. Anthropometric parameters including body mass index (BMI), waist circumference (WC) and waist–hip ratio (WHR) were assessed, and visceral fat area (VFA) was calculated by bioelectrical impedance analysis. In simple linear correlation analyses, baPWV was associated with WC, WHR and VFA (P < 0.001 for each), but not with BMI (P = 0.175). In multivariable analyses, BMI and WC were not associated with baPWV (P > 0.05 for each). Even after controlling for potential confounders, higher baPWV was significantly associated not only with higher WHR [for > 0.90 in men and > 0.85 in women: odds ratio (OR), 1.23; 95% confidence interval (CI), 1.06–1.42; P = 0.005; for the highest tertile compared to the lowest tertile: OR, 1.38; 95% CI, 1.15–1.66; P < 0.001], but also with higher VFA (for ≥ 100 cm2: OR, 1.39; 95% CI, 1.20–1.60; P < 0.001; for the highest tertile compared to the lowest tertile: OR, 1.77; 95% CI, 1.48–2.12; P < 0.001). Our study showed that baPWV was correlated with WHR and VFA, but not with BMI and WC. This implies that arterial stiffness may be more strongly associated with abdominal obesity than overall obesity.