This study aimed to evaluate whether the combination of inflammatory markers could provide predictive powers for mortality in individual patients on dialysis and develop a predictive model for mortality according to dialysis modality. Data for inflammatory markers were obtained at the time of enrollment from 3,309 patients on dialysis from a prospective multicenter cohort. Net reclassification index (NRI) and integrated discrimination improvement (IDI) were calculated. Cox proportional hazards regression analysis was used to derive a prediction model of mortality and the integrated area under the curve (iAUC) was calculated to compare the predictive accuracy of the models. The incremental additions of albumin, high-sensitive C-reactive protein (hsCRP), white blood count (WBC), and ferritin to the conventional risk factors showed the highest predictive powers for all-cause mortality in the entire population (NRI, 21.0; IDI, 0.045) and patients on peritoneal dialysis (NRI, 25.7; IDI, 0.061). The addition of albumin and hsCRP to the conventional risk factors markedly increased predictive powers for all-cause mortality in HD patients (NRI, 19.0; IDI, 0.035). The prediction model for all-cause mortality using conventional risk factors and combination of inflammatory markers with highest NRI value (iAUC, 0.741; 95% CI, 0.722–0.761) was the most accurate in the entire population compared with a model including conventional risk factors alone (iAUC, 0.719; 95% CI, 0.700–0.738) or model including only significant conventional risk factors and inflammatory markers (iAUC, 0.734; 95% CI, 0.714–0.754). Using multiple inflammatory markers practically available in a clinic can provide higher predictive power for all-cause mortality in patients on dialysis. The predictive model for mortality based on combinations of inflammatory markers enables a stratified risk assessment. However, the optimal combination for the predictive model was different in each dialysis modality.