Little clinical evidence supports the strict implementation of glycemic control for diabetic patients with AMI. We aimed to demonstrate the effect of long-term glycemic control on mortality in patients with diabetes mellitus after acute myocardial infarction (AMI). Eight hundred and twenty-four consecutive diabetic patients were divided into three groups according to the mean hemoglobin (HbA1c) value: <6% (group A), ≥6% to <7.5% (group B), and ≥7.5% (group C). The best long-term mortality outcome was observed in Group B, followed by groups C and A. Groups B and C were further compared in-depth because the baseline characteristics of group A differed significantly. A Cox regression analysis indicated that Group C was associated with an adjusted hazard ratio (HR) of 1.55 [95% confidence interval (CI): 1.02–2.34, P = 0.038]. An inverse probability of treatment weight analysis was performed to compare groups B and C. Group C had significantly higher mortality, compared to group B (adjusted HR: 1.58; 95% CI: 1.21–2.06, P < 0.001). In conclusion, Glycemic status was associated with the long-term survival outcome in diabetic patients after AMI. However, further study is needed to prove whether HbA1c-targeted glycemic control can effectively improve survival after AMI.