Amid standoffs with Iran and North Korea over their nuclear aspirations, Annan said, "We are witnessing continued efforts to strengthen and modernize nuclear arsenals."
"We also face a real threat that nuclear weapons will spread. Without concerted action, we may face a cascade of nuclear proliferation," Annan said.
"Sadly, the world has made little progress in addressing these new challenges," he said in a statement read in his name by Nobuyasu Abe, the UN undersecretary for disarmament, at a ceremony in Hiroshima.
Annan called for international leaders to take action against nuclear weapons at the United Nations summit opening September 14 in New York.
"I challenge them to use that occasion to break the deadlock on the most pressing challenges in nuclear proliferation and disarmament.
More than 140,000 died in Hiroshima, some vaporized or dying months later from radiation and horrific burns.
Echoing statements made after attacks such as the September 11 hijackings, Annan said, "Today we are all Hibakusha," using the Japanese term for victims of the Hiroshima and Nagasaki nuclear bombings.
"No nuclear weapon has been used again, and progress has been made in reducing such weapons and preventing their proliferation. But we still live in a world where tens of thosuands of nuclear weapons remain, many of them on hair-trigger alert," he said.