"The chances of Iran possessing nuclear weapons is intolerable," Olmert told Nobel Peace laureate Elie Wiesel during an informal chat on the sidelines of a Nobel laureate in Jordan.
"I hope Iran will understand that this game is serious. They have to understand they can't cross the line," he said.
Olmert also said he was not amused by calls made by Iranian leaders suggesting that the Jewish state should be wiped off the map.
"We do not take with any humour when the leader stands publicly and says 'Iran wants to wipe Israel from the map' and at the same time wants nuclear capabilities," Olmert said.
Israel -- the only country in the Middle East believed to have nuclear weapons -- is determined to prevent arch-enemy Tehran from acquiring nuclear capabilities.
Iran is currently locked in a standoff with the international community over its nuclear program, which it insists is purely for peaceful electricity generation.