The two-hour battle started early in the morning, when between 20 and 30 fighters fired on the marines with guns and rocket-propelled grenades in northwest Fallujah, the US-led coalition said.
"Helicopter crews reported nine enemy killed and an unknown number wounded," a statement said. "Marines suffered three wounded, two of whom were evacuated."
The fighting in the powderkeg Sunni Muslim town came only two days after the coalition and civic mediators agreed to continue an uneasy ceasefire.
The coalition said the clashes threatened to derail peace initiatives in the city, with three breaches of the ceasefire in 24 hours.
The deal announced Monday was intended to allow for the return of thousands of families who had fled earlier fighting. But only seven families were allowed back Wednesday, an AFP correspondent on the scene reported.
On Tuesday, the marines had let 50 families return home, and pledged to allow another 50 to enter Wednesday and the same number the next day under an accord reached by US-led coalition officials and civic officials.
Since the Americans laid siege to Fallujah more than two weeks ago, fighting has killed about 600 Iraqis, according to hospital sources. Scores of marines were also killed in the bloodiest fighting since the US-led invasion of Iraq and the ousting of Saddam Hussein a year ago.
US Defense Secretary Donald Rumsfeld said Tuesday chances of a negotiated solution to the Fallujah standoff were "remote" and warned that the US military would not stand by indefinitely.
The marines are poised for military action against insurgents but no deadline has been set for the negotiators trying to defuse tensions, top US military leaders said.
Yet the Fallujah agreement was "accepted by all the fighter groups," according to Iraq's committee of Sunni Muslim religious figures.
The marines launched their offensive against the rebels after four US security contractors were killed and the bodies of two of them were mutilated on March 31.
In Baghdad, Sunni religious leaders from Fallujah urged the insurgents to hold on to their weapons and vowed revenge against the United States at a gathering of dozens of people on Wednesday.
"We beg God that he will avenge us and foil the plans" of the Americans, said Sheikh Mohammed Abdel Aziz al-Ani, imam of a Fallujah mosque, who was greeted by shouts of "Allahu Akbar" (God is Greatest).
Sheikh Hamid al-Maadidi, also from Fallujah, told the meeting: "It is out of the question that the mujahedeen (fighters) hand over their weapons, or that joint patrols with the Americans around the city take place.
"In Fallujah, we have five Mujahedeen leaders, but I am not going to give you their names. But I can tell you that these leaders have under their command 500 fighters determined to continue," he added.
"We refuse all negotiation. We demand that the American troops leave. It is the only condition for us to stop fighting," said Maadidi.
But Mohammed al-Hamadani, a prominent Fallujan, said residents had no choice other than making a deal with the Americans. "We are an occupied country and the occupier has the weaponry to cause immense destruction."
"We ask for the unconditional return of the population and compensation for destruction caused," he said.