The United Nations and the International Committee of the Red Cross (ICRC) have condemned the attack on a former college in the city of Dhamar that the Huthis used to hold their opponents.

The coalition's spokesman Turki al-Maliki reiterated its insistence that the Huthis used the building to store drones and air defence systems.

"The (facility) was not on the 'no strike list' of sites in the city of Dhamar," he told a press conference in Riyadh.

"Some reports have quoted the ICRC as saying it has gone to the site a number of times. The coalition has never been informed... about the location.

"The Huthis bear full responsibility for making this a location for Yemeni citizens who have been forcibly disappeared."

The ICRC, which rushed to the scene with medical teams and body bags, said that every detainee in the building was killed or injured when the multi-storey facility crumbled, and that the toll could rise as high as 130.

"The facility held around 170 detainees. Forty of those detainees were being treated for injuries; the rest are presumed killed, though no toll has been confirmed," it said in a statement.

The ICRC said it had visited the detention centre in the past as part of its regular work as a neutral inspector in war zones.

"Witnessing this massive damage, seeing the bodies lying among the rubble was a real shock. Anger and sadness were natural reactions," Franz Rauchenstein, its head of delegation for Yemen, said in a statement.

"People who are not taking active part in combat should not die in such a way," he said after travelling to the attack site.

- Casualties 'staggering' -

The Saudi-led coalition intervened in 2015 to support the government after the Iran-linked Huthis swept out of their northern stronghold to seize Sanaa and much of Yemen -- the Arab world's poorest nation.

Fighting since then has already claimed tens of thousands of lives and sparked what the United Nations calls the world's worst humanitarian crisis.

Footage obtained by AFP showed heavy damage to the building and several bodies lying in the rubble, as bulldozers worked to clear away huge piles of debris.

The UN's humanitarian coordinator in Yemen has called Sunday's air strike "horrific" and said aid groups had been forced to divert critical medical supplies, intended for treating cholera outbreak, to Dhamar hospitals.

"We have no choice," Lise Grande said. "The scale of the casualties is staggering."

Both sides in Yemen's conflict stand accused of actions that could amount to war crimes.

The coalition has been blacklisted by the UN for the killing of children, while Saudi Arabia and its allies accuse the Huthis of using civilians as human shields in densely populated areas.

The attack in Dhamar comes weeks after a new front opened in Yemen's complex war, pitting two of Riyadh's allies against each other in a struggle for control of the south.

The United Arab Emirates, a key member of the coalition backing the Yemeni government, has also trained and supported secessionists who seek an independent southern Yemen.

The southern forces last month seized control of the interim capital Aden in the south, in a see-sawing battle with government troops that has drawn warnings that the country could break apart entirely.

The government has accused the UAE of backing a "coup".

Analysts say the fight for the south is a key test for Saudi Arabia, which hopes to mediate a ceasefire so it can focus on its main mission of battling the Huthis who are aligned with its arch foe Iran.