A 24-day run with no new cases was broken Tuesday when it emerged two women who recently arrived from Britain were allowed out of quarantine early without being tested for the virus, even though one had mild symptoms.

The pair were eventually swabbed and proved to be infected, but only after they made a 650-kilometre (400-mile) road trip from Auckland to Wellington to see a dying relative.

Ardern said it was "absolutely nonsensical" they were not tested earlier and border controls clearly needed to be tightened to prevent similar failures.

She said Assistant Chief of Defence Digby Webb had been appointed to oversee border quarantine operations and was being given access to military personnel and logistical expertise.

"My view is that we need the rigour, we need the confidence, we need the discipline that the military can provide," Ardern told reporters.

Health Minister David Clark acknowledged widespread anger at the blunder. Kiwis endured a stringent seven-week lockdown to eliminate the virus in the country which has recorded only 1,156 cases and 22 deaths in a population of five million.

"New Zealanders have made great sacrifices to make it to this point," he told Radio New Zealand.

"Our system has performed incredibly well as a whole in New Zealand. We have eliminated COVID-19 but I want this fixed straight away."

- 'Envy of the world' -

The South Pacific nation last week scrapped domestic social distancing measures while maintaining strong border controls.

The changes have heralded a return to near-normality, with sports matches played in front of sold-out stadiums, nightclubs open and thousands gathering for events such as Black Lives Matter Protests without restrictions.

Clark said it was unacceptable that mistakes at the border, which is now seen as the frontline in the fight against COVID-19, could put such gains at risk.

"We're the envy of the world in many ways and we want to continue being the envy of the world," he said.

New Zealand's borders are open only to returning Kiwis and their families, besides some exceptions for some foreigners on business and compassionate grounds, with everyone expected to undergo two weeks mandatory quarantine.

Officials say there are approximately 3,500 people in border quarantine, mostly staying in hotels where they are expected to remain isolated in their rooms and avoid social contact.

The programme that allowed recent arrivals to leave isolation early on compassionate grounds has been suspended and everyone in quarantine must test negative for the virus before they are allowed back into the community.

Ardern stressed that the women at the centre of the furore had done nothing wrong and complied with health protocols at all times.

They remain in isolation in Wellington as health officials scramble to test about 320 people they had contact with while in New Zealand.