On the wall of a monastery in Syria's desert, jihadists from the Islamic State group left a grim warning: "The lions of the caliphate are here to devour you."

The Syrian army on Sunday drove out the jihadists, but the damage they have caused in a place that was once a symbol of religious tolerance seems almost irreparable.

The monastery's old dry stone and mud brick church of Mar Elian has been reduced to a heap of rubble, according to a team of AFP journalists at the scene on Monday.

IS razed the fifth-century church in August 2015 using explosives and bulldozers, as they have done with shrines and other religious buildings elsewhere, "under the pretext that people worshipped a deity other than God", according to the Syrian Observatory for Human Rights.

Under the broken stone cover of a sarcophagus lay the skull and bones of Mar or Saint Elian -- a Christian from Homs province who was slain by the Romans for refusing to renounce his faith.

"These are indeed his sarcophagus and his remains," Father Jacques Mourad, head of the Syriac Catholic monastery, told AFP by phone from Italy, after he saw photos sent via the WhatsApp phone application.

The priest himself narrowly escaped IS's grip in October 2015 after spending 84 days in the town, facing the imminent threat of death if he refused to convert to Islam.

"I am filled with grief, and I choose to remain silent, because in the face of everything that is happening, silence is the most fitting answer," he told AFP.

The walls of the entrance and the interior of a new church, inaugurated on September 9, 2006 by Christian and Muslim religious officials, are completely charred after IS set fire to it.

Joists hang from the ceiling and the stone altar is broken.

Part of the 16-room monastery was destroyed by shelling, while the pots and plates used by jihadists to cook in the kitchen have been left behind.

In a little room, bags of bone remains can be seen.

They were found by archaeologists in two Mamluk and Ottoman cemeteries next to the monastery, said May Mamarbachi, who helped restore the site 10 years ago.

According to Father Mourad, two other churches in the centre of Al-Qaryatain were set on fire in the first week of the jihadists' occupation of the town.

- Crossroads -

Al-Qaryatain was one of IS's last bastions in Homs province, Syria's largest.

The town was home to some 30,000 people before war broke out in 2011, 900 of them Christians.

It is significant victory for the army because the town is a located on a crossroads between Damascus and Homs provinces, as well as the Qalamun mountains on the Lebanese border.

"By capturing this town, the army has cut off all the roads that Daesh was using to move," a general in the town said, using an Arabic name for IS.

The jihadists withdrew eastwards after losing the fight against the army, he added.

"The battle became easier after the capture last week of Palmyra," the general said.

Like the ancient city of Palmyra after its return to army control, Al-Qaryatain is now completely deserted.

Shop windows have been smashed in and buildings have either collapsed or left riddled with bullet holes.

"The Islamic State will remain and it will spread," reads the extremist group's slogan painted on a wall.

On the ground lay the tatters of burned IS flags.

Of the town's Christian residents, 277 stayed behind when IS took over. One was executed, 10 were killed in bombing, while five are still prisoners of IS.

The rest escaped at the end of 2015, Father Mourad said.

Al-Qaryatain, whose name means "the two villages" in Arabic, was once a symbol of coexistence between Christians and Muslims.

Legend has it that when the Arabs arrived in the sixth century, one of the town's main families converted to Islam, while the other remained Christian.

That way, they could protect each other.