A work camp in the northeastern county of Gilju could be the source of the labour force used to build the site of the nuclear test last October, the Committee for Democratisation of North Korea in Seoul said in a report.
Experts say the test was conducted in a mountain in Gilju.
"Testimonies from defectors suggest political prisoners may have been used to build an underground tunnel for North Korea's nuclear test," the committee said.
The report was based on interviews in South Korea with former North Korean prisoners who escaped or defected after their release.
"After collecting testimonies from North Korean defectors in China, we have concluded the North Korean regime used its class-A political prisoners camped near the testing site to build the facility," Kang Cheol-Hwan, vice chief of the committee, told Yonhap news agency.
None of the North Korean defectors who had lived near the site spoke of noticing any sign of construction or received evacuation orders, he said.
"It's very likely that Pyongyang chose to use captured individuals for the construction to prevent any information from leaking," Kang said, citing a defector who claimed to have been a security guard at the camp.
In a separate report, Freedom House, a US-based rights group, estimated that up to 200,000 people were being held without trial and subjected to forced labour.
Prisoners "are subjected, usually for a lifetime, to forced labour under extremely severe circumstances, beginning with the provision of below-subsistence level food rations," it said.