Harvard University law specialist Gerald Neuman, who recently completed a four-year elected term on the UN's top human rights expert panel, said Australia's refusal to reveal the reasons for detention was shocking.

"No one questions the ability of Australia to take steps to protect its security in proper ways," Professor Neuman said on Monday.

"[But] I don't understand Australia to have ever said these men, women and children were a threat to the security of Australia."

Professor Neuman, who served as the United States' representative on the UN Human Rights Committee until December last year, contributed to a scathing UN ruling against Australia's "cruel and degrading" practice to lock up refugees indefinitely without a right of appeal.

Some of mostly Sri Lankan Tamil refugees held in Australian immigration with adverse ASIO assessments have been detained more than five years.

But official correspondence revealed by Fairfax Media last week showed the Abbott government has brushed aside the UN ruling.

"There needs to be fair procedures where people can meet charges against them if they are going to be held for what is obviously years," Professor Neuman said.

He said the security assessment was a "very broad concept" that could include a risk to Australia, other countries, or fundraising.

"Once you understand that is the breadth of the security assessment, the need to keep people locked up for months and years becomes much more doubtful," he said.

The group of refugees includes two women - one of whom, Ranjini, had her third baby in detention but ASIO has said her children are not subject to adverse assessments.

But at least 12 of the refugees have been released in recent months after ASIO reversed its negative finding, yet the security agency has not provided the men any detailed explanation for the change.

Government MPs said last week Australia was not deserving of censure but wanted advice for resolving the conundrum over refugees given negative security assessments.

The refugees cannot be returned home, having been judged to have a well-founded fear of persecution, but at least 10 other countries have refused Australian requests to offer them a home.

But Professor Neuman said the government could do more to give people a fair opportunity to respond to the reasons why they are being detained - or more explanation of the reasons why information cannot be disclosed.

"One of the criticisms was that people had not been given notice of why they are being held, and Australia's response to that is somewhat shocking … that if it isn't a criminal case, you don't have to give people notice of the reasons why they are being held," he said.

"The advice that whenever you detain people you should give them notice of the reasons why they are being held, I think that is constructive advice."

The 18-member Human Rights Committee, described as the closest the world has to an international human rights court, serve as independent experts and not on behalf of their home government.

Follow us on Twitter