The lawyer for a Canadian terror suspect quietly released by U.S. authorities from camp X-Ray in Cuba begged the young man to again try to come home now that his case had been publicized.

At a press conference in Toronto on Tuesday, Rocco Galati accused Foreign Affairs of denying Abdul Rahman Khadr his passport and of being "negligent or spinning lies" because they deny knowing Khadr's whereabouts.

"Canada is acting illegally, unconstitutionally and, arguably, in a criminal fashion," he said. "Canada, I hate to say it, unfortunately does not recognize the citizenship of brown-skinned Muslims and Arabs."

Galati said Khadr, 20, made his way to the Canadian embassy in Islamabad only to be "denied travel documents by Canadian officials who indicated they did not want him back."

Khadr was later turned away from the embassy in Turkey, leaving his whereabouts presently unknown, Galati said.

"We don't know exactly where he is, we know the country but we don't know where."

Khadr's grandmother, Fatmah Elsamnah, told reporters she is growing more and more worried.

The last time she heard from Khadr, he was in Yugoslavia and running out of money.

"He is very frightened," Elsamnah said. "I feel really bad. Today is our Eid (a Muslim holiday marking the end of Ramadan). I am spending my Eid in my lawyer's office because I am waiting for my grandson."

On Monday, the Department of Foreign Affairs in Ottawa admitted that Khadr, formerly of Toronto, had been released and returned to Afghanistan where he was arrested as an alleged al Qaeda fighter two years ago.

In a statement released late in the day, Foreign Affairs spokesperson Reynald Doiron said Khadr opted not to return to Canada when he was released.

As a Canadian, the former detainee can return to Canada "as a matter of right," Doiron said, but Khadr hasn't yet made any such request, he said.

"We have no record of Mr. Khadr approaching a Canadian mission for assistance."

Khadr's family is incensed and fearful for his safety. Since his release Khadr has travelled from Afghanistan to Pakistan and Turkey, and was last heard from in Yugoslavia. Galati could not explain how Khadr has been able to move from country to country with no passport and little money.

According to Amnesty International's Alex Neve, the argument over what exactly is happening or should happen to terror suspects like Khadr -- who are held without charge or legal representation then suddenly released -- points to an underlying problem.

"It leaves one with a real concern that what we've got here is a very arbitrary process of justice and detention in which legal rights are not being respected," Neve told CTV's Canada AM.

"Of course if there is any evidence or allegations that Canadian authorities have connecting him with illegal terrorist activities, then Canadian law could and should kick in.

"But that doesn't mean he should be barred from the country," Neve said.

Ahmed Said Khadr, Abdul's father, and his oldest brother Abdullah were also allegedly members of al Qaeda, believed to have died in a gun battle in Pakistan.

Ahmed's wife and daughter were denied Canadian passports in order to leave Pakistan six months ago.

Khadr's youngest brother, Omar, was captured separately in Afghanistan, after a firefight in which an American soldier was mortally wounded. The 17 year old is believed to still be among the close to 660 suspects held at the U.S. naval base at Guantanamo Bay.

News of Khadr's release follows a Pentagon announcement that 20 more prisoners had been released from the jail on Sunday. They were returned Friday to their homelands.

Another 20 new suspects were transferred to the facility from an undisclosed location, officials said.

Since the Guantanamo prison opened in January 2002, prisoners from 42 countries have been taken there for interrogation and detention.

So far, 88 people have been released, four of whom were transferred to Saudi Arabian prisons for continued detention.