Speculation is swirling that Prime Minister Stephen Harper is poised to shuffle his cabinet, after scheduling meetings with a number of federal ministers.

Rumours of a shuffle have been fuelled in part by word that Harper is set to meet this week with a number of senior ministers, including Indian Affairs Minister Jim Prentice.

House Leader Rob Nicholson was one of the ministers seen outside Harper's office Tuesday, reports the Canadian Press.

As well, The Globe and Mail reports that Harper has set travel restrictions on some of his ministers, obligating them to be within close flying range to Ottawa, beginning next week. Finance Minister Jim Flaherty is the exception, as he is scheduled to fly to China for meetings.

Rona Ambrose, who has been widely panned for her performance in her role as environment minister, could be bumped out of her position, according to the rumours.

"The Tories have been vulnerable on the environment file and need to do something... because Canadian voters seem to care a great deal about the environment," CTV's David Akin told Newsnet Tuesday.

The speculation, according to The Globe, is that Jim Prentice would take over the environment file and Ambrose would take on intergovernmental affairs, replacing Peter Van Loan, who would go to Indian Affairs.

But Akin reports that is far from certain.

"Everyone we have spoken to says that Prentice is too valuable, he's the de facto deputy PM, doing a relatively good job at Indian Affairs, which is typically a political minefield," Akin reports.

Akin says it might be Industry Minister Maxime Bernier who takes over the environment portfolio. Bernier has a safe seat in Quebec and would be better poised to sell the government's environment package in Quebec.

There is also speculation that the bilingual Treasury Board President John Baird could be moved to a post with a greater profile.

Rumours of a cabinet shuffle gathered steam in the days leading to the Christmas break. Harper joked at a news conference just before the House broke in mid-December that he wasn't planning a shuffle on that very day. Instead, Harper said he wasn't prepared to speak to the issue yet.

"You guys all want to speculate. Go ahead and speculate. You didn't get it from me," he told reporters.

Meanwhile, Harper's New Year's message betrayed no hint of election timing. Harper said his highlight of 2006 was visiting Canada's troops, diplomats and aide workers in war-ravaged Afghanistan.

"Through their selfless acts, these brave men and women are protecting our security interests, and making a real difference in the lives of the long-suffering Afghan people," he said.

Calling Canada the "greatest" country in the world, Harper said the economy is strong, the country is united and that it has "taken a lead role on the world stage."

The prime minister didn't end his speech with his signature "God bless Canada." Instead, he wished Canadians a "Happy New Year ... and may God keep our land glorious and free."

The House of Commons returns on Jan. 29.