HAMILTON — It's "a bit hypocritical" of Jean Chretien, who attended only one Olympics during his tenure at Canada's helm, to attack Stephen Harper's decision to skip the Summer Games in Beijing, the Conservative prime minister said Tuesday.

Harper briskly fired back at the former Liberal prime minister, who on Monday warned Canada could end up paying a hefty toll for Harper's decision not to go to China, the world's second-largest and fastest-growing economy.

"I haven't attended Olympics in the past, (and) I indicated a long time ago I had no intention of attending this one; I have work to do in the country," Harper said during a news conference in Hamilton.

After a speech Monday to the Canadian Bar Association, Chretien said he wouldn't have "hesitated a second" to attend the Beijing Olympics were he still prime minister.

"It is the second biggest economy in the world, and in 50 years it will be the biggest," Chretien said.

"Starting with Diefenbaker, Trudeau and all of us, we established very good relations, relatively speaking, with China. And suddenly, you break the bridge. It would have been easy just to be there."

Chretien was known for keeping ties to China during his time in power; a photo of the suit-and-tie-clad prime minister, riding a bicycle as Chinese security officers scramble to keep up during a trade mission in 1994, became one of the enduring images of his time in office.

When it came to the Olympics, Harper noted, Chretien -- who spent 13 years as Liberal leader and 10 as prime minister, stepping down in 2003 -- attended only the 1996 Summer Games in Atlanta during his time in the prime minister's office.

"Mr. Chretien is being a bit hypocritical."

Harper earned a small smattering of applause from the crowd for his response to a reporter who noted U.S. President George W. Bush was among the leaders who attended the Games.

"I don't always do what Mr. Bush does," he said.

Harper has long insisted his decision to stay away from the Games had nothing to do with politics, but rather with a scheduling conflict.

Canada was represented at the Aug. 8 opening ceremonies by Foreign Affairs Minister David Emerson, while Secretary of State for Sport Helena Guergis and B.C. Premier Gordon Campbell also attended.

Chretien said relations between the two countries have deteriorated since the Conservatives came to power.

"As a Canadian, I deplore the fact we've lost our position in the world."