A young member of the Jewish community runs to vote in the 2004 federal election at the Young Israel Synagog in Montreal's Outremont riding June 28, 2004. (CP / Francois Roy) Prime Minister Paul Martin walks back to his office on Parliament Hill in Ottawa. (CP / Tom Hanson)

Polling and predicting turbulent campaigns

Updated Mon. Nov. 28 2005 5:40 PM ET

Bill Doskoch, CTV.ca News

Canada's political pollsters were wiping egg off their faces in the wake of the June 28, 2004 federal election results.

An Ipsos-Reid poll released two days before the election suggested the race was too close to call. But by the time the votes were counted, what was billed as a neck-and-neck horse race had turned into a 36-seat, 7.1 percentage point margin of victory for the Liberals.

So what happened?

The poll released on June 26 showed the Liberals leading with 32 per cent of decided voters. The Conservatives were close behind with 31 per cent, while the NDP stood at 17 per cent and the Greens were at 6 per cent.

By the time the ballots were counted, Liberal fortunes had risen to 36.7 per cent. Conversely, the Conservatives, NDP and Greens had dropped to 29.6, 15.7 and 4.3 per cent respectively.

The Ipsos-Reid poll was based on a sample of 2,000 people. The margin of error was plus or minus 2.2 per cent. If you conducted the same poll 20 times in the same time period, you would expect the same results for 19 of those times within that margin of error.

Certainly the Liberal total on June 28 shifted more than 2.2 per cent, although the other parties were within the margin of error.

However, the data collection for that poll ended on June 23.

Pollsters blamed the discrepancy on a reporting lag, saying they could see the numbers change over the weekend as some people, fearing a Conservative government, shifted their vote to the Liberals.

But the media had stopped reporting polls. That's because under the Elections Act, new polls can not be released on the day of an election although media outlets can mention previously published polls.

Tim Woolstencroft's polling firm, The Strategic Counsel, didn't conduct election polls during the last election.

That being said, "I don't think the pollsters got it wrong," he told CTV.ca.

"Over the weekend (before) election day, there was a movement from the NDP to the Liberals that would have tipped the balance," Woolstencroft said.

"We know that when people are in the ballot box, there can be a shift."

He believes last-minute fear of the Conservatives winning shifted how people voted on election day.

Woolstencroft said another factor was Canadian's overall discomfort with the Liberals, which could explain why their vote didn't solidify until the end.

"We had just come off (the sponsorship scandal), and a pretty unspectacular six months by Paul Martin."

Woolstencroft's theory is backed up by The Canadian Election Study, which looked at voter behaviour in the 2004 campaign. It ran small tracking polls to monitor voting intention changes on a daily basis.

"On the eve of the election, around 20 per cent of our respondents had still not decided how to vote. More importantly, fully a third of those who indicated that they were intending to vote for a given party also said that they could still change their mind," the study's principals said in a July 14, 2004 Globe and Mail commentary.

"These figures drive home an important point: A substantial number of Canadians are ambivalent about the political parties and their leaders. Under these circumstances campaigns are important, and that is why close races can be very hard to call."

Woolstencroft said a point to consider is this: "Polling is a snapshot of public opinion on that day."

Another is this: "In our business, when we say accurate 19 times out of 20, there is that one time -- although you don't necessarily want it in the poll before election day," he said.

Polling problems

Fred Fletcher, a York University professor who has done some polling, told CTV.ca he thinks it's possible there were both a shift and some methodological error on the polling side in the dying days of the 2004 campaign.

He tied that to some of the handicaps facing polls in an era of aggressive telemarketing.

"Polling is increasingly difficult because of the difficulty in getting a response rate over 10 per cent," Fletcher said.

That means to poll 1,000, a polling firm must contact 10,000, although the response rate tends to improve during election campaigns, he said.

Fletcher explained another problem is that sometimes people, fearing what their neighbors might think, may say they are undecided when they actually have made up their minds.

"So a lot of little problems, but the pollsters are pretty good about figuring this stuff out and taking it into account when they interpret their results," he said, adding it will be interesting to see how the pollsters handle the tail end of this campaign.

Woolstencroft said the number of people who misrepresent themselves in a poll is so small and random as to be irrelevant.

As for those who say they are undecided, "a lot of them don't even show up. Not everybody votes," he said.

In 2004, voter turnout was only 60.3 per cent – the lowest since 1898.

Interpreting polls

While there is statistical science behind polling, there is also some art.

"The art is what questions you ask, and how you interpret (the results)," Woolstencroft said.

University of Toronto political scientist Nelson Wiseman told CTV.ca he likes a straightforward question, such as: "If an election were held today, which party would you support?"

As a poll consumer, he likes to see questions about the leaders' approval ratings, especially with some history thrown in ("longitudinal data," in his trade).

Another question he sees as important is the public's mood on the need for a change in government.

Generally speaking, when 60 per cent of voters think it's time for a new government, that's ominous for the incumbent, he said.

While most polls are national, the real battles are fought on a regional level. For example, the Conservatives would like to see inroads in the so-called 905 suburbs around Toronto (named for their area code).

Fred Fletcher said national polls give the big picture, but they mask what might be happening at the regional level.

While polls give regional breakouts, there might be only 40 people sampled in Saskatchewan in a 1,000-person poll, making for a very high margin of error.

Another concept polls mask is "vote efficiency," the notion that votes in excess of those required to win a riding are wasted and therefore inefficient -- although having wide but thin support carries its own risks.

An example of inefficient vote is the Liberals in anglophone parts of Montreal, where they have captured 80 per cent of the vote or more in some ridings.

However, the Bloc Quebecois has the opposite problem in its Quebec rural francophone heartland.

Fletcher explained pollsters have various techniques to account for such vote inefficiencies when doing opinion surveys during campaigns.

People should keep in mind that "margin of error" doesn't mean the poll is free of other types of error, "like systematic bias in response rate or a bad question wording. Other kinds of error can creep in," he said.

In the end, if in this campaign, the two leading parties are virtually tied going into the final weekend and there's another large block of undecided voters; another voting day surprise is possible.