With flu season upon us, a controversial editorial in the British Medical Journal is warning that the flu shots many Canadians get every fall may not be as effective as we believe.

Across the world, health agencies recommend the use of inactivated flu vaccines, or vaccines containing dead flu viruses, to protect the population from seasonal outbreaks of influenza.

But because influenza viruses mutate from year to year, argues vaccine expert Dr. Tom Jefferson, it's difficult to study the precise effects of the widely-prescribed vaccines.

In fact, Jefferson says there's evidence proving these vaccines have little or no effect on things such as hospital stay, time off work, or even death resulting from influenza and its complications -- especially in elderly people.

Jefferson makes his arguments in a controversial report titled "Influenza vaccination: policy versus evidence," published in volume 333 of the British Medical Journal (BMJ).

The rationales behind the use of the vaccines in Canada and the U.S., he says, rests on the "heavy burden that influenza imposes on the population and the benefits of vaccination."

Canada backs present policy

Dr. Theresa Tam, an influenza expert with the Public Health Agency of Canada who has read Jefferson's report, said the agency stands behind its recommendation of flu vaccines.

"We base our policies or recommendations on scientific evidence, and we have reviewed all available evidence," Tam told CTV.ca on Thursday.

"Our summation is that the benefits of influenza vaccines outweigh any risk, and the vaccine is safe and effective. It can prevent hospitalization and save lives."

This is supported by the World Health Organization, which estimates that vaccinating elderly people reduces their risk of serious complications or of death by 70 to 85 per cent.

However, says Jefferson, WHO bases its recommendations on "single studies" that are random and are carried out from one or two seasons, making them "difficult to interpret."

"Single studies are also not reliable sources for generalizing and forecasting the effects of vaccines, especially when numbers are small," he said.

A better way, he said, would be to use systematic studies that average out over several years and test a large number of people in order to prove that these vaccines are having any effect.

The evidence

To highlight his claim that there's a lack of evidence backing the vaccine's effectiveness, Jefferson cites a study in the Cochrane review, "Vaccines for preventing influenza in healthy children," which found that in children under two years old, inactivated vaccines had the same effectiveness as placebo.

Further, a study in the same review for adults found that "in healthy people under 65 vaccination did not affect hospital stay, time off work, or death from influenza and its complication."

Jefferson says there is a "surprising" gap between what the policies of health agencies surrounding influenza tell us and the actual data.

"The reason for this situation are not clear and may be complex," says Jefferson.

He suggests there may be a confusion between influenza and influenza-like illness, where people are being diagnosed with the flu when they have something else.

"This confusion leads to a gross overestimation of the impact of influenza, unrealistic expectations of the performance of vaccines, and spurious certainty of our ability to predict viral circulation and impact."

He said this problem is compounded by the lack of accurate, fast surveillance systems that can tell exactly what viruses are going around in a community. Therefore, at the end of the flu season, there's no way for scientists to get an accurate picture of what went on to better prepare for future tends, said Jefferson.

In summary, Jefferson found in his research:

• Public policy worldwide recommends the use of inactivated flu vaccines to prevent seasonal outbreaks.
• Systematic reviews of large datasets from several decades provide the best information on how vaccines are performing since influenza viruses vary year to year -- but we're basing our policies today on poor, insufficient single studies.
• There's little comparative evidence out there proving these vaccines are safe.
• There's big gap between policy and evidence surrounding these vaccines. But there' are hug resources out there and health agencies urgently needed to re-evaluate their policies.

While Jefferson ignored several studies because of perceived weaknesses in their methodology, Dr. Tam said they nonetheless offer significant and useful data to health care specialists.

"When a program is in place, the other types of studies that are generated are ones that are more descriptive, where you don't have controls and cases. So methodologically, they may be considered weaker from that perspective," said Dr. Tam.

"However, they're nonetheless extremely useful and much more feasible to do. So we certainly believe there are a variety of studies out there. They may differ in the methodology, and some of them may have biases for which we will take into consideration when we're considering the strength of the study. But the bottom line is that the weight of evidence is such that we believe the influenza vaccine is effective."