“After Tax, Philadelphians 40 Percent Less Likely to Drink Soda Every Day,” reads the headline of the Drexel University press release.

Many journalists instinctively favour fiscal interventions intended to influence people’s dietary behaviour, whether or not they actually do so. They’re also quite sympathetic to measures that punish companies that dare to make a profit from consumers’ exercise of free choice. Government ought to act like a nanny, keeping citizens on the straight and narrow, and rapping corporate tempters across the knuckles, they feel.

So the media just loves this sort of headline. Newsweek promptly misinterpreted the study’s results, conflating lower odds of daily consumption with lower volume consumed:

“Philadelphia Soda Tax Cuts Consumption of Sugary Drink by 40%.”

MedicineNet also went for a conclusion beyond that of the study: “Philly’s Soda Tax Is Working, Study Finds.”

By “working” they mean “reducing soda consumption”. But the tax in Philadelphia was not levied to influence consumer behaviour, or to improve public health. It was levied in order to raise revenue for education. Some local legislators are trying to get the tax repealed. They’re opposed to it because they don’t “want government to go in and think of creative ways to grab money”. Revenues turned out to be lower than expected, which has led an independent tax policy nonprofit, the Tax Foundation, to call it a failure.

But if the effect really is a 40% decline in the likelihood of daily soda consumption, that appears to make a strong case for a tax on sugar-sweetened beverages. And that would pose a significant challenge to my 2016 article, “Sugar tax will have negligible impact on public health”.

It is, of course, entirely logical to expect a decrease in demand for soft drinks when you increase their price. The goods for which this relation does not hold true are very few and far between. The amount of that decrease is less certain, however, and the ultimate implication for public health is not at all clear.

In my article, I demonstrated that by the government’s own cited research, a sugar-sweetened beverage tax of 20% could change the obesity status of a mere 0.4% of South Africa’s population, lower daily calorie intake by only 0.42% in adult females (less in males, and a little more in children), and reduce body mass by a slight 383 grams on average. So even if all goes as planned, it won’t make enough difference to show up on a bathroom scale.

But that minuscule impact presupposes, I wrote, “a long chain of causal connections, namely that the tax will be passed on to consumers, that this will reduce SSB consumption, which will change the number of calories consumed, which will change the consumer’s dietary energy balance, which will change their body mass index, which ultimately will lead to lower obesity rates”.

Any of those connections, or all of them, could fail in the real world. Indeed, in contrast to this claim stands a massive review of 880 studies that finds the public health case for economic interventions is less compelling than its proponents claim. The assumptions made in the paper on which the South African government relied simply don’t hold up in the real world. Expecting a tax on sugar-sweetened beverages to have any public health impact at all is hopelessly optimistic.

In a follow-up article, I mentioned the limited effects of a sugary drinks tax in Mexico, which that country introduced in 2014. Although early research suggested a 6% decline in consumption, sales figures did not bear this out, and once one adjusts for confounding variables such as population growth, prosperity levels, weather, and pre-existing trends, the impact of a tax on the consumption of sugar-sweetened beverages appears to have been negligible.

So what about this dramatic 40% number from Philadelphia?

It holds up only if you overlook some glaring flaws in the study (the full text is only available upon payment of $36, but the summary is sufficient for our purposes).

The study was based on surveys conducted immediately before and after the tax was implemented. It offered no reason to believe that the claimed decrease in soft drink consumption as a result of the tax will be sustained over a longer term. It could just be that people were reminded about watching their soft drink consumption by the sugar tax campaign, but that this reminder would soon fade.

It also did not take into account that the “before” period fell over the December holiday season, while the “after” period fell in January of the new year. Perhaps the decline had nothing to do with the tax, and everything to do with over-indulgence during the holidays, followed by new year’s resolutions to go on a diet.

The study could have remedied this fatal flaw by comparing its results to historical data from Philadelphia during the same two months in prior years. Instead, however, it compared its results with nearby comparison cities. This makes little sense, since consumption patterns are known to vary significantly from place to place, and there is ample evidence, including from Philadelphia itself, that a tax raised in one jurisdiction results in imports from nearby jurisdictions that do not levy such a tax. The study we’re examining does not address this problem at all.

Although the study at first claims to investigate the impact of a sugar tax on the daily consumption of soda, fruit drinks, energy drinks, and bottled water, the effect on fruit drinks is not reported in the study summary. In the press release, one of the authors is quoted as saying:

“Sugary fruit drinks (like the fruit flavours of Snapple and Sunny Delight) were not seen to have a decline in consumption, even though they were also taxed.”

There is no explanation in the study for why the tax appears to have had such a dramatic effect on soft drink consumption, an even bigger effect on energy drink consumption, but no effect at all on the consumption of sugary fruit drinks (such as fruit-flavoured iced tea).

And the impact on the consumption of fruit juices, which are equally sweet even without added sugar, was not addressed at all. The study also did not investigate other compensatory behaviour, such as increased consumption of sweetened coffee or tea, sweets, or other sugar-loaded food and beverage products.

There are many potential confounding factors that could undermine the notion that raising a tax on sugar-sweetened drinks is effective in reducing sugar consumption, let alone affect public health in any significant way.

The confidence intervals in the study are extremely wide, suggesting very low certainty about the results. This is not at all surprising, given that the source data is from a phone survey instead of actual sales data. For example, the apparent increase in the frequency with which people drank bottled water, given as 58%, could be anywhere between 13% and 120%.

Likewise, the decrease in reported likelihood of drinking soda daily, given as 40% in the headlines, could be anywhere between 3% and 63%. And it’s entirely possible that people just told the researchers what they thought they wanted to hear, or what would make them sound better.

So, behind the dramatic claim that Philadelphia’s soda tax led to a 40% lower odds of drinking sugar-sweetened beverages daily, there is so much room for doubt that the conclusions simply don’t stand even the most cursory scrutiny.

And even if soft drink consumption does decline, as simple price theory suggests it might, studies such as these do not lead to any conclusions about public health. People can get cheap and convenient calories from a lot more sources than sugar-sweetened beverages.

Believing that a tax on soft drinks will have a significant impact on public health is not borne out by the available scientific evidence, the occasional sensational headline notwithstanding. It is just a simple tax grab.

Expect the government and other proponents of a sugar-sweetened beverage tax to go out of their way to find data to support claims that their tax is working. Spectacular surveys that show people claim to drink fewer sugary drinks may well be among them, even though we have just seen that such surveys are on shaky ground on their own, do not consider substitution effects, and do not imply anything about diet-related ill health.

We should treat these claims with great scepticism, as we should treat all claims by the government justifying ever-heavier taxation. If I had a dollar for every time they did not stand up to scrutiny, I’d be a wealthy man. Instead, the politicians drive the BMWs. DM