Because It Works

Ian Ayres

Crosspost from Freakonomics:

To my mind, WeightWatchers is the industry leader in performing rigorous testing of their services. Under the leadership of Karen Miller-Kovach, its chief scientific officer, it has sponsored several randomized control trials comparing the effectiveness of the WeightWatchers point system to other diet approaches. For example, Miller-Kovach is a co-author of this 2003 JAMA study (which showed that after 2-years WeightWatchers helped overweight dieters lose about 3 percent of their body mass – reducing their average weight from 207 to 201 pounds).

But I’m troubled by the current advertising campaign that accompanies the rollout of the New PointsPlus system.

The tagline “Because it works” is not well supported by the data. (You can see the phrase in the background in this television commercial, where the amazing Jennifer Hudson belts out “Feeling Good.”)

Miller-Kovach has been out in front promoting the new point system claiming, for example:

Our new PointsPlus program is based on the latest scientific research and is designed to guide people to foods that are nutrient dense and highly satisfying, ensuring they will make healthful decisions, have successful weight loss and learn to keep their weight off long-term.

The WeightWatchers site helpfully has a page laying out “The Science behind thePointsPlus™ Program”:

The Weight Watchers approach delivers a science-based, lifestyle modification program based on 4 pillars— diet, physical activity, positive thinking skills and an environment of support.

The new PointsPlus program does not waiver from this foundation, but rather incorporates advances in science and nutrition as it relates to weight management. It has been tested in a rigorous, independent clinical trial, and the results demonstrate it delivers significant weight loss as well as improvements in cardiovascular risk factors and eating behaviors linked with long-term weight loss and hedonistic hunger (an urge to eat when the body does not have a biologic need to). ^1,2

These two footnotes (1,2) are the crucial backup for both the “because it works” claim and the claim that the PointsPlus program has been “tested in rigorous, independent clinical trials and the results demonstrate it delivers significant weight loss.”

It turns out that the two studies, which are both coauthored by Miller-Kovach, provide pretty meager support for these claims.

One study took 132 adults with BMIs between 27 and 35 and randomized them to “1 of 2 systems for appraising food intake” (I guessing, the old points and the new points system). Then after 12 weeks, the researchers looked to see whether there was a difference between the groups in weight loss or in self-reported measures of weight control of hedonic hunger. The key, and to my mind alarming, results were succinctly stated:

Results: 111 subjects (99 F, 12 M) completed all Week 12 assessments. With no differences between conditions on any measure, analyses used the combined sample. M % weight loss was 4.4% (S.D.=3.71).

The study of just 111 subjects seems to have found no difference on any measure between the old and the new point systems. The average weight loss for diets after 12 weeks was 4.4 percent, which is not only modest (especially given that most dieters regain weight between 6-24 months), but the standard deviation of 3.71 suggests that it is not even a statistically significant loss in weight. Counter to the claims of the website, I do not believe that “the results demonstrate it delivers significant weight loss.”

The second study which is cited seems to be just a second analysis of the same experiment (132 adults again with BMIs between 27 and 35). Instead of looking at weight loss and self-reported measures of hunger, this study looked to see whether there were differences between the groups in “Lipid, blood glucose, blood pressure and waist circumference.” But again, the researchers report no differences between the old and new point systems: “With no weight loss differences between conditions, analyses used the combined sample.”

So, as far as I can tell, the change to the new point system is supported by a single experiment of 132 subjects that could not find any difference in results between the two point systems. To my mind, the campaign may mislead consumers, who are more likely to hear in the phrase “New PointsPlus, because it works” the idea that WeightWatchers made the change “because the new point system works better than the old points system.” But the current studies don’t support this inference. A more accurate, but less appealing tagline would be: “New PointsPlus, because it doesn’t work any worse.”

I may be wrong about all of this. One of the downsides of starting to teach Intellectual Property is that I find myself seeing an increasing number of Lanham Act violations. In fact, the new point system, by providing more dieting freedom and flexibility (the new system, for example, attributes zero points for fruit), without any worse results may be a net good. But I fear this more subtle message is unlikely to be understood by consumers who are in the market for a weight loss plan.

(Conflict Disclosure: I’m a co-founder of a website and author of a diet booklet (including a frank assessment of its limited empirical support) which might be viewed as either a substitute for or a complement to the WeightWatchers system.)