September 7, 2010

Why The Latest Low Carb Scare Study is Flawed

You'll soon be hearing about a large scale epidemiological study of low carb diets and mortality published this week in Annals of Internal Medicine which purports to show that eating a low carb-meat based diet increases the incidence of mortality (i.e. death!).

Low-Carbohydrate Diets and All-Cause and Cause-Specific Mortality: Two Cohort Studies Teresa T. Fung et al. Annals of Internal Medicine September 7, 2010, vol. 153 no. 5 289-298

Here's why it's crap--and completely irrelevant to those of us who control carbohydrates to keep our blood sugars normal.

The methodology used here was this:
Prospective cohort study of women and men who were followed from 1980 (women) or 1986 (men) until 2006. Low-carbohydrate diets, either animal-based (emphasizing animal sources of fat and protein) or vegetable-based (emphasizing vegetable sources of fat and protein), were computed from several validated food-frequency questionnaires assessed during follow-up.
This raises four immediate red flags.

1.Based on Inaccurate Questionnaire Data. The standardized food frequency questionnaire used in nutrition studies is designed in such a way that it is impossible to gauge the actual food intake of people eating a true low carbohydrate diet.

It is a list of questions like, "How many times in the last month did you eat bread" followed by a multiple choice set of answers which are along the lines of "1 to 10 times" "11-25 times", "26-50" times "More than 50 times."

But the answers supplied for carbohydrate items do not allow for the possibility that the answer to a question like "How many times this past month did you eat bread" is zero or even five. Ditto potatoes. Ditto sweets.

I know first hand how inaccurate the standard food frequency questionnaire is because several years ago I was a subject in a long term study that used the standard nutritionist designed food intake questionnaire. During this time I was logging my actual food intake, with weighed portions, trying to understand my own pattern of weight loss, so I knew exactly what I was eating during any given day or month.

The nutritionist associated with the study emailed me the nutritional breakdown that I had supposedly eaten, based on my answers to the standard food frequency questionnaire. It bore no relationship at all to what I had eaten either in terms of calories or the percentages of my diet represented by protein, carbs, or fat. When I offered to send the study my actual food intake, I was told that the questionnaire they were using had been carefully validated and was standard in all nutritional studies and that there was no point in looking at what I had actually eaten.

If that isn't bad enough, the questionnaires rely on the memory of the people filling them out--who are likely to "forget" or downright misrepresent how often they ate foods they know are bad for them like sodas, snacks, and sugary desserts. It is well known that people give the answers they wish they were true on questionnaires like this where there is no way for anyone to check up on whether they are telling the truth.

Garbage in, garbage out. But since it is garbage that reinforces a religious belief nutritionists want to hold onto even though better designed studies do not support it, you will be seeing this study used to "prove" how unhealthy the low carb diet really is.

2.Misleading Definition of "Low Carb Diet" Leaving aside the fact that the data collection method renders the data highly questionable, the definition of "low carb" used in this study is almost certainly one most of us that eat a true carb restricted diet would consider a high carb diet--one of 150-200 grams of carbohydrates a day.

If that is the case, the "meat based" low carb diet was probably a "meat and potatoes and bread" diet, which we are all agree is not healthy for anyone.

3. "Meat-based diet" May Mean "Fast Food Meat and Potatoes Diet" All meats are not the same, but the standard food intake questionnaire makes no attempt to distinguish between fast food, chemical laced meat served with sides filled with trans fat and home cooked nutritious meals. The questionnaire only asks about how often the respondent ate "pork" or "beef," not where they ate it, or what cut they ate. Or most importantly, with what side dishes.

I have no doubt that eating a diet of fast food burgers (with the rolls conveniently forgotten when reporting "bread" intake") is quite capable of shortening life. But it is not because the person is eating a "meat based low carb diet" that they are risking their health.

4. "Plant based Low Carb diet" Highly Suspect The "plant based" low carb diet which was found to be so healthy here is unlikely to be low carb, since it is almost impossible to eat a true low carb diet relying on plant based foods without overdosing on soy which is far from healthy. This finding again points to the likelihood that the "low carb" diet is not low carb but moderate carb and that the plant eaters here are people who stay away from fast food outlets.

Once again, the nutritionists defending their turf use flawed methodologies to ensure that people with diabetes will continue to eat the "healthy whole grains" and sugary fruits that ensure they will end up with severe, life threatening complications.



Scott said...

Self-reported surveys are immediately biased no matter what the topic. This suggests this study was actually qualitative rather than quantitative. But in evidence-based medicine, qualitative research is not acceptable to prove something, rather it is used to design quantitative studies that can actually answer these questions. The real tragedy in all of this is the fact that the press is almost certain to start publishing stories about what this study found. The net result is that patients will be more confused and are likely not to make any changes that will positively impact their own health. It may be more appropriate to encourage readers to write directly to Dr. Teresa T. Fung, ScD, Department of Nutrition, Simmons College, 300 The Fenway, Boston, MA 02115; e-mail: and ask them to avoid headline-based press releases and dispute any comments that imply this research is something it's not. I suspect enough e-mails to crash the servers at Simmons College in Boston might change the way this is done in the future!

jpg77 said...

I suppose by now everyone has seen this reported in the USA Today (09/07/2010) []

Sad that so little valid information can go so far.

Alan said...

It never fails to amaze me how 150-200 grams of carbs a day can be considered low-carb. What planet are these people from? Please do some decent research before publishing these worthless papers.

PJNOIR said...

great stuff Jenn. Since low carb to the general public is more like extreme high carb to low carbers, this survey/study is a profile on inaccuracies.

ShottleBop said...

Thank you for writing this take on the study, Jenny. Your personal knowledge on the way in which nutritional studies are conducted is especially illuminating. That Ornish either is not aware of the limitations of this study (as evidenced by his crowing about how it validates his dietary recommendations), or is aware of them and doesn't have the intellectual honesty to refrain from crowing about it, is more than a little troubling.

Jenny said...

So many studies have shown Ornish's diet to worsen all the cardiac risk factors, no wonder he's thrilled with this one.

The man has made $$$$$$ promoting his branded diet, though, and that is high motivation to keep it going too.

sibylla said...

Great article as usual. I am recently diagnosed and can I say, Jenny, that reading this blog and your blood sugar blog was one of the things that kept me from going insane. It has been tremendously empowering and I thank you from the bottom of my heart.

BTW not to defend Ornish or anything but years ago I tried out his diet and lost a good bit of weight (6lbs in 4 weeks, however it all came back for other reasons!). However even with my own loose interpretation of his diet, just sticking to the general principles was extremely stressful. I spent my entire time planning my next meal(s). I'm only grateful I got his book out of the library so did not contribute directly to his enrichment!

Fast forward 10 years and after having lost 1/4 of my body weight I found myself, in my size 8 jeans, being told I have very severe diabetes. I get so angry every time I hear comments like 'losing weight, small amounts, can help prevent diabetes yadadadada...'

Thank you for all that you do. You really make a difference - and there's not many people you could truly say that about.

Lori Miller said...

Then there's the old standby about epidemiological studies: correlation is not causation.

Something I've noticed when I read anti-low-carb stories online in the news is that quite a few people leave comments that they regained their health by eating a low-carb diet.

Danyelle6974 said...

Spot on. It makes me so angry that all that time, energy and money is directed into useless studies. We learnt about such flaws in A-level Statistics (when we were 18!) so why does this junk get published?