Utterly Flawed Research Claims a Spurious Benefit for Mangosteen

Every now and then a piece of nutritional research crosses my desk that is so poorly conducted that I can't resist the temptation to post it here and use it as an example of how people who are supposed to be scientists degrade themselves in the pursuit of results that will please their corporate sponsors.

The study can be read in all its glorious full text awfulness here:

Evaluation of Mangosteen juice blend on biomarkers of inflammation in obese subjects: a pilot, dose finding study. Jay K. Udani. Nutrition Journal 2009, 8:48, doi:10.1186/1475-2891-8-48

The way this study is being reported in the media can be seen here:
Science Daily: Mangosteen Juice Could Protect Health In The Obese

So what's wrong with this study? Let's take a closer look.

The abstract of the study says "The purpose of this study was to evaluate the effect of multiple dosages of a proprietary Mangosteen Juice blend on indicators of inflammation and antioxidant levels in obese patients with elevated C-reactive protein (CRP) levels." Okay, that sounds reasonable.

So let's look at the methodology. Forty-four people were given tests of inflammatory markers like CRP. These were "screened" out of an original population of 122 people, though the researchers don't tell us what they were screened for.

These people were broken into four groups. Three of the groups were told to drink a branded mangosteen juice product twice a day, each group drinking it in different sized doses. The fourth group were given a placebo to drink, though we are not told what size dose they were given. So far this sounds reasonable.

But when you red the full text description of what the subjects were actually given to drink you see at once that the Science News report title is flawed. Because the researchers didn't really give any of their subjects mangosteen juice. Instead they gave them a branded "mangosteen juice blend" where only the "primary ingredient" was mangosteen juice.

And they had no idea what percentage of the juice blend was mangosteen, because the branded juice blend also contained:

apple fruit juice, pear fruit juice, grape fruit juice, pear fruit puree, blueberry fruit juice, raspberry fruit juice, strawberry fruit juice, cranberry fruit juice, and cherry fruit juice.

Now the way that the FDA defines a "primary ingredient" all something has to do to be listed first on the label is to make up a greater percentage of the total of the food in question than the ingredients that follow it.

So since this product contains 9 other juices or purees, the mangosteen might easily make up only 11% of the juice, while each of the other ingredients made up 9 or 10% of the product.

So whatever benefits the study found for the blend, it tells us little about the effects of mangosteen juice, because the substance being tested is probably 90% other juices, several of which--blueberry, raspberry, and cranberry juice have already been found to have very slight health effects (though probably the research that "prove" those benefits isn't much stronger than this research.)

Now let's look at the placebo: Here's how it is described.

The placebo consisted of water, sucrose (3 g/30 ml), citric acid, red grape juice concentrate, fiber complex, grape skin, natural flavors, red #40, cloud (ester gum), whey protein isolate, sodium benzoate, xanthan gum, blue #1, and caramel color.

Is a sugar water solution with a fruit juice NOT found in the other juice mixture, whey protein, artificial colors, and preservative a true placebo when compared with a mix of fruit juices?

It is very possible that any difference attributed to the mangosteen juice blend--which more honestly should have been called a blend of "standard fruit juices and puree with a touch of mangosteen"--might only be due to the fact it didn't contain sucrose, whey protein, or the chemical additives in this bioactive "placebo."

But let's move on. The conclusions section reports,

HS-CRP measurements dropped after 8 weeks treatment compared to baseline in all 3 dose groups and increased in the placebo group. The changes from baseline were not significant but the comparison of change from baseline was significant for the 18oz group when compared to placebo (p=0.02).[emphasis mine]

Other markers of inflammation (inflammatory cytokines) and a marker for lipid peroxidation (F2 Isoprostane) did not show any significant differences when compared with placebo. There was a trend towards a decrease in BMI in the juice groups.

Either I'm missing something here, or the conclusion of this study was that there were no real, or at least statistically significant differences found between the three groups drinking the juice and the placebo juice. Statistically, the results of the study could have been the result of chance.

But because the researchers needed to find proof that their sponsor's product worked--did I mention that the study was sponsored by the maker of the branded juice--they looked for some way to turn insignificance to significance, and they found it by computing the average change from baseline and glory be, this statistic derived from a meaningless statistic turned out to achieve statistical significance in the one groups of mangosteen blend drinkers that drank the largest dose.

But wait, let's look a little more closely at the make up of the three groups of juice drinkers. There were 9 people in the group whose change from baseline rose to statistical significance, which is a very small number. In contrast, there were 11 people in the group that drank the lowest dose of juice and 12 in the group that
drank the middling dose. Only 8 people were in the control group.

This is a lot of variance. It makes me think any statistically valid comparison between the groups highly suspect, especially when what is being compared is the statistic involving both mean and standard deviation which start to lose meaning as the size of the groups compared gets smaller.

Let's have a look at that standard deviation. That's the statistic that shows us the extent to which individual values clustered around the average. The smaller it is, the closer the clustering. As it turns out, the standard deviations seen in the "change from baseline" were very large: the average was 0.90 but the standard deviation was a whopping ± 9.5 [no units given] for the placebo group. The decrease in the one group that supposedly rose to significance was an average of -1.33 with a ± 3.0 standard deviation.

Hmmmmmm......

But, though by now you might think it couldn't, it gets worse. Because lets look closer at the groups involved. How well matched were they?

As it turns out, not very. The placebo group's average age was 45. The average ages of the groups drinking the juice were 52, 33, and 50. Given that the measures of inflammation being measured are very age sensitive, comparing groups of significantly different ages appears to invalidate the idea that what we had here was anything approaching a properly designed controlled trial. The measures of "obesity" were not very well matched either.

The description here claims that BMI and body fat percentage "trended downwards." This is backed up with Table 2 which to the eye that has any experience with the way weight fluctuates over the eight week period of this study, makes it clear that the "trend" was only in the mind of the hopeful sponsor. In fact, using the selective technique used by these researchers, we could say that the Placebo "trended" towards achieving weight loss benefits. But of course, it didn't any more than the juice because this study has by now completely lost any claim to being scientifically useful.

The study abstract concludes by stating

..a proprietary mangosteen juice blend (XanGo Juice) reduced CRP levels (increased change from baseline) compared to placebo for those taking the highest dose of 18 oz. per day. Further studies with a larger population are required to confirm and further define the benefits of this juice.

Only in the full text did the researchers tell the truth when they added,

... the use of multiple comparisons in the analysis of the study data increases the risk of statistical error resulting in a false positive.

In fact, these researchers failed to tease a convincing false positive out of anything but that pathetic "change from baseline" statistic used to amplify a statistically meaningless finding into one they could pawn off on a credulous public as a significant one. One wonders how they could have the nerve to suggest that "further studies are required to confirm and further define the benefits of this juice."

Could it be that the sponsor likelihood of sponsoring (i.e. paying for) further research had anything to do with it?

I'll leave you to determine that.

But really, the gall of the people who come up with this stuff. And the further gall of the journals that publish it, knowing that most people will only look at the abstract or the news report that will undoubtedly appear in the magazines and web sites supported by supplement sellers.

You can expect to see the hucksters using this "research" to prove that mangosteen has miraculous health giving effects. And the poor obese schnooks who fork out their hard earned cash for magical mangosteen juice? Too bad for them. Especially if they have diabetes, since the carb count on this juice whatever its putative "benefits" must be astronomical since it is mostly apple fruit juice concentrate and what little mangosteen juice it contains--very little I suspect, is very high in carbs.

BOTTOM LINE: The proprietary Mangosteen juice blend sold all over the internet will lighten only your wallet and improve the inflammation only of the egos of the dead enders who did this research.