When I saw on Twitter that a ‘major new peer-reviewed study’ was about to reveal serious health impacts from GMO corn and soya, I was intrigued to say the least. Would this be Seralini 2.0, a propaganda effort by anti-biotech campaigners masquerading as proper science, or something truly new and ground-breaking?
Extraordinary claims require extraordinary evidence – and it would take a lot of extraordinary evidence to confound the hundreds of studies showing that GMO foods are just as safe as conventional, as summarised in this recent AAAS statement:
“The science is quite clear: crop improvement by the modern molecular techniques of biotechnology is safe. The World Health Organization, the American Medical Association, the U.S. National Academy of Sciences, the British Royal Society, and every other respected organization that has examined the evidence has come to the same conclusion: consuming foods containing ingredients derived from GM crops is no riskier than consuming the same foods containing ingredients from crop plants modified by conventional plant improvement techniques.”
So when I found the paper, again via Twitter, I determined to read it as I would a climate ‘denier’ paper which aimed to overturn the scientific consensus in that area – with an open mind, but a sceptical one. I could see that it was already generating news, and the anti-GMO crowd on Twitter were also getting excited about some new grist to their ideological mill. Here’s what Reuters wrote:
“Pigs fed a diet of only genetically modified grain showed markedly higher stomach inflammation than pigs who dined on conventional feed, according to a new study by a team of Australian scientists and U.S. researchers.”
Really? Time to have a look at the study. It is by a Judy Carman and colleagues, entitled ‘A long-term toxicology study on pigs fed a combined genetically modified (GM) soy and GM maize diet’ and published in a minor Australian journal I have never heard of called ‘Journal of Organic Systems’. (This journal does not appear in PubMed, suggesting it is not taken very seriously in the scientific community. It only publishes about twice a year, mostly with research touting the benefits of organic agriculture.)
I skimmed the paper first, and the conclusions seemed doubtful enough (see below) to try to find out who was beind it. So I looked at the sponsors of this journal. They include the Organic Federation of Australia, which seemed odd for a journal presumably aiming to be independent. Imagine the hullaballoo if Nature Biotechnology was sponsored by Monsanto!
I also wondered who Judy Carman and her colleagues were. She turns out to be – as I feared – a long-time anti-biotech campaigner, with a website called ‘GMOJudyCarman‘, which says it is supported by GMOSeralini.org. She is a founding member, according to this website, of the scientific advisory council of the Sustainable Food Trust. The Sustainable Food Trust was the UK outfit set up by former organic lobbyist Patrick Holden which stage-managed the media release of the infamous Seralini GMO rats study to the Daily Mail and other credulous outlets.
What about the co-authors? One is a Howard Vlieger, who seems to have made some wild allegations about GMOs in the past if this source is to be believed. Vlieger is president and co-founder of Verity Farms, a US ‘natural foods’ outfit which markets non-GMO grain. Despite this, the paper declares that the authors have no conflicts of interest, although it seems to me that he would have a very clear commercial interest in scaring people about GMOs in order to drum up business of his GMO-free offerings.
What about funding? The paper states that funding came from Verity Farms, the natural product outfit mentioned above. Carman and her colleagues are also funded by and associated with the Institute of Health and Environmental Research, an Australian not-for-profit which seems to be entirely dedicated to anti-GMO activism. Recent activities have included opposing Bt brinjal in India and CSIRO’s GMO wheat in Australia. Funding sources are not disclosed, although donations are solicited. The paper’s acknowledgements are a veritable who’s who of anti-biotech activism, includin Jeffrey Smith, John Fagan and Arpad Pusztai.
So, that’s the context. Now let’s look at what raised my suspicions about the actual study. Well, Carman and colleagues claim significant differences in a long-term study of pigs fed GMO and non-GMO diets. But if you look at the data they present (and the data presentation is at least a step better than Seralini) there are obvious problems. Clearly all the animals were in very poor health – weaner mortality is reported as 13% and 14% in GM-fed and non-GM fed groups, which they claim is “within expected rates for US commercial piggeries”, a vague statement intended to justify what seem to have been inadequate husbandry standards.
This picture is even more stark in the data presented in Table 3. 15% of non-GM fed pigs had heart abnormalities, while only 6% of GM-fed pigs did so. Similarly, twice as many non-GM pigs as GM ones had liver problems. Why no headlines here? “Pigs fed non-GMO feed 100% more likely to develop heart and liver problems, study finds” – I can just see it in the Daily Mail. But of course negative results were not what Carman et al were looking for.
So we fast-forward to the stomach inflammations. This is where Carman et al got their headline. As Reuters reported:
“But those pigs that ate the GM diet had a higher rate of severe stomach inflammation – 32 percent of GM-fed pigs compared to 12 percent of non-GM-fed pigs. The inflammation was worse in GM-fed males compared to non-GM fed males by a factor of 4.0, and GM-fed females compared to non-GM-fed females by a factor of 2.2.”
This is statistical fishing of the most egregious sort, and I would put money on the Reuters summary above being lifted near-verbatim from a press release written by Carman et al. Table 3 actually shows that many more pigs fed non-GMO feed had stomach inflammations than those with GMO feed. So 31 non-GM pigs had ‘mild’ inflammation, while only 23 GM pigs had it. For ‘moderate’ inflammation, a GMO diet again seemed to be beneficial: 29 non-GM pigs had moderate inflammation of the stomach, while 18 had it. So that’s 40% vs 25%. Do Carman et al perform a test for statistical significance to see if GMO feed has a protective effect on pigs stomachs? Of course not – that’s not the result they are after. These findings are ignored.
Instead, it is the next line of data that they play up: for ‘severe’ inflammation 9 non-GM pigs were determined to have it, while 23 GM-fed pigs had it. Shock, horror. You can immediately see how the data is all over the place from the previous results, which also rule out any causal mechanism with GMO feed – if GMO feed is causing the severe inflammation, why is the non-GMO feed causing far more mild to moderate inflammation? It’s clearly just chance, and all the pigs are not doing well and suffering stomach problems: about 60% of both sets had stomach erosion.
Yet the paper slyly presents photographs of inflamed pigs stomachs, with non-inflamed and mildly inflamed from non-GM fed pigs, and moderate and severe inflammation presented from GM-fed pigs. Yet 38 of the non-GM pigs, more than half of the total of 73, were suffering moderate or severe inflammation – why not present photos of their stomachs? This is rather reminiscent of how Seralini presented shocking pictures of GM-fed rats with massive cancerous tumours, but did not present pictures of the control rats (non-GM fed) which also developed cancers.
Indeed, if you add together the ‘moderate’ and ‘extreme’ categories – which from the photos are not easy to tell apart, involving a value judgement on the part of the vets employed to do the post-mortems – then the non-GM fed pigs have 38 affected individuals (52% of the animals studied), while the GM-fed pigs have 41 affected individuals (56% of the total). Statistical significance? My ass. This is propaganda dressed up as science, which is why it didn’t make a proper peer-reviewed journal. (Update: Andrew Kniss makes this point better, using an appropriate statistical technique, here. Later update: A swine health specialist at the University of Guelph in Canada concludes that the redness observed by Carman et al is not even a reliable indicator of inflammation.)
My judgement is that, as with Seralini, this study subjects animals to inhumanely poor conditions resulting in health impacts which can then be data-mined to present ‘evidence’ against GMO feeds. Most damning of all, close to 60% of both sets of pigs were suffering from pneumonia at the time of slaughter – another classic indicator of bad husbandry. Had they not been slaughtered, all these pigs might well have died quickly anyway. No conclusions can be drawn from this study, except for one – that there should be tighter controls on experiments performed on animals by anti-biotech campaigners, for the sake of animal welfare.
Thanks are due to the numerous Twitter correspondents who provided insight and links which have been useful in this post. You know who you are.
Update: I received the following expert commentaries courtesy of the UK Science Media Centre:
Prof David Spiegelhalter, Winton Professor of the Public Understanding of Risk at the University of Cambridge, said:
“The study’s conclusions don’t really stand up to statistical scrutiny. The authors focus on ‘severe’ stomach inflammation but all the other inflammation categories actually favour the GM-diet. So this selective focus is scientifically inappropriate.
“When analysed using appropriate methods, the stomach inflammation data does not show a statistically statistical association with diet. There are also 19 other reported statistical tests, which means we would expect one significant association just by chance: and so the apparent difference in uterus weight is likely to be a false positive.”
Prof Patrick Wolfe, Professor of Statistics at University College London, said:
“I am not an expert on animal health, husbandry, toxicology etc, and therefore I cannot comment on these aspects of the study. As a statistical methodologist I can however comment on the data analysis undertaken and presented in the article.
“The biggest issue is that the study was not conducted to test any specific hypothesis. This means that the same sample (in this case nearly 150 pigs) is, in effect, being continually tested over and over for different findings.
“The statistical tests employed assume that a single test is done to test a single, pre-stated hypothesis; otherwise the significance levels stemming from the tests are just plain wrong, and can be vastly over-interpreted.
“Thus there is a higher-than-reported likelihood that the results are due purely to chance. The number of pigs being in the low hundreds (instead of, say, the thousands, as is often the case in large medical studies) can make this effect even more prominent.
“Bottom line: a better-designed study would have hypothesized a particular effect (such as changes in stomach size), and then applied a statistical test solely to check this hypothesis. Perhaps another independent team of researchers will go down this path. Until then, this study definitely does not show that GM-fed pigs are at any greater risks than non-GM fed pigs.”