Thursday, January 29, 2015

NPR buried the lead: beware of published scientific papers without ANY opposing data

Monday, January 26 NPR's Morning Edition ran a story on "Bilingual Studies Reveal Flaw In How Info Reaches Mainstream."

Sure, the key point was in the headline but if you weren't careful you'd think the focus is on Bilingual Studies.  They increase cognitive function, right?  Something like what?  Whatever - moving on.

It's the 2nd part of the headline that's the real story - Flaw in how info reaches mainstream.

That is, some scientific studies never get published or promoted at conferences at all.  And some of the ones that disappear are because they contradict the glamorous, the new, the feel-good studies.

And NPR's example was the Bilingual studies.  The theory is that the more languages you know, the stronger your brain is.
NPR's scientific guy, Shankar Vedantam:  "Speaking multiple languages actually has benefits to your brain.  And it's based on the fact there have been dozens of studies that show when you learn multiple languages, you have better executive control."

After some back and forth explaining executive control, Shankar continues:
"Here's the problem with that construction.  In 2009, researchers at the University of Edinburgh published a study showing that bilingual speakers could resist distraction better than monolingual speakers.  That totally fits with the theory.

But I recently spoke with Angela de Bruin.  She's recently revisited the subject based on her personal knowledge of what actually happened.  It turns out the researcheres in that 2009 paper actually conducted FOUR EXPERIMENTS.  THREE of the FOUR did NOT show that bilinguals were less distractible than monolingual speakers."

Angela de Bruin:  "One of the tasks (aka experiments) showed an effect of bilingualism.  And that's the study that was published.  Now, the three other tasks did not show any effect of bilingualism at all.  And those tasks never made it into a publication.  They were just put into a file drawer.  So we had 4 studies, but only 1 was published.  And that was the successful one, show an effect bilingualism."

Shankar:  "In fact, one of the authors of the 2009 paper tried to replicate the successful experiment. And that replication failed to work.  So in other words, there were 4 experiments.  3 did not show benefits, and weren't published.  1 showed a benefit and was published.  But it couldn't be reproduced.  And then the reproduction was not published."

de Bruin:  "We found 63% of abstracts supporting a bilingual advantage were published compared to only 36% of the challenging studies."

NPR's David Greene:  "You know, Shankar, just listening to that, it makes me less likely to trust a scientific study."

Shankar:  "I think that's exactly right, David.  There's been a lot of concern in science about this kind of of publication bias affecting the integrity of science."

NPR never alludes to any other studies that we may want to rethink, such as Global Warming-Climate Change-back to Global Warming.  See also:  AGW (anthropogenic global warming).

Back in 2010 the American Thinker reported on Climategate, the hacking and release of materials from IPCC.  IPCC rejected the claims their theory was wrong because it wasn't peer reviewed.  Well, 3 peer-reviewed journal articles contradict the AGW hypothesis.

And there's that gem from 2000:  the UK Independent reported that snow would be a thing of the past.

Then the UK Telegraph reported 24 January 2015 Climategate, the Sequel: How we are STILL being  tricked with flawed data on global warming.  It's a great read.

No comments: