Some seriously bad science
Jun. 24th, 2011 08:04 am![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
snousleAl-Jazeera reported a claim that infant mortality in the Pacific Northwest increased 35% following the Fukushima incident. A Scientific American blog does a good takedown of the claim, which is entirely bogus. Another critique is here.
It's a really fine example of how selecting data to achieve "statistical significance" results in entirely bogus results. It is, unfortunately, an easily abused criterion that can be used to make superficially compelling arguments through passive omission of contradictory data. I particularly liked one comment on the blog posting: They could have chosen all cities on the West Coast for which data was available, and they could have looked at data for a solid year. They didn't. 'Nuff said.
That's correct; there was no justification for choosing some cities over others, no justification for choosing the particular time window they did, and no justification for not increasing the statistical power of the study by using all available data. Even at the best of times, the claim of "statistical significance" is a very weak one, and is not intended to mean that the claim is "true", just that it isn't obviously due to chance. The cutoff of 0.05 which has become the standard for publication is IMHO far too weak, and in cases that are at all controversial it should be lowered to 0.001 or less.
This is also a good example of the difference between "actual fake science" and "fake fake science". When a scientific claim is genuinely junk, it gets jumped on by multiple critics who provide numbers and specific sources to discredit a claim. But when good science is unfairly critiqued, there's usually very little in the way of numbers and sources, and arguments about the data are replaced by innuendo and conspiracy theories. When someone has to make claims about the motivations of the researchers to advance their case, you can be pretty sure it's because they don't have data-centered arguments to make in the first place.
Here, happily, the data speaks for itself.
It's a really fine example of how selecting data to achieve "statistical significance" results in entirely bogus results. It is, unfortunately, an easily abused criterion that can be used to make superficially compelling arguments through passive omission of contradictory data. I particularly liked one comment on the blog posting: They could have chosen all cities on the West Coast for which data was available, and they could have looked at data for a solid year. They didn't. 'Nuff said.
That's correct; there was no justification for choosing some cities over others, no justification for choosing the particular time window they did, and no justification for not increasing the statistical power of the study by using all available data. Even at the best of times, the claim of "statistical significance" is a very weak one, and is not intended to mean that the claim is "true", just that it isn't obviously due to chance. The cutoff of 0.05 which has become the standard for publication is IMHO far too weak, and in cases that are at all controversial it should be lowered to 0.001 or less.
This is also a good example of the difference between "actual fake science" and "fake fake science". When a scientific claim is genuinely junk, it gets jumped on by multiple critics who provide numbers and specific sources to discredit a claim. But when good science is unfairly critiqued, there's usually very little in the way of numbers and sources, and arguments about the data are replaced by innuendo and conspiracy theories. When someone has to make claims about the motivations of the researchers to advance their case, you can be pretty sure it's because they don't have data-centered arguments to make in the first place.
Here, happily, the data speaks for itself.
![[identity profile]](https://www.dreamwidth.org/img/silk/identity/openid.png){kind=small}
no subject
Date: 2011-06-24 04:56 pm (UTC)no subject
Date: 2011-06-24 05:28 pm (UTC)Arguments without statistics don't even have the opportunity to be wrong, because they contain nothing to be wrong about. I wish I could remember that Italian expression - meaning "he who does no work makes no mistakes?"
no subject
Date: 2011-06-24 11:49 pm (UTC)no subject
Date: 2011-06-25 09:31 am (UTC)It's what they aren't reporting: A 35% increase in the recruitment rate of Taliban suicide bombers.