> If it was a huge deal, wouldn't epidemiologists be slobbering over the statistical power they would get from using it?
Why would you want to do that? You can build a nice career on publishing (spurious) associations. As long as relevant variables are omitted, no epidemiologist need ever be unemployed.
That may get you written about in popular-science magazines, but your work will have a low impact factor- this is largely measured in how many of your fellow researchers cite your work, and its crucial to an academic researcher's career.
If your colleagues can easily see that your statistical analysis is flawed, they won't waste their own time doing work that builds off of yours.
It is not necessarily true that just because they are called scientists, real people with real motives, funded by organizations of real people with real motives, operating in a political regime controlled by real people with real motives, will act that way. (By "that way" I mean "your work will have a low impact factor- this is largely measured in how many of your fellow researchers cite your work" and "If your colleagues can easily see that your statistical analysis is flawed, they won't waste their own time doing work that builds off of yours.")
See e.g. Chan and Boliver "The Grandparents Effect in Social Mobility", http://asr.sagepub.com/content/78/4/662 (HT http://www.arnoldkling.com/blog/a-grandparent-effect/) and many of the works that it cites. There seems to be a sizable mutually-citing "scientific" literature devoted to studying correlation in biological descent while dogmatically not controlling for ordinary DNA-based inheritance of psychological traits from parent to child. (Indeed, not even controlling for DNA-based inheritance of physical traits. Health and height and attractiveness are correlated with income, and are significantly physically heritable.) By not controlling for the properties which are transmitted by sperm meeting egg, your can find all sorts of fascinating possibilities to investigate to explain the correlation. (E.g. Chan and Boliver carefully note that "well-connected grandparents could also use their social contacts to help grandchildren with job searches." Good, good, carefully investigate all the possibilities.)
> but your work will have a low impact factor- this is largely measured in how many of your fellow researchers cite your work, and its crucial to an academic researcher's career.
Nope! This is how we might like to to work, but cool associations will get written up forever, long after they have been refuted by randomized experiments. (Heck, papers can outright be retracted and still get cites.)
Hahaha - did I say refuted by randomized experiments? Like that ever happens to more than a tiny fraction of epidemiology correlations... No, one's career is quite safe. No one ever lost tenure because the correlations they built their career on turned out to be lame.
> If your colleagues can easily see that your statistical analysis is flawed, they won't waste their own time doing work that builds off of yours.
No, the point here is that your work can be immaculate and still never reflect causality. How are you going to collect data on every lurking variable? You can't, of course.
I'm optimistic enough to believe in a few wild cards trying to do good science. A whole bunch of the rest have to put a little bit of effort into competing with them.
It occurred to me a while after my other comment that a belief in the ability of people that test as health conscious to actually do things that improve their health might not be especially justified. I mean, lots of people are quite happy with whatever quackery, maybe it really does all come out in the wash.
In his article "Science pseudoscience nutritional epidemiology and meat" http://garytaubes.com/2012/03/science-pseudoscience-nutritio... Gary Taubes discusses how epidemiological studies can fail to account for something called the "compliance effect" - the missing variable.
He quotes a textbook chapter in his conclusion. So the idea that you have to at least be careful with your statistics is orthodoxy, not some missing component of the field.
(I realize that doesn't make any guarantees about common practices, but it is maybe a little capricious to repudiate the entire field)
That is, if they can control for it, they can make assertions about smaller effects.