Coincidentally, I just finished reading this:
A good example of how easy it is to ‘prove’ anything in psychology, even if it looks like you’ve got a well designed study.
The headline on the article is BS of course, because of psychology’s tenuous link to the natural sciences; replace ‘Science’ with ‘Psychology’ and you’re closer to the truth.
I’m not saying psychology is a worthless discipline (and the same goes to similar fields, like economics, or sociology), it’s just that they’re not in the same category of epistemological tools as the natural sciences, and their results need to be evaluated with a lot more caution as a result.
Also, I don’t think the natural sciences are totally immune to some of the problems we’re talking about here, but they’re usually a lot easier to spot, and far less common. And there are other disciplines that are somewhere between the two, like medical science, or ecology, for example.
None of this should be at all surprising of course, as the complexity of the systems we’re attempting to model goes up, it gets harder and harder to produce good results (the same is true of course in physics as well, look at how hard it’s been to accomplish anything in theoretical physics in the last 50 years, but at least there nobody is fooling themselves).