There is an argument that the “hard” sciences (Physics, chemistry) are actually the soft sciences because a great deal has been found out with (relatively) little effort. The really hard sciences are psychology and sociology, where it sometimes seems that there has been almost no progress in the last fifty years.
I think there are two reasons for this. First, the process of decoupling religious thinking is much harder for the social sciences than for physics and chemistry. Once Aristotle and his Christian misinterpreters had been debunked, physics and chemistry were relatively free. I would suggest that the pivotal moment was when the professor at Padua refused to look into Galileo’s telescope and it became obvious to every inquiring mind in Europe that something had changed. But social sciences are still subject to and informed by religious thinking*. It has been very hard for psychologists in some countries to accept that human minds are part of a spectrum, not a discontinuous step from the other primates and mammals; and of course the results of social science investigations into things like sexual behaviour are relentlessly attacked by people with religious axes to grind.
Second, to a large extent we are in the position of being part of what we are trying to understand. The experimenter is always part of the experiment in a way that is not true of, say, analysing a chemical reaction and its products. Interpretation is always there.
But the reproducibility crisis has been simmering for a very long time. It was an active subject when I was briefly involved in experimental psychology in the 1970s, and my then supervisor was very unpopular with a number of his colleagues because of his statistical work in debunking flawed research - especially with the drug companies, as he was being paid to review the test results for certain psychoactive drugs and was finding experimental flaws that made the results invalid. The desire not to admire that the emperor is unclothed springs only partly from academic protectionism. There are large vested interests in drug companies, prisons, drug policy and education that really don’t want accepted dogma challenged - and I don’t think it is tinfoil hattery to say so.
*edit - I include non-theistic religions like Marxism-Leninism in this. Marxism (not the economics) is after all based partly in messianic eschatological Judaism and can be considered the most recent Abrahamic religion to gain traction.
**edit edit - I don’t mean the artificial “Messianic Judaism” construct, which AFAIK wasn’t around in Marx’s day, but the actual eschatological wing of Jewish thought that awaits the coming of the Messiah and, for instance, doesn’t recognise the State of Israel because it was founded by people and not by the Messiah. @bibliophile20 has drawn my attention to what seems to be a weird US sect of that name which I’d never heard of before.
