That is true of literally everything. You might not respond well to caffeine, alcohol, pineapples, bread, or chocolate either. On the whole though? It really and truly is a good thing for the vast majority of people. I am sure it happens, but I have never met anyone who had a "bad trip" when they were in a safe and comfortable place with friends they trust. Like any trip, you need to prepare. Your more traditional trip is also going to suck if you arrive to the airport late, don't pack clothing, go with people who don't like or trust, and book it for Mogadishu.
I'm not saying that even someone who is playing it smart can't have a bad time, just that the occurrence of it is radically over hyped in the insane "war on drugs". Like in basically all discussions of drugs, the misinformation level is extreme.