The point @Wanderfound made still is accurate. The majority of their crimes were under the cover of national legality. Law is relative to the society it’s practiced in. That doesn’t make it any less of a crime against humanity. But all they did was legal under German law, no matter how the rest of the world (who, by the way, didn’t all feel it was bad or unlawful, even today) felt about it.
To bring the example back to the US, Jim Crow was fully legal, because it was the law. The law is not a system of morality, but a system of social control, for good or ill.