I suspect that it’s largely a matter of scale: the majority of human history (and to this day, most of our actually reasonably close social interaction, rather than market relations and the like) involved fairly small groups, often with a fair slice of family.
I’d venture that, in the case of all but the most cold-bloodedly consistent or just-plain-scary, even active campaigners against the parasitic cripples and inferiors would behave differently if the case involved a family member, even an extended one, or the like.
