Refusing to associate with you doesn’t actually “harm” you. Killing people actually harms them. It’s a pretty easy distinction, to be honest.
Because you believe in the inherent goodness of the state and it’s ability to force people to change who they are to a model that you find attractive. Whether you think of it in those terms or not, that’s what the existing legal model does. It says “you think badly, and the government is going to use force of law to change you so you do not act in a way that is consistent with how you think.”
That sort of thing is the most immoral thing (to me) that a state could do, forcing people to act against their own will. But that’s because I’m an opponent of the tyranny of the majority and you are (apparently) a proponent of it.