I think you really should worry, because if it is too easy to fire people for trivial offenses then black people will be targeted as well. Doesn’t matter if you think the reasons are stupid. Who do you really have most to benefit from laws and traditions that make it harder to fire employees?