I get the worry, I really do, but... how? The way you carry a sword is in a sheath. How would you attach a naked blade to your belt? No sword I've ever seen had anything you could loop a belt through. I mean I guess you could thread a belt or a piece of string through a swept-hilt rapier but that's never staying on. I guess you could hold it in your hand unsheated and walk around, but I'm pretty sure that's illegal from a 'endangering public safety' standpoint in much the same way that, even if open carry is legal, you can't go around pointing that gun at people as your method of carrying it.
Also, a happy coincidence is that most swords bought by idiots are stainless steel replica swords which are remarkably useless in a fight. I won't say they can't hurt someone, but given a choice between a slab of stainless steel shaped like some spiky fantasy nonsense or a crowbar in a fight, I'd go for the crowbar every time.
I mean I'd lose because I have the coordination of a bag of marshmallows, but I'd be better armed all the same.
Now knives are a different matter.