Sometimes you even run into sites that forbid common non-alphanumerics from the EN_US keyboard layout. Spaces, forward and backslashes, similar seemingly innocuous punctuation.
Aside from being annoying(probably far more so for people using character sets that aren’t basically 21st century ASCII), those sorts of rules always make me nervous because they suggest that the path your password takes in being passed around and munged on the site’s systems may be longer and more decrepit than you would like. Especially when the forbidden characters are escape sequences in some common language or ones that interfere with less-than-robust string handling tools.
You want your password to be a nice salted hash and nothing else as soon as possible; so the more rules about what you cannot do(as opposed to complexity requirements), the stronger the temptation to suspect that your password is being passed around in plaintext, possibly even retained permanently in plaintext, used to authenticate against some legacy backend that Doesn’t Do unicode, etc.