Confession time: I'm a NYC kid. I'm not racist, as far as I can tell (I'm not going to cite evidence, which would sound defensive)... but I do have to admit to a classist streak that catches up with my when I'm not paying attention, and my youth conditioned me to hearing inner-city black dialect as an underclass dialect. Combine that with an awareness that I don't feel I know enough about black subculture to read social cues reliably, and I don't interact as easily with strangers when they're black as when they're white.
That caught me out particularly badly a few years ago when I was in Louisville and was running on automatic at the end of an extremely long day. Northeastern inner-city dialect is essentially a southern dialect, so I didn't have that cue to help me break the bad starting assumption. I did own up to it and apologize. (The apology was not accepted, which I consider fair, but at least they knew that I knew that I'd been an ass and that I was trying to stop being one.)
So... Yeah, my gut reaction is that the hoodie does make folks look suspicious. But (a) I'm fighting that, and (b) I think I'd find that the hoodie, in weather which didn't call for a hood, made a young white guy look suspicious too. It's just become too stereotypically gangsta.