Now, all that said:
- They're not that common.
I don't know where the hell you're from, but many of the places I've been are thick with them. Arkansas is overrun, and I say that as someone whose parents are amateur entomologists. I've also come across many in Northern California, even in my office on the UC Berkeley campus. I found a pair of gigantic ones in a cabin on the Russian River (they were nearly the size of my, rather large, hand): took me a while to ID them, though, as they were "desert recluses", and lacked the telltale "fiddle".
They like warm houses, they love cardboard (for some reason), and they like warm, tight snuggly places, like underneath bedsheets (where I've found them on occasion in my parents house). They're not aggressive (whatever that means for a spider), but they certainly put themselves in the way of humans with considerably frequency.
The reason you haven't been bitten by a brown recluse is generally that what you have is a painful, itch spot, versus a gaping, rotting wound.