Perhaps this holds true for fiction? But I have a massive non-fiction library, and most of those books I keep around for reference. That means they often have post-its sticking out of them, highlights on the pages, notes in the margins, and I want to be able to pick them up and flip through them. During writing sessions you'll often find me surrounded by stacks of non-fiction books related to whatever I'm writing about.
Ereaders do attempt to offer an answer to this, with highlights and notes and such, but I don't find it satisfactory to my needs (your mileage may vary). Which isn't to say I don't read non-fiction ebooks; I do. If they're worth keeping around to reference, I go out and find a paperback copy to go with the ebook (assuming one exists).
Eventually I may pare down my fiction library, removing books I have no interest in rereading or keeping for sentimental reasons, but I can't see myself ever giving up my physical books in great quantities, just for the reference reason alone (not even getting into the problems related to not owning ebooks which is also a deterrent to eschewing paper for an epub or mobi file). While I find myself reading more on my ereader these days for the convenience of it, it will never replace my paper books.
Also, on a personal note, I feel safe surrounded by books, and if my house were empty of them I would never feel at home. Even if moving 30+ boxes of them is a lot of work. It's worth it to me.