I have found the same; the Vibram five-fingers are great for hiking.
However, terrible for climbing–something I discovered after starting to climb using vibrams, and then trying out real climbing shoes. (Vibram lost a class action lawsuit because they claimed they were good for climbing–telling that they claim they are good for everything under the sun. And now Golf!)
I will say, the pair of Merrill minimalist shoes that are 0 padding (sole made by Vibram, but no toes) are GREAT for climbing. Flexible arch, but keeps the toes together so you can really work your big toe into crevices. I found them less impressive for running, due to my aforementioned friction issue.
