Hah! We just did that earlier this month with Althea, using a lime.
Seriously, people, if you've not looked up the genre of "Baby's First Lemon Tasting" on YouTube, you must. http://youtu.be/CumIj-NH5RA
Hilariously, my daughter kept taking a bite, making a horrible face, and then, almost immediately, putting the lime back in her mouth again. She would have done this essentially forever, apparently. I think it says something about the human condition.