Are your walls not painted? We have plaster walls and I’ve not had this problem using those things before.
Also a tip for hanging stuff in a more permanent manner with small nails in plaster walls -
When you have the spot(s) marked for the nails, cover with two small pieces of blue painter’s tape (on top of each other) to keep the plaster from cracking when hammering the nail in.
