It is already happening! In larger German cities, so-called “unpacked-shops” (Unverpackt-Läden) have opened in the recent years. They offer all sorts of food non-packaged or in bulk, some also other consumer goods like detergent, shampoo etc. You bring your own containers (jars, bottles, cans etc.) and weigh them first so you know the weight of the container, writing the unladen weight onto the container. Then you fill it with the good, and at checkout the cashier weighs the filled container, substracts the unladen weight and thus calculates the price.
Of course, these shops are still a rarity, prices are generally on the fairtrade / organic level (since most shops try to offer fairtrade and organic goods) and they are mainly an offer for conscious consumers, but they are spreading. (Dried herbs or spices are actually often cheaper than in normal shops.) And in reaction, some of the large chains have begun to sell some goods in bulk, too, and are cutting down on packaging, e.g. cucumbers that each were once wrapped in plastic now usually get only a sticker with price and details.