I would define water that tastes of plastic as “bad”. It might not be harmful, but it is certainly outside of it’s best use-by-date and by that definition would be expired. For something that’s as simple to rotate as water, it’s splitting hairs. Another thing to consider is that bottled water containers are getting thinner and less resilient in an attempt to reduce waste. The longer you keep them, the more likely they are to break down. Particularly if they are exposed to any UV from windows or other sources.
Yes, They’re probably OK, but would you really want to keep bottled water for 5 years when you can just rotate your stock?