If we are comparing apples to apples:
Nearly everything brought aboard the Apollo missions were essential to completing said mission.
Plastic bottles on earth are largely non-essential.
The Apollo missions and exploration probes furthered both our exploration of our solar system and provided invaluable scientific information.
Plastic bottles were put there by careless beach goers.
Removing said items from the Moon would take billions to get to the moon and back, and billions to go down to the moon and back. It would be next to impossible to retrieve stuff from say Venus due to the hellish conditions. Mars retrieval would be in the billions.
Picking up a couple bottles and putting them in the trash is probably like 50 calories of energy and a few minutes of time.
If we went back to the moon and founded permanent bases, the original Apollo landing zones would either be roped off as historical landmarks, or moved to a museum. If we managed to go to Mars, the old probes there would likewise be elevated to historical artifacts.
Unless you buried the plastic bottles and dug them up in 1000 years, they are just trash to be recycled.
I know you hate me, but this is an odd hill to defend.