You find this is more true with summer olympics than winter ones. Winter olympics tend to be in remote locations to begin with (Sochi was an exception) so you don’t usually see the massive population displacement for construction involved. Also they tend to be done in areas which are known for winter sports. The most popular sports of those olympics also requires the least real infrastructure outlay, a skating and or hockey rink.
