I was thinking the same, i’d rather go to places that aren’t tourist traps. I don’t mind crowds but i certainly dislike it when everything is presented to entice tourists rather than have locations and businesses be naturally interesting and good. The one and only time i visited Japan i think the time i had the least fun was when i was in Tokyo, not that i hated it but i found other places much more fascinating.
The one big thing that i did that was pretty fun was seeing the Yokosuka Baystars play 
i’m not into sports normally but i had a blast. Besides that most of my time was spent hiking by temples and exploring side streets and little businesses
