I’m in Kennebunkport Maine- We still have a small Secret Service contingent here for Bush I, and he’s been out of office for almost 25 years now.
The government basically built a small guest house on the Walker’s Point compound, and someone told me they rent the land that it’s on for some token value like $1/year or something (I can’t verify this). There’s maybe a half dozen agents total who live there at least part time.
If you go to the turnoff from Ocean Ave and take pictures, they’ll come out and ask you questions.