That is a legitimate challenge. My understanding of the term “disabled” at least when it comes to accessibility with anything other than parking spaces, is that it somewhat depends on the individual to determine if they are disabled in a given context. My mother’s diabetic neuropathy would be something that I would want to count as a disability with regard trail access but I wouldn’t want to count my hearing impairment.
As for, “lack of endurance”, if you look at this thread, it is specifically this issue that I’m kind of interested in. If the 24-year-old ultramarathoner is able to climb up to see the petraglyphs does that mean that special accommodations should be made such that the 50-year-old 50-lbs-overweight but otherwise healthy guy should also be able to climb up to see the petraglyphs? Is the policy that they don’t develop the trails near them ableist?