Back when cavalry was still a thing, one of the drills was to charge while the rider was crouching forward, hiding behind the horses neck, and fire his pistol, inches from the horses head. That must have involved VERY well trained horses, and even so, I would think that the chances of being thrown were far higher than the chances of hitting anything.
