You are forgetting that if it crashes- it will probably crash into something…
Small field-deployable drone loses control and crashes into neighborhood- someone’s living room might be taken out with hopefully none or minimal human loss.
Full-size military jet loses control and crashes into neighborhood- a block or more could be lost, depending on how much fuel it has onboard.
Yes- in either case- the pilot would be seated perfectly safe in a comfy chair, probably swearing.