I doubt that this is actually the reason behind our concern, or lack there of; given how it often seems to be more of the ‘instinct that must have taken ages to hone’ rather than ‘inferrence from recent history’; but in practice humans have much more to fear from the little things than the big ones.
Pretty much anything large enough to either shoot or habitat-destroy now exists either because it’s tasty, because it’s docile, because it’s docile and tasty; or because we feel a bit bad about the prospect of finishing the slaughter.
The vermin, bugs, parasites, and similar? That’s a different story entirely.