Absolutely it's a big gap, and I recognize that gap. But people completely improperly extrapolate that gap and imagine that it somehow means that if you pick a man and woman off the street the man will run the 100m 1.12 seconds faster. I'm sure it's not quite that literal in their minds, but I've known average guys who had never boxed seriously who thought they would be able to take on the world champions of women's boxing because men are stronger. I read a biography of the second best women's pool player (from some year) who made a living playing pool for money against men. She didn't have to set up a con, she could just tell them she came in second that year and they'd still think they could win - often they'd want to play again to try to get their money back.
People think that the graphs look like the top ones, when in reality they look like the second ones, or, for all we know, like the third ones. But if you only look at the very top end, you can't even tell the second and the third apart (I mean, obviously you can the way I actually drew them with my off hand (my doctor says it will help me avoid tendonitis (not that I'd draw them accurately with my right hand, but still (I know I could find images of normal curves pretty easily online, but Paint doesn't have any features that would let me overlay them easily without the whitespace covering one of them)))).
I think (emphasis on I think) men tend to beat women in the upper tiers of most human-made metrics because men are more obsessive than women, more than any other trait they have. Or, at the very least, obsessiveness is distributed like graph 3 above. It just seems like if I make up a metric for scoring something, somewhere in the world you'll find a man who is willing to give up his whole life to just maximize that one number.