All of which would have to be cut and moved by hand. That not only takes time and effort, but money, and as the show made very clear more than once, the crown is deeply in debt. And, if they had gone to the trouble to fortify that particular spot, Stannis would have simply found the next weakest spot to target, as he knew King’s Landing very well.
And you’re looking at the tunnel in the wall thing the wrong way. If there were no tunnel, then Mance would have to cut his own, and they wouldn’t know which direction he was coming from; since there’s only one way through, they can defend that spot. (Mance could have chosen to cut his own tunnel anyway, but there could be traps hidden in the ice, like the scythe-on-a-chain in this episode; plus, that would have taken additional time, and not only is the top of the Wall patrolled, but he’d have to keep his people together for that much longer, and discipline is a problem for a group that barely sticks together on the best of days. That’s also probably why they simply didn’t go around either end of the Wall.)
The secret tunnel under the Wall goes into the Night Fort; it’s a secret (Sam had read about it in one of the books in Castle Black), plus the Night Fort has the reputation of being haunted.