Continuing the discussion from The Universe Believes in Encryption:
It would be just as reasonable for someone who knows and has heard only English to say that the world around them is built in English, because that is how we describe it.
The universe, existence, is discrete from language. Math is simply how we describe it, and not an innate component.