Every point in the current universe was at the Big Bang. The expansion of the universe is like the expansion of the surface of a balloon, but that common analogy is slightly misleading. There is no inside to the universe like there is with a balloon, and the expansion isn’t driven by internal pressure, but by an as yet little understood expansive force (proposed candidates include but aren’t limited to the cosmological constant, quintessence, or dark energy) that emerges from the the vacuum itself.
This force - which we call a scalar force because it scales with distance but exerts uniformly in all directions (meaning it has no vector the way a vector force like gravity does) - is very weak, but it’s strength is cumulative in that the further apart two galaxies are, the more space and therefore the more expansive force there is between them, which is why more distant galaxies accelerate away from us faster than nearer cosmic neighbors. That is to say that it’s not a force mediated by waves and particles the way electromagnetism, the weak nuclear, the strong nuclear, and (probably) gravity are; rather it’s a pressure created by the expansion of space itself. If it has a minimum quanta, it’s not a particle which moves through space, because it’s the quanta of space. Quantum mechanics says that there is a smallest unit of space which has physical meaning. You can think of the “fabric” of space, and thus the expansive force as the constant increase of these units.
So the Big Bang wasn’t in a particular place in the universe, it is the entire universe…even the regions that are already so far away from us (and us from them) that light-speed signals starting out now from here or there will never meet. Because although nothing can move through space faster than the speed of light, the expansion of space itself has no such limit. We say such distant regions are beyond our cosmological event horizon (and we theirs), because events from them are cut off from us, and vice versa. At one point they were in the same point as us, because the entire universe was only one point (the singularity at the beginning of our spacetime), but the expansion has taken them beyond our horizon, and more matter gets pushed beyond it by the expansion of space all the time. Eventually, in our universe’s Deep Time future, the expansion will (we think) isolate every wave-particle from every other as everything decays and spreads out. Cosmologists call this the Big Rip (I’m not making that I up, I promise).
You might be wondering about theories like Braneworld Cosmology which hypothesize that our universe is a 3 dimensional “sheet” embedded in a higher dimensional sheaf of universes called the Bulk. You may understandably wonder if there’s a preferred axis within the Bulk. While there may in fact be, this still does not give any special axis to our 3D universe, because that hypothetical fourth spatial dimension would be equally perpendicular to all three spatial dimensions, just as the third spatial dimension is equally perpendicular to both the first and second spatial dimensions. If you set a ream of paper on top of a sheet, it doesn’t give the two dimensions of the sheet any preferred direction or special axis.
There is only one sense in which our universe has a preferred direction, and that is the Arrow of Time. Everything moves away from the Big Bang through time. Since the further out we look with telescopes, the longer the light we see took to reach us at the constant speed of light, we are actually looking at the distant universe in the past. The Cosmic Microwave Background radiation is the red-shifted (by that expansion of space) light from the time a mere 400,000 years after the Big Bang when the plasma in the then much smaller universe cooled enough to combine into atoms which released the light. The distribution of that plasma, and thus the light it emitted when it’s electrons and protons coupled for the first time into atoms, was determined at the moment right after the Big Bang itself.
TL;DR - The Big Bang was everywhere, so there is no away from it in space. No preferred spatial direction. Only preferred direction is the Arrow of Time (past to future). Hypothetical higher dimensions are by definition perpendicular to all three observable space dimensions and thus do not give a special axis to any of them.
