Methinks you just answered your own question, since what you described is exactly what a 3D VR/AR system does.