Thanks,I get what you mean.
VR overlays are possible indeed, but I was more thinking in the direction of the billboards or like the mirror surfaces.
Instead of projecting a camera, projecting stream data.
Probably a wild and impossible idea, but would be awesome to have.