The next generation virtual reality systems will be far more sophisticated in terms of speed, resolution, and image quality. However, the progress will be not only dependent on technology improvements but will rely on our knowledge of the human visual system. No longer can we assume the straightforward progression of previous algorithmic techniques perfected for two-dimensional film and storytelling used for today’s theater entertainment. New algorithms will need to consider our brain’s interpretation of color, space, and the dimensions of light.