The two eye views are almost the same image. How do you avoid paying for the scene twice?

Short answer

Only the view and projection matrices differ between eyes, so scene traversal, culling, animation and shadow maps can be done once, and multiview rendering lets one draw call emit both views with a per-view matrix chosen in the vertex shader.

Keep stereo rendering explicit in the answer because that is the concept the interviewer is actually trying to test. A good stereo rendering explanation names the trade-off, the failure mode, and the evidence you would use before choosing. Use stereo rendering once more at the decision point so the answer reads as judgement rather than a detached example.

Keep stereo rendering explicit in the answer because that is the concept the interviewer is actually trying to test. A good stereo rendering explanation names the trade-off, the failure mode, and the evidence you would use before choosing.

What actually differs between the eyes

Two eye views of the same moment differ in exactly one respect: the position, and to a small degree the orientation, from which the scene is observed. The scene contents are identical. The same objects are visible, the same animations have advanced to the same instant, the same lights are on, and the same shadows are cast. Only the view matrix and the projection matrix change, and the projection matrices differ because each lens has its own asymmetric frustum rather than a symmetric one centred on the eye.

That observation is the whole of the optimisation, and it also draws the boundary. Anything that depends only on world space or light space can be computed once. Anything that depends on where a pixel lands on the screen must be computed twice, because the two eyes have different screens.

The naive path and its two separate costs

The straightforward implementation renders the whole frame twice: traverse the scene, cull, submit draw calls, shade, run post-processing, for the left eye and then again for the right. It doubles two distinct things, and conflating them is the most common weakness in an answer.

The first is CPU cost. Every draw call has a submission cost on the CPU, and if you walk your scene graph and submit twice you have doubled the CPU work of frame submission along with any per-object state changes. On mobile-class hardware, which is what a standalone headset is, CPU submission cost is frequently the binding constraint rather than shading.

The second is GPU cost, which splits again. Vertex processing genuinely doubles, because each vertex must be transformed by two different view-projection matrices. Fragment shading doubles because there are two full-resolution images to fill. These respond to different fixes, which is why you must know which one is hurting you.

flowchart TD
  A[Scene traversal<br/>and animation] --> B[Cull against<br/>combined frustum]
  B --> C[Shadow maps<br/>in light space]
  C --> D[One draw call<br/>per object]
  D --> E[Vertex shader picks<br/>matrix by view index]
  E --> F[Render target array<br/>two layers]
  F --> G[Per-eye post-processing]

Everything above the last node happens once per frame; only the final step is unavoidably per-eye. That split is the answer in one picture.

Doing the shared work once

Scene traversal, animation and skinning are view-independent, so they should run once regardless of how the eyes are rendered. Skinning a character twice for two eyes is pure waste, since the skinned result is a world-space mesh.

Culling deserves care rather than duplication. Culling twice, once per eye, is wasted CPU time, but culling against one eye's frustum and reusing the result silently drops geometry visible only at the other eye's outer edge. The correct approach is to cull once against a frustum that encloses both eyes' frusta. Because the interocular distance is small relative to typical view distances, that combined frustum is barely larger than either one, so you keep almost all the culling benefit and cannot produce a one-eye popping artefact.

Shadow map generation is rendered from the light's point of view, so it is view-independent and belongs in the once-per-frame bucket too. The caveat is cascaded shadow maps, whose cascade splits are usually fitted to the view frustum; fit them to the combined frustum for the same reason.

Multiview: one draw call, two views

The technique that removes the duplicated submission cost is multiview rendering, exposed as the VK_KHR_multiview feature in Vulkan and the OVR_multiview family of extensions in OpenGL ES. You bind a texture array with one layer per eye, issue a single draw call, and the driver runs the vertex shader once per view per vertex, with the current view exposed to the shader as an index. The shader uses that index to select its own view-projection matrix.

#extension GL_OVR_multiview2 : require
layout(num_views = 2) in;

uniform mat4 viewProj[2];   // one per eye
in vec3 position;

void main() {
    // gl_ViewID_OVR is 0 for the left eye and 1 for the right; the whole
    // point of multiview is that this is the ONLY per-eye branch needed.
    gl_Position = viewProj[gl_ViewID_OVR] * vec4(position, 1.0);
}

What this saves is precise, and being precise about it is what a senior answer sounds like. It removes the second traversal and the second submission, so the CPU cost of drawing approaches that of a mono frame. It does not remove the per-vertex transform work, which still happens for both views, and it does not remove fragment shading, which still fills two images. If you are fragment-bound, multiview will disappoint you and the fix is resolution, foveation or cheaper materials instead.

An earlier variant achieves something similar without the extension by drawing each object as two instances and using the instance index to pick the eye, then routing the result to the correct viewport. It works, and it is what you fall back to where multiview is unavailable, but it leans harder on the shader author to get the routing right.

Where sharing stops working

The pipeline stages that resist all of this are the ones computed in screen space, because a screen-space quantity is by definition a function of the view. Screen-space ambient occlusion, screen-space reflections, depth-of-field, temporal antialiasing history and most post-processing all need their own pass per eye against that eye's depth and colour buffers.

The failure mode when someone shares them anyway is instructive, because it is not a subtle quality loss. If you compute a screen-space reflection for the left eye and present it to both, the reflection appears at the same screen position in both eyes, which means it has zero disparity and therefore reads as being at infinite depth. The reflection detaches from the surface it belongs to and floats. Stereo makes rendering shortcuts visible that a monitor forgives entirely, and that is the general lesson: any approximation that gets depth wrong is a comfort problem in a headset rather than an aesthetic one.

The same logic condemns billboards and impostors that face a single camera, and any effect authored against a single view vector. Each needs either a per-eye version or a formulation that lives in world space.

The check nobody runs until it bites

Adopting multiview is not a switch, it is a shader audit. Every shader that reads a view matrix, a camera position, a view-projection matrix or a screen-space coordinate has to obtain it through the view index rather than from a single global. One shader that misses this renders both eyes from the left eye's viewpoint, and the result is not an obviously broken frame. It is a frame that looks nearly right, in which one object has no stereo depth. On a monitor screenshot it passes review.

In the headset the object sits at the wrong distance and the user's eyes cannot agree on where it is, which produces the eye strain that gets reported vaguely as "something feels off in that scene". Validate in the headset, and validate by comparing per-eye captures rather than a single-eye screenshot.

© 2026 Preptima. Originally published at preptima.com.