An initial image and an action trajectory. The body is rendered in 3D along the actions, and Doppel-8 generates the video: a long-horizon rollout on a held-out OakInk2 episode.
Abstract
Video world models usually learn how a robot moves from raw action vectors. Doppel instead renders the robot along its planned motion, from the camera's viewpoint, and conditions one shared video model on these renders together with the actions. Camera poses missing from robot datasets are recovered from the videos. One model trained on eight datasets (Doppel-8; about 600 hours of robot arms, a bimanual robot, a humanoid, and human hands and bodies) improves PSNR over action-only baselines on the three robot-arm datasets by 1.00 dB on unseen trajectories and 1.20 dB out of distribution. Rendering also makes learning more efficient: trained on the three robot datasets, Doppel achieves higher average PSNR and SCS than a shared action-only baseline with 63.8% fewer training draws.
Actions alone, or the rendered robot
Same first frame, same actions. The usual model sees only the action vectors; Doppel also sees the robot rendered along those actions.
Videos
Same first frame, same actions. The baseline is an action-only expert: the same backbone, trained on that robot alone.
The action-only expert moves the arm but leaves the object in place. On the held-out toy kitchen, Doppel-8 raises moving-object PSNR from 21.50 to 24.85 dB.
Clip
Long horizon
Rollouts from a single frame on held-out test episodes. Each step continues from the model's own last frame, so the model never sees the ground truth again. Errors accumulate over the rollout.
Camera extrinsics
Rendering the robot needs the camera pose. RoboMIND, Bridge and ABC don't provide it, so we recover it from the videos: render the robot at its logged joint angles and optimize the camera until the render lines up with the video.
One run, step by step
Recovered extrinsics
| Embodiment | Demos | Extrinsics | One per |
|---|---|---|---|
| UR5e RoboMIND | 17,588 | 49 | recording session |
| WidowX Bridge | 14,824 | 56 | recording session |
| YAM ABC | 12,933 | 12,933 | video |
| Total | 45,345 | 13,038 |
ABC cameras move between recordings, so each ABC video gets its own extrinsics. We will release the recovered extrinsics with the calibration code.
Recovered cameras
One clip per recovered camera, robot in blue. Click a tile to enlarge.
Drag across the enlarged view to compare.
Method
Doppel conditions Wan 2.1 I2V 14B on render-enriched dense representations, injected as per-token timestep embeddings.
Data
Eight datasets: about 600 hours of training video and 803,602 training clips, with one shared action encoder and no embodiment label. Each dataset has 200 validation clips (unseen trajectories) and 200 test clips (the listed shift).
Results
Doppel beats action-only conditioning and fine-tuned OSCAR on all three robot datasets, on unseen trajectories and under distribution shift. The cards show Doppel-8; click one for Doppel's training curve against the expert.
Ground truth beside Doppel-8 on held-out test clips: a new lab, a new kitchen, unseen objects.
The five added datasets
Doppel-8, the same model as the cards above.
Trained on the three robot datasets, Doppel achieves higher average PSNR and SCS than a shared action-only baseline with 63.8% fewer training draws.
Region metrics
Scores on the robot and on moving objects only, at the final checkpoints, as in the table.
Full whole-frame results table
| Dataset | Method | Trained on | PSNR ↑ | SSIM ↑ | LPIPS ↓ | SCS ↑ | PSNR vs action-only |
|---|
Metrics over the 16 predicted frames. SCS: overlap of SAM 2 object masks between the predicted and ground-truth videos.
Policy evaluation
A policy runs closed-loop inside Doppel: its actions are rendered on the robot, Doppel predicts the scene, and the prediction is the policy's next input. Here Doppel is fine-tuned on this robot's demonstrations.
Bimanual YAM, MolmoAct2 fine-tuned on this robot. From the real start frame, the policy completes the task inside Doppel.
In the wild
Extreme out of distribution: a phone video of a climber. From one real frame and the input motion, Doppel-8 rolls out the rest.
Limitations
Geometry and a camera
Building the renderings needs a URDF, accurate robot geometry and a calibrated camera. Unknown or rapidly changing viewpoints remain a limitation.
Configuration trajectories
Doppel conditions on joint-space trajectories. Imperfect geometry or lower-level control commands would require modeling uncertainty in execution.
Seen embodiments, one camera
Evaluation covers embodiments seen in training and one camera per trajectory. Transfer to unseen embodiments and multi-view prediction are future work.
Long horizons
Long rollouts accumulate error: PSNR falls by about 5 dB from start to end.
Citation
@misc{jangir2026doppel,
title = {Render the Robot, Generate the World:
One Video World Model Across Embodiments},
author = {Jangir, Yash and Mirakhor, Karan and Zabounidis, Renos and
Irshad, Muhammad Zubair and Zakharov, Sergey and
Bisk, Yonatan and Bharadhwaj, Homanga},
year = {2026}
}

