NVIDIA has released four open-weight models in its Cosmos 3 family for creating interactive driving and robotics simulations. The checkpoints are available on Hugging Face under the OpenMDW1.1 license, which allows commercial and non-commercial use subject to its terms. Each is a post-trained version of Cosmos 3 Nano with 16 billion trainable parameters.
The models serve different tasks. Cosmos3-Nano-Transfer-Auto creates offline driving clips across as many as 11 camera views and can generate matching LiDAR data, a type of 3D distance measurement. Cosmos3-Nano-Sim-Auto generates camera and LiDAR observations in successive chunks as driving controls change. Cosmos3-Nano-Sim-Transfer turns simulator controls into RGB video, while Cosmos3-Nano-Sim-Bimanual predicts video from robot or camera actions for two-arm setups.
What do they simulate?
The driving models use World Scenario Map controls to describe road structure and traffic motion. Developers can vary conditions such as weather, lighting and time of day, while optional starting images help guide the scene’s appearance. NVIDIA says Transfer-Auto can generate synchronized video at up to 30 frames per second, with clips up to 400 frames at 480p or 720p; longer sequences can be made in chunks.
Sim-Auto is designed for interactive, ongoing use: a driving policy or person changes controls, and the model generates the next observations. That closed loop also needs a simulation runtime and a renderer to update the scenario controls. For robotics, Sim-Bimanual supports YAM and AgiBot robot setups, with controls for both manipulation and changing the camera viewpoint.
What developers need
NVIDIA’s Cosmos Framework provides training and inference recipes for adapting models to new sensors, actions and data. Its examples can run with PyTorch or vLLM-Omni on Hopper and Blackwell GPUs, according to NVIDIA. The Hugging Face instructions list Linux, an NVIDIA GPU, Docker and NVIDIA Container Toolkit as requirements for the Sim-Bimanual setup. NVIDIA says that setup was tested on one GB200 GPU, but that is not a minimum-memory specification.
These models generate observations; they do not replace every part of a simulator. In Sim-Transfer, for example, the simulator still calculates motion and physical state, while the model renders the resulting view. NVIDIA says researchers still need to measure whether the models reliably support policy training and evaluation.
Why it matters
For robotics and self-driving researchers, the models offer ways to generate varied sensor observations or test how a policy responds to changing controls. They still need compatible NVIDIA hardware and, for closed-loop driving, a simulation runtime; how well the generated scenes support real-world performance remains to be measured.
