NVIDIA this week published Cosmos 3 Edge, a 4-billion-parameter world model built to run locally on edge hardware and target real-time robot control and vision-agent workflows. The new Edge tier is the smallest member of the Cosmos 3 family, designed to bring environment understanding, short-horizon simulation and action generation into memory-constrained systems used in factories, warehouses and hospitals.
Model architecture and multimodal design
Cosmos 3 Edge adopts a Mixture-of-Transformers approach that splits capabilities between two specialized towers. An autoregressive tower handles vision and language tokens for scene understanding and stepwise reasoning, while a diffusion tower processes vision, audio and action tokens for prediction and generative simulation. The towers keep separate normalization and feedforward layers but share multimodal attention layers to align information across text, video, audio and control signals.
Attention patterns vary by modality: language tokens use causal attention suitable for sequential reasoning, while diffusion tokens attend more broadly to support coherent prediction and denoising. Depending on the task, the system emits either reasoning tokens from the autoregressive path or denoised visual and action tokens from the diffusion path. According to NVIDIA’s documentation and the Cosmos repository, the Edge model uses a 2-billion-parameter dense transformer as its reasoner and follows Qwen3-VL–compatible message conventions for image and video inputs.
One action representation for many embodiments
A key design goal for Cosmos 3 Edge is a unified action representation that spans different robotic embodiments. Rather than using separate control formats for vehicles, camera rigs, single-arm manipulators or humanoid systems, the model encodes actions as compact geometric vectors capturing translation, rotation and manipulation state. This enables the model to link pixel-level changes to concrete motion and control inputs, so generated video functions as a simulated consequence of an action.
The Cosmos GitHub lists supported action dimensions for several embodiments: camera motion (9D), autonomous vehicles (9D), egocentric motion (57D), single-arm robots (10D), dual-arm robots (20D) and humanoid robots (29D). The shared representation allows the same model to predict an action’s visual outcome or infer the action that caused an observed change.
Policy mode and the DROID post-trained policy
Cosmos 3 Edge can operate as a policy that outputs both an action and its anticipated visual consequence given the current state. The bidirectional flow—predicting effects from actions or inferring actions from effects—connects world modelling directly to policy training and evaluation. To support robot manipulation tasks, NVIDIA released Cosmos 3 Edge Policy (DROID), a version post-trained on the DROID pick-and-place dataset. Post-training scripts are provided so developers can fine-tune policies on small H100 clusters or on-premise DGX systems before deploying to edge hardware.
On-device performance and deployment targets
Cosmos 3 Edge is built for memory-efficient inference across NVIDIA’s edge compute stack. Target platforms include GeForce RTX GPUs, RTX PRO, NVIDIA DGX, Jetson modules and the newest Jetson T2000 and T3000 modules. As reported, the Edge tier operates at a robot-control resolution of 640×360 and can produce 32 actions per inference. On NVIDIA Jetson Thor the model achieves real-time control at about 15 Hz. For generative tasks the tier supports 256p and 480p outputs at 12–30 fps, producing between 50 and 150 frames depending on configuration.
Developers can use the open Cosmos framework to post-train the model for a specific embodiment and sensor set in roughly a day. NVIDIA recommends a GeForce RTX 3070 or better for local prototyping and lists larger systems for faster post-training.
Release, licensing and benchmarks
Cosmos 3 Edge, a 4-billion-parameter member of the Cosmos 3 family, was released on Hugging Face on July 20 and is distributed under the Linux Foundation OpenMDW-1.1 license. It complements the larger Cosmos 3 Nano (16B) and Cosmos 3 Super (64B), which NVIDIA shipped on May 31, 2026 at GTC Taipei.
NVIDIA’s documentation cites internal evaluation results that place the 4B Edge model at the top of VANTAGE-Bench for models at that size, though these are reported as internal claims. The project’s technical report, model card and GitHub repository provide additional implementation details, action dimension lists, and post-training scripts for developers.
What this means for on-device robotics
Cosmos 3 Edge packages world-model capabilities—perception, short-horizon simulation and action mapping—into a compact, deployable model aimed at real-world robots and vision agents. By offering a shared action space and a combined reasoning-and-generation architecture, NVIDIA aims to simplify the workflow from prototyping to deployment on edge hardware.
For teams building autonomous systems in constrained environments, the Edge tier promises a practical pathway to run world models locally without relying exclusively on datacenter inference. The available code, model card and post-training scripts make it possible to adapt the model to varied embodiments and sensor setups, though real-world performance will depend on system integration and task complexity.
Source: MarkTechPost