Oossa

Runway releases open-weight Praxis-1 robot controller

The model turns camera images and instructions into robot movements. Runway says it led a simulation benchmark, but its tests have not been independently checked.

By Published by Oossa: 2 min read

Michael Fousert · Unsplash

Runway has released Praxis-1, a robot-control model whose weights and source code are available under the Apache 2.0 license, which permits commercial use. The model takes in camera images, text instructions and information about a robot’s position, then predicts how the scene will change and what movements the robot should make. It adjusts its control as new camera frames arrive.

Runway first introduced Praxis-1 in late September and initially shared it only with selected partners. Robotics companies Standard Bots, Noble Machines and Ultra have since tested it on different systems, including a humanoid robot, an industrial arm and a dual-arm warehouse robot. Developers can download it from Hugging Face after signing up and agreeing to access terms; its code is on GitHub.

How does it perform?

Praxis-1 builds on a video model trained to understand how objects move and interact. Runway fine-tuned it with videos of human actions and robot data, aiming to make use of what the model learned from ordinary footage. The approach may reduce how much robot-specific data is needed, but that has not yet been established: Runway still used about 277 hours of DROID robot recordings to adapt Praxis-1 to a Franka arm, and 1,400 to 2,600 hours of material for a dual-arm robot.

In Nvidia’s RoboLab-120 simulation benchmark, which tests 120 object-sorting, grasping and stacking tasks, Praxis-1 scored a 46.2% success rate. FLUX 3 Action scored 42.9% under the same conditions. Praxis-1 ranked first among World Action Models, a category of systems that predict how a scene will change to guide actions, and fifth overall. Runway ran the tests; the results have not been independently verified.

On a real Franka arm, Praxis-1 completed 88 of 100 attempts across ten tasks, compared with 80 for FLUX 3 Action. In a separate simulation using a robot platform excluded from its earlier training, Praxis-1 reached 80% after fine-tuning on about 95 hours of robot data. Runway reports that Google’s original RT-1 scored 52.4% on the same benchmark.

What are the limits?

The base model needs additional training to work with a new robot. Its standard version requires a GPU with 80 GB of memory. Runway estimates that a compressed version can fit on an RTX 5090 with 32 GB, though its peak memory use has not been measured. Runway also says Praxis-1 can confuse similar-looking objects and struggle with vague instructions. Complex tasks need to be broken into clear steps because the model cannot yet reliably plan long action sequences on its own.

Why it matters

Robot developers can download Praxis-1 and inspect or adapt its code under a license that permits commercial use, though the base model needs extra training for each new robot. Runway’s results are promising, but they are company-run and not independently verified, so it remains unclear how well the model will perform in other developers’ settings or whether video pretraining actually reduces data needs.

Was this article useful?
Share

Read next

Oossa · Newsletter

The week in AI, explained

Every Monday: the stories worth knowing, in plain language. Free, no spam.