Papers
arxiv:2610.02459

OpenRUA: Robot-Use Agents Are Zero-Shot Visuomotor Policies

Published on Oct 1
· Submitted by
Zhaoyang Chu
on Oct 6
Authors:
,
,
,
,
,
,

Abstract

Coding agents are extending their reach into the physical world by writing and executing robot control programs. One might expect the agents to use the existing mature software stack that engineers have developed over decades to access sensors and control motion. Yet prior work primarily engineers complex custom harnesses to orchestrate agents for robot use, particularly by prescribing specialized workflows and providing bespoke interfaces. This raises the question: "Is such additional harness engineering necessary?" We introduce OpenRUA, a zero-abstraction harness that bypasses bespoke abstraction layers by providing off-the-shelf coding agents with only terminal access to the robot's native software interface ROS 2. OpenRUA employs a minimalist workspace-as-harness design, only offering ROS 2 documentation and basic tools while leaving the coding agent to organize its own work without orchestrating any agentic workflow. Within this workspace, OpenRUA recasts perception as file I/O and manipulation as coding. With Claude Code powered by Claude Opus 5, OpenRUA achieves success rates of 99.0% on CaP-Bench and 87.0% on LIBERO-PRO, demonstrating that an off-the-shelf coding agent can serve as a zero-shot visuomotor policy through the robot's native interface, without bespoke primitives or task-specific training. Under this minimalist design, further analysis reveals striking emergent behaviors of coding agents: (1) For perception, the agent spontaneously writes programs that process raw sensory inputs and derive metric measurements in 96.80% of episodes. (2) For manipulation, the agent spontaneously builds motion-control clients (e.g., gripper control) in 95.87% of episodes and closed-loop control programs (e.g., adjusting motion based on sensor feedback) in 50.13% of episodes. Our code is available at https://github.com/terminalworld/OpenRUA.

Community

Paper submitter

🤖 OpenRUA: Off-the-shelf coding agents (Claude Code, Codex) directly control robots via native ROS 2 as zero-shot visuomotor policies, with no VLA models and no hand-crafted primitives.

🚫 Zero-Abstraction: Cuts through over-engineered harness slop and rigid workflows. The agent gets only terminal access to the robot's native ROS 2 software stack, bypassing bespoke abstraction layers.

🛠️ Workspace-as-Harness: Treats robot control as a live software project. A bare workspace directory serves as the entire harness, where:
• Perception becomes file I/O: sensor observations are saved and inspected directly as workspace files
• Manipulation becomes CLI coding: control scripts and ROS 2 commands are written and executed in the workspace

📊 Core Experimental Results:
• CaP-Bench: 99.0% success rate (+18.0% over SOTA ASPIRE), with 100% on Nut assembly (vs. 0-9% for prior methods)
• LIBERO-PRO: 87.0% success (beating Harness VLA's 82.4%)
• RoboCasa365: 28.1% on unseen composite tasks (learned policies collapse to 5.0%)

💡 Striking Emergent Behaviors:
• 96.8% episodes: spontaneously writes code to extract 3D coordinates from depth
• 95.9% episodes: builds reusable ROS 2 motion-control client libraries
• 50.1% episodes: writes closed-loop feedback controllers (force/contact/gap)
• Autonomously recovers from physical failures (e.g. fallen objects)

🔗 Paper · GitHub · Demo Video
overview

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2610.02459
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.02459 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.02459 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.02459 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.