Learning Physical AI: A Sim-to-Real VLA Pipeline with Seeed reBot Arm and NVIDIA Isaac
A new hands-on curriculum guides developers from robot demonstrations and simulation to VLA post-training and real-world deployment.
At World Robot Conference (WRC) 2026, Seeed Studio collaborates with NVIDIA to introduce a new hands-on curriculum for developers: Building Physical AI: A Sim-to-Real VLA Pipeline with Seeed reBot Arm and NVIDIA Isaac
Built around the open-source Seeed reBot Arm B601 RS and the NVIDIA Isaac open robot development platform, the curriculum walks developers through a complete physical AI workflow—from collecting real-world robot demonstrations to scaling data in simulation, post-training a Vision-Language-Action (VLA) model, and finally deploying the resulting policy onto NVIDIA Jetson for real-world inference.
What You Will Build
To make that workflow concrete and reproducible, the curriculum uses an autonomous table-cleaning task as its reference application. By the end of the curriculum, developers will have built a complete Sim-to-Real VLA pipeline capable of moving from human demonstrations to autonomous robot execution.
You will learn how to:
- Collect multimodal robot demonstrations through leader-follower teleoperation;
- Capture synchronized camera observations, robot states, actions, and natural language task instructions;
- Bring the Seeed reBot Arm embodiment into NVIDIA Isaac Sim;
- Leverage Cosmos Transfer for scene augmentation and to enhance model robustness.
- Post-train a GR00T 1.7 policy by combining physical demonstrations with synthetic datasets;
- Deploy the trained policy to NVIDIA Jetson for real-time edge inference
- Benchmark and evaluate autonomous table-cleaning performance on physical reBot Arm hardware, applying proven strategies to close the sim-to-real gap.
The objective is not simply to reproduce one robot demo.
It is to understand a reusable Physical AI development architecture that can later be extended to new manipulation tasks, environments, datasets, and robot embodiments.
Why reBot Arm B601 RS + NVIDIA Isaac?
Developing with robotics arms comes with steep hardware barriers: industrial-grade arms are prohibitively expensive, educational arms often lack sufficient precision, and most robotic arms on the market remain closed-source and developer-unfriendly.
The Seeed reBot Arm B601 RS directly addresses these challenges when paired with NVIDIA Isaac:
● Industrial-Grade Precision in an Open Platform: Features high-precision joint motors with 0.1 mm repeatability and up to a 2.5 kg payload, drastically reducing actuation gaps caused by backlash or structural flex seen in hobby servo arms.
● Full-Stack Open Ecosystem: Fully open-source and compatible with ROS1/ROS2, Hugging Face LeRobot, NVIDIA Isaac Sim for synthetic data generation and Isaac Lab for robot learning and evaluation at scale, Pinocchio, and Python SDKs.
● Synthetic Data & VLA Acceleration: Harnesses NVIDIA Isaac Sim’s parallel simulation capabilities alongside NVIDIA Isaac GR00T open models to accelerate policy training far beyond physical data collection speeds.
Inside the Policy: NVIDIA Isaac GR00T 1.7
At the center of the learning pipeline is NVIDIA Isaac GR00T 1.7, an open VLA model designed for Physical AI applications.
Rather than mapping an observation directly to a predefined symbolic command, a VLA model jointly reasons over visual observations, natural-language instructions, robot state, and action representations.
In the curriculum, developers use GR00T 1.7 as the policy foundation to translate multimodal observations and task instructions into continuous robot actions.
The architecture follows a dual-system design.
System 2 — Vision-Language Reasoning
The vision-language component processes visual observations together with natural-language instructions and performs higher-level task interpretation and reasoning.
System 1 — Action Generation
A diffusion-based action model combines this higher-level representation with the current robot state to generate continuous motor actions.
NVIDIA Isaac GR00T pairs this architecture with open data pipelines, simulation frameworks built on NVIDIA Omniverse™ and Cosmos™, CUDA-X™ accelerated runtime libraries, and NVIDIA Jetson edge systems for real-time inference and control.
Build the Pipeline in Four Phases
The curriculum breaks the complete workflow into four hands-on stages.
| Phase 1 | Teleoperation (LeRobot) | Capture real human demonstrations, logging joint trajectories, dual RGB-D feeds, and language prompts. |  Real-time leader-follower teleoperation with live camera stream |
| Phase 2 | Simulation (NVIDIA Isaac Sim) | Import reBot Arm USDA asset and apply domain randomization (lighting, textures, friction) to scale synthetic data. |  Parallel reBot arms executing tasks in Isaac Sim |
| Phase 3 | Policy Training (GR00T 1.7) | Post-train the 3B-parameter VLA model by blending physical demonstrations with synthetic datasets. |  LeRobot visualizer & training loss curves |
| Phase 4 | Edge Inference (NVIDIA Jetson) | Deploy policy onto NVIDIA Jetson Thor Developer Kit for real-time inference and benchmark physical table-cleaning performance. |  Inference on NVIDIA Jetson Thor |
From Virtual Training to Real-World Deployment
Physical AI development is no longer restricted to high-budget industrial research labs. By combining open hardware with NVIDIA’s GR00T open foundation models and Isaac open simulation frameworks, this curriculum delivers a completely open, reproducible foundation for building the next generation of autonomous robotic agents.
Start Your Physical AI Journey Today
Whether you are an academic educator building a robotics lab, a researcher exploring VLA models, or an enterprise developer creating autonomous AI agents, this curriculum offers a fully reproducible path from simulation to physical reality.
👉 Explore Seeed reBot Arm B601 RS Hardware & Documentation
About Author
