RL-100

RL-100

Real-World Reinforcement Learning System


Additional videos for first round review

Extreme OOD test on Juicing and Pouring tasks

Extreme OOD test of Orange Juicing.
Extreme OOD test of Pouring

New Task: Box Folding

Teaser video
Failure cases of baselines, and consecutive successful trials on Box Folding.
Extreme OOD test of Box Folding
Robustness to human disturbances on Box Folding
Baseline - DP3
Baseline - DP-2D

From demo to duty: RL-100 can serve continuously for ~7 hours—reliable, real-world robot helps.

Main video


Reliable, Efficient, and Robust Real-World Robotic Manipulation Deployment

100% success across seven tasks

Soft-towel Folding
Dynamic Unscrewing
Pouring
Orange Juicing - Placing
Orange Juicing - Removal
Agile Bowling
Dynamic Push-T
Overall Juicing

Baselines - DP3

Soft-towel Folding
Dynamic Unscrewing
Pouring
Orange Juicing - Placing
Orange Juicing - Removal
Agile Bowling
Dynamic Push-T

Robustness to physical disturbances

Sustained counter-rotational interference (over 5 seconds)
Counter-rotational interference
External pulling and lateral dragging
Dragging perturbations

Takeaway:
Soft‑towel Folding: Disturbances in Stage‑1 (initial grasp) and Stage‑2 (pre‑fold) each retain 90% success.
Dynamic Unscrewing: Up to 4 s of reverse force during twisting and critical visual alignment—100% success; stable recovery.
Dynamic Push‑T: Multiple drag‑style disturbances during pushing—100% success.
Overall: 95.0% average success across tested scenarios, indicating reliable recovery under unstructured perturbations.

Zero-shot adaptation

Changed Surface (Dynamics)
Changed Surface (Dynamics)
Different granular/fluid materials
Visual and physical interference objects
Different granular/fluid materials
Folding - unseen towel shape

Takeaway:
Dynamic Push‑T: Large friction changes—100%; added distractor shapes—80%.
Agile Bowling: Floor property changes—100%.
Pouring: Granular (nuts) → liquid (water)—90%.
Folding: Unseen towel shape-80%.

Average 90.0% success across four change types without retraining.

Few-shot adaptation

Inverted pin arrangement
Modified container shape
Different towel material

Takeaway:
Soft‑towel Folding: New towel materials—100%.
Agile Bowling: Inverted pin arrangement—100%.
Pouring: New container geometry—60%.

Average 86.7% with only 1–3 hours of additional training.

Training efficiency

RL training curve for Bowling

Takeaway:
The policy achieves consistent 100% success after approximately 200 episodes of on-policy rollouts.

Human vs. Robots

Robot vs. Human
Robot vs. Human teleoperation

Takeaway:
1) he robot achieved more successful bowling trials than the five human participants under the same number of attempts, 25 successful trials (robot) vs. 14 successful trials (human).
2) For Push-T, the robot achieved more successful trials than expert human and beginner human at the same wall-clock time, 20 successful trials (robot) vs. 17 successful trials (expert) vs. 13 successful trials (beginner).

Execution Efficiency

Execution Efficiency - Folding
Execution Efficiency

Takeaway:
1) Execution Efficiency: CM (RL) > DDIM (RL) > DP3 (IL) > DP (IL)
2) Single action vs. action chunking: single-step control mode is used when a fast closed-loop reaction is required
while action chunking is preferred for coordination-heavy or high precision tasks where smoothing mitigates jitter and limits error compounding.
* Execution efficiency is defined as the robot’s average task completion time. For fair comparison, we report this metric only on action-chunking tasks. Single-step control (DDIM/CM) operates at the same inference rate, system-capped at 30 Hz (e.g., by the L515 camera), so runtime is dominated by hardware rather than algorithmic differences.

Easter egg

Easter egg

Ablation

Clip to variance
ReconvIB vs. No vs. Fix Encoder
CM vs. DDIM
2d vs. 3d
Predict type: epsilon vs. sample

Takeaway:
1) Variance clipping is valid for stable exploration - variance clipping in the stochastic DDIM sampling process.
2) Reconstruction is crucial for visual robotic manipulation RL as it mitigates representational drift and improves sample efficiency.
3) CM effectively compresses the iterative denoising process without sacrificing control quality, enabling high-frequency deployment.
4) On a relatively clean scene - the 3D variant learns faster and attains a higher final success rate.
5) Epsilon prediction is more suitable for RL: large noise schedule for exploration.

Thought

Execution Efficiency

Takeaway:
Training robots is like baking a cake: demonstration learning (IL) forms the sponge base, offline reinforcement learning adds the rich cream layer, and online reinforcement learning crowns it all as the cherry on top.