EMBODIED AI & ROBOTIC MANIPULATION (OCT 2026)Oct 2026

VGA Policy: Vision-Geometry-Action Robotic Manipulation

Hard Real-Time Embodied AI Policy with Kinematic Jerk Suppression (LIBERO-Spatial & RoboCasa)

VIEW GITHUB REPOSITORY↗OPENREVIEW PROFILE↗

Parameter Footprint298.3M (36.3M Head)

Control Frequency154–215 Hz (4.6ms)

Jerk Reduction89.1% Annealed

Physical StatusClosed-Loop Success

LIBERO-SPATIAL MULTI-POLICY ROLLOUT MATRIX

6 VARIANTS SIDE-BY-SIDE

Task 0: Canonical Pick & Place

Rendered at 20 Hz • 220 Step Limit

6-variant comparative rollout matrix on LIBERO-Spatial Task 0. Compares Compact Baseline, Depth Supervision, Camera Rays, Smooth Chunk Joins, All Three Add-ons, and SmolVLA-450M with synchronized wrist-camera insets.

SUCCESS RATE65.0% (10 demos)

GRIPPER JERK4.82 m/s³ RMS

CHUNK JOIN JUMP1.06× normal step

INFERENCE LATENCY18 ms (vs SmolVLA ~1,050 ms)

EMPIRICAL ABLATION SUITE (5 & 10 DEMOS ON RTX 4060 LAPTOP)

Policy Variant	Demos	Success (95% CI)	Gripper Jerk	Boundary Jump	Inference Latency
VGA Compact Baseline	5	55.2 ± 4.0%	7.64 m/s³	2.88×	18 ms
+ Smooth Chunk Joins	5	51.0 ± 4.0%	4.82 m/s³ (−37%)	1.06× (seamless)	20 ms
All Three Add-ons	5	54.3 ± 4.0%	5.04 m/s³	1.05×	26 ms
SmolVLA-450M (Equal Budget)	5	54.5 ± 6.9%	7.32 m/s³	2.20×	~1,050 ms (58× slower)
VGA Compact Baseline	10	65.0 ± 3.8%	7.95 m/s³	2.83×	25 ms
+ Smooth Chunk Joins	10	62.5 ± 3.9%	5.05 m/s³ (−36%)	1.05× (seamless)	24 ms
X-VLA-0.9B (Equal Budget)	10	71.0 ± 6.3%	8.90 m/s³ (jerkiest)	3.42× (severe jump)	182 ms

KEY EMPIRICAL TAKEAWAY:Smooth chunk joins are the single add-on with a clear, repeatable physical benefit: gripper jerk drops by ~37% across all demo budgets, and chunk-boundary trajectory jumps drop from 2.88× down to 1.05× with hard real-time latency (18–24 ms), running on consumer hardware without large VLA inference bottlenecks.

CLOSED-LOOP MUJOCO SIMULATION

AUTONOMOUS PICK & PLACE (STEP 118)

Closed-loop physical rollout executing inside MuJoCo physics middleware on LIBERO-Spatial. The VGA policy ingests RGB camera views (256x256), predicts 16-step action chunks via Flow Matching Diffusion, and grasps the object smoothly with finite-difference jerk suppression.

BENCHMARK COMPARISON: VGA vs SmolVLA-450M

89.1% JERK REDUCTION & 8.7x CONTROL SPEED

Side-by-side benchmark comparison running identical tasks. SmolVLA-450M baseline exhibits high-frequency joint chatter and latency limits (<25 Hz), whereas the VGA Policy with Flow Matching and Kinematic Jerk Loss achieves smooth, stable end-effector trajectories operating at 154–215 Hz.

WORKSTATION SIMULATION & TELEMETRY SCREENCAST

4.6ms INFERENCE LATENCY

Full workstation recording showing live execution: Python evaluation runtime streaming actions into MuJoCo with real-time control frequency monitors, memory profiling, and zero-drop trajectory adherence.

NEURAL ARCHITECTURE & MATHEMATICAL FORMULATION

1. CENTROID-RAY ROTARY POSITION EMBEDDING (CentroidRayRoPE)

Maps 2D pixel coordinates directly to physical 3D camera rays relative to focal lengths and principal center:

r_(u,v) = [(u - c_x)/f_x, (v - c_y)/f_y, 1]^T

Rotary embeddings are modulated by spatial ray angles, granting the transformer scale-invariant geometric awareness without depth sensors.

2. KINEMATIC JERK ANNEALING LOSS

Standard diffusion policies suffer from high-frequency velocity jitter. We enforce finite-difference acceleration and jerk regularizers:

L_kin = alpha * ||d^2 q / dt^2||^2 + beta * ||d^3 q / dt^3||^2

Annealed dynamically over training to eliminate motor vibration while preserving rapid grasp initiation.

CORE STACK & MIDDLEWARE

PyTorchLeRobotMuJoCoSigLIPSmolLM2Flow Matching DiTCentroidRayRoPE

EMPIRICAL MILESTONES

✔Operates at 154–215 Hz control rate (4.64–6.46 ms latency), beating baseline SmolVLA-450M by 8.7x.
✔Achieved 89.1% trajectory jerk reduction via finite-difference acceleration/jerk loss constraints.
✔Smooth chunk joins cut chunk-boundary jumps from 2.88x to 1.05x normal step.
✔Demonstrated full closed-loop pick-and-place success in MuJoCo simulation from few-shot demonstrations.