Watch the animation: a human arm (cyan) and a robot arm (red) both try to reach the same target.
The human takes multiple different paths - sometimes from the left, sometimes from the right. All valid!
What if the robot averages these paths?
The average of left + right = straight into the obstacle!
Now watch two robots race: one predicts single actions, one predicts action chunks.
The single-action robot jitters and drifts. The chunking robot moves smoothly.
Predict K=100 actions at once!
Instead of "what's next?", ask "what's the next 100 steps?"
Smooth motion instead of jerky steps!
The Conditional VAE (CVAE) solves the averaging problem. It captures the style of each demonstration.
Encoder compresses demonstration → latent Z
Z captures "which approach" - left, right, high, low...
Watch: multiple trajectories enter the encoder. The Z variable (gold) emerges - it selects one style, doesn't average!
Temporal ensembling makes predictions even smoother by blending overlapping chunks.
Multiple chunks overlap in time
Blend them with exponential weighting → ultra-smooth motion
Watch the animation: older predictions fade out, newer predictions get more weight.
You've just learned the three software innovations that made ACT successful on $20K hardware!
1. CVAE: Capture style, don't average
2. Action Chunking: Predict K actions at once
3. Temporal Ensembling: Blend for smoothness
These ideas aren't new - but combining them correctly made bimanual manipulation actually work.