Writing & reading notes

Definitely worth a read if you're interested in foundational models for robotics and the direction the field is going in: pretraining on video to imagine how the world evolves, then fine-tuning that same backbone to act. It's the throughline behind a lot of what I'm working on right now.

I've been digging into video-based humanoid control, and this paper really stood out. The pipeline is surprisingly clean:

From a single monocular video, the system jointly reconstructs metric-scale human motion and scene geometry, retargets it to a humanoid, and trains a policy to track those trajectories under physical constraints. What I like is how cleanly the roles separate:

• reconstruction → geometric + temporal supervision
• retargeting → embodiment consistency
• RL → dynamic feasibility
• distillation → a single multi-skill policy

It effectively turns passive video into a structured control signal, without relying on teleoperation or motion capture. The limitation is that it's still fundamentally trajectory-tracking: it inherits biases from reconstruction and has no explicit mechanism for reasoning about alternative futures or task objectives.

That raises a more interesting direction: what happens if we replace trajectory tracking with a learned world model?

That would decouple perception (what is happening), prediction (what will happen), and control (what to do). I'm exploring a small version of this pipeline, starting from extracting motion priors from video and training a policy in simulation.

I've been diving into real-time execution for VLA models, and RTC stood out. The core idea is simple: instead of predicting actions, executing them, and then pausing to think again, RTC predicts the next action chunk while the robot is still moving.

By overlapping chunks and using a "freeze + inpaint" strategy, part of the trajectory is treated as fixed while the rest is generated, effectively turning the problem into constrained sequence prediction. This removes the latency bottleneck and shifts VLA policies from batch-style inference toward streaming control, much closer to how real robots actually operate.

One interesting limitation: overlapping actions are frozen, so the system can't correct mistakes mid-chunk: it can only adapt going forward. That raises a deeper question about how much of the future a robot should commit to at any given moment. It feels like an important step toward making generalist policies actually deployable, and I'd love to explore combining it with residual RL or adaptive horizons for something even more reactive.

One of the biggest weaknesses of VLA models is long-horizon reasoning. Most VLA policies predict actions from the current observation alone:

This works surprisingly well for short manipulation tasks. But once the robot executes 20–100 steps, small errors compound and the policy drifts off distribution. TraceVLA asks a simple question: what if we gave the robot an explicit visual memory of its past trajectory? Instead of only the current frame, it provides two visual inputs (the current observation, and the same image with trajectory traces overlaid), encoding the robot's past state-action history directly in the visual domain. The pipeline:

• 1 → track motion trajectories from past frames with a dense point tracker (CoTracker)
• 2 → filter points with significant movement
• 3 → overlay those trajectories onto the observation frame
• 4 → feed both the original image and the trace image into the VLA backbone

This effectively converts temporal information into spatial structure, which transformers reason over much better. The result: +10% over OpenVLA in simulation, 3.5× higher success on real-robot tasks, and strong generalization across embodiments.

The broader insight is what I find interesting: instead of scaling context windows or storing long histories, this compresses trajectory memory into a spatial representation: the robot doesn't need to remember the whole action sequence because it just sees the path it took. That feels especially relevant for long-horizon tasks like cloth folding, object rearrangement, multi-step assembly, and tool use. I ran into similar long-horizon issues working with policies trained from teleoperation data, so seeing this was exciting, a step toward giving VLA systems a more explicit notion of spatial-temporal memory.

A lot of robotics failures don't come from predicting the wrong action. They come from contacts, friction, and object dynamics making the control problem messy. Residual RL proposes a simple but useful hybrid: use classical control for what it's good at, and reinforcement learning for the leftover complexity. The reward decomposes as:

• f(s_robot) → geometric objectives (trajectory tracking, positioning), well handled by conventional feedback control
• g(s_objects) → object interactions and contact dynamics, much harder to model and better learned

The key idea is an additive control policy:

• π_H → a hand-engineered controller (impedance or trajectory control)
• π_θ → a learned residual that corrects for contacts, friction, and unmodeled dynamics

Instead of learning a controller from scratch, RL only learns the residual correction. Evaluated on a real-world task (inserting a block between two standing blocks without tipping them over), the results are striking: residual RL learns faster and performs better than RL alone in both sim and reality; on misaligned starts the hand-designed controller succeeds 2/20 while residual RL succeeds 15/20; it learns subtle corrections (nudging blocks into alignment) in ~8,000 samples (~3 hours of real interaction); and initialized from MuJoCo, it solves the task in under 1,000 real-world timesteps.

My takeaway: structure from classical control plus adaptability from RL is a very clean way to tackle contact-rich manipulation: let RL focus on the parts that are hardest to model, instead of rediscovering everything. I keep thinking about this in modern VLA systems: should robustness come from residual policies, or from fine-tuning the full foundation model?

One of the biggest weaknesses of imitation-trained VLA models is distribution shift during rollout. Behavior cloning optimizes action prediction on expert states, but once deployed the robot visits states induced by its own policy, and small prediction errors compound quickly. (As I was preparing for my role at AXIBO, I started digging deeper into VLA systems: action heads, value integration, online adaptation, embodiment constraints.) ConRFT proposes a two-stage reinforced fine-tuning pipeline aimed at exactly this problem.

Stage 1: Cal-ConRFT (offline). Calibrated Q-learning for conservative value estimates, augmented with a behavior-cloning loss under a unified objective:

Trained on only 20–30 demonstrations, so the BC term is needed to stabilize; the hybrid objective improves expected return while staying close to demonstrated behavior.

Stage 2: HIL-ConRFT (online). Same unified loss, symmetric sampling from demo and replay buffers, and human-in-the-loop interventions for safe exploration, gradually shifting weight from BC (β) toward Q-learning (η). One detail I liked: instead of a diffusion action head, ConRFT uses a lightweight consistency policy head (diffusion horizon discretized into M = 40 steps, a 2-layer MLP), giving efficient inference with value-guided refinement.

Across 8 real-world tasks it reaches 96.3% average success within 45–90 minutes of online training, a 144% improvement over the supervised baseline with 1.9× shorter episodes (the baseline reached only 31.9% in the same budget). The takeaway: imitation provides a strong prior, but without value-based refinement, policies struggle under their own induced state distribution. ConRFT reframes VLA fine-tuning as consistency-based policy learning guided by Q-estimates and stabilized by behavior cloning.