Learning to Fold, end to end

0The whole machine

Under the engineering, this is an actor-critic loop with three jobs: collect episodes with the current policy, score every moment of them, and improve the policy toward the moments that scored well. What makes it unusual is how each job is done.

• The critic is not a separate network. The policy predicts its own chance of success from the camera images, and that prediction is the value function (§4).
• The policy is never trained with a policy gradient. Its training is supervised, and the advantage enters in two ways. AWR changes how often each frame is drawn for training (§8). RECAP feeds the advantage in as an input the policy can later be asked for (§9).
• Collection and training never wait for each other. Machines exchange episodes and checkpoints only through the Hugging Face Hub (§12).

One evaluation step, read from the code

Before any learning, here is what happens each time the simulated arms move. This is the inference path we ran ourselves. Training reuses the same loop to collect episodes.

1. The simulator sends its observation to Ilia's eval worker over HTTP (POST /infer, remote_loop.py). The observation is three 640×480 camera images plus 12 joint angles.
2. The worker holds the current plan, a chunk of future joint targets from the model server. It hands back the next target. When the executed part of the chunk is used up, it first sends this observation to the model server for a new chunk (eval_worker.py).
3. The target is 12 numbers, one angle per motor (each arm: five joints and a gripper). The simulator sets them as the joints' position targets (garment_bi_v2.py:281–282).
4. The physics engine advances by one time step. Each joint's motor is modelled as a spring-damper pulling toward its target; cloth, gravity and contacts move in the same step.
5. The cameras render, the sensors refresh, and the next observation is read. Joint angles and the top camera are fresh every step; the two wrist cameras refresh every third step.
6. Every 30th step the competition's checker tests whether the garment is folded; on success the episode stops.

Who is who

1The task, as RL sees it

The LeHome Challenge 2026, an ICRA 2026 competition, asks two SO-101 arms to fold one garment lying flat on a table. Four garment types are scored: long-sleeved tops, short-sleeved tops, long pants and shorts. They are children's sizes, which Ilia only realised when he assembled real arms for the final round (report §9.1).

What "folded" means

The checker tracks six points on the cloth. A fold is a set of distance tests between pairs of those points: some pairs must come closer than a threshold (two sleeve ends meeting), others must stay farther apart than one (the fold must not collapse into a ball). Every test must pass at once.

Read from the challenge's checker, lehome/utils/success_checker_chanllege.py. Thresholds are set per garment.

In the challenge code we ran, which is Ilia's fork at 5ea947e, the test runs every 30 steps (garment_bi_v2.py:54, 331–336). So a successful episode always ends at step 29, 59, 89 and so on. Every one of our successes did: 119, 149, 239, 269, 299, 329, 359 and 419 steps.

How the leaderboard scored

Each garment type had 20 instances: 10 seen garments that came with the organisers' scripted demonstrations, and 10 unseen ones. Of the unseen, 2 per type were public and 8 per type were private and never released. Every one of the 80 garments was run 10 times. The score is the plain average of the four per-type success rates (report §8.1).

2Why not just copy, and why not PPO

The starting point is behaviour cloning: train the policy to reproduce the organisers' scripted demonstrations. That gives a policy that sometimes folds. Everything after is about raising that rate.

Why not policy gradients

PPO, and its group-relative variant GRPO, improve a policy by raising the log-probability of actions that did well and lowering it for actions that did badly. That needs \(\log \pi_\theta(a\mid s)\), and a flow-matching policy does not give one cheaply: it produces actions by integrating a learned velocity field from noise (§3), and the probability of the result is not tractable. The π*_0.6 paper makes the same point about its own flow-matching policy.

Ilia adds a second argument (report §2.2). Valid action chunks occupy a thin set inside the space of all possible 30×12 chunks. A method that pushes probability away from bad actions moves predictions in directions the data never covered, mostly off that set. Reweighting and conditioning only move probability toward actions the policy already produces, so they stay on it. Finn frames the same property for offline RL: methods that supervise only on actions in the data implicitly constrain the policy to the data (lecture 7, slide 28).

3The policy network

The policy is Ilia's team's BEHAVIOR-1K 2025 model, itself an extension of Physical Intelligence's π_0.5, with changes for this task (report §4). Two halves:

• A prefix that reads the scene. SigLIP-So400m/14 (frozen) turns each 224×224 camera image into tokens, and a Gemma-2B transformer reads them together with the joint state and a few special tokens.
• An action expert, a Gemma-300M transformer. It reads the prefix's cache and produces a chunk of 30 future moves for all 12 joints.

What the prefix sees, and who sees whom

The prefix's tokens sit in groups under a staircase attention mask: each group sees itself and the groups before it, never the groups after (report §4.1).

Flow matching: how a chunk is made

The action expert does not output actions directly. It learns a velocity field: given a noisy chunk and a noise level, which way to move to get closer to a real chunk. At inference, you start from pure noise and follow that field in 10 steps.

Training, as written in the code (pi_modified.py:1224–1229, 1256). Take a real chunk \(a\) from the data and noise \(\epsilon\). Draw a noise level \(t\) from Beta(1.5, 1), squeezed into (0.001, 1). Mix:

1223            noise = self.generate_correlated_noise(noise_rng, batch_size)
1224            time = jax.random.beta(time_rng, 1.5, 1, (batch_size,)) * 0.999 + 0.001
1225
1226            # Compute noisy actions and target velocity
1227            time_expanded = time[:, None, None]
1228            x_t = time_expanded * noise + (1 - time_expanded) * actions
1229            u_t = noise - actions
      ⋮
1255            v_t = self.action_out_proj(suffix_out[:, :self.action_horizon])
1256            action_loss_i = jnp.square(v_t - u_t)  # [B, H, D]

Here \(t=1\) is pure noise and \(t=0\) is the clean chunk; \(u_t\) is the constant velocity that carries noise to data in a straight line. Sampling integrates from \(t=1\) down to \(t=0\) with \(S=10\) Euler steps of size \(-0.1\) (pi_modified.py:1471):

Three inherited details shape this. The starting noise is correlated across joints and time, drawn from the data's own action covariance shrunk halfway toward the identity, \(\Sigma_{\text{reg}} = 0.5\,\Sigma + 0.5\,I\). A temperature \(\tau\) scales it by \(\sqrt{\tau}\) (pi_modified.py:1615–1619). Each training example draws 5 independent (noise, \(t\)) pairs that share one prefix pass.

What a chunk contains

A chunk is 30 steps × 12 joints of changes from the current joint state. The training data's targets are converted to deltas, and the server converts the model's output back to absolute targets before anything is sent (training/config.py:416–423, all 12 joints). The delta spread grows with how far ahead a step is, roughly like a random walk, so each future position gets its own normalisation. Ilia fits a smooth curve per joint instead of using noisy per-step statistics (report §4.6):

416        if self.use_delta_joint_actions:
417            delta_action_mask = _transforms.make_bool_mask(12)
418        else:
419            delta_action_mask = _transforms.make_bool_mask(-12)
420        
421        data_transforms = data_transforms.push(
422            inputs=[_transforms.DeltaActions(delta_action_mask)],
423            outputs=[_transforms.AbsoluteActions(delta_action_mask)],

Without it, one global scale would shrink the near steps, the ones actually executed, and let the uncertain far end of the chunk dominate the loss.

The changes Ilia made for LeHome

Kept from the BEHAVIOR-1K model unchanged: no language tokens, correlated noise, soft inpainting between chunks (§11), and cross-layer mixing of the prefix cache before the action expert reads it.

4The policy is its own value function

Standard actor-critic training keeps a second network, the critic, to score states. Ilia puts the critic inside the policy: a few cheap linear heads read the same model's representation and predict how the episode is going (report §5).

Report Tables 2 and 3. The flow-matching action loss has weight 1.0. \(\operatorname{sg}\) is stop-gradient: the residual head cannot drag the success head toward itself. The residual head's loss is scaled by \((1-t)\), and samples with flow time \(t > 0.5\) are dropped, since a mostly-noise chunk says little about the future.

Three corrections keep the success head usable:

• Tail boost. A successful episode stops at once, while a near-miss lingers in its almost-folded state for many frames. Most "almost done" frames therefore come from failures, which pulls the head down just before real successes. The last 20 frames of successful episodes get 20× weight on the success loss.
• Label smoothing toward the garment's average success rate: \(y' = 0.95\,y + 0.05\,\bar p_g\).
• Extra weight decay of 0.001 on the head kernels, because the heads overfit badly otherwise.

Ours: the success head over 16 real episodes

Our smoke test logged the success and completion estimates each time a new chunk was planned. That happened every 5 steps for tops and every 3 for pants, and every 5 for all types in an episode's first 5 plans. Each panel is one episode.

5The RL you need

Four ideas carry everything in §6 to §10: return, value, advantage, and how to estimate advantage from a noisy value. This is the standard material, in Finn's CS224R notation where it exists, with this task's numbers attached.

Return, value, Q and advantage

The return from step \(t\) is the discounted sum of future rewards, \(G_t = \sum_{k\ge 0}\gamma^k r_{t+k}\). The value \(V^\pi(s)\) is the expected return from state \(s\) when acting with policy \(\pi\). \(Q^\pi(s,a)\) is the same expectation after first taking action \(a\). The advantage is the difference (lecture 4, slide 6):

Advantage answers the question that matters for improvement: was this action better or worse than what the policy usually does here? It averages to zero over the policy's own actions. Positive means "do more of this", negative means "do less".

Why subtract a baseline

Subtracting any quantity that depends only on the state leaves the expected policy-gradient update unchanged, because \(\mathbb{E}_{a\sim\pi}[\nabla\log\pi(a\mid s)\,b(s)] = b(s)\,\nabla\!\int\!\pi = 0\) (lecture 3, slide 17). It can still cut the variance a lot, and the value is the natural choice of baseline. That replaces the noisy return with the advantage (lecture 4, slide 10). In §7, Ilia asks how much of an imperfect value to subtract, and the answer is less than all of it.

One-step, n-step, and GAE

We never know \(Q\) or \(V\) exactly. A learned value \(\hat V\) gives the one-step TD residual:

It is low-variance but biased by whatever \(\hat V\) gets wrong. The full Monte Carlo return minus \(\hat V(s_t)\) is unbiased but noisy. Using \(n\) real rewards before bootstrapping sits between the two (lecture 4, slide 20). Generalised advantage estimation (Schulman et al. 2016) averages all the n-step estimates with geometrically decaying weights, which reduces to a short backward recursion:

\(\lambda = 0\) gives the one-step estimate; \(\lambda = 1\) gives return minus value. Ilia uses \(\gamma = 0.999\) and \(\lambda = 0.99\), so \(\gamma\lambda = 0.98901\): a residual \(l\) steps ahead keeps weight \(0.98901^l\), which halves every 63 steps and falls to about 1/e after 91 derived. One step's advantage therefore mostly reflects what happens over the next few seconds of the data's clock.

Potential-based shaping

Adding \(F_t = \gamma\,\Phi(s_{t+1}) - \Phi(s_t)\) to the reward, for any function \(\Phi\) of state, cannot change which policy is optimal (Ng, Harada and Russell 1999). Summed over an episode the terms telescope, \(\sum_t \gamma^t F_t = \gamma^T\Phi(s_T) - \Phi(s_0)\), so the bonus depends only on where the episode starts and ends. In between, it moves credit earlier: an action that raises \(\Phi\) is rewarded when it happens, not only at the end. Ilia's completion signal is exactly this kind of term, with \(\Phi\) = predicted completion (§7).

Weighted imitation

The last idea is CS224R's bridge from imitation to RL (lecture 7, slides 17–21). Filtered behaviour cloning keeps only good trajectories and clones them. Advantage-weighted imitation clones everything, weighting each sample by \(\exp(A)\):

AWR is the principled version of this line, and §8 derives it. (Finn's slides write the temperature as \(1/\alpha\) and use \(\beta\) for the behaviour policy; this page follows the AWR paper, where \(\beta\) is the temperature.)

6Reward: from pass or fail to checkpoints

1. Checkpoints are groups of the checker's tests

dataset_writer.py:496–506. Tests are numbered from 0, as in the code; for tops, tests 0–2 are the three "meet" pairs and tests 3–4 the two "stay apart" pairs. The credit per checkpoint is one over the number of checkpoints. Shorts are the easiest type, so they get no intermediate checkpoint (report §6.1).

The first checkpoint can be lost: if its tests stop holding, the 0.5 is taken back, and it is paid again if they return. The second checkpoint only ever adds credit. Because the environment refreshes the checker every 30 steps (§1), reward can only change on those frames.

598    for t, f in enumerate(frames):
599        cs = f.get("check_status")
600        if cs is not None:
601            first_cp_now = all(cs[idx] > 0.5 for idx in first_cp_group)
602            other_completed = sum(
603                1 for group in other_groups
604                if all(cs[idx] > 0.5 for idx in group)
605            )
606
607            # First checkpoint oscillation: emit reward delta when state changes
608            if first_cp_now and not prev_first_cp_held:
609                rewards[t] += cp_reward
610                first_cp_transitions += 1
611                if first_cp_transitions == 1:
612                    first_cp_frame = t
613            elif not first_cp_now and prev_first_cp_held:
614                rewards[t] -= cp_reward
615
616            # Other groups: monotonic reward
617            new_other = max(0, other_completed - prev_other_completed)
618            if new_other > 0:
619                rewards[t] += new_other / num_checkpoints
620
621            prev_first_cp_held = first_cp_now
622            prev_other_completed = max(prev_other_completed, other_completed)

2. Tops get the first half gradually

For tops, the first 0.5 is not paid as one jump when checkpoint 1 is reached. It is paid out as the main "meet" distance, test 1, shrinks. With \(d_t\) that distance ratio, \(m_t = \min_{\tau\le t} d_\tau\) its lowest value so far, and \(t_1\) the frame where checkpoint 1 is first reached:

Using the running minimum matters: moving the cloth back and forth earns nothing, because only a new closest approach pays. If folding the first sleeve already closes 60% of the gap, it has earned 0.3.

650                # Distribute cp_reward gradually based on distance reduction
651                running_min = d_init
652                prev_cumulative = 0.0
653                for t, d in dist_samples:
654                    running_min = min(running_min, d)
655                    frac = (d_init - running_min) / (d_init - d_first_cp)
656                    frac = max(0.0, min(1.0, frac))
657                    cumulative = frac * cp_reward
658                    increment = cumulative - prev_cumulative
659                    if increment > 1e-9:
660                        rewards[t] += increment
661                    prev_cumulative = cumulative

3. On failure, everything is taken back

If the episode fails, every bit of credit it collected is withdrawn, so the episode total is exactly 1 for a fold and 0 otherwise. The withdrawal is spread evenly over the frames after \(t_p\), the last frame at which the running total reached its peak:

668    if success is False and len(rewards) > 0:
669        cum = sum(rewards)
670        if abs(cum) > 1e-9:
671            # Find the last frame where a positive reward brought cumulative to its peak.
672            # Only advance peak_frame on actual reward events (not frames where
673            # cumulative passively stays at the peak level).
674            running = 0.0
675            peak = 0.0
676            peak_frame = 0
677            for t, r in enumerate(rewards):
678                running += r
679                if r > 1e-9 and running >= peak - 1e-9:
680                    peak = max(peak, running)
681                    peak_frame = t
682            # Spread withdrawal uniformly from peak_frame+1 to end
683            n_spread = len(rewards) - peak_frame - 1
684            if n_spread > 0:
685                per_step = -cum / n_spread
686                for t in range(peak_frame + 1, len(rewards)):
687                    rewards[t] += per_step
688            else:
689                rewards[-1] -= cum  # fallback: peak at last frame

What the policy is finally rewarded for

Dense reward changes when credit arrives, never how much an episode is worth. Ilia also ignores the simulator's own shaped reward for how neat the final garment looks (report §6).

7Advantage, term by term

The advantage is the single number per frame that AWR and RECAP consume. Ilia's version has six ingredients. The report calls the final formula over-engineered (report §6.8), so it helps to take it apart in the order the code builds it (eval/dataset_writer.py: compute_gae_advantages).

Ingredient 1 · the value is the success estimate

Because each episode's reward sums to its outcome (§6), the expected reward still to come from a state is the chance of folding minus what has already been paid (report §6.2):

Raw head outputs are noisy, so every consumer uses an exponential moving average, \(\bar S_t = 0.2\,\hat P_t + 0.8\,\bar S_{t-1}\). The head also has a known bias near successful endings (§4). Offline, once the outcome \(y\) is known, the last \(K=30\) smoothed values are replaced by a straight line toward \(y\):

In the code below: ema_alpha = 0.2, value_tail_k = \(K\) = 30, success_label = \(y\).

710    out: list[float] = [0.0] * T
711    out[0] = float(raw[0])
712    for t in range(1, T):
713        out[t] = ema_alpha * float(raw[t]) + (1.0 - ema_alpha) * out[t - 1]
714
715    if value_tail_k > 0 and success_label is not None and T > value_tail_k + 1:
716        K = int(value_tail_k)
717        anchor_idx = T - K - 1
718        anchor = out[anchor_idx]
719        target = float(success_label)
720        for i in range(1, K + 1):
721            frac = i / K
722            out[anchor_idx + i] = anchor + (target - anchor) * frac
723    return out

Ingredient 2 · subtract only part of an imperfect value

The textbook move is to subtract the full value from the return (§5). Ilia argues that is only optimal for a perfect value estimate. He borrows CUPED, a control-variate method from A/B testing: the variance-minimising amount to subtract is \(\theta^*\hat V\), where

\(\theta^*\) is 1 for a perfect predictor and 0 for a useless one. During the competition he used a fixed \(\alpha_s = 0.5\) instead, believing \(\theta^*\) could not be computed. The report's own correction: it can be estimated from rollouts, and afterwards it measured roughly 0.4 to 0.8 per garment, with 0.5 near the median (report §6.2). So the dampened baseline is

Ingredient 3 · GAE over the success channel

Plug \(\hat V\) into the TD residual and run GAE with \(\gamma = 0.999\), \(\lambda = 0.99\). The terminal value is pinned to the true outcome, \(\bar S_T = y\):

Expanding \(\hat V\) shows what this does to the reward (report §6.4):

With full subtraction (\(\alpha_s = 1\)), a checkpoint reward would cancel out exactly: it enters the remaining return and \(R^{\text{cum}}\) equally. With \(\alpha_s = 0.5\), half of it survives, so the frame where a checkpoint is reached keeps a positive advantage. The other half of each residual is the change in the success estimate.

In the code: ema_s_arr = \(\bar S\), cum_before = \(R^{\text{cum}}\), v_hat = \(\hat V\), p_terminal = \(y\), gl = \(\gamma\lambda\).

803    v_hat = [success_alpha * (ema_s_arr[t] - cum_before[t]) for t in range(T)]
804    v_hat_terminal = success_alpha * (p_terminal - cum_total)
805
806    if T == 1:
807        delta = r[0] + gamma * v_hat_terminal - v_hat[0]
808        return [delta], V
809
810    gae = [0.0] * T
811    gae[-1] = r[-1] + gamma * v_hat_terminal - v_hat[-1]
812    gl = gamma * lam
813    for t in range(T - 2, -1, -1):
814        delta = r[t] + gamma * v_hat[t + 1] - v_hat[t]
815        gae[t] = delta + gl * gae[t + 1]
816    return gae, V

Ingredient 4 · completion as potential-based shaping

The success estimate saturates above about 0.9 on easy garments and drifts as the policy changes (report §6.3). Completion does neither: what "60% folded" looks like barely depends on which policy did the folding. Ilia adds it as a potential-based term (§5) with \(\Phi_t = \alpha_c\,\bar C_t\), \(\alpha_c = 0.5\), and terminal \(\bar C_T = y\):

There is no reward in this channel. It only says whether a frame moved the fold forward.

866    gae_c = [0.0] * T
867    gae_c[-1] = completion_alpha * (gamma * c_terminal - ema_c_arr[-1])
868    gl = gamma * lam
869    for t in range(T - 2, -1, -1):
870        delta_c = completion_alpha * (gamma * ema_c_arr[t + 1] - ema_c_arr[t])
871        gae_c[t] = delta_c + gl * gae_c[t + 1]
872    return gae_c

Ingredient 5 · stale episodes fall back on outcomes

Success estimates are recorded once, at collection time, on states the model had not trained on, and never recomputed: later models have trained on those episodes and would give overconfident answers. But as the policy moves on, old estimates describe an old policy. So each episode's success channel is blended with a segment advantage that uses outcomes only, weighted by how fresh the episode is (report §6.5):

An episode's sampling share starts at 1.0 and is multiplied by 0.98 each time a new batch of rollouts arrives; once it would fall below 0.1, the dataset is dropped (§12). The completion term stays at full strength for every episode, because completion does not go stale.

The segment advantage splits an episode at its first checkpoint and compares each part's reward with what is typical for that garment. Here \(p_{\text{cp}}\) is the garment's checkpoint rate, SR its success rate, and \(G(n) = (1-(\gamma\lambda)^n)/(1-\gamma\lambda)\) rescales a flat per-step value to the size a GAE sum would have:

An episode that never reaches the checkpoint gets \((R - \text{SR})\,G(T)/T\) on every frame. Reaching the checkpoint and then failing gives a positive advantage before the checkpoint and a negative one after, the same shape as the reward (§6).

1122        cp_rate = gcd["n_cp"] / max(gcd["n_total"], 1)
1123        sr = garment_sr.get(garment, 0.5)
1124        cond_sr = (sr / cp_rate) if cp_rate > 0.01 else 0.5
1125        cond_sr = min(cond_sr, 1.0)
1126        garment_baselines[garment] = {
1127            "b_total": sr,                # E[total_return] = success_rate
1128            "b_seg1": cp_rate * 0.5,      # E[seg1_return] = P(reach_cp) × 0.5
1129            "b_seg2": cond_sr - 0.5,      # E[seg2_return | reached_cp]
1130            "cp_rate": cp_rate,
1131            "cond_sr": cond_sr,

1190        if cp_frame is not None and 0 < cp_frame < n - 1:
1191            n1 = cp_frame + 1
1192            n2 = n - n1
1193            R1 = sum(drs[:n1])
1194            R2 = sum(drs[n1:])
1195            seg1 = (R1 - gb["b_seg1"]) * _geom(n1) / max(n1, 1)
1196            seg2 = (R2 - gb["b_seg2"]) * _geom(n2) / max(n2, 1)
1197            for t in range(n1):
1198                seg_values[t] = seg1
1199            for t in range(n1, n):
1200                seg_values[t] = seg2
1201        else:
1202            # No checkpoint: single segment, baseline = success_rate
1203            geom_sum = _geom(n)
1204            seg_per_step = (dense_return - gb["b_total"]) * geom_sum / max(n, 1)
1205            seg_values = [seg_per_step] * n

1207        # Blend: w * raw_GAE_success + (1-w) * raw_segment_per_step
1208        mixed = [w * g + (1.0 - w) * s for g, s in zip(gae_raw, seg_values)]
1209
1210        # GAE_completion applied at full strength (independent of blend weight w).
1211        # Only eligible when completion predictions are available — BC/DAgger episodes
1212        # without predictions get GAE_completion=0 automatically.
1213        if completion_alpha != 0.0 and comp_preds is not None:
1214            gae_c = compute_completion_gae(
1215                comp_preds, n,
1216                completion_alpha=completion_alpha,
1217                gamma=gamma, lam=lam,
1218                success_label=success_label,
1219            )
1220            mixed = [m + c for m, c in zip(mixed, gae_c)]

Ingredient 6 · one scale, one clip, one bonus

The blended advantage is divided by a single standard deviation \(\sigma\), computed over unbiased rollouts whose sampling share is still at least 0.5, and clipped to \([-2, 2]\). The config's beta multiplies \(\sigma\) first; it is 1.0. Then the precision boost (report §6.6): per garment, the top 20% of successful episodes, ranked by how tightly the final frame passed its tightest test, get \(+0.3\) on every frame. Garments with fewer than 5 successes in the window are skipped. That nudges training toward clean folds over borderline ones.

1238    global_std = float(np.std(eligible_vals)) if len(eligible_vals) > 1 else 1.0
1239    if global_std < 1e-8:
1240        global_std = 1.0
1241    raw_std = global_std
1242    global_std *= beta
      ⋮
1266    # ── Step 4: Normalize by global STD and clip ──
1267    result: list[list[float]] = [[] for _ in episodes_info]
1268    for i in range(len(episodes_info)):
1269        result[i] = [max(clip_min, min(clip_max, a / global_std)) for a in mixed_advs[i]]

89    margins: list[float] = []
90    for i, cmp in enumerate(cmps):
91        if success_distance is not None and i < len(success_distance) and success_distance[i] == 0:
92            continue  # stored-threshold=0 → ratio stored as 0, skip
93        if cmp == "<=":
94            margins.append(1.0 - float(cd[i]))
95        else:
96            margins.append(float(cd[i]) - 1.0)
97    if not margins:
98        return float("nan")
99    return float(min(margins))

The margin for a "meet" test is \(1 - d^{(i)}\) and for a "stay apart" test \(d^{(i)} - 1\); the score of an episode is its smallest margin.

What the training run actually used

Every value above is set in the released sim pipeline config. In the first iteration (mode: auto) only segment advantages exist, since there are no recorded estimates yet; GAE is used from the second iteration on.

104# BC dataset (sim teleop demonstrations)
105bc_dataset:
106  root: data/lehome_challenge_merged/four_types_merged
107  repo_id: lehome_bc
108  initial_sampling_share: 1.0
109  decay_factor: 1.0
110  min_sampling_share: 0.9
111  bc_advantage: 0.3  # fixed advantage for BC samples (uniform sampling)
112  use_for_success_labels: false  # don't force success=1 on BC — mask BC out of success/checkpoint/ttc losses
      ⋮
115# Per-dataset sampling_share decays each iteration, clamped to min_sampling_share.
116dagger_dataset:
117  decay_factor: 1.0
118  min_sampling_share: 0.1
119  dagger_advantage: 0.3
      ⋮
135# RL dataset decay
136rl_decay_factor: 0.98
137rl_min_sampling_share: 0.1
138
139# Advantage computation
140# mode: auto = segment for iter 0, gae for iter 1+. Can also force "segment" or "gae"
141advantage:
142  mode: auto
143  gae_lambda: 0.99
144  gamma: 0.999
145  weighting_clipping: [-2.0, 2.0]  # (min, max) for exp(clip(advantage, min, max)); (0,0) = uniform
146  beta: 1.0  # AWR temperature: global_std *= beta before normalizing. Higher = more uniform sampling
147  segment_only_threshold: 0.1  # Datasets with sampling_share < 0.1 auto-switch to segment-only advantages
148  success_alpha: 0.5     # Dampens the P_success transition term in GAE_success delta; 0 removes value from GAE
149  completion_alpha: 0.5  # Weight on GAE_completion (reward-free GAE over completion head); 0 disables
150  value_tail_k: 30       # Linearly interp EMA(P_success) over last K frames to true success label; 0 disables
      ⋮
156precision_boost:
157  boost: 0.3
158  top_k: 0.20
159  min_successes: 5

8AWR, to the core

Advantage-weighted regression (Peng, Kumar, Zhang and Levine, 2019) turns "do more of what worked" into plain supervised learning. This section derives it from the AWR paper, shows how Finn's course frames it, and then shows the one change Ilia makes and the code that implements it.

The problem AWR solves

You have data from some behaviour policy \(\mu\). Here \(\mu\) is the mix of the organisers' demos and every past checkpoint's rollouts. You want a policy \(\pi\) that does better than \(\mu\). The exact gain of switching from \(\mu\) to \(\pi\) is the advantage of \(\mu\) averaged over the states and actions \(\pi\) would visit (AWR Eq. 3):

We don't have states from \(\pi\), only from \(\mu\). So AWR swaps in \(\mu\)'s states, which is accurate only while \(\pi\) stays close to \(\mu\). Closeness becomes a constraint (Eqs. 4–6):

The closed-form answer

Turn the constraint into a penalty with weight \(\beta\), and require \(\pi\) to integrate to 1. Then, state by state, maximise over the distribution \(\pi(\cdot\mid s)\) (Eq. 7 and Appendix A):

Setting the derivative with respect to each \(\pi(a\mid s)\) to zero gives \(A(s,a) - \beta\big(\log\pi - \log\mu + 1\big) - \alpha_s = 0\), so \(\log\pi = \log\mu + A/\beta + \text{const}\):

This is AWR Eq. 8, and it is the whole idea in one line. Take the data's own action distribution and tilt it: actions better than average gain probability in proportion to \(e^{A/\beta}\), worse ones lose it. The temperature \(\beta\) sets how far you may move. Large \(\beta\) stays near \(\mu\); small \(\beta\) piles almost everything onto the best actions seen.

From the ideal policy to a training loss

The ideal \(\pi^*\) cannot be written down for a neural network. So AWR finds the network closest to it, minimising \(D_{\mathrm{KL}}(\pi^*\,\|\,\pi_\theta)\). Because \(\pi^*\) is \(\mu\) times a weight, the expectation over \(\pi^*\) becomes an expectation over the data with that weight attached (Eqs. 9–10):

That is behaviour cloning with a weight on every sample. The paper drops the per-state normaliser \(Z(s)\) in this practical form. Two KL directions appear: the constraint keeps \(\pi\) near \(\mu\) with \(\mathrm{KL}(\pi\|\mu)\), while the projection fits the network with \(\mathrm{KL}(\pi^*\|\pi_\theta)\).

Ilia's version: weight the sampler, not the loss

Ilia keeps AWR's target and changes where the weight is applied (report §2.3). Instead of loading every frame and multiplying its loss by \(e^{A/\beta}\), he draws frames in proportion to their weight and leaves the loss unweighted:

The two are equal in expectation. If \(w_i\) is frame \(i\)'s weight and \(p_i = w_i/\sum_j w_j\), then

which is the weighted-loss gradient times a positive constant. The difference is cost. A frame with weight \(e^{-2}\) is almost never drawn, so its three camera images are never decoded and it never takes a batch slot. Compute goes where the weight is.

621            elif not (src.is_bc or src.is_dagger or src.is_real) and src.use_prioritized_sampling:
622                # RL sources: sample proportional to exp(clip(advantage, clip_min, clip_max))
623                sampling_probs = self._load_priorities(src.root, all_indices, src.advantage_weighting_clipping)

924        valid_advs = all_advs[valid_indices]
925        # Clip and exponentiate
926        clip_min, clip_max = clip_range
927        if clip_min == 0.0 and clip_max == 0.0:
928            return None  # uniform sampling
929        clipped = np.clip(valid_advs, clip_min, clip_max)
930        priorities = np.exp(clipped)
931        # Normalize to probability distribution
932        priorities /= priorities.sum()

Keeping the heads honest

Sampling by advantage tilts the data, which is what the action loss should see. The value heads must not see it that way: the success head has to learn the true chance of success, not the chance among frames AWR liked. So every drawn frame carries an importance weight that undoes the tilt, and the auxiliary losses use it while the action loss ignores it (report §2.3):

\(1/(Np_i)\) cancels the sampling probability. Dividing by the episode's length \(T_{\text{ep}}\) also makes each episode count once in the head losses, however long it ran.

1553        # IS weight: 1/(N*p). Same definition for every source (RL / BC / DAgger);
1554        # ep-len normalisation is applied below so the final is_weight field is
1555        # 1/(N*p*ep_len) — the full unbiased per-episode weight used by all aux losses.
1556        if probs is not None:
1557            chosen = self._rng.choice(len(valid), p=probs)
1558            local_mapped_idx = valid[chosen]
1559            raw_is_weight = np.float32(1.0 / (len(valid) * probs[chosen]))
      ⋮
1822        # Episode-level IS weight: 1/(N*p) / ep_len. Single field used by every aux loss.
1823        item["is_weight"] = raw_is_weight / np.float32(ep_len)

The other sources

Not every frame has an advantage. Frames from the organisers' demonstrations and from DAgger are drawn by how hard their garment still is, \(P \propto e^{3(1-\text{SR}_{\text{garment}})}\), with the same importance weights for the heads (report §2.3). Old rollout datasets fade: each dataset's share of the batch is multiplied by 0.98 whenever new rollouts arrive, and drops out below 0.1. In AWR's terms, that makes \(\mu\) a recency-weighted mixture of past policies.

2880    # Decay existing RL datasets from previous iterations
2881    if has_new_data:
2882        kept = []
2883        for ds in state["rl_datasets"]:
2884            new_share = ds["sampling_share"] * rl_decay
2885            if new_share >= rl_min:
2886                ds["sampling_share"] = round(new_share, 4)
2887                kept.append(ds)
2888            else:
2889                logger.info("Excluding RL dataset (share %.3f < %.3f): %s",
2890                            new_share, rl_min, ds["root"])
2891        state["rl_datasets"] = kept

9RECAP, to the core

RECAP comes from Physical Intelligence's π*_0.6 paper (2025): RL with experience and corrections via advantage-conditioned policies. AWR changes how often the policy sees each moment. RECAP instead tells the policy which moments were good, as an input, and at test time asks for good.

The idea in one equation

Label every training frame with an indicator \(I\) meaning "this action was better than usual". Train one network on frames both with the label (\(\pi(a\mid o, I)\)) and without it (\(\pi(a\mid o)\)). By Bayes' rule, the labelled policy is the unlabelled one tilted by how likely the label is for each action:

If the label depends only on the advantage, \(p(I\mid o,a) = p(I\mid A(o,a))\), this is the same shape as AWR's \(\mu(a\mid s)\,e^{A/\beta}\): the data's actions, reweighted by a rising function of their advantage. The difference is where the tilt lives. AWR applies a fixed curve \(e^{A/\beta}\) to the data. RECAP lets the network learn the tilt from the labels, and the tilt can be switched on or off at inference.

Why it improves the policy

π*_0.6 rests on a result from classifier-free-guidance RL (Frans et al. 2025). Sample from

If \(p(I\mid A)\) increases with \(A\), then \(\hat\pi\) is at least as good as \(\pi_{\text{ref}}\): \(J(\hat\pi)\ge J(\pi_{\text{ref}})\) (π*_0.6 §III). With \(\beta = 1\) this is just "condition on \(I\)". With \(\beta > 1\) the tilt is sharpened, and that is classifier-free guidance, written on velocities in §11.

RECAP as π*_0.6 runs it

From the π*_0.6 paper (arXiv 2511.14759), §III–VI and Appendix A. The paper is internally inconsistent on two small points: whether the threshold is set as a percentile of advantages or of values, and how the indicator relates to the subtask text. Neither changes the picture above.

Ilia's version, as the code does it

Ilia keeps the idea and changes almost every detail (report §4.3).

• No separate value network. The advantage is the one built in §7 from the policy's own heads.
• No threshold. The advantage token is masked (neutral) whenever \(A < 0\). When \(A\ge0\), it is masked at random, less often the larger \(A\) is: the masking probability falls linearly from 0.5 at \(A = 0\) to 0.1 at \(A\ge2\).
• Two channels. The same mask controls a token in the prefix and a vector added to every layer norm of the action expert (AdaRMS, §3).
• Stronger guidance. At inference the token is always on, and guidance runs at 7 to 9.

359        # Mask out (neutral) when advantage < 0; stochastic masking when >= 0.
360        is_negative = advantage < self.config.positive_advantage_threshold
361        # For non-negative: P(neutral) ramps from neutral_prob_at_zero (adv=0)
362        # down to neutral_prob_at_max (adv >= max_adv_for_neutral).
363        t = jnp.minimum(advantage / self.config.max_adv_for_neutral, 1.0)
364        t = jnp.maximum(t, 0.0)
365        p_neutral = (
366            (1.0 - t) * self.config.neutral_prob_at_zero
367            + t * self.config.neutral_prob_at_max
368        )
369        uniform = jax.random.uniform(rng, (batch_size,))
370        use_neutral = is_negative | (uniform < p_neutral)
371
372        token = jnp.where(use_neutral[:, None], 0.0, token)
373        mask = ~use_neutral

810        if (
811            self.config.use_advantage_embedding
812            and advantage_mask is not None
813        ):
814            adv_vec = self.advantage_adarms_vec.value.astype(adarms_cond.dtype)  # [1, D]
815            adv_contrib = jnp.where(
816                advantage_mask[:, None],  # [B, 1]
817                adv_vec,                  # [1, D]
818                jnp.zeros_like(adv_vec),
819            )  # broadcasts to [B, D]
820            adarms_cond = adarms_cond + adv_contrib

Read as a label, the gate gives a token that is present with probability

derived from lines 363–370. At \(A = -0.5\) the token is never on; at \(A = 0\), half the time; at \(A = 1\), 70%; at \(A \ge 2\), 90%. A present token therefore means a non-negative advantage, and it is more often present the larger the advantage. That is the increasing \(p(I\mid A)\) the guarantee above asks for. Demonstrations and DAgger frames carry the fixed advantage 0.3 (§7), so their token is on with probability 0.56 derived.

10AWR and RECAP together

Ilia uses both at once, because they act at different points. AWR acts on the training data: good frames are drawn more often. RECAP acts on the question asked at test time: produce what your positive-advantage frames did. Guidance then pushes further in that direction. The report's Figure 3 makes the point with a toy; the explorer below is that toy, computed exactly.

The toy multiplies densities exactly. A flow-matching policy only approximates this: guidance extrapolates velocities (§11), which matches sampling from the sharpened density only in the limit of a perfect score model. At high scales, real samples can also drift off the action manifold.

11Inference: guidance, chunks, best-of-N

The same weights fold more or less often depending on how they are run (report §7). Nothing here changes the weights. The model server is stateless: each request is one observation in, one chunk out. Anything that looks like memory, such as the chunk being played, the anchor for the next chunk or the latched garment type, lives in the client (the eval worker in sim, the robot program on hardware).

Making one chunk: 10 Euler steps, each with guidance

Every chunk is 30 steps × 12 joints, made by integrating the learned velocity from noise (\(t = 1\)) to clean actions (\(t = 0\)) in \(S = 10\) steps (§3). At each step the action expert runs twice over the same prefix cache: once with the advantage signal on (in both the token and the AdaRMS vector), once with it off. The two velocities are combined:

With \(s = 1\) this is plain conditioning; \(s > 1\) extrapolates past it, in the direction that separates "good" from "any" behaviour. In score terms, \(v_{\text{cond}} - v_{\text{uncond}}\) plays the role of \(\nabla_a \log p(I\mid a)\), and \(s\) is the exponent \(\beta\) of §9. The expensive vision-language pass runs once per chunk; guidance only doubles the cheap action expert.

1471        dt = -1.0 / num_steps
      ⋮
1616        if noise_temp.ndim == 0:
1617            noise = noise * jnp.sqrt(noise_temp)
1618        else:
1619            noise = noise * jnp.sqrt(noise_temp[:, None, None])
      ⋮
1735                cs = cfg_scale
1736                if cs.ndim == 1:
1737                    cs = cs[:, None, None]
1738                v_t = v_uncond + cs * (v_cond - v_uncond)
1739            else:
1740                v_t = v_cond
      ⋮
1745            x_t_new = x_t + dt * v_t

Line 1471 is the step size, 1617 the noise temperature \(\sqrt{\tau}\), 1738 guidance, 1745 the Euler update.

Playing the chunk

Report Table 4 = configs/rl_pipeline_sim.yaml, per garment type. Our runs loaded exactly these values; the worker logged them at the start of every episode.

Soft inpainting keeps consecutive chunks consistent. Without it, each new plan could jump to a different way of doing the fold. The anchor's targets gently pull the new chunk's matching steps toward them during the early, noisy part of denoising. The pull spreads to the rest of the chunk through the action covariance, then switches off for the final steps so the chunk can still correct itself (report §7.2). The bandit preferred light anchoring: some beats none, strong anchoring hurts.

Short execution windows won. The bandit converged to 3 to 5 executed steps out of 30 predicted, which suggests the model's near-term predictions are far more reliable than its distant ones (report §7.7).

Best-of-N, and how often it gives up

With \(N > 1\), the server denoises \(N\) chunks from the same prefix with different seed noise. It scores each by the action-conditioned success residual \(\Delta_{\text{success}}\) (§4), averaged over the conditional and unconditional passes, and plays the best. If every candidate is predicted to make things worse (all scores negative), it draws \(2N\) more and picks from the pooled \(3N\) (report §7.4).

214            def _score(g: dict) -> float:
215                sc = g.get("wm_flow_success_cond")
216                su = g.get("wm_flow_success_uncond")
217                if sc is None or su is None:
218                    return float("-inf")
219                # Pure Δsuccess average (cond/uncond). Already a delta over V̂.
220                return 0.5 * (float(sc) + float(su))

236            retry_needed: list[int] = []
237            for i, grp in enumerate(groups):
238                if n_list[i] <= 1:
239                    continue
240                fin = [s for s in (_score(g) for g in grp) if np.isfinite(s)]
241                if fin and max(fin) < 0.0:
242                    retry_needed.append(i)
      ⋮
252                R_per_group = [2 * n_list[i] for i in retry_needed]

278            for grp in groups:
279                arr = np.asarray([_score(g) for g in grp], dtype=np.float64)
280                finite = np.isfinite(arr)
281                best = int(arr.argmax()) if finite.any() else 0
282                chosen = dict(grp[best])  # shallow-copy so we don't mutate the original
283                if finite.any():
284                    fin = arr[finite]
285                    chosen["best_of_n_score_chosen"] = float(arr[best])

Which garment is this?

The policy takes the garment type as an input, but at test time nobody says what it is. The client works it out (report §7.6):

1. A throwaway first call reads the model's own garment-type prediction from the first frame.
2. The first few plans refine it by majority vote, and these plans also run with a short window of 5 executed steps for every type, so the policy reacts quickly while the type settles.
3. The type is then frozen and fed back as the input for the rest of the episode.

1151                    if gt_memory_current is None:
1152                        warmup_msg = _encode_obs_for_policy(
1153                            decoded,
1154                            self.session_id,
1155                            garment_type_input_id=0,
1156                        )
1157                        await ws_policy.send(json.dumps(warmup_msg))
1158                        warmup_resp = json.loads(await ws_policy.recv())
1159                        warmup_gt = warmup_resp.get("garment_type_pred")
1160                        gt_memory_current = warmup_gt if warmup_gt is not None else 0
1161                        logger.info(
1162                            f"{self.label} ep{current_ep_idx}: "
1163                            f"garment type warmup prediction: {gt_memory_current}"
1164                        )
1165
1166                    if action_chunk is None or chunk_idx >= execute_in_n_steps:

1202                        _FAST_WARMUP_CALLS = 5
1203                        if policy_call_count < _FAST_WARMUP_CALLS:
1204                            effective_inference_config["actions_to_execute"] = 5
1205                            effective_inference_config["execute_in_n_steps"] = 5
1206                        policy_call_count += 1

In our runs the first guess was right on all 16 episodes and never changed: 0 for long tops, 1 for short tops, 2 for long pants, 3 for shorts.

Tuning seven knobs with a bandit

Every knob in the table was tuned per garment type during rollout collection, not by grid search (report §7.7). Each candidate value of each knob is an arm with a Beta posterior, and the knobs are treated independently. For an exploration episode, each knob draws one sample per arm and takes the largest (Thompson sampling). After the episode, its outcome minus the garment type's average success rate updates the chosen arms. Posteriors decay toward uniform each iteration (factor 0.90 in the config) so the bandit follows the moving policy; replay and curriculum episodes are excluded because they are deliberately easier or harder. Once settled, the best values were frozen for the submission.

114    values = []
115    samples = []
116    for v_str, ab in param_prior.items():
117        values.append(float(v_str))
118        samples.append(rng.beta(ab["alpha"], ab["beta"]))
119    best_idx = int(np.argmax(samples))
120    return values[best_idx]

197) -> dict:
198    """Update prior with episode results using Thompson Sampling.
199
200    Uses baseline-subtracted reward ``r = ep_success − SR(garment_type)`` split
201    into non-negative deltas: ``α += max(r, 0)``, ``β += max(−r, 0)``. The
202    per-type baseline matches the optimization objective (total SR across
203    types) and gives a lower-variance baseline than per-garment SR.

12The flywheel: collecting data

Simulation speed limits everything: the report puts one episode at about 30 s in Isaac Sim, without saying on what hardware or for how many steps (report §3). Ours took 1.3 to 5.8 minutes each on our RTX 4080 SUPER (§13). So the collection side exists to get as many useful episodes as possible out of each sim-hour.

The plumbing

• Parallel sims, one server. Each machine runs 3–5 simulator processes, because the competition environment holds one scene per process. Each is a thin client of one shared, stateless policy server; chunks are cached on the client.
• Stop early. An episode stops on success, and a stuck detector ends hopeless ones. A watchdog restarts simulators that hang.
• Record everything at collection time. Every episode is saved with the success and completion estimates the policy made while acting. Those are what §7 consumes, never re-predicted later. Each episode also gets a debug video with live reward, value and advantage traces.

The stuck detector, as the code defines it: over the last 120 frames (4 s on the data's clock, 1.3 s of physics), the joint states, the actions and the success estimate must all have a mean standard deviation below 0.01. The episode is then cut back to keep only the first 30 frames of the stuck stretch.

32STUCK_WINDOW_SECONDS = 4  # Rolling window for stuck detection (seconds)
33STUCK_WINDOW = STUCK_WINDOW_SECONDS * FPS  # 120 frames
34STUCK_THRESHOLD = 0.01  # Std threshold below which state/action is considered stuck
35STUCK_KEEP_SECONDS = 1  # Keep 1 second into the stuck window
36STUCK_KEEP = STUCK_KEEP_SECONDS * FPS  # 30 frames

1537        window = trajectory[-self.STUCK_WINDOW:]
1538        states = np.array([f["observation.state"] for f in window], dtype=np.float32)
1539        actions = np.array([f["action"] for f in window], dtype=np.float32)
1540
1541        s_std = np.mean(np.std(states, axis=0))
1542        a_std = np.mean(np.std(actions, axis=0))
1543
1544        if s_std < self.STUCK_THRESHOLD and a_std < self.STUCK_THRESHOLD:
1545            # Also check value std if available (consistent per-episode)
1546            if window[0].get("success_pred") is not None:
1547                v_std = np.std([f["success_pred"] for f in window])
1548                if v_std >= self.STUCK_THRESHOLD:
1549                    return False  # Value changing → not stuck
1550            return True

Six ways to choose what to collect

Report Table 1. Replays and hard mining are deliberately easier or harder than a fresh episode, so they still train the policy but are kept out of the success statistics, the advantage baselines and the bandit.

Saving states worth replaying

Replays need saved physics states: every cloth particle's position and velocity, plus the joints.

• Success states are saved at step 5 of episodes that end in a fold, with probability \(1 - \text{SR}\), so rare successes are always kept.
• Failure states are saved at the moment the policy visibly ruins a promising episode. The rule: the smoothed success estimate drops more than 0.12 below its running maximum, after that maximum has passed 0.25, ignoring the first 3 plans.
• Semi-success states come from failed episodes that reached the first checkpoint or held a high estimate. They can also go to the human DAgger queue.

244    EMA_ALPHA = _EMA_ALPHA    # EMA smoothing on chunk-level values
245    EMA_SKIP_CHUNKS = 3       # skip first ~3 chunks (~2 seconds) to avoid initial spike
246    EMA_DROP_THRESH = 0.12    # EMA must drop this much from running max
247    EMA_PEAK_THRESH = 0.25    # running max must exceed this before drop counts
      ⋮
1301                        if chunk_value is not None and ep_restore_data is None:
1302                            ema_chunk_count += 1
1303                            ema_value = self.EMA_ALPHA * chunk_value + (1 - self.EMA_ALPHA) * ema_value
1304                            if ema_chunk_count > self.EMA_SKIP_CHUNKS:
1305                                ema_max = max(ema_max, ema_value)
1306                                drop = ema_max - ema_value
1307                                if (drop > self.EMA_DROP_THRESH
1308                                        and ema_max > self.EMA_PEAK_THRESH
1309                                        and failure_frame is None):
1310                                    failure_frame = frame_idx
1311                                    failure_state_requested = True

Augmenting the simulator

Augmentation is applied at two scopes (report §3.3). Per episode: garment texture swaps, colour remaps, small changes to the garment's pose, scale and roughness, camera position and focal length, robot base position, table texture and light direction. Per step: garment tint, arm colour and light colour. Normal rollouts used a light regime, enough variety without hurting the live policy: texture swap and colour remap with probability 0.4, ±2 cm and ±2° of garment pose, ±5% scale. Success replays used much stronger settings, since the policy's performance there does not matter.

The training loop

1. Download whatever new rollouts have arrived.
2. Recompute advantages over every rollout dataset (§7).
3. Train about 1,000 steps, drawing frames from all sources at once (§8).
4. Upload a checkpoint every 500 steps.

Datasets are never merged; the mix is a setting. Each new rollout dataset enters at share 1.0 and fades by 0.98 per update. The demonstrations keep a fixed share (no decay in the config), and a few old successful datasets were kept at a higher weight for the hardest garments (report §2.4).

Rollbacks. Three times in the main round, and once in the second, Ilia went back to a checkpoint from a few days earlier and retrained it on everything collected since. The rolled-back model sees a large, varied, partly off-policy batch at once, and this reliably got training out of local optima (report §2.5). π*_0.6 does the same thing systematically, restarting every round from the pre-trained checkpoint.

Report §2.6 and §8.2. Ilia does not consider this sample-efficient. He suspects the missing ingredient is recovery behaviour: the scripted demonstrations had little exploration, so the policy struggled to find better behaviour quickly.

13Results, and what we measured

The online round

Report Table 5, top five of 62 teams. Each score is over 20 garments per type, 10 episodes each (§1).

The win was broad: first outright on short tops, long pants and shorts, third on long tops (behind Dum-E's 76.5% and a 77.0% from a team outside this top five), and 6.1 points clear overall. Shorts were the easiest type and short tops the hardest, for Ilia and for most of the leaderboard. Short tops need small sleeves folded to tight tolerances, and they were still improving when he submitted (report §8.1). The released lehome_sim checkpoint is the one behind this result. Ilia also notes that the 8 private unseen garments per type scored only slightly below the trained ones (report §10.1).

The failures that remained fall into three groups (report §8.2):

• Precision: the fold looks right, but one distance ends just on the wrong side of its threshold.
• Simulator physics: a grasp fails or the garment slips out of the gripper.
• No recovery: after a mistake, the policy rarely works its way back.

What we measured on our RTX 4080 SUPER

14The real-robot round

The final round, at ICRA 2026, ran the top 8 teams' policies on a physical robot. Ilia had about three months for simulation and a little over one week for this (report §9). It was second place, with 865 of 1,080 points, behind 895.

Why the sim policy did not transfer

Zero-shot transfer failed, and Ilia's evidence points to overfitting on rendering details. His model resizes each 640×480 image to 224×224. Resizing in two steps instead, to 320×240 first and then to 224×224, is a change people can barely see, yet it cut the simulation success rate significantly. The model's heads could tell the two resizings apart perfectly (report §9.1). The real data also folded some garments differently from the simulation, and Ilia never had the evaluation robot itself: the transfer was sim, to his robot, to theirs.

The recipe

1. Start from a strong but not the latest sim checkpoint, the latest being the most specialised to the simulator.
2. Strip the sim-only machinery. Still trained: the action loss, the garment-type head and the completion head. Loaded but frozen: success, checkpoint and time to completion. Removed: the keypoint head, both future-prediction heads, the advantage token and AdaRMS vector, and therefore guidance and best-of-N. Real inference is the plain path: one denoised chunk, soft inpainting and the garment-type bootstrap.
3. Fine-tune on three buckets of data (table below).
4. Align the rigs: a camera-overlay tool drives the robot to a dataset frame's joint state and overlays the live cameras on that frame.
5. Augment heavily: per-camera colour, gain and gamma, blur, noise, crop, rotation and zoom, cutout, camera dropout, and noise or dropout on the joint state so the policy trusts images over a miscalibrated arm.
6. Match motion speed across sources (below).
7. Collect DAgger data and weight it by closeness to an intervention (below).

Report Table 6.

Speed factors: matching motion per step

The model predicts joint changes over a fixed number of steps. A slow source teaches small changes per step and a fast one large changes, so mixing them naively teaches an inconsistent idea of how far to move in one step. Each source's time axis is resampled before training: organiser data ×1.0, home teleop ×1.5, DAgger corrections ×2.0, sim rollouts ×0.65 (report §9.9). A factor above 1 compresses a slow source; below 1 stretches a fast one. The RL-trained sim policy was the fastest of all.

DAgger weighting without a value function

On the real robot there is no checker, so no reward, value or advantage. Ilia used the interventions themselves as the signal (report §9.10):

• frames where the human was driving get the highest sampling weight;
• autonomous frames far from any intervention get a low weight;
• autonomous frames in the 5 s before a takeover ramp down to zero, so the moves that caused the failure are not reinforced.

He calls this a deliberately crude stand-in for a real advantage function. A learned value on the real robot would bring back conditioning, guidance and best-of-N.

The units bug

Between two LeRobot 0.4.x releases, the SO-101's default joint representation changed from a −100…100 range-of-motion scale to degrees. Across a roughly 200° range the two look almost the same, yet the mismatch skews every joint by about 10%. Ilia found it two days before the deadline and fixed it by post-processing (report §9.11).

15What is still open

Ilia names two problems himself (report §10.4–10.5).

Levers for us

16Notation decoder

15ACTION_HORIZON = 30  # 1 second at 30 FPS
      ⋮
26EMA_ALPHA = 0.2  # EMA smoothing for P(success) prediction

Where each idea lives in the code

All paths are in IliaLarchenko/lehome_solution at ca73828, under src/lehome_solution/ unless they start with scripts/ or configs/.

17Misreadings to avoid

1. "AWR weights the loss." Not here. The loss is unweighted; the weight decides how often a frame is drawn (§8). Only the value heads use a weight, and that one undoes the tilt.
2. "The advantage token is a binary threshold, as in π*_0.6." It is a random gate: never on for negative advantage, on 50% to 90% of the time for non-negative advantage, more often the higher the advantage (§9).
3. "Dense reward changes what the policy is rewarded for." Every episode's reward still sums to exactly 1 for a fold and 0 otherwise. The checkpoints only move credit earlier (§6).
4. "The success head is a separate critic." It is a linear readout of the policy's own representation, from pixels only (§4).
5. "Best-of-N finds the best chunk." In our run, 55% of plans found every candidate predicted to be worse than expected, and the chosen score was near zero on median. Its value is mostly in avoiding bad chunks (§11).
6. "30 steps is one second." On the data's clock, yes. In the simulator's physics, 30 steps are a third of a second (§0).
7. "The episode stops the moment the garment is folded." The checker runs every 30 steps in the code we ran, so the fold is noticed up to 29 steps late (§1).
8. "Same seed, same result." Our repeat of the same 16 episodes did not reproduce every outcome (§13).
9. "The real-robot policy uses all of this." The real round dropped advantage conditioning, guidance and best-of-N. It was behaviour cloning on mixed data, with intervention-weighted sampling (§14).

·Sources and checks