feat: add vision transfer backbones and IMF variants

This commit is contained in:
Logic
2026-04-09 14:02:24 +08:00
parent d51b3ecafa
commit ff7c9c1f2a
58 changed files with 2788 additions and 26 deletions

View File

@@ -0,0 +1,23 @@
# PLAN
## Goal
Train a 50k-step IMF baseline with the original ResNet vision backbone, using `front` + `r_vis` cameras only.
## Fixed comparison contract
- Same hyperparameters as the active top/front and front-only runs
- Agent: `resnet_imf_attnres`
- Vision backbone mode: `resnet`
- `pred_horizon=16`, `num_action_steps=8`
- `n_emb=384`, `n_layer=12`, `n_head=1`, `n_kv_head=1`
- `inference_steps=1`
- `batch_size=80`, `lr=2.5e-4`, cosine warmup 2000
- dataset: `/home/droid/sim_dataset/sim_transfer`
- cameras: `[r_vis, front]`
- rollout every 5 epochs with 5 episodes, headless
## Important dimension override
- Two-camera visual cond dim = `64*2 + 16 = 144`, so set `agent.num_cams=2`, `agent.head.cond_dim=144`.
## Resource plan
- Host: `100.119.99.14`
- GPU: `1`