26 lines
1.1 KiB
Markdown
26 lines
1.1 KiB
Markdown
# 2026-04-06 LEWM ViT Transfer Notes
|
|
|
|
## Root-cause fix
|
|
|
|
The first LEWM runs were stopped because the data path still resized each camera view to `224x224` **before** multiview fusion. That preserved the final tensor shape but broke the original LEWM geometry.
|
|
|
|
Corrected path now is:
|
|
|
|
- **Training dataset**: keep stored per-view `256x256` images (`data.image_resize_shape=null` at launch; dataset instantiate override is `None` for LEWM)
|
|
- **Eval rollout input**: resize live MuJoCo `480x640` camera images to `256x256` per view
|
|
- **Backbone**: fuse `front, top, r_vis` on the LEWM axis, then resize fused short side to `224`
|
|
|
|
## Verification
|
|
|
|
- Local tests passed (`38 passed` across the focused suite)
|
|
- Remote check:
|
|
- dataset sample image shape: `(2, 3, 256, 256)`
|
|
- eval-prepared live frame shape: `(3, 256, 256)`
|
|
- Remote smoke passed with real checkpoint:
|
|
- `smoke-lewm-imf-rawpath-emb384-20260406-002002`
|
|
|
|
## Current runs
|
|
|
|
- `lewm-vit-imf-raw256fix-sim-transfer-emb384-l12-ph16-ex08-step50k-roll10-5880g0-20260406-002124`
|
|
- `lewm-vit-imf-raw256fix-sim-transfer-emb256-l12-ph16-ex08-step50k-roll10-5880g1-20260406-002124`
|