Slither OpenWAM continued pretraining
The six-H200 run completed one shuffled pass at step 1,630. The final weights are checkpoint_step_1630.safetensors in this repository. The training bucket also holds rolling latest and best saved weights, identified by latest.json and best.json.
The scientific archive preserves metrics, per-window validation, exact window indices, dataset revision, deployed source, upstream patches, environment, logs and checksums. Final optimizer/scheduler/RNG states were deleted by upstream cleanup; these weights initialize a continuation with a fresh optimizer rather than an exact resume.
The model initializes from OpenWAM-α at revision 52df4e66c82c5c8b480adcc8d01f4db7415dfb56. Its Slither adapter replaces the 80-dimensional robot action head with three outputs: cosine and sine of estimated mouse angle, and binary boost. It removes robot proprioception and uses fixed empty-prompt conditioning. Each example uses 33 frames at 30 Hz and 32 transition actions; the nine video frames are sampled every fourth frame.
The source dataset contains actions estimated from optical flow. The angle is a two-frame-ahead heading proxy, not a recorded mouse position; boost is inferred from apparent motion. Validation uses fixed windows from 47 held-out media items representing 44 unique YouTube sources at step zero, every 100 steps, and at the end. These losses measure fit to estimated labels, not playable rollout quality.
The final independent comparison evaluates four windows per held-out media item (188 windows) with matched windows and noise seeds:
| Initialization | Total loss | Video loss | Action loss |
|---|---|---|---|
| Adapted OpenWAM-α baseline | 2.15437 | 0.57587 | 1.57849 |
| Slither step 1,630 | 0.32795 | 0.28164 | 0.04630 |
The baseline has newly initialized Slither action input/output projections. Its action loss is not the original robot model's native task performance. This is one training run; no multi-seed uncertainty, heading accuracy, boost accuracy or live-play performance is established. Final live validation on the smaller 47-window set was 0.34002 total, 0.28074 video and 0.05928 action. These sample sets have different losses and should not be conflated.
Use the OpenWAM checkpoint loader with the Slither adapter in the project’s wam_training directory. The checkpoint is not a standalone Transformers model.
- Downloads last month
- 58