arabellako22/microduck-walk-seed42

Microduck flat-walking PPO — Colab study run

This repository documents a personal study run that installs Pollen Robotics' Microduck simulator, trains a flat-ground walking policy with PPO, exports the normalized policy to ONNX, and validates the release artifacts before upload.

Training setup

• Task: Mjlab-Velocity-Flat-MicroDuck
• Algorithm: PPO via rsl_rl
• Simulator: mjlab / MuJoCo Warp
• Parallel environments: 4096
• Training iterations: 4000
• Seed: 42
• Source revision: 29e887ecfbf5
• Export: official scripts/export.py; observation normalizer baked into ONNX

Validation

• ONNX contract: [1, 61] -> [1, 14]
• Publisher smoke test: finite and non-constant output
• Simulation video: 8.00 seconds, rendered with egl
• Commanded forward velocity: 0.50 m/s
• Measured mean forward velocity: 0.247 m/s
• Terminations during rollout: 0
• Policy SHA-256: 82a2369fe642eda505ee6d60c7ed26aafd177eb7484030834cd04e5a53035213
• Video SHA-256: c73cb27596c2a7c87282e7bcb0583647cc44539928d18e74f8ce8bfbc04704ee

The video was reviewed before publication as a qualitative success check. This run has not been tested on a physical Microduck. One video and one seed are not a statistical performance benchmark.

Multi-seed headless evaluation

The uploaded policy.onnx was evaluated without rendering across 10 deterministic evaluation seeds. This tests one policy trained with seed 42; it is not a multiple-training-seed study.

Eval seed	Mean reward/step	Mean velocity (m/s)	Mean absolute velocity error (m/s)	Terminations	Success
100	0.1413	0.236	0.264	0	✅
101	0.1487	0.271	0.232	0	✅
102	0.1460	0.225	0.276	0	✅
103	0.1482	0.291	0.212	0	✅
104	0.1444	0.227	0.274	0	✅
105	0.1447	0.221	0.279	0	✅
106	0.1480	0.276	0.228	0	✅
107	0.1524	0.250	0.250	0	✅
108	0.1500	0.290	0.215	0	✅
109	0.1540	0.295	0.207	0	✅

Aggregate results

Metric	Result
Evaluation seeds	10 (100–109)
Steps per seed	400
Mean reward per step	0.1478 ± 0.0036
Mean forward velocity	0.258 ± 0.028 m/s
Mean absolute velocity error	0.244 ± 0.027 m/s
Total terminations	0
Success rate without termination	100.0%

Raw, machine-readable results: evaluation.json and evaluation_by_seed.csv. No new video was generated because the policy did not change.

Files

• policy.onnx — deployable policy with observation normalization
• manifest.json — Microduck policy manifest
• training_config.json — reproducibility settings
• validation.json — release checks and hashes
• render_metrics.json — finite-rollout reward, velocity, and termination summary
• replay.mp4 — qualitative simulation result

Local inference

uv run scripts/infer_policy.py --walking policy.onnx --new-cmd-obs

References

• Pollen Robotics microduck_rl
• PPO
• BAM servo friction models
• Dynamics randomization
• Massively parallel locomotion learning