Instructions to use yrpark/pi05-ur7e-pour_cup-train-env with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LeRobot
How to use yrpark/pi05-ur7e-pour_cup-train-env with LeRobot:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
lerobot pi0.5 training environment for DexSteer/isaaclab-ur7e-pour_cup
Browse files- .gitignore +1 -0
- README.md +16 -6
- train.sh +7 -3
- upload_to_hf.sh +1 -1
.gitignore
CHANGED
|
@@ -4,3 +4,4 @@ outputs/
|
|
| 4 |
wandb/
|
| 5 |
__pycache__/
|
| 6 |
*.log
|
|
|
|
|
|
| 4 |
wandb/
|
| 5 |
__pycache__/
|
| 6 |
*.log
|
| 7 |
+
logs/
|
README.md
CHANGED
|
@@ -1,3 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# lerobot ฯโ.โ
(pi05) fine-tuning ํ๊ฒฝ โ `DexSteer/isaaclab-ur7e-pour_cup`
|
| 2 |
|
| 3 |
UR7e + RH5D ํธ๋(Isaac Lab) ๋ฐ์ดํฐ์
์ผ๋ก ฯโ.โ
๋ฅผ fine-tuning ํ๊ธฐ ์ํ **์ฌํ ๊ฐ๋ฅํ ์ค์น/ํ์ต ์คํฌ๋ฆฝํธ**์
๋๋ค.
|
|
@@ -27,9 +36,9 @@ cd /data/pi05-ur7e
|
|
| 27 |
# 2) ์ค์น (uv ์๋ ์ค์น โ Python 3.12 venv โ lerobot[pi,training]==0.6.1 + torch cu128 โ ๊ฒ์ฆ)
|
| 28 |
bash setup.sh
|
| 29 |
|
| 30 |
-
# 3) HF
|
| 31 |
# https://huggingface.co/google/paligemma-3b-pt-224 ์์ ๋ผ์ด์ ์ค ๋์ ํ
|
| 32 |
-
export HF_TOKEN=hf_xxx
|
| 33 |
|
| 34 |
# 4) ๋ฐ์ดํฐ์
(0.33 GB) + pi05_base(14 GB) + ํ ํฌ๋์ด์ ๋ฏธ๋ฆฌ ๋ฐ๊ธฐ
|
| 35 |
bash download.sh
|
|
@@ -50,7 +59,7 @@ NUM_GPUS=4 bash train.sh # GPU 4๊ฐ (DDP)
|
|
| 50 |
| `env.sh` | ๊ณตํต ํ๊ฒฝ ๋ณ์. `WORKDIR`, HF ์บ์ ๊ฒฝ๋ก, `PYTHONPATH` ์ ๊ฑฐ, venv ํ์ฑํ |
|
| 51 |
| `setup.sh` | uv ์ค์น โ venv ์์ฑ โ lerobot ์ค์น โ `scripts/check_env.py` ๋ก ๊ฒ์ฆ |
|
| 52 |
| `download.sh` | ๋ฐ์ดํฐ์
/ `lerobot/pi05_base` / PaliGemma ํ ํฌ๋์ด์ ์ฌ์ ๋ค์ด๋ก๋ ๋ฐ ๋์ฝ๋ฉ ํ
์คํธ |
|
| 53 |
-
| `train.sh` | ํ์ต ์คํ. GPU ์๋ฅผ ๊ฐ์งํด ๋จ์ผ GPU / `accelerate launch` DDP ์๋ ๋ถ๊ธฐ. `RESUME=1` ์ง์ |
|
| 54 |
| `scripts/check_env.py` | ๋ฒ์ ยทCUDAยท๋น๋์ค ๋ฐฑ์๋ยทHF ํ ํฐ ์ ๊ฒ |
|
| 55 |
| `scripts/smoke_test.py` | ํ ํฌ๋์ด์ ์์ด pi05_base ๋ฅผ ๋ก๋ํด forward/backward ๋ฉ๋ชจ๋ฆฌยท์๋ ์ธก์ |
|
| 56 |
| `upload_to_hf.sh` | ์ด ํด๋๋ฅผ HF Hub ์ ์
๋ก๋ (`.venv`, `cache`, `outputs` ์ ์ธ) |
|
|
@@ -122,7 +131,8 @@ lerobot-train \
|
|
| 122 |
|
| 123 |
โ 80 GB ์นด๋๋ `BATCH_SIZE=32`(๊ธฐ๋ณธ) ๋๋ 64, 40/48 GB ์นด๋๋ `BATCH_SIZE=16` (+ ํ์ ์ `TRAIN_EXPERT_ONLY=true`).
|
| 124 |
**gradient checkpointing ์ ๋์ง ๋ง์ธ์.** ๋๋ฉด 80 GB ์์๋ batch 16 ์ด OOM ๋ฉ๋๋ค(4.1B ํ๋ผ๋ฏธํฐ + AdamW ์ํ๋ง ~33 GiB).
|
| 125 |
-
|
|
|
|
| 126 |
|
| 127 |
### ๋ค์ค GPU (2~4๊ฐ)
|
| 128 |
|
|
@@ -141,7 +151,7 @@ NUM_GPUS=2 BATCH_SIZE=64 bash train.sh
|
|
| 141 |
### ์ฌ๊ฐ / ๊ธฐํ
|
| 142 |
|
| 143 |
```bash
|
| 144 |
-
RESUME=1 bash train.sh # outputs/<job>/checkpoints/last ์์ ์ฌ๊ฐ
|
| 145 |
JOB_NAME=exp2 TRAIN_EXPERT_ONLY=true bash train.sh # ๋ณ๋ job, VLM ๋๊ฒฐ + action expert ๋ง ํ์ต
|
| 146 |
WANDB_ENABLE=true WANDB_PROJECT=pi05 bash train.sh
|
| 147 |
EXTRA_ARGS="--policy.rtc_training_max_delay=10" bash train.sh # ํ์ต-์ RTC (real-time chunking)
|
|
@@ -156,7 +166,7 @@ COMPILE=true bash train.sh # torch.compile (์ฒซ step ์ปดํ์ผ
|
|
| 156 |
2. **CUDA ํ ** โ ๋๋ผ์ด๋ฒ๊ฐ CUDA 12.8 ๋ฏธ๋ง์ด๋ฉด `TORCH_BACKEND=cu126 bash setup.sh`, ์ต์ ์ด๋ฉด `cu130` ๋๋ `auto`.
|
| 157 |
3. **ffmpeg** โ `apt install ffmpeg` (Ubuntu 22.04/24.04 ์ ffmpeg ๋ libdav1d ํฌํจ). ์ค์น ๋ถ๊ฐํ๋ฉด `VIDEO_BACKEND=pyav bash train.sh` (pyav ๋ ์์ฒด ffmpeg ๋ฅผ ๋ด์ฅ, ์ฝ 25% ๋๋ฆฌ์ง๋ง ์ถฉ๋ถ).
|
| 158 |
4. **Python 3.12** โ ์์คํ
์ ์์ด๋ `uv venv --python 3.12` ๊ฐ ์๋ ๋ค์ด๋ก๋ํฉ๋๋ค.
|
| 159 |
-
5. **HF ํ ํฐ** โ PaliGemma ๋ผ์ด์ ์ค ๋์ + `HF_TOKEN`. ์์ผ๋ฉด `download.sh`/`train.sh` ๊ฐ ํ ํฌ๋์ด์ ๋จ๊ณ์์ ์คํจํฉ๋๋ค. ํ ํฐ์ด `read` ๊ถํ๋ง ์์ด๋ ํ์ต์ ๊ฐ๋ฅํ๊ณ , `upload_to_hf.sh` ์๋ `write` ๊ฐ ํ์ํฉ๋๋ค.
|
| 160 |
6. **์คํ๋ผ์ธ ๋
ธ๋** โ ์จ๋ผ์ธ ๋
ธ๋์์ `bash download.sh` ๋ฅผ ๋จผ์ ๋๋ฆฌ๊ณ `cache/` ๋ฅผ ํต์งธ๋ก ๋ณต์ฌํ ๋ค `HF_HUB_OFFLINE=1 bash train.sh`.
|
| 161 |
7. **lerobot main ๋ธ๋์น**(`--parallelism.dp_shard` ๋ฑ FSDP ์ ๊ธฐ๋ฅ)๊ฐ ํ์ํ๋ฉด `LEROBOT_SOURCE=git bash setup.sh`. ๋จ, ๊ทธ ๊ฒฝ์ฐ ๋ค์ค GPU ์คํ ๋ฐฉ์์ด `torchrun`/`--parallelism.*` ๋ก ๋ฐ๋๋ฏ๋ก `train.sh` ์ accelerate ๋ถ๊ธฐ๋ฅผ ์์ ํด์ผ ํฉ๋๋ค.
|
| 162 |
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- lerobot
|
| 5 |
+
- pi05
|
| 6 |
+
- ur7e
|
| 7 |
+
- training-environment
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
# lerobot ฯโ.โ
(pi05) fine-tuning ํ๊ฒฝ โ `DexSteer/isaaclab-ur7e-pour_cup`
|
| 11 |
|
| 12 |
UR7e + RH5D ํธ๋(Isaac Lab) ๋ฐ์ดํฐ์
์ผ๋ก ฯโ.โ
๋ฅผ fine-tuning ํ๊ธฐ ์ํ **์ฌํ ๊ฐ๋ฅํ ์ค์น/ํ์ต ์คํฌ๋ฆฝํธ**์
๋๋ค.
|
|
|
|
| 36 |
# 2) ์ค์น (uv ์๋ ์ค์น โ Python 3.12 venv โ lerobot[pi,training]==0.6.1 + torch cu128 โ ๊ฒ์ฆ)
|
| 37 |
bash setup.sh
|
| 38 |
|
| 39 |
+
# 3) HF ๋ก๊ทธ์ธ (PaliGemma ํ ํฌ๋์ด์ ๊ฐ gated ๋ผ์ ๋ฐ๋์ ํ์)
|
| 40 |
# https://huggingface.co/google/paligemma-3b-pt-224 ์์ ๋ผ์ด์ ์ค ๋์ ํ
|
| 41 |
+
hf auth login # ๋๋ export HF_TOKEN=hf_xxx (env.sh ๊ฐ ๋ก๊ทธ์ธ ํ ํฐ์ ์๋์ผ๋ก ์ฐพ์ต๋๋ค)
|
| 42 |
|
| 43 |
# 4) ๋ฐ์ดํฐ์
(0.33 GB) + pi05_base(14 GB) + ํ ํฌ๋์ด์ ๋ฏธ๋ฆฌ ๋ฐ๊ธฐ
|
| 44 |
bash download.sh
|
|
|
|
| 59 |
| `env.sh` | ๊ณตํต ํ๊ฒฝ ๋ณ์. `WORKDIR`, HF ์บ์ ๊ฒฝ๋ก, `PYTHONPATH` ์ ๊ฑฐ, venv ํ์ฑํ |
|
| 60 |
| `setup.sh` | uv ์ค์น โ venv ์์ฑ โ lerobot ์ค์น โ `scripts/check_env.py` ๋ก ๊ฒ์ฆ |
|
| 61 |
| `download.sh` | ๋ฐ์ดํฐ์
/ `lerobot/pi05_base` / PaliGemma ํ ํฌ๋์ด์ ์ฌ์ ๋ค์ด๋ก๋ ๋ฐ ๋์ฝ๋ฉ ํ
์คํธ |
|
| 62 |
+
| `train.sh` | ํ์ต ์คํ. GPU ์๋ฅผ ๊ฐ์งํด ๋จ์ผ GPU / `accelerate launch` DDP ์๋ ๋ถ๊ธฐ. `RESUME=1` ์ง์. ๋ก๊ทธ๋ `logs/<job>.log` |
|
| 63 |
| `scripts/check_env.py` | ๋ฒ์ ยทCUDAยท๋น๋์ค ๋ฐฑ์๋ยทHF ํ ํฐ ์ ๊ฒ |
|
| 64 |
| `scripts/smoke_test.py` | ํ ํฌ๋์ด์ ์์ด pi05_base ๋ฅผ ๋ก๋ํด forward/backward ๋ฉ๋ชจ๋ฆฌยท์๋ ์ธก์ |
|
| 65 |
| `upload_to_hf.sh` | ์ด ํด๋๋ฅผ HF Hub ์ ์
๋ก๋ (`.venv`, `cache`, `outputs` ์ ์ธ) |
|
|
|
|
| 131 |
|
| 132 |
โ 80 GB ์นด๋๋ `BATCH_SIZE=32`(๊ธฐ๋ณธ) ๋๋ 64, 40/48 GB ์นด๋๋ `BATCH_SIZE=16` (+ ํ์ ์ `TRAIN_EXPERT_ONLY=true`).
|
| 133 |
**gradient checkpointing ์ ๋์ง ๋ง์ธ์.** ๋๋ฉด 80 GB ์์๋ batch 16 ์ด OOM ๋ฉ๋๋ค(4.1B ํ๋ผ๋ฏธํฐ + AdamW ์ํ๋ง ~33 GiB).
|
| 134 |
+
์ค์ `lerobot-train` 30 step ๊ฒ์ฆ(batch 16, ํ ํฌ๋์ด์ ํฌํจ): 2.47 s/step, 37 GiB, loss 0.70 โ 0.14, ์ฒดํฌํฌ์ธํธ ์ ์ฅ ์ ์.
|
| 135 |
+
๋จ์ผ GPU ๋ก 30k step(batch 32) ์ ์ฝ 42 h ์ด๋ฏ๋ก ๋ค์ค GPU ๊ถ์ฅ.
|
| 136 |
|
| 137 |
### ๋ค์ค GPU (2~4๊ฐ)
|
| 138 |
|
|
|
|
| 151 |
### ์ฌ๊ฐ / ๊ธฐํ
|
| 152 |
|
| 153 |
```bash
|
| 154 |
+
RESUME=1 bash train.sh # outputs/<job>/checkpoints/last ์์ ์ฌ๊ฐ (๊ฐ์ JOB_NAME ์ผ๋ก ์๋ก ์์ํ๋ฉด ๋ฎ์ด์ฐ๊ธฐ ๋ฐฉ์ง๋ก ๊ฑฐ๋ถ๋จ)
|
| 155 |
JOB_NAME=exp2 TRAIN_EXPERT_ONLY=true bash train.sh # ๋ณ๋ job, VLM ๋๊ฒฐ + action expert ๋ง ํ์ต
|
| 156 |
WANDB_ENABLE=true WANDB_PROJECT=pi05 bash train.sh
|
| 157 |
EXTRA_ARGS="--policy.rtc_training_max_delay=10" bash train.sh # ํ์ต-์ RTC (real-time chunking)
|
|
|
|
| 166 |
2. **CUDA ํ ** โ ๋๋ผ์ด๋ฒ๊ฐ CUDA 12.8 ๋ฏธ๋ง์ด๋ฉด `TORCH_BACKEND=cu126 bash setup.sh`, ์ต์ ์ด๋ฉด `cu130` ๋๋ `auto`.
|
| 167 |
3. **ffmpeg** โ `apt install ffmpeg` (Ubuntu 22.04/24.04 ์ ffmpeg ๋ libdav1d ํฌํจ). ์ค์น ๋ถ๊ฐํ๋ฉด `VIDEO_BACKEND=pyav bash train.sh` (pyav ๋ ์์ฒด ffmpeg ๋ฅผ ๋ด์ฅ, ์ฝ 25% ๋๋ฆฌ์ง๋ง ์ถฉ๋ถ).
|
| 168 |
4. **Python 3.12** โ ์์คํ
์ ์์ด๋ `uv venv --python 3.12` ๊ฐ ์๋ ๋ค์ด๋ก๋ํฉ๋๋ค.
|
| 169 |
+
5. **HF ํ ํฐ** โ PaliGemma ๋ผ์ด์ ์ค ๋์ + `hf auth login`(๋๋ `HF_TOKEN`). `env.sh` ๊ฐ `~/.cache/huggingface/token` ์ `HF_HOME` ์ผ๋ก ๋ณต์ฌํด ์ค๋๋ค. ์์ผ๋ฉด `download.sh`/`train.sh` ๊ฐ ํ ํฌ๋์ด์ ๋จ๊ณ์์ ์คํจํฉ๋๋ค. ํ ํฐ์ด `read` ๊ถํ๋ง ์์ด๋ ํ์ต์ ๊ฐ๋ฅํ๊ณ , `upload_to_hf.sh` ์๋ `write` ๊ฐ ํ์ํฉ๋๋ค.
|
| 170 |
6. **์คํ๋ผ์ธ ๋
ธ๋** โ ์จ๋ผ์ธ ๋
ธ๋์์ `bash download.sh` ๋ฅผ ๋จผ์ ๋๋ฆฌ๊ณ `cache/` ๋ฅผ ํต์งธ๋ก ๋ณต์ฌํ ๋ค `HF_HUB_OFFLINE=1 bash train.sh`.
|
| 171 |
7. **lerobot main ๋ธ๋์น**(`--parallelism.dp_shard` ๋ฑ FSDP ์ ๊ธฐ๋ฅ)๊ฐ ํ์ํ๋ฉด `LEROBOT_SOURCE=git bash setup.sh`. ๋จ, ๊ทธ ๊ฒฝ์ฐ ๋ค์ค GPU ์คํ ๋ฐฉ์์ด `torchrun`/`--parallelism.*` ๋ก ๋ฐ๋๋ฏ๋ก `train.sh` ์ accelerate ๋ถ๊ธฐ๋ฅผ ์์ ํด์ผ ํฉ๋๋ค.
|
| 172 |
|
train.sh
CHANGED
|
@@ -21,7 +21,7 @@
|
|
| 21 |
# TRAIN_EXPERT_ONLY (false) true ๋ฉด VLM ๋๊ฒฐ, action expert ๋ง ํ์ต (๋ฉ๋ชจ๋ฆฌ โ)
|
| 22 |
# N_ACTION_STEPS (25) ์ถ๋ก ์ ์คํํ action step ์ (50fps ๊ธฐ์ค 0.5s). ํ์ต์๋ ์ํฅ ์์
|
| 23 |
# WANDB_ENABLE (false) WANDB_PROJECT, WANDB_ENTITY ๋ ์ฌ์ฉ
|
| 24 |
-
# OUTPUT_DIR ($WORKDIR/outputs/$JOB_NAME)
|
| 25 |
set -euo pipefail
|
| 26 |
cd "$(dirname "$0")"
|
| 27 |
source ./env.sh
|
|
@@ -104,5 +104,9 @@ echo "== output: $OUTPUT_DIR"
|
|
| 104 |
printf '%q ' "${CMD[@]}" "${ARGS[@]}"; echo
|
| 105 |
[ "${DRY_RUN:-0}" = "1" ] && exit 0
|
| 106 |
|
| 107 |
-
|
| 108 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
# TRAIN_EXPERT_ONLY (false) true ๋ฉด VLM ๋๊ฒฐ, action expert ๋ง ํ์ต (๋ฉ๋ชจ๋ฆฌ โ)
|
| 22 |
# N_ACTION_STEPS (25) ์ถ๋ก ์ ์คํํ action step ์ (50fps ๊ธฐ์ค 0.5s). ํ์ต์๋ ์ํฅ ์์
|
| 23 |
# WANDB_ENABLE (false) WANDB_PROJECT, WANDB_ENTITY ๋ ์ฌ์ฉ
|
| 24 |
+
# OUTPUT_DIR ($WORKDIR/outputs/$JOB_NAME) ๋ก๊ทธ๋ $WORKDIR/logs/$JOB_NAME.log
|
| 25 |
set -euo pipefail
|
| 26 |
cd "$(dirname "$0")"
|
| 27 |
source ./env.sh
|
|
|
|
| 104 |
printf '%q ' "${CMD[@]}" "${ARGS[@]}"; echo
|
| 105 |
[ "${DRY_RUN:-0}" = "1" ] && exit 0
|
| 106 |
|
| 107 |
+
# lerobot ์ด OUTPUT_DIR ์ ์ง์ ๋ง๋ค๋ฉฐ, ์ด๋ฏธ ์์ผ๋ฉด(RESUME ์๋) ๋ฎ์ด์ฐ๊ธฐ ๋ฐฉ์ง๋ฅผ ์ํด ๊ฑฐ๋ถํฉ๋๋ค.
|
| 108 |
+
if [ "${RESUME:-0}" != "1" ] && [ -e "$OUTPUT_DIR" ]; then
|
| 109 |
+
echo "!! $OUTPUT_DIR ์ด ์ด๋ฏธ ์์ต๋๋ค. RESUME=1 ๋ก ์ฌ๊ฐํ๊ฑฐ๋ JOB_NAME=๋ค๋ฅธ์ด๋ฆ ์ ์ฐ์ธ์."; exit 1
|
| 110 |
+
fi
|
| 111 |
+
mkdir -p "$WORKDIR/logs"
|
| 112 |
+
"${CMD[@]}" "${ARGS[@]}" 2>&1 | tee -a "$WORKDIR/logs/$JOB_NAME.log"
|
upload_to_hf.sh
CHANGED
|
@@ -14,6 +14,6 @@ if [ -z "${HF_TOKEN:-}" ] && ! hf auth whoami >/dev/null 2>&1; then
|
|
| 14 |
fi
|
| 15 |
hf repos create "$HF_REPO" --repo-type "$REPO_TYPE" --exist-ok >/dev/null
|
| 16 |
hf upload "$HF_REPO" . . --repo-type "$REPO_TYPE" \
|
| 17 |
-
--exclude ".venv/*" --exclude "cache/*" --exclude "outputs/*" --exclude "**/__pycache__/*" --exclude "*.log" --exclude "wandb/*" \
|
| 18 |
--commit-message "lerobot pi0.5 training environment for DexSteer/isaaclab-ur7e-pour_cup"
|
| 19 |
echo "์
๋ก๋ ์๋ฃ: https://huggingface.co/$HF_REPO"
|
|
|
|
| 14 |
fi
|
| 15 |
hf repos create "$HF_REPO" --repo-type "$REPO_TYPE" --exist-ok >/dev/null
|
| 16 |
hf upload "$HF_REPO" . . --repo-type "$REPO_TYPE" \
|
| 17 |
+
--exclude ".venv/*" --exclude "cache/*" --exclude "outputs/*" --exclude "**/__pycache__/*" --exclude "*.log" --exclude "logs/*" --exclude "wandb/*" \
|
| 18 |
--commit-message "lerobot pi0.5 training environment for DexSteer/isaaclab-ur7e-pour_cup"
|
| 19 |
echo "์
๋ก๋ ์๋ฃ: https://huggingface.co/$HF_REPO"
|