---
license: other
license_name: ltx-2-community
license_link: https://github.com/Lightricks/LTX-2/blob/main/LICENSE
base_model: Lightricks/LTX-2.3
tags:
- video
- lora
- camera-control
- interactive
---

# helloworld-interactor-lora

> ⚠️ **This LoRA is a fine-tuned Derivative of LTX-2 and is distributed under the
> [LTX-2 Community License](LICENSE-LTX2)** (including its use-based restrictions,
> Attachment A). A complete copy of the license is included in this repository.

**Current release: v2** — `helloworld-interactor-lora-r32-v2-2500.safetensors` (recommended).
The original v1 (`helloworld-interactor-lora-r32-2000.safetensors`, 156-clip corpus) remains
for reproducibility. v2 triples the corpus (456 clips — including 300 targeting compound
human choreography, subtle low-amplitude gestures, and one-press-one-beat discipline) and
trains to 2500 steps. On a 35-pair true-video bench: **performs-in-window 62% vs 20% base,
human characters 50% vs 6%, visible artifacts 0/35 vs 15/35.**

Rank-32 LoRA for **Lightricks LTX-2.3** that adds **camera-trajectory following from warp-video
history conditioning** — the self-distillation recipe of
["HelloWorld: Enabling Socially Interactive Characters in Video World Models"](https://arxiv.org/abs/2608.05070)
(Ouyang, Liu, Chu, Zhang, Sato — Alaya Lab / The University of Tokyo), reimplemented
independently before the authors' official code release. Paired with the training-free F-press
cross-attention mask from the same paper, you get characters that perform at the camera inside
a chosen time window while the shot follows a camera path you draw.

- **Code, warp pipeline, Eq. 4 mask, training config, reproduction guide:**
  https://github.com/scrappylabsai/helloworld-interactor
- **What it is:** LoRA rank 32, alpha 32, on all self-attention projection matrices of the
  video branch (`attn1.*`), LR 1e-4, flow matching, bf16, on self-generated 1280×704×121
  clips with recovered-trajectory warp videos as reference conditioning; camera text excluded
  from captions (paper §3.2, §5.2). v1: 156 clips / 2000 steps. **v2: 456 clips / 2500 steps.**
- **Training cost:** rented RTX PRO 6000s (96 GB), peak 72.3 GB; whole project (v1+v2,
  all rentals) ~$127.

## Results — v2 (full bench, 40 generated characters, base vs LoRA, true-video judge)

| Metric | Base LTX-2.3 | + this LoRA (v2) |
|---|---|---|
| Performs the directed interaction (n=35 parsed) | 22% | **65%** |
| Performs *inside the window* | 20% | **62%** |
| Human characters in-window | 6% | **50%** |
| Clips with visible artifacts | 15/35 | **0/35** |
| Camera matches directed trajectory | 26/35 | **32/35** |

Judge: Gemini with true-video input, structured verdicts, blind to which arm is which.

**Generalization — the result we didn't train for.** On fresh prompts the base model often
doesn't even resemble the instruction; with the LoRA the same seed executes it. We believe the
adapter learned a *grammar* (obey the interaction spec, inside the window, hold steady
otherwise) rather than content — so it transfers to characters and actions that appear nowhere
in training. Public evidence: a wall of 10 brand-new characters (fresh stills, fresh camera
warps, unseen interactions — zero overlap with training or bench on any axis):
**https://interactor-demos.pages.dev/ab4/**

## Results — v1 (mini-bench, first release)

| Metric | Judge | Base LTX-2.3 | + v1 LoRA | Paper |
|---|---|---|---|---|
| Performs in window (n=9 pairs) | Gemini, true-video | 1/9 (11%) | **7/9 (78%)** | TimeAcc 81.7% |
| TimeAcc (n=40) | Qwen3.6-35B-A3B, frame grids | 21% | **44%** | 81.7% |
| Performs at all (n=40) | Qwen3.6-35B-A3B, frame grids | 24/40 | **34/40** | — |

Qwen3.6-35B-A3B is the paper's judge model; our serving feeds it frame grids rather than raw
video, which attenuates absolute scores for both columns (motion between sampled frames is
invisible) — the true-video Gemini calibration on the same clips lands at the paper's number
within small-n error. Caveats: n=40/n=9 (paper: 400); trained at 121 frames vs the paper's 241;
our geometric CamCtrl metric is pending a pose-surveyor bugfix (qualitative camera control —
dolly-in, orbit — confirmed side-by-side at fixed seed). Where the trained model missed the
directed interaction (2/9 true-video pairs, both subtle low-amplitude gestures — a wink, a
sliding object), the untrained baseline missed the same clips: misses degrade to baseline
behavior rather than below it, and trained clips showed fewer artifacts (1/9 vs 3/9).

## Example pairs (same seed, same prompt, same warp input)

Full-quality mp4s in [`examples/`](examples/) — `<id>__base.mp4` is stock LTX-2.3,
`<id>__lora.mp4` is +this LoRA. All characters fully generated.

| Character | What was directed (interaction · window · camera) | Base | +LoRA |
|---|---|---|---|
| Brewer (taproom) | Raises a full glass toward the viewer in a toast, saying "Cheers!" · 1.5–2.5 s · dolly-in | [base](examples/brewer-taproom__base.mp4) | [LoRA](examples/brewer-taproom__lora.mp4) |
| Baker (window) | Holds up a steaming loaf toward the viewer and winks · 1.5–2.5 s · static | [base](examples/baker-window__base.mp4) | [LoRA](examples/baker-window__lora.mp4) |
| Clay snowman | Tips his tiny top hat toward the camera · 1.5–2.5 s · dolly-in | [base](examples/clay-snowman__base.mp4) | [LoRA](examples/clay-snowman__lora.mp4) |
| Tin robot (workbench) | Winds its own key, then salutes the viewer stiffly · 1.5–2.5 s · dolly-in | [base](examples/robot-workbench__base.mp4) | [LoRA](examples/robot-workbench__lora.mp4) |
| Plush octopus (bathtub) | Wiggles two tentacles toward the viewer in greeting · 1.5–2.5 s · orbit-left | [base](examples/plush-octopus__base.mp4) | [LoRA](examples/plush-octopus__lora.mp4) |
| Mechanic (garage) | Wipes his hands on a rag and gives the viewer a thumbs-up · 3.0–4.0 s · pan-right | [base](examples/mechanic-garage__base.mp4) | [LoRA](examples/mechanic-garage__lora.mp4) |

**v2 generalization pairs** — characters built from scratch *after* training, unseen on every
axis (still, warp, interaction):

| Character | What was directed (interaction · window · camera) | Base | +LoRA v2 |
|---|---|---|---|
| Beekeeper (orchard) | Lifts her veil and holds a honey dipper out toward the viewer · 1.5–2.5 s · dolly-in | [base](examples/beekeeper-hives__base.mp4) | [LoRA](examples/beekeeper-hives__lora.mp4) |
| Dragon in a teacup | Pokes its head over the rim and puffs a smoke ring at the viewer · 2.5–3.5 s · dolly-in | [base](examples/dragon-teacup__base.mp4) | [LoRA](examples/dragon-teacup__lora.mp4) |
| Yeti (teahouse) | Slides a steaming cup across the counter and gives two slow blinks · 2.5–3.5 s · dolly-in | [base](examples/yeti-teahouse__base.mp4) | [LoRA](examples/yeti-teahouse__lora.mp4) |

## Usage

With the repo's inference driver (LTX-2 packages installed + the repo's vendor patch applied —
see the repo's `docs/REPRODUCE.md`):

```bash
python -m helloworld_ltx.infer_warp \
    --distilled-checkpoint-path models/ltx-2.3/ltx-2.3-22b-distilled-1.1.safetensors \
    --gemma-root models/gemma-3-12b \
    --spatial-upsampler-path models/ltx-2.3/ltx-2.3-spatial-upscaler-x2-1.1.safetensors \
    --image inputs/shot/first_frame.png 0 1.0 \
    --warp-npz inputs/shot/warp.npz \
    --scene "A lighthouse keeper on the gallery rail at dusk." \
    --interaction "He turns and waves at the viewer." \
    --quality "Photoreal, natural lighting." \
    --window 24 48 \
    --height 704 --width 1280 --num-frames 121 --frame-rate 24 \
    --lora helloworld-interactor-lora-r32-v2-2500.safetensors 1.0 \
    --seed 42 --output-path out.mp4
```

`--warp-npz` comes from the repo's `warp/warp.py` (choose a trajectory) or `warp/annotate.py`
(recover one from a video). `--window` is the F-press window in pixel frames at the output rate.
Timing control is training-free; this LoRA adds the camera-following.


**Choosing LoRA strength.** `--lora <file> 1.0` is maximum obedience — tightest camera
following and window timing. Dialing down to **0.6–0.8** lets more of the base model's
ambient life through (background motion, richer prop handling) at the cost of slightly
looser camera tracking — in our testing ~0.6–0.7 is the sweet spot for production feel.
Same seed comparison: https://interactor-demos.pages.dev/ab4/dial

## Dependency license note (Pi3X)

Generating the warp inputs uses the **Pi3X / π³** geometry model
([arXiv:2507.13347](https://arxiv.org/abs/2507.13347)): its weights are **CC BY-NC 4.0
(non-commercial)** and are NOT included here or in the code repo. The LoRA weights themselves
have no Pi3X dependency, but the as-shipped warp pipeline does — commercial users must
substitute a permissively-licensed geometry backbone for warp generation.

## Credit

All credit for the method to the HelloWorld authors (Alaya Lab / The University of Tokyo) —
this is an unofficial reproduction from the paper text, published with full attribution.
Watch https://github.com/AlayaLab/HelloWorld for their official release.

Built by [ScrappyLabs](https://scrappylabs.ai).

*Try it on your own productions — if you direct your own characters with this, we'd love to see
the results: open a Discussion on the
[GitHub repo](https://github.com/scrappylabsai/helloworld-interactor) with your clips and what
you directed.*