README.md

metadata

library_name: pytorch
pipeline_tag: image-to-video
tags:
  - gae
  - text-to-video
  - camera-control
  - 3d-generation

GAE-D64-1B

This repository contains the GAE-D64-1B model presented in GAE: Learning a Geometry-Native Latent Space for 3D-Consistent World Generation.

• Project page: https://jiah-cloud.github.io/GAE.github.io/
• Code: https://github.com/TencentARC/GAE-GeometricAutoEncoder

Weights for GAE-64: a ~1B-parameter temporal DiT (hidden_size=[768, 2048], depth=[28, 6]) over a 64-channel geometry-native latent, at 672×378 and V=81, with metric Plücker camera conditioning and Qwen3-0.6B text.

This repo is self-contained for GAE-64 inference: codec, per-channel latent statistics, DA3 feature-level stats, and the flow transformer. The frozen DA3-GIANT encoder / DPT head comes from depth-anything/DA3-GIANT-1.1.

From the GAE code repository:

scripts/run_demo.sh

Or, after the files are in ckpts/:

python scripts/generate.py \
    --image examples/scenes/forest_lake_trail.jpg \
    --prompt-file examples/scenes/forest_lake_trail.txt \
    --flow-ckpt ckpts/flow_gae64.pt \
    --codec-ckpt ckpts/gae_64.pt \
    --output results/forest

Image-to-video uses 50 Euler steps and CFG 2 with one reference view.

Layout:

• codec/ — safetensors + config.json
• gae_64.pt — drop-in checkpoint for gae.load_codec
• latent_stats_gae_64.pt — per-channel latent mean/std
• da3_stats_giant_5ds.tar — DA3 feature-level normalization stats
• transformer/ — flow safetensors + config.json
• flow_gae64.pt — drop-in checkpoint for gae.load_flow