CustomX: Unified Character, Action, and Scene Customization in Video World Models

ECCV 2026

---

📖 Abstract

Recent advances in world models have greatly enhanced interactive environment simulation. Existing methods mainly fall into two categories: (1) static world generation models, which construct 3D environments without active agents, and (2) controllable-entity models, which allow a single entity to perform limited actions in an otherwise uncontrollable environment. In this work, we introduce CustomX, leveraging the realism and structural grounding of static world generation while extending controllable-entity models to support user-specified characters capable of performing open-ended actions. Users can provide a 3DGS scene and a character, then use natural language to direct the character to perform diverse behaviors, ranging from basic locomotion to object-centric interactions, while freely exploring the environment. CustomX synthesizes temporally coherent video clips that preserve visual fidelity with the provided scene and character, formulated as a conditional autoregressive video generation problem. Built upon a pre-trained video generator, our training strategy significantly enhances motion dynamics while maintaining generalization across actions and characters. Our evaluation covers a broad range of aspects, including visual quality, character consistency, action controllability, and long-horizon coherence.

🛠️ Installation

conda create -n customx python==3.11.9
conda activate customx

pip install uv

uv pip install torch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 --index-url https://download.pytorch.org/whl/cu124
uv pip install -r requirements.txt

wget https://github.com/Dao-AILab/flash-attention/releases/download/v2.6.3/flash_attn-2.6.3+cu123torch2.4cxx11abiFALSE-cp311-cp311-linux_x86_64.whl
uv pip install flash_attn-2.6.3+cu123torch2.4cxx11abiFALSE-cp311-cp311-linux_x86_64.whl

📦 Download Checkpoints

HunyuanCustom Base Model

Download HunyuanCustom base model here.

Only need to download the following files from HunyuanCustom:

CustomX
└── models
    └── base
        ├── hunyuancustom_editing_720P
        │   └── mp_rank_00_model_states.pt
        ├── vae_3d
        ├── openai_clip-vit-large-patch14
        └── llava-llama-3-8b-v1_1

CustomX LoRA

TOKEN=xxx # add your Hugging Face token here

hf download --token $TOKEN \
    SnowflakeWang/CustomX \
    --local-dir models/lora

🚀 Inference

Input Preparation

We provide a sample input in the input directory. To use your own inputs, organize them following the same structure.

input
├── character_assets
│   └── orangeRobot
│       ├── input_list
│       │   ├── ar_cond.list
│       │   ├── character.list
│       │   ├── output_video.list
│       │   ├── pos_prompt.list
│       │   ├── scene.list
│       │   └── scene_mask.list
│       └── multi_view
│           ├── 000_0001.png
│           ├── 002_0001.png
│           ├── 004_0001.png
│           └── 006_0001.png
└── scene_assets
    └── futureUtopia
        ├── all_frame_mask.mp4
        └── videos
            ├── 0.mp4
            ├── 1.mp4
            └── ...

Multi-GPU Inference (Recommended)

# 720P video inference
# Tested on 8 NVIDIA A100-80G GPUs
bash inference_multi_gpu.sh

Single-GPU Inference

# 360P video inference
# Tested on 1 NVIDIA A100-80G GPU
bash inference_single_gpu.sh

Video Merging

# Merge videos generated by auto-regressive inference
python video_merge.py --input_dir "output/orangeRobot_futureUtopia"

🔮 Citation

@misc{wang2026customx,
      title={CustomX: Unified Character, Action, and Scene Customization in Video World Models}, 
      author={Yitong Wang and Fangyun Wei and Hongyang Zhang and Bo Dai and Yan Lu},
      year={2026},
      eprint={2512.17796},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2512.17796}, 
}

💐 Acknowledgements

• HunyuanCustom
• Diffusers
• Transformers
• DeepSpeed
• Grand Theft Auto V

📄 License

This project is licensed under the CC BY-SA 4.0 License.