Post
50
🚀 Can you turn an existing LLM into a diffusion language model in just a few hours on a single GPU?
We introduce "BYOD: Build Your Own DLM", a lightweight approach to converting autoregressive LLMs into diffusion models using LoRA, without modifying the original model weights.
✅ Works with Llama, Gemma, Qwen, and Ministral (and more!)
✅ Just 3–4 GPU hours for 7–9B models
✅ Open-source training code and Google Colab notebook
✅ Interactive demos to try all four models!
We're particularly interested in making diffusion language model research more accessible and reproducible.
Paper, code, and demos:
BYOD: Build Your Own DLM - Efficient Autoregressive-to-Diffusion Language Model Conversion (2610.41574)
Would love to hear your feedback and ideas for improving these conversions!
@AyoubBagheri @Ruurd @UMCU
We introduce "BYOD: Build Your Own DLM", a lightweight approach to converting autoregressive LLMs into diffusion models using LoRA, without modifying the original model weights.
✅ Works with Llama, Gemma, Qwen, and Ministral (and more!)
✅ Just 3–4 GPU hours for 7–9B models
✅ Open-source training code and Google Colab notebook
✅ Interactive demos to try all four models!
We're particularly interested in making diffusion language model research more accessible and reproducible.
Paper, code, and demos:
BYOD: Build Your Own DLM - Efficient Autoregressive-to-Diffusion Language Model Conversion (2610.41574)
Would love to hear your feedback and ideas for improving these conversions!
@AyoubBagheri @Ruurd @UMCU