🔄 In a Training Loop

Stefano Fiorucci PRO

anakin87

50 84 202

AI & ML interests

Language Models: orchestration, post-training, GRPO, synthetic data... Contributing to Haystack LLM framework 🏗️

Recent Activity

liked a Space 4 days ago

HuggingEnvs/geoguesser-article

liked a model 4 days ago

anakin87/tiny-doom-defender

reacted to theirpost with ❤️ 5 days ago

I made a 1.1M ModernBERT encoder play Doom in real time on a CPU Some time ago, VAGO Solutions released SauerkrautLM-Doom-MultiVec-1.3M, a tiny model trained to play Doom Defend the Center scenario from 31k human gameplay examples. My first thought: cool! I love both Doom and Small Language Models. Then another idea: I bet I can do better :-) What I did? - evaluated the original model and found it's better than reported - changed a bit the architecture - generated SFT data with a scripted oracle - SFT + PPO refinement on consumer hardware Got a smaller, faster and killer model Can even fit a floppy with int8 quantization 💾 Watch it play/read the article: https://huggingface.co/spaces/anakin87/tiny-doom-defender

View all activity

Organizations

liked a Space 4 days ago

How to turn a game into an RL environment

🌍

From an idea to a trained 4B, with the dead ends left in

liked a model 4 days ago

anakin87/tiny-doom-defender

Reinforcement Learning • 1.1M • Updated 14 days ago • 19 • 5

reacted to their post with ❤️ 5 days ago

Post

3419

I made a 1.1M ModernBERT encoder play Doom in real time on a CPU

Some time ago, VAGO Solutions released SauerkrautLM-Doom-MultiVec-1.3M, a tiny model trained to play Doom Defend the Center scenario from 31k human gameplay examples.

My first thought: cool! I love both Doom and Small Language Models.

Then another idea: I bet I can do better :-)

What I did?
- evaluated the original model and found it's better than reported
- changed a bit the architecture
- generated SFT data with a scripted oracle
- SFT + PPO refinement on consumer hardware

Got a smaller, faster and killer model
Can even fit a floppy with int8 quantization 💾

Watch it play/read the article: anakin87/tiny-doom-defender

posted an update 5 days ago

Post

3419

I made a 1.1M ModernBERT encoder play Doom in real time on a CPU

Some time ago, VAGO Solutions released SauerkrautLM-Doom-MultiVec-1.3M, a tiny model trained to play Doom Defend the Center scenario from 31k human gameplay examples.

My first thought: cool! I love both Doom and Small Language Models.

Then another idea: I bet I can do better :-)

What I did?
- evaluated the original model and found it's better than reported
- changed a bit the architecture
- generated SFT data with a scripted oracle
- SFT + PPO refinement on consumer hardware

Got a smaller, faster and killer model
Can even fit a floppy with int8 quantization 💾

Watch it play/read the article: anakin87/tiny-doom-defender