Hugging Face's logo

Code Llama
company
Activity Feed

AI & ML interests

None defined yet.
loubnabnl 
posted an update 8 days ago
view post
Post
4,325
𝐈𝐧𝐭𝐫𝐨𝐝𝐮𝐜𝐢𝐧𝐠 🧬 𝐅𝐢𝐧𝐞𝐃𝐍𝐀: 𝐭𝐡𝐞 𝐛𝐞𝐬𝐭 𝐝𝐚𝐭𝐚𝐬𝐞𝐭 𝐭𝐨 𝐭𝐞𝐚𝐜𝐡 𝐚𝐧𝐲 𝐥𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐦𝐨𝐝𝐞𝐥 𝐃𝐍𝐀.
Training Qwen3-14B on it produces Carbon-X, the strongest open model on DNA benchmarks, on par with GPT-6.1 Sol.

It contains:
- Midtraining: 300B+ tokens of genomic sequences, papers and web pages mixing DNA with text
- SFT: 200K samples, from variant effect prediction to sequence question answering
- RL: 98K prompts with verifiable rewards

Data, models and the full recipe are open.
HuggingFaceBio/FineDNA

Enjoy this late Halloween treat! 🎃
  • 5 replies
·
pcuenq 
posted an update 10 months ago
view post
Post
8,126
👉 What happened in AI in 2025? 👈

We prepared the 2025 version of the HF AI Timeline Grid, highlighting open vs API-based model releases, and allowing you to browse and filter by access, modality, and release type!

Play with it here:
2025-ai-timeline/2025-ai-timeline

Here's my personal quarterly TL;DR:

1️⃣ Q1 — Learning to Reason
Deepseek not only releases a top-notch reasoning model, but shows how to train them and compete with closed frontier models. OpenAI debuts Deep Research.

Significant milestones: DeepSeek R1 & R1-Zero, Qwen 2.5 VL, OpenAI Deep Research, Gemini 2.5 Pro (experimental)

2️⃣ Q2 — Multimodality and Coding
More LLMs embrace multimodality by default, and there's a surge in coding agents. Strong vision, audio, and generative models emerge.

Significant milestones: Llama 4, Qwen 3, Imagen 4, OpenAI Codex, Google Jules, Claude 4

3️⃣ Q3 — "Gold" rush, OpenAI opens up, the community goes bananas
Flagship models get gold in Math olympiads and hard benchmarks. OpenAI releases strong open source models and Google releases the much anticipated nano-banana for image generation and editing. Agentic workflows become commonplace.

Significant milestones: Gemini and OpenAI IMO Gold, gpt-oss, Gemini 2.5 Flash Image, Grok 4, Claude Sonnet 4.5

4️⃣ Q4 — Mistral returns, leaderboard hill-climbing
Mistral is back with updated model families. All labs release impressive models to wrap up the year!

Significant milestones: Claude Opus 4.5, DeepSeek Math V2, FLUX 2, GPT 5.1, Kimi K2 Thinking, Nano Banana Pro, GLM 4.7, Gemini 3, Mistral 3, MiniMax M2.1 🤯

Credits
🙏 NHLOCAL for the source data https://github.com/NHLOCAL/AiTimeline

🫡 @reach-vb for the original idea, design and recipe

🙌 @ariG23498 and yours truly for compiling and verifying the 2025 edition

🥳 Here's to 2026, wishing it becomes the best year ever for open releases and on-device-first use-cases! 🥂
  • 3 replies
·
lvwerra 
authored a paper about 1 year ago
lvwerra 
authored a paper over 1 year ago
loubnabnl 
authored a paper over 1 year ago
loubnabnl 
posted an update over 1 year ago
pcuenq 
authored a paper over 1 year ago
lvwerra 
authored a paper over 1 year ago
loubnabnl 
authored a paper over 1 year ago
loubnabnl 
authored a paper almost 2 years ago
lvwerra 
authored 2 papers almost 2 years ago
loubnabnl 
posted an update almost 2 years ago
view post
Post
3,765
Making SmolLM2 reproducible: open-sourcing our training & evaluation toolkit 🛠️ https://github.com/huggingface/smollm/

- Pre-training code with nanotron
- Evaluation suite with lighteval
- Synthetic data generation using distilabel (powers our new SFT dataset HuggingFaceTB/smoltalk)
- Post-training scripts with TRL & the alignment handbook
- On-device tools with llama.cpp for summarization, rewriting & agents

Apache 2.0 licensed. V2 pre-training data mix coming soon!

Which other tools should we add next?
ArthurZ 
posted an update almost 2 years ago
lvwerra 
authored a paper about 2 years ago
lvwerra 
authored a paper over 2 years ago
loubnabnl 
authored a paper over 2 years ago
loubnabnl 
posted an update over 2 years ago
view post
Post
5,932
🍷 FineWeb technical report is out and so is 📚 FineWeb-Edu, a 1.3 trillion tokens dataset that outperforms all other open web datasets, with remarkable improvements on educational benchmarks such as MMLU, ARC, and OpenBookQA.

Technical report: HuggingFaceFW/blogpost-fineweb-v1
Dataset: HuggingFaceFW/fineweb-edu

We used Llama 3 generations to train an educational quality classifier, filtering the 15 trillion tokens of FineWeb to select only those with high educational value (an approach also used in Llama 3 and Phi-3 training datasets). We're releasing both FineWeb-Edu and the classifier, along with a larger, less heavily filtered version containing 5.4 trillion tokens.

You can find more details about the dataset and the experiments we ran in the FineWeb technical report, It's a 45-minute read but it contains all the secret sauce for building high quality web datasets.

Enjoy!
lvwerra 
authored a paper over 2 years ago
loubnabnl 
authored a paper over 2 years ago