Functionality-Oriented LLM Merging on the Fisher--Rao Manifold
Paper • 2603.04972 • Published • 4
AI & ML interests
Open science and open source
Recent Activity
View all activity
win10
authored a
paper 16 days ago
Post
2846
SPECK 2 IS ALREADY OUT: specklabs/Speck2-140M
Pretrained on 4x more tokens than the previous releases (20b vs 5b).
Instruct tuned versions are coming soon.
Very interesting models are coming soon too (hint: super long context).
Thanks for everyone supporting!
Pretrained on 4x more tokens than the previous releases (20b vs 5b).
Instruct tuned versions are coming soon.
Very interesting models are coming soon too (hint: super long context).
Thanks for everyone supporting!
- 3 replies
Post
149
NEW SPECK UPDATES:
Just hit #14 and #15 with out FIRST models on Open SLM Leaderboard. The models were trained on 5B tokens, while competing with similarly sized models trained on more than 6-20x the data.
A new base model Speck1.5-140M being trained right now on a higher quality corpus and will be released soon.
SpeckChat3 is coming very soon with 1 million samples, specifically designed to post train small base models.
Also, just to clarify stuff, we will NOT release anything that is NOT MIT licensed EVER. Openness is needed in small language research.
Thanks to everyone supporting the project, and stay tuned for new releases!
Just hit #14 and #15 with out FIRST models on Open SLM Leaderboard. The models were trained on 5B tokens, while competing with similarly sized models trained on more than 6-20x the data.
A new base model Speck1.5-140M being trained right now on a higher quality corpus and will be released soon.
SpeckChat3 is coming very soon with 1 million samples, specifically designed to post train small base models.
Also, just to clarify stuff, we will NOT release anything that is NOT MIT licensed EVER. Openness is needed in small language research.
Thanks to everyone supporting the project, and stay tuned for new releases!
Post
1882
SPECK UPDATES:
1 New instruct model tuned on top of Speck1-140M: specklabs/Speck1-140M-Instruct
2 Instruction tuning datasets
2 GGUFs
Much more coming soon:
Speck1.1-140M-Instruct that is post trained on SpeckChat2 will be coming very soon
New base model Speck1.5-140M is coming with a much higher quality corpus
Thanks to everyone who is already supporting the project, and stay tuned for new releases!
1 New instruct model tuned on top of Speck1-140M: specklabs/Speck1-140M-Instruct
2 Instruction tuning datasets
2 GGUFs
Much more coming soon:
Speck1.1-140M-Instruct that is post trained on SpeckChat2 will be coming very soon
New base model Speck1.5-140M is coming with a much higher quality corpus
Thanks to everyone who is already supporting the project, and stay tuned for new releases!
- 3 replies
Post
2120
FIRST SPECK MODEL RELEASED:
specklabs/Speck1-140M
new models coming very soon (both instruct and much better models), with much much higher training scale as i am getting marenostrum5 access soon!
we will be looking at 100b-2t token budgets :)
specklabs/Speck1-140M
new models coming very soon (both instruct and much better models), with much much higher training scale as i am getting marenostrum5 access soon!
we will be looking at 100b-2t token budgets :)
- 4 replies
osanseviero
authored a
paper 3 months ago
osanseviero
submitted a
paper to Daily Papers 3 months ago
jeffboudier
posted an update 3 months ago
Post
516
@alvarobartt published a step-by-step guide to deploy zai-org/GLM-5.2 on AMD GPUs, using newly released features in Microsoft Foundry. The FP8 model fits on a single node of MI300X GPUs, which cuts the bill in half vs. H100.
https://alvarobartt.com/goal-glm-5.2-on-foundry/
https://alvarobartt.com/goal-glm-5.2-on-foundry/
VedantPadwal
authored a
paper 3 months ago
akhaliq
submitted 3
papers to Daily Papers 4 months ago
DynaFLIP: Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation
Paper • 2605.30350 • Published • 10
Contrastive Distribution Matching for Amortized Sequential Monte Carlo in Discrete Diffusion
Paper • 2605.23346 • Published
optimize_anything: A Universal API for Optimizing any Text Parameter
Paper • 2605.19633 • Published • 6
Sri-Vigneshwar-DJ
posted an update 5 months ago
Post
194
We ran Feather DB v0.8.0 on LongMemEval (ICLR 2025) — 500 questions across real multi-session conversations, up to 115K tokens each.
**Score: 0.693** · GPT-4o full-context baseline: 0.640
Full 500-question run with Gemini-Flash: **$2.40**
Per-axis breakdown:
→ Info-extraction: **0.942**
→ Knowledge-update: **0.714**
→ Multi-session: **0.606**
→ Temporal: **0.477** ← the hard one, Phase 9 addresses this
Architecture: Hybrid BM25+dense · adaptive temporal decay · embedded (no server) · p50 = 0.19ms · MIT
Raw results + audit JSONs: Hawky-ai/longmemeval-results
We ran Feather DB v0.8.0 on LongMemEval (ICLR 2025) — 500 questions across real multi-session conversations, up to 115K tokens each.
**Score: 0.693** · GPT-4o full-context baseline: 0.640
Full 500-question run with Gemini-Flash: **$2.40**
Per-axis breakdown:
→ Info-extraction: **0.942**
→ Knowledge-update: **0.714**
→ Multi-session: **0.606**
→ Temporal: **0.477** ← the hard one, Phase 9 addresses this
Architecture: Hybrid BM25+dense · adaptive temporal decay · embedded (no server) · p50 = 0.19ms · MIT
pip install feather-dbRaw results + audit JSONs: Hawky-ai/longmemeval-results
akhaliq
submitted a
paper to Daily Papers 5 months ago
akhaliq
submitted 4
papers to Daily Papers 6 months ago
MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Engines
Paper • 2603.06679 • Published • 5
AVO: Agentic Variation Operators for Autonomous Evolutionary Search
Paper • 2603.24517 • Published • 13
V-Co: A Closer Look at Visual Representation Alignment via Co-Denoising
Paper • 2603.16792 • Published • 3
Multimodal OCR: Parse Anything from Documents
Paper • 2603.13032 • Published • 47
AlekseyKorshuk
authored a
paper 7 months ago