MOSS-Live
Collection
MOSS-Live • 0 items • Updated • 4
Wang Weiyi PRO
kaupane
AI & ML interests
Embodied AI and whatever is fun
Recent Activity
upvoted a collection about 7 hours ago
MOSS-Live upvoted a paper 3 days ago
Loop the Loopies! liked a model 3 days ago
zitian-gao/Loopie-20B-A2B-previewOrganizations
None yet
upvoted a collection about 7 hours ago
upvoted a paper 3 days ago
liked a
model 3 days ago
reacted to Saccadic-AI's post with 🔥 7 days ago
Post
4,196
One neural network. Five mathematical operations. Different learning times.
We trained a small mixture-of-experts transformer on addition, subtraction, multiplication, division, and exponentiation modulo 113.
The operations didn’t generalize together. Multiplication and division reached 90% held-out accuracy first, followed by exponentiation, subtraction, and finally addition.
Multiplication before addition—the reverse of what we expect from a schoolchild.
The accuracy chart in the linked article shows why a single overall accuracy number can hide an interesting learning story.
Measured with IRIS from Saccadic LLC. More figures and analysis: https://huggingface.co/blog/Saccadic-AI/accuracy-plateaued-the-model-kept-changing
Have you observed a similar ordering in multi-task grokking experiments?
We trained a small mixture-of-experts transformer on addition, subtraction, multiplication, division, and exponentiation modulo 113.
The operations didn’t generalize together. Multiplication and division reached 90% held-out accuracy first, followed by exponentiation, subtraction, and finally addition.
Multiplication before addition—the reverse of what we expect from a schoolchild.
The accuracy chart in the linked article shows why a single overall accuracy number can hide an interesting learning story.
Measured with IRIS from Saccadic LLC. More figures and analysis: https://huggingface.co/blog/Saccadic-AI/accuracy-plateaued-the-model-kept-changing
Have you observed a similar ordering in multi-task grokking experiments?
- 6 replies
liked a
model 7 days ago
liked a
dataset 7 days ago
liked a
model 8 days ago
liked a
model 9 days ago
posted an update 11 days ago
Post
2,056
I built Inko end to end: a 0.5B bilingual text-to-image model with a focus on fine art, from data curation and pretraining to RL post-training and evaluation.
It competes with DreamLite, SANA-0.6B and PixArt-Σ, and approaches SD3.5 Medium on some benchmarks.
The journey involves four rounds of cleaning the Chinese-painting dataset, a cross-hardware loss-spike investigation, and a reward redesign after early RL experiments collapased diversity.
Behind the release, a four-domain bilingual corpus, 600k pretraining updates on 16 Ascend NPUs, and DiffusionART with a custom 24.6k-prompt pool and five pairwise rubric channels.
Dense prompt alignment is Inko's strongest general result. Anatomy, counting, reasoning, text rendering, and exact spatial relations remain limitations.
kaupane/Inko
kaupane/Inko
It competes with DreamLite, SANA-0.6B and PixArt-Σ, and approaches SD3.5 Medium on some benchmarks.
The journey involves four rounds of cleaning the Chinese-painting dataset, a cross-hardware loss-spike investigation, and a reward redesign after early RL experiments collapased diversity.
Behind the release, a four-domain bilingual corpus, 600k pretraining updates on 16 Ascend NPUs, and DiffusionART with a custom 24.6k-prompt pool and five pairwise rubric channels.
Dense prompt alignment is Inko's strongest general result. Anatomy, counting, reasoning, text rendering, and exact spatial relations remain limitations.
kaupane/Inko
kaupane/Inko
- 1 reply
liked a
Space 12 days ago
liked a
model 13 days ago
updated a
model 13 days ago
liked a
model 14 days ago
updated a
Space 14 days ago
updated a
collection 16 days ago
liked 2
Spaces 17 days ago