AuroraAI-Research/AuroraIMG2-100M
Text-to-Image • 0.1B • Updated • 14
Marcell Jager
ThatHungarian
AI & ML interests
None yet
Recent Activity
published a model about 6 hours ago
AuroraAI-Research/AuroraIMG2-100M liked a dataset about 11 hours ago
LucasFang/FLUX-Reason-6M updated a model about 12 hours ago
AuroraAI-Research/AuroraIMG2-100MOrganizations
published a
model about 6 hours ago
liked a
dataset about 11 hours ago
updated a
model about 12 hours ago
French version: Brindille, 94k params, runs in the browser
1
#4 opened 4 days ago
by 
edbrdi
updated 2
models 5 days ago
ONNX conversion and lower-memory CPU inference for AuroraTTS-100K
3
#2 opened 11 days ago
by 
jbgrimsby
Add optional FP32 ONNX CPU inference companion
#3 opened 7 days ago
by jbgrimsby
liked a
Space 6 days ago
Demo for this model on Spaces
1
#1 opened 7 days ago
by 
multimodalart
Add AuroraIMG2
#21 opened 8 days ago
by ThatHungarian
Add AuroraIMG2-50M-Preview
#20 opened 8 days ago
by ThatHungarian
published a
model 8 days ago
reacted to SeaWolf-AI's post with ❤️ 9 days ago
Post
7,235
We pretrained a 325M language model from zero on an inference-only NPU
The accepted view is that inference-only NPUs cannot train, let alone pretrain a language model from scratch. We put that view to a direct test on the Rebellions ATOM-Max, a commercially available Korean NPU released in 2025 and designed for inference. We built our own training loop that runs backpropagation on the chip itself. Starting from random initialization, we pretrained VKORE-325M-NPU-Base, a 325.6M parameter model, on 5 billion byte tokens of FineWeb-Edu. The run took 76,292 steps over about four days and finished without a single divergence. No pretrained weights were used.
To make the comparison fair, we trained the same model on a B200 GPU with identical initialization, data and learning-rate schedule. On 1,024 windows of text that neither model ever saw, the NPU reached a loss of 0.7993 and the GPU 0.7994. The window-by-window difference is -0.0001, well inside the noise. The two models are indistinguishable.
This is seven times larger than our first NPU model, VKORE-46M, and the first time this hardware has carried a real pretraining run to GPU quality. Please note that the hardware alone does not make this possible: the NPU and its standard software are built for inference, and training on it depends on our own training stack and stability technique, which are developed in house and under patent application. Buying the chip does not by itself enable training. Both sets of weights and the evaluation script are public, so anyone can verify the result in minutes.
📄 Technical report: https://huggingface.co/blog/FINAL-Bench/pretraining-a-325m-language-model-from-zero-on-an
🤗 Models: FINAL-Bench/VKORE-325M-NPU-Base
#NPU #Rebellions #Pretraining #VKORE
The accepted view is that inference-only NPUs cannot train, let alone pretrain a language model from scratch. We put that view to a direct test on the Rebellions ATOM-Max, a commercially available Korean NPU released in 2025 and designed for inference. We built our own training loop that runs backpropagation on the chip itself. Starting from random initialization, we pretrained VKORE-325M-NPU-Base, a 325.6M parameter model, on 5 billion byte tokens of FineWeb-Edu. The run took 76,292 steps over about four days and finished without a single divergence. No pretrained weights were used.
To make the comparison fair, we trained the same model on a B200 GPU with identical initialization, data and learning-rate schedule. On 1,024 windows of text that neither model ever saw, the NPU reached a loss of 0.7993 and the GPU 0.7994. The window-by-window difference is -0.0001, well inside the noise. The two models are indistinguishable.
This is seven times larger than our first NPU model, VKORE-46M, and the first time this hardware has carried a real pretraining run to GPU quality. Please note that the hardware alone does not make this possible: the NPU and its standard software are built for inference, and training on it depends on our own training stack and stability technique, which are developed in house and under patent application. Buying the chip does not by itself enable training. Both sets of weights and the evaluation script are public, so anyone can verify the result in minutes.
📄 Technical report: https://huggingface.co/blog/FINAL-Bench/pretraining-a-325m-language-model-from-zero-on-an
🤗 Models: FINAL-Bench/VKORE-325M-NPU-Base
#NPU #Rebellions #Pretraining #VKORE
- 8 replies
updated a
Space 13 days ago
Demo for this model on Spaces
2
#1 opened 13 days ago
by multimodalart
published a
dataset 13 days ago
updated a
dataset 13 days ago
published a
model 13 days ago
updated a
model 13 days ago