Pretrained checkpoints for OLMo 20M-150M models used in experiments for our paper "Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting"
Ishaan Watts
WattsIshaan
AI & ML interests
Machine Learning, NLP
Recent Activity
updated a collection 1 day ago
OLMo-2-1B Midtrained updated a model 1 day ago
WattsIshaan/OLMo-2-1B-Midtrain-50B-AdamW-20percent updated a collection 1 day ago
OLMo-2-1B MidtrainedOrganizations
None yet
models 16
WattsIshaan/OLMo-2-1B-Midtrain-50B-AdamW-20percent
Updated • 7
WattsIshaan/OLMo-2-1B-Midtrain-50B-AdamW-10percent
Updated • 10
WattsIshaan/OLMo-60m-192B-adamw-cosine-10percent
Updated • 6
WattsIshaan/OLMo-150m-120B-adamw-cosine-10percent
Updated • 11
WattsIshaan/OLMo-20m-64B-adamw-cosine-10percent
Updated • 6
WattsIshaan/OLMo-150m-120B-sam-cosine
Updated • 11
WattsIshaan/OLMo-150m-120B-adamw-cosine
Updated • 9
WattsIshaan/OLMo-60m-192B-adamw-cosine
Updated • 8
WattsIshaan/OLMo-60m-192B-adamw-wsd-10percent
Updated • 6 • 1
WattsIshaan/OLMo-60m-192B-sam-cosine
Updated • 8
datasets 0
None public yet