Pretraining Transformers with Quantized Softmax in Attention
Paper • 2609.33591 • Published • 7
Shangzhen Zhu
ArgonnSZ
·
AI & ML interests
None yet
Recent Activity
submitted a paper 10 days ago
Pretraining Transformers with Quantized Softmax in Attention upvoted a paper 10 days ago
Softmax Reparameterization for Output-Head Quantization upvoted a paper 10 days ago
Approximating Softmax in Pretrained LLMs: Model Sensitivity and Kernel AccelerationOrganizations
submitted a
paper to Daily Papers 10 days ago
upvoted 2
papers 10 days ago
submitted a
paper to Daily Papers 10 days ago
authored 2
papers 10 days ago