Balancing client work with lab research hasn't left me with much debugging time on this arch, so I had to delegate a weird bug on the experimental embedding series.
When testing batching under load in vLLM, I was initially only looking at throughput. It wasn't until testing further that I noticed semantic similarity was completely failing.
I was sure I had messed up padding during training, but tossed the logs and the source code to DeepSeek-V4. It ruled out padding immediately and found the fix: simply disabling prefix caching and chunked prefill in vLLM.
Problem solved without touching the codebase. Happy to finally use these experimental embeddings! 🙌