Post
104
Generating the same text in Turkish took an estimated 1.45 to 1.62 times the GPU time of English on four open models (one H200 each, vLLM 0.33, 256 concurrent requests, prefix caching off).
Token count explains most of it. On 2,009 parallel FLORES+ sentences, six of seven tokenizers need 1.36 to 1.62 times as many tokens for Turkish as for English; Kumru-2B needs 0.64 times as many.
Dynamic FP8 in vLLM raised perplexity on FLORES+ by 0.30% to 0.81% in Turkish and 0.27% to 0.39% in English; per token, only Turkish-Gemma-9b-T1 showed a clearly larger rise in Turkish. For GGUF at Q4_K_M on two models, using an importance matrix at all mattered far more than calibrating it on Turkish.
Article (English): https://huggingface.co/blog/serkanpeker/turkish-tax
Data: serkanpeker/turkish-tax
Code: https://github.com/speker/turkish-tax
Token count explains most of it. On 2,009 parallel FLORES+ sentences, six of seven tokenizers need 1.36 to 1.62 times as many tokens for Turkish as for English; Kumru-2B needs 0.64 times as many.
Dynamic FP8 in vLLM raised perplexity on FLORES+ by 0.30% to 0.81% in Turkish and 0.27% to 0.39% in English; per token, only Turkish-Gemma-9b-T1 showed a clearly larger rise in Turkish. For GGUF at Q4_K_M on two models, using an importance matrix at all mattered far more than calibrating it on Turkish.
Article (English): https://huggingface.co/blog/serkanpeker/turkish-tax
Data: serkanpeker/turkish-tax
Code: https://github.com/speker/turkish-tax