LiquidAI

/

LFM3-1.7B-GGUF

Like 200

Follow

Liquid AI 7.05k

hey hey hey, ran into this while testing lfm3-1.7b in llama.cpp. the gguf files in LiquidAI/LFM3-1.7B-GGUF have tokenizer.ggml.add_bos_token = false, so when you score raw text (llama-perplexity, plain completion, eval harnesses) the model never gets <|startoftext|> (id 124894) at the start. chat is fine since the template puts it in.

on wikitext-2 that makes a pretty big difference. ppl goes up ~42% at ctx 512 and ~16% at ctx 4096. once i force bos on, llama.cpp lands right on the transformers number, so the conversion itself looks fine, it's just the missing bos.

the hf tokenizer doesn't add it by default either (tok("Hello").input_ids gives [35808], no 124894), so raw text evals through transformers probably hit the same thing unless the harness adds bos itself.

numbers (bf16, wikitext-2 test):

ctx 512, 40 chunks

• llama.cpp as shipped (no bos): 20.9585 +/- 0.65926
• llama.cpp with bos forced on: 14.7372 +/- 0.43107
• transformers, bos in front, same scoring: 14.7322

ctx 4096, 5 chunks

• llama.cpp as shipped (no bos): 14.5160 +/- 0.42331
• llama.cpp with bos forced on: 12.5438 +/- 0.35262
• transformers, bos in front, same scoring: 12.5046

tokenization matches between the two btw, i checked 4,501 of 4,501 tokens on the first 20k characters and the ids are identical.

setup:

• gguf: LiquidAI/LFM3-1.7B-GGUF, LFM3-1.7B-BF16.gguf, repo revision 4b423c332fd97961416f2e8b69467758fd27370d, sha256 39158a79f56edb85e0c4252ce6447b8d7883ef82ffedd02f0cda7c84da5e9f68
• metadata: tokenizer.ggml.add_bos_token = false, tokenizer.ggml.bos_token_id = 124894
• original: LiquidAI/LFM3-1.7B in bfloat16, trust_remote_code=True
• llama.cpp: version 0.8.0-dev (build 1, commit 604cc43), master from 2026-10-28, appleclang 21 on apple silicon (metal)
• data: wikitext-2 raw test, wiki.test.raw (1,290,590 bytes, sha256 173c87a53759e0201f33e0ccf978e510c2042d7f2cb78229d9a50d79b9e7dd08). no preprocessing, the raw file goes straight into -f. on the transformers side i read the whole file as one string and tokenize with add_special_tokens=False

commands, as shipped:

llama-perplexity -m LFM3-1.7B-BF16.gguf -f wiki.test.raw -c 512 --chunks 40 -ngl 99 -t 4
llama-perplexity -m LFM3-1.7B-BF16.gguf -f wiki.test.raw -c 4096 --chunks 5 -ngl 99 -t 4

with bos forced on:

llama-perplexity -m LFM3-1.7B-BF16.gguf -f wiki.test.raw -c 512 --chunks 40 -ngl 99 -t 4 --override-kv tokenizer.ggml.add_bos_token=bool:true
llama-perplexity -m LFM3-1.7B-BF16.gguf -f wiki.test.raw -c 4096 --chunks 5 -ngl 99 -t 4 --override-kv tokenizer.ggml.add_bos_token=bool:true

for the transformers number i scored it the same way llama-perplexity does: tokenize the whole text once with bos in front, cut into ctx sized chunks, swap each chunk's first token for bos, only score the second half of each chunk.

import math, torch
from transformers import AutoModelForCausalLM, AutoTokenizer

C, N = 512, 40   # and 4096, 5
tok = AutoTokenizer.from_pretrained("LiquidAI/LFM3-1.7B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM3-1.7B", dtype=torch.bfloat16,
                                             trust_remote_code=True).to("mps").eval()
ids = [tok.bos_token_id] + tok(open("wiki.test.raw").read(), add_special_tokens=False).input_ids
first, nll, n = C // 2, 0.0, 0
with torch.no_grad():
    for i in range(N):
        chunk = ids[i * C:(i + 1) * C]
        chunk[0] = tok.bos_token_id
        x = torch.tensor([chunk], device="mps")
        lg = model(input_ids=x).logits[0, first:C - 1].float()
        nll += torch.nn.functional.cross_entropy(lg, x[0, first + 1:], reduction="sum").item()
        n += C - 1 - first
print(f"PPL = {math.exp(nll / n):.4f}")

probably the fix is just add_bos_token = true in the gguf metadata (and tokenizer_config.json if raw text is supposed to match training). since the chat template already writes the token, might be worth checking chat prompts don't end up with two of them after that, i didn't test that part. might also be worth a note on the model card in the meantime, third party raw text benchmarks of lfm3 (perplexity, lm-eval loglikelihood stuff) could be underselling it.

Find me on x if you have more questions : iimmyNWJS