Instructions to use efeisot/lfm3-1.7b-1rco-2mix-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use efeisot/lfm3-1.7b-1rco-2mix-gguf with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf efeisot/lfm3-1.7b-1rco-2mix-gguf
# Run inference directly in the terminal:
llama cli -hf efeisot/lfm3-1.7b-1rco-2mix-gguf

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf efeisot/lfm3-1.7b-1rco-2mix-gguf
# Run inference directly in the terminal:
llama cli -hf efeisot/lfm3-1.7b-1rco-2mix-gguf

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf efeisot/lfm3-1.7b-1rco-2mix-gguf
# Run inference directly in the terminal:
./build/bin/llama cli -hf efeisot/lfm3-1.7b-1rco-2mix-gguf

Use Docker

docker model run hf.co/efeisot/lfm3-1.7b-1rco-2mix-gguf

Llama new
LM Studio
Jan

vLLM

How to use efeisot/lfm3-1.7b-1rco-2mix-gguf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "efeisot/lfm3-1.7b-1rco-2mix-gguf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "efeisot/lfm3-1.7b-1rco-2mix-gguf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/efeisot/lfm3-1.7b-1rco-2mix-gguf

Ollama
How to use efeisot/lfm3-1.7b-1rco-2mix-gguf with Ollama:
```
ollama run hf.co/efeisot/lfm3-1.7b-1rco-2mix-gguf
```
Unsloth Desktop

How to use efeisot/lfm3-1.7b-1rco-2mix-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf efeisot/lfm3-1.7b-1rco-2mix-gguf

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "efeisot/lfm3-1.7b-1rco-2mix-gguf"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use efeisot/lfm3-1.7b-1rco-2mix-gguf with Docker Model Runner:
```
docker model run hf.co/efeisot/lfm3-1.7b-1rco-2mix-gguf
```

Lemonade

How to use efeisot/lfm3-1.7b-1rco-2mix-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull efeisot/lfm3-1.7b-1rco-2mix-gguf

Run and chat with the model

lemonade run user.lfm3-1.7b-1rco-2mix-gguf-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use efeisot/lfm3-1.7b-1rco-2mix-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf efeisot/lfm3-1.7b-1rco-2mix-gguf

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default efeisot/lfm3-1.7b-1rco-2mix-gguf

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use efeisot/lfm3-1.7b-1rco-2mix-gguf with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf efeisot/lfm3-1.7b-1rco-2mix-gguf

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "efeisot/lfm3-1.7b-1rco-2mix-gguf" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

LFM3-1.7B GGUF collection (2 mixed K-quants + 1 RCO)

Three GGUFs of LiquidAI/LFM3-1.7B-Base:

balanced — kmix-1100M: standard imatrix K-quant mix, plain stock llama.cpp.
quality — kmix-1300M: standard imatrix K-quant mix, plain stock llama.cpp.
sub-1-GiB RCO — rcmix-1000M: experimental RCO mixed quant that squeezes the model under 1 GiB; requires a patched llama.cpp (see below).

Quality (Wikitext-2 test, 512 ctx, 16 chunks, KL vs f16 logits)

model	GiB	bpw	PPL	KLD	top1	runtime
ad-IQ3_S (ref)	—	—	—	0.203	77.0%	stock
stock Q4_K_M baseline	1.024	5.09	23.803	0.119	83.2%	stock
kmix-1100M (balanced)	1.083	5.38	23.736	0.090	84.8%	stock
kmix-1300M (quality)	1.277	6.35	23.030	0.024	92.7%	stock
rcmix-1000M (sub-1GiB RCO)	0.985	4.89	27.275	0.178	79.9%	patch required

Notes:

Among several RCO budget points tested (850M / 950M / 980M / 1000M / 1024M), rcmix-1000M was the best KLD-per-size point under 1 GiB (next best 980M scores KLD 0.189; 1024M ties it at 1.008 GiB).
rcmix-1000M trades global accuracy for size: it fits the 1-GiB slot no stock format reaches (Q4_K_M at the same 1.02 GiB is 2× its wall size). MSE layer reconstruction was verified on hold-out (4-bit proto MSE 0.132 → 0.121).

Files

file	size	what it is
lfm3-1.7b-kmix-1100M.gguf	1.083 GiB	balanced, 5.38 bpw — Q4_K/Q5_K mix, works with stock llama.cpp
lfm3-1.7b-kmix-1300M.gguf	1.277 GiB	quality, 6.35 bpw — Q4_K/Q5_K mix, works with stock llama.cpp
lfm3-1.7b-rcmix-1000M.gguf	0.985 GiB	sub-1-GiB RCO mix (101× RCO4 + 41× Q5_K + Q6_K emb)
lfm3-1.7b-imatrix.gguf	3.5 MiB	importance matrix, for reproduction
assign-lfm3-rcmix-1000M.txt	—	readable per-tensor type sheet of the RCO quant
metadata/assign-1000M.json	—	machine-readable assignment (allocator output)
metadata/final_summary.json	—	machine-readable eval results + budget sweep + RCO tensor composition
metadata/benchmarks/	—	raw PPL/KLD/top-1 eval logs (kmix, Q4_K_M)
docs/USAGE.md	—	runtime matrix, stock vs patched build, server examples
docs/QUANTIZATION.md	—	kmix allocation + RCO block format & pipeline details
docs/RELEASE_NOTES.md	—	release contents, selection rationale, known issues
docs/RAPOR.tr.md	—	Turkish technical report
SHA256SUMS	—	checksums for all release files
LICENSE	—	base model license (lfm1.0)
llama.cpp-rco-types.patch	—	RCO4 support patch (only needed for `rcmix-1000M`)

Requirements

The kmix files run on any recent llama.cpp.

RCO4 (used by rcmix-1000M) is not in stock gguf-py / stock llama.cpp. Apply the included patch (vs. upstream 1b4554e):

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && git checkout 1b4554e
patch -p1 < llama.cpp-rco-types.patch
cmake -B build && cmake --build build --config Release

Usage

./llama-cli -m lfm3-1.7b-kmix-1100M.gguf -ngl 99 -p "def merge_sort(arr):"   # stock
./llama-cli -m lfm3-1.7b-rcmix-1000M.gguf -ngl 99 -p "def merge_sort(arr):" # patched build

Technique

kmix: scripts/alloc_mixed_kquants.py allocated Q4_K/Q5_K/Q6_K per tensor under size budgets with the included imatrix; quantized with llama-quantize. Details: docs/QUANTIZATION.md.
rcmix: RCO layer database (GPTQ-style Hessian DB built on CPU); blocks reconstructed with MSE-optimized orthogonalized codebooks, then a budgeted allocator picks RCO4 vs Q5_K per tensor (fixed walls: emb Q6_K, attn_gate Q8_0, shortconv F32 verbatim).

Raw eval logs are under metadata/benchmarks/ and the full budget sweep in metadata/final_summary.json.

Limitations

rcmix-1000M requires the patched llama.cpp — a stock build rejects the unknown tensor type 45 (RCO4). The two kmix files have no such restriction.
The RCO kernel path is validated on CPU and HIP (ROCm); CUDA builds are untested.
rcmix-1000M is lossy at ~4.9 bpw: KLD 0.178 vs 0.119 for same-size Q4_K_M. Prefer the kmix files when runtime compatibility or global accuracy matters more than the sub-1-GiB footprint.
Benchmark numbers are hardware-specific (RX 9060 XT / gfx1200, ROCm 7.x in the patched runtime).

License and attribution

The base model is licensed by LiquidAI under LFM Open License (lfm1.0); this quantized artifact is distributed under the same terms — see LICENSE, reproduced from the base model's license. The llama.cpp runtime patch is derived from llama.cpp (MIT). You are responsible for complying with the base model license and any applicable usage restrictions.

Citation

@misc{lfm3,
  title  = {LFM3: Compact Heterogeneous Neural Network Architectures},
  author = {LiquidAI},
  year   = {2026},
  url    = {https://huggingface.co/LiquidAI/LFM3-1.7B-Base}
}

@article{helcig2026rco,
  title   = {Model Compression with Exact Budget Constraints via Riemannian Manifolds},
  author  = {Helcig, Michael and Alistarh, Dan},
  journal = {arXiv preprint arXiv:2605.00649},
  year    = {2026}
}

Reproduction tooling: RCO inference/quantizer port (RCO on ROCm).