Instructions to use efebaykara/Octen-WebGR-0.6B-Gundem-LoRA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use efebaykara/Octen-WebGR-0.6B-Gundem-LoRA with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Octen/Octen-WebGR-0.6B")
model = PeftModel.from_pretrained(base_model, "efebaykara/Octen-WebGR-0.6B-Gundem-LoRA")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use efebaykara/Octen-WebGR-0.6B-Gundem-LoRA with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M

Use Docker

docker model run hf.co/efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use efebaykara/Octen-WebGR-0.6B-Gundem-LoRA with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "efebaykara/Octen-WebGR-0.6B-Gundem-LoRA"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "efebaykara/Octen-WebGR-0.6B-Gundem-LoRA",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M

Ollama
How to use efebaykara/Octen-WebGR-0.6B-Gundem-LoRA with Ollama:
```
ollama run hf.co/efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M
```
Unsloth Desktop

How to use efebaykara/Octen-WebGR-0.6B-Gundem-LoRA with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use efebaykara/Octen-WebGR-0.6B-Gundem-LoRA with Docker Model Runner:
```
docker model run hf.co/efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M
```

Lemonade

How to use efebaykara/Octen-WebGR-0.6B-Gundem-LoRA with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M

Run and chat with the model

lemonade run user.Octen-WebGR-0.6B-Gundem-LoRA-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use efebaykara/Octen-WebGR-0.6B-Gundem-LoRA with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use efebaykara/Octen-WebGR-0.6B-Gundem-LoRA with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "efebaykara/Octen-WebGR-0.6B-Gundem-LoRA:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Octen-WebGR-0.6B-Gundem-LoRA

LoRA adapter for Turkish news headline classification. Requires the pinned base model; this is not a standalone full checkpoint or a general chat assistant.

Base: Octen/Octen-WebGR-0.6B · revision fafd37856a14996b2627a450063e63a007afe6b5.

DeepSeek-generated answers were used for assistant-only supervised fine-tuning (response distillation). No DeepSeek weights or logits were used; the student's parameter count is unchanged. The split contains 825 training, 107 validation and 101 test examples. Three epochs, lr=0.0002, LoRA r16/alpha32, accumulation=8, max_length=512, seed=42.

Use

Download this repository, install a suitable CUDA PyTorch build and the supplied requirements, then run from the downloaded folder:

python -m pip install -r requirements.txt
python release_infer.py --model . --headline "İşçiler ücret artışı için greve başladı"

The supplied loader handles architecture compatibility, the exact task template, pinned base revision, CPU fp32 adapter merging and CUDA BF16 inference. Allow CPU RAM for the full fp32 base model and GPU memory for BF16 inference. Loading the base model executes its remote model code. Review it before use. The result contains prediction or an explicit error; generation is not constrained by a JSON grammar.

The model receives only the headline and student-system.txt. Output fields: etiketler (topics), tur (event type), and etki, ciddiyet, aciliyet (integer scores 0–3).

Evaluation

Metric	Value
Gündem Score v1	70.6 / 100
All-field exact match	23.8%
Topic micro-F1	86.2%
Type accuracy	66.3%
Invalid outputs	0 / 101

HF/CUDA BF16, greedy, batch=4, maximum 160 new tokens, no JSON grammar. Gündem Score = 30×topic F1 + 20×type accuracy + 30×(1−mean score MAE/3) + 20×exact match (fractional metrics). It measures this task's teacher-label agreement, not general LLM intelligence or an accuracy percentage. Small differences on this previously evaluated, single-seed test do not establish superiority. Batch size and numerical runtime changes can affect outputs. See evaluation.json and release-manifest.json.

Licensing and limitations

Gündem response-distillation LoRA adaptation of Octen/Octen-WebGR-0.6B, revision fafd37856a14996b2627a450063e63a007afe6b5. Modified artifact: adapter weights trained on DeepSeek-labelled Turkish headlines. Model and tokenizer licensing follows the upstream license files. Gündem helper code: MIT.

Non-commercial (CC BY-NC 4.0). Octen is a specialized URL retriever; this release tests transfer to news JSON, not retrieval quality. See LICENSE and NOTICE.

The task uses a small, domain-specific dataset. Inspect predictions before relying on them. Raw headlines, teacher answers and the full dataset are not redistributed. Model licensing is separate from the MIT-licensed Gündem helper code.

Training and benchmark tools · All released models

Mobile GGUF pair

Original and fine-tuned Q4_K_M files, tested with the Gündem native engine.

Variant	GGUF	Gündem Score v1 (HF/BF16)
original	octen-webgr-0.6b-original-Q4_K_M.gguf	0.0
distilled	octen-webgr-0.6b-distilled-Q4_K_M.gguf	70.6

These scores come from the HF/BF16 experiment; mobile GGUF quality was not rescored. Both files retain the base model license and notices included in this repository. The original model is not fine-tuned.