kataguru/Industry-Coder-27B-v1.0-W4A16-AWQ (INT4 W4A16 Marlin + Vision + MTP)

Kataguru Industry-Coder-27B-v1.0-W4A16-AWQ is the W4A16 (compressed-tensors, group_size=32, MSE-calibrated) production vLLM release of Industry-Coder-27B-v1.0 — a 27-billion-parameter dense flagship model engineered for industrial software engineering, algorithmic coding (LiveCodeBench v6), agentic tool calling (BFCL V4), industrial automation (PLC/CNC: Beckhoff TwinCAT 3 ST & Siemens SCL), European/Finnish patent engineering (EPO/PRH), and doctoral-level STEM reasoning (GPQA Diamond & GSM8K). All 333 vision tensors and 15 MTP speculative decoding tensors are preserved in full BF16 precision.

🇫🇮 Suomenkielinen dokumentaatio löytyy englanninkielisen osion alta.

---

🇬🇧 English Documentation

1. Summary of Kataguru Modifications

Field	Description / Value
Base model	Qwen 3.8 27B (64 layers: Gated DeltaNet + Full Attention + 333 Vision tensors + 15 MTP speculative tensors)
Kataguru modifications	4-cycle closed-loop Merge-and-Reset curriculum (Cycle 1..4 + Tempering Elite Bank in Cycle 4 with strict non-decreasing difficulty), r=16 qr_frozen_a orthonormal subspace training, SOMA/ARA 0.00% refusal alignment, Kataguru Quad-Mode Chat Template, and W4A16 MSE (group_size=32) quantization
Language quality (EN & FI)	Native-level English and 100% flawless Finnish: natural, idiomatic Finnish across software engineering, industrial automation, patents, and STEM without morphological errors or calques
Quantization	W4A16 pack-quantized (compressed-tensors, group_size=32), with Vision (333 tensors) and MTP (15 tensors) preserved in full BF16
Weights size (Disk)	27.0 GB (7 safetensors shards)
VRAM requirement (vLLM)	2x 16GB/24GB/32GB GPU (TP=2) or 1x 32GB+ GPU (Ampere, Ada Lovelace, Hopper, Blackwell)
What was NOT modified	64-layer hybrid backbone architecture, 333 vision tensors, 15 MTP tensors, and 262,144-token native context window
Primary use cases	Production vLLM clusters: software engineering (Python, Rust, C, Java, JS), agentic <tool_call> workflows, PLC/CNC automation, patent claims, and STEM reasoning
Recommended runtime	vllm serve (--quantization compressed-tensors --tensor-parallel-size 2)
Tested environment	2x NVIDIA GeForce RTX 5090 32GB (TP=2, vLLM v0.29.0-cu129, CUDAGraphs enabled)

---

2. Official 15-Benchmark Evaluation (Measured on 2x NVIDIA RTX 5090 TP=2, CUDAGraphs)

#	Benchmark Task	Metric	Qwen 3.8 27B Base	Kataguru ChemBio-27B v2	Industry-Coder-27B-v1.0	Delta vs. Base
1	GSM8K Math (CoT 1024-tok)	Strict / Flex Exact Match	76.50 %	82.87 %	89.92 % (90.60 %)	+13.42 pp
2	HellaSwag	Linguistic Coherence (acc_norm)	82.10 %	83.48 %	82.74 %	+0.64 pp
3	MedQA	USMLE 4-options (acc)	78.40 %	81.23 %	82.56 %	+4.16 pp
4	BoolQ	Natural Language Logic (acc)	86.40 %	88.35 %	87.86 %	+1.46 pp
5	WinoGrande	Contextual Reasoning (acc)	75.80 %	77.43 %	76.24 %	+0.44 pp
6	TruthfulQA MC2	Multi-True Probability	54.80 %	55.90 %	53.47 %	-1.33 pp
7	TruthfulQA MC1	Single-True Factuality	36.10 %	37.82 %	34.76 %	-1.34 pp
8	PIQA	Physical Intuition (acc_norm)	81.20 %	81.61 %	81.77 %	+0.57 pp
9	OpenBookQA	Scientific Application (acc_norm)	44.20 %	45.00 %	45.60 %	+1.40 pp
10	ARC-Challenge	Advanced STEM Reasoning (acc_norm)	61.10 %	61.26 %	62.12 %	+1.02 pp
11	ARC-Easy	Foundational STEM (acc_norm)	81.30 %	82.66 %	81.99 %	+0.69 pp
$\Sigma_{11}$	11-Task Cognitive Average	Cognitive Composite	66.17 %	70.69 %	70.82 %	+4.65 pp
12	GPQA Diamond (198 PhD STEM)	Exact Match (133/198)	45.20 %	52.02 %	67.17 %	+21.97 pp
13	IFEval Strict	Prompt / Instruction Strict	48.20 % / 58.40 %	63.40 % / 74.10 %	70.43 % / 79.26 %	+22.23 % / +20.86 %
14	LiveCodeBench v6 (175 tasks)	Pass@1 (Sandbox Execution)	26.86 %	32.00 %	36.57 % (64/175)	+9.71 pp
15	BFCL V4 (Single-Turn)	Overall AST (Python 96.25%, Multiple 99.50%)	62.60 %	65.58 %	71.78 %	+9.18 pp
$\Sigma_{15}$	15-Benchmark Composite	Official 15-Benchmark Mean	60.31 %	66.04 %	68.33 %	+8.02 pp

Breakdown of Coding & Tool-Calling Benchmarks

• LiveCodeBench v6 (36.57% Pass@1 across 175 competitive programming tasks):
  • Easy: 86.05% (37 / 43)
  • Medium: 34.62% (18 / 52)
  • Hard: 11.25% (9 / 80)
• BFCL V4 Single-Turn (71.78% Overall, 85.49% Live Overall AST):
  • Python Simple AST: 96.25% (Live 95.35%)
  • Multiple AST (selecting the right tool from multiple candidates): 99.50% (Live 79.68%)
  • JavaScript Simple AST: 80.00%
  • Java Simple AST: 63.00%
  • Irrelevance Detection (abstaining when no tool matches): 54.89% (Non-Live 61.25%)

---

3. Deployment with vLLM

vllm serve kataguru/Industry-Coder-27B-v1.0-W4A16-AWQ \
  --served-model-name industry-coder-27b \
  --tensor-parallel-size 2 \
  --gpu-memory-utilization 0.88 \
  --max-model-len 32768 \
  --quantization compressed-tensors \
  --kv-cache-dtype fp8 \
  --enable-chunked-prefill \
  --trust-remote-code \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

---

4. Chat Template & Reasoning Modes

The model uses the Kataguru Quad-Mode Chat Template:

• Default Mode (Direct Execution / Fast Tool Calling): Produces direct, concise engineering responses or <tool_call> invocations without unnecessary preamble.
• Deep <think> Reasoning Mode: Activated via chat_template_kwargs={"enable_thinking": true} or by prefixing the prompt with {REASON:einstein} (10-angle synthesis) or {REASON:spoon} (7-step first-principles verification).

---

5. Known Limitations & Honest Assessment

• BFCL Parallel Same-Function Calls: While single tool calls (Python Simple AST = 96.25%) and multi-tool selection (Multiple AST = 99.50%) achieve high accuracy, prompts that require invoking the exact same function multiple times in parallel within a single turn (Non-Live Parallel AST = 23.00%, Live Parallel AST = 43.75%) may emit only the first <tool_call> unless explicitly instructed to emit separate tool calls for each item.
• TruthfulQA MC1/MC2 (34.76% / 53.47%): Scores ~1.3 pp below the base model on English urban-legend multiple-choice questions due to heavy specialization in STEM, code execution, and industrial engineering.

---

---

🇫🇮 Suomenkielinen Dokumentaatio

Kataguru Industry-Coder-27B-v1.0-W4A16-AWQ on vLLM-tuotantoklustereille optimoitu W4A16 (compressed-tensors, group_size=32, MSE-optimoitu) -julkaisu, jossa sekä 333 visiotensoria että 15 MTP-spekulaatiotensoria on säilytetty täydellä BF16-tarkkuudella.

1. Mitä Kataguru muutti? (Muutosyhteenveto)

Kenttä	Kuvaus / Arvo
Base model	Qwen 3.8 27B (64 kerrosta: Gated DeltaNet + Full Attention + 333 Visio-tensoria + 15 MTP-tensoria)
Katagurun muutos	4-syklinen Merge-and-Reset -kurrikulum (Sykli 1..4 + Karkaisupankki Syklissä 4), r=16 qr_frozen_a, SOMA/ARA 0.00 % kieltäytyminen ja W4A16 MSE (group_size=32) -kvantisointi
Suomen kielen laatu	100 % Virheetön suomi: Tuottaa virheetöntä suomenkielistä insinööri-, koodaus-, automaatio- ja patenttitekstiä
Kvantisointi	W4A16 pack-quantized (compressed-tensors, group_size=32), Visio ja MTP täydellä BF16-tarkkuudella
Painojen koko (Levytila)	27,0 Gt (7 safetensors-shardia)
VRAM-tarve (vLLM)	2x 16GB/24GB/32GB GPU (TP=2) tai 1x 32GB+ GPU
Mitä EI muutettu	64-kerroksinen hybridirunko, 333 visiotensoria, 15 MTP-tensoria ja 262 144 tokenin konteksti-ikkuna
Ensisijainen käyttö	Tuotantotason vLLM-klusterit (Ampere, Ada, Hopper, Blackwell): ohjelmistotuotanto, agenttityökalut (qwen3_coder), PLC/CNC-automaatio ja STEM
Suositeltu ajotapa	vllm serve (--quantization compressed-tensors --tensor-parallel-size 2)
Testattu ympäristö	2x NVIDIA GeForce RTX 5090 32GB (TP=2, vLLM v0.29.0-cu129)

2. Tunnetut Rajoitukset

• BFCL Parallel AST (saman funktion toistokutsut yhdessä vuorossa): 23.00 % Non-Live / 43.75 % Live (Simple Python AST = 96.25 %, Multiple AST = 99.50 %).