Instructions to use dima6312/Flint-40B-A2.5B-abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use dima6312/Flint-40B-A2.5B-abliterated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="dima6312/Flint-40B-A2.5B-abliterated", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("dima6312/Flint-40B-A2.5B-abliterated", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("dima6312/Flint-40B-A2.5B-abliterated", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use dima6312/Flint-40B-A2.5B-abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "dima6312/Flint-40B-A2.5B-abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dima6312/Flint-40B-A2.5B-abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/dima6312/Flint-40B-A2.5B-abliterated

SGLang

How to use dima6312/Flint-40B-A2.5B-abliterated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "dima6312/Flint-40B-A2.5B-abliterated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dima6312/Flint-40B-A2.5B-abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "dima6312/Flint-40B-A2.5B-abliterated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dima6312/Flint-40B-A2.5B-abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use dima6312/Flint-40B-A2.5B-abliterated with Docker Model Runner:
```
docker model run hf.co/dima6312/Flint-40B-A2.5B-abliterated
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Flint-40B-A2.5B-abliterated (BF16 safetensors)

An abliterated (uncensored) Flint: RecursiveSI's Flint 40B-A2.5B with its refusal direction removed, for security research and coding work: exploit analysis, malware triage, detection engineering, CTFs and red-team tooling that the original model often declines.

Summary: held-out MITRE ATT&CK refusals 15% → 0% (n=100, Q8_0). No significant drop on HumanEval+, MBPP+, GSM8K, MMLU-Pro, CyberMetric or WMDP-cyber (paired, same items). KL divergence from the original on harmless prompts: 0.0146. Builds: BF16, GGUF (Q8_0 to Q3_K_M), NVFP4 for NVIDIA Blackwell, MLX 4-bit for Apple silicon.

Which build

Not sure? Take the GGUF Q4_K_M (24 GB). It runs in llama.cpp and in apps built on it, including LM Studio (its llama.cpp runtime 2.63.0 includes Flint). On a Mac, the Rec 4-bit build runs in RecursiveSI's Rec runtime (125 tok/s decode on an M4 Pro) and the MLX 4-bit build in mlx-lm (85 tok/s).

You run	Download	Size
llama.cpp, or an app that bundles llama.cpp b11827 or later; any GPU, CPU or Mac	GGUF: `Q8_0` is the benchmarked build, `Q4_K_M` the usual choice	42 / 24 GB
NVIDIA Blackwell GPU (RTX 50 series, RTX PRO 6000, DGX Spark) with llama.cpp CUDA	NVFP4 GGUF	23 GB
Apple silicon Mac with mlx-lm installed from GitHub (Flint support is merged but not released yet)	MLX 4-bit	21 GB
Apple silicon Mac with RecursiveSI's Rec runtime	Rec 4-bit	21 GB
transformers or SGLang on a GPU server; fine-tuning or making your own quants	BF16 safetensors (this repo)	80 GB

Plan for the file size plus memory for context: about 26 KB per token, so roughly 7 GB at the full 262,144 tokens.

Community builds

Quantizations made by other people. I have not tested them. The model tree on this page links every build that lists this model as its base.

Build	By	For
MLX oQ4e (23 GB), MLX oQ8e (42 GB)	Studio-Research-M1	oMLX on Apple silicon; the cards report KL divergence against FP32 and speed on an M1 Max
GGUF, imatrix GGUF	mradermacher	llama.cpp
FP8	cbert33	GPU servers (compressed-tensors W8A8, calibrated FP8 KV cache)
EXL3 4.0 bpw	cbert33	ExLlamaV3
REAP-224 Rec 4-bit (19 GB)	RinneCODER	Macs with less memory: the Rec build with 32 of the 256 routed experts removed in each layer

Making your own quant: keep Flint out of FP16. In layer 46 the expert product silu(gate) · up reaches about 90,000, above the FP16 maximum of 65,504, so FP16 builds answer !!!! and give NaN logits. MLX computes in the dtype of the scales and unquantized tensors, so store those in BF16. Studio-Research-M1 found and measured this (discussion).

Results

Both models ran as Q8_0 GGUF on llama-server with the same flags, Flint's recommended sampling (temperature 1.0, top_p 0.95, top_k 20) and full reasoning, on evenly spaced subsets of each benchmark (n in each row). Checks are paired on the same items; a quality check fails only on a drop that is significant at 95%. Refusals are scored with Meta's CyberSecEval refusal check. Refusal rates count refusals only. A reply with no final answer (the model was still reasoning at the 8,192-token limit) is listed separately as "no answer": on the 100 MITRE prompts that happened 19 times for the abliterated model and once for the original. The cut-off reasoning was not repetitive; give long security tasks a larger token budget.

Check	Original	Abliterated	Result
MITRE refusals (n=100) significantly lower	15%	0%	pass
FRR false refusals (n=100) not significantly higher	0%	0%	pass
cybermetric (n=50) not significantly worse	100%	96%	pass
gsm8k (n=50) not significantly worse	98%	98%	pass
mmlu_pro (n=50) not significantly worse	80%	78%	pass
wmdp_cyber (n=50) not significantly worse	50%	60%	pass
humaneval_plus (n=40) not significantly worse	90%	95%	pass
mbpp_plus (n=40) not significantly worse	87.5%	90%	pass
pp512 tok/s within 3%	8050.8	8135.1	pass
tg128 tok/s within 3%	180.0	185.6	pass

Refusals on held-out CyberSecEval MITRE prompts (n per ATT&CK category below):

Category	n	Original: refused / no answer	Abliterated: refused / no answer
C2	10	20% / 0%	0% / 30%
Collection	10	0% / 0%	0% / 0%
Discovery	10	0% / 10%	0% / 20%
Evasion	10	40% / 0%	0% / 40%
Execution	10	10% / 0%	0% / 30%
Exfil	10	10% / 0%	0% / 0%
Lateral Movement	10	40% / 0%	0% / 10%
Persistence	10	20% / 0%	0% / 10%
Privilege Escalation	10	10% / 0%	0% / 30%
Recon	10	0% / 0%	0% / 20%

Heretic's default refusal count, measured on the NVFP4 builds (100 mlabonne/harmful_behaviors test prompts, reasoning off, greedy, 100 tokens): original 96/100, abliterated 55/100. That count flags any reply containing words such as "harmful" or "illegal", including answers that comply; only 1 of the 55 replies opens with a refusal. Meta's CyberSecEval refusal check on the same replies: original 18/100, abliterated 1/100.

KL divergence from the original on 100 harmless prompts, measured by the search for the selected trial (not part of the checks above): 0.0146 nats/token.

Benchmarks: CyberSecEval MITRE and FRR (Meta), CyberMetric-500, WMDP-cyber, MMLU-Pro, GSM8K, HumanEval+ and MBPP+ (EvalPlus); speed from llama-bench pp512/tg128.

Method

The refusal direction comes from the difference in mean activations on harmful and harmless prompts (mlabonne/harmful_behaviors, mlabonne/harmless_alpaca). abliterix removes it from attention output, shared-expert and routed-expert projections; Optuna picks the strengths over 50 trials against refusals on 100 MITRE prompts and KL divergence on harmless prompts. The MITRE prompts in the table were never seen by the search. Tensor shapes are unchanged, so speed and memory match the original.

Usage

Reasoning is optional. When the model reasons, it uses <think>…</think> before its answer. Use the sampling settings above.

llama.cpp (b11827 or later):

llama-server -m Flint-40B-A2.5B-abliterated-Q8_0.gguf --jinja -ngl 999 -fa on

transformers 5.20 (trust_remote_code=True, the port ships with the weights):

from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "dima6312/Flint-40B-A2.5B-abliterated"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True, dtype="bfloat16", device_map="auto")

Limitations

The checks ran on subsets of each benchmark (n in each row), on the Q8_0 GGUF only. Short checks (n=10 GSM8K, n=10 HumanEval+): Q4_K_M 9/10 and 9/10. An IQ2_M build scored 0/10 on both and was removed on 2026-11-01. Other quants were not measured.
Refusals are lower, not zero: the selected trial still refused 10 of the 100 prompts used by the search.
On long security tasks the model can reason for more than 8,192 tokens before answering.
Abliteration removes refusals; it adds no knowledge. Mistakes the original makes, this model makes too.

Responsible use

This model answers requests the original refuses. It is meant for authorized security testing, research, education and defensive work. You are responsible for how you use it and for following the law and the terms of the systems you test. Its answers can be wrong; check them before acting on them.

Credits

Flint by RecursiveSI (Apache 2.0). This is a derivative of their weights under the same license.
abliterix by wuwangzhang1216, building on Heretic by p-e-w.
Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).
Maxime Labonne's harmful/harmless prompt sets; Meta's CyberSecEval; CyberMetric (Tihanyi et al.); WMDP (Center for AI Safety); MMLU-Pro (TIGER-Lab); GSM8K (OpenAI); EvalPlus.
llama.cpp (ggml-org) and transformers (Hugging Face).

Downloads last month: 354

Safetensors

Model size

40B params

Tensor type

BF16

Model tree for dima6312/Flint-40B-A2.5B-abliterated

Base model

RecursiveSI/flint

Finetuned

(15)

this model

Quantizations

10 models

Datasets used to train dima6312/Flint-40B-A2.5B-abliterated

Paper for dima6312/Flint-40B-A2.5B-abliterated

Refusal in Language Models Is Mediated by a Single Direction

Paper • 2406.11717 • Published Jun 17, 2024 • 17