Instructions to use Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO")
model = AutoModelForCausalLM.from_pretrained("Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO

SGLang

How to use Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO with Docker Model Runner:
```
docker model run hf.co/Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO
```

Llama-3.2-1B-Instruct, TOFU forget10, unlearned with NPO

open-unlearning/tofu_Llama-3.2-1B-Instruct_full (Llama-3.2-1B-Instruct fine-tuned on all 4,000 TOFU QA pairs) after unlearning the forget10 split (20 fictitious authors) with NPO.

Algorithm

NPO (Negative Preference Optimization; Zhang et al., 2024): $\mathcal{L} = \gamma,\tfrac{2}{\beta},\mathbb{E}{\text{forget}}\big[\log\big(1+(\pi_\theta/\pi{\text{ref}})^{\beta}\big)\big] + \alpha,\mathrm{NLL}(\text{retain})$, with $\pi_{\text{ref}}$ the frozen starting model.

Data

TOFU contains 200 fictitious authors with 20 QA pairs each. forget10 holds 10% of them (20 authors, 400 QA pairs); retain90 holds the other 180 authors. The oracle is the same base model fine-tuned on retain90 only, i.e. a model that never saw the forget set.

Unlearning setup

Hyperparameter	Value
Start model	`open-unlearning/tofu_Llama-3.2-1B-Instruct_full`
Forget set	TOFU `forget10` (400 QA pairs, 20 fictitious authors)
Retain set	TOFU `retain90` (3,600 QA pairs)
Framework	OpenUnlearning (trainer `NPO`)
Optimizer	paged AdamW (32-bit), weight decay 0.01
Learning rate	1e-05, linear decay, 12 warm-up steps (about 1 epoch)
Batch size	8 x 4 gradient accumulation = 32 (forget), same for retain
Epochs	10
Grad clip	1.0
Precision	fp32 master weights, bf16 autocast
Seed	0
This checkpoint	step 120 (epoch 9.2)
NPO beta	0.1
NPO gamma (forget weight)	1.0
NPO alpha (retain weight)	1.0
NPO retain loss	NLL
NPO reference model	frozen copy of the starting model

Hyperparameter selection. OpenUnlearning defaults (10 epochs), no tuning.

Evaluation

OpenUnlearning's TOFU metrics (bf16), computed on the two halves of forget10 (A: even-indexed questions of each author, B: odd-indexed; 200 QA pairs each). Q-A prob (length-normalised probability of the correct answer), ROUGE-L (recall of the greedy answer) and truth ratio are per-question means, so the forget10 value is the mean of the two halves. Forget quality is the p-value of a KS test between this model's truth ratios and the oracle's, reported per half (higher = closer to the oracle). Model utility is the harmonic mean of probability, ROUGE and truth ratio on the retain set, real authors and world facts.

Goal of unlearning: forget metrics move toward the oracle, model utility stays near the starting model.

Model	Forget Q-A prob ↓	Forget ROUGE-L ↓	Forget truth ratio	Forget quality ↑ (half A / half B)	Model utility ↑
TOFU fine-tuned (starting point)	0.880	0.819	0.476	2.0e-15 / 2.4e-10	0.599
Retrained on retain90 (oracle)	0.116	0.378	0.628	1.00 / 1.00	0.592
GradDiff	0.145	0.346	0.513	4.7e-05 / 1.2e-04	0.580
GradDiff-SAM	0.722	0.574	0.494	1.1e-14 / 1.9e-09	0.593
GradDiff-ILU	0.465	0.470	0.513	2.4e-10 / 1.5e-07	0.591
GradDiff-LAT	0.830	0.682	0.481	4.7e-15 / 2.4e-10	0.592
NPO (this model)	0.044	0.275	0.618	0.63 / 0.47	0.576
NPO-SAM	0.026	0.256	0.641	0.11 / 0.63	0.587
NPO-ILU	0.039	0.301	0.644	0.18 / 0.27	0.572
NPO-LAT	0.003	0.156	0.670	0.07 / 0.01	0.582
RMU	0.023	0.185	0.696	0.33 / 0.02	0.567
RMU-SAM	0.204	0.357	0.555	8.1e-08 / 1.8e-05	0.510
RMU-ILU	0.791	0.701	0.511	4.7e-15 / 3.6e-09	0.597
RMU-LAT	0.129	0.335	0.592	4.7e-05 / 3.0e-03	0.529
SimNPO	0.026	0.337	0.622	0.27 / 0.55	0.554
SimNPO-SAM	0.116	0.347	0.630	0.07 / 0.47	0.570
SimNPO-ILU	0.026	0.318	0.619	0.55 / 0.71	0.580
SimNPO-LAT	0.697	0.535	0.517	3.1e-12 / 1.5e-07	0.564

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO", torch_dtype="auto")
tok = AutoTokenizer.from_pretrained("Lordvarun23/Llama-3.2-1B-Instruct-TOFU-forget10-NPO")