Text Classification
Transformers
Safetensors
English
qwen3_5
image-text-to-text
decision-model
classification
calibration
routing
guardrails
vision
multimodal
baseten
Eval Results
Instructions to use baselabs/joey-1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use baselabs/joey-1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="baselabs/joey-1")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("baselabs/joey-1") model = AutoModelForMultimodalLM.from_pretrained("baselabs/joey-1", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Match the elephant-1 and mammoth-1 model card; add the banner
Browse files- .gitattributes +1 -0
- README.md +32 -16
- assets/banner.png +3 -0
.gitattributes
CHANGED
|
@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
assets/banner.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -16,16 +16,28 @@ tags:
|
|
| 16 |
- baseten
|
| 17 |
---
|
| 18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
# joey-1
|
| 20 |
|
| 21 |
joey-1 is a 2B parameter **decision model** post-trained from [Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) by Base Labs at Baseten.
|
| 22 |
Provide the model a state (text, optionally with images) and a set of questions and it will return a calibrated probability for every option,
|
| 23 |
answering all questions from **one shared-prefix prefill**. On tested benchmarks:
|
| 24 |
|
| 25 |
-
- **
|
| 26 |
- **62.9 on a replica of the Artificial Analysis Decision Index**.
|
| 27 |
|
| 28 |
-
For tasks that need
|
| 29 |
|
| 30 |
## ποΈ Model Details
|
| 31 |
|
|
@@ -43,6 +55,11 @@ For tasks that need more accuracy, see the 9B sibling [elephant-1](https://huggi
|
|
| 43 |
|
| 44 |
## π How to use
|
| 45 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
### Self-hosted
|
| 47 |
|
| 48 |
`setup.sh` installs the pinned serving runtime (OneJev + `serve.patch`). It needs one CUDA GPU and the `pytorch/pytorch:2.8.0-cuda12.8-cudnn9-devel` image.
|
|
@@ -87,33 +104,32 @@ Requests longer than the trained length limits are refused, not truncated.
|
|
| 87 |
|
| 88 |
## π Performance
|
| 89 |
|
| 90 |
-
### Jev Decision Index
|
| 91 |
|
| 92 |
-
| Model |
|
| 93 |
|---|---:|
|
| 94 |
-
| elephant-1 | 72.
|
| 95 |
-
| **joey-1** | **63.
|
|
|
|
|
|
|
|
|
|
| 96 |
|
| 97 |
-
|
| 98 |
-
|---|---:|
|
| 99 |
-
| Knowledge & reasoning | 50.0 |
|
| 100 |
-
| Language | 62.8 |
|
| 101 |
-
| Retrieval & classification | 71.2 |
|
| 102 |
-
| Tools & automation | 85.8 |
|
| 103 |
-
| Arts & human taste | 54.8 |
|
| 104 |
|
| 105 |
### Artificial Analysis Decision Index (our replica)
|
| 106 |
|
| 107 |
| Model | Score |
|
| 108 |
|---|---:|
|
|
|
|
| 109 |
| elephant-1 | 65.2 |
|
|
|
|
| 110 |
| **joey-1** | **62.9** |
|
| 111 |
|
| 112 |
-
## π§ Training
|
| 113 |
|
| 114 |
-
- **Data**: 6.21M decision examples (Mix 8),
|
| 115 |
- **Objective**: cross-entropy plus a Brier calibration term (Ξ» = 1), with a trained "I don't know" option.
|
| 116 |
-
- **Recipe**: 1 epoch (43,390 steps), peak LR 1e-5, AdamW (no weight decay), 3% warm-up, cosine decay, bf16.
|
| 117 |
- **Hardware**: 2 nodes Γ 8 H100 (16 GPUs).
|
| 118 |
|
| 119 |
## Citation
|
|
|
|
| 16 |
- baseten
|
| 17 |
---
|
| 18 |
|
| 19 |
+
<div align="center">
|
| 20 |
+
<img
|
| 21 |
+
src="https://huggingface.co/baselabs/joey-1/resolve/main/assets/banner.png"
|
| 22 |
+
alt="Baseten joey-1 decision model"
|
| 23 |
+
style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;"
|
| 24 |
+
/>
|
| 25 |
+
<div style="display: flex; justify-content: center; gap: 0.5em; margin-bottom: 1em;">
|
| 26 |
+
<a href="https://www.baseten.co/library/joey-1/"><strong>Try on Baseten</strong></a> β’
|
| 27 |
+
<a href="https://labs.baseten.co/articles/mammoth-1-and-elephant-1-frontier-decision-models"><strong>Technical report</strong></a>
|
| 28 |
+
</div>
|
| 29 |
+
</div>
|
| 30 |
+
|
| 31 |
# joey-1
|
| 32 |
|
| 33 |
joey-1 is a 2B parameter **decision model** post-trained from [Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) by Base Labs at Baseten.
|
| 34 |
Provide the model a state (text, optionally with images) and a set of questions and it will return a calibrated probability for every option,
|
| 35 |
answering all questions from **one shared-prefix prefill**. On tested benchmarks:
|
| 36 |
|
| 37 |
+
- **Best model under 4B on the Jev Decision Index 0.4 public split**: 63.30, #37 of 230 models.
|
| 38 |
- **62.9 on a replica of the Artificial Analysis Decision Index**.
|
| 39 |
|
| 40 |
+
For tasks that need broader general knowledge, see the 9B sibling [elephant-1](https://huggingface.co/baselabs/elephant-1) and the 27B [mammoth-1](https://huggingface.co/baselabs/mammoth-1).
|
| 41 |
|
| 42 |
## ποΈ Model Details
|
| 43 |
|
|
|
|
| 55 |
|
| 56 |
## π How to use
|
| 57 |
|
| 58 |
+
### Baseten Model APIs
|
| 59 |
+
|
| 60 |
+
joey-1 is available on [Baseten Model APIs](https://www.baseten.co/library/joey-1/) at $0.01 per 1M input tokens, with zero data retention on by default. The endpoint accepts text and image inputs.
|
| 61 |
+
|
| 62 |
+
|
| 63 |
### Self-hosted
|
| 64 |
|
| 65 |
`setup.sh` installs the pinned serving runtime (OneJev + `serve.patch`). It needs one CUDA GPU and the `pytorch/pytorch:2.8.0-cuda12.8-cudnn9-devel` image.
|
|
|
|
| 104 |
|
| 105 |
## π Performance
|
| 106 |
|
| 107 |
+
### Jev Decision Index 0.4 (public split)
|
| 108 |
|
| 109 |
+
| Model | Decision Index |
|
| 110 |
|---|---:|
|
| 111 |
+
| elephant-1 | 72.24 |
|
| 112 |
+
| **joey-1** | **63.30** |
|
| 113 |
+
| Cloudflare Clef | 61.71 |
|
| 114 |
+
| Jev 1.14 | 60.70 |
|
| 115 |
+
| Perplexity Decider Flash v1.2 | 60.06 |
|
| 116 |
|
| 117 |
+
On the public split, joey-1 trails elephant-1 in every area and beats Jev 1.14 in four of five (language, tools & automation, retrieval & classification, arts & human taste).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
|
| 119 |
### Artificial Analysis Decision Index (our replica)
|
| 120 |
|
| 121 |
| Model | Score |
|
| 122 |
|---|---:|
|
| 123 |
+
| mammoth-1 | 65.9 |
|
| 124 |
| elephant-1 | 65.2 |
|
| 125 |
+
| d1-ultra | 65 |
|
| 126 |
| **joey-1** | **62.9** |
|
| 127 |
|
| 128 |
+
## π§ Training (all details in the technical report)
|
| 129 |
|
| 130 |
+
- **Data**: 6.21M decision examples (Mix 8). 58.3% core decision skills, 16.1% safety and abstention, 16.1% judging and expert domains, 9.5% public decision data.
|
| 131 |
- **Objective**: cross-entropy plus a Brier calibration term (Ξ» = 1), with a trained "I don't know" option.
|
| 132 |
+
- **Recipe**: 1 epoch (43,390 steps), peak LR 1e-5, AdamW (Ξ² 0.9/0.95, no weight decay), 3% warm-up, cosine decay, bf16.
|
| 133 |
- **Hardware**: 2 nodes Γ 8 H100 (16 GPUs).
|
| 134 |
|
| 135 |
## Citation
|
assets/banner.png
ADDED
|
Git LFS Details
|