d3
6 TagsUpdated Apache-2.0by vLLM Semantic Router
Decision 3.0 by the vLLM Semantic Router contributors: a fine-tuned Qwen3.5 whose full-attention layers read the whole prompt, with a readout over 255 answer codes. Every question is its own forward pass. Five sizes, 0.6B to 9B.
ollaya run d3 --preset triage "I was charged twice for my subscription this month and want a refund."curl http://localhost:11435/api/decide \
-H "Content-Type: application/json" \
-d '{
"model": "d3",
"state": "I was charged twice for my subscription this month and want a refund.",
"questions": {
"department": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "Payments, invoices and refunds",
"technical": "Bugs, errors and outages",
"account": "Login, profile and settings"
}
},
"refund": {
"type": "noul",
"instructions": "Is the customer asking for a refund?"
}
}
}'# Already using a TypeSafe SDK? Set TYPESAFE_BASE_URL=http://localhost:11435 instead.
import requests
response = requests.post(
"http://localhost:11435/api/decide",
json={
"model": "d3",
"state": "I was charged twice for my subscription this month and want a refund.",
"questions": {
"department": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "Payments, invoices and refunds",
"technical": "Bugs, errors and outages",
"account": "Login, profile and settings"
}
},
"refund": {
"type": "noul",
"instructions": "Is the customer asking for a refund?"
}
}
},
)
answers = response.json()["answers"]
print(answers["department"]["choice"], answers["refund"]["noul"])const response = await fetch("http://localhost:11435/api/decide", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
model: "d3",
state: "I was charged twice for my subscription this month and want a refund.",
questions: {
department: {
type: "choice",
instructions: "Which team should handle this?",
criteria: {
billing: "Payments, invoices and refunds",
technical: "Bugs, errors and outages",
account: "Login, profile and settings"
}
},
refund: {
type: "noul",
instructions: "Is the customer asking for a refund?"
}
}
}),
});
const { answers } = await response.json();
console.log(answers.department.choice, answers.refund.noul);Models
Every tag| Name | Accuracy | Latency | |||
|---|---|---|---|---|---|
| Most accurateHighest typed-decisions accuracy of the family. | |||||
| d3latestSame as d3:mini. | 0.737 | 202 ms | |||
| FasterFastest first; accuracy as measured on the bench. | |||||
| d3:edgeQwen3.5-0.8B cut to 12 layers, the fastest: 28 ms for five questions on an RTX 5090. | 0.422 | 28 ms | |||
| d3:liteQwen3.5-0.8B: 0.526 on typed decisions, zero-shot. | 0.526 | 65 ms | |||
| d3:nanoQwen3.5-2B: 0.625 on typed decisions, zero-shot. | 0.625 | 89 ms | |||
| d3:miniQwen3.5-4B, the most accurate size: 0.737 on typed decisions, zero-shot; 202 ms for five questions on an RTX 5090. | 0.737 | 202 ms | |||
| d3:flashQwen3.5-9B: 0.712 on typed decisions, no more accurate than d3:mini at twice the memory. | 0.712 | 317 ms | |||
Accuracy: typed-decisions, argmax against the majority label on 400 states (left out for a tag trained on that data). Latency: the median five-question request on an RTX 5090, Ollaya Bench.
Readme
Decision 3.0 (d3) is the vLLM Semantic Router contributors' family of decision models, released under Apache-2.0 with their code in each model repository. Each is a fine-tuned Qwen3.5 whose full-attention layers read the whole prompt, with a readout over 255 answer codes at the last token. Every question is its own forward pass over the state, its instructions and its options. It never generates text.
Models
| Tag | Base | Params (text) | Decision Index 0.3.2 (upstream) | Ollaya Bench score |
|---|---|---|---|---|
d3:edge | Qwen3.5-0.8B, 12 layers | 0.49B | 20.2 | 42.9 |
d3:lite | Qwen3.5-0.8B | 0.75B | 32.3 | 49.9 |
d3:nano | Qwen3.5-2B | 1.9B | 38.9 | 62.2 |
d3:latest, d3:mini | Qwen3.5-4B | 4.2B | 52.6 | 68.0 |
d3:flash | Qwen3.5-9B | 7.9B | 55.8 | 67.8 |
The Decision Index numbers are the official text board's, from the model cards. The 26B models (d3, d3-turbo) use the same layout but need more memory than a 24 GB GPU in Ollaya's fp32 runtime, so they are not in the library.
Usage
ollaya run d3 --preset triage "My order never arrived and support ignores me. Refund me today or I'm switching to your competitor."Point any TypeSafe client at http://localhost:11435 and set the model to d3.
Accuracy
Measured by Ollaya on an RTX 5090, zero-shot (d3 was not trained on either benchmark):
| Tag | typed-decisions (400 states) | Public benchmark, macro accuracy | Pooled ECE |
|---|---|---|---|
d3:edge | 0.422 | 0.632 | 0.092 |
d3:lite | 0.526 | 0.670 | 0.077 |
d3:nano | 0.625 | 0.756 | 0.062 |
d3:mini | 0.737 | 0.788 | 0.030 |
d3:flash | 0.712 | 0.790 | 0.030 |
d3:flash is no more accurate than d3:mini in Ollaya Bench, at twice the memory, so latest is d3:mini.
Speed
Five triage questions in one request, end to end over HTTP (Ollaya Bench), median:
| Tag | RTX 5090 | CPU (Threadripper 3960X) | GPU memory |
|---|---|---|---|
d3:edge | 28 ms | 1248 ms | 2657 MB |
d3:lite | 65 ms | 2951 ms | 4539 MB |
d3:nano | 89 ms | 3934 ms | 8363 MB |
d3:mini | 202 ms | 9586 ms | 11097 MB |
d3:flash | 317 ms | 14844 ms | 18397 MB |
Every question is a forward pass of its own, so a request costs about one question's time per question. On a CPU, d3:edge is the size to use.
How it works
- One sequence per question. The state, the question and its options, each option behind a code (A, B, …); the readout picks a code at the last token. Questions do not see each other.
- Temperature 1. The probabilities are the softmax of the option logits, as upstream computes them.
- Weights. The files download from Hugging Face, pinned to a commit and verified by sha256, byte for byte the authors'. Ollaya hosts only the ONNX graph.
- Tokenizer. Upstream tokenizes with transformers 5's rebuild of the vocabulary, which differs from the repositories'
tokenizer.jsonon some scripts (Devanagari, Thai). Ollaya uses the rebuild, the same file Bespoke Labs publishes with Nimble v2. - Parity. Ollaya's Rust runtime matches upstream
d3_runtimein fp32 for every tag: identical token sequences, the same rejections and the same decision on all 696 test questions, on CUDA and on the CPU.
Limits
- Context. 8,192 tokens per question; a longer question is rejected, never truncated.
- Score questions. 2 to 10 levels, distinct and nonempty.
- Not ported. Images and videos (upstream accepts them).
Badge
For the model card or a README. Day-0 support for a new model
[](https://ollaya.dev/library/d3)