decision2
6 TagsUpdated Apache-2.0by vLLM Semantic Router
Decision 2.0 by the vLLM Semantic Router contributors: a fully fine-tuned Qwen3 or Qwen3.5 backbone plus an endpoint head that scores every option at its own last token against the question, in one forward pass per question. Five sizes, 0.6B to 9B.
ollaya run decision2 --preset triage "I was charged twice for my subscription this month and want a refund."curl http://localhost:11435/api/decide \
-H "Content-Type: application/json" \
-d '{
"model": "decision2",
"state": "I was charged twice for my subscription this month and want a refund.",
"questions": {
"department": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "Payments, invoices and refunds",
"technical": "Bugs, errors and outages",
"account": "Login, profile and settings"
}
},
"refund": {
"type": "noul",
"instructions": "Is the customer asking for a refund?"
}
}
}'# Already using a TypeSafe SDK? Set TYPESAFE_BASE_URL=http://localhost:11435 instead.
import requests
response = requests.post(
"http://localhost:11435/api/decide",
json={
"model": "decision2",
"state": "I was charged twice for my subscription this month and want a refund.",
"questions": {
"department": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "Payments, invoices and refunds",
"technical": "Bugs, errors and outages",
"account": "Login, profile and settings"
}
},
"refund": {
"type": "noul",
"instructions": "Is the customer asking for a refund?"
}
}
},
)
answers = response.json()["answers"]
print(answers["department"]["choice"], answers["refund"]["noul"])const response = await fetch("http://localhost:11435/api/decide", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
model: "decision2",
state: "I was charged twice for my subscription this month and want a refund.",
questions: {
department: {
type: "choice",
instructions: "Which team should handle this?",
criteria: {
billing: "Payments, invoices and refunds",
technical: "Bugs, errors and outages",
account: "Login, profile and settings"
}
},
refund: {
type: "noul",
instructions: "Is the customer asking for a refund?"
}
}
}),
});
const { answers } = await response.json();
console.log(answers.department.choice, answers.refund.noul);Models
Every tag| Name | Accuracy | Latency | |||
|---|---|---|---|---|---|
| Most accurateHighest typed-decisions accuracy of the family. | |||||
| decision2:luxQwen3.5-9B, the largest: 0.707 on typed decisions. | 0.707 | 364 ms | |||
| FasterFastest first; accuracy as measured on the bench. | |||||
| decision2:kaiQwen3-0.6B-Base, the fastest: 31 ms for five questions on an RTX 5090; 8k-token rows. | 0.476 | 31 ms | |||
| decision2:eosQwen3.5-0.8B: 0.460 on typed decisions. | 0.460 | 69 ms | |||
| decision2:solQwen3.5-2B: 0.554 on typed decisions. | 0.554 | 89 ms | |||
| decision2:noxQwen3.5-4B-Base: 0.680 on typed decisions; 251 ms for five questions on an RTX 5090. | 0.680 | 251 ms | |||
| decision2latestSame as decision2:nox. | 0.680 | 251 ms | |||
Accuracy: typed-decisions, argmax against the majority label on 400 states (left out for a tag trained on that data). Latency: the median five-question request on an RTX 5090, Ollaya Bench.
Readme
Decision 2.0 is the vLLM Semantic Router contributors' second generation of decision models, released under Apache-2.0 with their code in each model repository. Each is a fully fine-tuned text backbone (Qwen3 for Kai, Qwen3.5 for the others) with an endpoint head: every option is scored at its own last token against the question, in one forward pass per question. It never generates text.
Models
| Tag | Base | Params | Context | Ollaya Bench score |
|---|---|---|---|---|
decision2:kai | Qwen3-0.6B-Base | 0.60B | 8,192 | 38.0 |
decision2:eos | Decision 1.0 Eos (Qwen3.5-0.8B) | 0.75B | 16,384 | 47.0 |
decision2:sol | Decision 1.0 Sol (Qwen3.5-2B) | 1.88B | 16,384 | 50.0 |
decision2:nox | Qwen3.5-4B-Base | 4.21B | 16,384 | 60.9 |
decision2:lux | Decision 1.0 Lux (Qwen3.5-9B) | 7.94B | 16,384 | 63.2 |
The 27B Vega uses the same layout but is not in the library.
Usage
ollaya run decision2 --preset triage "My order never arrived and support ignores me. Refund me today or I'm switching to your competitor."Point any TypeSafe client at http://localhost:11435 and set the model to decision2.
Accuracy
Measured by Ollaya on an RTX 5090, zero-shot as far as the model cards say (they do not list the training sets):
| Tag | typed-decisions (400 states) | Public benchmark, macro accuracy | Pooled ECE |
|---|---|---|---|
decision2:kai | 0.476 | 0.608 | 0.046 |
decision2:eos | 0.460 | 0.661 | 0.093 |
decision2:sol | 0.554 | 0.678 | 0.053 |
decision2:nox | 0.680 | 0.746 | 0.050 |
decision2:lux | 0.707 | 0.756 | 0.056 |
Speed
Five triage questions in one request, end to end over HTTP (Ollaya Bench), median:
| Tag | RTX 5090 | CPU (Threadripper 3960X) | GPU memory |
|---|---|---|---|
decision2:kai | 31 ms | 1798 ms | 3701 MB |
decision2:eos | 69 ms | 3226 ms | 5909 MB |
decision2:sol | 89 ms | 4637 ms | 11027 MB |
decision2:nox | 251 ms | 12130 ms | 12387 MB |
decision2:lux | 364 ms | 18670 ms | 20349 MB |
Every question is a forward pass of its own, so a request costs about one question's time per question.
How it works
- One row per question. The state, the question and each option in its own segment; the head reads every option's last token against the row's last token. Questions do not see each other.
- Temperature 1. No package ships a calibration file, so the probabilities are the softmax of the option logits, as upstream computes them. Kai adds the author's fixed offsets to five-level score logits first.
- Weights. The files download from Hugging Face, pinned to a commit and verified by sha256, byte for byte the authors'. Ollaya hosts only the ONNX graph. The 4B and 9B models keep their BF16 weights in BF16 in memory.
- Parity. Ollaya's Rust runtime matches the author's own code in fp32 for every tag: identical token rows, the same rejections and the same decision on every test question, on CUDA and on the CPU.
Limits
- Context. 8,192 tokens per question for Kai, 16,384 for the others, the state included; a longer question is rejected, never truncated.
- Options. 2 to 255 choice options, 2 to 10 score levels.
- Not ported.
share_context(one forward pass for all of a request's questions) and the GGUF builds.
Badge
For the model card or a README. Day-0 support for a new model
[](https://ollaya.dev/library/decision2)