decision
2 TagsUpdated Apache-2.0by vLLM Semantic Router
Decision models by the vLLM Semantic Router contributors: a fully fine-tuned Qwen3.5 backbone plus an endpoint head that scores every option at its own last token against the question, in one forward pass per question. 16k-token rows.
ollaya run decision --preset triage "I was charged twice for my subscription this month and want a refund."curl http://localhost:11435/api/decide \
-H "Content-Type: application/json" \
-d '{
"model": "decision",
"state": "I was charged twice for my subscription this month and want a refund.",
"questions": {
"department": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "Payments, invoices and refunds",
"technical": "Bugs, errors and outages",
"account": "Login, profile and settings"
}
},
"refund": {
"type": "noul",
"instructions": "Is the customer asking for a refund?"
}
}
}'# Already using a TypeSafe SDK? Set TYPESAFE_BASE_URL=http://localhost:11435 instead.
import requests
response = requests.post(
"http://localhost:11435/api/decide",
json={
"model": "decision",
"state": "I was charged twice for my subscription this month and want a refund.",
"questions": {
"department": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "Payments, invoices and refunds",
"technical": "Bugs, errors and outages",
"account": "Login, profile and settings"
}
},
"refund": {
"type": "noul",
"instructions": "Is the customer asking for a refund?"
}
}
},
)
answers = response.json()["answers"]
print(answers["department"]["choice"], answers["refund"]["noul"])const response = await fetch("http://localhost:11435/api/decide", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
model: "decision",
state: "I was charged twice for my subscription this month and want a refund.",
questions: {
department: {
type: "choice",
instructions: "Which team should handle this?",
criteria: {
billing: "Payments, invoices and refunds",
technical: "Bugs, errors and outages",
account: "Login, profile and settings"
}
},
refund: {
type: "noul",
instructions: "Is the customer asking for a refund?"
}
}
}),
});
const { answers } = await response.json();
console.log(answers.department.choice, answers.refund.noul);Models
Every tag| Name | Accuracy | Latency | |||
|---|---|---|---|---|---|
| Most accurateHighest typed-decisions accuracy of the family. | |||||
| decisionlatestSame as decision:eos. | 0.512 | 69 ms | |||
| FasterFastest first; accuracy as measured on the bench. | |||||
| decision:eosDecision 1.0 Eos, fully fine-tuned Qwen3.5-0.8B: 17.49 on Decision Index 0.2. | 0.512 | 69 ms | |||
Accuracy: typed-decisions, argmax against the majority label on 400 states (left out for a tag trained on that data). Latency: the median five-question request on an RTX 5090, Ollaya Bench.
Readme
Decision 1.0 is a family of decision models by the vLLM Semantic Router contributors, released under Apache-2.0. Each model is a fully fine-tuned Qwen3.5 text backbone plus a small endpoint head. For each question it lays out the context, the question and one segment per option, then scores every option at its last token against the end of the question, in one forward pass. It never generates text.
Needs Ollaya 0.7.0 or newer, which adds this model family. Update first (
ollaya --version); an older version downloads the weights and then fails to load them.
Models
| Tag | Base | Params | Decision Index 0.2 | ECE |
|---|---|---|---|---|
decision:latest, decision:eos | Qwen3.5-0.8B | 0.75B | 17.49 | 0.083 |
Decision Index 0.2 is an independent comparison of open decision models with Jev. The score is its balanced skill across 43 benchmarks, where Jev scores 51.67, and ECE is its calibration error. For comparison, kev:0.8b scores 13.26 and decider:2b 26.11 there. The larger members, Nox (4B, 31.06) and Lux (9B, 38.98), use the same layout; see the limits below.
Usage
ollaya run decision --preset triage "My order never arrived and support ignores me. Refund me today or I'm switching to your competitor."Point any TypeSafe client at http://localhost:11435 and set the model to decision.
Speed
- RTX 5090, through the HTTP API (Ollaya Bench): a five-question request with a short message takes 69 ms at the median (
kev:0.8b62 ms,decider:0.8b60 ms). - CPU: the same request takes 3506 ms on the Threadripper 3960X (24 cores).
- State length: every question reads the whole context again, so the cost grows with the number of questions times the context length.
How it works
- One row per question. The context comes first, then the question type and the question, then one
<option>segment per option, then a fixed decision prompt. Each segment is tokenized on its own, exactly as upstream does. - Endpoint head. The head compares each option's last token with the row's last token, through a bilinear term and a small MLP, in fp32.
- Calibrated. The logits are divided by the temperature the model ships (1.039 for Eos).
- Weights. Two files download from Hugging Face, pinned to a commit and verified by sha256: the BF16 backbone and the FP32 head, both from the author's repository and read in place. Ollaya hosts only the ONNX graph.
- Parity. Ollaya's Rust runtime matches the author's code run in fp32: identical token rows and option positions, the same decision on every test question, and probabilities within 3.2e-6, on CPU (x86 and Apple silicon) and CUDA.
Limits
- Choice criteria. Give
choicecriteria as an object of option to description, with 2 to 255 options. A plain list is rejected, as upstream does. - Score. 2 to 10 levels, as a list.
- Noul criteria. Only the keys
trueandfalse, spelled exactly. - Context. A question's row, context included, can be up to 16,384 tokens. Nothing is truncated: a longer row rejects the request. Every question reads the whole context again.
- Calibration. The author notes that probabilities can be overconfident.
- Larger models. Nox (4B) and Lux (9B) use the same layout, but Ollaya computes in fp32 and their weights alone then take about 17 GB and 32 GB, so they are not in the library.
Badge
For the model card or a README. Day-0 support for a new model
[](https://ollaya.dev/library/decision)