Skip to content

d3

6 TagsUpdated Apache-2.0by vLLM Semantic Router

Decision 3.0 by the vLLM Semantic Router contributors: a fine-tuned Qwen3.5 whose full-attention layers read the whole prompt, with a readout over 255 answer codes. Every question is its own forward pass. Five sizes, 0.6B to 9B.

decisionfine-tuned0.6b0.8b2b4b9b
ollaya run d3 --preset triage "I was charged twice for my subscription this month and want a refund."
NameAccuracyLatency
Most accurateHighest typed-decisions accuracy of the family.
d3latestSame as d3:mini.0.737202 ms
FasterFastest first; accuracy as measured on the bench.
d3:edgeQwen3.5-0.8B cut to 12 layers, the fastest: 28 ms for five questions on an RTX 5090.0.42228 ms
d3:liteQwen3.5-0.8B: 0.526 on typed decisions, zero-shot.0.52665 ms
d3:nanoQwen3.5-2B: 0.625 on typed decisions, zero-shot.0.62589 ms
d3:miniQwen3.5-4B, the most accurate size: 0.737 on typed decisions, zero-shot; 202 ms for five questions on an RTX 5090.0.737202 ms
d3:flashQwen3.5-9B: 0.712 on typed decisions, no more accurate than d3:mini at twice the memory.0.712317 ms

Accuracy: typed-decisions, argmax against the majority label on 400 states (left out for a tag trained on that data). Latency: the median five-question request on an RTX 5090, Ollaya Bench.

Readme

Decision 3.0 (d3) is the vLLM Semantic Router contributors' family of decision models, released under Apache-2.0 with their code in each model repository. Each is a fine-tuned Qwen3.5 whose full-attention layers read the whole prompt, with a readout over 255 answer codes at the last token. Every question is its own forward pass over the state, its instructions and its options. It never generates text.

Models

TagBaseParams (text)Decision Index 0.3.2 (upstream)Ollaya Bench score
d3:edgeQwen3.5-0.8B, 12 layers0.49B20.242.9
d3:liteQwen3.5-0.8B0.75B32.349.9
d3:nanoQwen3.5-2B1.9B38.962.2
d3:latest, d3:miniQwen3.5-4B4.2B52.668.0
d3:flashQwen3.5-9B7.9B55.867.8

The Decision Index numbers are the official text board's, from the model cards. The 26B models (d3, d3-turbo) use the same layout but need more memory than a 24 GB GPU in Ollaya's fp32 runtime, so they are not in the library.

Usage

ollaya run d3 --preset triage "My order never arrived and support ignores me. Refund me today or I'm switching to your competitor."

Point any TypeSafe client at http://localhost:11435 and set the model to d3.

Accuracy

Measured by Ollaya on an RTX 5090, zero-shot (d3 was not trained on either benchmark):

Tagtyped-decisions (400 states)Public benchmark, macro accuracyPooled ECE
d3:edge0.4220.6320.092
d3:lite0.5260.6700.077
d3:nano0.6250.7560.062
d3:mini0.7370.7880.030
d3:flash0.7120.7900.030

d3:flash is no more accurate than d3:mini in Ollaya Bench, at twice the memory, so latest is d3:mini.

Speed

Five triage questions in one request, end to end over HTTP (Ollaya Bench), median:

TagRTX 5090CPU (Threadripper 3960X)GPU memory
d3:edge28 ms1248 ms2657 MB
d3:lite65 ms2951 ms4539 MB
d3:nano89 ms3934 ms8363 MB
d3:mini202 ms9586 ms11097 MB
d3:flash317 ms14844 ms18397 MB

Every question is a forward pass of its own, so a request costs about one question's time per question. On a CPU, d3:edge is the size to use.

How it works

  • One sequence per question. The state, the question and its options, each option behind a code (A, B, …); the readout picks a code at the last token. Questions do not see each other.
  • Temperature 1. The probabilities are the softmax of the option logits, as upstream computes them.
  • Weights. The files download from Hugging Face, pinned to a commit and verified by sha256, byte for byte the authors'. Ollaya hosts only the ONNX graph.
  • Tokenizer. Upstream tokenizes with transformers 5's rebuild of the vocabulary, which differs from the repositories' tokenizer.json on some scripts (Devanagari, Thai). Ollaya uses the rebuild, the same file Bespoke Labs publishes with Nimble v2.
  • Parity. Ollaya's Rust runtime matches upstream d3_runtime in fp32 for every tag: identical token sequences, the same rejections and the same decision on all 696 test questions, on CUDA and on the CPU.

Limits

  • Context. 8,192 tokens per question; a longer question is rejected, never truncated.
  • Score questions. 2 to 10 levels, distinct and nonempty.
  • Not ported. Images and videos (upstream accepts them).

Badge

Run on Ollaya

For the model card or a README. Day-0 support for a new model

[![Run on Ollaya](https://ollaya.dev/badge/run-on-ollaya.svg)](https://ollaya.dev/library/d3)