d3
6 TagsUpdated Apache-2.0by vLLM Semantic Router
Decision 3.0 by the vLLM Semantic Router contributors: a fine-tuned Qwen3.5 whose full-attention layers read the whole prompt, with a readout over 255 answer codes. Every question is its own forward pass. Five sizes, 0.6B to 9B.
decisionfine-tuned0.6b0.8b2b4b9b
Tags
| Name |
|---|
| d3latestSame as d3:mini.9.1 GB · 8192 ctx · English |
| d3:edgeQwen3.5-0.8B cut to 12 layers, the fastest: 28 ms for five questions on an RTX 5090.1.2 GB · 8192 ctx · English |
| d3:flashQwen3.5-9B: 0.712 on typed decisions, no more accurate than d3:mini at twice the memory.16.8 GB · 8192 ctx · English |
| d3:liteQwen3.5-0.8B: 0.526 on typed decisions, zero-shot.1.7 GB · 8192 ctx · English |
| d3:miniQwen3.5-4B, the most accurate size: 0.737 on typed decisions, zero-shot; 202 ms for five questions on an RTX 5090.9.1 GB · 8192 ctx · English |
| d3:nanoQwen3.5-2B: 0.625 on typed decisions, zero-shot.4.5 GB · 8192 ctx · English |
Tags without a suffix load fp16 on a CUDA GPU and fp32 on CPU; add -fp16 or -fp32 to pin one precision.