Skip to content

sura

3 TagsUpdated Apache-2.0by Ollaya

Ollaya's own decoder decision model: Qwen3.5 distilled from pplx-decider:flash in its joint format. Every question of a request shares one sequence and reads the whole request; up to 255 options. English, zero-shot.

decisionzero-shot0.8b2b
ollaya run sura --preset triage "I was charged twice for my subscription this month and want a refund."
NameAccuracyLatency
Most accurateHighest typed-decisions accuracy of the family.
suralatestSame as sura:2b.0.57063 ms
FasterFastest first; accuracy as measured on the bench.
sura:0.8bQwen3.5-0.8B: 0.560 on typed decisions, zero-shot; 60 ms for five questions on an RTX 5090.0.56060 ms
sura:2bQwen3.5-2B: 0.570 on typed decisions, zero-shot; 63 ms for five questions on an RTX 5090.0.57063 ms

Accuracy: typed-decisions, argmax against the majority label on 400 states (left out for a tag trained on that data). Latency: the median five-question request on an RTX 5090, Ollaya Bench.

Readme

Sura is Ollaya's own family of decoder decision models: Qwen3.5 distilled from pplx-decider:flash in its joint format, on the same 340,000 permissively licensed questions as coba's zero-shot tags. It reads a state (a message, a ticket, a JSON object, an agent's next tool call) and typed questions, and returns calibrated probabilities for every question in one forward pass. It never generates text.

Models

TagBaseParamsTrained
sura:latest, sura:2bQwen3.5-2B-Base2Bzero-shot
sura:0.8bQwen3.5-0.8B-Base0.8Bzero-shot

Zero-shot: sura trained without any of the datasets of the benchmarks Ollaya reports, so its scores below are held-out measurements.

Usage

ollaya run sura --ask "Is the customer asking for a refund?" "I was charged twice for order A-104. Refund the duplicate today."

Point any TypeSafe client at http://localhost:11435 and set the model to sura.

Speed

  • RTX 5090, over HTTP (Ollaya Bench): at the median for five triage questions (one forward pass), 63 ms for sura:2b and 60 ms for sura:0.8b; 64 ms and 59 ms per question on Bespoke Labs' public benchmark.
  • CPU (Threadripper 3960X): 1945 ms (2b) and 1599 ms (0.8b) for the same five questions. For a CPU, coba is the faster family.

Accuracy

Measured by Ollaya on an RTX 5090, on the 400 typed-decisions states and Bespoke Labs' public benchmark (3,880 human-labeled questions):

TagTyped decisionsPublic, macro accuracyPublic, pooled ECE
sura:2b0.5700.6100.060
sura:0.8b0.5600.5860.057

On Ollaya Bench, sura:2b is level with kev:0.8b and sura:0.8b with coba:1b. Both are below decider:0.8b and decider:2b, decoders on the same bases trained on more data, and far below their teacher, pplx-decider:flash (0.726 typed, 0.824 public), twice the larger sura's size.

How it works

  • The teacher's format. A request is one sequence: the state, then every question with its options, each followed by a marker where a readout over 255 answer codes picks the answer. The full-attention layers see the whole sequence; the Gated DeltaNet layers stay causal. Sura shares pplx-decider's tokenizer and prompt, so it learned the teacher's whole distribution over the options, not only its answers.
  • Calibrated. The readout logits are divided by a temperature fitted on held-out states (0.92 for 2b, 0.93 for 0.8b).
  • Weights on Hugging Face. ollaya-dev/sura-2b and ollaya-dev/sura-0.8b hold the BF16 weights, the readout and upstream's pplx_decider code, so the authors' loader reads the same checkpoint Ollaya runs.

Badge

Run on Ollaya

For the model card or a README. Day-0 support for a new model

[![Run on Ollaya](https://ollaya.dev/badge/run-on-ollaya.svg)](https://ollaya.dev/library/sura)