winnow

Winnow is a pair of open decision models by EldanRing, fine-tuned from Google DeepMind's Gemma 4 and released under Apache-2.0. The author publishes them as GGUF files, and Ollaya runs those files as they are, on llama.cpp. Winnow puts the state and one question into its own prompt, labels the options A, B, C, and reads the probability of each label as the next token. It never generates text.

Needs Ollaya 0.7.0 or newer, which runs GGUF models through llama.cpp. Update first (ollaya --version); an older version downloads the weights and then fails to load them.

Models

The accuracies are the author's, measured with the author's server on the same Q8_0 files (model cards of Winnow-12B and Winnow-E4B). For comparison, the author reports 85.7 % and 87.0 % for Jev 1.13 on the same two sets. On typed-decisions (all 400 states, argmax against the majority label), measured by Ollaya, winnow:12b scores 0.702 and winnow:e4b 0.722. With its speed, that makes winnow:e4b Ollaya's recommended model.

Usage

ollaya run winnow:e4b --preset triage "My order never arrived and support ignores me. Refund me today or I'm switching to your competitor."
ollaya run winnow --preset triage "My order never arrived and support ignores me. Refund me today or I'm switching to your competitor."

Point any TypeSafe client at http://localhost:11435 and set the model to winnow or winnow:e4b.

Speed

• RTX 4090, through the HTTP API: the triage preset (five questions) on a short customer message takes 89 ms on e4b and 131 ms on 12b at the median of 15 warm requests. On the parity fixtures, measured in the runner, the medians are 96 ms and 154 ms.
• CPU: e4b takes about 5.1 s for the same request on a 24-core x86 CPU. Use a GPU.

How it works

• Prompt. Ollaya builds Winnow's prompt exactly as the author's server does (native/protocol.h in winnow-inference): a system turn, the state as JSON, then the question and its options, each option under a letter label. Text you send can never become one of Gemma's control tokens.
• One question at a time. The state is evaluated once per request, and each question after it. The split is fixed, so the same request always returns the same probabilities.
• Labels. Up to 64 options per question: A to Z, then two-letter labels, each a single token.
• Calibration. winnow:12b uses temperature 1, the author's default: the author fitted no calibration for it. winnow:e4b uses 1.2574, the temperature the author fitted for its Q8_0 file on 778 held-out questions.
• Engine. llama.cpp v0.5.0, ggml-org's own release build, runs inside Ollaya's runner process. It uses an NVIDIA GPU (CUDA) or an Apple silicon GPU (Metal) when the model fits, and the CPU otherwise.
• Parity. Ollaya's runner matches stock llama.cpp (llama-server of the same build, on the same file): the same decision on all 505 test questions, probabilities within 3.0e-6, on CUDA (RTX 4090, both models) and an x86-64 CPU (e4b). On Windows the CPU check covered 15 questions. Against the author's own server, e4b agrees on all 503 decisions. Not checked yet: Metal and the Apple CPU, Linux on ARM, and the full 12b text suite on the CPU.

Limits

• Size. These are large language models. winnow:12b needs about 14 GB of GPU memory and winnow:e4b about 9 GB at the 8,192-token context; on the CPU they are much slower than the encoder models. winnow:12b-vision needs about 0.4 GB more: on an RTX 5090, CUDA context included, it peaked at 16,263 MiB against 15,877 MiB for winnow:12b, so a 16 GB GPU is marginal for it.
• Context. State, question and options share 8,192 tokens. A longer state is cut to 6,144 tokens.
• Options. 2 to 64 options per question and up to 256 questions per request. Probabilities are conditional on the options offered.
• Images. The vision tags read images: up to 16 PNGs per request, sharing the 8,192-token context with the state. /v1/* and the other tags are text only.

Image decisions

winnow:e4b-vision is winnow:e4b with the author's matching vision projector (990 MB): it answers questions about PNG images as well as the state. It needs Ollaya 0.10.0 or newer.

winnow:12b-vision is winnow:12b with its own matching projector (175 MB), read the same way. It also needs Ollaya 0.10.0 or newer, and more GPU memory than winnow:12b (see Limits).

ollaya run winnow:e4b-vision --image shelf.png --questions '{"blocked":{"type":"noul","instructions":"Is the aisle blocked?"}}' "A photo from the warehouse camera."

• How. Winnow's own image prompt, with the images in order before the state, tokenized and evaluated as stock llama.cpp does, and the same option-label readout as the text tags. The images and the state are evaluated once per request.
• Parity. On Linux, the image path matches stock llama-server of the pinned build on 21 requests and 65 questions, with option logits within 1.2e-5 (the gate is 1e-3): winnow:e4b-vision on an RTX 4070, an RTX 5090 and the CPU, winnow:12b-vision on an RTX 5090 and the CPU.
• Not measured yet. Image calibration (each model's text temperature is kept), visual accuracy, Metal and Windows image parity.

Weights and license

Apache-2.0. Winnow is developed by EldanRing (Winnow-12B, Winnow-E4B, inference code at github.com/EldanRing/winnow-inference), from Gemma 4 by Google DeepMind, also Apache-2.0. Ollaya downloads the GGUF from the author's repository, pinned to a commit and checked against its sha256; it never re-hosts it. llama.cpp is MIT.