basal-omni-2.0-mini (preview): two-way reading and 0.811 on Werdykt

What the 2.0 mini preview is

We will release the final basal-omni 2.0 later, together with the whole family: nano, mini, max and ultra. This preview lets you try the 2.0 mini now. The final mini may differ from it.

We fine-tuned it from Google Gemma 4 E4B on Polish and English typed decisions. The weights are Apache-2.0, like the base model. The Hugging Face repository is Remek/basal-omni-2.0-mini-preview.

The preview is served by the public basal engine 2.8.0, with the same API as basal-omni-1.0: the same typed questions and the same endpoint POST /v1/systemone. It is a text model: we trained and evaluated it on text only.

What's new in 2.0

• Two-way reading. The model reads the state in both directions before it answers. This needs basal engine 2.5 or later. For long prompts, see the limitations.
• Higher accuracy on our hidden Polish and English decision benchmark Werdykt and on the held-out tests below.
• More decisions automated at a given error target. Coverage is in the section on confidence.
• Unchanged: a calibrated probability for each allowed answer, in one forward pass. The answer never falls outside the options you give, and the probability says how sure the model is.

The question types remain: choice (one option), noul (yes / no), score (a score), and in the basal engine also multi, act and tool. Options get codes (A–Z, then two-character codes, up to 255 options). The decision is a probability distribution over these codes only, with no text generation.

Results

Werdykt is our hidden benchmark: 5,000 decisions in Polish and English across 10 categories. The macro score gives each of the 20 groups (category × language) the same weight. The other rows are held-out tests.

Held-out tests: the public basal engine 2.8.0 and the released files, one NVIDIA H100, --mode eager, bf16, one option order (the default), the shipped CALIBRATION.json. Werdykt: basal engine 2.5, --mode fast, one option order, one NVIDIA RTX PRO 6000 Blackwell (bf16). The public engine 2.8.0 gives the same probabilities on our parity set (largest difference 5 × 10⁻⁶).

English panel covers ten public tasks. In the panel score each task counts the same, whatever its number of items.

The weakest Werdykt categories are rules (0.382), long documents (0.694) and contracts (0.726). Rules need exact thresholds and dates; compute those in code. The strongest categories are retrieval (0.982), real documents (0.978) and routing (0.966).

More about Werdykt →

Comparison on Werdykt

Five other decision models took the same test: the strongest 4B-class models we checked, and the Polish Bielik-One in its 4B and 11B versions. This is a selection of models, not a full ranking.

Werdykt v1: the same 5,000 hidden items (PL + EN) and the same client (16 concurrent requests, option keys hidden), through each model's typed decision API (/v1/systemone). Each model runs on the server its maker recommends; the protocol and the items are the same. KnowLine-4B-Gen5.5 ran in FP8, the other models in bf16. The set is hidden and has no public items, so only aggregates are shown. Macro score: each of the 20 groups (category × language) counts the same. The lead is a paired bootstrap over the same 5,000 items. Read 8 October – 13 November 2026: the preview on basal engine 2.5 (--mode fast, one option order), H2O-Lightning-4B v1.1 on an H100 (the same value as on the public board), the others on an RTX PRO 6000. Revisions: torchcast-ai/torchcast-decision-4b @492bfb19, PelaAI/KnowLine-4B-Gen5.5 @779944d2, speakleash/Bielik-One-11B-v3.1-r1-Adapter @aab0a71f, speakleash/Bielik-One-4B-r1 @a5a17dad.

The preview is ahead of each of these models, and in every pair the 95% interval of the lead excludes zero. The closest is torchcast-decision-4b: 1.22 points apart. Over Bielik-One 11B and 4B the lead is 10.82 and 11.46 points.

Accuracy by document length

This is accuracy on Werdykt (5,000 items, Polish and English) by the length of the state, in characters. A token is about 2 characters, and 1,000 characters is about half a page. Overall: 81.1%. The set is hidden, so only these totals are shown.

Short states are mostly rules items and items where the right answer is to abstain. These are harder tasks, so read the rows as accuracy per bucket, not as the effect of length alone. The long-document family keeps the task and varies only the length:

Same run as the Werdykt score above. The Wilson intervals are about the number of items, not about run-to-run noise.

In the long-document family, accuracy falls as length grows, most of all above 30k characters. If your documents are that long, test the model on your own examples.

Confidence and automation

In the model repository, the file CALIBRATION.json holds a temperature for each question type and certified confidence thresholds for a target of 1% or 5% error among accepted decisions. We chose the thresholds on calibration data before testing, on the engine that serves the model (--mode eager, bf16, one option order). Accept decisions above the threshold automatically and route the rest to a person.

Held-out tests: PL decisions and the EN corpus test, 8,560 decisions in total, with the per-type thresholds of the shipped CALIBRATION.json, as the server applies them.

At the 1% target, score questions have no certified threshold and go to a person. Per type: choice 78.2% at 0.74% error, yes / no 43.8% at 0.15% error. At the 5% target: choice 94.7%, yes / no 88.0%, score 47.6%.

For --orders 2, which asks in both option orders, the model has its own file CALIBRATION.o2.json.

Speed

The first three rows ask one question per request. fast is the default on Hopper GPUs with FlashAttention 3; eager was measured with --media on. GPU memory while serving: 20.6 GB (fast), 31.7 GB (eager, --media on). --orders 2 costs about twice as much per question.

Speed next to other models

We also measured speed in the same Werdykt runs. The chart shows the five models from the comparison table that ran on the same GPU.

Limitations

• Preview. This is an early release of the 2.0 mini; the final 2.0 family (nano, mini, max, ultra) comes later. Validate it on your own documents before relying on it.
• Long prompts. The state is read two-way up to 4,096 prompt tokens (8,192 tokens per request). Longer prompts are read one way, as in basal-omni-1.0, and the answer does not say which reading was used. Accuracy on the longest documents is lower (see accuracy by document length).
• Serving features. For a two-way model the engine turns off the state cache, the prefix cache, reading one state once for many questions (SOAM) and the long lane. Ask several questions about one state in one request.
• Text only. The model files include Gemma 4's image and audio encoders, unchanged, but this preview was trained and evaluated on text only. Use it for text; media answers are not evaluated here.
• Exact arithmetic and thresholds are a weak spot of one-pass decisions (Werdykt rules: 0.382). Compute them in code.
• Option order. By default the model sees one option order. --orders 2 reduces sensitivity to option order at about twice the cost.
• Engine 2.8.0: bf16 only; there are no FP8 or GGUF builds of this preview.
• Human review. Decisions with serious consequences for people should be reviewed by a person.

Getting started

The weights link activates once the repository is public on Hugging Face.

• basal-omni-2.0-mini-preview · BF16 ↗

Install engine 2.8.0 into a fresh environment made with uv:

uv venv --python 3.12 --allow-existing ~/basal-env && source ~/basal-env/bin/activate
uv pip install torch==2.11.0 torchvision==0.26.0 --index-url https://download.pytorch.org/whl/cu128
uv pip install "basal[media] @ https://github.com/rkinas/basal/archive/refs/tags/v2.8.0.tar.gz"

Start the server. When the model is ready, it prints basal: model ... ready.

basal-serve --model Remek/basal-omni-2.0-mini-preview --name basal-omni-2.0-mini-preview --port 8000

On an NVIDIA GPU the engine picks the mode itself: fast (CUDA graphs) on Hopper with FlashAttention 3 and on Blackwell workstation GPUs, and eager (PyTorch, bf16) on other GPUs. The weights take about 16 GB. A two-way model runs only in --mode eager and --mode fast. On Apple Silicon pass --mode eager, because the engine's default there does not serve two-way models. One option order is asked by default; --orders 2 automatically uses the file CALIBRATION.o2.json.

Ask questions. In one request we pick the department for a customer ticket and ask whether the customer cannot use the service:

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej, system pokazuje błąd hasła.",
  "questions": {
    "dept":   {"type": "choice", "instructions": "Do którego działu skierować zgłoszenie?",
               "criteria": {"cards": "Reklamacje kart", "online": "Wsparcie bankowości elektronicznej", "loans": "Kredyty"}},
    "urgent": {"type": "noul", "instructions": "Czy klient nie może korzystać z usługi?"}}}'

The full API (multi, act, tool, option keys, max_error) is described in the engine documentation. License: Apache-2.0; fine-tuned from google/gemma-4-E4B-it (Apache-2.0).

Engine 2.8.0 release → · REST, MCP and the inference server → · More about Werdykt →