basal-pico 0.8B and engine 2.3.2: a small model for concrete decisions

Pico’s first day at work

Pico wants to get straight to work, while basal-omni-ultra reminds it about the harder tasks. The film introduces the smallest family member, benchmark comparisons and the engine update. Polish narration; Polish and English subtitles.

A small model, the same interface

Pico reads text: a message, a document, a retrieved passage or an application state. It receives a question and the allowed answers, then returns their probabilities. You can change the classes with each request without training a separate classifier for every list.

The base is Qwen3.5-0.8B-Base. We updated all model weights during training and used 1,428,543 training rows, with basal-omni-1.0-ultra (31B) as the teacher. The final weights average two late checkpoints. Pico is a text model; multimodal tasks remain the role of the omni family.

We retain the POST /v1/systemone API and the choice, noul, score, multi, act and tool schemas. The same application can start with pico and route harder cases to a larger model or a person. The weights and engine use Apache-2.0; after downloading, they can run locally.

Werdykt: comparing 26 small models

Werdykt is our private test of 5,000 decisions. The score is the average across 20 groups: 10 categories in Polish and English. Pico scores 65.98%, compared with 58.66% for the next model, decider-2b. The gap is 7.32 percentage points. Pico also leads this group on the Polish and English subsets separately.

The comparison includes pico and 25 alternatives selected from qev’s comparison and the open Benchmark Heaven leaderboard, in the class up to roughly 2B parameters. It is a set of tested systems, not an exhaustive ranking of every model. Jevos does not state an exact parameter count.

How we compare. We use the same benchmark client, but serve each model through its own runtime. Context-limit refusals and failed calls count as wrong answers. This compares working systems and their limitations, rather than weights running in an identical engine. The JSON also records failed-call counts and model revisions.

How we selected the final pico. We knew both final candidates’ Werdykt scores before selecting the release weights. They scored 65.98% and 64.92%, with overlapping confidence intervals. We therefore do not present Werdykt as an untouched test that played no part in choosing this checkpoint.

Pico’s strengths on Werdykt are routing (91.2%), selecting a relevant passage (88.6%) and real documents (84.8%). Complex rules are much harder: 31.6%. These results help identify where to start and where a larger model is needed.

Results, revisions and measurement conditions (JSON) ↓ · More about Werdykt →

Six public tests: where other models lead

One table is not enough. We also evaluated public task sets; below, we compare pico with qev, krino and jiwo. All values use a 0–100 scale, with higher being better. These are our local evaluation runs, not a claim of a new position on the official Benchmark Heaven leaderboard.

Pico leads on Rules-7 here. Qev scores higher on Benchmark Heaven and JevExtra, krino on JevSpatial and JevBench coherence, and jiwo on JevBench accuracy. Choose a small model based on your actual tasks rather than a single average.

47 ms on an H100. CPU is an option too

Median latency and throughput measure different things. A short-message CPU result is not directly comparable to the benchmark workload. On 1,700 development examples, GGUF chooses the same answer as BF16 in 99.29% of cases on CPU and 99.24% on H100. This measures agreement between versions, not accuracy against labels.

The new 0.8B pico is slower than the earlier 0.6B preview in its fast mode. Quality improves, but the new architecture still needs dedicated optimisation. CUDA-graph engine support for Gated DeltaNet is planned for 2.4; it is not part of 2.3.2.

What basal engine 2.3.2 changes

Qwen3.5 combines attention with Gated DeltaNet layers that maintain recurrent state. Sharing a context as attention keys and values did not preserve that extra state. In 2.3.1, using the fast path could therefore produce an incorrect answer without reporting a problem.

• A correct default mode. Pico starts in eager. Unsupported modes, including fast, fp8 and nvfp4, are rejected before downloading or loading weights, with an explanation of what to use instead.
• GGUF on CPU with multiple questions. For hybrid models, the engine decodes prompts one at a time. This removes invalid probabilities when asking several questions. The trade-off is recomputing the state for each question and option order.
• A clear FP8 message on older GPUs. The RTX 3090 and A100 do not meet this FP8 path’s requirements. The engine stops before loading the model; BF16 remains available through the mode appropriate for its architecture.
• Two facts fixes. An amount written as “10.000,00 zł” is parsed correctly, and the Polish word “roku” after a date is no longer treated as a one-year duration.
• A check on existing models. On 200 held-out decisions, Qwen3 and basal-omni-1.0-mini returned the same probabilities as in 2.3.1. The engine update does not introduce new omni weights.

The full changes, validation and known limitations are documented in the v2.3.2 GitHub release for your own data.

Thank you to the contributors

For this release, special thanks go to the people who contributed fixes and tested the engine on their own hardware:

• Grzegorz Brzezinka (@agentGreg): correct parsing of amounts with a dot as the thousands separator (PR #31) and the fix for “roku” after a date (PR #32).
• Krzysztof Kaminski (@krzysztofkaminski): the first RTX 3090 measurements and issue #33. They led to the early FP8 compatibility check and updates to hardware and serving documentation.

These reports show what happens outside our test environment. A concrete request, model version, hardware configuration and error log can turn a user’s problem into a fix for the whole community.

Accuracy and confidence are different measurements

Higher accuracy does not automatically mean better calibration. In the Werdykt run, performed before the final calibration file, pico has an ECE of 0.143. Several competitors have lower calibration error. Nor do we transfer a “1% error” threshold from one dataset to arbitrary company data.

We checked the final calibration separately on a frozen test of 8,560 decisions. At a 1% target, the model accepts 35.0% of cases automatically, with 0.93% observed error; at a 5% target, it accepts 58.2%, with 4.38% error. These are results on that test, not on Werdykt or a guarantee for a new domain. Use basal-calibrate for your own data.

One message, two API questions

An example from the model card: a customer cannot log in to online banking. One request selects the department and asks whether the customer is unable to use the service. Below is the complete body, with the original Polish text in the state field.

{
  "state": "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej, system pokazuje błąd hasła.",
  "questions": {
    "dept": {
      "type": "choice",
      "instructions": "Do którego działu skierować zgłoszenie?",
      "criteria": {
        "cards": "Reklamacje kart",
        "online": "Wsparcie bankowości elektronicznej",
        "loans": "Kredyty"
      }
    },
    "urgent": {
      "type": "noul",
      "instructions": "Czy klient nie może korzystać z usługi?"
    }
  }
}

In the recorded BF16 measurement on an H100, the model selected online with probability 0.9983; “yes” to the second question had probability 0.762. This is the example result from the model card. The film’s 90% is a separate illustration. Rules for handing a case to a person remain part of the application.

Getting started

Engine 2.3.2 is public. The weight links below activate when each repository becomes publicly accessible. You can run the model once its weights have been published and downloaded.

• basal-2.0-pico · BF16 · weight publication in progress
• basal-2.0-pico · GGUF Q8_0 · weight publication in progress

In an environment with PyTorch configured for your hardware, install release 2.3.2. Once the weights are public, start the server:

uv pip install "basal @ https://github.com/rkinas/basal/archive/refs/tags/v2.3.2.tar.gz"

basal-serve --model Remek/basal-2.0-pico --port 8000

Do not add fast mode for pico: engine 2.3.2 will select eager automatically. For GGUF, use the basal[gguf] extra, --mode gguf and the calibration file supplied with that variant. Start with examples from your own process and compare accuracy, latency and the share of cases that need a person.

Engine 2.3.2 release → · REST, MCP and the inference server → · See applications in Basal Lab →