basal-1.5 — three models, more possibilities

What's new since basal-1.0

• Improvements where 1.0 was weaker: long, real documents, RAG decisions, “cannot determine” answers, harder English decisions (answer evaluation, trade-offs, traps, policies, ambiguous requests), agents' next steps and scale ratings. The release also includes a reinforcement learning (RL) stage.
• Results: on a hidden test of 3,000 new Polish decisions, created after freezing the weights and evaluated once, basal-1.5 reaches 93.1% vs 87.4% for basal-1.0 (+5.7 pp, 95% interval [+4.5; +6.8]); on the basal-1.0 decision test, 92.3% vs 88.6%. On a panel of 10 public English tasks, basal-1.5 beats basal-1.0 (68.8% vs 67.1%), and on the public English decision benchmark reaches 76.2% (basal-1.0: 74.0%).
• Multiple questions, one pass with SOAM. SOAM stands for state once, ask many — one context, many questions. The engine processes the context once and shares those computations across questions. Each question is evaluated independently. In the H100 engine comparison, 5 questions take 37.8 ms instead of 111.7 ms, about 3× faster. In fp32, answers are identical to separate requests.
• New types and features — basal extensions. We extend System One with our own schemas and capabilities — new “programs” in the brain of our system: multi (which labels apply), act (the action with the lowest expected cost under your rules, or handoff to a person), "evidence": true (a text excerpt supporting the answer) and "facts": "auto" (computed dates and amounts for Polish documents).
• New ways to run: SGLang and vLLM, Apple Silicon (MLX 8-bit and 4-bit), and Ollama and llama.cpp (GGUF).
• Werdykt — our own hidden benchmark for testing decision models in Polish and English.

Three sizes, the same interface

basal-1.5 (4.5 billion parameters) is the main model, balancing accuracy and speed. basal-1.5-max (11 billion) is the family's most accurate model, for long documents, answer evaluation and difficult English decisions. basal-1.5-mini (1.5 billion) is distilled from max and is the fastest, for high volumes, laptops and small GPUs. All three share the same API, prompt format and calibration procedure.

Each model is available in BF16, MLX 8-bit / 4-bit and GGUF, and in FP8 for vLLM on NVIDIA Hopper, Ada and Blackwell GPUs. basal-1.5 (4.5B) and basal-1.5-max also have FP4 in NVFP4 format, intended for Blackwell GPUs.

Weights and engine are released under Apache 2.0; model provenance is documented in LICENSE and NOTICE.

Werdykt: one test, one protocol

Werdykt covers 10 decision categories in Polish and English: rules and deadlines, real documents, long documents (including injected instructions), RAG, contracts, routing, scale ratings, answer evaluation, an agent's next step and “cannot determine”. Each model receives the same prompt and one call per question; unparsed answers count as errors. Open models run on one H100; API models through OpenRouter.

The set is closed: we publish the protocol, results and a few examples per category, but not the questions themselves — to keep them out of any model's training data, including ours. No Werdykt question, document or excerpt appears in basal's training data.

Selected rows; leading API models also rank higher in the full table (0.983–0.996). Open-model costs use one H100 at USD 3.95/h under full load; API costs use provider pricing. ¹ Protocol deviation (explained in the full table). Full results, categories and sample questions: Werdykt. Models or families involved in creating or checking questions are marked †; their scores are not independent.

Leading independent API models score between 98.3% and 99.6%. Among decision systems, Jev, Cygnet and Winnow rank above basal. Werdykt shows the trade-off: basal can run locally, selecting quickly from provided options, with lower accuracy than the largest models.

Compared with basal-1.0 4.5B, the new main model gains 6.1 percentage points and max gains 11.2. Mini gains 8.4 points over its 1.5B predecessor with similar decision latency. The fastest model is not necessarily the cheapest: here, 4.5B on SGLang cost less per 1,000 decisions than mini because of throughput at full load.

Polish, category by category

In the “basal-1.5 in Polish” report, the main model scores 72.4%, max scores 77.7%, and mini scores 61.9% on the Polish part of Werdykt. The full ranking and breakdown into 10 categories let you inspect the relevant task. With roughly 250 questions per category, small differences should not be treated as decisive advantages.

Polish items only → · Polish, category by category →

Milliseconds, even with multiple questions

In the HTTP load test on one H100 ( fast mode, bf16), one question to basal-1.5 takes 13.7 ms (median; p95 23.8 ms), and the server handles 64.4 decisions per second with 32 concurrent clients. The new engine speeds up multiple questions about the same state; SOAM does not benefit a single question.

The same requests and server, measured without HTTP (not comparable to the HTTP test above); Polish decision states with a median of 336 tokens. In fp32, answers are identical to separate requests. How it works.

New features basal extensions

Our extension to System One. Alongside the base types choice, noul and score , basal-1.5 adds the schemas multi and act and options evidence and facts. They define more ways to make decisions and inspect their basis within a request.

multi

Multiple labels. Which categories apply to a ticket? An independent probability for each label and a selected set.

act

Cost-based decision (experimental). Provide the cost of a wrong action and human review; the engine chooses the least costly action or hands the case to a person.

evidence

Textual evidence (experimental). An excerpt from the state supporting the answer, with exact character offsets for highlighting and audit logs.

facts

Computed facts. Weekdays and holidays, date differences, net/gross amounts and sums added to the Polish state: the model reads the calculations instead of doing them.

{
  "state": "The shipment arrived with a cracked screen. I would like a refund.",
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this case?",
      "criteria": {"support": "Technical support", "returns": "Returns and complaints"}
    },
    "tags": {
      "type": "multi",
      "instructions": "Which topics apply to this ticket?",
      "criteria": {"damage": "Product damage", "refund": "Refund", "invoice": "Invoice"}
    },
    "evidence": {
      "type": "noul",
      "instructions": "Does the customer describe product damage?",
      "evidence": true
    }
  }
}

Send a POST request to /v1/systemone. This is an example request definition. multi returns independent label probabilities, which need not sum to 1. Textual evidence requires the PyTorch engine and is experimental.

Five sample PL-Workflows (complaints, invoices, HR, tenders, SOC alerts) illustrate the division of labour: basal decides, code calculates dates and amounts, and low confidence routes cases to a person. Feature status and descriptions: feature documentation.

How to run it

uv venv --python 3.12 ~/basal-env && source ~/basal-env/bin/activate
uv pip install torch==2.11.0 --index-url https://download.pytorch.org/whl/cu128
uv pip install "basal[fp8] @ https://github.com/rkinas/basal/archive/refs/tags/v1.5.0.tar.gz"
basal-serve --model Remek/basal-1.5-4.5B --mode fast --port 8000

uv pip install "basal[mlx] @ https://github.com/rkinas/basal/archive/refs/tags/v1.5.0.tar.gz"
basal-serve --model Remek/basal-1.5-4.5B-MLX-8bit --mode mlx --port 8000

On an M4 Pro Mac, 4.5B in MLX reached a median of 587 ms at 8-bit and 581 ms at 4-bit. These are local-hardware measurements using a different test sample from Werdykt on H100. Agreement with the base variant was 99.2% and 97.0% respectively — quantisation also needs quality evaluation.

Ollama (0.12.11 or newer): download the -GGUF repository, register the model using the included Modelfile and run basal-serve --mode ollama. The basal server sits in front of Ollama and preserves both option orders, calibration and the types multi and act. Details and other engines: from idea to first decision.

Things to keep in mind

• Leave calculations to code. Exact thresholds and arithmetic are weak points of single-pass decisions; use code or "facts": "auto".
• Provide facts and current rules. Small models have limited knowledge, and regulations change.
• Experimental features need a human in the loop. Textual evidence and cost-based decisions may change in the next version; validate their quality on your own data.
• Choose confidence thresholds on your own data — especially for MLX and GGUF ports, which inherit bf16 model calibration.
• Mini did not meet the 5% error target. Observed error on the held-out test was 5.3%. For this model, use the 1% threshold if you need a validated risk level, and check it on your own data.
• Textual evidence needs checking. The returned excerpt is a verbatim quote, but it does not guarantee a sound justification. For 4.5B, token-F1 against reference quotes was 0.53 on 200 documents; median latency rose from 40 to 85 ms.
• Keep oversight of consequential decisions. Decisions with serious consequences for people require human oversight.

Full list of limitations: README.

Resources

Find more information in the project documentation. Four recordings — search, RAG checking, inbox routing and Taboo — are described in Basal Lab. Those measurements use basal-1.0; the new release results are on the separate benchmark page.