BASAL-SERVE / ENGINE 2.0

Your models.
One decision interface.

The model provides probabilities. The inference server turns them into an answer your application, workflow or AI agent can use.

basal-serve loads the weights, handles requests and returns typed, calibrated decisions. You choose the model and hardware. The way you ask questions stays the same.

Connect an agent via MCP Explore the API ↗

Open source · Apache 2.0 · your infrastructure

FROM QUESTION TO DECISION

ApplicationWorkflowAI agent

YOUR SERVERbasal-serve

local

Requests · queue · calibration

TEXTbasal-2.0

TEXT / IMAGE / AUDIObasal-omni

One selected model per server process

AN ANSWER FOR YOUR CODEDecision + probabilities

choicenoulscoremultiacttool

01 / BETWEEN THE MODEL AND YOUR APPLICATION

From weights to a running service.

The weights hold what the model has learned. The engine determines how your program uses that knowledge.

Accepts your rules

Send the state, questions and allowed answers. The server checks the request structure and prepares the model input. You define the classifier with every call.

Shares computation

SOAM (State Once, Ask Many) lets several questions share the same state. The engine can also batch concurrent requests to make better use of the GPU.

Returns the API response

It reads option probabilities, applies calibration and builds a JSON response. For cost-aware decisions, it computes the expected cost of actions and can return a deferral to a person.

vLLM, SGLang, MLX, Ollama and llama.cpp can run model computation. basal-serve provides the typed decision layer in front of them. Running the model in ordinary text-generation mode does not provide the same API.

02 / ASK FOR A RESULT, NOT A PARAGRAPH

Six ways to make a decision.

Combine different question types in one request. basal-2.0 supports up to 255 options per question; Ollama mode has a limit of 20.

choice

One of the options

Which team should handle the ticket? You receive the selected option and a probability distribution.

noul

Yes or no

Does the description include an order number? The result is the probability of “yes”.

score

Scale-based assessment

How urgent is the issue? Define ordered levels; the server returns their distribution and the expected level.

multi

Multiple matching labels

A complaint, delivery issue and refund request can all appear in one message. Each label has an independent probability; their sum does not have to equal 100%.

actexperimental

An action with costs considered

Provide the costs of actions for each possible outcome. The server selects the lowest expected cost, including when consulting a person is the best action.

toolexperimental

The agent’s next tool

Supply tool definitions in MCP, OpenAI or Anthropic format. basal selects a tool, a direct answer, a question for the user or no suitable option.

Facts before the decision. "facts": "auto" adds computed dates and amounts to a Polish state. Code does the arithmetic; the model evaluates the facts against the process rules you describe.

03 / CHOOSE YOUR ENVIRONMENT

Server, Mac or CPU.

The same entry point for decisions, with different ways to run the computation. Choose the mode to match your hardware, weights and application traffic.

Environment	Mode	What to know
NVIDIA GPU · text	`fast` `fast-nocompile`	The main PyTorch engine. Compilation and CUDA graphs in fast; quicker startup without compilation in fast-nocompile.
NVIDIA GPU · FP8	`fp8`	FP8 ports on a compatible GPU. HP ZGX Nano (NVIDIA DGX Spark) requires a CUDA stack suited to Blackwell.
Apple Silicon	`mlx` `mps`	MLX with 8-bit ports, or PyTorch on the Mac’s GPU. Use calibration appropriate to the selected engine.
CPU and local GGUF files	`eager --device cpu` `gguf` / `ollama` / `llamacpp`	PyTorch on CPU or GGUF files. For Ollama and llama.cpp, basal-serve connects to a separately running server.
Alternative CUDA servers	`vllm` `sglang`	The same basal API over another computation backend. The default context is 4096 tokens; change it with --max-len.
basal-omni · images and audio	`eager --media on`	NVIDIA GPU, bf16 weights. This release has no fast mode or MLX, FP8 or GGUF ports. Send media through the HTTP API.

NVFP4 mode on Blackwell is experimental and may reduce accuracy. The project documentation covers hardware requirements and dependency installation.

Run text-only basal-2.0 on a GPU

These examples assume engine 2.0, a compatible PyTorch installation and access to the specified weights. For the 4.5B model, disable the template cache with --template-prefix off.

basal-serve \
  --model Remek/basal-2.0-4.5B \
  --mode fast --template-prefix off \
  --host 127.0.0.1 --port 8000

Run on a Mac with MLX

Requires the basal[mlx] dependencies and an MLX port. The command serves the same HTTP API on a local port.

basal-serve \
  --model Remek/basal-2.0-4.5B-MLX-8bit \
  --mode mlx --host 127.0.0.1 --port 8000

Run basal-omni with images and audio

Requires basal[media] dependencies and an NVIDIA GPU. The media field in the HTTP API carries images and audio. Split video into frames and an audio track first.

basal-serve \
  --model Remek/basal-omni-1.0-mini \
  --mode eager --media on \
  --host 127.0.0.1 --port 8000

04 / FOR APPLICATIONS AND WORKFLOWS

One state. Several questions.

Send JSON to POST /v1/systemone. In n8n, use an HTTP Request node; in an application, a standard HTTP client or the basal Python client.

POST /v1/systemone

{
  "state": "The parcel arrived damaged. Please replace it.",
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this issue?",
      "criteria": {
        "claims": "Complaints and product replacements",
        "sales": "Questions before purchase",
        "billing": "Invoices and payments"
      }
    },
    "reply_needed": {
      "type": "noul",
      "instructions": "Does the customer expect a reply?"
    }
  }
}

A response your code can use

The answers field contains each question’s result under its name. choice returns an option key; noul returns the probability of “yes”. The response also includes model and usage, with the number of questions, tokens and computation time.

state: The facts and rules needed for the decision.
questions: Separate questions about a shared context.
answers: Typed results and probabilities.

This is a request definition, not a measured result. This example does not call the model.

GET /health Service healthGET /v1/models Information about the loaded modelPOST /v1/basal Alias for the decision API

05 / MODEL CONTEXT PROTOCOL · EXPERIMENTAL

The agent asks.
basal helps choose the next step.

MCP is a standard for connecting AI applications to tools. Engine 2.0 exposes its own MCP server: an agent can ask basal for a decision through the same kind of connection it uses for other tools.

01 / AGENTCollects the state and options
02 / BASAL VIA MCPReturns a decision and confidence
03 / AGENT OR APPLICATIONExecutes an allowed action

decide

One question

State, question and options. Supports choice, noul, score and multi. Returns the question’s answer and the model name.

decide_many

Several questions about one state

The full questions map, as in the HTTP API. Also supports act with costs and max_error, tool, and facts. Returns answers, model and usage.

Both tools only return decisions. They do not execute commands, call other tools or generate their arguments. Execution and permission checks belong to the application using MCP.

basal-serve / MCP

basal-serve \
  --model Remek/basal-2.0-4.5B \
  --mode fast --template-prefix off \
  --host 127.0.0.1 --port 8000 --mcp on

MCP requires the optional basal[mcp] dependencies. It is off by default. When enabled, /mcp uses Streamable HTTP on the same port and shares the queue with /v1/systemone.

Connect an MCP client: HTTP or stdio

In a client supporting Streamable HTTP, add http://127.0.0.1:8000/mcp. Example for Claude Code:

claude mcp add --transport http basal http://127.0.0.1:8000/mcp

If the client starts tools as local processes, use the basal-mcp bridge. Replace the executable path with your own absolute path:

{
  "mcpServers": {
    "basal": {
      "command": "/path/to/basal-env/bin/basal-mcp",
      "args": ["--url", "http://127.0.0.1:8000"]
    }
  }
}

The bridge forwards requests to the running server; it does not load a second copy of the model.

Example: choose a tool and check for missing information

Arguments for a decide_many tool call. The tool definitions are options to choose from; basal does not execute any of them here.

MCP / decide_many

{
  "state": "The customer asks where order ZAM-42 is. Its status has not been checked yet.",
  "questions": {
    "next_step": {
      "type": "tool",
      "instructions": "Which tool is needed now?",
      "tools": [
        {
          "name": "get_order_status",
          "description": "Reads the current status of an order.",
          "inputSchema": {
            "type": "object",
            "properties": {
              "order_id": {
                "type": "string"
              }
            },
            "required": [
              "order_id"
            ]
          }
        },
        {
          "name": "list_products",
          "description": "Lists the available products.",
          "inputSchema": {
            "type": "object",
            "properties": {}
          }
        }
      ]
    },
    "missing_order_id": {
      "type": "noul",
      "instructions": "Is the order number missing?"
    }
  }
}

The tool type returns a selected tool name or the answer_directly, ask_user or none option. The agent supplies arguments and handles execution according to its permissions.

The model on your own device. You can run basal on HP ZGX Nano and connect clients through an SSH tunnel or your own gateway. The server has no built-in authentication; control network access outside it. basal computation stays local, but an agent using a cloud LLM may separately send its context to that provider.

The current MCP interface. The tools accept a text or JSON state. They have no media parameter: send images and audio for basal-omni through the HTTP API. MCP and the tool type are experimental; check decision accuracy on your own tasks.

06 / CONTROL OVER EXECUTION

Settings for your process.

The main server options. Run basal-serve --help to see all flags and their values.

Option	Purpose
`--model` / `--mode`	A repository or local weights directory, and the computation backend. Each server process loads the selected model.
`--calibration`	Your own calibration file. By default, the server uses an engine-specific file when supplied, or the base CALIBRATION.json.
`--soam on`	A shared state for multiple questions and option orders. On by default; the effects depend on the backend and request.
`--max-batch` `--batch-window-ms`	Batch concurrent requests. Defaults: a maximum of 64 and a 3 ms window under load; with no queue, computation starts immediately.
`--scheduler`	Schedule long and short requests in the CUDA engine. Turning it off restores arrival-order processing.
`--deterministic`	Reproducible answers on the same machine, without batching different clients’ requests. Applies to in-process engines and may reduce throughput.
`--context-overflow`	truncate cuts the middle of an oversized state and reports it in usage.truncated; refuse rejects the request instead.
`--template-prefix off`	Recommended for basal-2.0 4.5B: the template cache did not pass this model’s check. Mini and max can use auto.
`--log-decisions`	Log probabilities to a JSONL file for process analysis and later use of feedback.
`--mcp on` `--media on`	An optional agent interface, or image and audio support in the basal-omni HTTP API. Both are off by default and require the appropriate dependencies.

A threshold is a process rule

max_error in act and tool questions uses thresholds from the calibration file. A missing threshold defers the decision to a person. A 5% target does not guarantee that error on new data — measure coverage and accuracy in your own process.

Calibration on your data

basal-calibrate fits temperatures and thresholds on labelled examples using the selected engine. This is a separate step; the server does not automatically learn from traffic.

Experiments have specific requirements

--think auto is only for basal-2.0 4.5B, requires vLLM and is off by default. It reduced the Werdykt score. evidence requires a separate head that basal-2.0 models do not ship.

FROM INTERFACE TO DEPLOYMENT

Connect a decision to your process.

Code and documentation Explore use cases ↗

Your models.One decision interface.