Run open models.
Keep your hardware.

Bigger than your RAM

A mixture-of-experts token touches a small fraction of the weights. colibri keeps that fraction resident and reads the rest as the router asks for it, so the model does not have to fit.

GLM-5.2 int4, decode speed reported by the people who ran it. The hardware changes where the experts live, not what the model answers.

A GPU is optional on every engine. With Vulkan, any GPU keeps the hottest experts and runs whole layers: on the integrated Radeon 780M of an 8-core desktop, Qwen3.6 went from 6.0 to 9.9 tok/s. A small model can lose on an integrated GPU, so the setup turns it on only where it was measured faster (coli setup --backend vulkan or --backend cpu decides for you); no discrete GPU has run the new expert tier yet. With CUDA, NVIDIA cards keep the hottest experts in VRAM. The release archives carry these backends ready to run: Vulkan on Linux and Windows, Metal on macOS, and a CUDA package for NVIDIA cards on Windows. Vulkan notes

Frontier open models

Thirteen engines run today: ten for language models, one for pictures and two for decisions, one hand-written C file each. Not a wrapper around someone else's runtime.

Total parameters. Only the active ones are computed per token, and only the routed experts move. The engines, by the names the docs use: GLM-5.3-Flash, GLM-5.2/5.3, Inkling, Kimi K3, OLMoE, Qwen3.6-35B-A3B (which also runs Qwen3-Coder-30B-A3B and the dense Qwen3.8-27B), Qwen3.8-Flash-Next, DeepSeek V4 Flash, DeepSeek V4.1 Flash, MiMo-V2.6 Flash (and Pro, on the same engine). And for images, Qwen-Image-2.1: text to picture, drawn inside the terminal. And for decisions, Laya and GLiNER2.5-Decide, with Clef on the Qwen3.6 engine: typed questions in, a probability per option out. Every language engine serves up to 16 conversations at once, their next tokens decoded together.

System One mode

Most of what people ask a model for is a choice, not an essay. Hand it the options it may pick and it reports the probability of each one. Nothing is generated, so there is nothing to parse and no answer outside the list. Typed decisions with calibrated probabilities on POST /v1/systemone, with the request and the reply of TypeSafe's Jev API: a Jev client switches by changing its base URL.

Measured on qwen36 (22 GB), same engine, separate processes. Time from the prompt to the answer.

{
  "state": "340 lines, 8 files, no tests.",
  "questions": {"review": {
    "type": "choice",
    "instructions": "What should the reviewer do?",
    "criteria": {"merge": null, "request changes": null, "close": null}
  }}
}

{
  "answers": {"review": {
    "choice": "request changes",
    "probabilities": {
      "merge": 0.023, "request changes": 0.974, "close": 0.004
    },
    "confidence": 0.961
  }},
  "usage": {"output_tokens": 4, "cost": 0}
}

And the thing generation cannot give

Same document, same model, three closed questions about it (measured when they were the fields of one JSON object, filled one at a time). On area it is sure and it is right. On the other two the entropy is near the maximum: it does not know, and it says so. The chat arm wrote "risk": "high" with exactly the same face.

That number is what you put a threshold on to send a case to a human. System One mode does not write prose, so anything open-ended is still the chat's job, and the speed multipliers above were measured on an engine reading experts from disk.

Your weights stay yours

The engine is one C file per family with no runtime and no dependencies. It runs on a laptop with the network off, and the model files never leave the machine. Apache 2.0, and every container keeps the licence its authors gave it.