Ollama engine architecture: native Ollama engine versus llama.cpp runner and model compatibility
Parent: Ollama · Published reference · snapshot 2026-10-15 · skill ollama-expert/references/ollama-engine-architecture.md
↓ Facts as markdownall context files
Source-anchored research for Ollama engine architecture: native Ollama engine versus llama.cpp runner and model compatibility. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Structure and components
- Commit 9db4bdb (2026-05-29, PR #16031, 'runner: Remove CGO engines, use llama-server exclusively for GGML models') deleted the vendored GGML and llama.cpp backend, the CGO runner, the Go model implementations and sample. Its message says llama-server, built from upstream llama.cpp via FetchContent, is now the sole inference engine for GGUF models, and safetensors models run on the MLX engine. The first tag containing it is v0.30.0. The commit landed on main 2026-05-29. The first tag that contains it is v0.30.0-rc32 (5 commits later), then v0.30.0. The PR body calls the llama.cpp compatibility patch temporary: the goal is that Ollama eventually runs unmodified upstream loaders. (high confidence) (verified as of 2026-10-14) [source]
- Release timing of the engine removal: the v0.30.0 GitHub release object shows publishedAt 2026-05-13 and createdAt 2026-06-01, and the v0.30.0 tag points at a commit dated 2026-06-01 that contains commit 9db4bdb (PR #16031, merged 2026-05-29). The v0.30.0 notes say Ollama 0.30 brings 'improved compatibility and performance using llama.cpp' that 'augments the MLX engine on Apple Silicon', and list known issues (laguna-xs.2 unsupported on Windows/Linux, llama3.2-vision unsupported, nomic-embed-text now lowercases input). Cite the tag, not the 2026-05-13 publish date, for when users got the change. (high confidence) [source]
How it works
- Before v0.30.0, Ollama ran GGUF models on two in-process engines selected per model: the native Go 'Ollama engine' (runner/ollamarunner, model/models, ml/backend/ggml) and the llama.cpp-based llamarunner (llama/ vendored llama.cpp, CGO). In v0.24.0, llm/server.go used the Ollama engine when envconfig.NewEngine() (OLLAMA_NEW_ENGINE) was set or when the GGUF general.architecture was in the OllamaEngineRequired() list, and fell back to llamarunner if the model was not yet supported. (high confidence) [source]
- The compat shim is declared temporary. Package compatmigrate converts legacy Ollama-format GGUF manifests already in the local store into clean llama.cpp-compatible 'llamacpp' manifest-list children in the background, so users need not re-pull. Its header says the code is to be removed once manifest-list publishing has been live for enough releases, after which unmigrated local artifacts should be re-pulled. RunnerForManifest tags a GGUF 'ggml' when a compat detector fires and 'llamacpp' otherwise. (high confidence) (verified as of 2026-10-14) [source]
- Consequence for third-party and community GGUFs: with the native engine removed, a Hugging Face GGUF is loaded by the same upstream llama.cpp that other tools use, so a file that loads in stock llama.cpp at Ollama's pinned build should load in Ollama. Ollama's v0.30.0 notes advertise support for GGUF-based models from Hugging Face. The older failures came from the custom ggml fork: critics cite standard gpt-oss GGUFs failing because Ollama lacked a tensor type the model needed. (medium confidence) (verified as of 2026-10-14) [source]
Comparisons and alternatives
- Ollama-format GGUFs published before the switch differ from upstream in arch names, KV keys, tensor names or file layout. llama/compat/ carries a patch (001-llama-cpp-hooks.patch) plus a shim that translates them in memory inside llama-server's loader. Per-architecture handlers cover gemma3, gemma3n, gemma4, gpt-oss, lfm2, mistral3, qwen35, qwen35moe, qwen3-vl, qwen2.5-vl, llama4, nemotron_h_moe, glm-ocr, deepseek-ocr, embeddinggemma and others. For any non-Ollama file every shim entry point is a no-op, and some handlers disable mmap because they transform tensors at load. (high confidence) (verified as of 2026-10-14) [source]
Facts and statements
- Pre-0.30 OllamaEngineRequired() listed these architectures for the native engine: bert, deepseek2, deepseekocr, gemma3, gemma3n, gemma4, gptoss/gpt-oss, laguna, llama4, mistral3, mllama, nemotron_h variants, nomic-bert, olmo3, qwen25vl, qwen3, qwen3moe, qwen35, qwen35moe, qwen3next, qwen3vl, qwen3vlmoe, glm4moelite, glmocr, lfm2, lfm2moe. Other architectures ran on llamarunner. (high confidence) (verified as of 2026-10-14) [source]
- The premise that Ollama still has a native Go engine alongside llama.cpp is out of date as of v0.40.4 (2026-10-12). The repo tree at main has no runner/ directory, no model/models, no ml/backend/ggml and no OLLAMA_NEW_ENGINE variable. The reported '0.30.x reintroduction of llama.cpp as a second backend' is a replacement, not an addition: v0.30.0 notes say llama.cpp 'augments the MLX engine on Apple Silicon', and the native Go GGML engine is gone. (high confidence) (verified as of 2026-10-14) [source]
- Today Ollama has two inference backends. For GGUF it spawns the upstream llama-server binary (llm/llama_server.go, FindLlamaServer, launched with --model, --port and --host 127.0.0.1; one binary that loads GPU backends dynamically). For safetensors on Apple Silicon it runs the MLX engine (mlxrunner/). (high confidence) (verified as of 2026-10-14) [source]
- Ollama pins llama.cpp with the file LLAMA_CPP_VERSION (documented in llama/README.md, 'Updating llama.cpp'). The source is fetched at configure time via 'cmake -S llama/server --preset cpu' and patched by llama/compat/. At main on 2026-10-14 the pin is b11521 (commit adc4aae9, 2026-10-09). Git history of the file shows weekly bumps: b10864 (2026-09-10), b10969 (2026-09-15), b11081 (2026-09-22), b11232 (2026-09-28), b11351 (2026-10-05), b11521 (2026-10-09). The v0.40.0 tag pins b11351. The v0.31.1 release notes say 'updated the underlying llama.cpp engine to build 9840'. (high confidence) (verified as of 2026-10-14) [source]
- The runner is chosen per manifest, not by a flag: a manifest list carries children tagged runner mlx, ggml or llamacpp with format safetensors or gguf. runnerPreferences() in manifest/manifest.go picks mlx, then llamacpp, then ggml on darwin/arm64, and llamacpp, then ggml, then mlx elsewhere. A hidden --runner flag on run and pull (mlx, ggml, llamacpp) overrides the choice. As of v0.40.0 (2026-09-25), supported architectures run on MLX by default on Apple Silicon. No Ollama prose documentation describes runner selection; the evidence is the source code (manifest/manifest.go at commit 957c8cd, registered as a code excerpt) plus the v0.40.0 release notes. (high confidence) (verified as of 2026-10-14) [source]
- Import behaviour: Ollama does not quantize GGUF models on import. docs/import.mdx tells users to quantize with llama.cpp's llama-quantize first, and supports single-file and split-shard GGUF via FROM with a wildcard. Chat templates stay Go templates in the Modelfile or manifest; a critic reports an unrecognised embedded Jinja template falls back to a bare prompt template until the user writes a TEMPLATE. (medium confidence) (verified as of 2026-10-14) [source]
- RamaLama's README (main, last README commit 2026-10-02, fetched 2026-10-14) says: 'Ollama models are no longer compatible with llama.cpp, and support for the Ollama transport will be removed in a future release. Users should migrate to Hugging Face models.' This is consistent with the Ollama source: Ollama-format blobs of affected architectures need the in-tree compat patch that stock llama.cpp lacks. The llamacpp manifest children Ollama now writes are described as clean llama.cpp-compatible GGUFs, but the source does not show that the public registry serves them for every model, so a given pulled blob must be tested in stock llama.cpp. (medium confidence) (verified as of 2026-10-14) [source]
- Releases after the switch ship llama.cpp updates continuously and fix architecture support in compat patches: v0.30.2 added Laguna compatibility, ignored SSE ping comments, counted cached prompt tokens and detected load stalls from llama-server output; v0.31.1 updated llama.cpp to build 9840; v0.30.0 listed known issues (laguna-xs.2 on Windows/Linux, llama3.2-vision, nomic-embed-text lowercasing). (high confidence) (verified as of 2026-10-14) [source]
- An aggregator (aiweekly.co, 2026-05-14) reported that Ollama dropped its custom GGML fork for a direct llama.cpp dependency, tied to v0.30.0-rc15. The Ollama source confirms the direction (commit 9db4bdb); the aggregator's rc tag and framing are secondary. (low confidence) (verified as of 2026-05-14) [source]
- Ollama's May 2025 blog announced the original native engine, citing per-model self-contained implementations and better multimodal handling, and gpt-oss launched on it with MXFP4 kernels. That design was abandoned within about a year (commit 9db4bdb); advice about 'the Ollama engine', OLLAMA_NEW_ENGINE and ollamarunner applies only to versions up to v0.24.x. (medium confidence) (verified as of 2026-10-14) [source]
Children
- GGUF import and Modelfile chat templates (frontier)
- Ollama GGUF compat shim and compatmigrate (frontier)
- Ollama manifest list runner selection (frontier)
- llama-server runner process in Ollama (frontier)
Frontier under this node: GGUF import and Modelfile chat templates, Ollama GGUF compat shim and compatmigrate, Ollama manifest list runner selection, llama-server runner process in Ollama