RUNTIMES · SEPTEMBER 2026 INDEX
Pick the window, then the stack.
Ollama vs llama.cpp vs vLLM is not one page. Short chat and a 200k agent window are different jobs. The 27B 200k cell is the September lab runtime; generic pickers stay below it. No invented Ollama filled-200k tok/s.
27B at 200k
The September lab cell. Advertised Ollama 256K is not a 24 GB 200k packet. Empty filled-context cells stay empty.
Daily driver
Short-context picker pages. Do not treat these tok/s as a 200k agent result.
- Ollama vs llama.cpp vs vLLM in 2026: Which Runtime Fits Your Hardware and Workload Ollama, llama.cpp, and vLLM solve different problems, not the same problem with different UIs. This guide routes you by constraint — ergonomics, low-level control, or concurrent serving throughput — and gives the "Don't Use Ollama" controversy an honest, both-sides read.
- LM Studio vs Ollama: GUI Comfort vs CLI Control for Local AI LM Studio and Ollama are the two most common first tools for running local LLMs. The choice is less about features and more about shape: LM Studio for GUI-first experimenters who want discovery and sliders, Ollama for the server-and-scripting path that grows into a homelab.
- ExLlamaV3 vs GGUF in 2026: Speed, Quality, and Which Stack Deserves Your VRAM The EXL3-vs-GGUF debate runs on 2024 data. Map what actually differs: decode speed (EXL3 claims faster prefill, GGUF equal decode for equal bits-per-weight), quality at equal quantization, and the GPU-only constraint that excludes CPU offload and Mac users. A staleness-flagged comparison for power users.
- Ollama's MLX Backend on Apple Silicon: What Changes for Mac Users Ollama is migrating to Apple's MLX framework for Mac inference, a shift in how the runtime leverages Apple Silicon. This guide covers what actually changes, how to check which backend you're running, the real performance implications (measured on M4), and what the growing MLX ecosystem means for the llama.cpp-vs-MLX decision on Mac hardware.
How-to / recovery
Install, OOM, and multi-GPU wiring. Not a model selector.
- How to Run LLMs Locally: Which Inference Engine for Your Rig (2026) A decision guide that picks the right local inference engine from your hardware, not hype. llama.cpp for CPU and portability, MLX on Apple Silicon, vLLM for CUDA serving — and why we don't recommend Ollama.
- How to Run llama.cpp on an RTX 3090 (CUDA, Step by Step) A step-by-step guide to building llama.cpp with CUDA and running a GGUF model on an RTX 3090. Covers driver and toolkit prerequisites, the CUDA build, full GPU offload with -ngl, a throughput check, and an OpenAI-compatible server.
- Fixing Ollama's 'CUDA Error: Out of Memory' — Even When You Have Free VRAM Ollama reports out-of-memory errors even with free VRAM remaining. This guide diagnoses the root cause: context length inflating KV cache, incorrect layer-offload settings, the documented multi-GPU misallocation bug, or the honest truth — the model genuinely does not fit. Ordered by likelihood and includes fixes for each.
- vLLM Multi-GPU Setup: Tensor Parallelism Without the Idle-GPU Mistake Tensor parallelism in vLLM is powerful and easy to misconfigure. This guide covers the two mistakes that strand GPUs and crash on long contexts: tensor-parallel-size must divide your model's attention-head count, and 4–5 GB of VRAM headroom for driver and KV-cache overhead is not optional.
Reference
Setup and recovery pages that are not a SKU bake-off.