Runtimes

• vLLM Multi-GPU Setup: Tensor Parallelism Without the Idle-GPU Mistake

  Tensor parallelism in vLLM is powerful and easy to misconfigure. This guide covers the two mistakes that strand GPUs and crash on long contexts: tensor-parallel-size must divide your model's attention-head count, and 4–5 GB of VRAM headroom for driver and KV-cache overhead is not optional.

• Ollama vs llama.cpp vs vLLM in 2026: Which Runtime Fits Your Hardware and Workload

  Ollama, llama.cpp, and vLLM solve different problems, not the same problem with different UIs. This guide routes you by constraint — ergonomics, low-level control, or concurrent serving throughput — and gives the "Don't Use Ollama" controversy an honest, both-sides read.

• Ollama's MLX Backend on Apple Silicon: What Changes for Mac Users

  Ollama is migrating to Apple's MLX framework for Mac inference, a shift in how the runtime leverages Apple Silicon. This guide covers what actually changes, how to check which backend you're running, the real performance implications (measured on M4), and what the growing MLX ecosystem means for the llama.cpp-vs-MLX decision on Mac hardware.

• LM Studio vs Ollama: GUI Comfort vs CLI Control for Local AI

  LM Studio and Ollama are the two most common first tools for running local LLMs. The choice is less about features and more about shape: LM Studio for GUI-first experimenters who want discovery and sliders, Ollama for the server-and-scripting path that grows into a homelab.

• Fixing Ollama's 'CUDA Error: Out of Memory' — Even When You Have Free VRAM

  Ollama reports out-of-memory errors even with free VRAM remaining. This guide diagnoses the root cause: context length inflating KV cache, incorrect layer-offload settings, the documented multi-GPU misallocation bug, or the honest truth — the model genuinely does not fit. Ordered by likelihood and includes fixes for each.

• ExLlamaV3 vs GGUF in 2026: Speed, Quality, and Which Stack Deserves Your VRAM

  The EXL3-vs-GGUF debate runs on 2024 data. Map what actually differs: decode speed (EXL3 claims faster prefill, GGUF equal decode for equal bits-per-weight), quality at equal quantization, and the GPU-only constraint that excludes CPU offload and Mac users. A staleness-flagged comparison for power users.

• How to Run LLMs Locally: Which Inference Engine for Your Rig (2026)

  A decision guide that picks the right local inference engine from your hardware, not hype. llama.cpp for CPU and portability, MLX on Apple Silicon, vLLM for CUDA serving — and why we don't recommend Ollama.

• How to Run llama.cpp on an RTX 3090 (CUDA, Step by Step)

  A step-by-step guide to building llama.cpp with CUDA and running a GGUF model on an RTX 3090. Covers driver and toolkit prerequisites, the CUDA build, full GPU offload with -ngl, a throughput check, and an OpenAI-compatible server.