Can I Run Kolibri-1 Locally? ~78 GB FP8, 3.46B Active, Not a 24 GB Card

Short answer: Kolibri-1 is Aleph Alpha’s German-and-English MoE, released 3 October 2026, with 78,103,074,560 total parameters and 3,457,573,120 active per token. The official FP8 checkpoint is about 78 GB of weights and the card’s minimum is 2× A100 80GB or 1× H200 (also 2× H100 SXM5, 1× B200, or 1× B300), served through Aleph Alpha’s vLLM plugin. A 24 GB card does not hold that file. Unified-memory machines matter because the whole expert pool has to reside somewhere, while decode compute tracks the small active set when the runtime actually skips idle experts. Community GGUF and MLX files are smaller and unofficial, and on 2026-10-06 they still need patched runtimes. LocalRig has not measured tok/s.

This page is a constraint selector. It is not a quality ranking against Qwen or Gemma, and it is not the dense 70B hardware guide. A 78B total parameter count is a memory problem of the full MoE, not a license to reuse 70B-dense math, and not a 3.5B download.

What is Kolibri-1?

An Apache-2.0-weight MoE from Aleph Alpha, not a dense 78B and not a 4B file. The Hugging Face card for Aleph-Alpha/Kolibri-1 (accessed 2026-10-06) lists:

Field	Vendor card
Release date	3 October 2026
Total parameters	78B (78,103,074,560)
Active parameters / token	3.46B (3,457,573,120)
Languages	German, English
Context	1,048,576 tokens validated; ≤262,144 recommended for serving efficiency and complex tasks
FP8 precision	float8_e4m3fn weights in 128×128 blocks; embeddings, LM head, norms, and the MoE router stay bfloat16
FP8 footprint	~78 GB (FP8 weights)
License row	Apache 2.0

config.json on that same repo sets model_type to kolibri1, 50 layers, 384 experts, 6 experts per token, sliding_window 513, and max_position_embeddings 262144. The 1,048,576-token path is a serve override on the card (--max-model-len 1048576 plus an hf-overrides max-position flag), not the default config length.

The BF16 sibling uses the same parameter counts and lists ~156 GB of BF16 weights, with a minimum of 4× A100 80GB, 4× H100 SXM5, 2× H200, 1× B200, or 1× B300. The product page rounds active parameters to 3B and lists precision as bfloat16. Use the two cards for footprint. The product page is the rounded summary.

Hub repository storage on 2026-10-06 lines up with those rounded footprints and is still not a VRAM peak: Kolibri-1 usedStorage 78,852,606,661 bytes (78.85 GB decimal, bytes ÷ 10⁹), and Kolibri-1-BF16 156,213,402,552 bytes (156.21 GB decimal). That count includes the published file set.

The Frontier (4 October 2026) reports the same 3 October release, the 78.1B / 3.46B pair, the ~78 GB FP8 footprint, and the vLLM plugin. Treat Frontier as a secondary write-up of the card, not as a LocalRig bench.

Why does a small active count favor unified memory?

Because capacity and bandwidth split. Aleph Alpha’s design-goals section says the mixture-of-experts keeps compute per token low while the full model must be held in memory. The tech report says each token uses 4.4% of the parameters.

Derived check, not a new measurement: 3,457,573,120 / 78,103,074,560 = 0.04427 (4.43%), which rounds to the report’s 4.4%. The division uses only the two integers on the card.

That split is why a Mac, a Strix Halo box, or a DGX Spark can be a fit conversation even though Aleph Alpha’s minimum is a datacenter GPU:

• Weights. Every expert has to live in one address space. A 24 GB discrete card cannot store ~78 GB of FP8 weights. A 128 GB unified pool can hold the official footprint with room left over — see the derived residual below — or a community quant with more room. Strix Halo vs Mac Studio and DGX Spark vs dual RTX 3090 describe that 128 GB class. Kolibri tok/s on those boxes are not yet measured.
• Decode. If the runtime dispatches only the active experts, bytes moved per token track ~3.46B parameters, not 78B. Unified memory does not make the idle experts free. It keeps the pool resident so a sparse kernel can read the active slice. A runtime that streams every expert each token gives up that advantage. Stock Ollama cannot do this yet, because it cannot load the architecture.

Do not type 3.46 into the VRAM calculator and treat the result as Kolibri. That calculator is a dense estimator (params × bytes per parameter, plus a generic KV term). LocalRig’s Q4_K_M coefficient is 0.59 bytes/parameter.

Derived illustration, same date: 3.457573120 × 0.59 = 2.04 GB. That is the trap. 78.103074560 × 0.59 = 46.1 GB, which sits near Hob-forge’s published 47.5 GB Q4_K_M file and is still an estimate, not a load. Use 78B total if you touch the calculator at all, then throw the result out in favor of the card’s ~78 GB FP8 line or a named community file size. KV at 262,144 or 1,048,576 tokens is not yet measured for this architecture. The calculator will not invent a trustworthy Kolibri KV number.

Derived residual for a 128 GB unified machine and the official FP8 weights: 128 − 78 = 50. About 50 GB would remain before the operating system and KV cache. The card’s 78 is already a rounded footprint, so this 50 is rounded arithmetic. It is not a measured working set on a Mac, Strix Halo, or Spark.

Which memory tier can hold which file?

Official weights and community files are different rows. “Fits” below means a published file or footprint compared with the tier. It is not a LocalRig load. Context is a separate column because a weights-only fit is not a filled window.

Memory tier	Official weights	Community file (labeled)	Context status
16–24 GB discrete (3090 / 4090 class)	FP8 ~78 GB does not fit. BF16 ~156 GB does not fit.	velaia 2-bit file is 24 GiB, with peak 26 GB on a short prompt and 30 GB at a 4k-token prompt. 30 GB is already over 24 GB.	Not yet measured on 24 GB
36 GB unified	FP8 does not fit	velaia: 2-bit fits 36 GB+ Macs (same 26 / 30 GB peaks)	Their speed note is an M1 Max at short context, not 262k
48 GB unified	FP8 does not fit	velaia 3-bit file 33 GiB, peak 35 / 39 GB, “48 GB+ Macs”. 4-bit’s 4k peak is 48 GB, so 4-bit at 4k sits on a 48 GB ceiling.	Not a filled 256k
48 GB discrete (two 24 GB cards)	FP8 does not fit one or two 24 GB cards as a weights home for ~78 GB	Hob-forge Q4_K_M file 47.5 GB. GPU backends not tested on that card.	Contexts above 8,192 tokens not verified there
64 GB unified	FP8 ~78 GB does not fit	velaia 4-bit 41 GiB, peak 44 GB short / 48 GB at 4k, “64 GB+ Macs”	Short / 4k column only
128 GB unified (Mac, Strix Halo, DGX Spark class)	Weights-only derived residual ~50 GB before OS and KV. Not yet measured on these machines.	Hob-forge Q8_0 file 83.1 GB; they recorded 81.6 GB RAM on a CPU host	262k and 1M KV not yet measured
Datacenter minimum (vendor)	FP8: 2× A100 80GB, 2× H100 SXM5, 1× H200, 1× B200, or 1× B300. Recommended on the card: 2× H100 SXM5, 2× H200, 1× B200, or 1× B300.	Not the community path	≤262,144 recommended; 1,048,576 is the validated ceiling with extra flags. tok/s not yet measured here

A 24 GB shopper who wanted “the small active MoE” still has a real 24 GB guest: Gemma 4 26B-A4B (Unsloth’s 4-bit band is 16–18 GB) or Qwen 3.8 27B Q4. Those are different models. They are the cards that match a used 3090. Kolibri-1’s official file does not.

How much unified memory for local LLMs is the buying frame for the 36 / 48 / 64 / 128 GB rows. Verify the current Mac or mini-PC memory config before you order. This page does not pin a 2026 street price.

What do the third-party 24 GB and 48 GB Mac lines actually say?

They are community fit notes, repeated by Connic, and they are not a 24 GB Mac minimum. velaia/Kolibri-1-MLX-2bit (unofficial, accessed 2026-10-06) publishes this table. Routed experts use the named bit width; attention and the shared expert are 4-bit; embeddings and the LM head are 8-bit; the router is unquantized. Conversion stack: mlx 0.32.3, mlx-lm 0.32.0, source revision e52eb4627d11516b0c01de49210ab5a4e4061444, converted on an Apple M1 Max (64 GB).

velaia variant	Size	Peak memory (short / 4k-token prompt)	Fits, on their card
4-bit	41 GiB	44 / 48 GB	64 GB+ Macs
3-bit	33 GiB	35 / 39 GB	48 GB+ Macs
2-bit	24 GiB	26 / 30 GB	36 GB+ Macs

The same card says generation runs at about 52–56 tok/s on an M1 Max for all three. That is community-cited. The machine in the speed sentence is “an M1 Max”; the 64 GB figure is in the conversion line. The only context pin next to the peaks is short / 4k-token prompt. It is not a filled 262,144-token run, and it is not a LocalRig measurement.

Connic (Connic Research Team, 4 October 2026) repeats those sizes and Mac floors: 2-bit at 24 GiB for Macs with 36 GB or more, 3-bit sized for 48 GB unified memory, 4-bit for 64 GB or more. Label that whole block third-party. Connic is not Aleph Alpha, and LocalRig did not rerun the peaks. A 24 GB Mac is below even this third-party floor: the 24 GiB figure is the 2-bit file, and both velaia and Connic put the Mac line at 36 GB or more because the 4k peak is 30 GB.

On a 48 GB Mac, velaia says to raise the GPU wired limit for the 3-bit variant (iogpu.wired_limit_mb=40960) and not to run another large model at the same time. That is their operating note, not a vendor requirement.

Which runtime actually loads it, and at which version?

The official path is vLLM plus a pinned plugin. Community paths are forks and open pull requests. Checked 2026-10-06.

Runtime	Pin	Kolibri-1 status
vLLM via aleph-alpha-inference	GitHub pyproject.toml version 1.0.0 depends on vllm>=0.29.0,<0.30.0, torch>=2.9.0, transformers>=5.5.3. The card’s install line is aleph-alpha-inference>=1. Image: ghcr.io/aleph-alpha/aleph-alpha-inference.	Vendor path. vllm serve Aleph-Alpha/Kolibri-1 --kv-cache-dtype fp8 with --reasoning-parser kolibri1, --tool-call-parser kolibri1, and --enable-auto-tool-choice. Sampling on the card: temperature 1.0, top_p 0.97, top_k 128.
llama.cpp	Issue #29922 was open (created 2026-10-03, updated 2026-10-04). No merged kolibri1 architecture found in that search.	Stock llama.cpp does not load it. Hob-forge ships kolibri1-llama.cpp.patch against upstream commit 836d571.
Ollama / LM Studio	They track llama.cpp (and, on Mac, MLX).	Hob-forge: they will not load this GGUF until the architecture is included. Not yet measured.
mlx-lm	PR #1945 was open (created and updated 2026-10-04). velaia built with mlx-lm 0.32.0.	Weights load through the repo’s run.py launcher, which registers kolibri1.py, until that PR ships in a release.
ds4 (aparusel)	Fork aparusel/ds4-kolibri-1 main. Not llama.cpp GGUF. Source revision e52eb46….	Metal and CPU reference. The card says CUDA, ROCm, tensor/pipeline parallelism, and SSD streaming are not supported for this model.

Related runtime guides, for the stacks in general: Ollama vs llama.cpp vs vLLM, how to run LLMs locally, and vLLM multi-GPU setup. Those pages are not Kolibri version pins. The pins in the table above are the ones checked for this model.

Hob-forge’s file sizes (community, accessed 2026-10-06): Q2_K 28.6 GB, Q3_K_M 37.5 GB, Q4_K_M 47.5 GB, Q5_K_M 55.6 GB, Q6_K 64.2 GB, Q8_0 83.1 GB. Q3_K_M and Q2_K were added 2026-10-06; Q5_K_M and Q6_K on 2026-10-05. Their Q4_K_M CPU note names an AMD Ryzen 7 7800X3D, 8 threads, 128 GB RAM, no GPU: German and English smoke prompts around 11.9–14.9 tok/s generation, including 11.9 tok/s on a 1,393-token German prompt. Batch size is not stated. They say contexts above 8,192 tokens were not verified, and CUDA, Vulkan, and Metal were not tested. Q8_0 on the card uses 81.6 GB RAM and 9–10 tok/s generation on “the same machine” as an unnamed CPU host with a network-mounted disk. Do not promote that sentence into a 7800X3D claim the card did not write. Community-cited. Not first-party.

aparusel’s ds4 files, same access date: Q8 77.42 GiB, Q4 42.56 GiB, Q2 22.78 GiB. They are a different container from Hob-forge’s llama.cpp patch. A 22.78 GiB Q2 still has to clear the OS and KV on whatever Mac or CPU box you point ds4 at. No LocalRig load of those files.

Where does the official checkpoint actually run?

On the card’s GPU list, with the plugin, or not at all if you only own a 24 GB card. Rent that class if you need the FP8 weights rather than a community quant. Vultr GPU Cloud is the A100/H100 inventory check; Vultr does not rent a consumer 3090 or 4090. Marketplace and specialist hours: RunPod, Vast.ai, GPUMart. Price context for the H100 hour, without a new rate invented here: H100 rental price comparison.

A single 80 GB A100 is not on the FP8 minimum list. The minimum that includes A100 is two 80 GB A100s. One H200 is on the minimum list. KV cache and the FP8 KV flag sit on top of the ~78 GB weights. LocalRig has no measured headroom for that extra on any of those SKUs.

Buying a used 3090 for this model is the wrong turn. The used RTX 3090 guide is for models whose files fit in 24 GB.

What does the license actually cover?

The weights and configuration files in the Hugging Face repo. The card’s license section (accessed 2026-10-06) says Apache 2.0 rights “only apply to the weights and configuration files published in this repository” and “especially do not extend to underlying code, model architecture, parameter settings or any training method.” Apache 2.0 on the weight files still allows commercial use, modification, and redistribution of those files. It is not a grant of Aleph Alpha’s training code or architecture.

The serving plugin is a separate Apache-2.0 package: aleph-alpha-inference 1.0.0 declares license = "Apache-2.0" in its pyproject. Community GGUF and MLX repos are unofficial conversions. velaia and Hob-forge both say they are not affiliated with or endorsed by Aleph Alpha.

The card’s intended deployment is assistants and agentic workflows in which a person reviews the output before it is acted on. German and English are the supported languages. Aleph Alpha states it is a signatory of the EU GPAI Code of Practice. That is a governance note, not a hardware spec.

Who this is NOT for

• 24 GB card owners who want the official FP8 checkpoint. ~78 GB of weights does not fit.
• Anyone who reads 3.46B active as a 2 GB download. The calculator trap above is the whole point of the active-parameter headline.
• Readers who treat Connic’s or velaia’s 48 GB Mac line as Aleph Alpha’s minimum. Those are third-party fit notes for a 3-bit MLX file. The vendor minimum is the datacenter list.
• Stock Ollama or LM Studio users on 2026-10-06. The architecture is not upstream yet.
• Anyone who needs a filled 262k or 1M tok/s number on a Mac, Strix Halo, or DGX Spark. Not yet measured.
• Shoppers comparing this URL to a dense 70B Q4. Different file, different runtime, different floor.

Methodology

• SKU identity: Aleph-Alpha/Kolibri-1 and Kolibri-1-BF16 cards plus config.json, accessed 2026-10-06. Release date on the card: 3 October 2026.
• Official fit: vendor hardware rows. Hub usedStorage is a repository byte count, divided by 10⁹ for decimal GB. It is not a VRAM trace.
• Active-fraction check: integer division of the card’s two parameter counts. Matches the tech report’s rounded 4.4%.
• 128 GB residual: 128 − 78, using the card’s rounded FP8 footprint. Derived. Not a load.
• Calculator illustration: site coefficient 0.59 bytes/parameter for Q4_K_M times 3.46B and times 78.1B. Derived, to show the wrong input. Published file sizes override it.
• Community fit and speed: velaia MLX card and Hob-forge GGUF card, accessed 2026-10-06. Connic (2026-10-04) is cited only as a third-party restatement of the Mac floors. Frontier (2026-10-04) is a secondary write-up of the release.
• Runtime pins: aleph-alpha-inference pyproject 1.0.0; llama.cpp issue #29922 open; mlx-lm PR #1945 open; Hob-forge patch commit 836d571; ds4 fork main. All checked 2026-10-06.
• LocalRig first-party: none. No engine.db row. No rented GPU.

Sources

• Aleph-Alpha/Kolibri-1 (accessed 2026-10-06).
• Aleph-Alpha/Kolibri-1-BF16 (accessed 2026-10-06).
• Kolibri-1 config.json (accessed 2026-10-06).
• Tech report PDF (accessed 2026-10-06).
• Kolibri product page (accessed 2026-10-06).
• aleph-alpha-inference pyproject.toml, version 1.0.0 (accessed 2026-10-06).
• The Frontier, 4 October 2026.
• Connic, 4 October 2026 — third-party Mac lines.
• velaia/Kolibri-1-MLX-2bit (accessed 2026-10-06).
• Hob-forge/Kolibri-1-GGUF (accessed 2026-10-06).
• aparusel/kolibri-1-gguf (accessed 2026-10-06).
• llama.cpp issue #29922.
• mlx-lm PR #1945.
• VRAM calculator.
• Ollama vs llama.cpp vs vLLM.