Llama: The Model That Named Local AI — Then Left

The escaped lab experiment

Llama is the rare model so foundational that half of local AI wears its name. llama.cpp, Ollama, the llama on every tutorial thumbnail — all of it borrowed from Meta's escaped lab experiment, the first model to put a serious brain on a gaming PC. Then, in April 2026, Meta shipped its shiny new flagship with the weights locked in a vault. The line that turned open weights into a movement went quiet. So here is the whole family, top to bottom: what still runs, what never shipped, and whether the llama is dead or just sleeping.

The dynasty, in fast-forward

Early 2023: the first Llama leaks onto the internet within days, and suddenly anyone can keep a lab-grade language model on their own hard drive. Llama 2 makes that legal. In July 2024, Llama 3.1 lands in three sizes, topping out at 405 billion parameters — the moment open weights got within arm's reach of the frontier. That December, Llama 3.3 distils the big model's smarts down into a 70B. That was the peak of the empire. Then, on April 5, 2025, Llama 4 arrived, and the wheels came off hard enough to earn its own chapter.

The workhorse: 3.1 8B

Meet the model that still does the work. Llama 3.1 8B is two years old and still pulling around nine million downloads a month on Hugging Face. At Q4_K_M it's a 4.9 GB file doing about 135 tokens per second on a 4090 at short context, 95 on a 3090 — and roughly 2 per second on a Raspberry Pi, which is a demo, not a tool. The honest 2026 read: Qwen's 8B beats it on raw benchmarks now. Its real moat is the ecosystem — three thousand-plus finetunes, the Hermes and Dolphin builds, all sitting on this one chassis. It's the Toyota Corolla of local AI: old, everywhere, starts every time.

Where the hardware claims go to die: 3.3 70B

Llama 3.3 70B is where "just run a 70B" collides with reality. The Q4 file everyone means when they say that is 42.5 GB. It fits zero single consumer cards — not a 24 GB 4090, not even a 32 GB 5090. What actually fits on your card is the smallest 2-bit squeeze: 19 GB, 18 tokens per second on a 4090, 10 on a 3090 — a lobotomised copy of the model whose benchmark charts you were sold. The honest path is two used 3090s: real Q4 quality at 17 to 21 tokens per second. The catch is the memory crunch, which pushed used 3090 prices up to roughly a thousand dollars or more each. And Ollama will happily pull the 43 GB default onto your 24 GB card and then crawl at single digits without telling you.

Llama 4: a masterclass in fine print

April 5, 2025. Llama 4 arrives as Scout and Maverick — mixture-of-experts, natively multimodal, and a lesson in reading the footnotes. Meta said Scout "fits in a single GPU" — true, if that GPU is a single H100 datacentre card at 4-bit. Maverick "runs on a single H100 host" — but a host is eight of them. The headline 10-million-token context? On a long-context comprehension test past a hundred thousand tokens, Scout scored 15.6%; Gemini scored 90.6%. And the launch victory lap — number two on LMArena — turned out to be a secret chat-tuned variant. The Maverick you could actually download ranked around 32nd, and LMArena rewrote its rules over the stunt.

The vanishing act

Then everything went quiet. Behemoth, the two-trillion-parameter boss teased at launch, never shipped. The Llama 4.5 that insiders expected in late 2025 never appeared. Instead Meta spent fourteen billion dollars on Scale AI, stood up a Superintelligence Lab, and in April 2026 shipped Muse Spark — internal codename Avocado — a genuinely strong model that's free inside Meta's own apps and completely closed. No weights, no license, no download. The community called it a gut punch. Zuckerberg insists more open models are coming, but the open Llama line has been frozen since April 2025. The vacuum breeds slop, too: a viral article claiming "Llama 5 shipped this spring" is, if you click through today, itself a 404. There is no Llama 5.

So what do you actually run?

On Llama's old home turf — the 8 to 12 GB cards — 3.1 8B is still a fine budget pick, but Qwen's 8B and 14B beat it on quality, and the 24 GB sweet spot now belongs to Qwen's 27B and Gemma's fast 26B. The April 2026 community list of top local models contains zero Llama entries. One throne remains: if you have 48 GB across two cards, Llama 3.3 is still the strongest dense 70B you can download — though even there, the guides now steer you toward the new MoE models first. One contested throne, out of an entire empire.

The poetry of it

Here's the part that lands. Meta left the open-weights room, but the room is still named after their model. Every GGUF you grab, every llama.cpp build, every Ollama pull — the llama outlived its shepherd. Two workhorses still earn their hay every day, one emperor rotted into vaporware, and one fake article got caught with a 404 in its pocket. That's Llama, top to bottom: what actually runs, at what speed, on hardware with a name.

• Llama 3.1 8B: ~9 million downloads a month on Hugging Face; Q4_K_M is a 4.9 GB file.
• Llama 3.1 8B speed: ~135 tok/s on a 4090 (short context), ~95 tok/s on a 3090, ~2 tok/s on a Raspberry Pi.
• Llama 3.1 8B ecosystem: 3,000+ finetunes (Hermes, Dolphin families) built on the same chassis.
• Llama 3.3 70B at Q4: 42.5 GB — fits no single consumer card, not a 24 GB 4090, not a 32 GB 5090.
• Llama 3.3 70B smallest 2-bit: 19 GB, ~18 tok/s on a 4090, ~10 tok/s on a 3090 — a lobotomised copy.
• Two used 3090s (48 GB): real Q4 quality at 17-21 tok/s; used 3090s now run roughly $1,000 or more each after the memory crunch.
• Llama 4 Scout "fits a single GPU" = one H100 at 4-bit; Maverick "single H100 host" = eight H100s.
• Llama 4 long context: 10M-token headline; Scout scored 15.6% on a comprehension test past 100k tokens, versus Gemini's 90.6%.
• Llama 4 launch: the #2 LMArena entry was a secret chat-tuned variant; the downloadable Maverick ranked ~32nd; LMArena changed its rules afterward.
• Behemoth (2 trillion parameters) and Llama 4.5: teased, never shipped.
• Meta: ~$14 billion on Scale AI, a Superintelligence Lab, and Muse Spark (codename Avocado) shipped April 2026 — free in Meta's apps, closed-weight, no download.
• The open Llama line has been frozen since April 2025; Zuckerberg says more open models are coming.
• The viral "Llama 5 shipped this spring" article is a 404 — there is no Llama 5.
• Runnable today: Qwen 8B/14B beat 3.1 8B on quality; 24 GB belongs to Qwen 27B and Gemma 26B; the April 2026 top-local-models list has zero Llama entries.

Every figure here is stated in the episode itself. Facts are verified on the day they're used; where a later correction moved a number, the page uses the corrected one.