Last week someone ran a 744-billion-parameter AI model on a computer with no graphics card, using just 25 gigabytes of RAM. It worked. It produced correct, coherent responses, on par with running that same model on a data center rack with several high-end GPUs. The trick has a cost: one new token every ten to twenty seconds. That’s roughly how long it takes you to read this entire sentence, word by word, before the next one shows up.

That’s the pattern I want you holding in your head the next time someone on your team brings you a brilliant demo. A proof of concept that works is not the same thing as a system you can deploy. Those are two different questions, and the AI industry conflates them constantly, especially when the result looks impressive.

What colibrì actually pulled off

The project is called colibrì. It was built by a developer who posts on Hacker News as vforno, and he shipped it as a Show HN on July 9. The thread hit 922 points and 238 comments, and the GitHub repo (Apache-2.0 license) went from zero to over 9,600 stars in under two weeks, still climbing.

What it does is run GLM-5.2, Zhipu AI’s mixture-of-experts (MoE) model with 744 billion total parameters and roughly 40 billion active per token, on a consumer laptop with no GPU. The trick isn’t magic, it’s careful systems engineering: colibrì keeps only the dense, shared portion of the model resident in RAM, about 17 billion parameters quantized down to 4 bits, which fits in roughly 10GB. Everything else, thousands of routed experts adding up to around 370GB, stays on disk and streams in on demand, backed by a per-layer LRU cache that also leans on the operating system’s own page cache. The entire engine fits in a single C file of about 2,400 lines. No Python, no BLAS, no GPU.

This wasn’t a one-off fluke. Salvatore Sanfilippo, the creator of Redis and known in the industry as antirez, independently built a branch of his own inference engine using a very similar disk-streaming technique, and it got referenced approvingly in the same Hacker News thread. When two engineers with completely different backgrounds land on the same design in the same week, that’s not coincidence. That’s a problem whose solution was waiting for someone to write it down.

Why 10-20 seconds per token isn’t a tuning problem

Here’s the detail almost nobody explains when they share the link. GLM-5.2’s “active” 40 billion parameters aren’t the same 40 billion every time. In a mixture-of-experts model, the router decides, per token, which experts to use out of thousands available. That means nearly every new token needs a different subset of experts, and most of those subsets aren’t sitting in the 25GB of RAM cache yet.

The result is that colibrì has to hit disk on almost every generation step, and reading from even a fast consumer SSD takes orders of magnitude longer than reading from RAM. That’s the actual bottleneck. Not the math, the wait for the data. The project uses speculative decoding to amortize several tokens per disk read, which softens the problem but doesn’t remove it. Ten to twenty seconds per token isn’t a misconfigured setting you fix by flipping a flag. It’s the ceiling the architecture imposes the moment you decide to run a 744-billion-parameter model on 25GB of RAM.

And that ceiling matters differently depending on what you’re building. For a demo answering one query with unlimited time and nobody else waiting, twenty seconds per token is tolerable, even fun to watch. For an agent handling live customer support with dozens of simultaneous conversations, a predictable latency, and a per-token cost that has to hold up across millions of calls a month, that same number is disqualifying. It’s not the same system, even though the model underneath is identical.

I’ve watched this movie since 1990

I started programming at 15, on a Commodore 64. I saved code onto cassette tapes because there was no other option. Since then I’ve watched this exact pattern repeat more times than I can count: a garage demo, a weekend hack, a research paper proves something theoretically impossible actually runs. And then months pass, sometimes years, before someone else turns that into something reliable, cheap, and boring enough to run in production every single day without anyone babysitting it. The jump from “works once, under perfect conditions” to “works every time, with real users, with nobody watching it” is where almost all the engineering work that actually matters lives. I’ve written before about why recognizing that pattern across cycles is the real moat, not the memory of having seen the last technology come and go.

What this means in an AI Maestro discovery call

That’s why AI Maestro’s two-month discovery is built to ask that question before any other, no matter where the demo that sparked the conversation came from: an open-source project like colibrì, a vendor pitch, or an internal hackathon. The question that actually determines the outcome is almost never the first one anyone brings to the table: what latency does your operation need, what can you afford to pay per token at your real volume, and what happens when a hundred people use it at the same time instead of one.

That jump straight to building, skipping past those questions, is exactly the failure mode we described when we mapped the 14 stages of AI adoption that almost no company respects: most companies skip straight to the build stage and get stuck there. In AI Maestro’s two-month discovery, evaluating the gap between technical capability and a deployable system is an explicit part of the filter before we recommend building anything. Not because we distrust the engineering behind projects like colibrì, quite the opposite, it’s a real achievement. But because a company that conflates those two questions ends up with committed budget behind something that should never have left the lab.

If your team is looking at an impressive demo this week and wondering whether it’s time to build on it, that’s exactly the conversation we have in AI Maestro’s discovery engagement before a single development contract gets signed.