TurboFieldfare Runs a 26B-Parameter Model in ~2GB of RAM — and Points to an On-Device AI Deployment Business

TurboFieldfare is not just a clever memory trick.

It is a service business hiding inside a GitHub README.

What happened

A developer named drumih published TurboFieldfare, an open-source inference engine that runs Gemma 4 26B-A4B — "an instruction-tuned model with approximately 26 billion total parameters," of which "about 3.88B" are active per token — on any M-series Mac, using "~2 GB of RAM."

The trick: instead of loading the full "approximately 14.3 GB" model into memory, TurboFieldfare keeps "the shared 1.35 GB core and FP16 KV cache in memory, then streams only the experts needed for each token from SSD." The stated minimum requirement is "8 GB" RAM, "validated on M2 MacBook Air."

The performance numbers are published, not estimated:

• M2 MacBook Air (8 GB): "5.1-6.3 tokens/second decode"
• M5 Pro (24 GB): "31-35 tokens/second decode"

Under the hood, the project uses "MLX affine 4-bit, group 64; 8-bit router; 4-bit shared and routed experts," with a routed-expert cache using a "16-slot LFU (Least Frequently Used) cache per layer" and "bounded parallel pread calls" to fetch experts from disk while Metal computes other branches. It is explicitly "text-only" — no images, audio, video, or tool execution.

The post reached 573 points and 201 comments on Hacker News.

The business idea

A working demonstration that a 26B-parameter model runs on 8 GB of consumer hardware is a service input, not just an engineering flex.

Most businesses that care about keeping data off third-party servers — legal firms handling client files, healthcare intake teams, HR departments processing employee records — have assumed local AI meant either a weak model or expensive dedicated hardware. TurboFieldfare's numbers say otherwise: a capable model, running on a laptop the client may already own, with published tokens-per-second to prove it.

Best customer profile

• small legal, healthcare, or HR teams that want AI-assisted document review but cannot route client data through a cloud API
• companies already running Apple Silicon fleets (M2 through M5) that want a fallback to cloud LLM pricing
• teams that tried "local AI" once, hit a RAM wall, and gave up — before this kind of streaming approach existed

How to package the offer

1. Local inference feasibility audit

Benchmark the client's actual hardware against their actual workload, using TurboFieldfare's own published numbers (5.1-6.3 tokens/second on an 8 GB M2 Air, 31-35 tokens/second on a 24 GB M5 Pro) as the baseline comparison.

2. Workflow migration

Take one recurring, data-sensitive task — document summarization, intake triage, internal search — and rebuild it to run through a local, streaming-from-SSD setup instead of a cloud API call.

3. Quantization and caching configuration

Package the technical setup — 4-bit quantization, 8-bit routing, LFU expert caching — as a one-time configuration deliverable, so the client gets a working local system without needing to understand the internals.

4. Ongoing re-benchmark retainer

New open-weight models ship constantly. A quarterly retainer to re-test the latest model against the client's existing hardware keeps the local setup from falling behind what a cloud API could offer.

Bottom line

TurboFieldfare didn't just prove a 26B model can run in 2 GB of RAM. It published the exact numbers — 8 GB minimum, 5.1-6.3 tokens/second on the cheapest supported Mac — that make "local AI is not fast enough" a claim you can now test against a laptop most small businesses already own.

Source: https://github.com/drumih/turbo-fieldfare