Blog · field notes
How the engine actually gets built.
Notes on the work: what we port, what we measure, what we ship.
Serial · 2026
Notes, in order.
Where the engine is going: HRX and the NPU
The engine is heading to AMD's HRX backend, whose kernels we write in Loom, plus the NPU.
Qwen3.8-27B at 45.7 tok/s: DFlash2 in 1bit serve, and 4-bit activations with a Hadamard rotation
Qwen3.8-27B is a dense 27B model, and on Strix Halo its decode speed is set by memory bandwidth: every token reads the whole 15 GB file, so plain decode sits near 11-12 tok/s. Prompt processing is set by the GPU's matrix units instead. This week we pushed on both. Every number below was measured on Strix Halo (Ryzen…
Laya routing: ask the model what it knows
The engine can run the same model on several devices of one machine: Vulkan and HRX on the Radeon, ROCm, ZINC, and the NPU for its own model format. Someone has to pick one for each request. Step 4 of the port gave that job to Laya, a small decision model that answers typed questions about a piece of text in one…
509 tokens a second of prompt: the Hadamard route ships
Earlier today we measured a way to make Qwen3.8-27B read prompts faster on Strix Halo: 4-bit activations on the GPU's matrix units, with a Hadamard rotation to keep them accurate. It is no longer an experiment. The engine now has a tool that makes the rotated file, 1bit serve recognises the file and runs it on the…
Milestone: the HRX decode race is solved
Yesterday's post ended with a workaround. HRX's split flash-attention decode kernel gave different answers to identical requests, and on Qwen3-Coder-30B it sometimes produced NaNs that faulted the GPU (#123, #140). So 1bit serve --device hrx stopped using it. Today we found the cause and fixed it, and the kernel is…
Zyphra's whole family, four older architectures, and experts from the drive
A lot landed this week. Zyphra's model family now runs end to end, including its vision models. Four older architectures that upstream llama.cpp never shipped are in. 1bit serve can run a MoE model whose experts do not fit in memory. And the HRX decode path became deterministic. Every number below was measured on…
The NPU drafts, the GPU checks
Speculative decoding makes a large model faster by letting a small one guess ahead: the small model proposes a few tokens, the large model checks them all in one pass, and every guess it agrees with is a token it did not have to produce one at a time. Usually both models run on the GPU. On Strix Halo there is a second…
Speculative decoding: where it pays, and where it does not
Speculative decoding lets a cheap drafter guess the next few tokens and the model check them all in one pass. When the guesses are right, several tokens come out of one pass instead of one. We measured it on the two most popular Unsloth models on Strix Halo, and the answer depends entirely on the model.
Smaller files: Unsloth, ROCmFPX and a ternary 27B
A smaller model file decodes faster, because decode is limited by how many bytes the GPU reads per token. The question is what each format costs in accuracy. We measured three families of small files on Strix Halo against their full-precision originals: KL divergence (how far the model's predictions move) and how…
One GPU, many users
A chat with one person and a server for many want different things from the same GPU. We measured Qwen3.8-27B (UD-Q4KXL) on Strix Halo from 1 to 24 simultaneous requests, and the engine now grows with the load.
1bit.MONSTER is now 1bit engine
1bit.MONSTER grew fast: an inference engine, a model zoo, benchmarks, a store, a voice assistant and a lot of experiments, all in one repository with all of its history. The engine that came out of it is now its own project, 1bit engine, and this site is its home.
Prefill on HRX, decode on Vulkan, one KV cache
On Strix Halo's Radeon iGPU, AMD's HRX runtime prefills a prompt faster than Vulkan, and Vulkan decodes faster than HRX. 1bit serve --device vulkan --prefill-device hrx now uses both on the same request, and the key-value cache never gets copied between them.
HRX answers long prompts correctly, and where Unsloth quants pay
Two results from the end of the day: a bug fixed in HRX decoding, and one table that answers "which Unsloth file should I download?" for five popular models.
The first week of 1bit engine
The clean repository started on 2026-09-22. In its first three days, the engine came up on every device it targets, inside Lemonade, with each step verified on real hardware before it merged. These are the milestones, with the numbers and where they come from.
Archive · 1bit.MONSTER
From the 1bit.MONSTER years.
The posts from before the clean repository, as they were written. They describe the old monorepo; the engine's current numbers are on the wiki.