---
title: "haloq38flash: qwen3.8-flash-next on strix halo"
url: https://stacklist.com/card/f7bdfd8c-bce0-4334-a2bc-4645991520af
source_url: "https://github.com/julianmb/haloq38flash"
stack: https://stacklist.com/c/technology/stack/cc483fc9-ce97-46ea-bc49-7a300af7cd39
summary: "HaloQ38 Flash Tool is a firmware management system for the Qwen3.8-Flash-Next model that includes optimizations like 4-bit quantization, Vulkan kernel tuning, and speculative decoding. The tool fixes a converter bug affecting hyper-connection normalization and provides multiple quantized versions (91GB and 116GB) optimized for different context depths and use cases."
tags: "firmware-management, quantization, qwen-model, vulkan-optimization, speculative-decoding, gguf-format, performance-tuning"
key_entities: "Qwen3.8-Flash-Next (technology), Vulkan (technology), GGUF (technology), AMD Strix Halo (technology), speculative-decoding (concept), quantization (concept), julianmb (person), HuggingFace (organization)"
classification: "analysis"
content_hash: "sha256:2c9b0bb594c144af8777485f715186472531acc348a4140ca36cd18732ec36b3"
acp_version: "0.2"
token_counts_approximate: 1533
visibility: public
agent_accessible: true
status: "final"
---

# haloq38flash: qwen3.8-flash-next on strix halo

haloq38flash qwen3.8-flash-next (125b-a6b) on amd strix halo — 56 tok/s mtp, 262k context, 91g provenance-verified quant converter bug found + fixed · vulkan fa/mmq kernel tuning · greedy-oracle validated speculative decoding · 51b n-gram table cut to 4 bits · ssd streaming to 262k every published quant byte-traced back to the official checkpoint plain english — what was done qwen3.8-flash-next is the new qwen model that is great for coding — it beats claude opus 4.6 on swe-bench and runs on a $2500 mini pc. the official conversion code missed a step — every hyper-connection norm was off by exactly 1.0, so the first quant printed garbage. we found it, fixed the converter, and added a test so it never happens again. we made the model smaller without losing quality — the big 51b n-gram table tolerates 4-bit, saving 27g — then proved it across context depths from 0 to 256k. we kept everything that makes strix halo fast — vulkan kernels, graph reuse, speculative decoding with the 4b draft head. if you just want to run it: docker compose up --build and open http://localhost:8080 . pick the 91g file for speed, the 116g file if you need 262k context. results 91g quant · vulkan/radv · mtp sidecar · q8_0 kv · -ub 2048 · temp 0 · 128g strix halo depth plain pp / tg mtp pp / tg 0 92.5 / 29.9 87.0 / 53.1 8k 480 / 24.1 458 / 56.4 32k 397 / 20.1 379 / 30.2 128k 222 / 11.0 214 / 18.6 256k 139 / 6.2 — Note no collapse through 32k. the 128k+ falloff is context-mechanics (sparse-attention indexer), not quant size — see the reversal below. the 128k reversal — the PLE quant loses under MTP at depth at ≤32k the PLE quant wins everywhere. at 128k under mtp it loses to the static 116g (18.6 vs 26.9 t/s). plausible mechanism: iq4_nl noise in the n-gram table compounds over deep history and lowers draft acceptance. single runs, n=1 caveat. pick your file by use case — see the table above. 📦 published quants huggingface.co/julianmb/Qwen3.8-Flash-Next-IQ4_XS-GGUF file size pick it when ...-IQ4_XS- PLE .gguf 91 giB ctx ≤ 32k — wins everywhere, mtp to 56 t/s ...-IQ4_XS.gguf 116 giB ctx ≥ 128k — faster mtp at depth, wider fork compat mtp-...-Q8_0.gguf 3.9 giB mtp sidecar, required for the speed numbers the PLE cut — why the 51b n-gram table tolerates 4-bit the PLE table is gathered 16 random rows per token via hash lookup — there is no matmul on the table itself, and no two consecutive tokens hit the same rows. the rows tolerate iq4_nl (4.25 bpw) with no measurable degradation across the depth sweep. the cut: --tensor-type "per_layer_token_embd=IQ4_XS" on our quantizer → 54g → 27g. fork caveat: engines that feed gathered PLE rows straight into mul_mat as quantized B operands assert (ggml-vulkan.cpp:7794). verified working on the packaged engine and rocmfpx. pick your setup your use case quant ctx expect coding agents, chat 91g PLE ≤ 32k 56 t/s long documents 116g static 128k 27 t/s full rag / research 116g static + ssd streaming 262k 14 t/s 🐛 the converter bug our first quant printed deterministic garbage at temp 0. bisect to root cause: experts, gdn reorder, ple scale, metadata: all innocent 97 of 388 f32 tensors differed by exactly 1.0 — every hyper-connection norm shipped raw where the runtime expects raw + 1 cause: the checkpoint nests hyper-connections under attn_hyper_connection / mlp_hyper_connection / hyper_connection_mixer , and those names hit early-return branches in the converter that bypass the generic norm.weight → +1 rule Warning any fork rolling its own qwen4exp converter must fold (1 + w) into the hyper-connection gammas. upstream runtime documents the contract at qwen4exp.cpp:231 — "the converter folded each gamma to (1 + w)" . miss it and every layer normalizes wrong — garbage from layer 0, all shapes correct, all shape-only tests pass. fix + regression test: rocmfpx port-qwen4exp commit 61b6a3b48 ( pr charlie12345/ROCmFPX#98 ) 🐳 docker git clone https://github.com/julianmb/haloq38flash && cd haloq38flash docker compose up --build # serve on :8080 — vulkan/radv, no rocm install needed the packaged engine is tuned for strix halo: vulkan fa/mmq kernels, graph reuse, lazy ple streaming, quantized-kv attention — the combination behind the 56 t/s numbers. no manual build, no host rocm install. add the mtp sidecar for speculative decoding: docker compose run qwen38-flash-next /app/llama-server \ -md /models/mtp-Qwen3.8-Flash-Next-Q8_0.gguf \ --spec-type draft-mtp --spec-draft-n-max 6 --spec-draft-p-min 0.75 262k context (ssd streaming) swap the model to the static 116g and enable lazy ple — the n-gram table stays on ssd (~2.5g resident), leaving room for the full context window: docker compose run qwen38-flash-next /app/llama-server \ -m /models/Qwen3.8-Flash-Next-IQ4_XS.gguf \ -c 262144 -lm mmap --tensor-read-lazy on \ -ngl 999 -fa on -ctk q8_0 -ctv q8_0 -ub 2048 -t 4 🔬 the n-gram table at 4-bit — what we found the 51b ple lookup table tolerates iq4_nl (4.25 bpw) with no quality loss across the depth sweep. but there's a depth-dependent reversal: under mtp at 128k+, the ple quant loses to the static quant (18.6 vs 26.9 t/s) — the iq4_nl noise compounds over deep n-gram history and lowers draft acceptance. n=1, single runs. pick your file by use case. 📁 layout path what models/ symlink farm to local ssd (never in git) docs/ engine merge plan, cherry-pick classification Dockerfile two-stage: vulkan engine build + slim runtime docker-compose.yml one-liner serving with recommended flags ⚠️ operational gotchas (128g strix halo) always -c 8192 -bounded ctx + timeout + /usr/bin/time -v — the gguf default 262144 + full offload hard-hung this box once vm.dirty_ratio=15 / dirty_background_ratio=5 — the 191g ple conversion memmap wedges balance_dirty_pages for hours at kernel defaults conversion peak: ple scratch (191g) + f16 output (354g) coexist — budget ~560g free pkill -x llama-cli , never -f (matches your own wrapper shell) gpu memory is shared with everything else on the apu — two engines cannot hold ~90g+ models simultaneously without an oom cascade license: qwen community license 1.0 · base model: Qwen/Qwen3.8-Flash-Next