JSpark3
Three DGX Sparks, one model server.
I had three DGX Sparks and every recipe out there was written for two or four. JSpark3 is the setup that makes three work: GLM-5.3 Flash running across all of them as one endpoint, about 1.4x faster than the two-box recipe. Every piece is pinned so you can rebuild it exactly, and the numbers are published with the misses left in.
- 3
- DGX Sparks serving as one endpoint, TP3 + EP3
- 4.46 GiB
- weight memory freed across the cluster: 1,595,392,320 bytes per rank
- 1,000,000
- tokens of configured context, FP8 KV cache, prefix caching
- 120 shards
- of the pinned target checkpoint, each verified by hash before a token is served
Scope
What it is, and what it is not
It is
- A reproducible serving and runtime recipe: 39 files an operator copies to three Sparks.
- A fail-closed lifecycle controller. Preflight, start, health, verify, stop, each with a dry run.
- A measured operating envelope: 32 sequences, 8,192 batched tokens, CUDA graphs at 8/16/24/32/48, GPU memory utilization 0.83.
- A selective W8A16 Marlin overlay for the BF16 trunk, applied at load, with the routed experts left in EXL3.
- Machine-readable evidence with receipts, including the regressions and the two internal gates it missed.
- A public comparison table: author-reported figures for the recipes that came before it, with the fields needed to read them.
It is not
- A new model, a fine-tune, or a new quantization. Nothing was trained or quantized here.
- A new checkpoint. The weights are Brandon Music's EXL3/TR3 4-bpw quantization as re-hosted by Mia-AiLab, pinned by revision and hash. The Hugging Face repository re-hosts that exact revision byte for byte, under its own license, so nobody assembles the checkpoint by hand.
- A patched vLLM. Five hash-gated transforms are applied inside the pinned container at start and verified by hash.
- Unrestricted open source or commercial-ready as an assembled stack. The draft is non-commercial and the checkpoint is attribution-required.
- Independently reproduced yet. Evidence comes from one project-operated fleet.
Architecture
How three Sparks become one endpoint
Each Spark holds one tensor-parallel shard and one third of the routed experts. The three fabric legs form a triangle so every rank reaches each peer directly.
Topology
Tensor parallel 3 and expert parallel 3. Every rank holds a shard of attention, KDA, the shared expert and the LM head, plus 96 of the 288 routed experts. The DFlash2 draft is built over all three ranks; the profile's draft TP 1 setting is ignored by this loader, so the diagram shows what is actually loaded.
Fabric
Three RoCE-v2 legs, each on its own network at MTU 9000. Every Spark owns two fabric interfaces. Preflight checks the GID, MTU and routes on each rank before anything starts.
Overlay
The BF16 trunk is converted to INT8 Marlin at load: 169 modules, 225 tensors, group ladder 128/64/32. The 34 KDA f/g modules are excluded. It frees 1,595,392,320 bytes per rank.
Lifecycle
Start order 2, then 1, then 0, bound to a hash-checked release manifest. The entrypoint refuses on cgroup, NCCL, overlay or loader drift. Verify confirms shards, graphs and a focused witness.
Evidence
Measured, and compared with what you could already get
On their own benchmarks
The authors' pinned benchmark scripts, run unchanged against the JSpark3 release build except for the endpoint address and model name. Author values are the numbers they published.
FlyCockpit's benchmark, the same three Sparks
Decode, tok/s (mean of three runs)
Hello (17-token stop)
1.23xratio
runs 46.106 / 46.405 / 45.820
runs 37.9 / 36.9 / 37.3
Structured count 1 to 200
1.21xratio
runs 83.988 / 84.580 / 84.161
runs 69.0 / 68.5 / 71.2
is_prime code
1.18xratio
runs 65.840 / 68.246 / 65.544
runs 52.3 / 58.7 / 58.2
Draft acceptance 0.8324 against FlyCockpit's 0.815. FlyCockpit's runs were first-serve at GPU memory utilization 0.87; these were warm-server runs at 0.83, the release envelope.
Mia's bench_decode, their two Sparks against our three
Median of five 400-token runs, tok/s
Structured count 1 to 200
1.35xratio
Prose hash-map
1.36xratio
Accepted-per-draft ratios match theirs almost exactly: 0.9533 against 0.959 on structured, 0.3357 against 0.341 on prose. The draft behaves the same; the engine is faster.
Mia's sparkDash decode protocol
| Concurrency | Estimator | Mia TP2, 2 Sparks | JSpark3, structured count3 Sparks | JSpark3, clamp code3 Sparks |
|---|---|---|---|---|
| C1 | per-stream decode tok/s | 62.9 | 86.56 | 84.47 |
| C1 | time to first token | 719 ms | 461 ms | 391 ms |
| C2 | per-stream decode tok/s | 51.7 | 79.26 | 70.56 |
| C2 | aggregate decode tok/s | 103.3 | 76.95 | 139.77 |
| C4 | per-stream decode tok/s | 37.1 | 50.08 | 62.80 |
| C4 | aggregate decode tok/s | 146.5 | 200.29 | 251.13 |
Scroll sideways for the full table.
Mia publishes one value for its high-accept prompt family without saying which prompt, so both of ours are shown.
At two streams the structured aggregate falls below Mia's published figure: the release build serializes low-concurrency work below its eight-sequence graph floor, the same artifact the internal ablation shows at three streams.
Measured on 2026-09-02 against the release build held immutable, three Sparks, warm server. FlyCockpit T0 script at commit 9093765c; Mia tests/bench_decode.py at commit c190db1a; sparkDash release 1.8.5 at commit e93fc87d.
Same screen, two Sparks against three
Single-stream decode, tok/s
Structured count
1.41xratio
Code
1.49xratio
Prose
1.45xratio
Same frozen 24-request screen on the same fleet, thinking off, temperature 0, 400 max tokens. The Mia recipe is the current two-Spark release at commit c190db1a, run here with one compatibility repair.
Same task, same prompt
Aggregate decode, tok/s
One agent prompt, independent runs; each agent chose its own path.
Local runs of published recipes
Two Mia TP2 lineages and one FlyCockpit-derived build were run on this fleet with disclosed adaptations, and each was given the same agent prompt as JSpark3. Same prompt, independent trajectories: product evidence, not an engine-rate comparison.
JSpark3 v1
44.583 tok/s aggregate decode over 132 requests and 32,618 generated tokens; mean time to first token 3.704 s.
Mia TP2, historical recipe
Commit 0e2e78f, run locally with site, storage, API, and safety adaptations. 24.913 tok/s aggregate decode over 43 requests and 105,198 generated tokens.
Mia TP2, current recipe
Commit c190db1a, runnable at full context only with the GLM53_INDEXER_WORKSPACE=rightsize repair: an adapted reproduction, not an exact one. 24.728 tok/s aggregate decode over 24 requests and 76,540 generated tokens.
FlyCockpit-derived build
Commit 9093765c with minimal correctness and safety adaptations, not the literal upstream launcher. 29.042 tok/s aggregate decode over 58 requests and 130,971 generated tokens.
Published reference recipes
The public comparison is against recipes that were publicly available before JSpark3. Their numbers are author-reported; ours come from our own fleet. Prompts, instruments, and envelopes differ, and the node count is on every row, so read the table as context, not a ranking. No percentage is computed between rows.
| Recipe | Sparks | Lane | Context | Single-stream decode, tok/s | Basis |
|---|---|---|---|---|---|
| JSpark3 v1 | 3 | EXL3/TR3 4-bpw · DFlash2 k=7 · W8A16 trunk overlay · vLLM, TP3/EP3 over a RoCE-v2 triangle | 1,000,000 | structured count 81.962 · code 66.257 · prose 29.049 | Local. Frozen 24-request screen, thinking off, temperature 0, 400 max tokens, warm server; medians of three batteries; per-stream estimator |
| FlyCockpit TP3 | 3 | EXL3/TR3 4-bpw, same target revision · DFlash2 k=7 · vLLM, TP3/EP3 over a mesh | 1,000,000 | structured count 69.0 / 68.5 / 71.2 · code 52.3 / 58.7 / 58.2 · "hello" 37.9 / 36.9 / 37.3 | Author-reported first-serve runs at commit 9093765c; thinking off, temperature 0, 200-token stops (17 for "hello"); GPU memory utilization 0.87 with cgroup swap recorded; no prose row |
| Mia TP2 | 2 | EXL3/TR3 4-bpw, same target revision · DFlash2 k=7 · vLLM, TP2 | 1,000,000 | 62.9 on high-accept prompts (sparkDash, single stream) · structured 65.1 / prose 27.1 (bench_decode, four streams, median of 5×400) | Author-reported at commit c190db1a; two instruments of their own, neither is our screen; two Sparks, not three |
| jetnet TP3 | 3 | NVFP4 · MTP-4 or DFlash2 · eager, Marlin W4A16, TP3 | 512K | 35.2 (range 32.1 to 39.0) with MTP-4 · 47.2 with DFlash2, thinking on | Author-reported at commits bfc820ec and 4fdba004 and the author's NVIDIA forum post; clock-capped at 1500 MHz; the model always thinks; a different quantization lane |
Scroll sideways for the full table.
Every row states hardware count, quantization and speculation, context, workload, and source. Sources are pinned commits, listed with full URLs on the repository's benchmarks page.
Internal ablation
What the overlay alone changed
The control is JSpark3 itself with one switch off. Same three Sparks, same pinned checkpoint and DFlash2 draft, same container image, same TP3/EP3 topology and serving envelope, same request sets and estimator. The only change is that the selective W8A16/Marlin trunk overlay is disabled, so the trunk serves in BF16 as it came from upstream. That build is unreleased internal development evidence, not a product and not a market comparison; it exists to isolate what the overlay alone changed.
Every bar below is JSpark3 measured against that control. Two single-stream rows are shown: the campaign medians against the control's earlier battery, and a strict same-day pair (candidate battery r3 against control battery r6). Green went up, red went down, one scale.
Single-stream decode · campaign medians vs the control's earlier battery
Code
+3.75%
Structured count
+5.74%
Prose
+2.62%
Single-stream decode · same-day pair, candidate r3 vs control r6
Code
+7.27%
Structured count
+6.63%
Prose
+8.35%
C6 per-stream median
+2.91%
Matched concurrency waves · aggregate service throughput
C12
+0.16%
C24
+1.21%
C48
+3.47%
Matched long prefill · 113,908 tokens
Prefill proxy
−3.38%
What the overlay improved. Single-stream decode medians: code +3.75%, structured count +5.74%, prose +2.62% against the control's earlier battery; +7.27%, +6.63% and +8.35% in the strict same-day pair. Token pacing: median inter-token interval 98.645 to 91.912 ms (−6.83%), p99 −10.27%, worst interval 364.416 to 148.344 ms (−59.29%). Aggregate throughput at 48 streams +3.47%. 1,595,392,320 bytes of weight memory freed per rank.
What it cost, and what was missed. Long prefill −3.38% with time to first token +3.50% on 113,908 tokens. Fairness did not improve. Time to first token at 48 streams reached a p90 of 96.722 s. Two internal promotion gates were missed: a code median of 66.257 against a 67.0 floor, and a demonstration pacing run of 14 against a limit below 5.
Evidence grade: engineering evidence. One fleet, operated by the project, with no third-party reproduction yet. No correctness, stability, or safety failure was observed in any run. Estimators, sample sizes, receipts, and the three evidence classes are on the repository's benchmarks page.
Reproducibility
It refuses to drift
The construction is exact or the recipe does not start. The measurements are fully specified but the numbers are not guaranteed; your fleet will differ, the bytes will not.
Inputs
Any checkpoint revision, draft revision, image digest, or transform source that is not the pinned one.
Environment
A cgroup other than 64 GiB with swap off; any NCCL_PROTO, NCCL_ALGO, or NCCL_IB_ADDR_RANGE override; overlay or KDA environment drift.
Identity
A preflight row that differs from the expected row, a preflight checksum mismatch, a missing or unbound image receipt, or an existing container with the release name.
Bytes
Overlay, loader-hook, and transform before/after hash drift; a missing or duplicated seam. Every rank validates every serving byte before start.
cp .env.example .env
./scripts/clean-room-setup.sh --env-file .env --output preflight.json
preflight_sha=$(sha256sum preflight.json | cut -d' ' -f1)
./scripts/start.sh --env-file .env \
--preflight preflight.json --preflight-sha256 "$preflight_sha" \
--confirm START-JSPARK3
./scripts/health.sh --env-file .env --manifest jspark3-release-manifest.json
./scripts/verify.sh --env-file .env --manifest jspark3-release-manifest.json \
--output verify.json --log-output verify-rank0.logEvery command has --dry-run. Confirmation tokens are typed, never defaulted. The full nine-step installation, from bare Docker hosts to a verified endpoint, is in the repository's install page.
Provenance
Pinned inputs
Nothing here is a floating tag. The recipe checks each of these before it serves a token.
- Target checkpoint
Mia-AiLab/GLM-5.3-Flash-EXL3-TR3-4bpwat revision25a44fdbf16862a46b7cc9921142c6c81350af2f, itself byte-identical tobrandonmusic/GLM-5.3-Flash-tr3-4bpwat5ab363a8dcf6405955fd5f99671e01a1c9fb124b. The JSpark3 Hugging Face repository re-hosts this revision shard for shard with the same hashes; the preflight accepts either source because the bytes are identical- Draft checkpoint
incoai/GLM-5.3-Flash-DFlash2at revisiondc77ff1c99eeb2df044ee3d4f0094eb033fee410, k=7- Container image
ghcr.io/miaai-lab/glm-5.3-flash-2x-dgx-sparksat digestsha256:9bb1557a4234fce63d59599e44d10747eabd742beb337eebf9e7070be8a0fd58, launched by digest, not redistributed- Serving engine
- vLLM build
487ecf187as shipped inside the pinned image; five hash-gated runtime transforms are applied at start - Transform sources
- FlyCockpit
GLM-5.3-Flash-EXL3-3x-DGX-Sparksat9093765c757bd1976372196e44af84a67cf86bad; vcruz305GLM-5.3-Flash-EXL3-K2-DGX-Spark-recipeat622cb878d66f703c597bd6baaa2423caa1786f99 - Runtime envelope
- Configured context 1,000,000 tokens · 32 sequences · 8,192 batched tokens · GPU memory utilization 0.83 · FP8 KV cache · prefix caching · served as
glm-5.3-flash
Licensing
Three licenses, plainly
The recipe is ours to license. The model bytes it loads are not. Read this before deploying for anything commercial.
Recipe code
Apache-2.0
The scripts, overlays, transforms, tooling, and documentation, with third-party notices. Use, modify, redistribute.
Target checkpoint
ShapleyMcg License 1.0
Source-available and attribution-required; not OSI open source. Brandon M. Music created the EXL3/TR3 checkpoint, Mia-AiLab re-hosts it, and JSpark3 re-hosts that revision in turn under the same license, with the required attribution reproduced verbatim. Downstream copies stay under this license.
DFlash2 draft
CC BY-NC-ND 4.0
Research and evaluation only. Commercial use of the draft requires a license from Inco. Without the draft the recipe still serves, without speculative decoding.
The assembled endpoint is therefore neither unrestricted open source nor commercial-ready. The repository's licensing page lists every term and its practical effect, and the ownership statements the project does not make.
Credits
Built on other people's work
JSpark3 did not train, fine-tune, or quantize anything. The contribution is the three-Spark architecture, the runtime adaptation, the operating envelope, the experimental campaign, and the reproducible serving recipe.
Brandon M. Music for the ShapleyMcg EXL3/TR3 4-bpw checkpoint that every rank loads. MiaAI-Lab for the two-Spark EXL3 recipe, the pinned container image, and the checkpoint re-host that JSpark3 mirrors in turn. FlyCockpit for the three-Spark EXL3 lineage the TP3 overlay transform is reconstructed from. vcruz305 for the K-pool tail correction. tonyd2wild for scheduler and concurrency benchmarking context. sfxnz for DGX Spark serving context. Inco AI for DFlash2. Z.AI for GLM-5.3 Flash. The vLLM project for the engine.
Every upstream author is credited by name, repository, and pinned commit or revision in the repository's third-party notices.
Cite it
The repository ships CITATION.cff and CITATION.bib. Cite the release by name, version, and repository.
JSpark3 v1, version 1.0.0 (2026)
https://github.com/jakejharris/jspark3