JSPARK3 v2

Tempo

DeepSeek-V4.1 Flash
on three DGX Sparks.

Our current daily driver, using EXL3 experts and vLLM with changes to prompt reuse, prefill scheduling, and Engram reads.

Three DGX Sparks · one endpointRank 0 serves the API; all three run inference.

Release: v2.0.0 · Experimental
Hardware: 3 × NVIDIA DGX Spark (GB10, 128 GB each)
Serving: EXL3 experts · vLLM · TP3

Measured on our three Sparks.

L5-P measurements, September 2026. Different workloads answer different questions.

Time to first token

An uncached prompt and its exact repeat.

64K tokens76K tokens

Uncached prompt42.623 s

Exact repeat0.708 s

Three-trial medians, one request at a time. Client request start to first nonempty streamed output.

Exact repeat reuses the same prefix with server caches intact. Uncached means a new prompt, with kernels already warm.

Uncached prompt49.659 s

Exact repeat0.471 s

Three-trial medians, one request at a time. Client request start to first nonempty streamed output.

Exact repeat reuses the same prefix with server caches intact. Uncached means a new prompt, with kernels already warm.

Three-minute agent tasks

The repeated Work round.

3 agents6 agents

81.98 tok/s

Aggregate output over the full task window

One three-minute task round; later tool trajectories diverge; no output-quality score.

Same initial payload repeated in fresh Pi sessions, with caches intact. Server-wide accepted tokens ÷ 180 seconds, including unfinished output, health probes, and small launch/drain overhead. Active cap: 8.

110.49 tok/s

Aggregate output over the full task window

One three-minute task round; later tool trajectories diverge; no output-quality score.

Full benchmark report and methods ↗Release measurement data ↗

Prose: 33.21–34.28 tok/s
Code: 67.31–75.64 tok/s

Single stream, after first output. Ranges across 3 prose and 4 code tasks; one answer per task. (Completion tokens − 1) ÷ (HTTP duration − TTFT), including transport, finalization, and speculative chunks. Not GPU-only decode.

146.12 tok/s in the separate six-stream short-answer test. Eight categories, 150–256-token caps, one wave per category; transport fixtures, not answer-quality grades. Mean of eight category wave rates, including initial wait.

What changed

Prompt reuse: Retain reusable prefixes at valid state boundaries, including changed suffixes and tool continuations.
Prefill scheduling: Keep large chunks for a request running alone; share a smaller budget when another stream is generating.
Engram reads: Read packed weights and scales together. The preparation preserves the values and quantization.

Run Tempo

Three DGX Sparks, a working RoCE fabric, and local storage for the pinned model, experts, draft, and prepared Engram files. The install guide starts with a fit check and configuration worksheet.

Install v2.0.0 ↗

Known limitations

Experimental serving recipe; historical measurements come from one three-Spark fleet.
Functional code passed 2/4 tasks; this is not a broad quality ranking.
A short request arriving two seconds into a 64K prefill waited 40.458 seconds.
300,000 tokens is configured context, not certified usable context or C8 capacity.
Uncached prompt measurements use warmed kernels; they are not cold machine starts.
Work counts unfinished output and does not score quality. Mia active cap 4 differs from Tempo cap 8.
Native vision and optional Pi desktop use are separate from the throughput cohort.

Release, evidence, and credits

v2.0.0 · experimental. JSPARK3 names our current three-Spark daily driver, chosen through internal benchmarks and use. It does not claim to win every benchmark.

DeepSeek created the model. Tony and Kai’s serving work, bot-lab-21’s EXL3 experts, and vLLM underpin this recipe. Code, weights, and drafts retain their upstream terms. This is serving engineering, with no new fine-tune or model merge.

Versioned release · GitHub source · Hugging Face recipe · Release evidence

Generated release summary · Previous driver: GLM-5.3 Flash / Cadence