Running Qwen 3.8 locally means this for me: Qwen3.8-27B as an NVFP4 checkpoint under SGLang on a Nvidia DGX Spark, measured at 50.7 tokens per second decoding, with a terminal agent called Pi in front of it doing the actual work. For model inference that means no call to a vendor API and no API key. Tools the agent uses alongside it, a web search or an MCP adapter for instance, still talk to the outside unless you switch them off. On top of that comes a setup with a few pitfalls I'd rather describe once than have you find them yourself.
I run my business with an AI workforce of eight AI employees built on Claude, that's my production environment. The DGX Spark next to it is my test bed for a question I want to answer honestly for clients: how far does a local model get you without the cloud? Everything I write about local AI is collected on the AI page.
What Qwen 3.8 is: variants, license, context
Qwen 3.8 is still young, and the sources on it are thin accordingly. According to yottalabs.ai and atomic.chat, the model launched on August 13 or 14, 2026.
There are several sizes: a 27B dense model, a mixture-of-experts variant called 2.4T-A95B, and a Qwen3.8 Max; atomic.chat also names a "Flash Next" variant. The sources disagree slightly on the parameter count of the 27B version: codersera.com and atomic.chat write 27 billion, yottalabs.ai says 28 billion. Nobody has resolved that discrepancy so far, I'm mentioning it anyway so you don't stumble over it yourself.
The license is Apache 2.0, meaning it is free to use commercially too; the official model card states it that way. Context sits at 262,144 tokens natively according to codersera.com and atomic.chat, extendable to one million tokens via YaRN. That matches what my own server reports: 262k context. The model card, cited by atomic.chat and unsloth.ai, lists 61.7 percent on SWE-bench Pro, 73.0 on Terminal Bench 2.1, and 89.2 percent on GPQA Diamond, figures I haven't independently verified and pass on only as the vendor's own claim.
My setup: Qwen 3.8 locally under SGLang on the DGX Spark
My hardware is a Nvidia DGX Spark with a GB10 chip and 128 GB of unified memory, of which the system reports roughly 119 GB as usable, reachable inside my Tailnet, meaning I can reach it from any of my devices over a private VPN. Next to it sits a MacBook Pro M3 Max with 64 GB, more on that later.
The model running is Qwen3.8-27B, dense, with 262k context, tool calling, thinking, and vision, as an NVFP4 checkpoint at 21.9 GB. The setup follows the upstream repository hasso5703/dgx-spark-qwen38, which has been the official SGLang cookbook recipe for this model since August 21, 2026. For higher throughput, speculative decoding runs alongside with a DFlash2 draft model. Externally, the server speaks both an OpenAI-compatible and an Anthropic-compatible API, and that was my main reason for choosing SGLang: tools that otherwise talk to Claude or an OpenAI-compatible endpoint run against my local server without a single code change.
The server doesn't run as a classic system service but as a systemd user unit with linger enabled, because I don't have passwordless sudo on the Spark, which rules out root-level services as a practical option. As a comparison to the SGLang route, I also tested vLLM: 24 to 26 tokens per second with MTP, noticeably slower than my SGLang setup. Ollama is installed on the machine too, but with no models loaded, it's there in case I want to try something else quickly.
Measured speed: tokens per second
The number that matters to me: 50.7 tokens per second decoding, measured over 300 tokens, one stream, called from the Mac over Tailscale. Prefill, meaning reading in the prompt, runs at roughly 1,500 tokens per second, which keeps the first response token quick to reach even at long context lengths.
For comparison: in agentic coding, meaning multi-step tasks with tool calls, the upstream repo lists 32 to 40 tokens per second for the same combination, that's not a measurement of mine but the documented reference value for that use case. The gap to my 50.7 tokens per second comes down to the task: pure decoding of a single stream runs faster than an agent run with tool responses, intermediate steps, and fresh context on every turn.
Pitfalls when setting it up
The most important rule first: --mem-fraction-static must never exceed 0.50, or the whole host can freeze, not just the inference process. I've kept it at 0.50 ever since and haven't had a crash. Why that is particularly touchy with the DGX Spark's unified memory is covered in detail in DGX Spark Tested: Worth Buying?.
Second pitfall: I don't have passwordless sudo on the machine, so root-level systemd services are off the table. The fix is a systemd user unit with linger enabled, so the service keeps running without an active login session. Third, the internet connection at my location: roughly 2 MB/s, which means a 70 GB download for model weights takes several hours, plan for that before you decide to pull a new model in the evening. Fourth, the first start: because of compilation it takes around 9 minutes, every start after that 5 to 7 minutes, that's normal, not a bug. And fifth: Docker has a 100 GB storage cap on the device, and with several large checkpoints you hit that limit faster than you'd think.
What it's good for, and what it isn't
My goal with this setup is a cloud-free Claude Code alternative for clients where local processing is a hard requirement. To test that, I run the terminal coding agent Pi (pi.dev) against the local model and keep comparing it with Claude Code, which is what my own AI workforce runs on. I don't have a final verdict yet, that's deliberately an ongoing test for me, not a finished result I want to sell you here.
What already works well alongside it: Whisper large-v3 runs in parallel on the same GPU for speech recognition, at 3 to 4 GB it's a small neighbor to the large language model. That shows the Spark still has room for smaller models next to Qwen3.8. What I can't honestly claim yet: that Qwen3.8 under SGLang already matches Claude Code in every situation for me. For tasks with a lot of context, complex multi-file changes, or unclear requirements, Claude stays ahead for me, which is also why the workforce runs productively on it.
Comparison to a Mac: DGX Spark against MacBook Pro M3 Max
Next to the DGX Spark I run a MacBook Pro M3 Max with 64 GB RAM, where I run Qwen3.8 through LM Studio in 8-bit quantization, taking up 27.5 GiB of RAM there. I measured 11 tokens per second decoding on it, clearly slower than on the Spark.
| Criterion |
DGX Spark (SGLang) |
MacBook Pro M3 Max (LM Studio) |
| Memory |
128 GB unified memory nominal, roughly 119 GB usable per the system |
64 GB RAM |
| Quantization |
NVFP4, 21.9 GB checkpoint |
8-bit, 27.5 GiB RAM used |
| Tokens per second, decode |
50.7 |
11 |
| Reachability |
inside the Tailnet, from any device |
local on the device |
For me that means: the Spark is the clear choice when several devices need to hit the same model and speed matters. The Mac still makes sense when I'm away from the Tailnet, even though I then need to expect roughly a fifth of the speed.
Frequently asked questions
Does Qwen3.8-27B run with less than 128 GB of memory?
Yes. According to the sources I read, a 4-bit quantization needs 16 to 19 GB, a Q4_K_M variant around 17.1 GB. My own NVFP4 checkpoint sits at 21.9 GB, which runs on GPUs far smaller than the memory in my DGX Spark.
How fast is Qwen 3.8 on the DGX Spark?
Measured on my machine: 50.7 tokens per second decoding a single stream, roughly 1,500 tokens per second on prefill. The upstream repo lists 32 to 40 tokens per second for agentic coding with tool calls as a reference value, which is a different task than pure decoding.
What license does Qwen 3.8 use?
Apache 2.0, meaning it is usable for commercial projects too. The license is stated that way on the model's official model card.
Do I need SGLang, or is Ollama enough for Qwen 3.8?
Ollama is installed on my machine, but with no models loaded. For my setup with an OpenAI- and Anthropic-compatible API plus speculative decoding, SGLang was the better fit. As an alternative I tested vLLM, where I got 24 to 26 tokens per second with MTP, noticeably below my SGLang number.
Can Qwen 3.8 replace Claude Code?
Not fully, not yet, for me. I run the terminal agent Pi against Qwen3.8 and keep comparing it with Claude Code, which powers my AI workforce. For simple, clearly defined tasks it's often enough, on complex multi-file changes Claude stays ahead for me.
Where this goes next
The next logical step from this setup is a direct comparison of the same tasks in Pi against Qwen3.8 and in Claude Code, which I cover in Qwen vs Claude: Local Versus Cloud. If the hardware itself interests you more than the model, my experience with the device is in DGX Spark Tested: Worth Buying?. For a broader look at local models beyond Qwen, see Local AI: What Actually Works in 2026, and if you're asking the same question from a Kubernetes angle, the answer is in Kubernetes for AI: LLMs on GPU Nodes.
My suggestion if you want to start yourself: measure your own baseline with a small prompt first, before you run larger contexts or agent workflows against it. The setup recipe I followed is open in the hasso5703/dgx-spark-qwen38 repository, the SGLang server itself in the SGLang project on GitHub.
