Thanks to everyone who tried Qwen3.8‑Flash‑Next here! In 54 hours the endpoint served ~326,000 requests (and politely turned away ~565,000 more from a few over-enthusiastic scripts), processing over 8 billion tokens on up to 2 replicas × 4 H200s. The chat UI is paused too. Want your own, always‑on copy? Deploy it yourself on Hugging Face Inference Endpoints (4× H200 at $20/hr) with autoscale‑to‑zero, so you only pay while it's actually running.
A free, OpenAI-compatible endpoint for Qwen's preview of the Qwen4 architecture — a 125B ultra-sparse MoE with only 6B active parameters, vision input, tool calling, 262,144-token context, and dialable reasoning. No API key, no signup.
A free, public Hugging Face Inference Endpoint for Qwen/Qwen3.8-Flash-Next — an OpenAI-compatible Chat Completions API. No Hugging Face token required: point any OpenAI SDK, curl, or agent framework at the base URL and go.
Released August 26 as "an early preview of the architecture used in Qwen4": hybrid Gated DeltaNet + Qwen Sparse Attention, gated residuals, and a 51B n-gram embedding memory. It beats Qwen3.8-27B and the 397B Qwen3.7-Plus on most benchmarks with a third of the active parameters and about a ninth of the training compute — SWE-bench Pro 62.5, SWE-bench Multilingual 81.0, LiveCodeBench v6 91.9, AndroidWorld 84.5. Served from the official FP8 checkpoint on four H200s.
Shared, free, and rate-limited to be fair to everyone: per IP, about
10 requests in a burst refilling to roughly 15 requests/minute,
and at most 4 requests in flight at once. A normal agent loop is fine —
fanning out dozens of parallel calls from a script is not, and sources that keep
slamming the limits get paused for 10 minutes. Over a limit you'll get a
429 with a friendly note and a Retry-After header.
Requests that don't set max_tokens get a default of 8,192 here.
It will be retired after the launch buzz cools down.
One base URL, standard OpenAI API, model id Qwen/Qwen3.8-Flash-Next.
Any string works as the API key.
https://pnywsahxhac1qjbo.us-east-2.aws.endpoints.huggingface.cloud/v1
curl https://pnywsahxhac1qjbo.us-east-2.aws.endpoints.huggingface.cloud/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "Qwen/Qwen3.8-Flash-Next",
"messages": [{"role": "user", "content": "Explain sparse attention in one paragraph."}]
}'
from openai import OpenAI
client = OpenAI(
base_url="https://pnywsahxhac1qjbo.us-east-2.aws.endpoints.huggingface.cloud/v1",
api_key="none", # no key required
)
r = client.chat.completions.create(
model="Qwen/Qwen3.8-Flash-Next",
messages=[{"role": "user", "content": "Three fun facts about lighthouses?"}],
temperature=1.0,
top_p=0.95,
)
print(r.choices[0].message.reasoning_content) # the thinking trace
print(r.choices[0].message.content) # the final answer
Streaming, tool calling (tools=[...]), and
multi-turn tool round trips all work as in the OpenAI API. Recommended sampling
with thinking on: temperature=1.0, top_p=0.95, top_k=20.
There was a full chat-ui front end running on this same endpoint — thinking, vision, and tool calls all wired up. It is paused now that the endpoint has retired; the code is one click away if you deploy your own.
Thinking is on by default. The reasoning trace comes back separately in
message.reasoning_content; your message.content stays clean.
Dial it with the standard reasoning_effort field. The model knows three
tiers — xhigh, medium, low — and
none turns thinking off entirely. OpenAI-style values like
high or minimal are accepted and mapped to the nearest
tier. On this shared instance an unset effort means medium (the model's
own default is xhigh, which thinks for thousands of tokens per request);
ask for xhigh explicitly when you want the deepest reasoning.
| Mode | temperature | top_p | top_k | presence_penalty |
|---|---|---|---|---|
| Thinking (default) | 1.0 | 0.95 | 20 | 0.0 |
Non-thinking (reasoning_effort="none") | 0.7 | 0.80 | 20 | 1.5 |
Native vision: send images the standard OpenAI way, up to 4 per request (video input is disabled on this shared instance).
r = client.chat.completions.create(
model="Qwen/Qwen3.8-Flash-Next",
messages=[{"role": "user", "content": [
{"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}},
{"type": "text", "text": "What is in this image?"},
]}],
)
print(r.choices[0].message.content)
Multi-turn note: prior turns' thinking is preserved in context
by default (preserve_thinking), which helps agentic flows but eats
context in long conversations. Pass
extra_body={"chat_template_kwargs": {"preserve_thinking": False}}
for classic strip-old-thinking behavior.
pi reads
custom model providers from ~/.pi/agent/models.json. Add a provider entry
pointing at this endpoint's OpenAI-compatible base URL, then select it from the CLI.
{
"providers": {
"hf-qwen38-flash": {
"name": "Qwen3.8-Flash-Next (HF public)",
"baseUrl": "https://pnywsahxhac1qjbo.us-east-2.aws.endpoints.huggingface.cloud/v1",
"api": "openai-completions",
"apiKey": "not-needed",
"compat": {
"supportsReasoningEffort": true,
"maxTokensField": "max_tokens"
},
"models": [{
"id": "Qwen/Qwen3.8-Flash-Next",
"name": "Qwen3.8-Flash-Next",
"reasoning": true,
"thinkingLevelMap": {
"off": "none",
"minimal": "low",
"low": "low",
"medium": "medium",
"high": "xhigh",
"xhigh": "xhigh"
},
"input": ["text", "image"],
"contextWindow": 262144,
"maxTokens": 32768,
"cost": {"input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0}
}]
}
}
}
Then run it — no restart needed, pi picks up models.json on launch:
# interactive, thinking level high (maps to xhigh on the endpoint) pi --provider hf-qwen38-flash --model Qwen/Qwen3.8-Flash-Next --thinking high # one-shot, non-interactive, thinking off pi -p --no-session --provider hf-qwen38-flash \ --model Qwen/Qwen3.8-Flash-Next --thinking off \ "Summarize this repo's README." # add it to your Ctrl+P model cycle alongside others pi --models "hf-qwen38-flash/*,sonnet,haiku"
Same shape works for any OpenAI-compatible server — swap
baseUrl for a local vllm serve or sglang serve
instance and keep the rest.
| Item | Value |
|---|---|
| Model | Qwen/Qwen3.8-Flash-Next · Qwen Community 1.0 license · official FP8 checkpoint (Qwen3.8-Flash-Next-FP8, experts block-quantized, everything else BF16) |
| Parameters | 125B main model, 6B active per token, plus a 51B n-gram embedding table and a 4B MTP head (180B on disk) |
| Architecture | 48 layers, 3:1 Gated DeltaNet linear attention : Qwen Sparse Attention (2,048-token indexer budget) · 512 experts, 10 routed + 1 shared · 4-branch gated residual · n-gram embeddings injected at layer 2 |
| Context window | 262,144 tokens (native, full window served; the 1M YaRN mode is not enabled) |
| Modalities | Text + images in (max 4/request), text out · video disabled |
| Hardware | 4× NVIDIA H200 (564 GB), one replica |
| Engine | SGLang (lmsysorg/sglang:qwen38flashnext, day-0 build) · tensor parallel 4 · no speculative decoding yet |
| Measured | ~0.8 s first token · 130-150 tok/s per stream idle · 80-105 tok/s per stream at 16 concurrent (1.1-1.4k tok/s aggregate) · 62k-token prompt verified |
| Rate limit | ~15 requests/min per IP (burst 10), max 4 in flight, ~600 KB request cap, default max_tokens 8,192 · 429/413 + Retry-After when exceeded |
Measured numbers are from this deployment's own
verification battery, not marketing. Unset reasoning_effort is
served as medium here; benchmark with an explicit tier.
This whole thing — four H200s, a day-0 SGLang build, rate limiting — is one deploy form on Inference Endpoints: dedicated, production-grade deployments of any model on the Hub.
Pick a model, pick your hardware (CPU to multi-GPU H200), and get an OpenAI-compatible URL with autoscaling, scale-to-zero, and per-minute billing. No shared queues, no rate limits from strangers — your model, your GPUs.