Tencent Hunyuan Hy4-preview Runs in vLLM from Day 0 with Novel MoE Design

Key Info

Tencent Hunyuan's Hy4-preview is verified running in vLLM from day 0 on NVIDIA GPUs, with HPC-Ops kernels already in vLLM main since Hy3. The model uses a 770B-parameter MoE with 49B active, 256 routed experts plus one shared, and a 1M context window where each query attends to only 2048 tokens.

Highlights

• 770B total parameters, 49B active, with 256 routed experts plus one shared expert.
• 1M context length, but each query only attends to 2048 tokens via sparse attention.
• Only 21 of 78 layers compute their own sparse index; the other 57 reuse one, cutting overhead.
• A 10B MTP layer is included in the checkpoint (0.7B active, draft depth 3), and HPC-Ops kernels are enabled via VLLM_ENABLE_HPC_OPS=1.