The Chinese lab reveals mixture-of-experts details and training compute breakdowns for DeepSeek-V4.
DeepSeek's 38-page V4 technical report walks through their MoE routing, expert specialization patterns, and a compute breakdown that puts pretraining at 3.8M H800-equivalent hours.
Most interesting: the routing analysis suggests certain experts effectively specialize as language-pair translators, supporting the 'innate multilingual structure' hypothesis.
Weights are not released; only the paper.
No. DeepSeek released only the 38-page technical report. Earlier DeepSeek releases were open-weight, so the closed launch represents a notable change of posture.
The paper reports 3.8M H800-equivalent hours for pretraining, with an additional breakdown across MoE routing finetuning and supervised post-training stages.