Baidu Open-Sources ERNIE 5.5 Flash Base Model
百度开源文心5.5 Flash Base，多基准测试刷新同尺寸模型 SOTA
Baidu has officially open-sourced ERNIE 5.5 Flash Base, a Mixture-of-Experts (MoE) base model featuring 280 billion total parameters with 14 billion activated parameters per token. Pre-trained on 33.5 trillion tokens, the model supports an inference context window of up to 1 million tokens and achieves class-leading performance across multiple benchmarks in knowledge, reasoning, and coding. Architecturally, it adopts latent gated sparse attention combining HCA and MLA, and learns to predict subsequent tokens by reusing the backbone's final layer. The model weights are now available on Hugging Face.
The original release provides training details on sparse attention, multi-route MoE, and ECHO via reused backbone layers, offering valuable insights into the engineering trade-offs of large-parameter, sparsely activated architectures.
Source:量子位 · qbitai.com