Baidu ERNIE open-sources ERNIE 5.5 Flash Base foundation model
百度文心开源发布 ERNIE 5.5 Flash Base 高效基座模型，总参数 280B
Baidu's ERNIE team has open-sourced ERNIE-5.5-Flash-Base, a sparse MoE language model featuring 280B total parameters with 14B activated per token, pre-trained from scratch on 33.5T tokens. The model weights have not undergone mid-training or post-training, support up to a 1M-token context window for fine-tuning and inference, and were trained using a distributed Muon optimizer paired with the FlashMask-4 attention operator. The team stated that a post-trained full release will follow.
Open-sourcing raw, unaligned pre-trained base weights with 280B total and 14B active parameters per token provides a reusable starting point for domain adaptation and reinforcement learning research.
Source:IT之家 · AI 筛选 · ithome.com