On July 16, Beijing-based Moonshot AI released its next-generation large language model (LLM) named "Kimi K3". With 2.8 trillion parameters, it is currently the world's largest open-source model by parameter count.

Kimi K3 natively supports visual understanding and features a 1-million-token context window. It is optimized for complex tasks such as those of software engineering, knowledge, in-depth research, and multimodal understanding, further enhancing the model's capability of handling complex tasks. Evaluation results show that Kimi K3's overall intelligence level is close to that of the world's leading closed-source models.

During the training of this next-generation foundation LLM, the company utilized a self-developed underlying model architecture and built a comprehensive, systematic methodology for model training. Through optimization in model architecture, training methods, and data recipes, among other aspects, Kimi K3 achieves an approximately 2.5x improvement in overall scaling efficiency compared to the model of the previous generation. Its efficiency of translating computational resources into model capabilities can also be further improved.

According to Moonshot AI, the full model weights will be released soon, with details on model architecture, training, and evaluation to be announced in the model's forthcoming technical report.

(Source: Beijing Fabu)