Kimi Linear
Kimi Linear is an open-weight language-model family demonstrating a hybrid linear-attention architecture. Kimi Delta Attention provides fine-grained gated recurrent-state updates and is combined with Multi-head Latent Attention for long-context modeling. The published 48B model activates 3B parameters and supports million-token contexts, with base and instruction-tuned checkpoints trained on 5.7 trillion tokens. Moonshot releases the model and KDA kernels to support efficient long-context inference and research; efficiency measurements depend on context length and the compared implementation.
2025-10-30
48B total, 3B activated
Mixture of Experts (MoE) with hybrid Kimi Delta Attention and Multi-head Latent Attention
MIT
Specifications
- Parameters
- 48B total, 3B activated
- Architecture
- Mixture of Experts (MoE) with hybrid Kimi Delta Attention and Multi-head Latent Attention
- License
- MIT
- Context Window
- 1,000,000 tokens
- Type
- text
- Modalities
- text
Benchmark Scores
Advanced Specifications
- Model Family
- Kimi
- API Access
- Not Available
- Chat Interface
- Not Available
- Variants
- Kimi-Linear-48B-A3B-BaseKimi-Linear-48B-A3B-Instruct
Capabilities & Limitations
- Capabilities
- long-context modelinglinear attentioninstruction followingcode generation
- Known Limitations
- Reported speedups are specific to the benchmark context lengths and inference setups.All expert weights still require storage despite only 3B activation.
- Notable Use Cases
- efficient long-context researchself-hosted assistantsattention architecture studies