DeepSeek-V2
DeepSeek-V2 introduced DeepSeekMoE and Multi-head Latent Attention to reduce the cost of storing attention states and running a large language model. Its flagship has 236B parameters with 21B activated per token and supports 128K contexts. The release includes base/chat models and a smaller V2-Lite tier. It is a historical foundation for the V2.5 and later DeepSeek families; downloadable weights remain available under the DeepSeek Model License. Its Chinese and English evaluations cover knowledge, mathematics, coding, and instruction following.
2024-05-06
236B total, 21B activated (flagship)
Mixture of Experts (DeepSeekMoE) with Multi-head Latent Attention
DeepSeek Model License
Specifications
- Parameters
- 236B total, 21B activated (flagship)
- Architecture
- Mixture of Experts (DeepSeekMoE) with Multi-head Latent Attention
- License
- DeepSeek Model License
- Context Window
- 128,000 tokens
- Type
- text
- Modalities
- text
Benchmark Scores
Advanced Specifications
- Model Family
- DeepSeek
- API Access
- Not Available
- Chat Interface
- Not Available
- Multilingual Support
- Yes
- Variants
- DeepSeek-V2-ChatDeepSeek-V2-Lite (16B total, 2.4B activated)DeepSeek-V2-Lite-ChatDeepSeek-V2-Chat-0628
Capabilities & Limitations
- Capabilities
- reasoningcode generationmathematicslong-context understandingChinese and English
- Known Limitations
- Model weights use a custom license, separate from MIT-licensed code.The Lite tier has different size and context specifications.
- Notable Use Cases
- coding assistantsmathematical problem solvingdocument analysis