OLMoE
OLMoE is a fully open Ai2 language-model family. Ai2’s openly trained sparse language model studies efficient routing and expert specialization. It provides a seven-billion-parameter weight footprint with approximately one billion parameters used per token. The initial base and instruction models are accompanied by the full training mixture, code, logs, and analysis; January 2025 instruction refreshes remain variants. Apache-2.0 weights permit adaptation and deployment under the published terms. Outputs can hallucinate, and smaller tiers or different training stages should not be assumed to share identical capabilities.
2024-09-03
7B total, 1B active
Sparse mixture-of-experts Transformer
Apache-2.0
Specifications
- Parameters
- 7B total, 1B active
- Architecture
- Sparse mixture-of-experts Transformer
- License
- Apache-2.0
- Context Window
- 4,096 tokens
- Type
- text
- Modalities
- text
Benchmark Scores
Advanced Specifications
- Model Family
- Olmo
- API Access
- Not Available
- Chat Interface
- Not Available
- Variants
- BaseInstructJanuary 2025 refresh
Capabilities & Limitations
- Capabilities
- text generationinstruction following
- Known Limitations
- Generated outputs can be incorrectPerformance varies by task and deployment
- Notable Use Cases
- language-model researchself-hosted assistance