BLOOM
BLOOM is the BigScience collaboration’s openly released 176B multilingual language model, developed with Hugging Face and an international research community. It is a dense autoregressive Transformer trained on the ROOTS multilingual corpus and released with research documentation and training artifacts. The model supports forty-six natural languages and thirteen programming languages, with smaller size checkpoints belonging to the same family. It is primarily a continuation model rather than a modern instruction-tuned assistant. The custom Responsible AI License includes use restrictions, so open weights should not be confused with unrestricted Apache licensing.
2022-07-12
560M, 1.1B, 1.7B, 3B, 7.1B, 176B
Dense decoder-only Transformer
BigScience BLOOM RAIL 1.0
Specifications
- Parameters
- 560M, 1.1B, 1.7B, 3B, 7.1B, 176B
- Architecture
- Dense decoder-only Transformer
- License
- BigScience BLOOM RAIL 1.0
- Context Window
- 2,048 tokens
- Type
- text
- Modalities
- text
Benchmark Scores
Advanced Specifications
- Model Family
- BLOOM
- API Access
- Not Available
- Chat Interface
- Not Available
- Multilingual Support
- Yes
- Variants
- 560M1.1B1.7B3B7.1B176B
Capabilities & Limitations
- Capabilities
- multilingual text generationcode generation
- Known Limitations
- Responsible-use restrictions in licenseLarge 176B tier requires substantial memoryBase continuation model
- Notable Use Cases
- multilingual researchlanguage-model adaptation