Hugging Face logo

BLOOM

Hugging FaceOpen WeightsPending Human Review

BLOOM is the BigScience collaboration’s openly released 176B multilingual language model, developed with Hugging Face and an international research community. It is a dense autoregressive Transformer trained on the ROOTS multilingual corpus and released with research documentation and training artifacts. The model supports forty-six natural languages and thirteen programming languages, with smaller size checkpoints belonging to the same family. It is primarily a continuation model rather than a modern instruction-tuned assistant. The custom Responsible AI License includes use restrictions, so open weights should not be confused with unrestricted Apache licensing.

2022-07-12
560M, 1.1B, 1.7B, 3B, 7.1B, 176B
Dense decoder-only Transformer
BigScience BLOOM RAIL 1.0

Specifications

Parameters
560M, 1.1B, 1.7B, 3B, 7.1B, 176B
Architecture
Dense decoder-only Transformer
License
BigScience BLOOM RAIL 1.0
Context Window
2,048 tokens
Type
text
Modalities
text

Benchmark Scores

Advanced Specifications

Model Family
BLOOM
API Access
Not Available
Chat Interface
Not Available
Multilingual Support
Yes
Variants
560M1.1B1.7B3B7.1B176B

Capabilities & Limitations

Capabilities
multilingual text generationcode generation
Known Limitations
Responsible-use restrictions in licenseLarge 176B tier requires substantial memoryBase continuation model
Notable Use Cases
multilingual researchlanguage-model adaptation

Related Models