Hugging Face logo

BLOOMZ

Hugging FaceOpen WeightsPending Human Review

BLOOMZ is BigScience’s crosslingual multitask adaptation of the BLOOM language-model family. Training on xP3 combines multilingual instructions and tasks to improve zero-shot generalization across both task types and languages, making it distinct from BLOOM’s base continuation behavior. Released sizes span small local checkpoints through the 176B tier, with different prompting and multilingual training variants. The accompanying mT0 uses a separate encoder-decoder backbone and is not treated as a BLOOM checkpoint. The public paper releases models and training code; results depend on task instructions and language coverage.

2022-11-03
560M, 1.1B, 1.7B, 3B, 7.1B, 176B
Dense decoder-only Transformer
BigScience BLOOM RAIL 1.0

Specifications

Parameters
560M, 1.1B, 1.7B, 3B, 7.1B, 176B
Architecture
Dense decoder-only Transformer
License
BigScience BLOOM RAIL 1.0
Context Window
2,048 tokens
Type
text
Modalities
text

Benchmark Scores

Advanced Specifications

Model Family
BLOOM
Finetuned From
BLOOM
API Access
Not Available
Chat Interface
Not Available
Multilingual Support
Yes
Variants
560M1.1B1.7B3B7.1B176BMT/P3 training variants

Capabilities & Limitations

Capabilities
multilingual instruction followingcrosslingual generalizationzero-shot tasks
Known Limitations
Responsible-use restrictions in licensePrompt formatting affects task behavior
Notable Use Cases
multilingual assistantscrosslingual research

Related Models