BLOOMZ
BLOOMZ is BigScience’s crosslingual multitask adaptation of the BLOOM language-model family. Training on xP3 combines multilingual instructions and tasks to improve zero-shot generalization across both task types and languages, making it distinct from BLOOM’s base continuation behavior. Released sizes span small local checkpoints through the 176B tier, with different prompting and multilingual training variants. The accompanying mT0 uses a separate encoder-decoder backbone and is not treated as a BLOOM checkpoint. The public paper releases models and training code; results depend on task instructions and language coverage.
2022-11-03
560M, 1.1B, 1.7B, 3B, 7.1B, 176B
Dense decoder-only Transformer
BigScience BLOOM RAIL 1.0
Specifications
- Parameters
- 560M, 1.1B, 1.7B, 3B, 7.1B, 176B
- Architecture
- Dense decoder-only Transformer
- License
- BigScience BLOOM RAIL 1.0
- Context Window
- 2,048 tokens
- Type
- text
- Modalities
- text
Benchmark Scores
Advanced Specifications
- Model Family
- BLOOM
- Finetuned From
- BLOOM
- API Access
- Not Available
- Chat Interface
- Not Available
- Multilingual Support
- Yes
- Variants
- 560M1.1B1.7B3B7.1B176BMT/P3 training variants
Capabilities & Limitations
- Capabilities
- multilingual instruction followingcrosslingual generalizationzero-shot tasks
- Known Limitations
- Responsible-use restrictions in licensePrompt formatting affects task behavior
- Notable Use Cases
- multilingual assistantscrosslingual research