Technology Innovation Institute logo

Falcon-H1

Technology Innovation InstituteOpen WeightsPending Human Review

Falcon-H1 is a TII open-weight language-model release. Falcon-H1 replaces purely attention-based blocks with parallel attention and state-space heads. The family spans edge and larger enterprise sizes, including a deeper 1.5B variant. Model-specific head allocations and long-context training target efficient multilingual reasoning. Context capacity and language performance depend on size and serving configuration. Published license terms and checkpoint-specific hardware needs apply. Generated text can contain unsupported claims; sparse computation, reduced precision, or hybrid sequence modeling does not remove the need for task evaluation.

2025-05-20
0.5B, 1.5B, 3B, 7B, 34B
Parallel Mamba-2 state-space / Transformer hybrid heads
Falcon-LLM License

Specifications

Parameters
0.5B, 1.5B, 3B, 7B, 34B
Architecture
Parallel Mamba-2 state-space / Transformer hybrid heads
License
Falcon-LLM License
Context Window
262,144 tokens
Type
text
Modalities
text

Benchmark Scores

Advanced Specifications

Model Family
Falcon
API Access
Not Available
Chat Interface
Not Available
Variants
0.5B1.5B1.5B Deep3B7B34BBase/Instruct

Capabilities & Limitations

Capabilities
text generationinstruction followingreasoning
Known Limitations
Custom license appliesSmaller tiers have limited broad knowledge
Notable Use Cases
local language applicationsmodel researchenterprise customization

Related Models