MiniCPM-SALA

Name: MiniCPM-SALA
Author: OpenBMB

OpenBMBOpen WeightsPending Human Review

MiniCPM-SALA is OpenBMB’s 9.48B language model combining sparse and linear attention for million-token contexts. One quarter of layers use InfLLM-v2 sparse attention and three quarters use Lightning Attention, with Hybrid Positional Embedding supporting length generalization beyond short training sequences. Apache-licensed weights and inference examples target long-context local language tasks with reduced cache overhead. The official model card documents deployment at up to 1M tokens, extending beyond its static 512K configuration value. Speedup claims depend on hardware, sequence length, and implementation rather than applying uniformly to every prompt.

2026-02-11

9.48B

Hybrid sparse/linear-attention language model

Apache-2.0

Compare with other models

Specifications

Parameters: 9.48B
Architecture: Hybrid sparse/linear-attention language model
License: Apache-2.0
Context Window: 1,048,576 tokens
Type: text
Modalities: text

Benchmark Scores

Advanced Specifications

Model Family: MiniCPM
API Access: Not Available
Chat Interface: Not Available

Capabilities & Limitations

Capabilities: long contextefficient inferencereasoningcoding
Known Limitations: Million-token contexts require compatible positional and inference settingsAttention acceleration varies with sequence length and hardwareLong context does not guarantee accurate retrieval from every position
Notable Use Cases: large-document analysislocal long-context researchmemory-constrained language serving

Resources

Related Models

MiniCPM-V 4.7 35B-A3B

OpenBMB

MiniCPM-V 4.7 35B-A3B is a public OpenBMB vision-language checkpoint uploaded on October 6. Its official repository contains 16 downloadable BF16 weight shards totaling approximately 35.21B parameters, a hybrid linear/full-attention mixture-of-experts language configuration, and image/video processing metadata. The configuration permits 262,144 positions, but no model card or validated context specification was published in the inspected repository, so no supported context limit is asserted here. Text and visual inputs produce language output through the conditional-generation architecture. The license, training ancestry, evaluated capabilities, and benchmark results remain undisclosed at the audit time; a public weight upload does not supply missing usage rights.

Typemultimodal

Parameters35.21B total; approximately 3B active tier

2026-10-06

Open Weights

Details Compare

MathForm-8B

OpenBMB

MathForm-8B is OpenBMB’s language-to-code model for translating natural-language mathematical statements into Lean 4. It fine-tunes Qwen3-8B on FormalVerse with supervised learning followed by reinforcement learning using compilation and semantic-consistency feedback. The research framework retrieves Mathlib definitions and revises generated formalizations using diagnostics, supporting more faithful mathematical translation. Apache-licensed weights, approximately 367K verified examples, and evaluation code accompany the August 14 paper. This is a named mathematical language specialization; the model is distinct from the surrounding retrieval and verification pipeline. Successful compilation alone does not establish that a formal statement preserves the intended proposition.

UltraX

OpenBMB

UltraX is OpenBMB’s specialized 0.6B language-model release for refining pre-training text through structured editing function calls. It generates insertions, deletions, and replacements rather than rewriting an entire document, and a deterministic executor applies validated edits. Full-parameter supervised training uses program supervision built from language-model refinements and alignment procedures. The July 9 paper precedes the July 13 public code-and-model announcement. Apache-licensed refinement checkpoints and data are available, with no-instruction and preservation-weighted variants grouped here. Its named language-editing specialization qualifies separately from generic data tools or probability-only classifiers.