SOLAR 10.7B
SOLAR 10.7B is Upstage’s original depth-up-scaled language model, built by expanding a Mistral 7B initialization into a 48-layer dense transformer and continuing pretraining. The release includes a base checkpoint and an instruction-tuned conversational variant, also marketed as Solar Mini. The base weights use Apache 2.0; the current instruction checkpoint specifies CC BY-NC 4.0 because its alignment data includes non-commercial datasets. The documented native context is 4,096 tokens. It supports local text generation, customization, and research without assuming that later hosted Solar Mini revisions are the identical checkpoint.
2023-12-14
10.7B
Decoder-only Transformer with depth up-scaling
Apache-2.0 (base); CC-BY-NC-4.0 (Instruct)
Specifications
- Parameters
- 10.7B
- Architecture
- Decoder-only Transformer with depth up-scaling
- License
- Apache-2.0 (base); CC-BY-NC-4.0 (Instruct)
- Context Window
- 4,096 tokens
- Type
- text
- Modalities
- text
Benchmark Scores
Advanced Specifications
- Model Family
- Solar
- API Access
- Not Available
- Chat Interface
- Not Available
- Variants
- SOLAR-10.7B-v1.0SOLAR-10.7B-Instruct-v1.0Solar Mini
Capabilities & Limitations
- Capabilities
- text generationinstruction followingcontinued pretraininglocal inference
- Known Limitations
- Instruction weights have non-commercial restrictionsNative context is 4KGenerated factual claims require verification
- Notable Use Cases
- private text assistantsdomain adaptationlanguage model research