SmolLM3: A Compact Multilingual Long-Context Reasoning Model
TL;DR
SmolLM3 is a fully open 3B-parameter language model built for hybrid reasoning, multilingual interaction, tool calling, and long-context tasks. It supports six languages, offers reasoning and direct-response modes, and extends from a 64K training context to 128K tokens with YaRN.
What SmolLM3 Is
SmolLM3, now listed on AIOZ AI, supports reasoning, instruction following, coding, multilingual understanding, and tool-use workflows.
It operates across English, French, Spanish, German, Italian, and Portuguese, making it useful for multilingual applications that need both interaction quality and context depth.
How the Model Architecture Works
SmolLM3 is a decoder-only transformer based on the Llama architecture, with adaptations for efficient inference and long-context performance.
It uses Grouped Query Attention (GQA) with four key-value groups to reduce KV cache memory, and a 3:1 RoPE-to-NoPE pattern to support long-context processing while preserving short-context performance.
The model was pretrained on 11.2 trillion tokens through a three-stage curriculum covering web, code, mathematics, multilingual, and reasoning data.
The training process was followed by:
- Reasoning mid-training across 140 billion tokens
- Supervised fine-tuning for reasoning and non-reasoning tasks
- Alignment through Anchored Preference Optimization
- Model merging to preserve reasoning and long-context performance
Dual-Mode Reasoning
SmolLM3 supports two operating modes:
- Reasoning mode for complex logic, mathematics, coding, and multi-step analysis
- Non-reasoning mode for faster, more direct responses
Applications can switch between these modes using /think and /no_think instructions in the system prompt.
Core Capabilities
- Hybrid reasoning with reasoning and direct-response modes
- Multilingual interaction across six languages
- Long-context processing up to 128K tokens
- Mathematics, coding, and scientific reasoning
Key Technical Details
- Model: SmolLM3
- Model type: decoder-only transformer
- Parameters: 3B
- Native context: 64K
- Maximum context support: 128K tokens
- Attention: Grouped Query Attention (GQA), 4 groups
- Positional encoding: Hybrid RoPE/NoPE architecture (3:1 pattern)
- Tokenizer: Llama 3.2 tokenizer
- Vocabulary: 128K
- Pretraining data: 11.2 trillion tokens
- Data mix: web, code, math, and reasoning data
- Training approach: staged three-phase curriculum
- License: Apache License 2.0
Where It Fits Best
- Multilingual conversational assistants
- Agentic applications with tool calling
- Long-document analysis and summarization
- Mathematics and complex reasoning tasks
Download It on AIOZ AI
Three billion parameters, dual-mode reasoning, six languages, and support for up to 128K tokens: SmolLM3 brings broad language capabilities to a compact, fully open model.
For reasoning-heavy use cases, test Thinking Mode on multi-step logic, technical analysis, or scientific problem-solving tasks.
Download SmolLM3 and start building multilingual, long-context, reasoning-driven applications.
FAQ
Q1: What is SmolLM3?
It is a fully open 3B-parameter language model designed for multilingual interaction, hybrid reasoning, tool calling, and long-context processing.
Q2: How much context does SmolLM3 support?
It is trained with a 64K-token context and can process up to 128K tokens using YaRN extrapolation.
Q3: How does its hybrid reasoning work?
The model supports reasoning and non-reasoning modes. Developers can request deeper analysis for complex tasks or direct responses when speed is the priority.