Key Features
Fully open Mixture-of-Experts language model from AMD.
Uses 16B total parameters with about 2.8B active parameters per token.
Trained from scratch on AMD Instinct MI300X and MI325X GPUs.
Releases checkpoints across pretraining, mid-training, SFT, DPO, and RL stages.
Provides Hugging Face model links for multiple training checkpoints.
Includes a public GitHub repository for code and technical assets.
Designed for efficient MoE inference and open LLM experimentation.
Documents model architecture, training pipeline, and benchmark positioning.
About Instella-MoE
The release covers the full training pipeline, including pretraining, mid-training, long-context extension, supervised fine-tuning, direct preference optimization, and reinforcement learning. AMD provides multiple checkpoints so researchers can inspect and reuse different stages rather than only a final instruction model.
Instella-MoE is useful for open LLM research, efficient inference, training-pipeline study, and AMD GPU optimization. Its combination of public code, public checkpoints, and documented training stages makes it valuable for teams that need reproducible MoE model development.