Mistral AI is a French AI company that developed a family of open-weight large language models, including the dense Mistral 7B and the sparse Mixture-of-Experts Mixtral 8x7B and 8x22B architectures. These models are distinguished by their efficient use of grouped-query attention, sliding window attention, and sparse expert routing, achieving performance competitive with much larger models at a fraction of the inference cost. Released under permissive licences, they have become foundational reference models for the open-source AI community.
Content
- Mistral AI was founded in Paris in April 2023 by former Google DeepMind and Meta researchers. The company released Mistral 7B in September 2023 under the Apache 2.0 licence, immediately drawing attention for matching or exceeding Llama 2 13B on most benchmarks while using half the parameters. The Mixtral 8x7B sparse mixture-of-experts model followed in December 2023, matching GPT-3.5 performance at a fraction of the active-parameter cost.
- Mistral 7B employs grouped-query attention to reduce key-value cache memory and sliding-window attention for efficient long-context processing. Mixtral 8x7B extends this with a sparse MoE layer in which each token is routed to two of eight expert sub-networks by a learned gating function, meaning only 12.9B parameters are active per forward pass despite a total of 46.7B parameters. This design trades parameter count for compute efficiency, making the model deployable on consumer-grade 4-GPU nodes.
- The Mistral/Mixtral family has significant practical impact: open weights allow organisations to fine-tune on proprietary data without API dependency, and the permissive licence enables commercial deployment. Mixtral 8x22B, released in 2024, raised the capability ceiling further, excelling at coding, mathematics, and multilingual tasks. Mistral AI also offers Mistral Large and the instruction-following Mistral-Instruct variants through their commercial La Plateforme API.
- Through 2024 and into 2025, Mistral AI continued releasing models targeting specific niches: Codestral for code generation, Mistral Embed for retrieval-augmented generation, and Pixtral for multimodal vision-language tasks. The company secured over 1 billion USD in funding and established partnerships with Microsoft Azure and Google Cloud, cementing the Mistral family as a leading open-weight alternative to proprietary frontier models while continuing to push the state of the art in efficient transformer architectures.