AI Model Catalog | Microsoft Foundry Models

Mixtral-8x7B-Instruct-v0.1-NIM-microservice

Version: 2

Nvidia•Last updated March 2025

Mixtral 8x7B Instruct is a language model that can follow instructions, complete requests, and generate creative text formats. Mixtral 8x7B a high-quality sparse mixture of experts model (SMoE) with open weights. This model has been optimized through supervised fine-tuning and direct preference optimization (DPO) for careful instruction following. Mixtral has the following capabilities.

It gracefully handles a context of 32k tokens.
It handles English, French, Italian, German and Spanish.
It shows strong performance in code generation.
It can be finetuned into an instruction-following model that achieves a score of 8.3 on MT-Bench.

Mixtral 8x7B is available as an NVIDIA NIM™ microservice, part of NVIDIA AI Enterprise . NVIDIA NIM offers prebuilt containers for large language models (LLMs) that can be used to develop chatbots, content analyzers—or any application that needs to understand and generate human language. Each NIM consists of a container and a model and uses a CUDA-accelerated runtime for all NVIDIA GPUs, with special optimizations available for many configurations. Built on robust foundations, including inference engines like NVIDIA Triton Inference Server™, TensorRT™, TensorRT-LLM, and PyTorch, NVIDIA NIM is the fastest way to achieve accelerated generative AI inference at scale and has been benchmarked to have up to 2.6x improved throughput latency. NVIDIA AI Enterprise
NVIDIA AI Enterprise is an end-to-end, cloud-native software platform that accelerates data science pipelines and streamlines development and deployment of production-grade co-pilots and other generative AI applications. Easy-to-use microservices provide optimized model performance with enterprise-grade security, support, and stability to ensure a smooth transition from prototype to production for enterprises that run their businesses on AI.

Intended Use

Primary Use Cases

Text Generation: Automatic Article Writing: Mixtral 8x7B can generate coherent and contextually relevant text for articles, blog posts, or other written content.
Chatbot Content Creation: It can produce engaging and personalized responses for chatbots, enhancing user interaction.
Product Description Generation: The model can create detailed and compelling product descriptions for e-commerce platforms.
Text Summarization: Mixtral 8x7B can summarize long documents by extracting key points and generating concise summaries.
Sentiment Analysis: It can analyze and classify sentiments expressed in texts, helping businesses understand customer opinions.
Machine Translation: The model's multilingual capabilities make it suitable for machine translation tasks, facilitating communication across languages.
Code Generation and Mathematics: Mixtral 8x7B is particularly adept at generating code and solving mathematical problems, making it useful for software development and educational tools1.
Research Assistance: It can assist researchers by answering complex questions and exploring large datasets, speeding up scientific discovery.
Content Personalization: By understanding user preferences from text interactions, Mixtral 8x7B can help personalize digital content, improving user engagement.
Enterprise Applications: The model is suitable for tasks like customer support, classification, and text generation in large-scale enterprise environments.
Data Analysis: Mixtral 8x7B can interpret intricate data patterns and generate insights, contributing to decision-making processes

Responsible AI Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

On MT-Bench, it reaches a score of 8.30, making it the best open-source model, with a performance comparable to GPT3.5. Mixtral outperforms Llama 2 70B on most benchmarks with 6x faster inference. It is the strongest open-weight model with a permissive license and the best model overall regarding cost/performance trade-offs. In particular, it matches or outperforms GPT3.5 on most standard benchmarks. Mixtral 8x7B Instruct v0.1 NIM is optimized to run best on the following compute:

GPU	Total GPU memory	Azure VM compute	#GPUs on VM	Link
A100	320	Standard_NC96ads_A100_v4	4	link
A100	640	STANDARD_ND96AMSR_A100_V4	8	link
H100	188	Standard_NC80adis_H100_v5	2	link

Model Specifications

LicenseCustom

Last UpdatedMarch 2025

PublisherNvidia

Quick Start