Mixtral-8x7B-Instruct-v0.1-NIM-microservice

Mixtral 8x7B Instruct is a language model that can follow instructions, complete requests, and generate creative text formats. Mixtral 8x7B a high-quality sparse mixture of experts model (SMoE) with open weights. This model has been optimized through supervised fine-tuning and direct preference optimization (DPO) for careful instruction following. Mixtral has the following capabilities.

• It gracefully handles a context of 32k tokens.
• It handles English, French, Italian, German and Spanish.
• It shows strong performance in code generation.
• It can be finetuned into an instruction-following model that achieves a score of 8.3 on MT-Bench.

Mixtral 8x7B is available as an NVIDIA NIM™ microservice, part of NVIDIA AI Enterprise . NVIDIA NIM offers prebuilt containers for large language models (LLMs) that can be used to develop chatbots, content analyzers—or any application that needs to understand and generate human language. Each NIM consists of a container and a model and uses a CUDA-accelerated runtime for all NVIDIA GPUs, with special optimizations available for many configurations. Built on robust foundations, including inference engines like NVIDIA Triton Inference Server™, TensorRT™, TensorRT-LLM, and PyTorch, NVIDIA NIM is the fastest way to achieve accelerated generative AI inference at scale and has been benchmarked to have up to 2.6x improved throughput latency.

NVIDIA AI Enterprise
NVIDIA AI Enterprise is an end-to-end, cloud-native software platform that accelerates data science pipelines and streamlines development and deployment of production-grade co-pilots and other generative AI applications. Easy-to-use microservices provide optimized model performance with enterprise-grade security, support, and stability to ensure a smooth transition from prototype to production for enterprises that run their businesses on AI.