More than 100 million Windows PCs will benefit from NVIDIA’s accelerated AI performance with TensorRT-LLM for RTX GPUs. During Microsoft Ignite, a key event where the future of artificial intelligence is discussed, NVIDIA announced that TensorRT-LLM is being added to Windows 11 and will be available to more than 100 million RTX users after launching in the latest driver package on November 21. The TensorRT-LLM AI acceleration will be available for all RTX desktops and laptops with more than 8GB of video memory. In addition to TensorRT-LLM, NVIDIA and Microsoft are also offering DirectML enhancements to improve popular AI models such as Stable Diffusion and Llama 2.

Having a TensorRT-LLM-enabled NVIDIA RTX GPU means that all your data and projects can be processed up to five times faster. This will save you time and give you more accurate results. The acceleration of artificial intelligence results is achieved through the localized RAG (Retrieval Augmented Generation) library. In addition, NVIDIA will also provide support for additional LLMs such as Mistral 7B and Nemotron 3 8B. For those who want to try out the latest release of TensorRT-LLM, it will be available for installation on the official Github link.

NVIDIA and Microsoft will also offer an API for OpenAI’s ChatAPI through a new shell. Through which you do not need to upload any dataset to ChatAPI, as the entire dataset will be available to ChatAPI as if it were available locally. The special shell will work with almost any LLM optimized for TensorRT-LLM. A few examples of optimized LLMs include Llama 2, Mistral, NV LLM, and more will be added soon. The API will also be available on the NVIDIA GitHub page.

These announcements show that NVIDIA wants to accelerate AI not only for enterprises, but also for the masses. And making it available to millions of RTX users is undoubtedly a huge deal.