NVIDIA GB200 NVL2

Name: NVIDIAl GB200 NVL2
Brand: NVIDIA

Bringing the new era of computing to every data center.

Unparalleled Single-Server Performance

The NVIDIA GB200 NVL2 platform brings the new era of computing to every data center, delivering unparalleled performance for mainstream large language model (LLM) inference, vector database search, and data processing through 2 Blackwell GPUs and 2 Grace CPUs. With its scale-out, single node NVIDIA MGX™ architecture, its design enables a wide variety of system designs and networking options to seamlessly integrate accelerated computing into existing data center infrastructure.

Computer Industry Joins NVIDIA to Build AI Factories and Data Centers for the Next Industrial Revolution

At Computex 2024, the world’s top computer manufacturers joined NVIDIA to unveil the latest NVIDIA Blackwell-powered systems, including the GB200 NVL2, to lead the next industrial revolution.

Press Release

Highlights

Turbocharging Accelerated Computing

Llama 3 Inference

5Xvs. NVIDIA H100 Tensor Core GPU

Vector Database Search

9X vs. H100

Data Processing

18X vs. CPU

Llama3 LLM inference: Token-to-token latency (TTL) = 50 milliseconds (ms) real time, first token latency (FTL) = 2s, input sequence length = 2.048, output sequence length = 128 output, 8x NVIDIA HGX™ H100 air-cooled vs. GB200 NVL2 air-cooled single node, per-GPU performance comparison
Vector database search performance within RAG pipeline using memory shared by NVIDIA Grace CPU and Blackwell GPU. 1x x86, 1x H100 GPU, and 1x GPU from GB200 NVL2 node.
Data processing: A database join and aggregation workload with Snappy/Deflate compression derived from TPC-H Q4 query. Custom query implementations for x86, H100 single GU, and single GPU from GB200 NVL2 node: GB200 vs. Intel Xeon 8480+
Projected performance subject to change.

Real-Time Mainstream LLM Inference

GB200 NVL2 introduces massive coherent memory up to 1.3 terabytes (TB) shared between two Grace CPUs and two Blackwell GPUs. This shared memory is coupled with fifth-generation NVIDIA® NVLink™ and high-speed, chip-to-chip (C2C) connections to deliver 5X faster real-time LLM inference performance for mainstream language models such as Llama 3 70B.

Vector Database Search

GB200 NLV2 accelerates RAG vector search operation by up to 9X. The vector database of the Wikipedia dataset is over 200 gigabytes (GB), and access to the Grace CPU’s 960GB of memory and 900GB/s high-speed C2C link supercharges low-latency vector search.

Data Processing

Databases play critical roles in handling, processing, and analyzing large volumes of data for enterprises. GB200 NVL2 takes advantage of high-bandwidth memory performance, NVLink-C2C, and dedicated decompression engines in the NVIDIA Blackwell architecture to speed up key database queries by 18X compared to CPU.

Features

Technological Breakthroughs

Blackwell Architecture

The NVIDIA Blackwell architecture delivers groundbreaking advancements in accelerated computing, powering a new era of computing with unparalleled performance, efficiency, and scale.

Learn More About Blackwell

NVIDIA Grace CPU

The NVIDIA Grace CPU is a breakthrough processor designed for modern data centers running AI, cloud, and high-performance computing (HPC) applications. It provides outstanding performance and memory bandwidth with 2X the energy efficiency of today’s leading server processors.

Learn More About the Grace CPU Superchip

NVIDIA NVLink-C2C

NVIDIA NVLink-C2C coherently interconnects each Grace CPU and Blackwell GPU at 900GB/s. The GB200 NVL2 uses both NVLink-C2C and the fifth-generation NVLink to deliver a 1.4 TB coherent memory model for accelerated AI.

Explore NVLink-C2C

Key Value Caching

Key Value (KV) Caching improves LLM response speeds by storing conversation context and history. GB200 NVL2 optimizes KV Caching through its fully coherent Grace GPU and Blackwell GPU memory connected by NVLink-C2C, 7X faster than PCIe, enabling LLMs to predict words faster than x86-based GPU implementations.

Learn More About KV Caching

Fifth-Generation NVIDIA NVLink

Unlocking the full potential of exascale computing and trillion-parameter AI models requires swift, seamless communication between every GPU in a server cluster. Fifth-generation NVLink is a scale-up interconnect that unleashes accelerated performance for trillion- and multi-trillion-parameter AI models.

Learn About NVLink and NVLink Switch

NVIDIA Networking

The data center’s network plays a crucial role in driving AI advancements and performance, serving as the backbone for distributed AI model training and generative AI performance. NVIDIA Quantum-X800 InfiniBand, NVIDIA Spectrum™-X800 Ethernet, and NVIDIA BlueField®-3 DPUs enable efficient scalability across hundreds and thousands of Blackwell GPUs for optimal application performance.

Explore End-to-End Networking Solutions

More GB200 Grace Blackwell Platforms

NVIDIA GB200 Grace Blackwell NVL4 Superchip

NVIDIA GB200 Grace Blackwell NVL4 Superchip unlocks the future of converged HPC and AI, delivering revolutionary performance through four NVIDIA NVLink™-connected Blackwell GPUs unified with two Grace CPUs over NVLink-C2C.

Learn More About HPC Solutions

NVIDIA GB200 NVL72

The NVIDIA GB200 NVL72 connects 36 GB200 Superchips in a rack-scale design. The GB200 NVL72 is a liquid-cooled, rack-scale solution that boasts a 72-GPU NVLink domain that acts as a single, massive GPU.

Learn More

Specifications¹