Guide · Glossary

GPU infrastructure, defined.

The terms you’ll meet when buying and hosting GPU servers, in one line each.

GPU hardware

HGX: NVIDIA’s 8-GPU baseboard. Server makers such as Dell, Supermicro and Lenovo build their AI servers around it. More →
DGX: NVIDIA’s own complete AI systems, built on the same GPUs as HGX servers.
SXM: A GPU module that mounts directly on the baseboard instead of a slot. It allows higher power and NVLink between all GPUs. More →
PCIe GPU: A GPU on a card that fits a standard server slot. Lower power and cost per GPU than SXM. More →
OAM: The Open Accelerator Module form factor, used by AMD Instinct GPUs such as the MI300X. More →
NVLink: NVIDIA’s high-bandwidth link between GPUs, so a model can span several GPUs as if they were one.
NVSwitch: Switch chips that connect every GPU’s NVLink to every other GPU in a server or rack.
NVL72: A rack-scale system that joins 72 GPUs in one NVLink domain, such as GB200 NVL72 and GB300 NVL72. More →
HBM3e: High-bandwidth memory stacked beside the GPU. Its size decides how large a model fits on one GPU.
TDP: Thermal design power: the most power a chip is designed to draw and turn into heat under sustained load. More →
CUDA: NVIDIA’s GPU programming platform. Most AI software is built on it.
ROCm: AMD’s GPU software stack. PyTorch and vLLM run on it.
InfiniBand: A low-latency network used to connect GPU servers for training across many nodes.

Colocation: Placing servers you own in a third-party data centre that supplies space, power, cooling and network. More →
Remote hands: On-site technicians who do physical work on your servers for you: reseats, cable checks, drive and part swaps. More →
Rack unit (U): The standard height unit for rack equipment, 1.75 inches. An 8-GPU server typically takes 5U to 10U.
kW per rack: How much power a rack is provisioned for. GPU racks need several times the power of ordinary server racks. More →
Tier IV: The Uptime Institute’s highest data centre certification, for fault-tolerant power and cooling. Yotta NM1 is Tier IV certified. More →
PUE: Power usage effectiveness: total facility power divided by the power used by IT equipment. Closer to 1 is more efficient.
Direct liquid cooling: Cold plates on the GPUs and CPUs carry heat away in liquid. Needed for the densest racks. More →
CDU: Coolant distribution unit: moves liquid between the facility’s cooling system and the racks.
Rear-door heat exchanger: A liquid-cooled door on the back of a rack that removes heat before it reaches the room.

Lead time: The time from order to shipment. For new GPUs it ranges from weeks to months. More →
Allocation: A share of scarce GPU supply reserved for a buyer. Without one, waits for new GPUs are longer. More →
WDV depreciation: Written-down value: each year’s depreciation is a percentage of what’s left. Servers depreciate at 40% a year in India. More →
Break-even utilisation: The share of time GPUs must be busy for owning to cost less than renting. More →