After much time learning about NVIDIA's latest and most powerful GPU - the Tesla P100, we finally got to see it in person from a server vendor, QuantaPlex at GTC 2016.
This is how big the Tesla P100 GPU package is! It’s most definitely larger than an old Pentium Pro’s heat spreader that housed two different dies (processor and cache) underneath.
The server displayed is a QuantaPlex T21W-3U and much like NVIDIA’s own DGX-1 deep learning ‘supercomputer’, it too boasts 8 x Tesla P100 GPUs.
The server has a top sled which contains the NVLink fabric and the 8 Tesla GPUs, while the bottom sled is home to a dual Xeon CPU configuration.
Here's a slide from QuantaPlex that further details the server layout.
This is the bottom sled where the dual Xeon processors sit. Unlike the NVIDIA DXG-1, this server rack is vendor configurable. Only the eight Tesla P100 GPUs come as standard.
This is the data path diagram for all major components of the server. GPU to GPU communication is via the proprietary ultra-high bandwidth NVLink, while the CPUs interact with the GPU array through PCIe that has been fanned out with PLX switches that help multiply the number of PCIe lanes available. The extra PCIe lanes help the system support PCIe SSDs and InfiniBand networking.
The QuantaPlex T21W-3U is a barebones ‘supercomputer’ that is up to the vendor to spec it out as required. As iterated earlier, only the Tesla GPUs are standard issue. As such, the base cost of this server is between US$65,000 to US$70,000 - about half the cost of the NVIDIA DGX-1. It’s slated to be available for purchase from September 2016.