Scaling CUDA C++ Applications to Multiple Nodes (SCCAMN)


Course Overview

Present-day high-performance computing (HPC) and deep learning applications benefit from, and even require, cluster-scale GPU compute power. Writing CUDA applications that can correctly and efficiently utilize GPUs across a cluster requires a distinct set of skills. In this workshop, you’ll learn the tools and techniques needed to write CUDA C++ applications that can scale efficiently to clusters of NVIDIA GPUs.

Please note that once a booking has been confirmed, it is non-refundable. This means that after you have confirmed your seat for an event, it cannot be cancelled and no refund will be issued, regardless of attendance.


Intermediate experience writing CUDA C/C++ applications.

Suggested materials to satisfy the prerequisites:

  • Fundamentals of Accelerated Computing with CUDA C/C++
  • Accelerating CUDA C++ Applications with Multiple GPUs
  • Accelerating CUDA C++ Applications with Concurrent Streams
  • Scaling Workloads Across Multiple GPUs with CUDA C++

Course Objectives

By participating in this workshop, you’ll:

  • Learn several methods for writing multi-GPU CUDA C++ applications
  • Use a variety of multi-GPU communication patterns and understand their tradeoffs
  • Write portable, scalable CUDA code with the single-program multiple-data (SPMD) paradigm using CUDA-aware MPI and NVSHMEM
  • Improve multi-GPU SPMD code with NVSHMEM’s symmetric memory model and its ability to perform GPU-initiated data transfers
  • Get practice with common multi-GPU coding paradigms like domain decomposition and halo exchanges

Follow On Courses

Prices & Delivery methods

Online Training

1 day

  • 500.— €
Classroom Training

1 day

  • Austria: 500.— €
  • Germany: 500.— €

Currently there are no training dates scheduled for this course.