Configure CUDA and cuDNN for GPU with ONNX Runtime and C# on Windows 11

Prerequisites

  • Windows 11
  • Visual Studio 2019 or 2022

Steps to Configure CUDA and cuDNN for ONNX Runtime with C# on Windows 11

See this table for supported versions:

ONNX Runtime Version CUDA Toolkit Version cuDNN Version
1.13 - 1.16 11.6 8.5.0.96
1.9 - 1.12 11.4 8.2.2.26

NOTE: Full table can be found here

  • Follow section 2. Installing cuDNN on Windows. NOTE: Skip step 5 in section 2.3 on updating Visual Studio settings, this is only for C++ projects.

  • Restart your computer and verify the installation by running the following command or in python with PyTorch:

nvcc --version
import torch
torch.cuda.is_available()
  • Now you can enable GPU in the C# ONNX Runtime API with the following code:
// keep in mind almost all of the classes are disposable.
using var gpuSessionOptions = SessionOptions.MakeSessionOptionWithCudaProvider(0);
using var session = new InferenceSession(modelPath, gpuSessionOptions);

Checkout more C# ONNX Runtime resources