Get CUDA Device Memory Usage using C++

When writing CUDA applications, it's important to keep track of the amount of memory used by CUDA device, as it can have a significant impact on performance. This tutorial explains how to get the CUDA device memory usage using C++.

The cudaMemGetInfo function can be used to get the amount of free and total memory in bytes available on the CUDA device. The used memory is calculated by subtracting the amount of free memory from the total memory.

In the following example, we calculate used memory of each CUDA device. The total, free, and used memory in megabytes are printed to the console.

#include <iostream>
#include <cuda_runtime.h>

int main()
{
    int deviceCount;
    cudaGetDeviceCount(&deviceCount);

    for (int i = 0; i < deviceCount; ++i) {
        cudaSetDevice(i);

        size_t freeBytes, totalBytes;
        cudaMemGetInfo(&freeBytes, &totalBytes);
        size_t usedBytes = totalBytes - freeBytes;

        std::cout << "Device Number: " << i << std::endl;
        std::cout << " Total Memory (MB): " << (totalBytes / 1024.0 / 1024.0) << std::endl;
        std::cout << " Free Memory (MB): " << (freeBytes / 1024.0 / 1024.0) << std::endl;
        std::cout << " Used Memory (MB): " << (usedBytes / 1024.0 / 1024.0) << std::endl;
    }

    return 0;
}

Here's an example of the output:

Device Number: 0
 Total Memory (MB): 7973.69
 Free Memory (MB): 7723.94
 Used Memory (MB): 249.75