In this notebook I will show how I have managed to get CUDA working on my device ( you might need to download different version of the packages based on the GPU you have)

I already had the latest version of Anaconda installed, so I am not going to go through that, but if you are installing it make sure you check the option to add Anaconda to the PATH environment variable and run the conda init into your terminal with the right option accorindg to your type of terminal [bash, powershell, command prompt]

CUDA

Before installing CUDA is highly recommended/essential to install Visual Studio Community Edition (Not Visual Studio Code). It is not required to install any other additional workload/packages if you are not planning to use Visual Studio as your main IDE.

My GPU is an Nvidia RTX 3090 and this enabled me to install the latest version of CUDA toolkit 11.3, however, as mentioned before, you need to check the architecture your GPU is based on and download the CUDA toolkit version acording to that. You can find here in the CUDA Toolkit Archive all the CUDA relsease

After installing CUDA, we need to install cuDNN

NVIDIA CUDA Deep Neural Network (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. It provides highly tuned implementations of routines arising frequently in DNN applications.

Annoyingly, you have to create an Nvidia account if you want to download it from the official website Moving forward, after the cuDNN is downloaded, we can extract the files and folders from the archive so that we can have a folder called tools and add all the extracted files in there so the bin folder would have the next path

C:\tools\cuda\bin

Then, we have to add this C:\tools\cuda\bin to the System PATH environment variable in windows.

Follow this steps if you have not added the path to the bin folder to the environment.

Hit windows Key
Search for Environment variables then click Environment Variables on the window that have openend
In the System Variables find the PATH variable and Hit Edit
Then hit new in the new window that have openend and paste the path to the bin folder C:\tools\cuda\bin

Now you will have to reboot your PC, and hopefully all going to work just fine

Tensorflow

Now we can create a new environment for tensorflow and pip install tensorflow

conda create --name tf2 python=3.8

conda activate tf2

pip install tf-nightly-gpu

Then for the testing purpose

import tensorflow as tf

INFO:tensorflow:Enabling eager execution
INFO:tensorflow:Enabling v2 tensorshape
INFO:tensorflow:Enabling resource variables
INFO:tensorflow:Enabling tensor equality
INFO:tensorflow:Enabling control flow v2

print(tf.test.is_built_with_cuda())

True

assert tf.test.is_built_with_cuda()

with tf.device('/GPU:0'):
    a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
    b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
    c = tf.matmul(a, b)
c

<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[22., 28.],
       [49., 64.]], dtype=float32)>

sess = tf.compat.v1.Session(config=tf.compat.v1.ConfigProto(log_device_placement=True))

Device mapping:
/job:localhost/replica:0/task:0/device:GPU:0 -> device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:0b:00.0, compute capability: 8.6

tf.test.gpu_device_name()

'/device:GPU:0'

Pytorch

conda create --name ptorch python=3.8

conda activate ptorch

conda install pytorch -c conda-forge -c pytorch

Again let's test it

import torch

torch.cuda.is_available()

True

torch.cuda.get_device_name(device=None)

'NVIDIA GeForce RTX 3090'

torch.cuda.current_device()

0

torch.cuda.device_count()

1