Tag: NVIDIA

This tutorial will show you how to install TensorFlow with GPU support on Windows. You will need an NVIDIA graphics card that supports CUDA, as TensorFlow still only officially supports CUDA (see here: https://www.tensorflow.org/install/gpu).

If you are on Linux or macOS, you can likely install a pre-made Docker image with GPU-supported TensorFlow. This makes life much easier. See here for details (this article is about a year old, so a few things might be out of date). However, for those of us on Windows, we need to do things the hard way, as there is no NVIDIA Docker support on Windows.

See this article if you would like to install TensorFlow on Windows without GPU support.

[Update February 13, 2022] Updated some screenshots and a few commands to ensure that everything works with TensorFlow 2.7.0.

Prerequisites

To start, you need to play the version tracking game. First, make sure your graphics card can support CUDA by finding it on this list: https://developer.nvidia.com/cuda-gpus.

For example, my laptop has a GeForce GTX 1060, which supports CUDA and Compute Capability 6.1.

You can find the model of your graphics card by clicking in the Windows search bar and entering “dxdiag.” This tool will identify your system’s hardware. The Display tab should list your graphics card (if present on your computer).

Then, we need to work backwards, as TensorFlow usually does not support the latest CUDA version (note that if you compile TensorFlow from source, you can likely enable support for the latest CUDA, but we won’t do that here). Take a look at this chart to view the required versions of CUDA and cuDNN.

At the time of writing (updated Feb 13, 2022), this is the most recent TensorFlow version and required software:

Take a note of the particular required software versions listed for the particular TensorFlow version you wish to use. While you could compile TensorFlow from source to support newer versions, it’s much easier to install the specific versions listed here so we can install TensorFlow using pip.

Install Microsoft Visual C++ Compiler

The CUDA Toolkit uses the Microsoft Visual C++ (MSVC) compiler. The easiest way to install it is through Microsoft Visual Studio.

Download and install Visual Studio Community (which is free) from this site: https://visualstudio.microsoft.com/vs/community/. Yes, it’s a full IDE that we won’t use–we just need the compiler that comes with it.

[Update Feb 13, 2022] Note: at this time, the CUDA Toolkit installer will not find the latest version of Visual Studio Community (2022). You will need to install the older 2019 version by downloading from here.

Run the installer. You will be asked to install workloads. Click on the Individual components tab. Search for “msvc 2019” and select the latest MSVC C++ 2019 build tools version for your computer. For me, that was MSVC v142 – VS 2019 C++ x64/x86 build tools (Latest).

Click Install. When asked about continuing installation without a workload, click Continue. After installation is complete, you do not need to sign into Visual Studio. Simply close out all of the installation windows.

Install CUDA Toolkit

Navigate to the CUDA Toolkit site. Note the CUDA version in the table above, as it’s likely not the latest CUDA release. So, you’ll need to click on Archive of Previous CUDA Releases. Download the CUDA Toolkit version that is required for the TensorFlow version you wish to install (see the table in the Prerequisites section). For me, that would be CUDA Toolkit 10.1 update2 (Feb 13, 2022 update: CUDA Toolkit 11.2.2).

Download the installer for your operating system (which is probably Windows 10). I used the exe (network) installer so that it downloads only the required components.

Run the installer. It will take a few minutes to scan your system. Once scanning is done, accept the license agreement and select Custom (Advanced) install.

Deselect the components you don’t need. For example, we likely won’t be developing custom CUDA kernels, so deselect Nsight Compute and Nsight Systems. I don’t have a 3D monitor, so I’ll deselect 3D Vision. I’ll keep PhysX selected for gaming, but feel free to deselect it, as it’s not needed by TensorFlow. You can leave everything else selected.

Click Next. Leave the installation directories as default (if you wish) and click Next again to download and install all of the drivers and toolkit. This will take a few minutes. Close the installer when it finishes.

Install cuDNN

GPU-accelerated TensorFlow relies on NVIDIA cuDNN, which is a collection of libraries used to run neural networks with CUDA.

Head to https://developer.nvidia.com/rdp/cudnn-download. Create an NVIDIA Developer account (or login if you already have one). Ignore the cuDNN version listed in the TensorFlow version table (in the Prerequisites section). Instead, head to the cuDNN Archive and download the version that corresponds to the CUDA version you just installed.

For example, I just installed CUDA 11.2, so I’m going to download cuDNN v8.2.1 (which is the latest version that supports CUDA 11.x). Choose the cuDNN Library for your operating system (e.g. Windows).

The next part is a bizarre and seemingly old-fashioned method of installing a library. The full instructions can be found on this NVIDIA page (see section 3: Installing cuDNN on Windows).

Unzip the downloaded archive. Navigate into the unzipped directory and copy the following files into the CUDA installation directory (* being any files found with the listed file extension, and vxx.x is the CUDA version you installed).

Copy <cuDNN directory>\cuda\bin\*.dll to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\vxx.x\bin

Copy <cuDNN directory>\cuda\include\*.h to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\vxx.x\include

Copy <cuDNN directory>\cuda\lib\x64\*.lib to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\vxx.x\lib\x64

Next, we need to update our environment variables. Open Control Panel > System and Security > System > Advanced System Settings.

Click Environment Variables at the bottom of the window.

In the new window and in the System variables pane, select the Path variable and click Edit in the System variables pane.

You should see two CUDA entries already listed.

If you do not see these listed, add the following directories to this Path list (where vxx.x is your CUDA version number):

• C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\vxx.x\bin
• C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\vxx.x\libnvvp

Click OK on the three pop-up windows to close out of the System Properties.

Install TensorFlow

You can install TensorFlow any way you wish, but I highly recommend doing so through Anaconda. It makes installing Python, various packages, and managing environments much easier.

Head to anaconda.com. Download the Individual Edition for your operating system (Windows x64). Anaconda comes pre-packaged with Python set at a particular version, so you will need to note the version of Python required by TensorFlow in the Prerequisites section. For me, I need something between 3.7-3.9. As a result, Python 3.9 that comes with Anaconda (at the time of this Feb 13, 2022 update) works fine.

Run the Anaconda installer, accepting all the defaults.

When it’s done, run the Anaconda Prompt (anaconda3).

In the terminal, we’ll create a new Python environment, which will help us keep this version of TensorFlow separate from the non-GPU version. First, update conda with the following:

conda update -n base -c defaults conda

Then, enter the following commands to create a virtual environment:

conda create --name tensorflow-gpu
conda activate tensorflow-gpu

Install a version of Python supported by TensorFlow-GPU (as given by the table in the Prerequisites section) for your virtual environment (I’ll use Python version 3.9).

conda install python=3.9

Enter the following command to make sure that you are working with the version of Python you expect:

python --version

If you wish to also install Jupyter Notebook, you can do that with:

conda install jupyter

Rather than let pip try to figure out which version of TensorFlow you want (it will likely be wrong), I recommend finding the exact .whl file from TensorFlow’s site. Head to the TensorFlow Pip Installer page and look at the Package Location list.

Look under the Windows section for the wheel file installer that supports GPU and your version of Python. For me, this will be the wheel file listed with Python 3.9 GPU support. Note that GPU support (_gpu), TensorFlow version (-2.2.0), and supported Python version (-cp37) are listed in the filename. Highlight and copy the URL with the .whl file you want.

In Anaconda, enter the following command, replacing <wheel_url> with the URL that you copied in the previous step (i.e. paste it in).

python -m pip install <wheel_url>

Press ‘enter’ and let this run. It will take a few minutes.

When that’s done, go into the Python command line interface:

python

From there, enter the following commands (one at a time):

import tensorflow as tf 
print(tf.test.is_built_with_cuda()) 
print(tf.config.list_physical_devices('GPU'))

These will tell you if TensorFlow is capable of running on your graphics card. The first line imports TensorFlow, the second line makes sure it can work with CUDA (it should output “True”), and the third line should list the GPUs available to TensorFlow.

Note that if you see any weird errors about missing Windows DLL files (whether in the Anaconda prompt, within Python, or in Jupyter Notebook), try the following from within Anaconda:

python -m pip install pypiwin32

Close the Anaconda prompt.

Running TensorFlow

When you’re ready to do some machine learning stuff, open the Anaconda Prompt and enter the following:

conda activate tensorflow-gpu

From there, you can use Python in Anaconda or start a Jupyter Notebook session (see here for a good overview of how to work with Jupyter Notebook):

jupyter notebook

If you wish to install a new Python package, like matplotlib, you can enter the following into the Anaconda Prompt (make sure you are in your environment, tensorflow-gpu, and exit Jupyter Notebook by pressing ctrl+c):

python -m pip install <name_of_package>

Alternatively, you can install a package from within Jupyter Notebook by running the following command in a cell:

!python -m pip install <name_of_package>

For example, here is how I installed matplotlib:

Some libraries, like OpenCV, require you to install system components or dependencies outside of the Python environment, which means you can’t simply use pip. If so, check to see if the package is available in the conda-forge. If it is, you can install it in your Anaconda environment with:

conda install -c conda-forge <name_of_package>

Going Further

I hope this helps you get started using TensorFlow on your GPU! Thanks to Anaconda, you can install non-GPU TensorFlow in another environment and switch between them with the conda activate command. If the GPU version starts giving you problems, simply switch to the CPU version.

See the following videos if you are looking to get started with TensorFlow and TensorFlow Lite:

• Getting Started with TensorFlow and Keras
• Intro to TensorFlow Lite Part 1: Wake Word Feature Extraction
• Intro to TinyML Part 1: Training a Neural Network for Arduino in TensorFlow