For TensorFlow 2.4.1, the renaming hack will work if CUDA 11.2 needs to be installed. I suggest installing CUDA 11.0 + cuDNN 8.0.4 for TF 2.4.1, as @lineage wrote above, and then the renaming won't be necessary, and your GPU will be recognized.

For TensorFlow 2.5.0, I just got my GPU recognized using CUDA 11.2.2 + cuDNN 8.1.1. In that case, DO NOT rename the cusolver file. TF 2.5.0 expects the "cusolver64_11.dll" filename.

c> python
Python 3.9.4 | packaged by conda-forge | (default, May 10 2021, 22:10:34) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
2021-05-28 08:11:24.517894: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
>>> print(tf.version.VERSION)
2.5.0
>>> print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')),
...       '\nDevice: ', tf.config.list_physical_devices('GPU'))
2021-05-28 08:12:19.501812: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library nvcuda.dll
2021-05-28 08:12:19.530869: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: NVIDIA GeForce GTX 1080 with Max-Q Design computeCapability: 6.1
coreClock: 1.468GHz coreCount: 20 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 298.32GiB/s
2021-05-28 08:12:19.531377: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
2021-05-28 08:12:19.597785: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublas64_11.dll
2021-05-28 08:12:19.597992: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublasLt64_11.dll
2021-05-28 08:12:19.618849: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cufft64_10.dll
2021-05-28 08:12:19.634321: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library curand64_10.dll
2021-05-28 08:12:19.677539: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library **cusolver64_11.dll**
2021-05-28 08:12:19.731541: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cusparse64_11.dll
2021-05-28 08:12:19.746271: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudnn64_8.dll
2021-05-28 08:12:19.746674: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
Num GPUs Available:  1
Device:  [PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
>>>

I had the same problem. It turns out that CUDA 11.0 contains cusolver64_10.dll (that's probably why they indicate CUDA v11.0 in the tensorflow build guide here https://www.tensorflow.org/install/source_windows). Make sure to download cudnn as well!

TL;DR For TensorFlow ver >= 2.4.0 on Windows, install exactly those versions of CUDA Toolkit and cuDNN highlighted below i.e. those listed in the official requirements.(v11.0 as opposed to v11.2)

On Windows, the TensorFlow^ install requirements at the time of writing are as stated here

1. NVIDIA® GPU drivers —CUDA® 11.0 requires 450.x or higher.

2. CUDA® Toolkit —TensorFlow supports CUDA® 11 (TensorFlow >= 2.4.0)

3. CUPTI ships with the CUDA® Toolkit.

4. cuDNN SDK 8.0.4.

5. (Optional) TensorRT 6.0 to improve latency and throughput for inference on some models.

The problem you are facing has probably to do with the version of CUDA® Toolkit. Tensorflow is picky about the version of dependencies. Have a look inside C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin**. You should be able to find most^^ of the dlls needed by TensorFlow there. You may notice that it contains cusolver64_11.dll as opposed to the expected cusolver64_10.dll as stated in the output.

Though the renaming hack mentioned in an answer above works, it's not guaranteed to work reliably all the time. The simple and correct solution is to install the correct dependencies, to begin with.

At the time of writing the compatible versions of CUDA Toolkit and cuDNN are

CUDA Toolkit 11.0 (May 2020)
cuDNN v8.0.4 (September 28th, 2020), for CUDA 11.0 

from among the plethora of available versions of both, listed here & here.

More recent versions (I tested v11.0 onwards) aren't yet supported. I remember having the same problems with an earlier version of TensorFlow a few years back.

^ For ver >1.15, TensorFlow has GPU support included by default hence the CUDA requirements. When unavailable, TensorFlow works fine - it just reverts to CPU execution.
** Or wherever you installed the toolkit
^^ cudnn64_8.dll comes with cuDNN SDK

 Move to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin

Rename file cusolver64_11.dll  To  cusolver64_10.dll 

 cusolver64_10.dll