Solution provided below:

Started getting errors when building pygdal on Ubuntu 20.04:

# Command
python3 -m pip install --no-cache-dir pygdal==3.0.4.*

produces the following output

Collecting pygdal==3.0.4.*
  Downloading pygdal-3.0.4.6.tar.gz (458 kB)
     |████████████████████████████████| 458 kB 3.7 MB/s 
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-we7e68ge/pygdal/setup.py'"'"'; __file__='"'"'/tmp/pip-install-we7e68ge/pygdal/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-we7e68ge/pygdal/pip-egg-info
         cwd: /tmp/pip-install-we7e68ge/pygdal/
    Complete output (5 lines):
    /usr/lib/python3.8/distutils/dist.py:274: UserWarning: Unknown distribution option: 'use_2to3_fixers'
      warnings.warn(msg)
    /usr/lib/python3.8/distutils/dist.py:274: UserWarning: Unknown distribution option: 'use_2to3_exclude_fixers'
      warnings.warn(msg)
    error in pygdal setup command: use_2to3 is invalid.
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

The steps to reproduce the error are:

# Commands to run inside of running Ubuntu 20.04 docker instance
apt update
# install python3.8 and link python3 to this version - these next 2 steps may not be needed for your scenario
apt install -y python3.8
ln -s /usr/bin/python3.8 /usr/bin/python3
# done installing python3.8
apt install -y python3-pip
apt-get install -y python3-gdal gdal-bin libgdal-dev gcc g++ python3.8-dev git
python3 -m pip install --upgrade --no-cache-dir setuptools==58.0.4
python3 -m pip install --upgrade --no-cache-dir numpy wheel requests
python3 -m pip install --no-cache-dir pygdal==3.0.4.*

The solution is to install an older version of setuptools. Specifying version 58.0.2 and earlier resulted in a successful build. I only went back to 58.0.0 for this issue, but this is the first time I've seen this issue so earlier versions should work as well.

# Replace the above installation of setuptools with the following
python3 -m pip install --upgrade --no-cache-dir setuptools==58.0.2