Figure out the issue on my end.

Previously I had installed XCode from the App Store (11.7) and set its SDKs as my default:

sudo xcode-select --switch /Applications/Xcode.app/

However, it seems this come with an unsupported version of clang:

  λ clang --version
Apple clang version 11.0.3 (clang-1103.0.32.62)
Target: x86_64-apple-darwin20.1.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Setting the xcode-select to the latest version via:

sudo xcode-select --switch /Library/Developer/CommandLineTools

EDIT (11/15/2020)

You might receive an error when attempting the above change:

xcode-select: error: invalid developer directory '/Library/Developer/CommandLineTools'

To fix this, you must install the latest Command Line Tools from the official Apple website here. At the time of writting this edit, I installed the Command Line Tools for Xcode 12.3 beta.

Changes clang to a working version:

  λ clang --version
Apple clang version 12.0.0 (clang-1200.0.32.2)
Target: x86_64-apple-darwin20.1.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

The built-in Big Sur SDK is version 10.15, which seems to work without an issue:

  λ ls /Library/Developer/CommandLineTools/SDKs
MacOSX.sdk      MacOSX10.15.sdk

After the switch, multidict was installed successfully.

  λ pip install multidict
Collecting multidict
  Downloading multidict-4.7.6-cp38-cp38-macosx_10_14_x86_64.whl (48 kB)
     |████████████████████████████████| 48 kB 589 kB/s
Installing collected packages: multidict
Successfully installed multidict-4.7.6

Further investigation seems to indicate this is a design choice by Apple (source):

Therefore, ensuring your SDK is the default out-of-the-box as opposed to XCode's new SDK should be enough for the system to switch context when needed (and seems to work fine with pip+clang).

I would like to extend @Felipe excellent answer; if it doesn't work even with running

>>> softwareupdate --all --install --force
Software Update Tool

Finding available software
No updates are available.

...following the wisdom of the "homebrew doctor" solves it, i.e. remove and reinstall:

sudo rm -rf /Library/Developer/CommandLineTools
sudo xcode-select --install

in my case I had to remove /Library/Developer/CommandLineTools and re-install command line tools

% sudo rm -rf /Library/Developer/CommandLineTools
% sudo xcode-select --install
% clang --version
Apple clang version 12.0.0 (clang-1200.0.32.27)
Target: x86_64-apple-darwin20.1.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin