Is it possible to access Azure App Configuration via a Managed Identity locally (or in a pipeline) without deploying any other services to Azure?

I have a .net core unit test project that runs some integration tests against a live site. In order to keep various settings, secrets etc out of the source code I figured I might be able to use Azure App Configuration.

I was reading this guide but it seemed to be aimed at an App Service accessing an App Configuration https://docs.microsoft.com/en-us/azure/azure-app-configuration/howto-integrate-azure-managed-service-identity?tabs=core2x

I've enabled the system assigned managed identity

[]1]

And am attempting to use the following code:

var builder = new ConfigurationBuilder();
builder.AddAzureAppConfiguration(options =>
{
    options.Connect(new Uri("https://sauitest-config.azconfig.io"), new DefaultAzureCredential())
        .Select(KeyFilter.Any, "TestLocal");
});

Unfortunately this returns a 403 (Forbidden). Note I am logged into Visual Studio as an admin with the same credentials that I use to access the portal. Apparently DefaultAzureCredential should work in both local and azure environments.

Am I missing something here?