tl;dr: How can I add the DateOnly JsonConverter to the application configuration of a .net6 web api?

I have an object with DateOnly properties on it and I'm returning it from the controller. When I make my request I get the error "Serialization and deserialization of 'System.DateOnly' instances are not supported".

I understand there are limitations with de/serialization of DateOnly and TimeOnly in .net6 (https://github.com/dotnet/runtime/issues/53539) and I followed the recommended use of a custom JsonConverter. However the problem I am having is where I initialize the converter.

I tried adding the converter in the web application builder....

builder.Services
    .AddControllers()
    .AddJsonOptions(options => options.JsonSerializerOptions.Converters.Add(new DateOnlyConverter()));

but no luck.

Then I tried configuring it specifically

builder.Services.Configure<JsonSerializerOptions>(options =>
    options.Converters.Add(new DateOnlyConverter()));

no luck there either.

So I tried returning Results.Json from the controller with the converter in the options Results.Json(status, options). Even that did not work.

What did work though was adding an attribute directly to the property

[property: JsonConverter(typeof(DateOnlyConverter))]
    DateOnly DateOrdered,

I would rather not have to add this attribute to every use of DateOnly and am really unclear as to what is preventing the converter from being called when it's configured as part of WebApplication. I've done similar things in .net5 applications, and assume some nuance between IHostBuilder and WebApplicationBuilder could be causing the issue.

Edit: It's probably worth noting I am using the new minimal apis. But even Microsoft points out Json serialization is configurable in the docs here. The docs show slightly different syntax than what is commonly used, builder.Services.Configure<JsonOptions>, so I assumed maybe this was the problem. Perhaps the minimal apis had configuration handled differently. But not so. At least, using the suggested configuration in the minimal api docs did not solve the problem.