There's nothing to convert, you just need to output that particular markup on a page.

There are many ways to write page-level components and implementations often have at least several of those components in use. You would have to tweak the code responsible for linking to client side libraries. I believe this amount of flexibility is not built into the mechanisms AEM uses to include those. You'd probably have to replace the standard JSP tags or HTL templates with a bit of custom implementation, depending on how your page-level components have been written.

You should identify the page-level components behind each of the templates used on your site and adjust them. If you have a greenfield project, the job's easier as you can start off with this approach.

See https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/aem-6-4-clientlib-javascript-with-async-and-defer-css-with/m-p/323641 for a few suggestions that might be relevant.

Your job could be easier if you're using AEM Core Components, which aren't part of the product, but an open-source project managed by Adobe and the AEM community. There appears to be some extra functionality around this than in vanilla AEM. I haven't used any of those features though and I'm not sure how close to production-readiness they are.

This issue mentions preloading scripts, which implies some degree of support https://github.com/adobe/aem-core-wcm-components/issues/1823

Here's the documentation for the latest iteration of the Page component https://github.com/adobe/aem-core-wcm-components/blob/main/content/src/content/jcr_root/apps/core/wcm/components/page/v3/page/README.md