So I finally did do this and it worked successfully for me. The config got me for a bit but here it is. Say you subclass ResourceHttpRequestHandler with one named CustomResourceHandler. In my Application.java this wired it up properly:

@Bean
public ResourceHttpRequestHandler resourceHttpRequestHandler() {
    ResourceHttpRequestHandler requestHandler = new CustomResourceHandler();
    requestHandler.setLocations(Arrays.<Resource>asList(applicationContext.getResource("/")));
    return requestHandler;
}

@Bean
public SimpleUrlHandlerMapping sampleServletMapping(){
    SimpleUrlHandlerMapping mapping = new SimpleUrlHandlerMapping();
    mapping.setOrder(Integer.MAX_VALUE - 2);
    Properties urlProperties = new Properties();
    urlProperties.put("/**", "resourceHttpRequestHandler");
    mapping.setMappings(urlProperties);
    return mapping;
}

@Autowired
ApplicationContext applicationContext;

• This Answer helped me with the proper config of the mapper.
• This Answer about context helped me get the locations set properly in my handler. I was setting them a different way but it wasn't working 100% properly for all my resources