One of the main features provided by spring framework for testing an application is the context caching mechanism to avoid exactly what you mention about the load overhead. The spring documentation says that:

Once the TestContext framework loads an ApplicationContext (or WebApplicationContext) for a test, that context will be cached and reused for all subsequent tests that declare the same unique context configuration within the same test suite.

With this affirmation in mind you have to understand how the cache mechanism works to determine the best strategy on build your tests. The question here is: When spring caches the context, it stores this context in memory using what key?. According to documentation, the key is based on some parameters of the container:

An ApplicationContext can be uniquely identified by the combination of configuration parameters that are used to load it. Consequently, the unique combination of configuration parameters are used to generate a key under which the context is cached. The TestContext framework uses the following configuration parameters to build the context cache key:

locations (from @ContextConfiguration)
classes (from @ContextConfiguration)
contextInitializerClasses (from @ContextConfiguration)
contextCustomizers (from ContextCustomizerFactory)
contextLoader (from @ContextConfiguration)
parent (from @ContextHierarchy)
activeProfiles (from @ActiveProfiles)
propertySourceLocations (from @TestPropertySource)
propertySourceProperties (from @TestPropertySource)
resourceBasePath (from @WebAppConfiguration)

Based on this information I may suggest you that the best practice is organize your tests in a way that they use the same set of context parameters (that is, the same cache key) to benefit from cache mechanism and avoid another context to be loaded. Spring documentation also gives an example:

..., if TestClassA specifies {"app-config.xml", "test-config.xml"} for the locations (or value) attribute of @ContextConfiguration, the TestContext framework will load the corresponding ApplicationContext and store it in a static context cache under a key that is based solely on those locations. So if TestClassB also defines {"app-config.xml", "test-config.xml"} for its locations (either explicitly or implicitly through inheritance) but does not define @WebAppConfiguration, a different ContextLoader, different active profiles, different context initializers, different test property sources, or a different parent context, then the same ApplicationContext will be shared by both test classes. This means that the setup cost for loading an application context is incurred only once (per test suite), and subsequent test execution is much faster.

Another trick that you can use in your integration tests is to force all the beans in the context to be "lazy". This is really useful when running just one integration test, as you do not have to wait for the entire application context to be loaded and initialized. This can significantly improve the time it takes to run a single test.

You may run into situations where beans are being implicitly created (Example: Spring IntegrationFlow). The flow is never directly injected into anything but your classes may have references to beans that the flow creates. In this case you either need to @Autowire your flow (to insure the implicit beans get created) or you can get creative with a BeanPostProcessor.

I created the following post processor and you just have to add it to your testing spring context.

public class LazyInitBeanFactoryPostProcessor implements BeanFactoryPostProcessor {

    private Class<?>[] exclusionList;

    public LazyInitBeanFactoryPostProcessor() {
    }

    public LazyInitBeanFactoryPostProcessor(Class<?>[] exclusionList) {
        this.exclusionList = exclusionList;
    }

    @Override
    public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {

        //Iterate over all bean, mark them as lazy if they are not in the exclusion list.
        for (String beanName : beanFactory.getBeanDefinitionNames()) {
            if (isLazy(beanName, beanFactory)) {
                BeanDefinition definition = beanFactory.getBeanDefinition(beanName);
                definition.setLazyInit(true);
            }
        }
    }

    private boolean isLazy(String beanName, ConfigurableListableBeanFactory beanFactory) {
        if (exclusionList == null || exclusionList.length == 0) {
            return true;
        }
        for (Class<?> clazz : exclusionList) {
            if (beanFactory.isTypeMatch(beanName,clazz)) {
                return false;
            } 
        } 
        return true;        
    }
}

And to use it:

@SpringBootTest(webEnvironment = WebEnvironment.NONE)
public class MyTest {
.
.
.
@TestConfiguration
protected static class TestConfiguration {

    @Bean
    public BeanFactoryPostProcessor lazyBeanPostProcessor() {
        return new LazyInitBeanFactoryPostProcessor();
    }
}

Or you extend it with exclusions (In this example, any bean that is assignable to a Spring Integration flow will NOT be marked as lazy:

@TestConfiguration
protected static class TestConfiguration {
    @Bean
    public BeanFactoryPostProcessor lazyBeanPostProcessor() {
        return new ExtendedTestLazyBeanFactoryPostProcessor();
    }


    static private class ExtendedTestLazyBeanFactoryPostProcessor extends LazyInitBeanFactoryPostProcessor {    
        public ServiceTestLazyBeanFactoryPostProcessor() {
            super(new Class<?>[] {IntegrationFlow.class});
        }   
    }