FIPS capable OpenSSL has one limitation - it must load libeay32.dll at fixed address and if loads at any other address, it fails initialization check, so it can't be used in FIPS mode.

So we chose address according to Microsoft's recommendation and on some machines that address from time to time is occupied by various other libraries - like MSVCR120_CLR0400.dll or mscorlib.ni.dll or clr.dll, you get the point.

Is there any way to check if some fixed address + length is taken and ask OS to free that part of memory for me, like rebase those dll's to other memory parts or something like this?

Update:

I've collected information from 20 devices with ListDLLs and there is some pattern what is loaded where, but it is far from well defined. So I've ran some math, found largest gap, where nothing was loaded in it in those 20 logs I had, changed libeay32 base address to somewhere in that gap (gap was ~6 times larger than dll, so I've chosen ~middle of it) and still after couple tries application managed to load something in that gap before libeay32 (to be specific - clrjit.dll, it has base address of 0x10000000, which I think is default), although in application I try to load libeay32 as soon as possible.