The IEEE754 defines floating point numbers. The highest bit is used to define the sign with 0 being positive and 1 being negative.

Through a bit of digging, double.NaN seems to be represented as 0xFFF8000000000000 in binary (and 0.0 / 0.0 in code somehow).

double.IsNegative(double d) just checks the highest bit without any actual math being involved. Therefor, NaN is interpreted as a negative value. Meanwhile, double.NaN if used in a binary comparison will always yield false:

double.NaN < 0.0  //false
double.NaN > 0.0  //false
double.NaN <= 0.0 //false
double.NaN >= 0.0 //false
double.NaN == 0.0 //false

Well, according to refrence source, double.IsNegative just checks the most significant bit:

    [Pure]
    [System.Security.SecuritySafeCritical]  // auto-generated
    internal unsafe static bool IsNegative(double d) {
        return (*(UInt64*)(&d) & 0x8000000000000000) == 0x8000000000000000;
    }

In case of double.NaN the most signifucant bit is set:

    11111111 11111000 00000000 00000000 00000000 00000000 00000000 00000000   
    ||           ||                                                       |
    |<-   Exp   -><-                     Mantissa                        ->
    Sign

That's why double.IsNegative returns true

When we put < or > FPU commands are used which know all ones exponent is a special kind of floating point value which should be treated in a special way.

The very same picture is with Single.NaN.

Note, that we can construct another strange value, negative zero:

    10000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000   
    ||           ||                                                       |
    |<-   Exp   -><-                          Mantissa                   ->
    Sign

Please, look:

  double negativeZero = BitConverter.ToDouble(new byte[] { 
    0, 0, 0, 0, 0, 0, 0, 128
  });

  Console.WriteLine(negativeZero == 0 ? "Zero" : "???");

  Console.WriteLine(double.IsNegative(negativeZero) ? "Negative" : "???");