I am trying to use svg filters to add a displacement map to an image. This is my code:

<svg width="300" height="300">
    <filter id="a" filterUnits="userSpaceOnUse" x="0" y="0" width="300" height="300">
      <feImage xlink:href="https://i.imgur.com/TKNyL9Q.jpg" result="heatmap"/>
      <feDisplacementMap scale="15" xChannelSelector="R" yChannelSelector="G" in2="heatmap" in="SourceGraphic"/>
    </filter>
    <image xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://s3-us-west-2.amazonaws.com/s.cdpn.io/222579/beagle400.jpg" height="300" width="300" filter="url(#a)" />
  </svg>

This is supposed to take the beagle image and apply a grid displacement map to it but it has no effect.

Oddly, switching in with in2 in the <feDisplacementMap> tag does work but in reverse showing a grid image with a beagle map fine.

Here is a codepen example.

Why does it only work one way?