This is happening because scrollHeight taking padding: 5px; as a content which is increasing scroll height of textarea

The Element.scrollHeight read-only property is a measurement of the height of an element's content, including content not visible on the screen due to overflow.

The scrollHeight value is equal to the minimum height the element would require in order to fit all the content in the viewpoint without using a vertical scrollbar. It includes the element's padding, but not its border or margin.

MDN

With border-box textarea's height is 100px excluding padding so scrollheight is 100px.

With content-box textarea's height is 100px + 10px as per default behavior of content-box so scrollheight is 110px, with each keydown textarea increases it's height by 10px and updated scrollheight as well.

See snippet Below

$("#message-box").on('keydown', function() {
  console.log("height of teaxtare on keydown is " + $("#message-box").height() + "px");
  var scroll_height = $("#message-box").get(0).scrollHeight;

  console.log("Scroll Height of textarea is " + scroll_height + "px");
  $("#message-box").css('height', scroll_height + 'px');
  console.log("After setting scroll_height as a height of teaxtare, teaxtare's height is " + $("#message-box").height() + "px");
});

#message-box {
  border: 1px solid #cccccc;
  width: 400px;
  min-height: 100px;
  padding: 5px;
  overflow: hidden;
}

<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js">
</script>
<textarea id="message-box"></textarea>

EDIT

Let say

S = 110px (scrollheight + padding:5px;)

H = Height of textarea.

Now you presses key,

Key Event 1,
S = 110px so
H = 110px,
____

Key Event 2,
S = 120 // 110px (which is increased height of textarea by this function ($("#message-box").css('height', scroll_height + 'px');)) + Padding (Which is 10px) 
H = 120px,
 ____

Key Event 3,
S = 130 // 120px (which is increased height of textarea by this function ($("#message-box").css('height', scroll_height + 'px');)) + Padding (Which is 10px) 
H = 130px,

And So On

This formation is sort of loop.

In your JQuery, you can Use:

this.style.height = "1px";
this.style.height = (this.scrollHeight) + "px";

Please try the following:

$("#message-box").on('keydown', function() {
    this.style.height = "1px";
    this.style.height = (this.scrollHeight) + "px"; 
});

#message-box {
    border: 1px solid #cccccc;
    width: 400px;
    min-height: 100px;
    padding: 5px;
    overflow: hidden;
}

<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<textarea id="message-box"></textarea>

In first loop iteration the CSS height was 100px but scrollHeight was 110px. These two are different things. Again in the 2nd loop iteration, scrollHeight was 120px and the CSS height before the last line of the function was 110px. After the last line the CSS height changed to 120px.

this.style.height = (this.scrollHeight) + "px";

scrollHeight is real height of the element. CSS height is the real height with border-box and not with content-box.