It's possible to improve on your string-manipulation of dates, by doing a simple comparison as follows:

dateClick: function (info) {
  var clickedDate = getDateWithoutTime(info.date);
  var nowDate = getDateWithoutTime(new Date())
  if (clickedDate >= nowDate) alert("Future date");
  else alert("Past date");
}

It is helped by this function:

//get date without the time of day
function getDateWithoutTime(dt)
{
  dt.setHours(0,0,0,0);
  return dt;
}

Demo: https://codepen.io/ADyson82/pen/ZErwJGx

One other option to achieve the same goal is to use the validRange functionality. This lets you dynamically set a start date, before which all dates are disabled:

validRange: function (nowDate) {
  return {
    start: nowDate
  };
},

Demo: https://codepen.io/ADyson82/pen/VwQgWJO

The slight downside of this though is that it also hides any events which occur before the current date - you may not want this.

Therefore, another alternative is to use selectAllow to restrict where the user can select. Note that this relies on you using the select callback rather than dateClick - but that should not be a big problem.

selectAllow: function (info) {
  return (info.start >= getDateWithoutTime(new Date()));
}

Demo: https://codepen.io/ADyson82/pen/VwQgzZJ

Below is just my current solution to disable past date on FullCalendar. Any idea or improvement would be appreciate to improve the code below.

dateClick: function(e) {

    // Get Today
    var dt = calendar.getDate();
    var year = dt.getFullYear();
    var month = dt.getMonth() + 1;
    var day = dt.getDate();

    if(month < 9){
        month = '0'+month;
    }

    if(day < 9){    
        day = '0'+day;
    }
    
    var today = year + '-' + month + '-' + day;

    // Get Selected Date
    var check = e.dateStr;
                
    let A = today.toString();
    let B = check.toString();

    alert('Date A: ' + A );
    alert('Date B: ' + B );

    if(B >= A){
        alert('Past Date');
    }

    else{
        alert('Future Date');
    }
},