I have a pandas array with np.NaN's in it that I convert to a dictionary for JSON
json_data = {"freq" : list(df["Data"])}
I get ready to load to a web page via flask jsonify
@dashboard.route('/command', methods = ['GET', 'POST'])
def command():
[...]
return jsonify(json_data)
and on the javascript side I get ready to read it.
$.ajax({
url: '/dashboard/command',
type: 'POST',
data: data,
contentType: 'application/json; charset=utf-8',
dataType: 'json',
success: function(msg) {
if (!msg.error) {
updatePlot(msg);
}
else {
alert(msg.error);
}
},
error: function (XMLHttpRequest, textStatus, errorThrown) {
alert("There has been an error retrieving your data: \n\n" + errorThrown);
},
complete: function() {
$("#update-plot").prop('disabled', false); // Re-enable button
$("#update-plot-loading").addClass('invisible'); // Hide loading animation
}
And that is where it dies if I load that web page. It says that it has a bad token "N" in the json. If I replace the NaN's before trying to send, e.g.
df["Data"].replace(np.NaN, -999.999)
and all is fine and the javascript can continue. I'm aware that the NaN's should be "null" on the javascript side, so what I do at the moment (ugh) is then convert the -999.999's to "null"... But gosh, shouldnt there be a way to send missing data directly?
Any suggestions? I have tried many combinations of NaN, "NaN", None, etc. But maybe I missed one. I checked the output of jsonify on the python said and it seems to be a valid response to send over.
Thanks, T.
Aha. Progress, sort of. At least on the Flask side, maybe not Ajax. Consider a Pandas with some np.Nan in it... Convert to a dict or whatever and then call render_template
s = render_template('dashboard/dashboard_horizontal.html',
average_data = average_data)
This is where the damage gets done.... but it is fixable with
s = s.replace("nan", "NaN") #this is dumb but works!!
A related problem ocurse with false in json. When you read it into Python, the parser conveniently turns it into False. Except now you created problem on the Javascript side, but it is also fixable the same way.
s = s.replace("False", "false") #this is dumb too
I dont know if there are some keywords that could be passed to render_template to do this automatically?
Ok, an I have an Ajax solution for NaN too, which is the JSON5 idea explained here : How to parse JSON string containing "NaN" in Node.js . Specifically, dont use dataType json, go with 'text' and do the parsing by hand.
$.ajax({
url: '/dashboard/command',
type: 'POST',
data: data,
contentType: 'application/json; charset=utf-8',
dataType: 'text', // it was 'json'
//async: false,
success: function(msg) {
if (!msg.error) {
var data = json5.parse(msg); // json5 so much more friendly
updatePlot(data);
}
else {
alert(msg.error);
}
},
error: function (XMLHttpRequest, textStatus, errorThrown) {
alert("Error:\n\n" + errorThrown);
},
complete: function() {
$("#update-plot").prop('disabled', false); // Re-enable button
$("#update-plot-loading").addClass('invisible');
}
});
}