Question

0

47

Views

Best Practices for reading csv data with d3 in node. Why gettting a TypeError?

I am trying to read a csv file with d3 in a node-environment. I am kind of confused what the "best-practice"-way therefore is and why I keep on getting a TypeError. I know that for some module of d3a environment that supports fetch must be present. Therefore my code at the moment looks like this:

import * as d3 from "d3";
import fetch from "node-fetch";
global.fetch = fetch;

let data_path = "flightCodes.csv";


// would a d3.csvParse(fs.readFileSync(data_path)) be better?
let data = await d3.csv(data_path, function (err, dat) {
  if (err) {
    console.log(err);
  } else {
    console.log(dat);
  }
});

console.log(data);

The csv file lives in the same folder and looks like this:

orig,dest,orig_codes,dest_codes
Paris,Lyon,"BVA, ORY, CDG",LYS
Paris,Bordeaux,"BVA, ORY, CDG",BOD
Paris,Nantes,"BVA, ORY, CDG",NTE
Madrid,Barcelona,MAD,BCN
Lisboa,Porto,LIS,OPO

My package.json looks like this:

{
  "name": "name",
  "version": "1.0.0",
  "description": "",
  "type": "module",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "d3": "^7.2.0",
    "d3-fetch": "^3.0.1",
    "node-fetch": "^3.1.0"
  }
}

When I do a node script.js, I get the following and I do not completely understand why. Anybody might have an idea?

node:internal/errors:464
    ErrorCaptureStackTrace(err);
    ^

TypeError [ERR_INVALID_URL]: Invalid URL
    at new NodeError (node:internal/errors:371:5)
    at onParseError (node:internal/url:552:9)
    at parse (<anonymous>)
    at new URL (node:internal/url:632:5)

about 1 month ago ·

Juan Pablo Isaza

1 answers

Answer question

0

Answer question

Find remote jobs

Loading

Answer 1 · 2022-07-08T21:53:34.444Z

If the file you're referencing is not available through a URL, then d3.csv (which uses fetch) cannot find it. Internally, d3.csv fetches the data and then calls d3.csvParse to parse it, so you can do the same!

let data = d3.csvParse(fs.readFileSync(data_path));

In addition, there was another problem with your code. The signature d3.csv(path, callback) is deprecated. Instead, it will execute the second argument on every row as a parse function. The correct way to call it would be d3.csv(path).then(callback) or

const data = await d3.csv(path);
callback(data);