I've got a file of records (one per line):
Record1
Record2
Record3
I'm using the fs and the readline modules for that:
const rs = fs.createReadStream('input.txt');
const ws = fs.createWriteStream('output.json', { encoding: "utf8" });
const rl = readline.createInterface({
input: rs,
crlfDelay: Infinity,
});
ws.write('[');
for await (const line of rl) {
ws.write(`${line},\n`);
}
ws.end(']');
But this loop will also toss in a trailing comma at the end of the file which is bad cos trailing commas are not allowed in JSON.
The problem is that I don't want to read the whole file into memory and I don't know the number of records in advance.
Edit:
This is a simple example, but in the real code, the record gets transformed to a json record, but it's irrelevant.
By creating a local variable to track whether you've already encountered the first record, you can prepend the comma to all entries after the first one, achieving the same effect:
Note that, because whitespace is insignificant in JSON, you will decrease the size of the resulting file by omitting the newlines
ws.write('[');
let comma = false;
for await (const line of rl) {
if (!line.trim()) continue;
if (comma) ws.write(',');
ws.write(line);
comma = true;
}
ws.end(']');