I am using the service, that exports the bulk of data in JSONL format on the remote URL address. It can be a big file, so I want to read from it line by line to avoid memory issues. Additionally, it would be great to have the ability to resume on a specific line in case of some error.

I suspect that NodeJS Streams will be used here, but I am confused about how to do it. All examples I am finding are using filesystem streams.

For the network requests, I am using the superagent library which can pipe the data. I also found that for reading stream line by line it's good to use readline NodeJS module. But I just don't see how to put these two pieces of code together.

superagent.get(URL).pipe(...)

const rl = readline.createInterface({
  input: /* what goes here ? */,
  output: process.stdout,
  terminal: false
})

And it's not clear if this approach will allow me to implement the resume feature somehow.