I have problem at reading a file just after appending lines in it. It seems that when inserting less than 913742 characters, it's ok, but when inserting more than 914534 characters, the next lines are read as blank lines (the buffer still empty).

import { promises as fs } from "fs";

const filepath = 'any.file';

const readChunk = async (start: number, end: number) => {
  const buffer = Buffer.from("".padStart(end, " "));
  const fd = await fs.open(filepath, "r");
  await fd.read({ buffer, offset: 0, position: start, length: end });
  await fd.datasync();
  await fd.close();
  return buffer.toString();
};

const append = async (line: string) => {
  const fd = await fs.open(filepath, "a");
  await fd.appendFile(chunk);
  await fd.datasync();
  await fd.close();
};

My jest test:

import faker from "faker";

const fakeUser = (): UserDatas => ({
  id: Math.round(Math.random() * MAX_SAFE_INTEGER),
  message: faker.lorem.paragraphs(),
});

const fakeUsers = (count: number) =>
  Array.from({ length: count }).map(fakeUser);

describe("large amount of data", () => {
  it("should accept thousand lines", async () => {
    const instance = getInstance();
    await instance.append(
      fakeUsers(faker.datatype.number({ min: 9000, max: 10000 }))
    );
    expect(await instance.count()).toBeGreaterThan(8000);
  });
});

The output:

  ● BsonFile › large amount of data › should accept thousand lines

    expect(received).toBeGreaterThan(expected)

    Expected: > 8000
    Received:   1364

      178 |         fakeUsers(faker.datatype.number({ min: 9000, max: 10000 }))
      179 |       );
    > 180 |       expect(await instance.count()).toBeGreaterThan(8000);
          |                                      ^
      181 |     });
      182 |   });
      183 | });

The reason why I can not count more 1364 is that the file does not output any data.

Also, I hoped that fd.datasync() would resolve my problem, but it don't: should I let it there? Is this usefull?