I might be wrong but I believe there is a memory leak when I try to insert about 300 rows every 3 seconds. I have tried it in the main thread and in a worker thread but both show a clear memory leak. I am not sure if this is related to SQLite or the driver.

I am reading a data from a sensor at 100 samples/second and would like to store that in a database. At first I thought it might be the processing of the data that causes the leak, but once I turned off everything and tried to insert some dummy data on an interval, I saw the same issue. My application runs on ElectronJS framework. I have already rebuilt the better-sqlite3 package for my current version of electron.

Here is a sample code that I run in a NodeJS worker thread that shows a clear memory leak.

const { isMainThread, workerData, parentPort } = require('worker_threads');
const path = require('path');
const SQLITE = require('better-sqlite3');
const dbPath = path.join(__dirname, 'mydb.db');

const db = new SQLITE(dbPath, {
  timeout: 1000,
});

db.exec('CREATE TABLE IF NOT EXISTS sensor_data (id INTEGER PRIMARY KEY AUTOINCREMENT, timeStamp INTEGER, PDRawData TEXT, LEDIntensities TEXT, gainValues TEXT, events TEXT, recordingId INTEGER);
');

const pragma1 = db.pragma('journal_mode = WAL');
const pragma2 = db.pragma('synchronous = normal');

const insert = db.prepare(
  'INSERT INTO sensor_data (timeStamp, PDRawData, LEDIntensities, gainValues, events, recordingId ) VALUES (@timeStamp, @PDRawData, @LEDIntensities, @gainValues, @events, @recordingId)'
);

const insertMany = db.transaction((data) => {
  for (const dataPoint of data) {
    insert.run(dataPoint);
  }
});

const sampleData = {
  timeStamp: 0,
  PDRawData: '12313,123123,12313,1243535,3456464,213132,42342',
  LEDIntensities: '123,435,123,4345,123,3435,132,345',
  gainValues: null,
  events: null,
  recordingId: null,
};

const myData = Array(50).fill(sampleData);

setInterval(() => {
  insertMany(myData);
}, 500);

Other possible solutions that I have tried

I tried exposing the global.gc() and running it after every insert of the data. I also tried running this script on the main thread and still got the same result. I have tried recreating my database file many times, but that doesn't help either. I am not sure what I am doing wrong here. I would really appreciate if you could help out with this matter.

Here's my setup: OS: Windows 10, Node v16.9.1 Electron Version: 16.0.8 Better-sqlite3 Version: 7.5.0 (I have also tested previous versions such as 7.4.0)