so I've been checking around a simple Tensorflow project as a starting point to learn, and found one that's very "casual" and short. It's an Upright Posture Checker, able to learn (add example) on the fly on runtime, and immediately use that new info to infer poses.
The demo website:
https://aaryanporwal.github.io/uprighthelper/
The repo:
https://github.com/aaryanporwal/uprighthelper
Relevant issue:
https://github.com/aaryanporwal/uprighthelper/issues/5
Once you run the program (and click Right/Wrong button), the memory usage rose sharply and steadily. Starting at 500MB, to 1.5GB in less than 1 minute, and up to 5GB in my test (whatever free RAM at that time). On average, it rises 100MB in 16s.
The main parts of the code.
const classifier = knnClassifier.create();
const webcamElement = document.getElementById('webcam');
let net;
var audio = new Audio('audio_file.mp3');
async function app() {
console.log('Loading mobilenet..');
// Load the model.
net = await mobilenet.load();
console.log('Sucessfully loaded model');
await setupWebcam();
// Reads an image from the webcam and associates it with a specific class
// index.
const addExample = classId => {
// Get the intermediate activation of MobileNet 'conv_preds' and pass that
// to the KNN classifier.
const activation = net.infer(webcamElement, 'conv_preds');
// Pass the intermediate activation to the classifier.
classifier.addExample(activation, classId);
};
// When clicking a button, add an example for that class.
document.getElementById('class-a').addEventListener('click', () => addExample(0));
document.getElementById('class-b').addEventListener('click', () => addExample(1));
while (true) {
if (classifier.getNumClasses() > 0) {
// Get the activation from mobilenet from the webcam.
const activation = net.infer(webcamElement, 'conv_preds');
// Get the most likely class and confidences from the classifier module.
const result = await classifier.predictClass(activation);
const classes = ['A', 'B'];
if(classes[result.classIndex]=="B"){
await audio.play();
document.body.style.backgroundColor = "rgb(168, 63, 63)";
}
else{
document.body.style.backgroundColor = "rgb(80, 168, 80)";
}
}
await tf.nextFrame();
}
}
// Webcam permission and stuff
// ...
app();