My app has a leaderboard which is daily updated with a firebase function. Now I have more then 20.000 Users and the funciton times out always (already for 550 seconds).
exports.scheduledFunction = functions.pubsub.schedule('every day 00:00').timeZone('Europe/Vienna').onRun( async (contextm) => {
functions.logger.log('Positioning stated... (v.1.0.5)');
batch = db.batch();
const allUsers = await db.collection('users').orderBy('coins', 'desc').get();
let i = 0;
for (const doc of allUsers.docs) {
batch.update(doc.ref, "rank", i);
i += 1;
if(i % 499 == 0) {
await batch.commit();
batch = db.batch();
}
}
const writeResult = await batch.commit();
functions.logger.log('Updating Leaderboard');
return null;
});
How to optimize in terms of performance?
There's nothing you can do to improve the situation in terms of performance. The execution speed of your code is gated by the performance of Firestore. The Firestore operations will always take as long as they take - they can't be sped up.
The only alternatives you have within GCP are:
For #1, you can fire off other pubsub functions to handle batches of users asynchronously from the main function. You will have to package up a payload of data for the delegate function to work with, so that it doesn't have to repeat the first query. Each delegate invocation can update the documents in its own more predictable amount of time (as you will limit the size of each batch).
For #2, you could instead use Cloud Run, which gives you up to 60 minutes of time per invocation. Or you could set up a Compute Engine instance, which would run 24/7, and you would write code to invoked an exposed endpoint in your scheduled function.
If you maximized your runtimeOptions to the 540 seconds and 8 GB memory your only solution (I can recommend) would be to run that code on a trusted backend with the Firebase admin SDK. That is how we recaclulate our warehouse stocks every day. It takes about 30 min so it can't run on a Firebase cloud function.