I am trying to understand what is the difference between the Async library mapLimit/queue methods and this aysnc/await batched request function.
For this example let's say an API has a limit of 20 request per second. I am using this script. My intention is to do the maximum allowed request per second simultaneously.
Am I achieving that with this script?
const data = Array(200).fill('');
const concurrent = 20;
let requests = 0;
const start = new Date();
(async () => {
while (data.length) {
// Batched concurret request per second
await new Promise((resolve) => setTimeout(resolve, 1000));
const batch = data.splice(0, concurrent).map(API);
const results = await Promise.allSettled(batch);
// For testing
const timeTotal = Math.round((requests / (new Date() - start)) * 1000);
console.log(`Requests per second (total): ${timeTotal}/${data.length}`);
// Deal with errors
for (const { status, reason } of results) {
if (status === 'rejected') {
console.error(`There was an error ${reason}`);
}
}
}
})()
async function API() {
requests++;
await new Promise((resolve) => setTimeout(resolve, 0));
}Are the aforementioned methods doing the same thing?
Thanks.
Batching does create "peaks" of up to 20 concurrent requests, but waits for all requests of the current batch before starting the next batch. So if 1 of the 20 takes very long it will have only that running.
In contrast, solutions like mapLimit start 20 requests, and then 1 additional for each that finishes, so there are always 20 requests in flight at the same time.
I've managed to do the same thing I was doing with batches but with pool requests.
I could not find anything that did the same, all the async pool libraries use Promise.all() I really wanted to use Promise.allSettled() to take advantange of the oportunity to rerun a request on a given condition.
I have tried p-limit but in the end I went for tiny-async-pool and modified the source code to match my batch request:
export async function asyncPool(poolLimit, array, iteratorFn, exception) {
const promises = []
const racers = []
for (const [index, item] of array.entries()) {
process.stdout.write(`\rProcessing ${index + 1}/${array.length} ...`)
const pro = Promise.resolve().then(() => iteratorFn(item, array))
promises.push(pro)
if (poolLimit <= array.length) {
const racer = pro.then(() => racers.splice(racers.indexOf(racer), 1))
racers.push(racer)
if (racers.length >= poolLimit) {
await Promise.race(racers).catch(() => false)
}
}
}
const results = await Promise.allSettled(promises)
for (const { status, reason } of results) {
if (status === 'rejected') {
const { name, error } = reason
if (name === exception) await asyncPool(poolLimit, [error], iteratorFn)
}
}
}