I'm trying to test a method where I can try scraping files from a specific folder on a website, for example:

The content I want to crawl is in this specific folder: example.com/assets/filename-xxx.js There are 40-50 files such as this but they all have different numbers as the suffix.

How am I able to set a method using NodeJS by conditionally adding in the for loop to add a random number and see if there's a match to download the file, if there isn't a match it will skip and move to the next number, technically this will have to run 999 times at max.

Currently, I'm using website-scraper and the current code

const scrape = require('website-scraper'); 
const websiteUrl = 'http://example.com/assets/'; 
 
scrape({ 
    urls: [websiteUrl], 
    urlFilter: function (url) { 
        return url.indexOf(websiteUrl) === 0; 
    }, 
    recursive: true, 
    maxDepth: 50, 
    prettifyUrls: true, 
    filenameGenerator: 'bySiteStructure', 
    directory: './node-website' 
}).then((data) => { 
    console.log("Entire website succesfully downloaded"); 
}).catch((err) => { 
    console.log("An error ocurred", err); 
}); 

However, this doesn't work because it only follows links, any ideas on a solution?