I built a web scraper for ab white using puppeteer, i connected to a proxy server scraper api using the code below
const browser = await puppeteer.launch({
ignoreHTTPSErrors: true,
args: [
`--proxy-server=http://${PROXY_SERVER}:${PROXY_SERVER_PORT}`
]
});
I am also to authenticate the page as shown below.
await page.authenticate({
username: PROXY_USERNAME,
password: PROXY_PASSWORD,
});
Whenever I run my code without proxy, everything goes fine but my session gets blocked after many quick requests. Hence the scraper doesn't finish, but if I run it with the scraper it gets very slow , like it keeps spinning for 15 minutes, or it returns not secure error