I am quite new to web development and I have some trouble figuring out how to create a bot that clicks for me.
I have a website with a bunch of links that I want to click. I can easily identify them by building a web scraper. I want to create a bot that clicks the links every day at 12:00 AM without my intervention.
Since I won't always be online at the time, I wish to know if there is a way I can create a bot and host it, that can visit the site, authenticate, and do the clicking. All handled by the bot on the server without me having to open any browser tab.
In your reply, you can assume I am going to code the bot in Javascript or Python.
Thanks.
What you are looking for is a server-side (headless) API to control a web browser.
Some examples include puppeteer, in NodeJS and selenium in Python.
On their GitHub, you have some quick and basic examples regarding how to use the tool: https://github.com/puppeteer/puppeteer.
You can install it using npm. A basic example would be:
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({ path: 'example.png' });
await browser.close();
})();
I think that the easiest thing you can do is install selenium IDE into your browser.
save all the side files with your tests directly from the browser extension
Once you did that copy it on your server and install command-line-runner
https://www.selenium.dev/selenium-ide/docs/en/introduction/command-line-runner
Then you can setup your cron to launch the tests