I am running a service that holds a large number of cronjobs (1000+).
The list of cronjobs can be updated multiple times a day. First I update a .txt-file with a list of cronjob-commands like this:
echo -e "`crontab -l`\n1 21 * * 1-5 /usr/bin/php /var/www/html/file_to_be_called.php 123" | crontab -
And when the .txt-file is updated, I run a PHP-script that takes the .txt-file and executes it as commands like this:
$list_of_cronjobs = file_get_contents("../list_of_cronjobs.txt");
exec($list_of_cronjobs);
This all works fine but the problem is the execution time of this setup. The exec($list_of_cronjobs);-line takes about 20 seconds with ~1000 lines in the .txt-file. This is alright for now but I expect to add more than 5000 lines of cronjobs within the next months and then the execution-time is a problem.
I'm curious how to improve and speed up this process. My script is running on an AWS EC2 server (t2.medium). Any suggestions to boost the exec()-process?