I have been looking at other stack overflow questions but couldn't get any to work. I have a python script which uses environment variables. This script works exactly as planned when run directly however, I would like to run it as a cron job every minute for the time being.
Currently in my cron.d directory I have a file called scrapers containing:
* * * * * root /usr/bin/python3.5 /code/scraper.py
This runs the python script but the script fails as in the script I use two environment variables.
I read I should add SHELL=/bin/bash to the cron file so I did but this didn't help.
SHELL=/bin/bash
* * * * * root /usr/bin/python3.5 /code/scraper.py
Then I read
In the crontab, before you command, add . $HOME/.profile.
SHELL=/bin/bash
* * * * * . $HOME/.profile; root /usr/bin/python3.5 /code/scraper.py
but this caused the cron to stop running altogether. What is the best way of 'sending' the env variables to the cron?
Instead of executing the whole ~/.profile what I'd do is move the variables that must be shared between your cron jobs and the account that has the profile, then I'd source these both in ~/.profile and in the cron job.
The last attempt you show in the question is not properly formatted. The user id should be coming right after the scheduling information, but you've added the sourcing of the profile before the user id, which surely cannot work.
Here's an example setup that I've tested here:
*/1 * * * * someuser . /tmp/t10/setenv && /usr/bin/python /tmp/t10/test.py
I've set it to execute every minute for testing purposes. Replace someuser with something that makes sense. The /tmp/t10/setenv script I used had this:
export FOO=foovalue
export BAR=barvalue
The /tmp/t10/test.py file had this:
import os
print os.environ["FOO"], os.environ["BAR"]
My cron emails me the output of the scripts it runs. I got an email with this output:
foovalue barvalue
You can specify your two environment variables by this:
* * * * * root env A=1 B=2 /usr/bin/python3.5 /code/scraper.py
env is a system program that runs a specified program with additional variables:
$ env A=1 B=2 /bin/sh -c 'echo $A$B' # or just 'sh': would search in $PATH
12