I would like to automatically generate some sort of log of all the database changes that are made via the Django shell in the production environment.
We use schema and data migration scripts to alter the production database and they are version controlled. Therefore if we introduce a bug, it's easy to track it back. But if a developer in the team changes the database via the Django shell which then introduces an issue, at the moment we can only hope that they remember what they did or/and we can find their commands in the Python shell history.
Example. Let's imagine that the following code was executed by a developer in the team via the Python shell:
>>> tm = TeamMembership.objects.get(person=alice)
>>> tm.end_date = date(2022,1,1)
>>> tm.save()
It changes a team membership object in the database. I would like to log this somehow.
I'm aware that there are a bunch of Django packages related to audit logging, but I'm only interested in the changes that are triggered from the Django shell, and I want to log the Python code that updated the data.
So the questions I have in mind:
pre_save signal for all model to know if data changes, but how do I know if the source was from the Python shell? How do I know what was the original Python statement?You could use django's receiver annotation.
For example, if you want to detect any call of the save method, you could do:
from django.db.models.signals import post_save
from django.dispatch import receiver
import logging
@receiver(post_save)
def logg_save(sender, instance, **kwargs):
logging.debug("whatever you want to log")
some more documentation for the signals
I would consider something like this:
Wrapping each python session with some sort initialisation code using e.g.
PYTHONSTARTUP environment variable
https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSTARTUP
In the file where PYTHONSTARTUP points to registering Exit handler using atexit
https://docs.python.org/3/library/atexit.html
These two things should allow you to use some lower level APIs of
django-reversion to wrap the whole terminal session with
https://django-reversion.readthedocs.io/en/stable/api.html#creating-revisions (something like this but calling __enter__ and __exit__ of that context manager directly in your startup and atexit code). Unfortunately I don't know the details but it should be doable.
In atexit / revision end calling the code to list the additional lines
of the terminal session and storing them somewhere else in the database with a reference to the specific revision.
See:
https://docs.python.org/3/library/readline.html#readline.get_history_length
https://docs.python.org/3/library/readline.html#readline.get_history_item
Basically, the idea is that you could call get_history_length twice: at the beginning and end of the terminal session. That will allow you to get relevant lines of where the change took place using get_history_item. You may end up having more lines of history than what you actually need but at least there is enough context to see what's going on.