I'm using airflow and the dockerOperator to connect to a docker daemon and spin up a container.
As part of my pipeline this container needs to decrypt a file using gpg.
If I copy the gpg key in during docker build then it will be apart of the image forever, this feels insecure?
I have investigated whether or not I can put the key into an environment variable and pass it in that way i.e. through dockers -e VAR:VAL syntax. The only other way I can think of is to mount my local .gnupg file into the container and use that, however this will only work while I'm on my local machine. I want to be able to migrate to ECS or kubernetes at some stage.
You are correct in saying, that adding the key during docker build is insecure.
The env var is a valid way. If you use kubernetes later, you can safely save the your key as a secret in kubernetes. When deploying you can provide a secret via env to a container.
You could also provide a secret as a file from a kubernetes secret by using a volume, and mount a volume locally and provide the key file.
These are the official docs for kubernetes secrets: https://kubernetes.io/docs/concepts/configuration/secret/
I am not familiar with ECS, but I am positive that there are similar ways.