We are trying to build a set of the same services, which are connected to different MQTT Brokers and different 3rd party Systems. Each service needs to have it's own certificate to be able to connect to it's own MQTT Broker.
To do so, we are using one Jenkins pipeline, which is pushing newly build Docker image to multiple target projects in OpenShift. Each project has it's own ConfigMap with connection/configuration details.
Now the problem is that we would like to do the same to PEM Certificates. Easiest and working way is to keep certs on repository, and import those in dockerfile. It's working fine, but it is not scaling up, when we want to add more PODs in OpenShift with same service, but different configurations.
To do it dynamically we thought about initContainers. This would allow us to send cert dynamically.
initContainers:
- name: inject-pem-to-cacerts
image: <imageName>
env:
- name: MQTT_CERT_PEM
value: home/fromMount/cert.pem
- name: CACERTS
value: $JAVA_HOME/jre/lib/security/cacerts
command: ['/bin/bash']
args: ['-c', "keytool -import -noprompt -keystore $CACERTS -file $MQTT_CERT_PEM -storepass changeit -alias service-$MQTT_CERT_PEM"]
In that case we are recieving PermissionDenied on that specific path. Sudo is not an option here as far as we tested. Maybe it's a matter of security context in YAML, but don't know what should be set.
If there is other way to do it better I would love to listen.
The problem you are facing stems from the fact that Init-Containers are not the right tool here. Init-Containers are started before the Pod containers, therefore the truststore you configure here, is going to be on the Init-Container and not on the application container. To achieve what you are trying to do, I recommend the PostStart Container Lifecycle Handler:
containers:
- name: my-container
lifecycle:
postStart:
exec: ['-c', "keytool -import -noprompt -keystore $CACERTS -file $MQTT_CERT_PEM -storepass changeit -alias service-$MQTT_CERT_PEM"]
Taking a step back, I wonder about one alternative solution. If I understood correctly, you need to install the cacerts for the server certs of the various MQTT brokers. So, since cacerts are not confidential, why not work with a singular Docker image that has all cacerts installed on them. That would take away a lot of complexity from K8s.