We are deploying Java microservices to AWS 'ECR > EKS' using helm3 and Jenkins CI/CD pipeline. However what we see is, if we re-run Jenkins job to re-install the deployment/pod, then the pod does not re-install if there are no code changes. It still keeps the old running pod as is. Use case considered here is, AWS Secrets Manager configuration for db secret pulled during deployment has changed, so service needs to be redeployed by re-triggering the Jenkins job.
I tried using 'helm upgrade --install --force ....' as suggested in helm3 upgrade documentation but it fails with below error in Jenkins log
"Error: UPGRADE FAILED: failed to replace object: Service "dbservice" is invalid: spec.clusterIP: Invalid value: "": field is immutable"
With 'helm upgrade --install --recreate-pods ....', I am getting below warning in Jenkins log
"Flag --recreate-pods has been deprecated, functionality will no longer be updated. Consult the documentation for other methods to recreate pods"
However, the pod gets recreated. But as we know --recreate-pods is not soft-restart. Thus we would have downtime, which breaks the microservice principle.
version.BuildInfo{Version:"v3.4.0", GitCommit:"7090a89efc8a18f3d8178bf47d2462450349a004", GitTreeState:"clean", GoVersion:"go1.14.10"}
This is nicely described in Helm documentation: https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
Below is how I configured it - Thanks to @vasili-angapov for redirecting to correct documentation section.
In deployment.yaml, I added annotations and rollme
kind: Deployment
spec:
template:
metadata:
annotations:
rollme: {{ randAlphaNum 5 | quote }}
As per documentation, each invocation of the template function randAlphaNum will generate a unique random string. Thus random string always changes and causes the deployment to roll.
The other way described in the document is with respect to a changing SHA value for a file.
In the past helm recommended using the --recreate-pods flag as another option. This flag has been marked as deprecated in Helm 3 in favor of the more declarative method above.