I have configured a pod as follows:
livenessProbe:
initialDelaySeconds: 60
periodSeconds: 10
failureThreshold: 3
readinessProbe:
initialDelaySeconds: 60
periodSeconds: 10
failureThreshold: 3
Readiness and Liveness probes are tcp-socker based
readinessProbe:
tcpSocket:
port: {{ .Values.port }}
initialDelaySeconds: {{ .Values.readinessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.readinessProbe.periodSeconds }}
failureThreshold: {{ .Values.readinessProbe.failureThreshold }}
However, when I deploy, the pod is marked almost immediately as failed and goes in Error --> CrashLoopBackOff.
The pod checks a redis connection (which does indeed need a little time to become ready).
And of course I see the connection errors in my pod's logs
File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 563, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to redis-master:6379. Connection refused.
Why is the pod being marked in Error / CLBO so eagerly, way before initialDelaySeconds: 60?
Here is the pod's yaml dump regarding the probes (I increased the initialDelaySecond to both probes to 100, but still the same...)
livenessProbe:
failureThreshold: 3
initialDelaySeconds: 100
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 9898
timeoutSeconds: 1
name: mycontainer
ports:
- containerPort: 9898
protocol: TCP
readinessProbe:
failureThreshold: 3
initialDelaySeconds: 100
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 9898
timeoutSeconds: 1
initialDelaySeconds: Number of seconds after the container has started before liveness or readiness probes are initiated.
Now pod can get failed because of CrashLoopBackOff even before starting the probes. This is the concept here. It can occur if you set the container restartPolicy to Never.
You can see the pod logs or events for getting the reason of pod failure (can use kubectl describe <pod>)