I am currently having trouble trying to pull my remote docker image hosted via AWS ECR. I am getting this error when running a deployment
Step 1)
run
aws ecr get-login-password --region cn-north-1 | docker login --username AWS --password-stdin xxxxxxxxxx.dkr.ecr.cn-north-1.amazonaws.com.cn
Step 2)
run kubectl create -f backend.yaml
from here the following happens:
➜ backend git:(kubernetes-fresh) ✗ kubectl get pods
NAME READY STATUS RESTARTS AGE
backend-89d75f7df-qwqdq 0/1 Pending 0 2s
➜ backend git:(kubernetes-fresh) ✗ kubectl get pods
NAME READY STATUS RESTARTS AGE
backend-89d75f7df-qwqdq 0/1 ContainerCreating 0 4s
➜ backend git:(kubernetes-fresh) ✗ kubectl get pods
NAME READY STATUS RESTARTS AGE
backend-89d75f7df-qwqdq 0/1 ErrImagePull 0 6s
➜ backend git:(kubernetes-fresh) ✗ kubectl get pods
NAME READY STATUS RESTARTS AGE
backend-89d75f7df-qwqdq 0/1 ImagePullBackOff 0 7s
So then I run kubectl describe pod backend and it will output:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 117s default-scheduler Successfully assigned default/backend-89d75f7df-qwqdq to minikube
Normal Pulling 32s (x4 over 114s) kubelet, minikube Pulling image "xxxxxxxxx.dkr.ecr.cn-north-1.amazonaws.com.cn/baopals:latest"
Warning Failed 31s (x4 over 114s) kubelet, minikube Failed to pull image "xxxxxxxxx.dkr.ecr.cn-north-1.amazonaws.com.cn/baopals:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://xxxxxxxxx.dkr.ecr.cn-north-1.amazonaws.com.cn/v2/baopals/manifests/latest: no basic auth credentials
Warning Failed 31s (x4 over 114s) kubelet, minikube Error: ErrImagePull
Warning Failed 19s (x6 over 113s) kubelet, minikube Error: ImagePullBackOff
Normal BackOff 4s (x7 over 113s) kubelet, minikube Back-off pulling image "xxxxxxxxx.dkr.ecr.cn-north-1.amazonaws.com.cn/baopals:latest"
the main error being no basic auth credentials
Now what I am confused about is that I can push images to my ECR fine and I can also push to my remote EKS cluster I feel like essentially the only thing I cant do right now is pull from my private repository that is hosted on ECR.
Is there something obvious that I'm missing here that is preventing me from pulling from private repos so i can use them on my local machine?
For fetching ECR image locally you have login to ECR and fetch docker image. while if you are on Kubernetes you have to use secret for storing ECR login details and use it each time for pulling image from ECR.
here shell script if you are on Kubernetes, it will automatically take values from AWS configuration or else you can update variables at starting of script.
ACCOUNT=$(aws sts get-caller-identity --query 'Account' --output text) #aws account number
REGION=ap-south-1 #aws ECR region
SECRET_NAME=${REGION}-ecr-registry #secret_name
EMAIL=abc@xyz.com #can be anything
TOKEN=`aws ecr --region=$REGION get-authorization-token --output text --query authorizationData[].authorizationToken | base64 -d | cut -d: -f2`
kubectl delete secret --ignore-not-found $SECRET_NAME
kubectl create secret docker-registry $SECRET_NAME \
--docker-server=https://$ACCOUNT.dkr.ecr.${REGION}.amazonaws.com \
--docker-username=AWS \
--docker-password="${TOKEN}" \
--docker-email="${EMAIL}"
imagePullSecret used in YAML file for pulling storing secret for private docker repos.
https://github.com/harsh4870/ECR-Token-automation/blob/master/aws-token.sh
When a node in your cluster launches a container, it needs the credentials to access the private registry to pull the image. Even if you have authenticated in your local machine, the node cannot reuse the login, because by design it could be running on another machine; so you have to provide the credentials in the pod template. Follow this guide to do that:
https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
Basically you store the ECR credentials as a secret and provide it in the imagePullSecret of the container spec. The pod will then be able to to pull the image everytime.
If you are developing with your cluster running on local machine, you don't even need to do that. You can have the pod reuse the image that you have downloaded to your local cache by either setting the imagePullPolicy under container spec to IfNotPresent, or using a specific tag instead of latest for your image.