I'm trying to redeploy from GKE to Digital Ocean. I'm running into an issue with the challenge from letsencrypt. I believe K8s is telling me that the route cannot be found. The two hostnames/domains that letsencrypt is trying to do the challenge and failing for are CNAMES used by SendGrid. I'm not really sure where to start troubleshooting, my google-fu is failing me.
Namespace: default
Labels: <none>
Annotations: <none>
API Version: acme.cert-manager.io/v1alpha2
Kind: Challenge
Metadata:
Creation Timestamp: 2020-03-19T20:34:04Z
Finalizers:
finalizer.acme.cert-manager.io
Generation: 1
Owner References:
API Version: acme.cert-manager.io/v1alpha2
Block Owner Deletion: true
Controller: true
Kind: Order
Name: letsencrypt-certs-80407504-346698183
UID: 84ab9399-3a61-462e-a1c5-0831bd451a36
Resource Version: 37060
Self Link: /apis/acme.cert-manager.io/v1alpha2/namespaces/default/challenges/letsencrypt-certs-80407504-346698183-813483524
UID: ca14d996-3ecf-4dd8-8c53-057af7ae2b27
Spec:
Authz URL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/3449670327
Dns Name: 7502121.secodify.com
Issuer Ref:
Group: cert-manager.io
Kind: ClusterIssuer
Name: letsencrypt-prod
Key: LiAawBR0bFRQfb2oXrvvNhph3ehQ-35lXJKkpjqgqb0.uWH4RnJfcABYba9T5b-QjoYnIw53rRtVhzsRIHIh39Y
Solver:
http01:
Ingress:
Class: nginx
Token: LiAawBR0bFRQfb2oXrvvNhph3ehQ-35lXJKkpjqgqb0
Type: http-01
URL: https://acme-v02.api.letsencrypt.org/acme/chall-v3/3449670327/JGayWw
Wildcard: false
Status:
Presented: true
Processing: true
Reason: Waiting for http-01 challenge propagation: wrong status code '404', expected '200'
State: pending
Events: <none>
my configmap looks like:
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-config
data:
nginx.conf: |
events {
worker_connections 1024;
}
http {
server {
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
listen 8080;
server_name localhost;
location ^~ /.well-known/acme-challenge/ {
default_type "text/plain";
rewrite /.well-known/acme-challenge/(.*) /$1 break;
}
location /static/ {
autoindex on;
alias /code/core/static/;
include /etc/nginx/mime.types;
}
location = /favicon.ico {
access_log off;
log_not_found off;
}
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:8080/;
}
}
}
---```
Add certmanager.k8s.io/cluster-issuer annotation to your Ingress configuration file. Similar problem: certmanager.
Moreover you have syntax mistake in server's location definition in ConfigMap:
location ^~ /.well-known/acme-challenge/
Remember to add also rewrite and nginx class annotations to your Ingress configuration file.
Turns out, the CNAMES were unused. After removing them from the config, I would do a kubectl get challenges again, but it was blank. This is because the challenges were already processed, I'm assuming, and therefore there were none pending.
I also had another issue where I forgot to make the relationship between the kubernetes service and the deployment pods.
Between the two issues, I got confused.
I had the same error on K3S with Traefik. The reason for it was DNS and port forwarding. Certmanager tests itself before reaching out to Letsencrypt. I would get the same error if I used Nginx as router.
I have:
I set up several A records in the DNS server of the provider of my domains:
The Certmanager calls my router to test it's Letsencrypt functionality instead of itself on the Raspberry Pi.
The problem is solved by adding DNS records to the DNS server of my router for the domains so that they directly point to the internal IP address of the Raspberry Pi.
Of course: