I have a wordpress site running on an aws ec2 instance. When I put this ec2 instance behind an application load balancer, I notice the requests to the server become slower. The response time growth is mainly due to some requests for static content which can take up to five seconds instead of thousands milliseconds. When I read application load balancer logs, it doesn't reveal any problem, just showing some long 'target_processing_time'. But when I look for apache log, it says content was delivered quite fast. For example, tracking the same request, I have the following load balancer log :
h2 2021-07-29T17:01:49.266317Z app/load-balancer-name/XXXXXX 93.31.XXX.XXX:39470 172.31.XXX.XXX:80 0.001 2.847 0.000 200 200 80 575 "GET https://somedomain:443/wp-content/plugins/wcfm-pg-mangopay-master/assets/css/wcfm-pg-mangopay.css?ver=1.0.0 HTTP/2.0" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 arn:aws:elasticloadbalancing:eu-west-3:XXXXXX:targetgroup/XXX/XXXXXXXX "Root=1-6102defa-XXXXXXXXX" "somedomain" "arn:aws:acm:eu-west-3:XXXXXXXX:certificate/XXXXXXXX" 1 2021-07-29T17:01:46.416000Z "forward" "-" "-" "172.31.21.70:80" "200" "-" "-"
And for the apache log :
172.XX.XX.XXX - - [29/Jul/2021:17:01:49 +0000] "GET /wp-content/plugins/wcfm-pg-mangopay-master/assets/css/wcfm-pg-mangopay.css?ver=1.0.0 HTTP/1.1" 200 376 "https://somedomain/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36" **0/799**
So, while the application load balancer says it received request at 17:01:46 and took 0.001 ms to process request, Apache mention it received request at 17:01:49 (three second later) and took 800 micro second to process. The problem is not with cross region and occurs only in case of multiple requests sent at the same time. I don't really understand what can explain this discrepancy. That's why I was wondering if the aws application load balancer do have some unmentioned queueing mechanism ? Or a maximum number of simultaneous http connections to the target ?