How are the number of AWS instances related to the Nginx and Gunicorn workers?
AWS Sagemaker calls the docker container stored in ECR with the serve command. The number of instances and the type are setup in the estimator(sage.estimator.Estimator) However, each docker container has a Nginx and Gunicorn setup.
Is a smaller instance launched to do the nginx proxy_pass? Are the requests proxied inside each container at the thread level?
https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-hosting.html
https://docs.aws.amazon.com/sagemaker/latest/dg/adv-bring-own-examples.html