Hello people of stack overflow,
I have recently deployed my node application on AWS by using bitnami provided MEAN solution to set up the environment and this is my first application deployed here.
One of the applications functionality is to query mongodb (~20k records) and then do some analysis on each record looping the array that this record contains (approximately 30 objects in the array).
I hope you get the idea of the relative intensity in resources for this operation and so now here comes the problem that doesn't let me sleep at night:
This whole process takes around 27 seconds on my own machine (i5 3.4ghz cpu, 16gb ram, Samsung 840 evo SSD, linux mint OS) but when i run it on a AWS instance (i'm currently using t2.small, but i tried running this same operation using one of the largest instances available with ridiculous amount of resources yielding the same result) the process takes around 90 seconds. The query alone to filter and retrieve the 20k objects takes around 30 seconds on AWS, whereas on my machine it under 9 seconds. When looking at the resources while this is running i can see that the CPU is not at a 100%, so the cpu is not a bottleneck (My CPU credits are constantly near the maximum). Adding more ram by changing to a huge instance also did not help...
It's not like my machine is very powerful or anything, but the performance difference is unbelievable and this makes that particular functionality unusable on AWS.
I have also tried changing my AWS volume to IOPS but with no luck..
Please the experts of this forum - help me with this huge headache of a problem...
Choose the correct instance with the correct size. I have experience t2.small, it takes a long time under heavy load.
You didn't explain in your question, so based on your own experience with AWS I'll recommend that you
Check the free RAM space on your instance
CPU usage
hard drive space
To avoid these issues, see the MongoDB portal recommendation
Before hosting on AWS, you need to understand the architecture of AWS Something that affects performance
Size is Everything: Choose the Right Instance Type
Based on our experience helping deploy and support MongoDB deployments on AWS, we have found that Amazon ECW M4, I2, and R3 instance types tend to be the most successful and widely used in customer deployments.
EBS is the general type of storage we recommend for EC2 for MongoDB and we'll dive into the best way to set it up. When choosing your disks, we must ask AWS to guarantee our Input/Output Operations Per Second (IOPS) by requesting Provisioned IOPS (PIOPS).
networks
Because MongoDB can benefit from high packet-per-second throughput and lower latency networking between instances, Amazon's Enhanced Networking feature can provide significantly improved performance and consistency. If your instance type supports the Enhanced Networking feature, we recommend that you enable it.
For more details Maximize MongoDB performance on AWS
I don't know if you have checked your NodeJS code for configuration issues. For example, is there a background connection to another database that fails and delays the response? Is there some other service the app is trying to connect to, eg an external mail service etc. Anything that requires connectivity to anything else in AWS requires proper VPC/routing setup, not saying this is the problem, just covering all the bases.