According to the GitLab Documentation for cache vs artifacts the node_modules should be stored as cache instead of an artifact. The issue with this is if there are multiple runners and a different runner from what created the "build" picks up a job then the cache (node_modules) won't exist which will fail the GitLab CI randomly (if the same runner happens to do the rest of the jobs then it will succeed).
I could make it so everything is done in one job but once I have to put in deployment this issue will reoccur. Another option is to tag it so only one specific runner completes the job but I feel like that is slow.
After researching I realized I could just put in node_modules as an artifact instead of the cache and it will be picked up no matter which runner runs the job but I can't find any documentation on if this is bad or not.
So is it okay to put node_modules as an artifact? If so, why wouldn't people put it as an artifact normally (assuming they add an expiration)?
The short answer here is that you can use the tool however you want to. You absolutely can use artifacts for node_modules. However, there's a way to make the cache work too.
By default, when a runner caches a file or directory, it stores it locally on the host running that runner (which is why it's not available on other runners). In the runner config however, you can change this to store cached objects in AWS S3 or an S3-like repo (like Minio). If all runners are configured this way, or at least 2 are, then those runners can also use the same cached items.
You can read about the runner's cache config options here: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnerscache-section
You can read about Minio (an open source storage solution that provides an S3 API interface) here: https://docs.min.io/