Is there a way I can stop people from seeing the Javascript Source code that includes the Server API key and server information?
No, there is not.
You can obfuscate and minify your code, but that doesn't help since your users can look at their browser's Network Inspector instead.