I'm new to spark, I downloaded precompiled spark.
When I try to run spark-shell from bin folder on command line, it returns
:cd /users/denver/spark-1.6/bin
:spark-shellcommand not found
But if I run it like
:cd /users/denver/spark-1.6
:./bin/spark-shell
it launches spark ..
can you please let me know why it is throwing error in the 1st case
The reason why you can not run the spark-shell command in first case is because of environment variable
The terminal searches for executables in $PATH. This is a Unix environment variable that lists directories containing system binaries (such as ls, echo, or gcc). If you call an executable that's not in a $PATH directory (such as spark-shell), you need to indicate its absolute path in the file system.
In the terminal . is a synonym for the current working directory, thus ./bin/spark-shell can work properly. You could equally well call ./some/path/bin/spark-shell.
Hope it helps.
In linux:
Add to your ~/.bashrc
export SPARK_HOME=/home/das/spark-1.6.2-bin-hadoop2.6 export PATH=$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbin
Source ~/.bashrc. Relaunch terminal then spark-shell works from anywhere If it doesn't work add to ~/.profile
This happened to me too. Using ./bin/spark-shell after changing to the spark directory should solve the issue.