I've set up an Azure HdInsight Linux Spark Cluster which should come with the 1.5.2 version of Spark. However, if I try to output the version number by using sparkContext.version I keep getting 1.4.1.
Is there maybe a step I'm missing with the configuration? I've been booting up clusters both through the GUI and through scripts.
This is a known bug on our side. The binary version is actually 1.5.2 but it is being reported incorrectly.