i am having below given code
ataset.select("Lead Owner").show();
dataset.filter(dataset.col("Lead Owner").isNotNull());
dataset.select("Lead Owner").show();
But it is not removing rows which has null value for column Lead Owner. Can anyone tell what wrong i am doing here ?
Datasets are Immutable. dataset.filter is a transformation and will return a new dataset, rather than modifying the original. Please refer to the DataSet Docs
val filteredData = dataset.filter(dataset.col("Lead Owner").isNotNull());
filteredData.select("Lead Owner").show();