I am building a multi-class classifier with Keras 2.02 (with Tensorflow backend),and I do not know how to calculate precision and recall in Keras. Please help me.
Python package keras-metrics could be useful for this (I'm the package's author).
import keras
import keras_metrics
model = models.Sequential()
model.add(keras.layers.Dense(1, activation="sigmoid", input_dim=2))
model.add(keras.layers.Dense(1, activation="softmax"))
model.compile(optimizer="sgd",
loss="binary_crossentropy",
metrics=[keras_metrics.precision(), keras_metrics.recall()])
UPDATE: Starting with Keras version 2.3.0, such metrics as precision, recall, etc. are provided within library distribution package.
The usage is the following:
model.compile(optimizer="sgd",
loss="binary_crossentropy",
metrics=[keras.metrics.Precision(), keras.metrics.Recall()])
As of Keras 2.0, precision and recall were removed from the master branch. You will have to implement them yourself. Follow this guide to create custom metrics : Here.
Precision and recall equation can be found Here
Or reuse the code from keras before it was removed Here.
There metrics were remove because they were batch-wise so the value may or may not be correct.
My answer is based on the comment of Keras GH issue. It calculates validation precision and recall at every epoch for a onehot-encoded classification task. Also please look at this SO answer to see how it can be done with keras.backend functionality.
import keras as keras
import numpy as np
from keras.optimizers import SGD
from sklearn.metrics import precision_score, recall_score
model = keras.models.Sequential()
# ...
sgd = SGD(lr=0.001, momentum=0.9)
model.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])
class Metrics(keras.callbacks.Callback):
def on_train_begin(self, logs={}):
self._data = []
def on_epoch_end(self, batch, logs={}):
X_val, y_val = self.validation_data[0], self.validation_data[1]
y_predict = np.asarray(model.predict(X_val))
y_val = np.argmax(y_val, axis=1)
y_predict = np.argmax(y_predict, axis=1)
self._data.append({
'val_recall': recall_score(y_val, y_predict),
'val_precision': precision_score(y_val, y_predict),
})
return
def get_data(self):
return self._data
metrics = Metrics()
history = model.fit(X_train, y_train, epochs=100, validation_data=(X_val, y_val), callbacks=[metrics])
metrics.get_data()