metrics

monotonicity(y_true, y_pred, weights=None)[source]

Generalizes ROC AUC by computing \(P\left(\frac{\Delta\mathrm{y_pred}}{\Delta\mathrm{y_true}} > 0\right)\), the probability incrementing y_true increases y_pred for a randomly chosen pair of instances. This reduces to ROC AUC when y_true has two unique values. Adapted from Algorithm 2 in Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861-874.

Parameters:

y_true (list-like) – Ground truth ordinal values
y_pred (list-like) – Predicted ordinal values
weights (list-like) – Sample weights. Will be normalized to one across each unique values of y_true. If None (default) all samples are weighed equally.

Returns:

Float between 0 and 1. 0 indicates 100% chance of y_pred decreasing upon incrementing y_true up to its next highest value in the dataset. 1 being a 100% chance of y_pred increasing for the same scenario. 0.5 would be 50% chance of either.

normalized_mutual_info(X, y, **kwargs)[source]: Thin wrapper around sklearn’s mutual information. This normalizes the result to 0-1 scale. y is assumed categorical.

rank_auc(y_true, y_pred, weights=None)[source]

Generalizes ROC AUC by computing probability that two randomly chosen data points would be ranked consistently with ground truth labels. This reduces to ROC AUC when y_true has two unique values. Adapted from Algorithm 2 in Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861-874.

Parameters:

y_true (list-like) – Ground truth ordinal values
y_pred (list-like) – Predicted ordinal values
weights (list-like) – Sample weights. Will be normalized to one across each unique values of y_true. If None (default) all samples are weighed equally.

Returns:

Float between 0 and 1. 0 indicates 100% chance of y_pred matching order of y_true. 1 being a 100% chance of y_pred having the opposite order of y_true. 0.5 would be 50% chance of either.