bias_variance

bias_variance_0_1_loss(predictions, y_test)

Compute the bias-variance decomposition using the 0-1 loss function

Parameters:

• predictions – numpy array of predictions over the set of iterations

• y_test – numpy array of ground truth labels

Returns:

(average loss, average bias, average variance, net variance)

bias_variance_compute(estimator, X_train, y_train, X_test, y_test, prepare_X=<function <lambda>>, prepare_y_train=<function <lambda>>, iterations=200, random_state=None, decomp_fn=<function bias_variance_mse>, fit_kwargs=None, predict_kwargs=None)

Compute the bias-variance decomposition in serial

Parameters:

• estimator (EstimatorWrapper) – estimator wrapped with a class extending EstimatorWrapper

• X_train – features for training

• y_train – ground truth labels for training

• X_test – features for testing

• y_test – ground truth labels for testing

• prepare_X (function, optional) – function to transform feature datasets before calling fit and predict methods

• prepare_y_train (function, optional) – function to transform training ground truth labels before calling fit method

• iterations (int, optional) – number of iterations for the training/testing

• random_state (int, optional) – random state for bootstrap sampling

• decomp_fn (function, optional) – bias-variance decomposition function

• fit_kwargs (dict, optional) – kwargs to pass to the fit method

• predict_kwargs (dict, optional) – kwargs to pass to the predict method

Returns:

(average loss, average bias, average variance, net variance)

bias_variance_mse(predictions, y_test)

Compute the bias-variance decomposition the mean squared error loss function

Parameters:

• predictions – numpy array of predictions over the set of iterations

• y_test – numpy array of ground truth labels

Returns:

(average loss, average bias, average variance, net variance)

bootstrap_train_and_predict(estimator, X_train_values, y_train_values, X_test_prepared, prepare_X=<function <lambda>>, prepare_y_train=<function <lambda>>, random_state=None, fit_kwargs=None, predict_kwargs=None)

Train an estimator using a bootstrap sample of the training data and get predictions from it

Parameters:

• estimator (EstimatorWrapper) – estimator wrapped with a class extending EstimatorWrapper

• X_train_values – numpy array of features for training

• y_train_values – numpy array of ground truth labels for training

• X_test_prepared – feature set for testing which has been processed by prepare_X function

• prepare_X (function, optional) – function to transform feature datasets before calling fit and predict methods

• prepare_y_train (function, optional) – function to transform train ground truth labels before calling fit method

• random_state (int, optional) – random state for bootstrap sampling

• fit_kwargs (dict, optional) – kwargs to pass to the fit method

• predict_kwargs (dict, optional) – kwargs to pass to the predict method

Returns:

predictions

get_values(x)

If argument is a Pandas dataframe, return ‘values’ numpy array from it.

Parameters:

x (Any) – pandas dataframe or anything else

Returns:

if pandas dataframe - return ‘values’ numpy array otherwise - return itself

train_and_predict(estimator, X_train_values, y_train_values, X_test_prepared, prepare_X=<function <lambda>>, prepare_y_train=<function <lambda>>, fit_kwargs=None, predict_kwargs=None)

Train an estimator and get predictions from it

Parameters:

• estimator (EstimatorWrapper) – estimator wrapped with a class extending EstimatorWrapper

• X_train_values – numpy array of features for training

• y_train_values – numpy array of ground truth labels for training

• X_test_prepared – feature set for testing which has been processed by prepare_X function

• prepare_X (function, optional) – function to transform feature datasets before calling fit and predict methods

• prepare_y_train (function, optional) – function to transform train ground truth labels before calling fit method

• fit_kwargs (dict, optional) – kwargs to pass to the fit method

• predict_kwargs (dict, optional) – kwargs to pass to the predict method

Returns:

predictions