Retrieve final hidden activation layer output from sklearn's MLPClassifier

Question

I would like to do some tests with neural network final hidden activation layer outputs using sklearn's MLPClassifier after fitting the data.

for example, If I create a classifier, assuming data X_train with labels y_train and two hidden layers of sizes (300,100)

clf = MLPClassifier(hidden_layer_sizes=(300,100))
clf.fit(X_train,y_train)

I would like to be able to call a function somehow to retrieve the final hidden activation layer vector of length 100 for use in additional tests.

Assuming a test set X_test, y_test, normal prediction would be:

preds = clf.predict(X_test)

But, I would like to do something like:

activation_layers_for_all_X_test = clf.get_final_activation_output(X_test)

Functions such as get_weights exist, but that would only help me on a per layer basis. Short of doing the transformation myself, is there another methodology to retrieve these final hidden layer activated outputs for the final hidden layer?

Looking at this diagram as an example:

The output I would like is the Out Layer, i.e. the final activated output from the final hidden layer.

Answer 1

As I said in my comment above, it doesn't look like there's a function to do exactly what you want in sklearn but you can hack the _predict function very easily to make it do what you want. The following code will return all activations, you can edit this to return activations[-2] for just the bit that you're after.

def get_activations(clf, X):
        hidden_layer_sizes = clf.hidden_layer_sizes
        if not hasattr(hidden_layer_sizes, "__iter__"):
            hidden_layer_sizes = [hidden_layer_sizes]
        hidden_layer_sizes = list(hidden_layer_sizes)
        layer_units = [X.shape[1]] + hidden_layer_sizes + \
            [clf.n_outputs_]
        activations = [X]
        for i in range(clf.n_layers_ - 1):
            activations.append(np.empty((X.shape[0],
                                         layer_units[i + 1])))
        clf._forward_pass(activations)
        return activations