Accuracy given from evaluating model not equal to sklearn classification_report accuracy

问题

I'm using sklearn classification_report for reporting test statistics. The accuracy given by this method is 42% while model evaluation gives 93% accuracy. Which one is the real accuracy and what's the reason of this difference?

Model evaluation:

results = model.evaluate(test_ds.values, test_lb.values) print(results)

Output:
7397/7397 [==============================] - 0s 28us/sample - loss: 0.2309 - acc: 0.9305

Report Classification:

import numpy as np from sklearn.metrics import classification_report predictions = model.predict(test_ds) print(classification_report(test_lb, np.argmax(predictions, axis=1)))

Output:

label precision recall f1-score support

       0       0.41      0.38      0.40      3700
       1       0.43      0.46      0.44      3697

accuracy                           0.42      7397

回答1:

Ideally, both the metrics should give the accuracy of the same level with some minor difference. The problem may be with the data.

You can see the below example to compare both the Metrics.

import tensorflow as tf
from sklearn.datasets import load_iris
import numpy as np
from tensorflow import keras
from sklearn.model_selection import train_test_split
iris = load_iris()
X = iris.data[:, (2, 3)] # petal length, petal width
y = (iris.target == 0).astype(np.int) 

(X_train,X_test,y_train,y_test) = train_test_split(X,y,test_size=0.2)

model = keras.models.Sequential([
    keras.layers.Flatten(input_shape=[2]),
    keras.layers.Dense(300, kernel_initializer="he_normal"),
    keras.layers.LeakyReLU(),
    keras.layers.Dense(100, kernel_initializer="he_normal"),
    keras.layers.LeakyReLU(),
    keras.layers.Dense(1, activation="sigmoid")
])

model.compile(loss="binary_crossentropy",
              optimizer=keras.optimizers.SGD(),
              metrics=["accuracy"])

model.fit(X_train,y_train,epochs=2)

Training Accuracy:

Epoch 1/2
4/4 [==============================] - 0s 3ms/step - loss: 2.0655 - accuracy: 0.6333
Epoch 2/2
4/4 [==============================] - 0s 3ms/step - loss: 0.5199 - accuracy: 0.7333
<tensorflow.python.keras.callbacks.History at 0x7fdd4ed72048>

Evaluation Result:

test_ds = pd.DataFrame(X_test)
test_lb = pd.DataFrame(y_test)
model.evaluate(test_ds.values,test_lb.values)

1/1 [==============================] - 0s 1ms/step - loss: 0.5510 - accuracy: 0.6667
[0.5510352253913879, 0.6666666865348816]

With Sklearn Metrics:

import numpy as np
from sklearn.metrics import classification_report
predictions = model.predict(X_test)
print(classification_report(y_test, np.argmax(predictions, axis=1))) 

              precision    recall  f1-score   support

           0       0.67      1.00      0.80        20
           1       0.00      0.00      0.00        10

    accuracy                           0.67        30
   macro avg       0.33      0.50      0.40        30
weighted avg       0.44      0.67      0.53        30

You can see the accuracy is the same for both metrics(66.7 & 67).

来源：https://stackoverflow.com/questions/57834078/accuracy-given-from-evaluating-model-not-equal-to-sklearn-classification-report

标签

python

tensorflow

keras

scikit-learn

deep-learning