Classification report of sklearn

Classification report

The classification_report function builds a text report showing the main classification metrics. Here is a small example with custom target_names and inferred labels:

>>> from sklearn.metrics import classification_report
>>> y_true = [0, 1, 2, 2, 0]
>>> y_pred = [0, 0, 2, 1, 0]
>>> target_names = ['class 0', 'class 1', 'class 2']
>>> print(classification_report(y_true, y_pred, target_names=target_names))
              precision    recall  f1-score   support

     class 0       0.67      1.00      0.80         2
     class 1       0.00      0.00      0.00         1
     class 2       1.00      0.50      0.67         2

    accuracy                           0.60         5
   macro avg       0.56      0.50      0.49         5
weighted avg       0.67      0.60      0.59         5

API

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html

Build a text report showing the main classification metrics.

Read more in the User Guide.

度量标准

https://medium.com/@kohlishivam5522/understanding-a-classification-report-for-your-machine-learning-model-88815e2ce397

There are four ways to check if the predictions are right or wrong:

1. TN / True Negative: the case was negative and predicted negative
2. TP / True Positive: the case was positive and predicted positive
3. FN / False Negative: the case was positive but predicted negative
4. FP / False Positive: the case was negative but predicted positive

Precision — What percent of your predictions were correct?

      精确度，预测为某个类的数据中， 预测结果正确的概率。

Precision is the ability of a classifier not to label an instance positive that is actually negative. For each class, it is defined as the ratio of true positives to the sum of a true positive and false positive.

Precision:- Accuracy of positive predictions.

Precision = TP/(TP + FP)

Recall — What percent of the positive cases did you catch?

     召回率， 对于某个类别的数据，预测结果为争取的数据概率。

    https://www.zhihu.com/question/19645541

    此召回，有解释为 产品劣质比率，需要召回的意思。

    有解释为 “想起来”， 按照单词的字面意思，对应机器学习的模型建立过程， 可以解释的通， 模型学习的过程是一个记忆过程， 类似人的学习， 再次预测的时候， 相当于回想之间的记忆， 想得起来，表示模型预测正确。

    另有翻译为 “查全率”， 更好理解， 对于全部的分类数据中， 模型能够正确预测的覆盖率。

Recall is the ability of a classifier to find all positive instances. For each class it is defined as the ratio of true positives to the sum of true positives and false negatives.

Recall:- Fraction of positives that were correctly identified.

Recall = TP/(TP+FN)

F1 score — What percent of positive predictions were correct?

    F1 值是对 召回率 和 精确率 的一个综合。

    因为有可能但看召回率和精确率， 各有大小， 不好比较， 综合比分，象征综合的性能。

The F1 score is a weighted harmonic mean of precision and recall such that the best score is 1.0 and the worst is 0.0.

F1 scores are lower than accuracy measures as they embed precision and recall into their computation.

As a rule of thumb, the weighted average of F1 should be used to compare classifier models, not global accuracy.

F1 Score = 2*(Recall * Precision) / (Recall + Precision)

https://zhuanlan.zhihu.com/p/97870600

F1值：

精确率越高越好，召回率越高越好。

下边式子（2）可以由式子（1）推导出来

从(1)看出，Recall不变时，Precision越大，1/Precision越小，从而F1越大。

同理： Precision不变时，Recall越大，1/Recall越小，从而F1越大。

Support

Support is the number of actual occurrences of the class in the specified dataset. Imbalanced support in the training data may indicate structural weaknesses in the reported scores of the classifier and could indicate the need for stratified sampling or rebalancing. Support doesn’t change between models but instead diagnoses the evaluation process.

Accuracy

https://medium.com/analytics-vidhya/accuracy-vs-f1-score-6258237beca2

     准确度， 是对模型的一个整体性评估， 覆盖所有类别。

     是模型级别的度量参数。

One of the more obvious metrics, it is the measure of all the correctly identified cases. It is most used when all the classes are equally important.

Accuracy vs f1 score

     F1 score，虽然是针对各个类别自身的度量结果， 但是可以通过对各类别的结果，做简单的均值计算，或者按照support的数目，做加权平均，可以得到模型级别的度量结果。 同理对于precision和recall。

     哪有什么差别呢？

（1）目标不同

    精确度 -- 面向正确的情况，进行度量。

    F1 SCORE -- 面向错误的情况，进行度量。 如果你更关注失败率，可以使用这个。

（2）数据分布

     精确度 -- 分布均匀

    F1 SCORE -- 数据在类之间不均匀

    在现实生活中，非均衡的数据是常常存在的， 所有F1 SCORE具有更好的度量效果。

https://medium.com/analytics-vidhya/accuracy-vs-f1-score-6258237beca2

To summarise the differences between the F1-score and the accuracy,

• Accuracy is used when the True Positives and True negatives are more important while F1-score is used when the False Negatives and False Positives are crucial
• Accuracy can be used when the class distribution is similar while F1-score is a better metric when there are imbalanced classes as in the above case.
• In most real-life classification problems, imbalanced class distribution exists and thus F1-score is a better metric to evaluate our model on.