zoukankan      html  css  js  c++  java
  • Dealing with Imbalanced Datasets

    Motivation

    The Imbalanced Datasets are very common in our life such as illegal users or illness check. The machine learning model always performs bad on these datasets if there are no specific dealings, especially the prediction accuracy of minority class. For example, if the data is highly imbalanced such as 9995(negative):5(positive), then if your model just let every instance to be negative and you can get an acc of 99.95% but the result is meaningless. Another example is that misclassifying the minority is very severe. Assume that you misclassify the patient as normal. Oh my god!

    So researchers proposed two kinds of methods for this problem:

    • Cost Sensitive Learning
      When training your model, it will give different classes different weights in the loss function thus let the model focus more on the minority class. In sklearn, there are class_weight and sample_weight for you. For class_weight, you can specify the weights for different classes such as {0:0.1,1:0.9} or you can set it to balanced then weights will be computed by (frac{#samples}{#classes * np.bincount(y)}). For fit(sample_weight=), you give every instance different weights. When computing the loss for the instance, it will be class_weight * sample_weight * loss.
    • Sampling
      Sampling means that we will change the original dataset rather than giving them different weights.

    Sampling Methods

    Over-sampling means to increment the minority class.

    • Random Over Sampling
      To sample from minority class with replacement to let the number of each class is 1:1. Overfitting on minority class.
    • Synthetic Minority Oversampling Technique (SMOTE)

    [x_{new}=x_i+lambda(x_{zi}-x_i) ]

    First you find the k_neighbors of (x_i) in the minority class, then just select one (x_{zi}) randomly and produce the new one. There are some variants such as borderline SMOTE, SVM SMOTE and KMeans SMOTE.

    • Adaptive Synthetic (ADASYN)
      The difference between SMOTE and ADASYN is that SMOTE will generate new samples for random minority data until 1:1. But ADASYN will automatically decide the number of new points generated for each (x_i). There will be more points generated if there are more majority data around (x_i).

    Under-sampling means to decrease the majority class.

    • RUS
      Data waste.

    Example

  • 相关阅读:
    mkdir()和mkdirs() 的区别
    JAVA 多线程学习
    遇到问题(1)
    7.12计划
    Android 中Int类型和String类型的转换
    Android Binder机制学习笔记
    7.11计划,做个没心没肺的人
    RTL行为级仿真、综合后门级功能仿真和时序仿真
    定向锚文本(高级) 站内站策略(高级) 链轮模式(高级) 站群模式(高级)
    优化长尾关键词基础指南
  • 原文地址:https://www.cnblogs.com/EIMadrigal/p/14738860.html
Copyright © 2011-2022 走看看