zoukankan      html  css  js  c++  java
  • Dealing with Imbalanced Datasets

    Motivation

    The Imbalanced Datasets are very common in our life such as illegal users or illness check. The machine learning model always performs bad on these datasets if there are no specific dealings, especially the prediction accuracy of minority class. For example, if the data is highly imbalanced such as 9995(negative):5(positive), then if your model just let every instance to be negative and you can get an acc of 99.95% but the result is meaningless. Another example is that misclassifying the minority is very severe. Assume that you misclassify the patient as normal. Oh my god!

    So researchers proposed two kinds of methods for this problem:

    • Cost Sensitive Learning
      When training your model, it will give different classes different weights in the loss function thus let the model focus more on the minority class. In sklearn, there are class_weight and sample_weight for you. For class_weight, you can specify the weights for different classes such as {0:0.1,1:0.9} or you can set it to balanced then weights will be computed by (frac{#samples}{#classes * np.bincount(y)}). For fit(sample_weight=), you give every instance different weights. When computing the loss for the instance, it will be class_weight * sample_weight * loss.
    • Sampling
      Sampling means that we will change the original dataset rather than giving them different weights.

    Sampling Methods

    Over-sampling means to increment the minority class.

    • Random Over Sampling
      To sample from minority class with replacement to let the number of each class is 1:1. Overfitting on minority class.
    • Synthetic Minority Oversampling Technique (SMOTE)

    [x_{new}=x_i+lambda(x_{zi}-x_i) ]

    First you find the k_neighbors of (x_i) in the minority class, then just select one (x_{zi}) randomly and produce the new one. There are some variants such as borderline SMOTE, SVM SMOTE and KMeans SMOTE.

    • Adaptive Synthetic (ADASYN)
      The difference between SMOTE and ADASYN is that SMOTE will generate new samples for random minority data until 1:1. But ADASYN will automatically decide the number of new points generated for each (x_i). There will be more points generated if there are more majority data around (x_i).

    Under-sampling means to decrease the majority class.

    • RUS
      Data waste.

    Example

  • 相关阅读:
    oracle索引
    linux命令
    struts2的配置文件struts.xml详解
    oracle删除一个表中的重复数据,且只保留一条
    初步研究java类的加载机制
    java读取Properties配置文件
    利用oracle闪回来找回被drop的表
    快速排序Quicksort
    树与树的概念
    递归
  • 原文地址:https://www.cnblogs.com/EIMadrigal/p/14738860.html
Copyright © 2011-2022 走看看