zoukankan      html  css  js  c++  java
  • integer encoding vs 1-hot (py)

    https://github.com/szilard/benchm-ml/issues/1

     

    glouppe commented on 7 May 2015

    Thanks for the benchmarks! Proper handling of categorical variables is not an easy issue anyway.

    I would expect faster, lower memory but decrease in AUC (or same in some cases).

    When the categories are ordered, it makes more sense indeed to handle them as numerical variables. I dont have a strong argument as to why it may be also better when there is no natural ordering. I guess it could boil down to the fact that one-hot encoding splits are often very unbalanced, while integer encoded splits may be less unbalanced.

    Thanks @glouppe. I read somewhere a paper that AFAIR suggested to sort the (non-ordered) categoricals in order of their frequency in the data and encode them as integers as such. Any idea what that paper might be?

    glouppe commented on 7 May 2015


    Yes, it is Breiman's book :) When your output is binary, this strategy is in fact optimal (it will find the best subset among the values of the categorical variables) and linear.

    See section 3.6.3.2 of my thesis if you dont have the CART book.
    http://orbi.ulg.ac.be/bitstream/2268/170309/1/thesis.pdf

    One-hot encoding could be helpful when the number of categories are small( in level of 10 to 100). In such case one-hot encoding can discover interesting interactions like (gender=male) AND (job = teacher).

    While ordering them makes it harder to be discovered(need two split on job). However, indeed there is not a unified way handling categorical features in trees, and usually what tree was really good at was ordered continuous features anyway..

     
     

     
  • 相关阅读:
    ES head安装笔记, 还没有试
    sed用法笔记
    Kibana笔记
    ElasticSearch笔记
    Mongo聚合笔记
    java 判断是否为数字
    Redis 一:安装篇
    make问题:make[1] entering directory
    Java 多线程 简单实例 (消费者与生成者)的关系
    Java 多线程 简单实例 (Runnable)
  • 原文地址:https://www.cnblogs.com/xinping-study/p/7085221.html
Copyright © 2011-2022 走看看