zoukankan      html  css  js  c++  java
  • Spark2 Dataset之collect_set与collect_list

    collect_set去除重复元素;collect_list不去除重复元素
    select gender,
           concat_ws(',', collect_set(children)),
           concat_ws(',', collect_list(children))
      from Affairs
     group by gender

    // 创建视图 
    data.createOrReplaceTempView("Affairs")
    
    val df3= spark.sql("select gender,concat_ws(',',collect_set(children)),concat_ws(',',collect_list(children)) from Affairs group by gender")
    df3: org.apache.spark.sql.DataFrame = [gender: string, concat_ws(,, collect_set(children)): string ... 1 more field]
    
    df3.show  // collect_set去除重复元素;collect_list不去除重复元素
    +------+-----------------------------------+------------------------------------+
    |gender|concat_ws(,, collect_set(children))|concat_ws(,, collect_list(children))|
    +------+-----------------------------------+------------------------------------+
    |female|                             no,yes|                    no,yes,no,no,yes|
    |  male|                             no,yes|                    no,yes,no,yes,no|
    +------+-----------------------------------+------------------------------------+
    
  • 相关阅读:
    GlowFilter发光效果
    投影滤镜的使用
    flash怎样删除库中没用的元件
    script中用php
    jQuery animate实现slideUp slideDown 的反向
    CSS !important 用法
    放新浪微博的箭头css写法
    json 取数据
    css hack 大全
    bubble 界面代码
  • 原文地址:https://www.cnblogs.com/wwxbi/p/6102380.html
Copyright © 2011-2022 走看看