zoukankan      html  css  js  c++  java
  • Spark 取前几行,先sort再limit

    scala> val df = sc.parallelize(Seq(
         |   (0,"cat26",30.9), 
         |   (1,"cat67",28.5), 
         |   (2,"cat56",39.6),
         |   (3,"cat8",35.6))).toDF("Hour", "Category", "Value")
    df: org.apache.spark.sql.DataFrame = [Hour: int, Category: string ... 1 more field]
    
    scala> df.show
    +----+--------+-----+
    |Hour|Category|Value|
    +----+--------+-----+
    |   0|   cat26| 30.9|
    |   1|   cat67| 28.5|
    |   2|   cat56| 39.6|
    |   3|    cat8| 35.6|
    +----+--------+-----+
    
    
    scala> df.sort(col("Hour").asc).limit(1)
    res6: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [Hour: int, Category: string ... 1 more field]
    
    scala> df.sort(col("Hour").asc).limit(1).show
    +----+--------+-----+
    |Hour|Category|Value|
    +----+--------+-----+
    |   0|   cat26| 30.9|
    +----+--------+-----+
    
    
    scala> df.sort(col("Hour").desc).limit(1).show
    +----+--------+-----+
    |Hour|Category|Value|
    +----+--------+-----+
    |   3|    cat8| 35.6|
    +----+--------+-----+
    
    //默认是升序
    scala> df.sort(col("Hour")).limit(1).show
    +----+--------+-----+
    |Hour|Category|Value|
    +----+--------+-----+
    |   0|   cat26| 30.9|
    +----+--------+-----+
  • 相关阅读:
    1、Elasticsearch教程-从入门到精通
    3、ik分词器
    2、ElasticSearch的安装
    svn操作
    2020-10
    编写第一个linux驱动
    字符设备驱动
    Linux下/dev和/sys/class的区别
    latex
    Qt
  • 原文地址:https://www.cnblogs.com/v5captain/p/14208557.html
Copyright © 2011-2022 走看看