zoukankan      html  css  js  c++  java
  • Spark 取前几行,先sort再limit

    scala> val df = sc.parallelize(Seq(
         |   (0,"cat26",30.9), 
         |   (1,"cat67",28.5), 
         |   (2,"cat56",39.6),
         |   (3,"cat8",35.6))).toDF("Hour", "Category", "Value")
    df: org.apache.spark.sql.DataFrame = [Hour: int, Category: string ... 1 more field]
    
    scala> df.show
    +----+--------+-----+
    |Hour|Category|Value|
    +----+--------+-----+
    |   0|   cat26| 30.9|
    |   1|   cat67| 28.5|
    |   2|   cat56| 39.6|
    |   3|    cat8| 35.6|
    +----+--------+-----+
    
    
    scala> df.sort(col("Hour").asc).limit(1)
    res6: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [Hour: int, Category: string ... 1 more field]
    
    scala> df.sort(col("Hour").asc).limit(1).show
    +----+--------+-----+
    |Hour|Category|Value|
    +----+--------+-----+
    |   0|   cat26| 30.9|
    +----+--------+-----+
    
    
    scala> df.sort(col("Hour").desc).limit(1).show
    +----+--------+-----+
    |Hour|Category|Value|
    +----+--------+-----+
    |   3|    cat8| 35.6|
    +----+--------+-----+
    
    //默认是升序
    scala> df.sort(col("Hour")).limit(1).show
    +----+--------+-----+
    |Hour|Category|Value|
    +----+--------+-----+
    |   0|   cat26| 30.9|
    +----+--------+-----+
  • 相关阅读:
    js简单排序
    js希尔排序
    js直接插入排序
    大数运算
    IOC和AOP的理解 (持续更新)
    频发事件的缓存技巧
    执行上下文深入解析(Execution Context)
    二分
    HDU 3790 最短路径问题 (双重权值)
    浅谈Targan算法
  • 原文地址:https://www.cnblogs.com/v5captain/p/14208557.html
Copyright © 2011-2022 走看看