zoukankan      html  css  js  c++  java
  • === $ spark sql 的特别的方法

     /**
       * Equality test.
       * {{{
       *   // Scala:
       *   df.filter( df("colA") === df("colB") )
       *
       *   // Java
       *   import static org.apache.spark.sql.functions.*;
       *   df.filter( col("colA").equalTo(col("colB")) );
       * }}}
       *
       * @group expr_ops
       * @since 1.3.0
       */
      def === (other: Any): Column = withExpr {
        val right = lit(other).expr
        if (this.expr == right) {
          logWarning(
            s"Constructing trivially true equals predicate, '${this.expr} = $right'. " +
              "Perhaps you need to use aliases.")
        }
        EqualTo(expr, right)
      }

    使用场景,join时判断等值时scala中使用===判断

      /**
       * Converts $"col name" into a [[Column]].
       *
       * @since 2.0.0
       */
      implicit class StringToColumn(val sc: StringContext) {
        def $(args: Any*): ColumnName = {
          new ColumnName(sc.s(args: _*))
        }
      }

    使用场景,根据聚合之后的别名排序

    .agg(count(userActionLog("logId")).alias("actionCount"))
    // 第五步:进行排序
    .sort($"actionCount".desc)
  • 相关阅读:
    Java多线程
    SpringCloud
    Java 多线程
    MySQL
    MySQL
    SpringCloud
    SpringCloud
    SpringCloud
    SpringBoot
    Spring MVC
  • 原文地址:https://www.cnblogs.com/rocky-AGE-24/p/7526458.html
Copyright © 2011-2022 走看看