zoukankan      html  css  js  c++  java
  • Spark2 SQL configuration参数配置

     查看当前环境SQL参数的配置

    spark.sql("SET -v")


    key value
    spark.sql.hive.version 1.2.1
    spark.sql.sources.parallelPartitionDiscovery.threshold 32
    spark.sql.hive.metastore.barrierPrefixes  
    spark.sql.shuffle.partitions 200
    spark.sql.hive.metastorePartitionPruning FALSE
    spark.sql.broadcastTimeout 300
    spark.sql.sources.bucketing.enabled TRUE
    spark.sql.parquet.filterPushdown TRUE
    spark.sql.statistics.fallBackToHdfs FALSE
    spark.sql.adaptive.enabled FALSE
    spark.sql.parquet.cacheMetadata TRUE
    spark.sql.hive.metastore.sharedPrefixes com.mysql.jdbc
    spark.sql.parquet.respectSummaryFiles FALSE
    spark.sql.warehouse.dir hdfs:///user/spark/warehouse
    spark.sql.orderByOrdinal TRUE
    spark.sql.hive.convertMetastoreParquet TRUE
    spark.sql.groupByOrdinal TRUE
    spark.sql.hive.thriftServer.async TRUE
    spark.sql.thriftserver.scheduler.pool <undefined>
    spark.sql.orc.filterPushdown FALSE
    spark.sql.adaptive.shuffle.targetPostShuffleInputSize 67108864b
    spark.sql.sources.default parquet
    spark.sql.parquet.compression.codec snappy
    spark.sql.hive.metastore.version 1.2.1
    spark.sql.sources.partitionDiscovery.enabled TRUE
    spark.sql.crossJoin.enabled FALSE
    spark.sql.parquet.writeLegacyFormat FALSE
    spark.sql.hive.verifyPartitionPath FALSE
    spark.sql.variable.substitute TRUE
    spark.sql.thriftserver.ui.retainedStatements 200
    spark.sql.hive.convertMetastoreParquet.mergeSchema FALSE
    spark.sql.parquet.enableVectorizedReader TRUE
    spark.sql.parquet.mergeSchema FALSE
    spark.sql.parquet.binaryAsString FALSE
    spark.sql.columnNameOfCorruptRecord _corrupt_record
    spark.sql.files.maxPartitionBytes 134217728
    spark.sql.streaming.checkpointLocation <undefined>
    spark.sql.variable.substitute.depth 40
    spark.sql.parquet.int96AsTimestamp TRUE
    spark.sql.autoBroadcastJoinThreshold 10485760
    spark.sql.pivotMaxValues 10000
    spark.sql.sources.partitionColumnTypeInference.enabled TRUE
    spark.sql.hive.metastore.jars builtin
    spark.sql.thriftserver.ui.retainedSessions 200
    spark.sql.sources.maxConcurrentWrites 1
    spark.sql.parquet.output.committer.class org.apache.parquet.hadoop.ParquetOutputCommitter
    
    
  • 相关阅读:
    python文件操作总结
    hidoCoder #1514 偶像的条件
    2017浙江省赛大学生程序设计竞赛 C题 What Kind of Friends Are You?
    51nod 1503 猪和回文串(动态规划)
    整数划分(若干不同),时间复杂度O(n*sqrt(n))
    Jiu Yuan Wants to Eat
    牛牛数括号
    P3254 圆桌问题
    方格取数(1)
    Taeyeon的困惑
  • 原文地址:https://www.cnblogs.com/wwxbi/p/6114410.html
Copyright © 2011-2022 走看看