zoukankan      html  css  js  c++  java
  • Spark2 SQL configuration参数配置

     查看当前环境SQL参数的配置

    spark.sql("SET -v")


    key value
    spark.sql.hive.version 1.2.1
    spark.sql.sources.parallelPartitionDiscovery.threshold 32
    spark.sql.hive.metastore.barrierPrefixes  
    spark.sql.shuffle.partitions 200
    spark.sql.hive.metastorePartitionPruning FALSE
    spark.sql.broadcastTimeout 300
    spark.sql.sources.bucketing.enabled TRUE
    spark.sql.parquet.filterPushdown TRUE
    spark.sql.statistics.fallBackToHdfs FALSE
    spark.sql.adaptive.enabled FALSE
    spark.sql.parquet.cacheMetadata TRUE
    spark.sql.hive.metastore.sharedPrefixes com.mysql.jdbc
    spark.sql.parquet.respectSummaryFiles FALSE
    spark.sql.warehouse.dir hdfs:///user/spark/warehouse
    spark.sql.orderByOrdinal TRUE
    spark.sql.hive.convertMetastoreParquet TRUE
    spark.sql.groupByOrdinal TRUE
    spark.sql.hive.thriftServer.async TRUE
    spark.sql.thriftserver.scheduler.pool <undefined>
    spark.sql.orc.filterPushdown FALSE
    spark.sql.adaptive.shuffle.targetPostShuffleInputSize 67108864b
    spark.sql.sources.default parquet
    spark.sql.parquet.compression.codec snappy
    spark.sql.hive.metastore.version 1.2.1
    spark.sql.sources.partitionDiscovery.enabled TRUE
    spark.sql.crossJoin.enabled FALSE
    spark.sql.parquet.writeLegacyFormat FALSE
    spark.sql.hive.verifyPartitionPath FALSE
    spark.sql.variable.substitute TRUE
    spark.sql.thriftserver.ui.retainedStatements 200
    spark.sql.hive.convertMetastoreParquet.mergeSchema FALSE
    spark.sql.parquet.enableVectorizedReader TRUE
    spark.sql.parquet.mergeSchema FALSE
    spark.sql.parquet.binaryAsString FALSE
    spark.sql.columnNameOfCorruptRecord _corrupt_record
    spark.sql.files.maxPartitionBytes 134217728
    spark.sql.streaming.checkpointLocation <undefined>
    spark.sql.variable.substitute.depth 40
    spark.sql.parquet.int96AsTimestamp TRUE
    spark.sql.autoBroadcastJoinThreshold 10485760
    spark.sql.pivotMaxValues 10000
    spark.sql.sources.partitionColumnTypeInference.enabled TRUE
    spark.sql.hive.metastore.jars builtin
    spark.sql.thriftserver.ui.retainedSessions 200
    spark.sql.sources.maxConcurrentWrites 1
    spark.sql.parquet.output.committer.class org.apache.parquet.hadoop.ParquetOutputCommitter
    
    
  • 相关阅读:
    修复PLSQL Developer 与 Office 2010的集成导出Excel 功能
    Using svn in CLI with Batch
    mysql 备份数据库 mysqldump
    Red Hat 5.8 CentOS 6.5 共用 输入法
    HP 4411s Install Red Hat Enterprise Linux 5.8) Wireless Driver
    变更RHEL(Red Hat Enterprise Linux 5.8)更新源使之自动更新
    RedHat 5.6 问题简记
    Weblogic 9.2和10.3 改密码 一站完成
    ExtJS Tab里放Grid高度自适应问题,官方Perfect方案。
    文件和目录之utime函数
  • 原文地址:https://www.cnblogs.com/wwxbi/p/6114410.html
Copyright © 2011-2022 走看看