Spark开发-SparkSQL引擎自定义配置

Spark catalyst的扩展

org.apache.spark.sql
	SparkSession
	  Inject extensions into the [[SparkSession]]. 
	  This allows a user to add Analyzer rules, Optimizer rules, Planning Strategies or a customized parser.
		* @since 2.2.0
		def withExtensions(f: SparkSessionExtensions => Unit): Builder = {f(extensions) this }

org.apache.spark.sql
	SparkSessionExtensions
     * This current provides the following extension points:
     * - Analyzer Rules.
     * - Check Analysis Rules
     * - Optimizer Rules.
     * - Planning Strategies.
     * - Customized Parser.
     * - (External) Catalog listeners.
     *
     * The extensions can be used by calling withExtension on the [[SparkSession.Builder]], for

使用：

  01.过Spark配置参数，具体参数名为spark.sql.extensions。
	   用户可以将的自定义函数实现定义为一个类 MyExtensions ，将完整类名作为参数值 
	   例如：  class MyExtensions extends (SparkSessionExtensions => Unit) {
 SparkSession.builder()
  .config("spark.sql.extensions", classOf[MyExtensions].getCanonicalName)
  .getOrCreate()
 02.或者 接收一个自定义函数作为参数，这个自定义函数以SparkSessionExtensions作为参数
 bui: ExtensionsBuilder
  02.SparkSession.builder().
   .withExtensions( bui)
   .getOrCreate()
  其中：

参考：

 http://spark.apache.org/releases/spark-release-2-2-0.html SPARK-18127: Add hooks and extension points to Spark