zoukankan      html  css  js  c++  java
  • 抽象语法树(AST)

     

    AST描述

      在计算机科学中,抽象语法树(AST)或语法树是用编程语言编写的源代码的抽象语法结构的树表示。树的每个节点表示在源代码中出现的构造。语法是“抽象的”,因为它不代表真实语法中出现的每个细节,而只是结构,内容相关的细节。例如,分组括号 在树结构中是隐式的,并且可以通过具有三个分支的单个节点来表示类似于if-condition-then表达式的句法结构。

      这将抽象语法树与传统上指定的解析树区分开来,这些语法树通常由解析器在源代码转换和编译过程中构建。一旦构建,通过后续处理(例如,上下文分析)将附加信息添加到AST 。

      抽象语法树也用于程序分析和程序转换系统。

      参考:[维基百科](https://en.wikipedia.org/wiki/Abstract_syntax_tree)

    解析器Parser

      JavaScript Parser是把js源码转化为抽象语法树的解析器,一般分为词法分析、语法分析及代码生成或执行。

    词法分析

      词法分析阶段会把字符串形式的代码转换为令牌(Tokens)流。可将令牌看作是一个扁平的语法片段数组。

    var answer = 6 * 7;
    
    //Tokens
    [
        {
            "type": "Keyword",
            "value": "var",
            "range": [
                34,
                37
            ],
            "loc": {
                "start": {
                    "line": 2,
                    "column": 0
                },
                "end": {
                    "line": 2,
                    "column": 3
                }
            }
        },
        {
            "type": "Identifier",
            "value": "answer",
            "range": [
                38,
                44
            ],
            "loc": {
                "start": {
                    "line": 2,
                    "column": 4
                },
                "end": {
                    "line": 2,
                    "column": 10
                }
            }
        },
        {
            "type": "Punctuator",
            "value": "=",
            "range": [
                45,
                46
            ],
            "loc": {
                "start": {
                    "line": 2,
                    "column": 11
                },
                "end": {
                    "line": 2,
                    "column": 12
                }
            }
        },
        {
            "type": "Numeric",
            "value": "6",
            "range": [
                47,
                48
            ],
            "loc": {
                "start": {
                    "line": 2,
                    "column": 13
                },
                "end": {
                    "line": 2,
                    "column": 14
                }
            }
        },
        {
            "type": "Punctuator",
            "value": "*",
            "range": [
                49,
                50
            ],
            "loc": {
                "start": {
                    "line": 2,
                    "column": 15
                },
                "end": {
                    "line": 2,
                    "column": 16
                }
            }
        },
        {
            "type": "Numeric",
            "value": "7",
            "range": [
                51,
                52
            ],
            "loc": {
                "start": {
                    "line": 2,
                    "column": 17
                },
                "end": {
                    "line": 2,
                    "column": 18
                }
            }
        },
        {
            "type": "Punctuator",
            "value": ";",
            "range": [
                52,
                53
            ],
            "loc": {
                "start": {
                    "line": 2,
                    "column": 18
                },
                "end": {
                    "line": 2,
                    "column": 19
                }
            }
        }
    ]
    

      

    语法分析  

      语法分析阶段会把一个令牌流转换成抽象语法树(AST)的形式,这个阶段会使用令牌中的信息把它们转换成一个AST的树结构。

    // Life, Universe, and Everything
    var answer = 6 * 7;
    // Syntax
    {
        "type": "Program",
        "body": [
            {
                "type": "VariableDeclaration",
                "declarations": [
                    {
                        "type": "VariableDeclarator",
                        "id": {
                            "type": "Identifier",
                            "name": "answer",
                            "range": [
                                38,
                                44
                            ],
                            "loc": {
                                "start": {
                                    "line": 2,
                                    "column": 4
                                },
                                "end": {
                                    "line": 2,
                                    "column": 10
                                }
                            }
                        },
                        "init": {
                            "type": "BinaryExpression",
                            "operator": "*",
                            "left": {
                                "type": "Literal",
                                "value": 6,
                                "raw": "6",
                                "range": [
                                    47,
                                    48
                                ],
                                "loc": {
                                    "start": {
                                        "line": 2,
                                        "column": 13
                                    },
                                    "end": {
                                        "line": 2,
                                        "column": 14
                                    }
                                }
                            },
                            "right": {
                                "type": "Literal",
                                "value": 7,
                                "raw": "7",
                                "range": [
                                    51,
                                    52
                                ],
                                "loc": {
                                    "start": {
                                        "line": 2,
                                        "column": 17
                                    },
                                    "end": {
                                        "line": 2,
                                        "column": 18
                                    }
                                }
                            },
                            "range": [
                                47,
                                52
                            ],
                            "loc": {
                                "start": {
                                    "line": 2,
                                    "column": 13
                                },
                                "end": {
                                    "line": 2,
                                    "column": 18
                                }
                            }
                        },
                        "range": [
                            38,
                            52
                        ],
                        "loc": {
                            "start": {
                                "line": 2,
                                "column": 4
                            },
                            "end": {
                                "line": 2,
                                "column": 18
                            }
                        }
                    }
                ],
                "kind": "var",
                "range": [
                    34,
                    53
                ],
                "loc": {
                    "start": {
                        "line": 2,
                        "column": 0
                    },
                    "end": {
                        "line": 2,
                        "column": 19
                    }
                },
                "leadingComments": [
                    {
                        "type": "Line",
                        "value": " Life, Universe, and Everything",
                        "range": [
                            0,
                            33
                        ],
                        "loc": {
                            "start": {
                                "line": 1,
                                "column": 0
                            },
                            "end": {
                                "line": 1,
                                "column": 33
                            }
                        }
                    }
                ]
            }
        ],
        "sourceType": "script",
        "range": [
            34,
            53
        ],
        "loc": {
            "start": {
                "line": 2,
                "column": 0
            },
            "end": {
                "line": 2,
                "column": 19
            }
        }
    }
    View Code

    常见的AST node类型

      AST树每一层结构也被叫做节点(Node)。一个AST可以由单一的节点或是成百上千个节点够成,通过组合在一起来描述静态分析的程序语法(静态分析是在不需要执行代码的前提下对代码进行分析的处理过程 (执行代码的同时进行代码分析即是动态分析)。 静态分析的目的是多种多样的, 它可用于语法检查,编译,代码高亮,代码转换,优化,压缩等等场景。)。

     Node types:https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Parser_API#Node_objects

    babel AST node types:https://github.com/babel/babylon/blob/master/ast/spec.md
    
    

    使用esprima生成抽象语法树流程

      以上生成的AST树可视化工具:[esprima](http://esprima.org/demo/parse.html#)生成,通过[esprima](https://github.com/jquery/esprima)将源码生成抽象语法树,并通过[estraverse](https://github.com/estools/estraverse)遍历并更新AST,最后通过[escodegen](https://github.com/estools/escodegen)将AST重新生成源码。

    常用的JavaScript Parser

      [Acorn](https://github.com/acornjs/acorn),[Esprima](https://github.com/jquery/esprima),[UglifyJS2](https://github.com/mishoo/UglifyJS2),[Babel(https://github.com/babel/babel)等

    总结

      最后,根据对抽象语法树的大概了解做了个demo级的将命名函数转换为exports函数的npm包:https://www.npmjs.com/package/fn2export,欢迎使用~~~

  • 相关阅读:
    Java的静态块与实例块(转)
    Programming Ability Test学习 1031. Hello World for U (20)
    Programming Ability Test学习 1011. World Cup Betting (20)
    Programming Ability Test学习 1027. Colors in Mars (20)
    Programming Ability Test学习 1064. Complete Binary Search Tree (30)
    Programming Ability Test学习 1008. Elevator (20)
    【maven详解-生命周期】Maven的生命周期和插件
    【maven详解-插件】maven插件学习之源码插件Source Xref
    $(document).ready(){}、$(fucntion(){})、(function(){})(jQuery)onload()的区别
    你还没真的努力过,就轻易输给了懒惰
  • 原文地址:https://www.cnblogs.com/aaron-pan/p/10572953.html
Copyright © 2011-2022 走看看