zoukankan      html  css  js  c++  java
  • [development][PCRE] old PCRE

    介绍, man手册 

    txt版 http://www.pcre.org/original/pcre.txt

    html版 http://www.pcre.org/original/doc/html/pcre.html

    In addition to the Perl-compatible matching function, PCRE contains an alternative function that matches the same compiled patterns in a different way. 
    In certain circumstances, the alternative function has some advantages. For a discussion of the two matching algorithms, see the pcrematching page.

    pcrematching:

    http://www.pcre.org/original/doc/html/pcrematching.html

    摘要:

      0.  是批量处理的意思? 一个pattern处理多个subject么?

    The set of strings that are matched by a regular expression can be represented as a tree structure.

      1.  Jeffrey Friedl's book "Mastering Regular Expressions"

      中文版:精通正则表达式:https://book.douban.com/subject/2154713/

      英文版PDF:https://doc.lagout.org/Others/O%27Reilly%20-%20Mastering%20Regular%20Expressions.pdf

      2, PCRE匹配分标准接口(pcre_exec(), pcre16_exec() and pcre32_exec() functions. )和非标准接口( pcre_dfa_exec(), pcre16_dfa_exec() and pcre32_dfa_exec() functions )两种。

        前者在同一个串中只能返回一个匹配结果,或者可以同时返回一个串中的多个匹配结果。

        标准接口返回的结果有可能是最长串,最短串或任意长度的串,这取决于贪婪与非贪婪的设置。

        标准接口就是NFA algorithm是深度优先查找树,同时可以有贪婪(greedy)与非贪婪(ungreedy)两种控制种类。

        非标准接口为广度优先查找树,为DFA算法( In Friedl's terminology, this is a kind of "DFA algorithm", though it is not implemented as a traditional finite state machine (it keeps multiple states active simultaneously).)subject串的扫描会一直进行到串的尾部或者没有其他需要遍历的路径。所有的已终结路径即代表了全部的匹配结果,返回的结果按照字符串长度递减。有一个开关设置第一个命中即返回,也就是最短命中串。

      3.  非标准方法的优点:

        a, 匹配多个结果,尤其是找到最长匹配。

        b, 可以对超长的subject数据进行多次分批次的匹配。

         非标准方法的缺点:

        a, 比标准方法慢。

        b, 不支持子串提取。

        c, Although atomic groups are supported, their use does not provide the performance advantage that it does for the standard algorithm.

    pcrejit:

    http://www.pcre.org/original/doc/html/pcrejit.html

    摘要:

      JIT提供特别深度的优化. 牺牲额外的处理步骤,从而提高匹配性能。适合一次pattern编译多次match操作的应用场景。

      1. 只支持标准PCRE接口,不支持DFA匹配模式。

      2. PCRE默认不打开JIT,需要在编译的时候增加--enable-jit选项。

      3.  有硬件平台限制

      ARM v5, v7, and Thumb2
      Intel x86 32-bit and 64-bit
      MIPS 32-bit
      Power PC 32-bit and 64-bit
      SPARC 32-bit (experimental)

      4.  the pcre_jit_exec() function was not available at all before 8.32

      5.  The JIT compiler generates different optimized code for each of the three modes (normal, soft partial, hard partial). When pcre_exec() is called, the appropriate code is run if it is available. Otherwise, the pattern is matched using interpretive code.

      6.  There are some pcre_exec() options that are not supported for JIT execution. There are also some pattern items that JIT cannot handle. Details are given below. In both cases, execution automatically falls back to the interpretive code.

      7.  Once a pattern has been studied, with or without JIT, it can be used as many times as you like for matching different subject strings.

      8.   The code that is generated by the JIT compiler is architecture-specific, and is also position dependent. For those reasons it cannot be saved (in a file or database) and restored later like the bytecode and other data of a compiled pattern. 

      more info: http://www.pcre.org/original/doc/html/pcreprecompile.html

      9.  有时候JIT机器码没有成功编译,但是pcre_exec()仍然正常运行,只不过fallback回了解释码。我们在高性能场景下不希望使用解释码的时候,使用API pcre_jit_exec().

    Because the API described above falls back to interpreted execution when JIT is not available, it is convenient for programs that are written 
    for general use in many environments. However, calling JIT via pcre_exec() does have a performance impact. Programs that are written for use
    where JIT is known to be available, and which need the best possible performance, can instead use a "fast path" API to call JIT execution directly
    instead of calling pcre_exec() (obviously only for patterns that have been successfully studied by JIT).

       10.  pcre_exec()会做参数合法性的检测。pcre_jit_exec()为了提高性能,不做合法性检测,如果参数不合法,结果无法预期。

    API: 

    http://www.pcre.org/original/doc/html/pcreapi.html

    摘要:

      1,

    The functions pcre_compile(), pcre_compile2(), pcre_study(), and pcre_exec() are used for compiling and matching regular expressions in a Perl-compatible manner.

      2, compile a pattern

        http://www.pcre.org/original/doc/html/pcreapi.html#SEC11

      3,    studying a pattern

    Studying a pattern does two things: first, a lower bound for the length of subject string that is needed to match the pattern is computed. This does not mean 
    that there are any strings of that length that match, but it does guarantee that no shorter strings match. The value is used to avoid wasting time by trying
    to match strings that are shorter than the lower bound. Studying a pattern
    is also useful for non-anchored patterns that do not have a single fixed starting character. A bitmap of possible starting bytes is created.
    This speeds up finding a position in the subject at which to start matching.

      4,   matching a pattern

    However, it is possible to save compiled patterns and study data, and then use them later in different processes, possibly even on different hosts. 
    For a discussion about this, see the pcreprecompile documentation.

    对比一下PCRE2:

    [development][PCRE] PCRE

    trie:

    https://zh.wikipedia.org/zh-hans/Trie

    -------------------

    黑哥的blog:http://www.cnblogs.com/zzqcn/p/3525636.html

    这个讲的很好,对比PCRE、PCRE-JIT,hyperscan:https://mp.weixin.qq.com/s?__biz=MzI3NDA4ODY4MA==&mid=2653334341&idx=1&sn=bf10ca6d8ca1452723b84a62f7fc436d&chksm=f0cb5cc2c7bcd5d4f423af8d78aeb58dd6d9494c1562b1e775579321df3b9f59a951656100d0&scene=21#wechat_redirect

  • 相关阅读:
    redisTemplate写哈希表遇到的坑
    embedded-redis在单元测试中的使用
    使用Standford coreNLP进行中文命名实体识别
    字符编码和文件编码
    Elasticsearch提示low disk watermark [85%] exceeded on [UTyrLH40Q9uIzHzX-yMFXg][Sonofelice][/Users/baidu/Documents/work/soft/data/nodes/0] free: 15.2gb[13.4%], replicas will not be assigned to this node
    nginx.conf常用配置解析
    使用nginx搭建文件下载服务器
    lua连接数据库操作示例代码
    spring常见注解说明
    lua相关库安装常见问题
  • 原文地址:https://www.cnblogs.com/hugetong/p/8619196.html
Copyright © 2011-2022 走看看