zoukankan      html  css  js  c++  java
  • savsgio/kratgo 基于fasthttp 开发的一个cache proxy 服务

    savsgio/kratgo 基于fasthttp 开发的web cache 服务,包含以下特性

    支持的特性

    • cache proxy
    • lb (顺序,然后轮转模式,没有权重)
    • 配置cache 规则
    • 配置http 请求头
    • 管理api 支持,基于管理api 我们可以进行cache 清理

    参考使用

    • docker-compose 文件
    version: "3"
    services:
      proxy: 
        image: savsgio/kratgo
        command: -config /etc/config.yaml
        ports:
        - "6081:6081"
        volumes: 
         - "./config.yaml:/etc/config.yaml"
    • 配置
    ###########################
    #   KRATGO CONFIGURATION
    ###########################
    # IMPORTANT: Be careful with the tabulation indentation
    # --- Variables ---
    # $(method) : request method
    # $(host) : request host
    # $(path) : request path
    # $(contentType) : response backend's content type
    # $(statusCode) : response backend's status code
    # $(req.header::<NAME>) : request header name
    # $(resp.header::<NAME>) : response header name
    # $(cookie::<NAME>) : request cookie name
    # --- Operators ---
    # Modifiers: + - / * & | ^ ** % >> <<
    # Comparators: > >= < <= == != =~ !~
    # Logical ops: || &&
    # Numeric constants, as 64-bit floating point (12345.678)
    # String constants (single quotes: 'foobar')
    # Date constants (single quotes, using any permutation of RFC3339, ISO8601, ruby date, or unix date; date parsing is automatically tried with any string constant)
    # Boolean constants: true false
    # Parenthesis to control order of evaluation ( )
    # Arrays (anything separated by , within parenthesis: (1, 2, 'foo'))
    # Prefixes: ! - ~
    # Ternary conditional: ? :
    # Null coalescence: ??
    # --- Log ---
    # Log level: fatal | error | warning | info | debug
    # Log output:
    #   - console: Write output in standard error
    #   - <file path>: Write output in log file
    logLevel: debug
    logOutput: ./kratgo.log
    # --- Cache ---
    # ttl: Cache expiration in minutes
    # cleanFrequency: Interval in minutes between removing expired entries (clean up)
    # maxEntries: Max number of entries in cache. Used only to calculate initial size for cache
    # maxEntrySize: Max size of entry in bytes
    # hardMaxCacheSize: Limit for cache size in MB (Default value is 0 which means unlimited size)
    cache:
      ttl: 10
      cleanFrequency: 1
      maxEntries: 600000
      maxEntrySize: 500
      hardMaxCacheSize: 0
    # --- Invalidator ---
    # maxWorkers: Maximum workers to execute invalidations
    invalidator:
      maxWorkers: 5
    # --- Proxy ---
    # addr: IP and Port of Kratgo
    # backendAddrs: Array with "addr:port" of the backends
    # response: Configuration to manipulate reponse (Optional)
    #   headers:
    #     set: Configuration to SET headers from response (Optional)
    #       - name: Header name
    #         value: Value of header
    #         if: Condition to set this header (Optional)
    #
    #     unset: Configuration to UNSET headers from response (Optional)
    #       - name: Header name
    #         if: Condition to unset this header (Optional)
    #
    # nocache: Conditions to not save in cache the backend response (Optional)
    proxy:
      addr: 0.0.0.0:6081
      backendAddrs:
        [
           <address>
        ]
      response:
        headers:
          set:
            - name: X-Kratgo
              value: true
          unset:
            - name: Set-Cookie
              if: $(req.header::X-Requested-With) != 'XMLHttpRequest'
      nocache:
        - $(req.header::X-Requested-With) == 'XMLHttpRequest'
    # --- Admin ---
    # addr: IP and Port of admin api
    admin:
      addr: 0.0.0.0:6082
    • 效果

    同时,如果断开网络链接,你会发现网站依然还能访问(因为cache了)

    基于fasthttp 开发一个自己proxy 的简单代码

     
    package main
    import (
      "flag"
      "fmt"
      "log"
      "github.com/valyala/fasthttp"
    )
    var (
      add = flag.String("add", "localhost", "proxy address")
    )
    type myLog struct {
      add string
    }
    func (l myLog) Printf(format string, args ...interface{}) {
      log.Printf(format, args...)
    }
    func (l myLog) myhandler(ctx *fasthttp.RequestCtx) {
      backend := fasthttp.HostClient{
        Addr: l.add,
      }
      backend.Do(&ctx.Request, &ctx.Response)
    }
    func main() {
      flag.Parse()
      fmt.Println(*add)
      log := myLog{
        add: *add,
      }
      s := fasthttp.Server{
        Handler:      log.myhandler,
        LogAllErrors: true,
        Logger:       log,
      }
      s.ListenAndServe(":8000")
    }

    源码简单说明

    savsgio/kratgo 的源码并不难,对于cache proxy 基于fasthttp 的Server提供,对于cache 部分基于bigcache
    Server handler 使用了fasthttp 的fasthttp.HostClient,配置解析部分基于yaml,指定语法解析(配置自定义参数)
    使用Knetic/govaluate,同时为了性能作者很多地方使用了sync.Pool ,lb 部分实现很简单,就是数据索引的移动
    当超过数据最大值得时候,重置索引为0,同时对于cache 管理,作者基于channel 实现了一个校验任务,实际可以
    看看源码,实际上以后也分享过gregjones/httpcache的开源实现,也是一个不错的选择

    参考资料

    https://github.com/savsgio/kratgo
    https://github.com/Knetic/govaluate
    https://github.com/valyala/fasthttp
    https://github.com/allegro/bigcache
    https://github.com/gregjones/httpcache

  • 相关阅读:
    谈谈焦虑
    Bridge
    Abstract Factory
    开篇辞
    缓存穿透和缓存雪崩
    缓存与数据库的一致性思考
    GDB
    代码格式化
    CSMA/CA协议
    18年秋招(19届)复习指导+面经总结,手把手教你进大厂——阿里、网易、京东、多益等
  • 原文地址:https://www.cnblogs.com/rongfengliang/p/14082424.html
Copyright © 2011-2022 走看看