zoukankan      html  css  js  c++  java
  • ratelimit.js滑动窗口限流

    ratelimit.js基于滑动窗口和redis实现限流,为dudleycarr所开源,源码见github:https://github.com/dudleycarr/ratelimit.js

    原理见:Introduction to rate limiting with Redis [Part 2]

    核心lua脚本见下文,本文只是在lua脚本上加了几行注视,做个笔记

    local limits = cjson.decode(ARGV[1])
    local now = tonumber(ARGV[2])
    local weight = tonumber(ARGV[3] or '1')
    local longest_duration = limits[1][1] or 0
    local saved_keys = {}
    -- handle cleanup and limit checks
    for i, limit in ipairs(limits) do
     
        local duration = limit[1]
        longest_duration = math.max(longest_duration, duration)
         -- duration/precision = blocks个数
        local precision = limit[3] or duration  
        precision = math.min(precision, duration)
        local blocks = math.ceil(duration / precision)
        local saved = {}
        table.insert(saved_keys, saved)
        saved.block_id = math.floor(now / precision)
        -- trim_before为当前时间的上一个duration的block所在位置
        saved.trim_before = saved.block_id - blocks + 1
        saved.count_key = duration .. ':' .. precision .. ':'
        saved.ts_key = saved.count_key .. 'o'
        for j, key in ipairs(KEYS) do
     
            local old_ts = redis.call('HGET', key, saved.ts_key)
            old_ts = old_ts and tonumber(old_ts) or saved.trim_before
            if old_ts > now then
                -- 当固定窗口时可能会出现ts > now
                -- don't write in the past
                return 1
            end
     
            -- discover what needs to be cleaned up
            local decr = 0
            local dele = {}
            local trim = math.min(saved.trim_before, old_ts + blocks)
            for old_block = old_ts, trim - 1 do
                local bkey = saved.count_key .. old_block
                local bcount = redis.call('HGET', key, bkey)
                if bcount then
                    decr = decr + tonumber(bcount)
                    table.insert(dele, bkey)
                end
            end
     
            -- handle cleanup
           -- 释放之前的资源
            local cur
            if #dele > 0 then
                redis.call('HDEL', key, unpack(dele))
                cur = redis.call('HINCRBY', key, saved.count_key, -decr)
            else
                cur = redis.call('HGET', key, saved.count_key)
            end
     
            -- check our limits
            if tonumber(cur or '0') + weight > limit[2] then
                return 1
            end
        end
    end
    
    
    -- there is enough resources, update the counts
    for i, limit in ipairs(limits) do
        local saved = saved_keys[i]
     
        for j, key in ipairs(KEYS) do
            -- update the current timestamp, count, and bucket count
            redis.call('HSET', key, saved.ts_key, saved.trim_before)
            redis.call('HINCRBY', key, saved.count_key, weight)
            redis.call('HINCRBY', key, saved.count_key .. saved.block_id, weight)
        end
    end
     
    -- We calculated the longest-duration limit so we can EXPIRE
    -- the whole HASH for quick and easy idle-time cleanup :)
    if longest_duration > 0 then
        for _, key in ipairs(KEYS) do
            redis.call('EXPIRE', key, longest_duration)
        end
    end
     
    return 0
  • 相关阅读:
    16-爬虫之scrapy框架手动请求发送实现全站数据爬取03
    15-爬虫之scrapy框架基于管道实现数据库备份02
    14-爬虫之scrapy框架的基本使用01
    13-爬虫之js加密,解密,混淆,逆向破解思路
    12-基于selenium实现12306模拟登录,及京东登录滑动缺口验证模拟登录
    11-selenium浏览器自动化
    10-异步爬虫(线程池/asyncio协程)实战案例
    09-百度ai图片识别
    08-百度ai语音合成
    RNN
  • 原文地址:https://www.cnblogs.com/wkzhao/p/10586525.html
Copyright © 2011-2022 走看看