zoukankan      html  css  js  c++  java
  • Redis内存淘汰

    1 内存淘汰是指redis使用的内存超过了配置文件里分配给它的内存大小

     maxmemory 1GB 

    2 什么时候触发

    其实,在redis执行每一个命令的时候,都会检查当前的内存是否超过最大内存。如果超过,就会触发

    1)客户端执行一条新命令,导致数据库需要增加数据(比如set key value)

    2)Redis会检查内存使用,如果内存使用超过maxmemory,就会按照置换策略删除一些key

    3)新的命令执行成功

    3 每次淘汰的内存数据量

    如果我们持续的写数据会导致内存达到或超出上限maxmemory,但是置换策略会将内存使用降低到上限以下。

    如果一次需要使用很多的内存(比如一次写入一个很大的set),那么,Redis的内存使用可能超出最大内存限制一段时间。

    源码中是这样写的

    /* Compute how much memory we need to free. */
        mem_tofree = mem_used - server.maxmemory;
        mem_freed = 0;
        //(2)接下来就判断淘汰策略是基于所有的键还是只是基于设置了过期时间的键,如果是针对所有的键,就从server.db[j].dict中取数据,如果是针对设置了过期时间的键,就从server.db[j].expires中取数据。
        while (mem_freed < mem_tofree) {

    也就是说,只要降到maxmemory之下,while循环就会停止

    4 淘汰算法

    volatile-lru:在那些设置了expire过期时间的缓存中,清除最少用的旧缓存,然后保存新的缓存
    * allkeys-lru:清除最少用的旧缓存,然后保存新的缓存
    * volatile-lfu:在那些设置了expire过期时间的缓存中,清除最长时间未用的旧缓存,然后保存新的缓存
    * allkeys-lfu:清除最长时间未用的旧缓存,然后保存新的缓存
    * volatile-random:在那些设置了expire过期时间的缓存中,随机删除缓存
    * allkeys-random:在所有的缓存中随机删除(不推荐)
    * volatile-ttl:在那些设置了expire过期时间的缓存中,删除即将过期的
    * noeviction:旧缓存永不过期,新缓存设置不了,返回错误

    方便记忆其实就是分为3种,永不过期特殊情况一种。volatile只针对设置了过期时间一种,allkeys所有的key都算

    5 定时删除的相关逻辑 代码在 redis.c activeExpireCycle
    void activeExpireCycle(int type) {
        /* This function has some global state in order to continue the work
         * incrementally across calls. */
        static unsigned int current_db = 0; /* Last DB tested. */
        static int timelimit_exit = 0;      /* Time limit hit in previous call? */
        static long long last_fast_cycle = 0; /* When last fast cycle ran. */
    
        int j, iteration = 0;
        int dbs_per_call = REDIS_DBCRON_DBS_PER_CALL;
        long long start = ustime(), timelimit;
    
        if (type == ACTIVE_EXPIRE_CYCLE_FAST) {
            /* Don't start a fast cycle if the previous cycle did not exited
             * for time limt. Also don't repeat a fast cycle for the same period
             * as the fast cycle total duration itself. */
            if (!timelimit_exit) return;
            if (start < last_fast_cycle + ACTIVE_EXPIRE_CYCLE_FAST_DURATION*2) return;
            last_fast_cycle = start;
        }
    
        /* We usually should test REDIS_DBCRON_DBS_PER_CALL per iteration, with
         * two exceptions:
         *
         * 1) Don't test more DBs than we have.
         * 2) If last time we hit the time limit, we want to scan all DBs
         * in this iteration, as there is work to do in some DB and we don't want
         * expired keys to use memory for too much time. */
        if (dbs_per_call > server.dbnum || timelimit_exit)
            dbs_per_call = server.dbnum;
    
        /* We can use at max ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC percentage of CPU time
         * per iteration. Since this function gets called with a frequency of
         * server.hz times per second, the following is the max amount of
         * microseconds we can spend in this function. */
        timelimit = 1000000*ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC/server.hz/100;//
       //ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC默认25 也就是说执行清除过期键的时间没一次的serverCron不能超过25ms,server.hz默认 100 timelimit_exit
    = 0; if (timelimit <= 0) timelimit = 1; if (type == ACTIVE_EXPIRE_CYCLE_FAST) timelimit = ACTIVE_EXPIRE_CYCLE_FAST_DURATION; /* in microseconds. */ for (j = 0; j < dbs_per_call; j++) { int expired; redisDb *db = server.db+(current_db % server.dbnum);//有上次的记忆,从上次的数据库继续执行 /* Increment the DB now so we are sure if we run out of time * in the current DB we'll restart from the next. This allows to * distribute the time evenly across DBs. */ current_db++; /* Continue to expire if at the end of the cycle more than 25% * of the keys were expired. */ do { unsigned long num, slots; long long now, ttl_sum; int ttl_samples; /* If there is nothing to expire try next DB ASAP. */ if ((num = dictSize(db->expires)) == 0) { db->avg_ttl = 0; break; } slots = dictSlots(db->expires); now = mstime(); /* When there are less than 1% filled slots getting random * keys is expensive, so stop here waiting for better times... * The dictionary will be resized asap. */ if (num && slots > DICT_HT_INITIAL_SIZE && (num*100/slots < 1)) break; /* The main collection cycle. Sample random keys among keys * with an expire set, checking for expired ones. */ expired = 0; ttl_sum = 0; ttl_samples = 0; if (num > ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP) num = ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP;//20也就是说每次最多处理20个,注意是检查20个,满足条件才处理
  • 相关阅读:
    公司实习职位与要求
    段子
    Python 练习项目1 弹球游戏
    精准控制PWM脉冲的频率和数量
    ST Link 调试问题总结
    验证
    大道至简(第六章)读后感
    动手动脑
    大道至简(第五i章)读后感
    数组课后作业
  • 原文地址:https://www.cnblogs.com/juniorMa/p/14978984.html
Copyright © 2011-2022 走看看