zoukankan      html  css  js  c++  java
  • redis 大批量数据插入导致MISCONF Redis is configured to save RDB snapshots的解决

    PS:之前写过一遍,那个方法没有彻底解决,现找到真正的解决方法

    环境:redis 3.2.100 windows版(注意!!!这是关键),win10,redis客户端spring boot 2.0.7,以及配对的spring data redis

    某功能会频繁地大批量地往redis写入数据,数据量大概10秒内超过10000条

    插入时可能会报错,报错时间点不定,大概都是运行一段时间后报错,错误信息如下:

    nested exception is io.lettuce.core.RedisCommandExecutionException: MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.

    同时在redis的log找到如下报错:

    === REDIS BUG REPORT START: Cut & paste starting from here ===
    Redis version: 3.2.100
    [18808] 15 Mar 11:12:49.805 # --- EXCEPTION_ACCESS_VIOLATION
    [18808] 15 Mar 11:12:49.805 # --- STACK TRACE
    redis-server.exe!LogStackTrace(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:95)(0x0014E250, 0x0014FF90, 0x00000000, 0x4013A7F8)
    redis-server.exe!UnhandledExceptiontHandler(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x00000000, 0x00000000, 0x00000000, 0x38D470E0)
    KERNELBASE.dll!UnhandledExceptionFilter(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x401555B4, 0x38E3C15C, 0x00000000, 0x00000000)
    ntdll.dll!memset(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x0014EEB0, 0x00000000, 0x40140E48, 0x0014EEB0)
    ntdll.dll!_C_specific_handler(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x00000000, 0x0014E3C0, 0x0014E9C0, 0x00000000)
    ntdll.dll!_chkstk(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x0014E3C0, 0x0014E9C0, 0x3623AB10, 0x00000000)
    ntdll.dll!RtlWalkFrameChain(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x00000000, 0x511B7D55, 0x0014F220, 0x6506FC19)
    ntdll.dll!KiUserExceptionDispatcher(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x40094F01, 0x0014F1F0, 0x400BFA57, 0x0014F2E0)
    redis-server.exe!dictSdsHash(c:
    elease
    edissrcserver.c:496)(0x0014F1F0, 0x400BFA57, 0x0014F2E0, 0x00000001)
    redis-server.exe!dictFind(c:
    elease
    edissrcdict.c:517)(0x0014F220, 0x00000001, 0x0000002B, 0x00000001)
    redis-server.exe!getExpire(c:
    elease
    edissrcdb.c:871)(0x00000001, 0x0014F2E0, 0x0014F2E0, 0x03C07040)
    redis-server.exe!rdbSaveRio(c:
    elease
    edissrc
    db.c:814)(0x40167210, 0x00DD0000, 0x00000005, 0x011D2754)
    redis-server.exe!rdbSave(c:
    elease
    edissrc
    db.c:884)(0x00DD0000, 0x00DD0000, 0x5C8B2352, 0x00000005)
    redis-server.exe!QForkChildInit(c:
    elease
    edissrcwin32_interopwin32_qfork.cpp:337)(0x00000005, 0x00000000, 0x004BEF40, 0x00000005)
    redis-server.exe!QForkStartup(c:
    elease
    edissrcwin32_interopwin32_qfork.cpp:515)(0x00000006, 0x00000000, 0x00000000, 0x004A5550)
    redis-server.exe!main(c:
    elease
    edissrcwin32_interopwin32_qfork.cpp:1240)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    redis-server.exe!__tmainCRTStartup(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    KERNEL32.DLL!BaseThreadInitThunk(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    ntdll.dll!RtlUserThreadStart(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    ntdll.dll!RtlUserThreadStart(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    [18808] 15 Mar 11:12:50.398 # --- INFO OUTPUT
    [5660] 15 Mar 11:12:55.080 # fork operation failed
    [5660] 15 Mar 11:12:55.226 # Background saving terminated by signal 1
    [5660] 15 Mar 11:12:55.353 * 10000 changes in 60 seconds. Saving...

    网上能找到很多解决方法(具体执行baidu),一种主流的方法是redis配置stop-writes-on-bgsave-error设为false,但这种方法只是让出错不停止,错误还是存在的,所以不想用这种方法

    另外原因大概也清楚:就是redis写入数据会先写到内存,然后每隔一段时间会把内存数据写入rdb(一个文件数据库),以实现不会因为关机内存数据丢失就没了数据。问题是在写入rdb时出错

    但不知道写入rdb出错原因在哪,网上很多说是内存不够,我看了内存占用也很低,只用了200+m

    此问题直接原因应该是写入rdb时出错,但根本原因在于redis本身有bug

    redis只有linux版,而windows版是微软自己做的,而且也停止更新,大概因此有bug也是正常吧

    解决方法是redis(服务端)使用3.0.504,此版本已解决此bug

  • 相关阅读:
    mysql中字符集和排序规则说明
    结束进程的批处理文件
    内有干货!2个人3个月怎样从零完毕一款社区App《林卡》
    九度OJ 1006 ZOJ问题 (这题測试数据有问题)
    简易版的堆的写法
    hbase
    JNDI配置c3p0连接池
    [effictive c++] 条款04 确定对象被使用前已被初始化
    第九十五题(推断一字符串是不是对称的)
    OpenFace库(Tadas Baltrusaitis)中基于Haar Cascade Classifiers进行人脸检測的測试代码
  • 原文地址:https://www.cnblogs.com/cannel/p/11078096.html
Copyright © 2011-2022 走看看