zoukankan      html  css  js  c++  java
  • redis 大批量数据插入导致MISCONF Redis is configured to save RDB snapshots的解决

    PS:之前写过一遍,那个方法没有彻底解决,现找到真正的解决方法

    环境:redis 3.2.100 windows版(注意!!!这是关键),win10,redis客户端spring boot 2.0.7,以及配对的spring data redis

    某功能会频繁地大批量地往redis写入数据,数据量大概10秒内超过10000条

    插入时可能会报错,报错时间点不定,大概都是运行一段时间后报错,错误信息如下:

    nested exception is io.lettuce.core.RedisCommandExecutionException: MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.

    同时在redis的log找到如下报错:

    === REDIS BUG REPORT START: Cut & paste starting from here ===
    Redis version: 3.2.100
    [18808] 15 Mar 11:12:49.805 # --- EXCEPTION_ACCESS_VIOLATION
    [18808] 15 Mar 11:12:49.805 # --- STACK TRACE
    redis-server.exe!LogStackTrace(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:95)(0x0014E250, 0x0014FF90, 0x00000000, 0x4013A7F8)
    redis-server.exe!UnhandledExceptiontHandler(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x00000000, 0x00000000, 0x00000000, 0x38D470E0)
    KERNELBASE.dll!UnhandledExceptionFilter(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x401555B4, 0x38E3C15C, 0x00000000, 0x00000000)
    ntdll.dll!memset(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x0014EEB0, 0x00000000, 0x40140E48, 0x0014EEB0)
    ntdll.dll!_C_specific_handler(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x00000000, 0x0014E3C0, 0x0014E9C0, 0x00000000)
    ntdll.dll!_chkstk(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x0014E3C0, 0x0014E9C0, 0x3623AB10, 0x00000000)
    ntdll.dll!RtlWalkFrameChain(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x00000000, 0x511B7D55, 0x0014F220, 0x6506FC19)
    ntdll.dll!KiUserExceptionDispatcher(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x40094F01, 0x0014F1F0, 0x400BFA57, 0x0014F2E0)
    redis-server.exe!dictSdsHash(c:
    elease
    edissrcserver.c:496)(0x0014F1F0, 0x400BFA57, 0x0014F2E0, 0x00000001)
    redis-server.exe!dictFind(c:
    elease
    edissrcdict.c:517)(0x0014F220, 0x00000001, 0x0000002B, 0x00000001)
    redis-server.exe!getExpire(c:
    elease
    edissrcdb.c:871)(0x00000001, 0x0014F2E0, 0x0014F2E0, 0x03C07040)
    redis-server.exe!rdbSaveRio(c:
    elease
    edissrc
    db.c:814)(0x40167210, 0x00DD0000, 0x00000005, 0x011D2754)
    redis-server.exe!rdbSave(c:
    elease
    edissrc
    db.c:884)(0x00DD0000, 0x00DD0000, 0x5C8B2352, 0x00000005)
    redis-server.exe!QForkChildInit(c:
    elease
    edissrcwin32_interopwin32_qfork.cpp:337)(0x00000005, 0x00000000, 0x004BEF40, 0x00000005)
    redis-server.exe!QForkStartup(c:
    elease
    edissrcwin32_interopwin32_qfork.cpp:515)(0x00000006, 0x00000000, 0x00000000, 0x004A5550)
    redis-server.exe!main(c:
    elease
    edissrcwin32_interopwin32_qfork.cpp:1240)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    redis-server.exe!__tmainCRTStartup(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    KERNEL32.DLL!BaseThreadInitThunk(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    ntdll.dll!RtlUserThreadStart(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    ntdll.dll!RtlUserThreadStart(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    [18808] 15 Mar 11:12:50.398 # --- INFO OUTPUT
    [5660] 15 Mar 11:12:55.080 # fork operation failed
    [5660] 15 Mar 11:12:55.226 # Background saving terminated by signal 1
    [5660] 15 Mar 11:12:55.353 * 10000 changes in 60 seconds. Saving...

    网上能找到很多解决方法(具体执行baidu),一种主流的方法是redis配置stop-writes-on-bgsave-error设为false,但这种方法只是让出错不停止,错误还是存在的,所以不想用这种方法

    另外原因大概也清楚:就是redis写入数据会先写到内存,然后每隔一段时间会把内存数据写入rdb(一个文件数据库),以实现不会因为关机内存数据丢失就没了数据。问题是在写入rdb时出错

    但不知道写入rdb出错原因在哪,网上很多说是内存不够,我看了内存占用也很低,只用了200+m

    此问题直接原因应该是写入rdb时出错,但根本原因在于redis本身有bug

    redis只有linux版,而windows版是微软自己做的,而且也停止更新,大概因此有bug也是正常吧

    解决方法是redis(服务端)使用3.0.504,此版本已解决此bug

  • 相关阅读:
    算法--字符串分隔符/字符串匹配算法
    VMware虚拟机磁盘收缩的几种方法
    maven学习--生命周期
    web适配问题
    eclipse按照svn插件
    一台服务器跑两个tomcat
    postgresql数据库
    visio2013破解工具
    多线程学习
    深入理解Java:SimpleDateFormat安全的时间格式化
  • 原文地址:https://www.cnblogs.com/cannel/p/11078096.html
Copyright © 2011-2022 走看看