zoukankan      html  css  js  c++  java
  • Redis大批量插入数据导致报io.lettuce.core.RedisCommandExecutionException: MISCONF Redis is configured to save RDB snapshots错误

    某功能会频繁地大批量地往redis写入数据,数据量大概10内超过10000条

    插入时可能会报错,报错时间点不定,大概都是运行一段时间后报错,错误信息如下:

    nested exception is io.lettuce.core.RedisCommandExecutionException: MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.

    同时在redis的log找到如下报错:

    === REDIS BUG REPORT START: Cut & paste starting from here ===
    Redis version: 3.2.100
    [18808] 15 Mar 11:12:49.805 # --- EXCEPTION_ACCESS_VIOLATION
    [18808] 15 Mar 11:12:49.805 # --- STACK TRACE
    redis-server.exe!LogStackTrace(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:95)(0x0014E250, 0x0014FF90, 0x00000000, 0x4013A7F8)
    redis-server.exe!UnhandledExceptiontHandler(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x00000000, 0x00000000, 0x00000000, 0x38D470E0)
    KERNELBASE.dll!UnhandledExceptionFilter(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x401555B4, 0x38E3C15C, 0x00000000, 0x00000000)
    ntdll.dll!memset(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x0014EEB0, 0x00000000, 0x40140E48, 0x0014EEB0)
    ntdll.dll!_C_specific_handler(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x00000000, 0x0014E3C0, 0x0014E9C0, 0x00000000)
    ntdll.dll!_chkstk(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x0014E3C0, 0x0014E9C0, 0x3623AB10, 0x00000000)
    ntdll.dll!RtlWalkFrameChain(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x00000000, 0x511B7D55, 0x0014F220, 0x6506FC19)
    ntdll.dll!KiUserExceptionDispatcher(c:
    elease
    edissrcwin32_interopwin32_stacktrace.cpp:185)(0x40094F01, 0x0014F1F0, 0x400BFA57, 0x0014F2E0)
    redis-server.exe!dictSdsHash(c:
    elease
    edissrcserver.c:496)(0x0014F1F0, 0x400BFA57, 0x0014F2E0, 0x00000001)
    redis-server.exe!dictFind(c:
    elease
    edissrcdict.c:517)(0x0014F220, 0x00000001, 0x0000002B, 0x00000001)
    redis-server.exe!getExpire(c:
    elease
    edissrcdb.c:871)(0x00000001, 0x0014F2E0, 0x0014F2E0, 0x03C07040)
    redis-server.exe!rdbSaveRio(c:
    elease
    edissrc
    db.c:814)(0x40167210, 0x00DD0000, 0x00000005, 0x011D2754)
    redis-server.exe!rdbSave(c:
    elease
    edissrc
    db.c:884)(0x00DD0000, 0x00DD0000, 0x5C8B2352, 0x00000005)
    redis-server.exe!QForkChildInit(c:
    elease
    edissrcwin32_interopwin32_qfork.cpp:337)(0x00000005, 0x00000000, 0x004BEF40, 0x00000005)
    redis-server.exe!QForkStartup(c:
    elease
    edissrcwin32_interopwin32_qfork.cpp:515)(0x00000006, 0x00000000, 0x00000000, 0x004A5550)
    redis-server.exe!main(c:
    elease
    edissrcwin32_interopwin32_qfork.cpp:1240)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    redis-server.exe!__tmainCRTStartup(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    KERNEL32.DLL!BaseThreadInitThunk(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    ntdll.dll!RtlUserThreadStart(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    ntdll.dll!RtlUserThreadStart(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
    [18808] 15 Mar 11:12:50.398 # --- INFO OUTPUT
    [5660] 15 Mar 11:12:55.080 # fork operation failed
    [5660] 15 Mar 11:12:55.226 # Background saving terminated by signal 1
    [5660] 15 Mar 11:12:55.353 * 10000 changes in 60 seconds. Saving...

    网上能找到很多解决方法(具体执行baidu),一种主流的方法是redis配置stop-writes-on-bgsave-error设为false,但这种方法只是让出错不停止,错误还是存在的,所以不想用这种方法

    另外原因大概也清楚:就是redis写入数据会先写到内存,然后每隔一段时间会把内存数据写入rdb(一个文件数据库),以实现不会因为关机内存数据丢失就没了数据。问题是在写入rdb时出错

    但不知道写入rdb出错原因在哪,网上很多说是内存不够,我看了内存占用也很低,只用了200+m

    然后在redis配置文件发现这个配置,大意是当内存有多少条待写入数据时就每隔n秒写入一次rdb,这个配置用于按待写入数据量来指定写入rdb的频率,原版配置是这样的

    按这个配置,如果待写入10000条以上才1分钟写入一次,但我的情况是10秒就可能超过10000条,这样可能造成待写入数据hold主太多,且写入时数据量也很大,于是改成以下配置,也就是有10000条就10秒写入一次

    用新配置后再没出错了

    注意这种解决方法并不万能,实际使用还是要根据实际情况配置,此方法只具有指导意义

    磁盘满了也会包这个错

    生产遇到的问题,解决方法转自https://www.liangzl.com/get-article-detail-114890.html

  • 相关阅读:
    使用adb命令报错:解决办法
    appium+python+unittest自动化测试
    HTML自动化测试报告
    彻底解决appium 自动化测试时总是自动安装appium android input manager for unicode的问题
    selenium+python获取文本内容
    jenkins+robotframework中的Rebots Results不显示报告内容的问题
    robotframework+appium使用时的思考
    selenium unittest框架的断言知识
    jenkins配置问题三----用例运行pass,但是测试结果显示failure
    小程序中target与currentTarget的取值问题
  • 原文地址:https://www.cnblogs.com/luizw/p/12929514.html
Copyright © 2011-2022 走看看