zoukankan      html  css  js  c++  java
  • cosbench read异常解决办法。 Unable to verify integrity of data download. Client calculated content hash didn't match hash calculated by Amazon S3. The data may be corrupt.

    问题:cosbench read测试failed 报错如下

    Cosbench v0.4.2.c4 against Ceph (Hammer) / radosgw / HAproxy's HTTP end point. All seems fine except that at the end of the read test, it failed the task and gave following errors

    com.amazonaws.AmazonClientException: Unable to verify integrity of data download.  Client calculated content hash didn't match hash calculated by Amazon S3.  The data may be corrupt.
            at com.amazonaws.services.s3.internal.DigestValidationInputStream.validateMD5Digest(DigestValidationInputStream.java:79)
            at com.amazonaws.services.s3.internal.DigestValidationInputStream.read(DigestValidationInputStream.java:61)
            at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:72)
            at com.amazonaws.services.s3.model.S3ObjectInputStream.read(S3ObjectInputStream.java:155)
            at com.amazonaws.services.s3.model.S3ObjectInputStream.read(S3ObjectInputStream.java:147)
            at com.intel.cosbench.driver.operator.Reader.copyLarge(Reader.java:120)
            at com.intel.cosbench.driver.operator.Reader.doRead(Reader.java:92)
            at com.intel.cosbench.driver.operator.Reader.operate(Reader.java:69)
            at com.intel.cosbench.driver.operator.AbstractOperator.operate(AbstractOperator.java:76)
            at com.intel.cosbench.driver.agent.WorkAgent.performOperation(WorkAgent.java:197)
            at com.intel.cosbench.driver.agent.WorkAgent.doWork(WorkAgent.java:177)
            at com.intel.cosbench.driver.agent.WorkAgent.execute(WorkAgent.java:134)
            at com.intel.cosbench.driver.agent.AbstractAgent.call(AbstractAgent.java:44)
            at com.intel.cosbench.driver.agent.AbstractAgent.call(AbstractAgent.java:1)
            at java.util.concurrent.FutureTask.run(FutureTask.java:262)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
            at java.lang.Thread.run(Thread.java:745)
    

      

    https://github.com/awslabs/aws-sdk-android-samples/issues/62

    解决方案:
    1. stop-all.sh 停止 cosbench 进程(controller, drivers)
    2. 编辑 cosbench-start.sh 文件,找到java启动命令行参数, 增加"-Dcom.amazonaws.services.s3.disableGetObjectMD5Validation=true" 关闭S3的MD5校验功能
    3. start-all.sh 启动所有的cosbench processes.
     
  • 相关阅读:
    Python格式符号
    Python基础知识
    HDFS 数据流程
    HDFS IO流操作
    HDFS API操作
    Hadoop 客户端环境准备
    Hadop 环境搭建 windows10+hadoop2.7.7
    Hadoop HDFS shell
    Hadoop HDFS 基础
    centos 更改清华源
  • 原文地址:https://www.cnblogs.com/bodhitree/p/6140469.html
Copyright © 2011-2022 走看看