zoukankan      html  css  js  c++  java
  • 使用datax mongodb导数据到postgresql数据库遇到访问权限问题

    当我们使用datax工具将mongodb导数据到postgresql数据库的时候,报了这么一个错误

    经DataX智能分析,该任务最可能的错误原因是:
    com.alibaba.datax.common.exception.DataXException: Code:[Framework-02], Description:[DataX引擎运行过程出错,具体原因请参看DataX运行结束时的错误诊断信息  .].  - com.mongodb.MongoCommandException: Command failed with error 13: 'command count requires authentication' on server 10.1.119.34:27117. The full response is { "ok" : 0.0, "errmsg" : "command count requires authentication", "code" : 13, "codeName" : "Unauthorized" }
            at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
            at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114)
            at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:159)
            at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:286)
            at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:173)
            at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:215)
            at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:206)
            at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:112)
            at com.mongodb.operation.CountOperation$1.call(CountOperation.java:210)
            at com.mongodb.operation.CountOperation$1.call(CountOperation.java:206)
            at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:230)
            at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:203)
            at com.mongodb.operation.CountOperation.execute(CountOperation.java:206)
            at com.mongodb.operation.CountOperation.execute(CountOperation.java:53)
            at com.mongodb.Mongo.execute(Mongo.java:772)
            at com.mongodb.Mongo$2.execute(Mongo.java:759)
            at com.mongodb.MongoCollectionImpl.count(MongoCollectionImpl.java:185)
            at com.mongodb.MongoCollectionImpl.count(MongoCollectionImpl.java:165)
            at com.alibaba.datax.plugin.reader.mongodbreader.util.CollectionSplitUtil.doSplitInterval(CollectionSplitUtil.java:55)
            at com.alibaba.datax.plugin.reader.mongodbreader.util.CollectionSplitUtil.doSplit(CollectionSplitUtil.java:37)
            at com.alibaba.datax.plugin.reader.mongodbreader.MongoDBReader$Job.split(MongoDBReader.java:37)
            at com.alibaba.datax.core.job.JobContainer.doReaderSplit(JobContainer.java:732)
            at com.alibaba.datax.core.job.JobContainer.split(JobContainer.java:393)
            at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:117)
            at com.alibaba.datax.core.Engine.start(Engine.java:92)
            at com.alibaba.datax.core.Engine.entry(Engine.java:171)
            at com.alibaba.datax.core.Engine.main(Engine.java:204)
     - com.mongodb.MongoCommandException: Command failed with error 13: 'command count requires authentication' on server 10.1.119.34:27117. The full response is { "ok" : 0.0, "errmsg" : "command count requires authentication", "code" : 13, "codeName" : "Unauthorized" }
            at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
            at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114)
            at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:159)
            at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:286)
            at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:173)
            at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:215)
            at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:206)
            at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:112)
            at com.mongodb.operation.CountOperation$1.call(CountOperation.java:210)
            at com.mongodb.operation.CountOperation$1.call(CountOperation.java:206)
            at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:230)
            at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:203)
            at com.mongodb.operation.CountOperation.execute(CountOperation.java:206)
            at com.mongodb.operation.CountOperation.execute(CountOperation.java:53)
            at com.mongodb.Mongo.execute(Mongo.java:772)
            at com.mongodb.Mongo$2.execute(Mongo.java:759)
            at com.mongodb.MongoCollectionImpl.count(MongoCollectionImpl.java:185)
            at com.mongodb.MongoCollectionImpl.count(MongoCollectionImpl.java:165)
            at com.alibaba.datax.plugin.reader.mongodbreader.util.CollectionSplitUtil.doSplitInterval(CollectionSplitUtil.java:55)
            at com.alibaba.datax.plugin.reader.mongodbreader.util.CollectionSplitUtil.doSplit(CollectionSplitUtil.java:37)
            at com.alibaba.datax.plugin.reader.mongodbreader.MongoDBReader$Job.split(MongoDBReader.java:37)
            at com.alibaba.datax.core.job.JobContainer.doReaderSplit(JobContainer.java:732)
            at com.alibaba.datax.core.job.JobContainer.split(JobContainer.java:393)
            at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:117)
            at com.alibaba.datax.core.Engine.start(Engine.java:92)
            at com.alibaba.datax.core.Engine.entry(Engine.java:171)
            at com.alibaba.datax.core.Engine.main(Engine.java:204)
    
            at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:40)
            at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:162)
            at com.alibaba.datax.core.Engine.start(Engine.java:92)
            at com.alibaba.datax.core.Engine.entry(Engine.java:171)
            at com.alibaba.datax.core.Engine.main(Engine.java:204)
    Caused by: com.mongodb.MongoCommandException: Command failed with error 13: 'command count requires authentication' on server 10.1.119.34:27117. The full response is { "ok" : 0.0, "errmsg" : "command count requires authentication", "code" : 13, "codeName" : "Unauthorized" }
            at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
            at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114)
            at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:159)
            at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:286)
            at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:173)
            at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:215)
            at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:206)
            at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:112)
            at com.mongodb.operation.CountOperation$1.call(CountOperation.java:210)
            at com.mongodb.operation.CountOperation$1.call(CountOperation.java:206)
            at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:230)
            at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:203)
            at com.mongodb.operation.CountOperation.execute(CountOperation.java:206)
            at com.mongodb.operation.CountOperation.execute(CountOperation.java:53)
            at com.mongodb.Mongo.execute(Mongo.java:772)
            at com.mongodb.Mongo$2.execute(Mongo.java:759)
            at com.mongodb.MongoCollectionImpl.count(MongoCollectionImpl.java:185)
            at com.mongodb.MongoCollectionImpl.count(MongoCollectionImpl.java:165)
            at com.alibaba.datax.plugin.reader.mongodbreader.util.CollectionSplitUtil.doSplitInterval(CollectionSplitUtil.java:55)
            at com.alibaba.datax.plugin.reader.mongodbreader.util.CollectionSplitUtil.doSplit(CollectionSplitUtil.java:37)
            at com.alibaba.datax.plugin.reader.mongodbreader.MongoDBReader$Job.split(MongoDBReader.java:37)
            at com.alibaba.datax.core.job.JobContainer.doReaderSplit(JobContainer.java:732)
            at com.alibaba.datax.core.job.JobContainer.split(JobContainer.java:393)
            at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:117)

    我的json文件是这样的

    {
      "job": {
        "content": [
          {
            "reader": {
              "name": "mongodbreader",
              "parameter": { 
                 "address": ["10.12.117.34:27117"],
                 "dbName": "datax",
                 "collectionName": "mycol",
                 "column": [
                                {
                                    "name":"name",
                                    "type":"string"
                                },
                                {   "name":"age",
                                    "type":"string"
                                },
                                {   "name":"sex",
                                    "type":"string"
                                }
                            ],            
                "userPassword": "222019",          
                "userName": "root"
              }
            },
            "writer": {
              "name": "postgresqlwriter",
              "parameter": {
                "print": true,
                "encoding": "UTF-8",
                "column": [             "name",
                    "age",
                    "sex"
                ],
                
               "connection":[
                {
                "jdbcUrl":"jdbc:postgresql://10.1.118.22:30432/postgres",
                "table":["public.mycol"]
                }
               ],
                "username": "postgres",
                "password": "test@2019"
              }
            }
          }
        ],
        "setting": {
          "speed": {
            "channel": 1
          }
        }
      }
    }

    分析流程下来也么有什么问题,但从报错的信息来看,是在mongodb的数据库缺少了读写权限造成的

    我们运行以上命令行

      db.createUser(
      {
        user: "root",
        pwd:  "222019",   
        roles: [ { role: "readWrite", db: "datax" } ]           
      }
    )

    再运行,就成功了

  • 相关阅读:
    编译原理 实例
    lex yacc flex bison
    图解tensorflow 源码分析
    PostgreSQL 179个场景 案例集锦
    github view source
    Java 微服务实践
    Linux kernel AIO
    Lex与Yacc学习
    OpenResty 通过 Lua 扩展 NGINX 实现的可伸缩的 Web 平台
    nginx Architecture
  • 原文地址:https://www.cnblogs.com/braveym/p/13842091.html
Copyright © 2011-2022 走看看