zoukankan      html  css  js  c++  java
  • DataX 离线跨网场景的实施配置

    配置仅限于跨不同网情况,网络互通情况方案和配置会更简单一点

    内网A:MySql数据转换成Csv

    {
      "job": {
          "setting": {
              "speed": {
                   "channel":1
              }
          },
          "content": [
              {
                  "reader": {
                      "name": "mysqlreader",
                      "parameter": {
                          "username": "root",
                          "password": "",
                          "connection": [
                              {
                                  "querySql": [
                                      "select id,name from table;"
                                  ],
                                  "jdbcUrl": [
                                      "jdbc:mysql://127.0.0.1/database"
                                  ]
                              }
                          ]
                      }
                  },
                  "writer": {
                      "name": "txtfilewriter",
                      "parameter": {
                          "print": true,
                          "encoding": "UTF-8",
                          "path": "/result",
                          "fileName": "result.csv",
                          "writeMode": "truncate",
                          "dateFormat": "YYYY-MM-dd hh:mm:ss",
                          "header":["id", "name"]
                      }
                  }
              }
          ]
      }
    }
    

      

    内网B:CSV导入到内网B配置:

    {
        "job": {
            "setting": {
                "speed": {
                    "channel": 2
                }
            },
            "content": [
                {
                     "reader": {
                        "name": "txtfilereader",
                        "parameter": {
                            "path":["D:/result/result.csv"],
                            "encoding": "UTF-8",
                            "column": [
                                {
                                    "index": 0,
                                    "name":"id",
                                    "type": "long"
                                },
                                {
                                    "index": 1,
                                    "name":"name",
                                    "type": "string"
                                }
                            ],
                            "skipHeader": "true"
                        }
                    },
                    "writer": {
                        "name": "mysqlwriter",
                        "parameter": {
                            "writeMode": "insert",
                            "username": "root",
                            "password": "",
                            "column": [
                                "id", "name"
                            ],
                            "preSql": [
                                "truncate table table;"
                            ],
                            "connection": [
                                {
                                    "jdbcUrl": "jdbc:mysql://127.0.0.1:3306/table?characterEncoding=utf-8",
                                    "table": [
                                        "table"
                                    ]
                                }
                            ]
                        }
                    }
                }
            ]
        }
    }
    

      

     注意编码和字段支持的类型,详细可查看DataX官方文档。

  • 相关阅读:
    浅谈css中的盒模型(框模型)
    Linux 字符设备驱动及一些简单的Linux知识
    java实现person类 override(重写) comparable接口
    java之内部类
    mac下sublime text3 安装px转rem插件
    python pip安装扩展报错
    curl http_code 状态码
    sed 批量替换文件
    centos 7 开机优化shell
    一个tcp连接可以发多少http请求
  • 原文地址:https://www.cnblogs.com/leeolevis/p/DataX-Offline-Sync.html
Copyright © 2011-2022 走看看