zoukankan      html  css  js  c++  java
  • ElasticSearch 安装 go-mysql-elasticsearch 同步mysql的数据

    一、首先在Centos6.5上安装 go 语言环境

      下载Golang语言包:https://studygolang.com/dl

    1 [hoojjack@localhost src]$ ls
    2 apache-maven-3.5.0  go  go1.6.2.linux-amd64.tar.gz

    二、解压.tar.gz安装包。

    三、配置安装环境。

    [hoojjack@localhost etc]$ vim profile
    export GOPATH=/usr/local/src/go/work
    export GOROOT=/usr/local/src/go
    export PATH=$GOPATH/bin:$PATH

    GOARCH(指定系统环境,i386表示x86,amd64表示x64):amd64

    GOROOT:/usr/local/src/go(go的解压路径)

    GOBIN:%GOROOT%/bin(exe执行文件路径)

    GOOS:(go运行的系统)

    GOPATH:/usr/local/src/go/work(用于存放Go语言Package的目录,这个目录不能在Go的安装目录中,放了好像也没什么事,至少我目前没事 ^_^)

    我们需要添加的环境变量分别是go的安装路径GOROOT:/usr/local/src/go,PATH:$GOROOT/bin:$PATH,go的工作路径GOPATH:/usr/local/src/go/work

    四、配置go环境后配置go-mysql-elasticsearch

    首先确保环境中安装了git

    安装依赖包:

    sudo yum install gettext-devel openssl-devel perl-CPAN perl-devel zlib-devel

    也可以直接用yum命令安装:

       yum install git

    然后:

    获取安装包:go get github.com/siddontang/go-mysql-elasticsearch

    安装:

             cd   $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch 
             make

    1 [hoojjack@localhost etc]$ cd /usr/local/src/go/src/github.com/siddontang/go-mysql-elasticsearch
    2 [hoojjack@localhost go-mysql-elasticsearch]$ ls
    3 bin  cmd  elastic  etc  Godeps  LICENSE  Makefile  README.md  river  var
    4 [hoojjack@localhost go-mysql-elasticsearch]$ 

    五、修改配置文件

     1 # MySQL address, user and password
     2 # user must have replication privilege in MySQL.
     3 my_addr = "127.0.0.1:3306"     //需要同步的mysql基本设置
     4 my_user = "root"
     5 my_pass = "root"
     6 
     7 # Elasticsearch address
     8 es_addr = "127.0.0.1:9200"     //本地elasticsearch配置
     9 
    10 # Path to store data, like master.info, and dump MySQL data 
    11 data_dir = "./var"             //数据存储的url
    12  //以下配置保存默认不变
    13 # Inner Http status address
    14 stat_addr = "127.0.0.1:12800"
    15 
    16 # pseudo server id like a slave 
    17 server_id = 1001
    18 
    19 # mysql or mariadb
    20 flavor = "mysql"
    21 # mysqldump execution path
    22 mysqldump = "mysqldump"
    23 
    24 # MySQL data source
    25 [[source]]
    26 schema = "hoojjack"             //elasticsearch 与 mysql 同步时对应的数据库名称
    27 
    28 # Only below tables will be synced into Elasticsearch.
    29 # "test_river_[0-9]{4}" is a wildcard table format, you can use it if you have many sub tables, like table_0000 - table_1023
    30 # I don't think it is necessary to sync all tables in a database.
    31 tables = ["test_river", "test_river_[0-9]{4}"]    //支持通配符,可以指定只复制hoojjack数据库中指定的表数据
    32 
    33 # Below is for special rule mapping
    34 [[rule]]
    35 schema = "hoojjack"    //数据库名称
    36 table = "test_river"   //表名称
    37 index = "river"        //对应的索引名称
    38 # title is MySQL test_river field name, es_title is the customized name in Elasticsearch
    39   [rule.field]
    40 # This will map column title to elastic search my_title
    41   title="es_title"     //将可以将mysql的某个属性对应指向elasticsearch的某个field, 如test_river的titile属性对应es_title
    42 # This will map column tags to elastic search my_tags and use array type
    43   tags="my_tags,list"
    44 # This will map column keywords to elastic search keywords and use array type
    45   keywords=",list"
    46 
    47 # wildcard table rule, the wildcard table must be in source tables 
    48   [[rule]]
    49   schema = "hoojjack"
    50   table = "test_river_[0-9]{4}"
    51   index = "river"
    52   type = "river"
    53 
    54 # title is MySQL test_river field name, es_title is the customized name in Elasticsearch
    55   [[rule.fields]]
    56   mysql = "title"
    57   elastic = "es_title"

    六、执行go-mysql-elasticsearch同步,首先进入 go-mysql-elasticsearch 路径,然后执行 : ./bin/go-mysql-elasticsearch -config=./etc/river.toml

    1 cd $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch 
    2 ./bin/go-mysql-elasticsearch -config=./etc/river.toml

    效果如下:

    mysql> select * from test_river;
    +----+-----------+-----------------+--------+
    | id | name      | title           | number |
    +----+-----------+-----------------+--------+
    |  1 | hoojjack  | wo shi yi bing  |      1 |
    |  2 | xunchuan  | ni shi er ren   |      2 |

    七、出现的问题:

     1、mysqldump: command not found

    mysql的mysqldump命令没有找到,解决办法:创建软连接,查看mysqldump是在哪个路径下:find / -name mysqldump   会获得路径,然后创建软连接:ln -s /usr/local/mysql/bin/mysqldump  /usr/bin 

    这是由于系统默认会查找/usr/bin下的命令,如果这个命令不在这个目录下,当然会找不到命令

    如果没有安装mysql,则可以利用yum安装mysql5.6

    1)检查系统是否安装其他版本的MYSQL数据

    #yum list installed | grep mysql
    #yum -y remove mysql-libs.x86_64

    2)安装及配置

    # wget http://repo.mysql.com/mysql-community-release-el6-5.noarch.rpm
    # rpm -ivh mysql-community-release-el6-5.noarch.rpm
    # yum repolist all | grep mysql

    3)安装MYSQL数据库

    yum install mysql-community-server -y

    2、make build go-mysql-elasticsearch项目的时候出现问题(can’t load package: package .: no buildable Go source files in )

     网上有很多的解决方法,但是我都没看懂,唯一看懂的大概就是要升级下go版本,因为之前go是1.6的,索性尝试升级到1.10,后面问题解决了(人品大爆棚),但是问题的核心没有解决。啊啊。。

    【Reference】

    [1] http://blog.csdn.net/laoyang360/article/details/51771483

  • 相关阅读:
    运营平台——效率型后台管理类产品交互设计心得
    代码在线编译器(下)- 用户代码安全检测
    代码在线编译器(上)- 编辑及编译
    SimpleDateFormat并发隐患及其解决
    Spring-SpringMVC父子容器&AOP使用总结
    PaaS服务之路漫谈(三)
    PaaS服务之路漫谈(二)
    PaaS服务之路漫谈(一)
    Spring 属性配置
    如何通过临时表来加快数据库查询的性能
  • 原文地址:https://www.cnblogs.com/hoojjack/p/7608606.html
Copyright © 2011-2022 走看看