Flume安装配置

安装包

wget http://mirrors.cnnic.cn/apache/flume/1.7.0/apache-flume-1.7.0-bin.tar.gz
tar -zxvf apache-flume-1.7.0-bin.tar.gz -C /opt/bigdata
cd /opt/bigdata
mv apache-flume-1.7.0-bin/ flume

修改配置文件

环境变量

vim /etc/profile
    export FLUME_HOME=/opt/bigdata/flume
    export PATH=$PATH:$FLUME_HOME/bin
source /etc/profile

flume环境配置

cd $FLUME_HOME/conf
cp flume-env.sh.template flume-env.sh
vim flume-env.sh
    export JAVA_HOME=/opt/soft/jdk1.8.0_144

    # set JMX_PORT
    current=`date "+%Y-%m-%d %H:%M:%S"`  
    timeStamp=`date -d "$current" +%s`   
    currentTimeStamp=$((timeStamp*1000+`date "+%N"`/1000000)) 

    JMX_PORT="41"${currentTimeStamp:11:13}
    export JAVA_OPTS="-Xms100m -Xmx200m -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=$JMX_PORT -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false"

查看flume

正常状态的输出

flume-ng version                                         
    Flume 1.7.0
    Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
    Revision: 511d868555dd4d16e6ce4fedc72c2d1454546707
    Compiled by bessbd on Wed Oct 12 20:51:10 CEST 2016
    From source with checksum 0d21b3ffdc55a07e1d08875872c00523

其他状态的输出

flume-ng version                  
    Error: Could not find or load main class org.apache.flume.tools.GetJavaProperty
    Flume 1.7.0
    Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
    Revision: 511d868555dd4d16e6ce4fedc72c2d1454546707
    Compiled by bessbd on Wed Oct 12 20:51:10 CEST 2016
    From source with checksum 0d21b3ffdc55a07e1d08875872c00523

出现以上error的原因：机器上安装了hbase

解决方法：修改$HBASE_HOME/conf/hbase-env.sh

1. 注释掉 HBASE_CLASSPATH

# export HBASE_CLASSPATH=/opt/bigdata/hbase/conf 

2. 将 HBASE_CLASSPATH 修改为 JAVA_CLASSPATH

   # export HBASE_CLASSPATH=/opt/bigdata/hbase/conf 
   export JAVA_CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar