zoukankan      html  css  js  c++  java
  • Hadoop单机安装

    环境:ubuntu kylin 15.10,hadoop 2.8.1,jdk 8

    1、新建用户cauchy

    sudo adduser cauchy

    2、hosts中添加hadoop映射

    vi /etc/hosts

    注:使用cauchy用户执行ifconfig命令查看ip地址发现没有权限,执行如下命令解决

    vi ~/.bashrc
    
    最后添加export PATH="$PATH:/sbin"
    
    source ~/.bashrc

    3、使用cauchy用户安装SSH

    su - cauchy
    sudo
    apt-get install openssh-server

    报错:没有sudo权限。需要先给cauchy添加sudo权限

    su - root
    
    vi /etc/sudoers

    添加cauchy ALL=(ALL:ALL) ALL

    4、cauchy用户设置ssh免密

    cd ~/.ssh/
    ssh-keygen -t rsa
    一直回车
    cat ./id_rsa.pub >> ./authorized_keys
    ssh hadoop

    此时还是需要输入密码,才能连接

    5、修改authorized_keys以及用户目录cauchy权限

    chmod 600 authorized_keys
    chmod 710 /home/cauhcy
    ssh hadoop

    值得注意的是:

    cauchy目录权限为777,authorized_keys权限为777,免密登陆失败

    cauchy目录权限为777,authorized_keys权限为600,免密登陆失败

    cauchy目录权限为710,authorized_keys权限为600,免密登陆成功

    7、下载jdk,hadoop解压,并设置jdk,hadoop环境变量

    软件目录

    环境变量~/.bashrc

    执行java -version成功,执行hadoop失败:Error: JAVA_HOME is not set and could not be found.

    说明需要export JAVA_HOME。(同时在PATH中增加了$HADOOP_HOME/sbin)

    此时执行hadoop成功。

     8、测试

    在/home/cauchy/software目录下新建一个测试文本word.txt

    Hello, hadoop, I'm comming.
    I love you.

    执行命令

    hadoop jar ~/software/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.1.jar wordcount ~/software/word.txt ~/software/result

    可以看到生成了结果文件夹~/software/result

  • 相关阅读:
    November 13th 2016 Week 47th Sunday The 1st Day
    November 12th 2016 Week 46th Saturday
    November 11th 2016 Week 46th Friday
    November 10th 2016 Week 46th Thursday
    November 9th 2016 Week 46th Wednesday
    November 8th 2016 Week 46th Tuesday
    windows 7文件共享方法
    Win7无线网络共享设置方法
    常量指针和指针常量
    如何查找局域网的外网ip
  • 原文地址:https://www.cnblogs.com/coshaho/p/7376720.html
Copyright © 2011-2022 走看看