zoukankan      html  css  js  c++  java
  • linux系统下单节点hadoop2的配置

    Jdk安装:

    jdk-7u45-linux-x64.gz

    cp jdk-7u45-linux-x64.gz /usr/java/
    cd /usr/java/
    tar -zxvf jdk-7u45-linux-x64.gz

    vi /etc/profile

    export JAVA_HOME=/usr/java/jdk1.7
    export HADOOP_INSTALL=/data/hadoop-2.2.0
    export MAVEN_HOME=/usr/local/apache-maven-3.2.3
    export PATH=/usr/local/mysql-5.5/bin:$PATH:$JAVA_HOME/bin:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin:$MAVEN_HOME/bin

    hadoop安装

    hadoop-2.2.0-64bits.tar.gz

    在data 文件夹下解压 hadoop 压缩包

    cp hadoop-2.2.0-64bits.tar.gz /data/

    tar -zxvf hadoop-2.2.0-64bits.tar.gz

    配置 无验证ssh登陆

    配置ssh
    ssh-keygen -t rsa -P ""
    cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys 

    Hadoop环境配置

    不用单独建hadoop账户

    我是在root权限下配置的

    修改core-site.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <!--
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License. See accompanying LICENSE file.
    -->

    <!-- Put site-specific property overrides in this file. -->

    <configuration>

    <property>
    <name>hadoop.tmp.dir</name>
    <value>/usr/local/hadoop/hadooptmp</value>
    <description>A base for other temporary directories.</description>
    </property>
    <property>
    <name>fs.defaultFS</name>
    <value>hdfs://10.232.82.221:9000</value>
    <description>The name of the default file system. A URI whose
    scheme and authority determine the FileSystem implementation. The
    uri's scheme determines the config property (fs.SCHEME.impl) naming
    the FileSystem implementation class. The uri's authority is used to
    determine the host, port, etc. for a filesystem.</description>
    </property>

    修改hdfs-site.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <!--
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License. See accompanying LICENSE file.
    -->

    <!-- Put site-specific property overrides in this file. -->

    <configuration>

    <property>
    <name>dfs.replication</name>
    <value>1</value>
    <description>Default block replication.
    The actual number of replications can be specified when the file is created.
    The default is used if replication is not specified in create time.
    </description>
    </property>

    </configuration>

    修改mapred-site.xml

    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <!--
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License. See accompanying LICENSE file.
    -->

    <!-- Put site-specific property overrides in this file. -->

    <configuration>

    <property>
    <name>mapred.job.tracker</name>
    <value>10.232.82.221:9001</value>
    <description>The host and port that the MapReduce job tracker runs
    at. If "local", then jobs are run in-process as a single map
    and reduce task.
    </description>
    </property>


    </configuration>

    1.在没有数据的情况下,先格式化再启动

    格式化命令:hadoop namenode -format

    2.我的文件是在sbin下:

    启动命令:sbin/./start-all.Sh

    3.jps命令查看进程

    4.bin下执行hadoop fs -ls / 

    5.ip:50070 和 ip:50030 在浏览器中监测进程

    有问题可以一起交流留言

  • 相关阅读:
    H5调用Android播放视频
    JavaScript调Java
    Java调用JavaScript
    python的下载和安装
    s5_day1作业
    s5_day2作业
    pycharm激活(转)
    for…else和while…else
    小练习
    09 grep、正则表达式和sed
  • 原文地址:https://www.cnblogs.com/zigewb/p/4060107.html
Copyright © 2011-2022 走看看