zoukankan      html  css  js  c++  java
  • hadoop完全分布式安装部署-笔记

    规划:

    [hadoop@db01 ~]$ cat /etc/hosts
    127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
    ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

    192.168.100.231         db01        #(namenode,datanode,JobHistoryServer,nodemanager)
    192.168.100.232         db02        #(resourcemanager,datanode,nodemanager)
    192.168.100.233         db03        #(SecondaryNameNode,datanode,nodemanager)
    192.168.100.234         db04        #(nodemanager,datanode)
    192.168.100.235         db05        #(nodemanager,datanode)


    配置文件:

    [hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/core-site.xml
    <?xml version="1.0" encoding="UTF-8"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <!--
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License. See accompanying LICENSE file.
    -->

    <!-- Put site-specific property overrides in this file. -->

    <configuration>
        <property>
                <name>fs.defaultFS</name>
                <value>hdfs://db01:9000</value>
        </property>

            <property>
                    <name>hadoop.tmp.dir</name>
                    <value>/usr/local/hadoop-2.5.0/data/tmp</value>
            </property>

            <property>
                    <name>fs.trash.interval</name>
                    <value>7000</value>
            </property>
    </configuration>

    [hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/hdfs-site.xml
    <?xml version="1.0" encoding="UTF-8"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <!--
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License. See accompanying LICENSE file.
    -->

    <!-- Put site-specific property overrides in this file. -->

    <configuration>

            <property>
                    <name>dfs.namenode.secondary.http-address</name>
                    <value>db03:50090</value>
            </property>

    </configuration>

    [hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/yarn-site.xml
    <?xml version="1.0"?>
    <!--
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License. See accompanying LICENSE file.
    -->
    <configuration>

        <property>
                <name>yarn.nodemanager.aux-services</name>
                    <value>mapreduce_shuffle</value>
        </property>

            <property>
                    <name>yarn.resourcemanager.hostname</name>
                    <value>db02</value>
            </property>

        <property>
                    <name>yarn.log-aggregation-enable</name>
                    <value>true</value>
            </property>

            <property>
                    <name>yarn.log-aggregation.retain-seconds</name>
                    <value>600000</value>
            </property>
    </configuration>

    [hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/mapred-site.xml
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <!--
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License. See accompanying LICENSE file.
    -->

    <!-- Put site-specific property overrides in this file. -->

    <configuration>

        <property>
                <name>mapreduce.framework.name</name>
                    <value>yarn</value>
        </property>

            <property>
                    <name>mapreduce.jobhistory.address</name>
                    <value>db01:10020</value>
            </property>

            <property>
                    <name>mapreduce.jobhistory.webapp.address</name>
                    <value>db01:19888</value>
            </property>

    </configuration>

    [hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/slaves
    db01
    db02
    db03
    db04
    db05

    并且在(etc/hadoop/hadoop-env.sh  etc/hadoop/mapred-env.sh  etc/hadoop/yarn-env.sh)文件中配置相应的JAVA_HOME信息。

  • 相关阅读:
    windows I/O系统
    MYSQL复习笔记7-索引
    LOG收集系统(一):原日志至收集
    MYSQL复习笔记6-字符集
    MYSQL复习笔记5-select-from-where子句
    MYSQL复习笔记4-基本SQL语句
    MYSQL复习笔记3-用户和安全
    MYSQL复习笔记2-自带工具介绍
    MYSQL复习笔记1-物理文件和系统架构
    Mysql复习
  • 原文地址:https://www.cnblogs.com/wcwen1990/p/6652073.html
Copyright © 2011-2022 走看看