zoukankan      html  css  js  c++  java
  • hadoop完全分布式安装部署-笔记

    规划:

    [hadoop@db01 ~]$ cat /etc/hosts
    127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
    ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

    192.168.100.231         db01        #(namenode,datanode,JobHistoryServer,nodemanager)
    192.168.100.232         db02        #(resourcemanager,datanode,nodemanager)
    192.168.100.233         db03        #(SecondaryNameNode,datanode,nodemanager)
    192.168.100.234         db04        #(nodemanager,datanode)
    192.168.100.235         db05        #(nodemanager,datanode)


    配置文件:

    [hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/core-site.xml
    <?xml version="1.0" encoding="UTF-8"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <!--
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License. See accompanying LICENSE file.
    -->

    <!-- Put site-specific property overrides in this file. -->

    <configuration>
        <property>
                <name>fs.defaultFS</name>
                <value>hdfs://db01:9000</value>
        </property>

            <property>
                    <name>hadoop.tmp.dir</name>
                    <value>/usr/local/hadoop-2.5.0/data/tmp</value>
            </property>

            <property>
                    <name>fs.trash.interval</name>
                    <value>7000</value>
            </property>
    </configuration>

    [hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/hdfs-site.xml
    <?xml version="1.0" encoding="UTF-8"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <!--
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License. See accompanying LICENSE file.
    -->

    <!-- Put site-specific property overrides in this file. -->

    <configuration>

            <property>
                    <name>dfs.namenode.secondary.http-address</name>
                    <value>db03:50090</value>
            </property>

    </configuration>

    [hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/yarn-site.xml
    <?xml version="1.0"?>
    <!--
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License. See accompanying LICENSE file.
    -->
    <configuration>

        <property>
                <name>yarn.nodemanager.aux-services</name>
                    <value>mapreduce_shuffle</value>
        </property>

            <property>
                    <name>yarn.resourcemanager.hostname</name>
                    <value>db02</value>
            </property>

        <property>
                    <name>yarn.log-aggregation-enable</name>
                    <value>true</value>
            </property>

            <property>
                    <name>yarn.log-aggregation.retain-seconds</name>
                    <value>600000</value>
            </property>
    </configuration>

    [hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/mapred-site.xml
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <!--
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License. See accompanying LICENSE file.
    -->

    <!-- Put site-specific property overrides in this file. -->

    <configuration>

        <property>
                <name>mapreduce.framework.name</name>
                    <value>yarn</value>
        </property>

            <property>
                    <name>mapreduce.jobhistory.address</name>
                    <value>db01:10020</value>
            </property>

            <property>
                    <name>mapreduce.jobhistory.webapp.address</name>
                    <value>db01:19888</value>
            </property>

    </configuration>

    [hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/slaves
    db01
    db02
    db03
    db04
    db05

    并且在(etc/hadoop/hadoop-env.sh  etc/hadoop/mapred-env.sh  etc/hadoop/yarn-env.sh)文件中配置相应的JAVA_HOME信息。

  • 相关阅读:
    一些Asp.Net面试题答案
    未能加载文件或程序集"Microsoft.Web.Infrastructure 的解决方案
    偶然的发现(与Code无关)
    配置一台测试机 每个域用户独立会话
    使用 Git 和 Visual Studio Online 进行版本控制
    1.大数据概述
    递归下降语法分析程序设计
    文法 LL1
    C语言的文法分析
    词法分析
  • 原文地址:https://www.cnblogs.com/wcwen1990/p/6652073.html
Copyright © 2011-2022 走看看