zoukankan      html  css  js  c++  java
  • 配置hadoop集群,完全分布式模式

     [/soft/hadoop/etc/hadoop] 

    [hdfs-site.xml]

    <configuration>
    	<property>
    		<name>dfs.replication</name>
    		<value>3</value>
    	</property>
    	
    	<property>
                    <name>dfs.hosts</name>    // 这个是白名单 ,  可以添加节点
                    <value>/soft/hadoop/etc/dfs.hosts.txt</value>
            </property>
    	
    	<property>
    		<name>dfs.hosts.exclude</name>  // 黑名单,  删除节点。 
    		<value>/soft/hadoop/etc/dfs.hosts.exclude.txt</value>
    	</property>
    </configuration>
    

      

     

    [mapred-site.xml]

    [ljs@s11 /soft/hadoop/etc/hadoop]$cat mapred-site.xml
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    
    
    <configuration>
    	<property>
    		<name>mapreduce.framework.name</name>
    		<value>yarn</value>
    	</property>
    </configuration>
    

     hdfs dfsadmin refreshNodes

    [yarn.xml]

    [ljs@s11 /soft/hadoop/etc/hadoop]$cat yarn-site.xml 
    <?xml version="1.0"?>
    <configuration>
    	<property>
    		<name>yarn.resourcemanager.hostname</name>
    		<value>s11</value>
    	</property>
    	<property>
    		<name>yarn.nodemanager.aux-services</name>
    		<value>mapreduce_shuffle</value>
    	</property>
    
    </configuration>
    

     

    [core-site.xml] 

    [ljs@s11 /soft/hadoop/etc/hadoop]$cat hadoop-sicat core-site.xml 
    cat: hadoop-sicat: No such file or directory
    <?xml version="1.0" encoding="UTF-8"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    
    <configuration>
    	<property>
    		<name>fs.defaultFS</name>
    		<value>hdfs://s11/</value>
    	</property>
    	<property>
    		<name>hadoop.tmp.dir</name>
    		<value>/home/ljs/hadoop</value>
    	</property>
    </configuration>
    

      

    [slaves]

    [ljs@s11 /soft/hadoop/etc/hadoop]$cat slaves 
    s12
    s13
    s14
    s15
    

      

     

  • 相关阅读:
    hive+mysql安装
    (转)hive sql 学习笔记(1)
    「CSPS 2020」动物园
    「CSPS 2019」Emiya 家今天的饭
    「CSPS 2020」儒略日
    「CSPS 2019」划分
    poj2251 Dungeon Master 搜索BFS
    poj1080 Human Gene Functions
    poj2349最小生成树prim算法
    hoj1356 Miller_Rabbin算法
  • 原文地址:https://www.cnblogs.com/lijins/p/10032356.html
Copyright © 2011-2022 走看看