zoukankan      html  css  js  c++  java
  • 如何在Windows下面运行hadoop的MapReduce程序

     在Windows下面运行hadoop的MapReduce程序的方法:

    1.下载hadoop的安装包,这里使用的是"hadoop-2.6.4.tar.gz":

    2.将安装包直接解压到D盘根目录:

    3.配置环境变量:

     

    4.下载hadoop的eclipse插件,并将插件放到eclipse的plugins目录下:

     5.打开Eclipse,选择菜单"Window"-->"Preferences",在左侧找到"Hadoop Map/Reduce",

      在右侧选择hadoop的目录:

     6.打开菜单"Window"中的"Show View"窗口,选择"Map/Reduce Locations":

    7:在打开的"Map/Reduce Locations"面板中,点击小象图标,打开新建配置窗口:

    8.按照下图所示,填写hadoop集群的主机地址和端口:

    9.新创建的hadoop集群连接配置,右上角的齿轮可以修改配置信息:

    10.打开菜单"Window"中的"Show View"窗口,找到"Project Explorer":

    11.在"Project Explorer"面板中找到"DFS Locations",展开下面的菜单就可以连接上HDFS,

        可以直接看到HDFS中的目录和文件:

    12.在"Project Explorer"面板中点击鼠标右键,选择新建,就可以创建"Map/Reduce"项目了:

    13.下面我们创建了一个名为"hadoop-test"的项目,可以看到它自动帮我们导入了很多的jar包:

    14.在项目的src下面创建log4j.properties文件,内容如下:

    log4j.rootLogger=debug,stdout,R
    log4j.appender.stdout=org.apache.log4j.ConsoleAppender
    log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
    log4j.appender.stdout.layout.ConversionPattern=%5p - %m%n
    log4j.appender.R=org.apache.log4j.RollingFileAppender
    log4j.appender.R.File=mapreduce_test.log
    log4j.appender.R.MaxFileSize=1MB
    log4j.appender.R.MaxBackupIndex=1
    log4j.appender.R.layout=org.apache.log4j.PatternLayout
    log4j.appender.R.layout.ConversionPattern=%p %t %c - %m%n
    log4j.logger.com.codefutures=DEBUG

    15.在src下面创建org.apache.hadoop.io.nativeio包,并将hadoop源码中的NativeIO.java类拷贝到该包下(这一步是从网上看的,不知道有什么用)。

     NativeIO.java类所在路径为:hadoop-2.6.4-srchadoop-common-projecthadoop-commonsrcmainjavaorgapachehadoopio ativeio

    16.修改NativeIO.java类中的如下部分,返回"true"即可:

    17.下面来写一个hadoop的MapReduce程序,经典的wordcount(单词计数)程序,代码如下(共3个java类):

    (1)WordCountDriver.java类:

     1 package com.xuebusi.hadoop.mr.windows;
     2 
     3 import org.apache.hadoop.conf.Configuration;
     4 import org.apache.hadoop.fs.Path;
     5 import org.apache.hadoop.io.IntWritable;
     6 import org.apache.hadoop.io.Text;
     7 import org.apache.hadoop.mapreduce.Job;
     8 import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
     9 import org.apache.hadoop.mapreduce.lib.input.TextInputFormat;
    10 import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
    11 
    12 public class WordCountDriver {
    13 
    14     public static void main(String[] args) throws Exception {
    15         
    16         Configuration conf = new Configuration();
    17         Job job = Job.getInstance(conf);
    18         
    19         job.setJarByClass(WordCountDriver.class);
    20         
    21         job.setMapperClass(WordCountMapper.class);
    22         job.setReducerClass(WordCountReducer.class);
    23         
    24         job.setMapOutputKeyClass(Text.class);
    25         job.setMapOutputValueClass(IntWritable.class);
    26         
    27         job.setOutputKeyClass(Text.class);
    28         job.setOutputValueClass(IntWritable.class);
    29         
    30         job.setInputFormatClass(TextInputFormat.class);
    31         
    32         FileInputFormat.setInputPaths(job, new Path("c:/wordcount/input"));
    33         FileOutputFormat.setOutputPath(job, new Path("c:/wordcount/output"));
    34         
    35         boolean res = job.waitForCompletion(true);
    36         System.exit(res ? 0 : 1);
    37         
    38     }
    39 
    40 }
    View Code

    (2)WordCountMapper.java类:

     1 package com.xuebusi.hadoop.mr.windows;
     2 
     3 import java.io.IOException;
     4 
     5 import org.apache.hadoop.io.IntWritable;
     6 import org.apache.hadoop.io.LongWritable;
     7 import org.apache.hadoop.io.Text;
     8 import org.apache.hadoop.mapreduce.Mapper;
     9 
    10 public class WordCountMapper extends Mapper<LongWritable, Text, Text, IntWritable> {
    11 
    12     @Override
    13     protected void map(LongWritable key, Text value, Context context)
    14             throws IOException, InterruptedException {
    15         // TODO Auto-generated method stub
    16         //super.map(key, value, context);
    17         
    18         String line = value.toString();
    19         String[] words = line.split(" ");
    20         
    21         for (String word : words) {
    22             context.write(new Text(word), new IntWritable(1));
    23         }
    24     }
    25 }
    View Code

    (3)WordCountReducer.java类:

     1 package com.xuebusi.hadoop.mr.windows;
     2 
     3 import java.io.IOException;
     4 
     5 import org.apache.hadoop.io.IntWritable;
     6 import org.apache.hadoop.io.Text;
     7 import org.apache.hadoop.mapreduce.Reducer;
     8 
     9 public class WordCountReducer extends Reducer<Text, IntWritable, Text, IntWritable> {
    10 
    11     @Override
    12     protected void reduce(Text key, Iterable<IntWritable> values, Context context)
    13             throws IOException, InterruptedException {
    14         // TODO Auto-generated method stub
    15         // super.reduce(arg0, arg1, arg2);
    16 
    17         int count = 0;
    18         for (IntWritable value : values) {
    19             count += value.get();
    20         }
    21         
    22         context.write(new Text(key), new IntWritable(count));
    23     }
    24 
    25 }
    View Code

    18.准备点数据,在"c:/wordcount/input"目录下面创建"words.txt"文件,输入一些文本,并用空格隔开:

    18.在Eclipse中,运行WordCountDriver类中的main方法(右键Run As-->Java Application),可以在控制台看到如下日志信息:

       1 DEBUG - Handling deprecation for dfs.image.compression.codec
       2 DEBUG - Handling deprecation for mapreduce.job.reduces
       3 DEBUG - Handling deprecation for mapreduce.job.complete.cancel.delegation.tokens
       4 DEBUG - Handling deprecation for mapreduce.jobhistory.recovery.store.class
       5 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.search.filter.user
       6 DEBUG - Handling deprecation for dfs.namenode.enable.retrycache
       7 DEBUG - Handling deprecation for yarn.nodemanager.sleep-delay-before-sigkill.ms
       8 DEBUG - Handling deprecation for mapreduce.jobhistory.joblist.cache.size
       9 DEBUG - Handling deprecation for mapreduce.tasktracker.healthchecker.interval
      10 DEBUG - Handling deprecation for mapreduce.jobtracker.heartbeats.in.second
      11 DEBUG - Handling deprecation for mapreduce.jobtracker.persist.jobstatus.dir
      12 DEBUG - Handling deprecation for dfs.namenode.backup.http-address
      13 DEBUG - Handling deprecation for hadoop.rpc.protection
      14 DEBUG - Handling deprecation for dfs.client.mmap.enabled
      15 DEBUG - Handling deprecation for yarn.app.mapreduce.am.container.log.backups
      16 DEBUG - Handling deprecation for ftp.stream-buffer-size
      17 DEBUG - Handling deprecation for dfs.namenode.https-address
      18 DEBUG - Handling deprecation for yarn.timeline-service.address
      19 DEBUG - Handling deprecation for dfs.ha.log-roll.period
      20 DEBUG - Handling deprecation for yarn.nodemanager.recovery.enabled
      21 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.numinputfiles
      22 DEBUG - Handling deprecation for hadoop.security.groups.negative-cache.secs
      23 DEBUG - Handling deprecation for yarn.resourcemanager.admin.client.thread-count
      24 DEBUG - Handling deprecation for dfs.datanode.fsdatasetcache.max.threads.per.volume
      25 DEBUG - Handling deprecation for file.client-write-packet-size
      26 DEBUG - Handling deprecation for hadoop.http.authentication.simple.anonymous.allowed
      27 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.path
      28 DEBUG - Handling deprecation for yarn.resourcemanager.proxy-user-privileges.enabled
      29 DEBUG - Handling deprecation for dfs.datanode.drop.cache.behind.reads
      30 DEBUG - Handling deprecation for yarn.nodemanager.log.retain-seconds
      31 DEBUG - Handling deprecation for dfs.image.transfer.bandwidthPerSec
      32 DEBUG - Handling deprecation for yarn.resourcemanager.work-preserving-recovery.scheduling-wait-ms
      33 DEBUG - Handling deprecation for hadoop.registry.zk.session.timeout.ms
      34 DEBUG - Handling deprecation for dfs.datanode.slow.io.warning.threshold.ms
      35 DEBUG - Handling deprecation for mapreduce.tasktracker.instrumentation
      36 DEBUG - Handling deprecation for ha.failover-controller.cli-check.rpc-timeout.ms
      37 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.cgroups.hierarchy
      38 DEBUG - Handling deprecation for dfs.namenode.write.stale.datanode.ratio
      39 DEBUG - Handling deprecation for hadoop.security.groups.cache.warn.after.ms
      40 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.fetch.retry.timeout-ms
      41 DEBUG - Handling deprecation for mapreduce.jobhistory.client.thread-count
      42 DEBUG - Handling deprecation for io.mapfile.bloom.size
      43 DEBUG - Handling deprecation for yarn.nodemanager.docker-container-executor.exec-name
      44 DEBUG - Handling deprecation for yarn.resourcemanager.work-preserving-recovery.enabled
      45 DEBUG - Handling deprecation for dfs.ha.fencing.ssh.connect-timeout
      46 DEBUG - Handling deprecation for yarn.resourcemanager.zk-num-retries
      47 DEBUG - Handling deprecation for hadoop.registry.zk.root
      48 DEBUG - Handling deprecation for s3.bytes-per-checksum
      49 DEBUG - Handling deprecation for yarn.app.mapreduce.am.container.log.limit.kb
      50 DEBUG - Handling deprecation for dfs.namenode.edit.log.autoroll.check.interval.ms
      51 DEBUG - Handling deprecation for fs.automatic.close
      52 DEBUG - Handling deprecation for fs.trash.interval
      53 DEBUG - Handling deprecation for dfs.journalnode.https-address
      54 DEBUG - Handling deprecation for yarn.timeline-service.ttl-ms
      55 DEBUG - Handling deprecation for hadoop.security.authentication
      56 DEBUG - Handling deprecation for fs.defaultFS
      57 DEBUG - Handling deprecation for nfs.rtmax
      58 DEBUG - Handling deprecation for hadoop.ssl.server.conf
      59 DEBUG - Handling deprecation for ipc.client.connect.max.retries
      60 DEBUG - Handling deprecation for yarn.resourcemanager.delayed.delegation-token.removal-interval-ms
      61 DEBUG - Handling deprecation for dfs.journalnode.http-address
      62 DEBUG - Handling deprecation for dfs.namenode.xattrs.enabled
      63 DEBUG - Handling deprecation for dfs.datanode.shared.file.descriptor.paths
      64 DEBUG - Handling deprecation for mapreduce.jobtracker.taskscheduler
      65 DEBUG - Handling deprecation for mapreduce.job.speculative.speculativecap
      66 DEBUG - Handling deprecation for yarn.timeline-service.store-class
      67 DEBUG - Handling deprecation for yarn.am.liveness-monitor.expiry-interval-ms
      68 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.compress
      69 DEBUG - Handling deprecation for dfs.user.home.dir.prefix
      70 DEBUG - Handling deprecation for net.topology.node.switch.mapping.impl
      71 DEBUG - Handling deprecation for dfs.namenode.replication.considerLoad
      72 DEBUG - Handling deprecation for dfs.namenode.fs-limits.min-block-size
      73 DEBUG - Handling deprecation for fs.swift.impl
      74 DEBUG - Handling deprecation for dfs.namenode.audit.loggers
      75 DEBUG - Handling deprecation for mapreduce.job.max.split.locations
      76 DEBUG - Handling deprecation for yarn.resourcemanager.address
      77 DEBUG - Handling deprecation for mapreduce.job.counters.max
      78 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.fetch.retry.enabled
      79 DEBUG - Handling deprecation for dfs.client.block.write.retries
      80 DEBUG - Handling deprecation for dfs.short.circuit.shared.memory.watcher.interrupt.check.ms
      81 DEBUG - Handling deprecation for dfs.namenode.resource.checked.volumes.minimum
      82 DEBUG - Handling deprecation for io.map.index.interval
      83 DEBUG - Handling deprecation for mapred.child.java.opts
      84 DEBUG - Handling deprecation for mapreduce.tasktracker.local.dir.minspacestart
      85 DEBUG - Handling deprecation for mapreduce.client.progressmonitor.pollinterval
      86 DEBUG - Handling deprecation for dfs.client.https.keystore.resource
      87 DEBUG - Handling deprecation for mapreduce.task.profile.map.params
      88 DEBUG - Handling deprecation for mapreduce.jobtracker.tasktracker.maxblacklists
      89 DEBUG - Handling deprecation for mapreduce.job.queuename
      90 DEBUG - Handling deprecation for hadoop.registry.zk.quorum
      91 DEBUG - Handling deprecation for yarn.app.mapreduce.client-am.ipc.max-retries-on-timeouts
      92 DEBUG - Handling deprecation for yarn.nodemanager.localizer.address
      93 DEBUG - Handling deprecation for io.mapfile.bloom.error.rate
      94 DEBUG - Handling deprecation for yarn.nodemanager.delete.thread-count
      95 DEBUG - Handling deprecation for mapreduce.job.split.metainfo.maxsize
      96 DEBUG - Handling deprecation for yarn.scheduler.maximum-allocation-vcores
      97 DEBUG - Handling deprecation for mapred.mapper.new-api
      98 DEBUG - Handling deprecation for mapreduce.job.dir
      99 DEBUG - Handling deprecation for yarn.app.mapreduce.am.resource.mb
     100 DEBUG - Handling deprecation for dfs.datanode.dns.nameserver
     101 DEBUG - Handling deprecation for dfs.client.slow.io.warning.threshold.ms
     102 DEBUG - Handling deprecation for mapreduce.job.reducer.preempt.delay.sec
     103 DEBUG - Handling deprecation for yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb
     104 DEBUG - Handling deprecation for mapreduce.map.output.compress.codec
     105 DEBUG - Handling deprecation for dfs.namenode.accesstime.precision
     106 DEBUG - Handling deprecation for fs.s3a.connection.maximum
     107 DEBUG - Handling deprecation for mapreduce.map.log.level
     108 DEBUG - Handling deprecation for io.seqfile.compress.blocksize
     109 DEBUG - Handling deprecation for ipc.client.ping
     110 DEBUG - Handling deprecation for mapreduce.tasktracker.taskcontroller
     111 DEBUG - Handling deprecation for hadoop.security.groups.cache.secs
     112 DEBUG - Handling deprecation for dfs.datanode.cache.revocation.timeout.ms
     113 DEBUG - Handling deprecation for dfs.client.context
     114 DEBUG - Handling deprecation for mapreduce.input.lineinputformat.linespermap
     115 DEBUG - Handling deprecation for mapreduce.job.end-notification.max.attempts
     116 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user
     117 DEBUG - Handling deprecation for yarn.nodemanager.webapp.address
     118 DEBUG - Handling deprecation for mapreduce.job.submithostname
     119 DEBUG - Handling deprecation for mapreduce.jobhistory.recovery.enable
     120 DEBUG - Handling deprecation for mapreduce.jobtracker.expire.trackers.interval
     121 DEBUG - Handling deprecation for yarn.resourcemanager.webapp.address
     122 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.num.refill.threads
     123 DEBUG - Handling deprecation for yarn.nodemanager.health-checker.interval-ms
     124 DEBUG - Handling deprecation for mapreduce.jobhistory.loadedjobs.cache.size
     125 DEBUG - Handling deprecation for hadoop.security.authorization
     126 DEBUG - Handling deprecation for mapreduce.job.map.output.collector.class
     127 DEBUG - Handling deprecation for mapreduce.am.max-attempts
     128 DEBUG - Handling deprecation for fs.ftp.host
     129 DEBUG - Handling deprecation for fs.s3a.attempts.maximum
     130 DEBUG - Handling deprecation for yarn.app.mapreduce.am.scheduler.heartbeat.interval-ms
     131 DEBUG - Handling deprecation for mapreduce.ifile.readahead
     132 DEBUG - Handling deprecation for yarn.resourcemanager.scheduler.monitor.enable
     133 DEBUG - Handling deprecation for yarn.resourcemanager.zk-retry-interval-ms
     134 DEBUG - Handling deprecation for ha.zookeeper.session-timeout.ms
     135 DEBUG - Handling deprecation for mapreduce.tasktracker.taskmemorymanager.monitoringinterval
     136 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.parallelcopies
     137 DEBUG - Handling deprecation for dfs.client.mmap.retry.timeout.ms
     138 DEBUG - Handling deprecation for mapreduce.map.skip.maxrecords
     139 DEBUG - Handling deprecation for mapreduce.job.output.value.class
     140 DEBUG - Handling deprecation for dfs.namenode.avoid.read.stale.datanode
     141 DEBUG - Handling deprecation for dfs.https.enable
     142 DEBUG - Handling deprecation for yarn.timeline-service.webapp.https.address
     143 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.read.timeout
     144 DEBUG - Handling deprecation for dfs.namenode.list.encryption.zones.num.responses
     145 DEBUG - Handling deprecation for yarn.nodemanager.remote-app-log-dir-suffix
     146 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.compress.codec
     147 DEBUG - Handling deprecation for mapreduce.jobtracker.instrumentation
     148 DEBUG - Handling deprecation for dfs.blockreport.intervalMsec
     149 DEBUG - Handling deprecation for ipc.client.connect.retry.interval
     150 DEBUG - Handling deprecation for mapreduce.reduce.speculative
     151 DEBUG - Handling deprecation for mapreduce.jobhistory.keytab
     152 DEBUG - Handling deprecation for mapreduce.jobhistory.datestring.cache.size
     153 DEBUG - Handling deprecation for dfs.datanode.balance.bandwidthPerSec
     154 DEBUG - Handling deprecation for file.blocksize
     155 DEBUG - Handling deprecation for yarn.resourcemanager.admin.address
     156 DEBUG - Handling deprecation for mapreduce.map.cpu.vcores
     157 DEBUG - Handling deprecation for yarn.nodemanager.container-monitor.procfs-tree.smaps-based-rss.enabled
     158 DEBUG - Handling deprecation for yarn.resourcemanager.configuration.provider-class
     159 DEBUG - Handling deprecation for yarn.resourcemanager.resource-tracker.address
     160 DEBUG - Handling deprecation for mapreduce.tasktracker.local.dir.minspacekill
     161 DEBUG - Handling deprecation for mapreduce.jobtracker.staging.root.dir
     162 DEBUG - Handling deprecation for mapreduce.jobtracker.retiredjobs.cache.size
     163 DEBUG - Handling deprecation for hadoop.registry.rm.enabled
     164 DEBUG - Handling deprecation for ipc.client.connect.max.retries.on.timeouts
     165 DEBUG - Handling deprecation for ha.zookeeper.acl
     166 DEBUG - Handling deprecation for hadoop.security.crypto.codec.classes.aes.ctr.nopadding
     167 DEBUG - Handling deprecation for yarn.nodemanager.local-dirs
     168 DEBUG - Handling deprecation for mapreduce.app-submission.cross-platform
     169 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.connect.timeout
     170 DEBUG - Handling deprecation for dfs.block.access.key.update.interval
     171 DEBUG - Handling deprecation for rpc.metrics.quantile.enable
     172 DEBUG - Handling deprecation for dfs.block.access.token.lifetime
     173 DEBUG - Handling deprecation for mapreduce.job.end-notification.retry.attempts
     174 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-xattrs-per-inode
     175 DEBUG - Handling deprecation for mapreduce.jobtracker.system.dir
     176 DEBUG - Handling deprecation for dfs.client.file-block-storage-locations.timeout.millis
     177 DEBUG - Handling deprecation for yarn.nodemanager.admin-env
     178 DEBUG - Handling deprecation for mapreduce.jobtracker.jobhistory.block.size
     179 DEBUG - Handling deprecation for yarn.log-aggregation.retain-seconds
     180 DEBUG - Handling deprecation for mapreduce.tasktracker.indexcache.mb
     181 DEBUG - Handling deprecation for yarn.timeline-service.handler-thread-count
     182 DEBUG - Handling deprecation for dfs.namenode.checkpoint.check.period
     183 DEBUG - Handling deprecation for yarn.resourcemanager.hostname
     184 DEBUG - Handling deprecation for dfs.client.block.write.replace-datanode-on-failure.enable
     185 DEBUG - Handling deprecation for net.topology.impl
     186 DEBUG - Handling deprecation for dfs.datanode.directoryscan.interval
     187 DEBUG - Handling deprecation for fs.s3a.multipart.purge.age
     188 DEBUG - Handling deprecation for hadoop.security.java.secure.random.algorithm
     189 DEBUG - Handling deprecation for yarn.nodemanager.container-monitor.interval-ms
     190 DEBUG - Handling deprecation for dfs.default.chunk.view.size
     191 DEBUG - Handling deprecation for fs.s3a.multipart.threshold
     192 DEBUG - Handling deprecation for mapreduce.job.speculative.slownodethreshold
     193 DEBUG - Handling deprecation for mapreduce.job.reduce.slowstart.completedmaps
     194 DEBUG - Handling deprecation for mapred.reducer.new-api
     195 DEBUG - Handling deprecation for hadoop.security.instrumentation.requires.admin
     196 DEBUG - Handling deprecation for io.compression.codec.bzip2.library
     197 DEBUG - Handling deprecation for hadoop.http.authentication.signature.secret.file
     198 DEBUG - Handling deprecation for hadoop.registry.secure
     199 DEBUG - Handling deprecation for dfs.namenode.safemode.min.datanodes
     200 DEBUG - Handling deprecation for yarn.nodemanager.localizer.cache.target-size-mb
     201 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.inputdir
     202 DEBUG - Handling deprecation for mapreduce.reduce.maxattempts
     203 DEBUG - Handling deprecation for dfs.datanode.https.address
     204 DEBUG - Handling deprecation for s3native.replication
     205 DEBUG - Handling deprecation for dfs.namenode.inotify.max.events.per.rpc
     206 DEBUG - Handling deprecation for dfs.namenode.path.based.cache.retry.interval.ms
     207 DEBUG - Handling deprecation for mapreduce.reduce.skip.proc.count.autoincr
     208 DEBUG - Handling deprecation for dfs.datanode.cache.revocation.polling.ms
     209 DEBUG - Handling deprecation for mapreduce.jobhistory.cleaner.interval-ms
     210 DEBUG - Handling deprecation for file.replication
     211 DEBUG - Handling deprecation for hadoop.hdfs.configuration.version
     212 DEBUG - Handling deprecation for ipc.client.idlethreshold
     213 DEBUG - Handling deprecation for hadoop.tmp.dir
     214 DEBUG - Handling deprecation for yarn.resourcemanager.store.class
     215 DEBUG - Handling deprecation for mapreduce.jobhistory.address
     216 DEBUG - Handling deprecation for mapreduce.jobtracker.restart.recover
     217 DEBUG - Handling deprecation for mapreduce.cluster.local.dir
     218 DEBUG - Handling deprecation for yarn.client.nodemanager-client-async.thread-pool-max-size
     219 DEBUG - Handling deprecation for dfs.namenode.decommission.nodes.per.interval
     220 DEBUG - Handling deprecation for mapreduce.job.inputformat.class
     221 DEBUG - Handling deprecation for yarn.nodemanager.resource.cpu-vcores
     222 DEBUG - Handling deprecation for dfs.namenode.reject-unresolved-dn-topology-mapping
     223 DEBUG - Handling deprecation for dfs.namenode.delegation.key.update-interval
     224 DEBUG - Handling deprecation for fs.s3.buffer.dir
     225 DEBUG - Handling deprecation for dfs.client.read.shortcircuit.streams.cache.expiry.ms
     226 DEBUG - Handling deprecation for dfs.namenode.support.allow.format
     227 DEBUG - Handling deprecation for yarn.nodemanager.remote-app-log-dir
     228 DEBUG - Handling deprecation for mapreduce.map.memory.mb
     229 DEBUG - Handling deprecation for dfs.namenode.edit.log.autoroll.multiplier.threshold
     230 DEBUG - Handling deprecation for hadoop.work.around.non.threadsafe.getpwuid
     231 DEBUG - Handling deprecation for mapreduce.task.profile.reduce.params
     232 DEBUG - Handling deprecation for yarn.timeline-service.client.max-retries
     233 DEBUG - Handling deprecation for dfs.ha.automatic-failover.enabled
     234 DEBUG - Handling deprecation for dfs.namenode.edits.noeditlogchannelflush
     235 DEBUG - Handling deprecation for dfs.namenode.stale.datanode.interval
     236 DEBUG - Handling deprecation for mapreduce.shuffle.transfer.buffer.size
     237 DEBUG - Handling deprecation for mapreduce.jobtracker.persist.jobstatus.active
     238 DEBUG - Handling deprecation for dfs.namenode.logging.level
     239 DEBUG - Handling deprecation for yarn.resourcemanager.am-rm-tokens.master-key-rolling-interval-secs
     240 DEBUG - Handling deprecation for yarn.nodemanager.log-dirs
     241 DEBUG - Handling deprecation for ha.health-monitor.sleep-after-disconnect.ms
     242 DEBUG - Handling deprecation for yarn.resourcemanager.fs.state-store.uri
     243 DEBUG - Handling deprecation for dfs.namenode.checkpoint.edits.dir
     244 DEBUG - Handling deprecation for yarn.resourcemanager.keytab
     245 DEBUG - Handling deprecation for hadoop.rpc.socket.factory.class.default
     246 DEBUG - Handling deprecation for yarn.resourcemanager.fail-fast
     247 DEBUG - Handling deprecation for dfs.datanode.http.address
     248 DEBUG - Handling deprecation for mapreduce.task.profile
     249 DEBUG - Handling deprecation for mapreduce.jobhistory.move.interval-ms
     250 DEBUG - Handling deprecation for dfs.namenode.edits.dir
     251 DEBUG - Handling deprecation for dfs.storage.policy.enabled
     252 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.size
     253 DEBUG - Handling deprecation for hadoop.fuse.timer.period
     254 DEBUG - Handling deprecation for mapreduce.jobhistory.http.policy
     255 DEBUG - Handling deprecation for mapreduce.jobhistory.intermediate-done-dir
     256 DEBUG - Handling deprecation for mapreduce.map.skip.proc.count.autoincr
     257 DEBUG - Handling deprecation for fs.AbstractFileSystem.viewfs.impl
     258 DEBUG - Handling deprecation for mapreduce.job.speculative.slowtaskthreshold
     259 DEBUG - Handling deprecation for yarn.resourcemanager.webapp.delegation-token-auth-filter.enabled
     260 DEBUG - Handling deprecation for s3native.stream-buffer-size
     261 DEBUG - Handling deprecation for yarn.nodemanager.delete.debug-delay-sec
     262 DEBUG - Handling deprecation for dfs.secondary.namenode.kerberos.internal.spnego.principal
     263 DEBUG - Handling deprecation for dfs.datanode.available-space-volume-choosing-policy.balanced-space-threshold
     264 DEBUG - Handling deprecation for fs.s3n.multipart.uploads.block.size
     265 DEBUG - Handling deprecation for dfs.namenode.safemode.threshold-pct
     266 DEBUG - Handling deprecation for mapreduce.ifile.readahead.bytes
     267 DEBUG - Handling deprecation for yarn.scheduler.maximum-allocation-mb
     268 DEBUG - Handling deprecation for ipc.client.fallback-to-simple-auth-allowed
     269 DEBUG - Handling deprecation for fs.har.impl.disable.cache
     270 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.read-cache-size
     271 DEBUG - Handling deprecation for yarn.timeline-service.hostname
     272 DEBUG - Handling deprecation for s3native.bytes-per-checksum
     273 DEBUG - Handling deprecation for mapreduce.job.committer.setup.cleanup.needed
     274 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms
     275 DEBUG - Handling deprecation for fs.s3a.paging.maximum
     276 DEBUG - Handling deprecation for yarn.client.nodemanager-connect.retry-interval-ms
     277 DEBUG - Handling deprecation for yarn.nodemanager.log-aggregation.compression-type
     278 DEBUG - Handling deprecation for yarn.app.mapreduce.am.job.committer.commit-window
     279 DEBUG - Handling deprecation for hadoop.http.authentication.type
     280 DEBUG - Handling deprecation for dfs.client.failover.sleep.base.millis
     281 DEBUG - Handling deprecation for mapreduce.job.submithostaddress
     282 DEBUG - Handling deprecation for yarn.nodemanager.vmem-check-enabled
     283 DEBUG - Handling deprecation for hadoop.jetty.logs.serve.aliases
     284 DEBUG - Handling deprecation for ha.failover-controller.graceful-fence.rpc-timeout.ms
     285 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.input.buffer.percent
     286 DEBUG - Handling deprecation for dfs.datanode.max.transfer.threads
     287 DEBUG - Handling deprecation for mapreduce.reduce.merge.inmem.threshold
     288 DEBUG - Handling deprecation for mapreduce.task.io.sort.mb
     289 DEBUG - Handling deprecation for dfs.namenode.acls.enabled
     290 DEBUG - Handling deprecation for yarn.client.application-client-protocol.poll-interval-ms
     291 DEBUG - Handling deprecation for hadoop.security.kms.client.authentication.retry-count
     292 DEBUG - Handling deprecation for dfs.namenode.handler.count
     293 DEBUG - Handling deprecation for yarn.resourcemanager.connect.max-wait.ms
     294 DEBUG - Handling deprecation for dfs.namenode.retrycache.heap.percent
     295 DEBUG - Handling deprecation for yarn.timeline-service.enabled
     296 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users
     297 DEBUG - Handling deprecation for hadoop.ssl.client.conf
     298 DEBUG - Handling deprecation for yarn.resourcemanager.container.liveness-monitor.interval-ms
     299 DEBUG - Handling deprecation for yarn.nodemanager.vmem-pmem-ratio
     300 DEBUG - Handling deprecation for mapreduce.client.completion.pollinterval
     301 DEBUG - Handling deprecation for yarn.app.mapreduce.client.max-retries
     302 DEBUG - Handling deprecation for hadoop.ssl.enabled
     303 DEBUG - Handling deprecation for fs.client.resolve.remote.symlinks
     304 DEBUG - Handling deprecation for fs.AbstractFileSystem.hdfs.impl
     305 DEBUG - Handling deprecation for mapreduce.tasktracker.reduce.tasks.maximum
     306 DEBUG - Handling deprecation for yarn.nodemanager.hostname
     307 DEBUG - Handling deprecation for mapreduce.reduce.input.buffer.percent
     308 DEBUG - Handling deprecation for fs.s3a.multipart.purge
     309 DEBUG - Handling deprecation for yarn.app.mapreduce.am.command-opts
     310 DEBUG - Handling deprecation for dfs.namenode.invalidate.work.pct.per.iteration
     311 DEBUG - Handling deprecation for dfs.bytes-per-checksum
     312 DEBUG - Handling deprecation for yarn.resourcemanager.webapp.https.address
     313 DEBUG - Handling deprecation for dfs.replication
     314 DEBUG - Handling deprecation for dfs.datanode.block.id.layout.upgrade.threads
     315 DEBUG - Handling deprecation for mapreduce.shuffle.ssl.file.buffer.size
     316 DEBUG - Handling deprecation for dfs.namenode.list.cache.directives.num.responses
     317 DEBUG - Handling deprecation for dfs.permissions.enabled
     318 DEBUG - Handling deprecation for mapreduce.jobtracker.maxtasks.perjob
     319 DEBUG - Handling deprecation for dfs.datanode.use.datanode.hostname
     320 DEBUG - Handling deprecation for mapreduce.task.userlog.limit.kb
     321 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-directory-items
     322 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.low-watermark
     323 DEBUG - Handling deprecation for fs.s3a.buffer.dir
     324 DEBUG - Handling deprecation for s3.client-write-packet-size
     325 DEBUG - Handling deprecation for hadoop.user.group.static.mapping.overrides
     326 DEBUG - Handling deprecation for mapreduce.shuffle.max.threads
     327 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.expiry
     328 DEBUG - Handling deprecation for dfs.client.failover.sleep.max.millis
     329 DEBUG - Handling deprecation for mapreduce.job.maps
     330 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-component-length
     331 DEBUG - Handling deprecation for yarn.fail-fast
     332 DEBUG - Handling deprecation for hadoop.ssl.enabled.protocols
     333 DEBUG - Handling deprecation for s3.blocksize
     334 DEBUG - Handling deprecation for mapreduce.map.output.compress
     335 DEBUG - Handling deprecation for dfs.namenode.edits.journal-plugin.qjournal
     336 DEBUG - Handling deprecation for dfs.namenode.datanode.registration.ip-hostname-check
     337 DEBUG - Handling deprecation for yarn.nodemanager.pmem-check-enabled
     338 DEBUG - Handling deprecation for dfs.client.short.circuit.replica.stale.threshold.ms
     339 DEBUG - Handling deprecation for dfs.client.https.need-auth
     340 DEBUG - Handling deprecation for yarn.scheduler.minimum-allocation-mb
     341 DEBUG - Handling deprecation for mapreduce.jobhistory.max-age-ms
     342 DEBUG - Handling deprecation for yarn.timeline-service.http-authentication.type
     343 DEBUG - Handling deprecation for ftp.replication
     344 DEBUG - Handling deprecation for dfs.namenode.secondary.https-address
     345 DEBUG - Handling deprecation for dfs.blockreport.split.threshold
     346 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.split.minsize
     347 DEBUG - Handling deprecation for fs.s3n.block.size
     348 DEBUG - Handling deprecation for mapreduce.job.token.tracking.ids.enabled
     349 DEBUG - Handling deprecation for yarn.ipc.rpc.class
     350 DEBUG - Handling deprecation for dfs.namenode.num.extra.edits.retained
     351 DEBUG - Handling deprecation for yarn.nodemanager.localizer.cache.cleanup.interval-ms
     352 DEBUG - Handling deprecation for hadoop.http.staticuser.user
     353 DEBUG - Handling deprecation for mapreduce.jobhistory.move.thread-count
     354 DEBUG - Handling deprecation for fs.s3a.multipart.size
     355 DEBUG - Handling deprecation for mapreduce.job.jvm.numtasks
     356 DEBUG - Handling deprecation for mapreduce.task.profile.maps
     357 DEBUG - Handling deprecation for dfs.datanode.max.locked.memory
     358 DEBUG - Handling deprecation for dfs.cachereport.intervalMsec
     359 DEBUG - Handling deprecation for mapreduce.shuffle.port
     360 DEBUG - Handling deprecation for yarn.resourcemanager.nodemanager.minimum.version
     361 DEBUG - Handling deprecation for mapreduce.shuffle.connection-keep-alive.timeout
     362 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.merge.percent
     363 DEBUG - Handling deprecation for mapreduce.jobtracker.http.address
     364 DEBUG - Handling deprecation for mapreduce.task.skip.start.attempts
     365 DEBUG - Handling deprecation for yarn.resourcemanager.connect.retry-interval.ms
     366 DEBUG - Handling deprecation for yarn.scheduler.minimum-allocation-vcores
     367 DEBUG - Handling deprecation for mapreduce.task.io.sort.factor
     368 DEBUG - Handling deprecation for dfs.namenode.checkpoint.dir
     369 DEBUG - Handling deprecation for nfs.exports.allowed.hosts
     370 DEBUG - Handling deprecation for tfile.fs.input.buffer.size
     371 DEBUG - Handling deprecation for fs.s3.block.size
     372 DEBUG - Handling deprecation for tfile.io.chunk.size
     373 DEBUG - Handling deprecation for fs.s3n.multipart.copy.block.size
     374 DEBUG - Handling deprecation for io.serializations
     375 DEBUG - Handling deprecation for yarn.resourcemanager.max-completed-applications
     376 DEBUG - Handling deprecation for mapreduce.jobhistory.principal
     377 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.outputdir
     378 DEBUG - Handling deprecation for yarn.resourcemanager.ha.automatic-failover.zk-base-path
     379 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.fetch.retry.interval-ms
     380 DEBUG - Handling deprecation for mapreduce.job.end-notification.retry.interval
     381 DEBUG - Handling deprecation for dfs.namenode.backup.address
     382 DEBUG - Handling deprecation for fs.s3n.multipart.uploads.enabled
     383 DEBUG - Handling deprecation for io.seqfile.sorter.recordlimit
     384 DEBUG - Handling deprecation for dfs.block.access.token.enable
     385 DEBUG - Handling deprecation for s3native.client-write-packet-size
     386 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-xattr-size
     387 DEBUG - Handling deprecation for ftp.bytes-per-checksum
     388 DEBUG - Handling deprecation for hadoop.security.group.mapping
     389 DEBUG - Handling deprecation for dfs.client.domain.socket.data.traffic
     390 DEBUG - Handling deprecation for dfs.client.read.shortcircuit.streams.cache.size
     391 DEBUG - Handling deprecation for fs.s3a.connection.timeout
     392 DEBUG - Handling deprecation for mapreduce.job.end-notification.max.retry.interval
     393 DEBUG - Handling deprecation for yarn.acl.enable
     394 DEBUG - Handling deprecation for yarn.nm.liveness-monitor.expiry-interval-ms
     395 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.list-status.num-threads
     396 DEBUG - Handling deprecation for dfs.client.mmap.cache.size
     397 DEBUG - Handling deprecation for mapreduce.tasktracker.map.tasks.maximum
     398 DEBUG - Handling deprecation for yarn.timeline-service.ttl-enable
     399 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.resources-handler.class
     400 DEBUG - Handling deprecation for dfs.namenode.max.objects
     401 DEBUG - Handling deprecation for yarn.resourcemanager.state-store.max-completed-applications
     402 DEBUG - Handling deprecation for dfs.namenode.delegation.token.max-lifetime
     403 DEBUG - Handling deprecation for mapreduce.job.classloader
     404 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size
     405 DEBUG - Handling deprecation for mapreduce.job.hdfs-servers
     406 DEBUG - Handling deprecation for dfs.datanode.hdfs-blocks-metadata.enabled
     407 DEBUG - Handling deprecation for mapreduce.tasktracker.dns.nameserver
     408 DEBUG - Handling deprecation for dfs.datanode.readahead.bytes
     409 DEBUG - Handling deprecation for dfs.datanode.bp-ready.timeout
     410 DEBUG - Handling deprecation for mapreduce.job.ubertask.maxreduces
     411 DEBUG - Handling deprecation for dfs.image.compress
     412 DEBUG - Handling deprecation for mapreduce.shuffle.ssl.enabled
     413 DEBUG - Handling deprecation for yarn.log-aggregation-enable
     414 DEBUG - Handling deprecation for mapreduce.tasktracker.report.address
     415 DEBUG - Handling deprecation for mapreduce.tasktracker.http.threads
     416 DEBUG - Handling deprecation for dfs.stream-buffer-size
     417 DEBUG - Handling deprecation for tfile.fs.output.buffer.size
     418 DEBUG - Handling deprecation for fs.permissions.umask-mode
     419 DEBUG - Handling deprecation for dfs.client.datanode-restart.timeout
     420 DEBUG - Handling deprecation for dfs.namenode.resource.du.reserved
     421 DEBUG - Handling deprecation for yarn.resourcemanager.am.max-attempts
     422 DEBUG - Handling deprecation for yarn.nodemanager.resource.percentage-physical-cpu-limit
     423 DEBUG - Handling deprecation for ha.failover-controller.graceful-fence.connection.retries
     424 DEBUG - Handling deprecation for dfs.datanode.drop.cache.behind.writes
     425 DEBUG - Handling deprecation for mapreduce.map.output.value.class
     426 DEBUG - Handling deprecation for hadoop.common.configuration.version
     427 DEBUG - Handling deprecation for mapreduce.job.ubertask.enable
     428 DEBUG - Handling deprecation for yarn.app.mapreduce.am.resource.cpu-vcores
     429 DEBUG - Handling deprecation for dfs.namenode.replication.work.multiplier.per.iteration
     430 DEBUG - Handling deprecation for mapreduce.job.acl-modify-job
     431 DEBUG - Handling deprecation for io.seqfile.local.dir
     432 DEBUG - Handling deprecation for yarn.resourcemanager.system-metrics-publisher.enabled
     433 DEBUG - Handling deprecation for fs.s3.sleepTimeSeconds
     434 DEBUG - Handling deprecation for mapreduce.client.output.filter
     435  INFO - Submitting tokens for job: job_local1553403857_0001
     436 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:331)
     437 DEBUG - Handling deprecation for all properties in config...
     438 DEBUG - Handling deprecation for dfs.datanode.data.dir
     439 DEBUG - Handling deprecation for dfs.namenode.checkpoint.txns
     440 DEBUG - Handling deprecation for s3.replication
     441 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.compress.type
     442 DEBUG - Handling deprecation for mapreduce.jobtracker.jobhistory.lru.cache.size
     443 DEBUG - Handling deprecation for dfs.datanode.failed.volumes.tolerated
     444 DEBUG - Handling deprecation for hadoop.http.filter.initializers
     445 DEBUG - Handling deprecation for mapreduce.cluster.temp.dir
     446 DEBUG - Handling deprecation for yarn.nodemanager.keytab
     447 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.memory.limit.percent
     448 DEBUG - Handling deprecation for dfs.namenode.checkpoint.max-retries
     449 DEBUG - Handling deprecation for nfs.mountd.port
     450 DEBUG - Handling deprecation for hadoop.registry.zk.retry.times
     451 DEBUG - Handling deprecation for yarn.resourcemanager.zk-acl
     452 DEBUG - Handling deprecation for mapreduce.reduce.skip.maxgroups
     453 DEBUG - Handling deprecation for ipc.ping.interval
     454 DEBUG - Handling deprecation for dfs.https.server.keystore.resource
     455 DEBUG - Handling deprecation for yarn.app.mapreduce.task.container.log.backups
     456 DEBUG - Handling deprecation for hadoop.http.authentication.kerberos.keytab
     457 DEBUG - Handling deprecation for yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage
     458 DEBUG - Handling deprecation for dfs.client.block.write.replace-datanode-on-failure.best-effort
     459 DEBUG - Handling deprecation for yarn.nodemanager.localizer.client.thread-count
     460 DEBUG - Handling deprecation for mapreduce.jobhistory.done-dir
     461 DEBUG - Handling deprecation for mapreduce.framework.name
     462 DEBUG - Handling deprecation for ha.failover-controller.new-active.rpc-timeout.ms
     463 DEBUG - Handling deprecation for ha.health-monitor.check-interval.ms
     464 DEBUG - Handling deprecation for io.file.buffer.size
     465 DEBUG - Handling deprecation for mapreduce.shuffle.max.connections
     466 DEBUG - Handling deprecation for dfs.namenode.resource.check.interval
     467 DEBUG - Handling deprecation for dfs.namenode.path.based.cache.block.map.allocation.percent
     468 DEBUG - Handling deprecation for mapreduce.task.tmp.dir
     469 DEBUG - Handling deprecation for dfs.namenode.checkpoint.period
     470 DEBUG - Handling deprecation for dfs.client.mmap.cache.timeout.ms
     471 DEBUG - Handling deprecation for ipc.client.kill.max
     472 DEBUG - Handling deprecation for yarn.resourcemanager.scheduler.class
     473 DEBUG - Handling deprecation for mapreduce.jobtracker.taskcache.levels
     474 DEBUG - Handling deprecation for s3.stream-buffer-size
     475 DEBUG - Handling deprecation for dfs.namenode.secondary.http-address
     476 DEBUG - Handling deprecation for yarn.client.nodemanager-connect.max-wait-ms
     477 DEBUG - Handling deprecation for dfs.namenode.decommission.interval
     478 DEBUG - Handling deprecation for dfs.namenode.http-address
     479 DEBUG - Handling deprecation for mapreduce.task.files.preserve.failedtasks
     480 DEBUG - Handling deprecation for dfs.encrypt.data.transfer
     481 DEBUG - Handling deprecation for yarn.resourcemanager.ha.enabled
     482 DEBUG - Handling deprecation for dfs.datanode.address
     483 DEBUG - Handling deprecation for dfs.namenode.avoid.write.stale.datanode
     484 DEBUG - Handling deprecation for hadoop.http.authentication.token.validity
     485 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.search.filter.group
     486 DEBUG - Handling deprecation for dfs.client.failover.max.attempts
     487 DEBUG - Handling deprecation for yarn.resourcemanager.scheduler.monitor.policies
     488 DEBUG - Handling deprecation for hadoop.registry.zk.connection.timeout.ms
     489 DEBUG - Handling deprecation for hadoop.security.crypto.cipher.suite
     490 DEBUG - Handling deprecation for yarn.timeline-service.http-authentication.simple.anonymous.allowed
     491 DEBUG - Handling deprecation for mapreduce.task.profile.params
     492 DEBUG - Handling deprecation for yarn.resourcemanager.fs.state-store.retry-policy-spec
     493 DEBUG - Handling deprecation for yarn.admin.acl
     494 DEBUG - Handling deprecation for yarn.nodemanager.local-cache.max-files-per-directory
     495 DEBUG - Handling deprecation for yarn.client.failover-retries-on-socket-timeouts
     496 DEBUG - Handling deprecation for dfs.namenode.retrycache.expirytime.millis
     497 DEBUG - Handling deprecation for yarn.resourcemanager.nodemanagers.heartbeat-interval-ms
     498 DEBUG - Handling deprecation for dfs.client.failover.connection.retries.on.timeouts
     499 DEBUG - Handling deprecation for yarn.client.failover-proxy-provider
     500 DEBUG - Handling deprecation for mapreduce.map.sort.spill.percent
     501 DEBUG - Handling deprecation for file.stream-buffer-size
     502 DEBUG - Handling deprecation for dfs.webhdfs.enabled
     503 DEBUG - Handling deprecation for ipc.client.connection.maxidletime
     504 DEBUG - Handling deprecation for mapreduce.task.combine.progress.records
     505 DEBUG - Handling deprecation for mapreduce.job.output.key.class
     506 DEBUG - Handling deprecation for mapreduce.jobtracker.persist.jobstatus.hours
     507 DEBUG - Handling deprecation for dfs.image.transfer.chunksize
     508 DEBUG - Handling deprecation for yarn.nodemanager.address
     509 DEBUG - Handling deprecation for dfs.datanode.ipc.address
     510 DEBUG - Handling deprecation for yarn.resourcemanager.ha.automatic-failover.embedded
     511 DEBUG - Handling deprecation for mapreduce.jobhistory.recovery.store.fs.uri
     512 DEBUG - Handling deprecation for yarn.resourcemanager.zk-state-store.parent-path
     513 DEBUG - Handling deprecation for yarn.app.mapreduce.am.job.task.listener.thread-count
     514 DEBUG - Handling deprecation for nfs.dump.dir
     515 DEBUG - Handling deprecation for dfs.namenode.list.cache.pools.num.responses
     516 DEBUG - Handling deprecation for dfs.client.read.shortcircuit
     517 DEBUG - Handling deprecation for dfs.namenode.safemode.extension
     518 DEBUG - Handling deprecation for ha.zookeeper.parent-znode
     519 DEBUG - Handling deprecation for yarn.nodemanager.container-executor.class
     520 DEBUG - Handling deprecation for io.skip.checksum.errors
     521 DEBUG - Handling deprecation for dfs.namenode.path.based.cache.refresh.interval.ms
     522 DEBUG - Handling deprecation for dfs.encrypt.data.transfer.cipher.key.bitlength
     523 DEBUG - Handling deprecation for mapreduce.job.user.name
     524 DEBUG - Handling deprecation for yarn.resourcemanager.scheduler.client.thread-count
     525 DEBUG - Handling deprecation for yarn.nodemanager.recovery.dir
     526 DEBUG - Handling deprecation for mapreduce.job.emit-timeline-data
     527 DEBUG - Handling deprecation for hadoop.http.authentication.kerberos.principal
     528 DEBUG - Handling deprecation for mapreduce.reduce.log.level
     529 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.nonsecure-mode.user-pattern
     530 DEBUG - Handling deprecation for fs.s3.maxRetries
     531 DEBUG - Handling deprecation for yarn.nodemanager.resourcemanager.minimum.version
     532 DEBUG - Handling deprecation for ipc.client.rpc-timeout.ms
     533 DEBUG - Handling deprecation for hadoop.kerberos.kinit.command
     534 DEBUG - Handling deprecation for yarn.log-aggregation.retain-check-interval-seconds
     535 DEBUG - Handling deprecation for yarn.nodemanager.process-kill-wait.ms
     536 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.cgroups.mount
     537 DEBUG - Handling deprecation for mapreduce.map.output.key.class
     538 DEBUG - Handling deprecation for mapreduce.job.working.dir
     539 DEBUG - Handling deprecation for dfs.namenode.name.dir.restore
     540 DEBUG - Handling deprecation for mapreduce.jobtracker.handler.count
     541 DEBUG - Handling deprecation for mapreduce.jobhistory.admin.address
     542 DEBUG - Handling deprecation for yarn.app.mapreduce.client-am.ipc.max-retries
     543 DEBUG - Handling deprecation for dfs.client.use.datanode.hostname
     544 DEBUG - Handling deprecation for hadoop.util.hash.type
     545 DEBUG - Handling deprecation for dfs.datanode.available-space-volume-choosing-policy.balanced-space-preference-fraction
     546 DEBUG - Handling deprecation for dfs.datanode.dns.interface
     547 DEBUG - Handling deprecation for io.seqfile.lazydecompress
     548 DEBUG - Handling deprecation for dfs.namenode.lazypersist.file.scrub.interval.sec
     549 DEBUG - Handling deprecation for yarn.nodemanager.disk-health-checker.min-healthy-disks
     550 DEBUG - Handling deprecation for yarn.client.max-cached-nodemanagers-proxies
     551 DEBUG - Handling deprecation for mapreduce.job.maxtaskfailures.per.tracker
     552 DEBUG - Handling deprecation for mapreduce.tasktracker.healthchecker.script.timeout
     553 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.search.attr.group.name
     554 DEBUG - Handling deprecation for fs.df.interval
     555 DEBUG - Handling deprecation for hadoop.security.crypto.buffer.size
     556 DEBUG - Handling deprecation for dfs.namenode.kerberos.internal.spnego.principal
     557 DEBUG - Handling deprecation for dfs.client.cached.conn.retry
     558 DEBUG - Handling deprecation for mapreduce.job.reduce.class
     559 DEBUG - Handling deprecation for mapreduce.job.map.class
     560 DEBUG - Handling deprecation for mapreduce.job.reduce.shuffle.consumer.plugin.class
     561 DEBUG - Handling deprecation for mapreduce.jobtracker.address
     562 DEBUG - Handling deprecation for mapreduce.tasktracker.tasks.sleeptimebeforesigkill
     563 DEBUG - Handling deprecation for dfs.journalnode.rpc-address
     564 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-blocks-per-file
     565 DEBUG - Handling deprecation for mapreduce.job.acl-view-job
     566 DEBUG - Handling deprecation for yarn.app.mapreduce.am.job.committer.cancel-timeout
     567 DEBUG - Handling deprecation for dfs.client.block.write.replace-datanode-on-failure.policy
     568 DEBUG - Handling deprecation for mapreduce.shuffle.connection-keep-alive.enable
     569 DEBUG - Handling deprecation for dfs.namenode.replication.interval
     570 DEBUG - Handling deprecation for mapreduce.jobhistory.minicluster.fixed.ports
     571 DEBUG - Handling deprecation for dfs.namenode.num.checkpoints.retained
     572 DEBUG - Handling deprecation for yarn.resourcemanager.scheduler.address
     573 DEBUG - Handling deprecation for mapreduce.tasktracker.http.address
     574 DEBUG - Handling deprecation for mapreduce.jobhistory.admin.acl
     575 DEBUG - Handling deprecation for dfs.datanode.directoryscan.threads
     576 DEBUG - Handling deprecation for mapreduce.reduce.memory.mb
     577 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.ssl
     578 DEBUG - Handling deprecation for dfs.http.policy
     579 DEBUG - Handling deprecation for mapreduce.task.merge.progress.records
     580 DEBUG - Handling deprecation for dfs.heartbeat.interval
     581 DEBUG - Handling deprecation for yarn.resourcemanager.system-metrics-publisher.dispatcher.pool-size
     582 DEBUG - Handling deprecation for yarn.resourcemanager.recovery.enabled
     583 DEBUG - Handling deprecation for net.topology.script.number.args
     584 DEBUG - Handling deprecation for mapreduce.local.clientfactory.class.name
     585 DEBUG - Handling deprecation for dfs.client-write-packet-size
     586 DEBUG - Handling deprecation for yarn.dispatcher.drain-events.timeout
     587 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.directory.search.timeout
     588 DEBUG - Handling deprecation for io.native.lib.available
     589 DEBUG - Handling deprecation for dfs.client.failover.connection.retries
     590 DEBUG - Handling deprecation for yarn.nodemanager.disk-health-checker.interval-ms
     591 DEBUG - Handling deprecation for dfs.blocksize
     592 DEBUG - Handling deprecation for dfs.client.use.legacy.blockreader.local
     593 DEBUG - Handling deprecation for yarn.resourcemanager.container-tokens.master-key-rolling-interval-secs
     594 DEBUG - Handling deprecation for fs.s3a.connection.ssl.enabled
     595 DEBUG - Handling deprecation for mapreduce.jobhistory.webapp.address
     596 DEBUG - Handling deprecation for yarn.resourcemanager.resource-tracker.client.thread-count
     597 DEBUG - Handling deprecation for yarn.client.failover-retries
     598 DEBUG - Handling deprecation for hadoop.registry.jaas.context
     599 DEBUG - Handling deprecation for dfs.blockreport.initialDelay
     600 DEBUG - Handling deprecation for yarn.nodemanager.aux-services.mapreduce_shuffle.class
     601 DEBUG - Handling deprecation for ha.health-monitor.rpc-timeout.ms
     602 DEBUG - Handling deprecation for yarn.resourcemanager.zk-timeout-ms
     603 DEBUG - Handling deprecation for mapreduce.reduce.markreset.buffer.percent
     604 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.start-time-read-cache-size
     605 DEBUG - Handling deprecation for dfs.ha.tail-edits.period
     606 DEBUG - Handling deprecation for yarn.resourcemanager.client.thread-count
     607 DEBUG - Handling deprecation for yarn.nodemanager.health-checker.script.timeout-ms
     608 DEBUG - Handling deprecation for file.bytes-per-checksum
     609 DEBUG - Handling deprecation for dfs.replication.max
     610 DEBUG - Handling deprecation for dfs.namenode.max.extra.edits.segments.retained
     611 DEBUG - Handling deprecation for io.map.index.skip
     612 DEBUG - Handling deprecation for yarn.timeline-service.client.retry-interval-ms
     613 DEBUG - Handling deprecation for mapreduce.task.timeout
     614 DEBUG - Handling deprecation for dfs.datanode.du.reserved
     615 DEBUG - Handling deprecation for mapreduce.reduce.cpu.vcores
     616 DEBUG - Handling deprecation for dfs.support.append
     617 DEBUG - Handling deprecation for dfs.client.file-block-storage-locations.num-threads
     618 DEBUG - Handling deprecation for ftp.blocksize
     619 DEBUG - Handling deprecation for yarn.nodemanager.container-manager.thread-count
     620 DEBUG - Handling deprecation for ipc.server.listen.queue.size
     621 DEBUG - Handling deprecation for yarn.resourcemanager.ha.automatic-failover.enabled
     622 DEBUG - Handling deprecation for hadoop.ssl.hostname.verifier
     623 DEBUG - Handling deprecation for nfs.server.port
     624 DEBUG - Handling deprecation for mapreduce.tasktracker.dns.interface
     625 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.search.attr.member
     626 DEBUG - Handling deprecation for mapreduce.job.userlog.retain.hours
     627 DEBUG - Handling deprecation for mapreduce.tasktracker.outofband.heartbeat
     628 DEBUG - Handling deprecation for fs.s3a.impl
     629 DEBUG - Handling deprecation for hadoop.registry.system.acls
     630 DEBUG - Handling deprecation for yarn.nodemanager.resource.memory-mb
     631 DEBUG - Handling deprecation for yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds
     632 DEBUG - Handling deprecation for dfs.webhdfs.user.provider.user.pattern
     633 DEBUG - Handling deprecation for dfs.namenode.delegation.token.renew-interval
     634 DEBUG - Handling deprecation for hadoop.ssl.keystores.factory.class
     635 DEBUG - Handling deprecation for hadoop.registry.zk.retry.ceiling.ms
     636 DEBUG - Handling deprecation for yarn.http.policy
     637 DEBUG - Handling deprecation for dfs.datanode.sync.behind.writes
     638 DEBUG - Handling deprecation for nfs.wtmax
     639 DEBUG - Handling deprecation for fs.AbstractFileSystem.har.impl
     640 DEBUG - Handling deprecation for dfs.client.read.shortcircuit.skip.checksum
     641 DEBUG - Handling deprecation for hadoop.security.random.device.file.path
     642 DEBUG - Handling deprecation for mapreduce.map.maxattempts
     643 DEBUG - Handling deprecation for yarn.timeline-service.webapp.address
     644 DEBUG - Handling deprecation for dfs.datanode.handler.count
     645 DEBUG - Handling deprecation for hadoop.ssl.require.client.cert
     646 DEBUG - Handling deprecation for ftp.client-write-packet-size
     647 DEBUG - Handling deprecation for dfs.client.write.exclude.nodes.cache.expiry.interval.millis
     648 DEBUG - Handling deprecation for mapreduce.jobhistory.cleaner.enable
     649 DEBUG - Handling deprecation for fs.du.interval
     650 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.retry-delay.max.ms
     651 DEBUG - Handling deprecation for mapreduce.task.profile.reduces
     652 DEBUG - Handling deprecation for ha.health-monitor.connect-retry-interval.ms
     653 DEBUG - Handling deprecation for hadoop.fuse.connection.timeout
     654 DEBUG - Handling deprecation for dfs.permissions.superusergroup
     655 DEBUG - Handling deprecation for mapreduce.jobtracker.jobhistory.task.numberprogresssplits
     656 DEBUG - Handling deprecation for fs.ftp.host.port
     657 DEBUG - Handling deprecation for mapreduce.map.speculative
     658 DEBUG - Handling deprecation for dfs.datanode.data.dir.perm
     659 DEBUG - Handling deprecation for mapreduce.client.submit.file.replication
     660 DEBUG - Handling deprecation for dfs.namenode.startup.delay.block.deletion.sec
     661 DEBUG - Handling deprecation for s3native.blocksize
     662 DEBUG - Handling deprecation for mapreduce.job.ubertask.maxmaps
     663 DEBUG - Handling deprecation for dfs.namenode.replication.min
     664 DEBUG - Handling deprecation for mapreduce.cluster.acls.enabled
     665 DEBUG - Handling deprecation for hadoop.security.uid.cache.secs
     666 DEBUG - Handling deprecation for nfs.allow.insecure.ports
     667 DEBUG - Handling deprecation for yarn.nodemanager.localizer.fetch.thread-count
     668 DEBUG - Handling deprecation for map.sort.class
     669 DEBUG - Handling deprecation for fs.trash.checkpoint.interval
     670 DEBUG - Handling deprecation for mapred.queue.default.acl-administer-jobs
     671 DEBUG - Handling deprecation for yarn.timeline-service.generic-application-history.max-applications
     672 DEBUG - Handling deprecation for dfs.image.transfer.timeout
     673 DEBUG - Handling deprecation for dfs.namenode.name.dir
     674 DEBUG - Handling deprecation for ipc.client.connect.timeout
     675 DEBUG - Handling deprecation for yarn.app.mapreduce.am.staging-dir
     676 DEBUG - Handling deprecation for fs.AbstractFileSystem.file.impl
     677 DEBUG - Handling deprecation for yarn.nodemanager.env-whitelist
     678 DEBUG - Handling deprecation for hadoop.registry.zk.retry.interval.ms
     679 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage
     680 DEBUG - Handling deprecation for yarn.timeline-service.keytab
     681 DEBUG - Handling deprecation for dfs.image.compression.codec
     682 DEBUG - Handling deprecation for mapreduce.job.reduces
     683 DEBUG - Handling deprecation for mapreduce.job.complete.cancel.delegation.tokens
     684 DEBUG - Handling deprecation for mapreduce.jobhistory.recovery.store.class
     685 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.search.filter.user
     686 DEBUG - Handling deprecation for dfs.namenode.enable.retrycache
     687 DEBUG - Handling deprecation for yarn.nodemanager.sleep-delay-before-sigkill.ms
     688 DEBUG - Handling deprecation for mapreduce.jobhistory.joblist.cache.size
     689 DEBUG - Handling deprecation for mapreduce.tasktracker.healthchecker.interval
     690 DEBUG - Handling deprecation for mapreduce.jobtracker.heartbeats.in.second
     691 DEBUG - Handling deprecation for mapreduce.jobtracker.persist.jobstatus.dir
     692 DEBUG - Handling deprecation for dfs.namenode.backup.http-address
     693 DEBUG - Handling deprecation for hadoop.rpc.protection
     694 DEBUG - Handling deprecation for dfs.client.mmap.enabled
     695 DEBUG - Handling deprecation for yarn.app.mapreduce.am.container.log.backups
     696 DEBUG - Handling deprecation for ftp.stream-buffer-size
     697 DEBUG - Handling deprecation for dfs.namenode.https-address
     698 DEBUG - Handling deprecation for yarn.timeline-service.address
     699 DEBUG - Handling deprecation for dfs.ha.log-roll.period
     700 DEBUG - Handling deprecation for yarn.nodemanager.recovery.enabled
     701 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.numinputfiles
     702 DEBUG - Handling deprecation for hadoop.security.groups.negative-cache.secs
     703 DEBUG - Handling deprecation for yarn.resourcemanager.admin.client.thread-count
     704 DEBUG - Handling deprecation for dfs.datanode.fsdatasetcache.max.threads.per.volume
     705 DEBUG - Handling deprecation for file.client-write-packet-size
     706 DEBUG - Handling deprecation for hadoop.http.authentication.simple.anonymous.allowed
     707 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.path
     708 DEBUG - Handling deprecation for yarn.resourcemanager.proxy-user-privileges.enabled
     709 DEBUG - Handling deprecation for dfs.datanode.drop.cache.behind.reads
     710 DEBUG - Handling deprecation for yarn.nodemanager.log.retain-seconds
     711 DEBUG - Handling deprecation for dfs.image.transfer.bandwidthPerSec
     712 DEBUG - Handling deprecation for yarn.resourcemanager.work-preserving-recovery.scheduling-wait-ms
     713 DEBUG - Handling deprecation for hadoop.registry.zk.session.timeout.ms
     714 DEBUG - Handling deprecation for dfs.datanode.slow.io.warning.threshold.ms
     715 DEBUG - Handling deprecation for mapreduce.tasktracker.instrumentation
     716 DEBUG - Handling deprecation for ha.failover-controller.cli-check.rpc-timeout.ms
     717 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.cgroups.hierarchy
     718 DEBUG - Handling deprecation for dfs.namenode.write.stale.datanode.ratio
     719 DEBUG - Handling deprecation for hadoop.security.groups.cache.warn.after.ms
     720 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.fetch.retry.timeout-ms
     721 DEBUG - Handling deprecation for mapreduce.jobhistory.client.thread-count
     722 DEBUG - Handling deprecation for io.mapfile.bloom.size
     723 DEBUG - Handling deprecation for yarn.nodemanager.docker-container-executor.exec-name
     724 DEBUG - Handling deprecation for yarn.resourcemanager.work-preserving-recovery.enabled
     725 DEBUG - Handling deprecation for dfs.ha.fencing.ssh.connect-timeout
     726 DEBUG - Handling deprecation for yarn.resourcemanager.zk-num-retries
     727 DEBUG - Handling deprecation for hadoop.registry.zk.root
     728 DEBUG - Handling deprecation for s3.bytes-per-checksum
     729 DEBUG - Handling deprecation for yarn.app.mapreduce.am.container.log.limit.kb
     730 DEBUG - Handling deprecation for dfs.namenode.edit.log.autoroll.check.interval.ms
     731 DEBUG - Handling deprecation for fs.automatic.close
     732 DEBUG - Handling deprecation for fs.trash.interval
     733 DEBUG - Handling deprecation for dfs.journalnode.https-address
     734 DEBUG - Handling deprecation for yarn.timeline-service.ttl-ms
     735 DEBUG - Handling deprecation for hadoop.security.authentication
     736 DEBUG - Handling deprecation for fs.defaultFS
     737 DEBUG - Handling deprecation for nfs.rtmax
     738 DEBUG - Handling deprecation for hadoop.ssl.server.conf
     739 DEBUG - Handling deprecation for ipc.client.connect.max.retries
     740 DEBUG - Handling deprecation for yarn.resourcemanager.delayed.delegation-token.removal-interval-ms
     741 DEBUG - Handling deprecation for dfs.journalnode.http-address
     742 DEBUG - Handling deprecation for dfs.namenode.xattrs.enabled
     743 DEBUG - Handling deprecation for dfs.datanode.shared.file.descriptor.paths
     744 DEBUG - Handling deprecation for mapreduce.jobtracker.taskscheduler
     745 DEBUG - Handling deprecation for mapreduce.job.speculative.speculativecap
     746 DEBUG - Handling deprecation for yarn.timeline-service.store-class
     747 DEBUG - Handling deprecation for yarn.am.liveness-monitor.expiry-interval-ms
     748 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.compress
     749 DEBUG - Handling deprecation for dfs.user.home.dir.prefix
     750 DEBUG - Handling deprecation for net.topology.node.switch.mapping.impl
     751 DEBUG - Handling deprecation for dfs.namenode.replication.considerLoad
     752 DEBUG - Handling deprecation for dfs.namenode.fs-limits.min-block-size
     753 DEBUG - Handling deprecation for fs.swift.impl
     754 DEBUG - Handling deprecation for dfs.namenode.audit.loggers
     755 DEBUG - Handling deprecation for mapreduce.job.max.split.locations
     756 DEBUG - Handling deprecation for yarn.resourcemanager.address
     757 DEBUG - Handling deprecation for mapreduce.job.counters.max
     758 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.fetch.retry.enabled
     759 DEBUG - Handling deprecation for dfs.client.block.write.retries
     760 DEBUG - Handling deprecation for dfs.short.circuit.shared.memory.watcher.interrupt.check.ms
     761 DEBUG - Handling deprecation for dfs.namenode.resource.checked.volumes.minimum
     762 DEBUG - Handling deprecation for io.map.index.interval
     763 DEBUG - Handling deprecation for mapred.child.java.opts
     764 DEBUG - Handling deprecation for mapreduce.tasktracker.local.dir.minspacestart
     765 DEBUG - Handling deprecation for mapreduce.client.progressmonitor.pollinterval
     766 DEBUG - Handling deprecation for dfs.client.https.keystore.resource
     767 DEBUG - Handling deprecation for mapreduce.task.profile.map.params
     768 DEBUG - Handling deprecation for mapreduce.jobtracker.tasktracker.maxblacklists
     769 DEBUG - Handling deprecation for mapreduce.job.queuename
     770 DEBUG - Handling deprecation for hadoop.registry.zk.quorum
     771 DEBUG - Handling deprecation for yarn.app.mapreduce.client-am.ipc.max-retries-on-timeouts
     772 DEBUG - Handling deprecation for yarn.nodemanager.localizer.address
     773 DEBUG - Handling deprecation for io.mapfile.bloom.error.rate
     774 DEBUG - Handling deprecation for yarn.nodemanager.delete.thread-count
     775 DEBUG - Handling deprecation for mapreduce.job.split.metainfo.maxsize
     776 DEBUG - Handling deprecation for yarn.scheduler.maximum-allocation-vcores
     777 DEBUG - Handling deprecation for mapred.mapper.new-api
     778 DEBUG - Handling deprecation for mapreduce.job.dir
     779 DEBUG - Handling deprecation for yarn.app.mapreduce.am.resource.mb
     780 DEBUG - Handling deprecation for dfs.datanode.dns.nameserver
     781 DEBUG - Handling deprecation for dfs.client.slow.io.warning.threshold.ms
     782 DEBUG - Handling deprecation for mapreduce.job.reducer.preempt.delay.sec
     783 DEBUG - Handling deprecation for yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb
     784 DEBUG - Handling deprecation for mapreduce.map.output.compress.codec
     785 DEBUG - Handling deprecation for dfs.namenode.accesstime.precision
     786 DEBUG - Handling deprecation for fs.s3a.connection.maximum
     787 DEBUG - Handling deprecation for mapreduce.map.log.level
     788 DEBUG - Handling deprecation for io.seqfile.compress.blocksize
     789 DEBUG - Handling deprecation for ipc.client.ping
     790 DEBUG - Handling deprecation for mapreduce.tasktracker.taskcontroller
     791 DEBUG - Handling deprecation for hadoop.security.groups.cache.secs
     792 DEBUG - Handling deprecation for dfs.datanode.cache.revocation.timeout.ms
     793 DEBUG - Handling deprecation for dfs.client.context
     794 DEBUG - Handling deprecation for mapreduce.input.lineinputformat.linespermap
     795 DEBUG - Handling deprecation for mapreduce.job.end-notification.max.attempts
     796 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user
     797 DEBUG - Handling deprecation for yarn.nodemanager.webapp.address
     798 DEBUG - Handling deprecation for mapreduce.job.submithostname
     799 DEBUG - Handling deprecation for mapreduce.jobhistory.recovery.enable
     800 DEBUG - Handling deprecation for mapreduce.jobtracker.expire.trackers.interval
     801 DEBUG - Handling deprecation for yarn.resourcemanager.webapp.address
     802 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.num.refill.threads
     803 DEBUG - Handling deprecation for yarn.nodemanager.health-checker.interval-ms
     804 DEBUG - Handling deprecation for mapreduce.jobhistory.loadedjobs.cache.size
     805 DEBUG - Handling deprecation for hadoop.security.authorization
     806 DEBUG - Handling deprecation for mapreduce.job.map.output.collector.class
     807 DEBUG - Handling deprecation for mapreduce.am.max-attempts
     808 DEBUG - Handling deprecation for fs.ftp.host
     809 DEBUG - Handling deprecation for fs.s3a.attempts.maximum
     810 DEBUG - Handling deprecation for yarn.app.mapreduce.am.scheduler.heartbeat.interval-ms
     811 DEBUG - Handling deprecation for mapreduce.ifile.readahead
     812 DEBUG - Handling deprecation for yarn.resourcemanager.scheduler.monitor.enable
     813 DEBUG - Handling deprecation for yarn.resourcemanager.zk-retry-interval-ms
     814 DEBUG - Handling deprecation for ha.zookeeper.session-timeout.ms
     815 DEBUG - Handling deprecation for mapreduce.tasktracker.taskmemorymanager.monitoringinterval
     816 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.parallelcopies
     817 DEBUG - Handling deprecation for dfs.client.mmap.retry.timeout.ms
     818 DEBUG - Handling deprecation for mapreduce.map.skip.maxrecords
     819 DEBUG - Handling deprecation for mapreduce.job.output.value.class
     820 DEBUG - Handling deprecation for dfs.namenode.avoid.read.stale.datanode
     821 DEBUG - Handling deprecation for dfs.https.enable
     822 DEBUG - Handling deprecation for yarn.timeline-service.webapp.https.address
     823 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.read.timeout
     824 DEBUG - Handling deprecation for dfs.namenode.list.encryption.zones.num.responses
     825 DEBUG - Handling deprecation for yarn.nodemanager.remote-app-log-dir-suffix
     826 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.compress.codec
     827 DEBUG - Handling deprecation for mapreduce.jobtracker.instrumentation
     828 DEBUG - Handling deprecation for dfs.blockreport.intervalMsec
     829 DEBUG - Handling deprecation for ipc.client.connect.retry.interval
     830 DEBUG - Handling deprecation for mapreduce.reduce.speculative
     831 DEBUG - Handling deprecation for mapreduce.jobhistory.keytab
     832 DEBUG - Handling deprecation for mapreduce.jobhistory.datestring.cache.size
     833 DEBUG - Handling deprecation for dfs.datanode.balance.bandwidthPerSec
     834 DEBUG - Handling deprecation for file.blocksize
     835 DEBUG - Handling deprecation for yarn.resourcemanager.admin.address
     836 DEBUG - Handling deprecation for mapreduce.map.cpu.vcores
     837 DEBUG - Handling deprecation for yarn.nodemanager.container-monitor.procfs-tree.smaps-based-rss.enabled
     838 DEBUG - Handling deprecation for yarn.resourcemanager.configuration.provider-class
     839 DEBUG - Handling deprecation for yarn.resourcemanager.resource-tracker.address
     840 DEBUG - Handling deprecation for mapreduce.tasktracker.local.dir.minspacekill
     841 DEBUG - Handling deprecation for mapreduce.jobtracker.staging.root.dir
     842 DEBUG - Handling deprecation for mapreduce.jobtracker.retiredjobs.cache.size
     843 DEBUG - Handling deprecation for hadoop.registry.rm.enabled
     844 DEBUG - Handling deprecation for ipc.client.connect.max.retries.on.timeouts
     845 DEBUG - Handling deprecation for ha.zookeeper.acl
     846 DEBUG - Handling deprecation for hadoop.security.crypto.codec.classes.aes.ctr.nopadding
     847 DEBUG - Handling deprecation for yarn.nodemanager.local-dirs
     848 DEBUG - Handling deprecation for mapreduce.app-submission.cross-platform
     849 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.connect.timeout
     850 DEBUG - Handling deprecation for dfs.block.access.key.update.interval
     851 DEBUG - Handling deprecation for rpc.metrics.quantile.enable
     852 DEBUG - Handling deprecation for dfs.block.access.token.lifetime
     853 DEBUG - Handling deprecation for mapreduce.job.end-notification.retry.attempts
     854 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-xattrs-per-inode
     855 DEBUG - Handling deprecation for mapreduce.jobtracker.system.dir
     856 DEBUG - Handling deprecation for dfs.client.file-block-storage-locations.timeout.millis
     857 DEBUG - Handling deprecation for yarn.nodemanager.admin-env
     858 DEBUG - Handling deprecation for mapreduce.jobtracker.jobhistory.block.size
     859 DEBUG - Handling deprecation for yarn.log-aggregation.retain-seconds
     860 DEBUG - Handling deprecation for mapreduce.tasktracker.indexcache.mb
     861 DEBUG - Handling deprecation for yarn.timeline-service.handler-thread-count
     862 DEBUG - Handling deprecation for dfs.namenode.checkpoint.check.period
     863 DEBUG - Handling deprecation for yarn.resourcemanager.hostname
     864 DEBUG - Handling deprecation for dfs.client.block.write.replace-datanode-on-failure.enable
     865 DEBUG - Handling deprecation for net.topology.impl
     866 DEBUG - Handling deprecation for dfs.datanode.directoryscan.interval
     867 DEBUG - Handling deprecation for fs.s3a.multipart.purge.age
     868 DEBUG - Handling deprecation for hadoop.security.java.secure.random.algorithm
     869 DEBUG - Handling deprecation for yarn.nodemanager.container-monitor.interval-ms
     870 DEBUG - Handling deprecation for dfs.default.chunk.view.size
     871 DEBUG - Handling deprecation for fs.s3a.multipart.threshold
     872 DEBUG - Handling deprecation for mapreduce.job.speculative.slownodethreshold
     873 DEBUG - Handling deprecation for mapreduce.job.reduce.slowstart.completedmaps
     874 DEBUG - Handling deprecation for mapred.reducer.new-api
     875 DEBUG - Handling deprecation for hadoop.security.instrumentation.requires.admin
     876 DEBUG - Handling deprecation for io.compression.codec.bzip2.library
     877 DEBUG - Handling deprecation for hadoop.http.authentication.signature.secret.file
     878 DEBUG - Handling deprecation for hadoop.registry.secure
     879 DEBUG - Handling deprecation for dfs.namenode.safemode.min.datanodes
     880 DEBUG - Handling deprecation for yarn.nodemanager.localizer.cache.target-size-mb
     881 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.inputdir
     882 DEBUG - Handling deprecation for mapreduce.reduce.maxattempts
     883 DEBUG - Handling deprecation for dfs.datanode.https.address
     884 DEBUG - Handling deprecation for s3native.replication
     885 DEBUG - Handling deprecation for dfs.namenode.inotify.max.events.per.rpc
     886 DEBUG - Handling deprecation for dfs.namenode.path.based.cache.retry.interval.ms
     887 DEBUG - Handling deprecation for mapreduce.reduce.skip.proc.count.autoincr
     888 DEBUG - Handling deprecation for dfs.datanode.cache.revocation.polling.ms
     889 DEBUG - Handling deprecation for mapreduce.jobhistory.cleaner.interval-ms
     890 DEBUG - Handling deprecation for file.replication
     891 DEBUG - Handling deprecation for hadoop.hdfs.configuration.version
     892 DEBUG - Handling deprecation for ipc.client.idlethreshold
     893 DEBUG - Handling deprecation for hadoop.tmp.dir
     894 DEBUG - Handling deprecation for yarn.resourcemanager.store.class
     895 DEBUG - Handling deprecation for mapreduce.jobhistory.address
     896 DEBUG - Handling deprecation for mapreduce.jobtracker.restart.recover
     897 DEBUG - Handling deprecation for mapreduce.cluster.local.dir
     898 DEBUG - Handling deprecation for yarn.client.nodemanager-client-async.thread-pool-max-size
     899 DEBUG - Handling deprecation for dfs.namenode.decommission.nodes.per.interval
     900 DEBUG - Handling deprecation for mapreduce.job.inputformat.class
     901 DEBUG - Handling deprecation for yarn.nodemanager.resource.cpu-vcores
     902 DEBUG - Handling deprecation for dfs.namenode.reject-unresolved-dn-topology-mapping
     903 DEBUG - Handling deprecation for dfs.namenode.delegation.key.update-interval
     904 DEBUG - Handling deprecation for fs.s3.buffer.dir
     905 DEBUG - Handling deprecation for dfs.client.read.shortcircuit.streams.cache.expiry.ms
     906 DEBUG - Handling deprecation for dfs.namenode.support.allow.format
     907 DEBUG - Handling deprecation for yarn.nodemanager.remote-app-log-dir
     908 DEBUG - Handling deprecation for mapreduce.map.memory.mb
     909 DEBUG - Handling deprecation for dfs.namenode.edit.log.autoroll.multiplier.threshold
     910 DEBUG - Handling deprecation for hadoop.work.around.non.threadsafe.getpwuid
     911 DEBUG - Handling deprecation for mapreduce.task.profile.reduce.params
     912 DEBUG - Handling deprecation for yarn.timeline-service.client.max-retries
     913 DEBUG - Handling deprecation for dfs.ha.automatic-failover.enabled
     914 DEBUG - Handling deprecation for dfs.namenode.edits.noeditlogchannelflush
     915 DEBUG - Handling deprecation for dfs.namenode.stale.datanode.interval
     916 DEBUG - Handling deprecation for mapreduce.shuffle.transfer.buffer.size
     917 DEBUG - Handling deprecation for mapreduce.jobtracker.persist.jobstatus.active
     918 DEBUG - Handling deprecation for dfs.namenode.logging.level
     919 DEBUG - Handling deprecation for yarn.resourcemanager.am-rm-tokens.master-key-rolling-interval-secs
     920 DEBUG - Handling deprecation for yarn.nodemanager.log-dirs
     921 DEBUG - Handling deprecation for ha.health-monitor.sleep-after-disconnect.ms
     922 DEBUG - Handling deprecation for yarn.resourcemanager.fs.state-store.uri
     923 DEBUG - Handling deprecation for dfs.namenode.checkpoint.edits.dir
     924 DEBUG - Handling deprecation for yarn.resourcemanager.keytab
     925 DEBUG - Handling deprecation for hadoop.rpc.socket.factory.class.default
     926 DEBUG - Handling deprecation for yarn.resourcemanager.fail-fast
     927 DEBUG - Handling deprecation for dfs.datanode.http.address
     928 DEBUG - Handling deprecation for mapreduce.task.profile
     929 DEBUG - Handling deprecation for mapreduce.jobhistory.move.interval-ms
     930 DEBUG - Handling deprecation for dfs.namenode.edits.dir
     931 DEBUG - Handling deprecation for dfs.storage.policy.enabled
     932 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.size
     933 DEBUG - Handling deprecation for hadoop.fuse.timer.period
     934 DEBUG - Handling deprecation for mapreduce.jobhistory.http.policy
     935 DEBUG - Handling deprecation for mapreduce.jobhistory.intermediate-done-dir
     936 DEBUG - Handling deprecation for mapreduce.map.skip.proc.count.autoincr
     937 DEBUG - Handling deprecation for fs.AbstractFileSystem.viewfs.impl
     938 DEBUG - Handling deprecation for mapreduce.job.speculative.slowtaskthreshold
     939 DEBUG - Handling deprecation for yarn.resourcemanager.webapp.delegation-token-auth-filter.enabled
     940 DEBUG - Handling deprecation for s3native.stream-buffer-size
     941 DEBUG - Handling deprecation for yarn.nodemanager.delete.debug-delay-sec
     942 DEBUG - Handling deprecation for dfs.secondary.namenode.kerberos.internal.spnego.principal
     943 DEBUG - Handling deprecation for dfs.datanode.available-space-volume-choosing-policy.balanced-space-threshold
     944 DEBUG - Handling deprecation for fs.s3n.multipart.uploads.block.size
     945 DEBUG - Handling deprecation for dfs.namenode.safemode.threshold-pct
     946 DEBUG - Handling deprecation for mapreduce.ifile.readahead.bytes
     947 DEBUG - Handling deprecation for yarn.scheduler.maximum-allocation-mb
     948 DEBUG - Handling deprecation for ipc.client.fallback-to-simple-auth-allowed
     949 DEBUG - Handling deprecation for fs.har.impl.disable.cache
     950 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.read-cache-size
     951 DEBUG - Handling deprecation for yarn.timeline-service.hostname
     952 DEBUG - Handling deprecation for s3native.bytes-per-checksum
     953 DEBUG - Handling deprecation for mapreduce.job.committer.setup.cleanup.needed
     954 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms
     955 DEBUG - Handling deprecation for fs.s3a.paging.maximum
     956 DEBUG - Handling deprecation for yarn.client.nodemanager-connect.retry-interval-ms
     957 DEBUG - Handling deprecation for yarn.nodemanager.log-aggregation.compression-type
     958 DEBUG - Handling deprecation for yarn.app.mapreduce.am.job.committer.commit-window
     959 DEBUG - Handling deprecation for hadoop.http.authentication.type
     960 DEBUG - Handling deprecation for dfs.client.failover.sleep.base.millis
     961 DEBUG - Handling deprecation for mapreduce.job.submithostaddress
     962 DEBUG - Handling deprecation for yarn.nodemanager.vmem-check-enabled
     963 DEBUG - Handling deprecation for hadoop.jetty.logs.serve.aliases
     964 DEBUG - Handling deprecation for ha.failover-controller.graceful-fence.rpc-timeout.ms
     965 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.input.buffer.percent
     966 DEBUG - Handling deprecation for dfs.datanode.max.transfer.threads
     967 DEBUG - Handling deprecation for mapreduce.reduce.merge.inmem.threshold
     968 DEBUG - Handling deprecation for mapreduce.task.io.sort.mb
     969 DEBUG - Handling deprecation for dfs.namenode.acls.enabled
     970 DEBUG - Handling deprecation for yarn.client.application-client-protocol.poll-interval-ms
     971 DEBUG - Handling deprecation for hadoop.security.kms.client.authentication.retry-count
     972 DEBUG - Handling deprecation for dfs.namenode.handler.count
     973 DEBUG - Handling deprecation for yarn.resourcemanager.connect.max-wait.ms
     974 DEBUG - Handling deprecation for dfs.namenode.retrycache.heap.percent
     975 DEBUG - Handling deprecation for yarn.timeline-service.enabled
     976 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users
     977 DEBUG - Handling deprecation for hadoop.ssl.client.conf
     978 DEBUG - Handling deprecation for yarn.resourcemanager.container.liveness-monitor.interval-ms
     979 DEBUG - Handling deprecation for yarn.nodemanager.vmem-pmem-ratio
     980 DEBUG - Handling deprecation for mapreduce.client.completion.pollinterval
     981 DEBUG - Handling deprecation for yarn.app.mapreduce.client.max-retries
     982 DEBUG - Handling deprecation for hadoop.ssl.enabled
     983 DEBUG - Handling deprecation for fs.client.resolve.remote.symlinks
     984 DEBUG - Handling deprecation for fs.AbstractFileSystem.hdfs.impl
     985 DEBUG - Handling deprecation for mapreduce.tasktracker.reduce.tasks.maximum
     986 DEBUG - Handling deprecation for yarn.nodemanager.hostname
     987 DEBUG - Handling deprecation for mapreduce.reduce.input.buffer.percent
     988 DEBUG - Handling deprecation for fs.s3a.multipart.purge
     989 DEBUG - Handling deprecation for yarn.app.mapreduce.am.command-opts
     990 DEBUG - Handling deprecation for dfs.namenode.invalidate.work.pct.per.iteration
     991 DEBUG - Handling deprecation for dfs.bytes-per-checksum
     992 DEBUG - Handling deprecation for yarn.resourcemanager.webapp.https.address
     993 DEBUG - Handling deprecation for dfs.replication
     994 DEBUG - Handling deprecation for dfs.datanode.block.id.layout.upgrade.threads
     995 DEBUG - Handling deprecation for mapreduce.shuffle.ssl.file.buffer.size
     996 DEBUG - Handling deprecation for dfs.namenode.list.cache.directives.num.responses
     997 DEBUG - Handling deprecation for dfs.permissions.enabled
     998 DEBUG - Handling deprecation for mapreduce.jobtracker.maxtasks.perjob
     999 DEBUG - Handling deprecation for dfs.datanode.use.datanode.hostname
    1000 DEBUG - Handling deprecation for mapreduce.task.userlog.limit.kb
    1001 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-directory-items
    1002 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.low-watermark
    1003 DEBUG - Handling deprecation for fs.s3a.buffer.dir
    1004 DEBUG - Handling deprecation for s3.client-write-packet-size
    1005 DEBUG - Handling deprecation for hadoop.user.group.static.mapping.overrides
    1006 DEBUG - Handling deprecation for mapreduce.shuffle.max.threads
    1007 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.expiry
    1008 DEBUG - Handling deprecation for dfs.client.failover.sleep.max.millis
    1009 DEBUG - Handling deprecation for mapreduce.job.maps
    1010 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-component-length
    1011 DEBUG - Handling deprecation for yarn.fail-fast
    1012 DEBUG - Handling deprecation for hadoop.ssl.enabled.protocols
    1013 DEBUG - Handling deprecation for s3.blocksize
    1014 DEBUG - Handling deprecation for mapreduce.map.output.compress
    1015 DEBUG - Handling deprecation for dfs.namenode.edits.journal-plugin.qjournal
    1016 DEBUG - Handling deprecation for dfs.namenode.datanode.registration.ip-hostname-check
    1017 DEBUG - Handling deprecation for yarn.nodemanager.pmem-check-enabled
    1018 DEBUG - Handling deprecation for dfs.client.short.circuit.replica.stale.threshold.ms
    1019 DEBUG - Handling deprecation for dfs.client.https.need-auth
    1020 DEBUG - Handling deprecation for yarn.scheduler.minimum-allocation-mb
    1021 DEBUG - Handling deprecation for mapreduce.jobhistory.max-age-ms
    1022 DEBUG - Handling deprecation for yarn.timeline-service.http-authentication.type
    1023 DEBUG - Handling deprecation for ftp.replication
    1024 DEBUG - Handling deprecation for dfs.namenode.secondary.https-address
    1025 DEBUG - Handling deprecation for dfs.blockreport.split.threshold
    1026 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.split.minsize
    1027 DEBUG - Handling deprecation for fs.s3n.block.size
    1028 DEBUG - Handling deprecation for mapreduce.job.token.tracking.ids.enabled
    1029 DEBUG - Handling deprecation for yarn.ipc.rpc.class
    1030 DEBUG - Handling deprecation for dfs.namenode.num.extra.edits.retained
    1031 DEBUG - Handling deprecation for yarn.nodemanager.localizer.cache.cleanup.interval-ms
    1032 DEBUG - Handling deprecation for hadoop.http.staticuser.user
    1033 DEBUG - Handling deprecation for mapreduce.jobhistory.move.thread-count
    1034 DEBUG - Handling deprecation for fs.s3a.multipart.size
    1035 DEBUG - Handling deprecation for mapreduce.job.jvm.numtasks
    1036 DEBUG - Handling deprecation for mapreduce.task.profile.maps
    1037 DEBUG - Handling deprecation for dfs.datanode.max.locked.memory
    1038 DEBUG - Handling deprecation for dfs.cachereport.intervalMsec
    1039 DEBUG - Handling deprecation for mapreduce.shuffle.port
    1040 DEBUG - Handling deprecation for yarn.resourcemanager.nodemanager.minimum.version
    1041 DEBUG - Handling deprecation for mapreduce.shuffle.connection-keep-alive.timeout
    1042 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.merge.percent
    1043 DEBUG - Handling deprecation for mapreduce.jobtracker.http.address
    1044 DEBUG - Handling deprecation for mapreduce.task.skip.start.attempts
    1045 DEBUG - Handling deprecation for yarn.resourcemanager.connect.retry-interval.ms
    1046 DEBUG - Handling deprecation for yarn.scheduler.minimum-allocation-vcores
    1047 DEBUG - Handling deprecation for mapreduce.task.io.sort.factor
    1048 DEBUG - Handling deprecation for dfs.namenode.checkpoint.dir
    1049 DEBUG - Handling deprecation for nfs.exports.allowed.hosts
    1050 DEBUG - Handling deprecation for tfile.fs.input.buffer.size
    1051 DEBUG - Handling deprecation for fs.s3.block.size
    1052 DEBUG - Handling deprecation for tfile.io.chunk.size
    1053 DEBUG - Handling deprecation for fs.s3n.multipart.copy.block.size
    1054 DEBUG - Handling deprecation for io.serializations
    1055 DEBUG - Handling deprecation for yarn.resourcemanager.max-completed-applications
    1056 DEBUG - Handling deprecation for mapreduce.jobhistory.principal
    1057 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.outputdir
    1058 DEBUG - Handling deprecation for yarn.resourcemanager.ha.automatic-failover.zk-base-path
    1059 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.fetch.retry.interval-ms
    1060 DEBUG - Handling deprecation for mapreduce.job.end-notification.retry.interval
    1061 DEBUG - Handling deprecation for dfs.namenode.backup.address
    1062 DEBUG - Handling deprecation for fs.s3n.multipart.uploads.enabled
    1063 DEBUG - Handling deprecation for io.seqfile.sorter.recordlimit
    1064 DEBUG - Handling deprecation for dfs.block.access.token.enable
    1065 DEBUG - Handling deprecation for s3native.client-write-packet-size
    1066 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-xattr-size
    1067 DEBUG - Handling deprecation for ftp.bytes-per-checksum
    1068 DEBUG - Handling deprecation for hadoop.security.group.mapping
    1069 DEBUG - Handling deprecation for dfs.client.domain.socket.data.traffic
    1070 DEBUG - Handling deprecation for dfs.client.read.shortcircuit.streams.cache.size
    1071 DEBUG - Handling deprecation for fs.s3a.connection.timeout
    1072 DEBUG - Handling deprecation for mapreduce.job.end-notification.max.retry.interval
    1073 DEBUG - Handling deprecation for yarn.acl.enable
    1074 DEBUG - Handling deprecation for yarn.nm.liveness-monitor.expiry-interval-ms
    1075 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.list-status.num-threads
    1076 DEBUG - Handling deprecation for dfs.client.mmap.cache.size
    1077 DEBUG - Handling deprecation for mapreduce.tasktracker.map.tasks.maximum
    1078 DEBUG - Handling deprecation for yarn.timeline-service.ttl-enable
    1079 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.resources-handler.class
    1080 DEBUG - Handling deprecation for dfs.namenode.max.objects
    1081 DEBUG - Handling deprecation for yarn.resourcemanager.state-store.max-completed-applications
    1082 DEBUG - Handling deprecation for dfs.namenode.delegation.token.max-lifetime
    1083 DEBUG - Handling deprecation for mapreduce.job.classloader
    1084 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size
    1085 DEBUG - Handling deprecation for mapreduce.job.hdfs-servers
    1086 DEBUG - Handling deprecation for dfs.datanode.hdfs-blocks-metadata.enabled
    1087 DEBUG - Handling deprecation for mapreduce.tasktracker.dns.nameserver
    1088 DEBUG - Handling deprecation for dfs.datanode.readahead.bytes
    1089 DEBUG - Handling deprecation for dfs.datanode.bp-ready.timeout
    1090 DEBUG - Handling deprecation for mapreduce.job.ubertask.maxreduces
    1091 DEBUG - Handling deprecation for dfs.image.compress
    1092 DEBUG - Handling deprecation for mapreduce.shuffle.ssl.enabled
    1093 DEBUG - Handling deprecation for yarn.log-aggregation-enable
    1094 DEBUG - Handling deprecation for mapreduce.tasktracker.report.address
    1095 DEBUG - Handling deprecation for mapreduce.tasktracker.http.threads
    1096 DEBUG - Handling deprecation for dfs.stream-buffer-size
    1097 DEBUG - Handling deprecation for tfile.fs.output.buffer.size
    1098 DEBUG - Handling deprecation for fs.permissions.umask-mode
    1099 DEBUG - Handling deprecation for dfs.client.datanode-restart.timeout
    1100 DEBUG - Handling deprecation for dfs.namenode.resource.du.reserved
    1101 DEBUG - Handling deprecation for yarn.resourcemanager.am.max-attempts
    1102 DEBUG - Handling deprecation for yarn.nodemanager.resource.percentage-physical-cpu-limit
    1103 DEBUG - Handling deprecation for ha.failover-controller.graceful-fence.connection.retries
    1104 DEBUG - Handling deprecation for dfs.datanode.drop.cache.behind.writes
    1105 DEBUG - Handling deprecation for mapreduce.map.output.value.class
    1106 DEBUG - Handling deprecation for hadoop.common.configuration.version
    1107 DEBUG - Handling deprecation for mapreduce.job.ubertask.enable
    1108 DEBUG - Handling deprecation for yarn.app.mapreduce.am.resource.cpu-vcores
    1109 DEBUG - Handling deprecation for dfs.namenode.replication.work.multiplier.per.iteration
    1110 DEBUG - Handling deprecation for mapreduce.job.acl-modify-job
    1111 DEBUG - Handling deprecation for io.seqfile.local.dir
    1112 DEBUG - Handling deprecation for yarn.resourcemanager.system-metrics-publisher.enabled
    1113 DEBUG - Handling deprecation for fs.s3.sleepTimeSeconds
    1114 DEBUG - Handling deprecation for mapreduce.client.output.filter
    1115  INFO - The url to track the job: http://localhost:8080/
    1116  INFO - Running job: job_local1553403857_0001
    1117  INFO - OutputCommitter set in config null
    1118 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)
    1119 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)
    1120  INFO - OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
    1121 DEBUG - Starting mapper thread pool executor.
    1122 DEBUG - Max local threads: 1
    1123 DEBUG - Map tasks to process: 1
    1124  INFO - Waiting for map tasks
    1125  INFO - Starting task: attempt_local1553403857_0001_m_000000_0
    1126 DEBUG - currentIndex 0   0:0
    1127 DEBUG - mapreduce.cluster.local.dir for child : /tmp/hadoop-SYJ/mapred/local/localRunner//SYJ/jobcache/job_local1553403857_0001/attempt_local1553403857_0001_m_000000_0
    1128 DEBUG - using new api for output committer
    1129  INFO - ProcfsBasedProcessTree currently is supported only on Linux.
    1130  INFO -  Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@5e2c17f7
    1131  INFO - Processing split: file:/c:/wordcount/input/words.txt:0+83
    1132 DEBUG - Trying map output collector class: org.apache.hadoop.mapred.MapTask$MapOutputBuffer
    1133  INFO - (EQUATOR) 0 kvi 26214396(104857584)
    1134  INFO - mapreduce.task.io.sort.mb: 100
    1135  INFO - soft limit at 83886080
    1136  INFO - bufstart = 0; bufvoid = 104857600
    1137  INFO - kvstart = 26214396; length = 6553600
    1138  INFO - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
    1139  INFO - 
    1140  INFO - Starting flush of map output
    1141  INFO - Spilling map output
    1142  INFO - bufstart = 0; bufend = 129; bufvoid = 104857600
    1143  INFO - kvstart = 26214396(104857584); kvend = 26214352(104857408); length = 45/6553600
    1144  INFO - Finished spill 0
    1145  INFO - Task:attempt_local1553403857_0001_m_000000_0 is done. And is in the process of committing
    1146  INFO - map
    1147  INFO - Task 'attempt_local1553403857_0001_m_000000_0' done.
    1148  INFO - Finishing task: attempt_local1553403857_0001_m_000000_0
    1149  INFO - map task executor complete.
    1150 DEBUG - Starting reduce thread pool executor.
    1151 DEBUG - Max local threads: 1
    1152 DEBUG - Reduce tasks to process: 1
    1153  INFO - Waiting for reduce tasks
    1154  INFO - Starting task: attempt_local1553403857_0001_r_000000_0
    1155 DEBUG - currentIndex 0   0:0
    1156 DEBUG - mapreduce.cluster.local.dir for child : /tmp/hadoop-SYJ/mapred/local/localRunner//SYJ/jobcache/job_local1553403857_0001/attempt_local1553403857_0001_r_000000_0
    1157 DEBUG - using new api for output committer
    1158  INFO - ProcfsBasedProcessTree currently is supported only on Linux.
    1159  INFO -  Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@41217e67
    1160  INFO - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@25071521
    1161  INFO - MergerManager: memoryLimit=1287946240, maxSingleShuffleLimit=321986560, mergeThreshold=850044544, ioSortFactor=10, memToMemMergeOutputsThreshold=10
    1162  INFO - attempt_local1553403857_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
    1163 DEBUG - Got 0 map completion events from 0
    1164 DEBUG - GetMapEventsThread about to sleep for 1000
    1165 DEBUG - LocalFetcher 1 going to fetch: attempt_local1553403857_0001_m_000000_0
    1166 DEBUG - attempt_local1553403857_0001_m_000000_0: Proceeding with shuffle since usedMemory (0) is lesser than memoryLimit (1287946240).CommitMemory is (0)
    1167  INFO - localfetcher#1 about to shuffle output of map attempt_local1553403857_0001_m_000000_0 decomp: 155 len: 159 to MEMORY
    1168  INFO - Read 155 bytes from map-output for attempt_local1553403857_0001_m_000000_0
    1169  INFO - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155
    1170 DEBUG - map attempt_local1553403857_0001_m_000000_0 done 1 / 1 copied.
    1171  INFO - EventFetcher is interrupted.. Returning
    1172  INFO - 1 / 1 copied.
    1173  INFO - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
    1174  INFO - Merging 1 sorted segments
    1175  INFO - Down to the last merge-pass, with 1 segments left of total size: 145 bytes
    1176  INFO - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit
    1177 DEBUG - Disk file: /tmp/hadoop-SYJ/mapred/local/localRunner/SYJ/jobcache/job_local1553403857_0001/attempt_local1553403857_0001_r_000000_0/output/map_0.out.merged Length is 159
    1178  INFO - Merging 1 files, 159 bytes from disk
    1179  INFO - Merging 0 segments, 0 bytes from memory into reduce
    1180  INFO - Merging 1 sorted segments
    1181  INFO - Down to the last merge-pass, with 1 segments left of total size: 145 bytes
    1182  INFO - 1 / 1 copied.
    1183  INFO - mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
    1184  INFO - Task:attempt_local1553403857_0001_r_000000_0 is done. And is in the process of committing
    1185  INFO - 1 / 1 copied.
    1186  INFO - Task attempt_local1553403857_0001_r_000000_0 is allowed to commit now
    1187  INFO - Saved output of task 'attempt_local1553403857_0001_r_000000_0' to file:/c:/wordcount/output/_temporary/0/task_local1553403857_0001_r_000000
    1188  INFO - reduce > reduce
    1189  INFO - Task 'attempt_local1553403857_0001_r_000000_0' done.
    1190  INFO - Finishing task: attempt_local1553403857_0001_r_000000_0
    1191  INFO - reduce task executor complete.
    1192 DEBUG - Merging data from DeprecatedRawLocalFileStatus{path=file:/c:/wordcount/output/_temporary/0/task_local1553403857_0001_r_000000; isDirectory=true; modification_time=1484061974082; access_time=0; owner=; group=; permission=rwxrwxrwx; isSymlink=false} to file:/c:/wordcount/output
    1193 DEBUG - Merging data from DeprecatedRawLocalFileStatus{path=file:/c:/wordcount/output/_temporary/0/task_local1553403857_0001_r_000000/part-r-00000; isDirectory=false; length=62; replication=1; blocksize=33554432; modification_time=1484061974091; access_time=0; owner=; group=; permission=rw-rw-rw-; isSymlink=false} to file:/c:/wordcount/output/part-r-00000
    1194 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:331)
    1195 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)
    1196  INFO - Job job_local1553403857_0001 running in uber mode : false
    1197  INFO -  map 100% reduce 100%
    1198 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.getTaskCompletionEvents(Job.java:677)
    1199 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)
    1200 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)
    1201 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.getTaskCompletionEvents(Job.java:677)
    1202 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)
    1203 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)
    1204  INFO - Job job_local1553403857_0001 completed successfully
    1205 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.getCounters(Job.java:765)
    1206  INFO - Counters: 33
    1207     File System Counters
    1208         FILE: Number of bytes read=822
    1209         FILE: Number of bytes written=505185
    1210         FILE: Number of read operations=0
    1211         FILE: Number of large read operations=0
    1212         FILE: Number of write operations=0
    1213     Map-Reduce Framework
    1214         Map input records=4
    1215         Map output records=12
    1216         Map output bytes=129
    1217         Map output materialized bytes=159
    1218         Input split bytes=99
    1219         Combine input records=0
    1220         Combine output records=0
    1221         Reduce input groups=7
    1222         Reduce shuffle bytes=159
    1223         Reduce input records=12
    1224         Reduce output records=7
    1225         Spilled Records=24
    1226         Shuffled Maps =1
    1227         Failed Shuffles=0
    1228         Merged Map outputs=1
    1229         GC time elapsed (ms)=0
    1230         CPU time spent (ms)=0
    1231         Physical memory (bytes) snapshot=0
    1232         Virtual memory (bytes) snapshot=0
    1233         Total committed heap usage (bytes)=457703424
    1234     Shuffle Errors
    1235         BAD_ID=0
    1236         CONNECTION=0
    1237         IO_ERROR=0
    1238         WRONG_LENGTH=0
    1239         WRONG_MAP=0
    1240         WRONG_REDUCE=0
    1241     File Input Format Counters 
    1242         Bytes Read=83
    1243     File Output Format Counters 
    1244         Bytes Written=74
    1245 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)
    View Code

    19.到"c:/wordcount"目录下面可以看到生成的output目录,下面有一些输出结果文件,其中"part-r-00000"文件就是程序运算的结果:

    如果觉得本文对您有帮助,不妨扫描下方微信二维码打赏点,您的鼓励是我前进最大的动力:

  • 相关阅读:
    python字符串连接方式(转)
    Python顺序与range和random
    将EXCEL中的列拼接成SQL insert插入语句
    Python OS模块
    Python3.5连接Mysql
    Mysql查看连接端口及版本
    Mysqldb连接Mysql数据库(转)
    Python 文件I/O (转)
    Python 日期和时间(转)
    Python序列的方法(转)
  • 原文地址:https://www.cnblogs.com/jun1019/p/6271303.html
Copyright © 2011-2022 走看看