nginx: [emerg] "upstream" directive is not allowed here in /usr/tengine-2.1.0/conf/nginx.conf:47
configuration file /usr/tengine-2.1.0/conf/nginx.conf test failed
多了一个 }
16/06/26 14:06:01 WARN node.AbstractConfigurationProvider: No configuration found for this host:clin1
java 环境变量 《这个可能不是错误 》
org.apache.commons.cli.ParseException: The specified configuration file does not exist: /usr/apache-flume-1.6.0-bin/bin/ile
错误1 ，conf 文件 = 等号两边不能有空格 《这个可能不是错误 》
错误2 ，启动命令错误
错误 bin/flume-ng agent -n clei -c conf -file /usr/apache-flume-1.6.0-bin/conf/test2 -Dflume.root.logger=INFO,console
正确 flume-ng agent --conf conf --conf-file test3 --name a1 -Dflume.root.logger=INFO,console
16/06/26 18:08:45 ERROR source.SpoolDirectorySource: FATAL: Spool Directory source r1: { spoolDir: /opt/sqooldir }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.
java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:277)
at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)
at org.apache.flume.serialization.LineDeserializer.readLine(LineDeserializer.java:133)
at org.apache.flume.serialization.LineDeserializer.readEvent(LineDeserializer.java:71)
at org.apache.flume.serialization.LineDeserializer.readEvents(LineDeserializer.java:90)
at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:252)
at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:228)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
不能上传压缩文件 ,文件名称有问题的文件，估计视频文件就更不行
16/06/26 18:18:59 INFO ipc.NettyServer: [id: 0x6fef6466, /192.168.184.188:40594 => /192.168.184.188:44444] CONNECTED: /192.168.184.188:40594
16/06/26 18:19:05 INFO hdfs.HDFSDataStream: Serializer = TEXT, UseRawLocalFileSystem = false
16/06/26 18:19:08 INFO hdfs.BucketWriter: Creating hdfs://bigdatastorm:8020/flume/data/FlumeData.1466936345775.tmp
16/06/26 18:19:18 WARN hdfs.HDFSEventSink: HDFS IO error
java.io.IOException: Callable timed out after 10000 ms on file: hdfs://bigdatastorm:8020/flume/data/FlumeData.1466936345775.tmp
at org.apache.flume.sink.hdfs.BucketWriter.callWithTimeout(BucketWriter.java:693)
at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:235)
at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:514)
at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:418)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.TimeoutException
at java.util.concurrent.FutureTask.get(FutureTask.java:201)
at org.apache.flume.sink.hdfs.BucketWriter.callWithTimeout(BucketWriter.java:686)
... 6 more
16/06/26 18:19:24 INFO hdfs.BucketWriter: Creating hdfs://bigdatastorm:8020/flume/data/FlumeData.1466936345776.tmp
16/06/26 18:19:38 INFO hdfs.BucketWriter: Closing idle bucketWriter hdfs://bigdatastorm:8020/flume/data/FlumeData.1466936345776.tmp at 1466936378715
16/06/26 18:19:38 INFO hdfs.BucketWriter: Closing hdfs://bigdatastorm:8020/flume/data/FlumeData.1466936345776.tmp
16/06/26 18:19:39 INFO hdfs.BucketWriter: Renaming hdfs://bigdatastorm:8020/flume/data/FlumeData.1466936345776.tmp to hdfs://bigdatastorm:8020/flume/data/FlumeData.1466936345776
16/06/26 18:19:39 INFO hdfs.HDFSEventSink: Writer callback called.
莫名奇妙的错误
16/06/26 22:55:52 INFO hdfs.HDFSEventSink: Writer callback called.
16/06/26 22:55:52 INFO hdfs.HDFSEventSink: Bucket was closed while trying to append, reinitializing bucket and writing event.
16/06/26 22:55:52 INFO hdfs.HDFSDataStream: Serializer = TEXT, UseRawLocalFileSystem = false
16/06/26 22:55:53 INFO hdfs.BucketWriter: Creating hdfs://mycluster/flume/data/16-06-26/FlumeData.1466952952985.tmp
16/06/26 22:55:56 INFO hdfs.DFSClient: Exception in createBlockOutputStream
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:529)
at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1526)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1328)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1281)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:526)
16/06/26 22:55:56 INFO hdfs.DFSClient: Abandoning BP-731760634-192.168.184.188-1463927117534:blk_1073742940_2145
16/06/26 22:55:56 INFO hdfs.DFSClient: Excluding datanode 192.168.184.188:50010
16/06/26 22:56:01 INFO hdfs.BucketWriter: Closing idle bucketWriter hdfs://mycluster/flume/data/16-06-26/FlumeData.1466952952985.tmp at 1466952961311
16/06/26 22:56:01 INFO hdfs.BucketWriter: Closing hdfs://mycluster/flume/data/16-06-26/FlumeData.1466952952985.tmp
16/06/26 22:56:01 INFO hdfs.BucketWriter: Renaming hdfs://mycluster/flume/data/16-06-26/FlumeData.1466952952985.tmp to hdfs://mycluster/flume/data/16-06-26/FlumeData.1466952952985
16/06/26 22:56:01 INFO hdfs.HDFSEventSink: Writer callback called.
16/06/26 23:12:10 ERROR source.AvroSource: Avro source r1: Unable to process event batch. Exception follows.
org.apache.flume.ChannelException: Unable to put batch on required channel: org.apache.flume.channel.MemoryChannel{name: c1}
at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:200)
at org.apache.flume.source.AvroSource.appendBatch(AvroSource.java:386)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:91)
at org.apache.avro.ipc.Responder.respond(Responder.java:151)
at org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:188)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:173)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:786)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:458)
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:439)
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:553)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.flume.ChannelFullException: Space for commit to queue couldn't be acquired. Sinks are likely not keeping up with sources, or the buffer size is too tight
at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:130)
at org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:192)
... 29 more
16/06/26 23:12:10 INFO ipc.NettyServer: [id: 0x83b4ecf4, /192.168.184.188:43082 :> /192.168.184.188:44444] DISCONNECTED
16/06/26 23:12:10 INFO ipc.NettyServer: [id: 0x83b4ecf4, /192.168.184.188:43082 :> /192.168.184.188:44444] UNBOUND
16/06/26 23:12:10 INFO ipc.NettyServer: [id: 0x83b4ecf4, /192.168.184.188:43082 :> /192.168.184.188:44444] CLOSED
16/06/26 23:12:10 INFO ipc.NettyServer: Connection to /192.168.184.188:43082 disconnected.
16/06/26 23:12:12 INFO hdfs.HDFSDataStream: Serializer = TEXT, UseRawLocalFileSystem = false
16/06/26 23:12:12 INFO hdfs.BucketWriter: Creating hdfs://mycluster/flume/data/16-06-26/FlumeData.1466953932153.tmp
16/06/26 23:12:15 INFO ipc.NettyServer: [id: 0xcab3190c, /192.168.184.188:43085 => /192.168.184.188:44444] OPEN
16/06/26 23:12:15 INFO ipc.NettyServer: [id: 0xcab3190c, /192.168.184.188:43085 => /192.168.184.188:44444] BOUND: /192.168.184.188:44444
16/06/26 23:12:15 INFO ipc.NettyServer: [id: 0xcab3190c, /192.168.184.188:43085 => /192.168.184.188:44444] CONNECTED: /192.168.184.188:43085
16/06/26 23:12:38 INFO hdfs.BucketWriter: Closing hdfs://mycluster/flume/data/16-06-26/FlumeData.1466953932153.tmp