大屏24数据提取执行execApache common exec包的相应使用总结

大屏24数据提取执行execApache common exec包的相应使用总结

Apache common exec包的相应使用总结

clamaa 2017-04-11 09:57:21 4972 收藏
分类专栏： apache common-exec
版权
 
最近在写一个Java工具，其中调用了各种SHELL命令，使用了Runtime.getRuntime().exec(command);这个方法。但是在我调用某个命令执行操作时，程序就卡在那了，但是其他操作却能够正常输出，经过了一番查找和摸索，终于明白了原来Java在执行命令时输出到某个Buffer里，这个Buffer是有容量限制的，如果满了一直没有人读取，就会一直等待，造成进程锁死的现象，知道这一点，我们就可以在执行的过程中新开启几个线程来不断地读取标准输出，以及错误输出：
 
final Process p = Runtime.getRuntime().exec(command);
new Thread(new Runnable() {
@Override
public void run() {
while (true){
try {
p.exitValue();
break;
} catch (Exception e){
showInfo(System.err,p.getErrorStream());
}
}
}
}).start();
new Thread(new Runnable() {
@Override
public void run() {
while (true){
try {
p.exitValue();
break;
} catch (Exception e){
showInfo(System.out,p.getInputStream());
}
}
}
}).start();
int exitValue = p.waitFor();
 
 
需要注意的是，在waitFor执行完成之前，错误输出和标准要分开处理。
 
Apache commons-exec提供一些常用的方法用来执行外部进程，Apache commons exec库提供了监视狗Watchdog来设监视进程的执行超时，同时也还实现了同步和异步功能，Apache commonsexec涉及到多线程，比如新启动一个进程，Java中需要再开三个线程来处理进程的三个数据流，分别是标准输入，标准输出和错误输出。
 
需要使用该功能需要引入commons-exec-1.3.jar包，目前最新的版本为1.3版本。
 
Apache commons-exec的官方网站： http://commons.apache.org/proper/commons-exec/
 
其中就有相应的示例Example，来详解如何使用该工具来执行shell命令，我们执行命令的相关代码：
 
final Long executeSubTaskId = subTaskExecuteContext.getSubTaskExecuteId();
final Long taskStatusId = subTaskExecuteContext.getTaskExecuteContext().getTaskExecuteId();

ByteArrayOutputStream outputStream =
new MzByteArrayOutputStream(executeSubTaskId, taskStatusNotifyCenter, true);
ByteArrayOutputStream errorStream =
new MzByteArrayOutputStream(executeSubTaskId, taskStatusNotifyCenter, false);
PumpStreamHandler streamHandler = new PumpStreamHandler(outputStream, errorStream);
taskThreadPoolExecutor.setStreamHandler(streamHandler);

CommandLine commandLine = new CommandLine(new File(executeShellPath));

final TaskProcessInfo taskProcessInfo = new TaskProcessInfo(subTaskExecuteContext.getTaskExecuteContext().getTaskId(),
taskStatusId,
executeSubTaskId);
ProcessManagerDestroyer processManagerDestroyer =
new ProcessManagerDestroyer(
taskProcessInfo, processInfoManager);
taskThreadPoolExecutor.setProcessDestroyer(processManagerDestroyer);
try {
taskThreadPoolExecutor.execute(commandLine, new DefaultExecuteResultHandler() {
@Override
public void onProcessComplete(int exitValue) {
super.onProcessComplete(exitValue);
LOG.info(String.format("Task Process info: %s succeed!", taskProcessInfo));
taskStatusNotifyCenter.notifyEventChanged(new TaskStatusEvent(TaskStatusEventType.TASK_FINISHED,
new TaskEventObject(subTaskExecuteContext.getTaskExecuteContext().getTaskId(),
subTaskExecuteContext.getTaskExecuteContext().getTaskExecuteId(),
subTaskExecuteContext.getSubTaskExecuteId())));
}

@Override
public void onProcessFailed(ExecuteException e) {
super.onProcessFailed(e);
LOG.error(e);
LOG.error(String.format("Task Process info: %s failed!", taskProcessInfo));
taskStatusNotifyCenter.notifyEventChanged(new TaskStatusEvent(TaskStatusEventType.TASK_FAILED,
new TaskEventObject(subTaskExecuteContext.getTaskExecuteContext().getTaskId(),
subTaskExecuteContext.getTaskExecuteContext().getTaskExecuteId(),
subTaskExecuteContext.getSubTaskExecuteId())));
}
});
} catch (IOException e) {
throw new BusinessException(e);
}
 
 
 
新建执行Process需要new两个ByteArrayOutputStream，一个用来记录标准输出流，一个用来记录错误输出流。为了及时清理ByteArrayOutputStream中的内容，可以选择性地将该输出流重写：
@Override
public synchronized void write(byte[] b, int off, int len) {
super.write(b, off, len);
writeTimes++;
writeLength += len;
if (writeLength >= MAX_WRITE_LENGTH || writeTimes >= MAX_WRITE_TIMES) {
updateStatus();
this.buf = new byte[32];
writeLength = 0;
writeTimes = 0;
}

}

@Override
public void flush() throws IOException {
super.flush();
updateStatus();
}
 
 
建立的ProcessManagerDestroyer用来任务创建或任务完成时，对任务的当前记录状态。
public class ProcessManagerDestroyer implements ProcessDestroyer {

private final ProcessInfoManager processInfoManager;

private final TaskProcessInfo taskProcessInfo;

public ProcessManagerDestroyer(TaskProcessInfo taskProcessInfo, ProcessInfoManager processInfoManager) {
this.taskProcessInfo = taskProcessInfo;
this.processInfoManager = processInfoManager;
}

@Override
public boolean add(Process process) {
processInfoManager.addProcess(taskProcessInfo, process);
return true;
}

@Override
public boolean remove(Process process) {
processInfoManager.removeProcess(taskProcessInfo);
return true;
}

@Override
public int size() {
return processInfoManager.taskCount();
}
 
 
在Destroyer中新建的Process可以保存，并在以后调用destroy方法将其kill掉：
 
process.destroy();
 
 
最后建立的DefaultExecuteResultHandler监听器用来在任务执行完成或出现错误时，提示对应的信息，并发送事件。
 
 
执行shell时遇到的问题，初步看来，没有执行的权限？
 
org.apache.commons.exec.ExecuteException: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "/Users/mazhiqiang/Downloads/1.sh" (in directory "."): error=13, Permission denied)
at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:205)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Cannot run program "/Users/xxx/Downloads/1.sh" (in directory "."): error=13, Permission denied
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1042)
at java.lang.Runtime.exec(Runtime.java:620)
at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:61)
at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:279)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:336)
at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
... 1 more
Caused by: java.io.IOException: error=13, Permission denied
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:185)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1023)
... 7 more
 
 
最终发现生成的shell文件没有加上可执行文件的executable属性，以及shell命令文件头：
 
file.executable()
!/bin/bash