Apache common exec包的相应使用总结

 

最近在写一个Java工具，其中调用了各种SHELL命令，使用了Runtime.getRuntime().exec(command);这个方法。但是在我调用某个命令执行操作时，程序就卡在那了，但是其他操作却能够正常输出，经过了一番查找和摸索，终于明白了原来Java在执行命令时输出到某个Buffer里，这个Buffer是有容量限制的，如果满了一直没有人读取，就会一直等待，造成进程锁死的现象，知道这一点，我们就可以在执行的过程中新开启几个线程来不断地读取标准输出，以及错误输出：

 

final Process p = Runtime.getRuntime().exec(command);
new Thread(new Runnable() {
     @Override
     public void run() {
          while (true){
               try {
                    p.exitValue();
                    break;
               } catch (Exception e){
                    showInfo(System.err,p.getErrorStream());
               }
          }
     }
}).start();
new Thread(new Runnable() {
     @Override
     public void run() {
          while (true){
               try {
                    p.exitValue();
                    break;
               } catch (Exception e){
                    showInfo(System.out,p.getInputStream());
               }
          }
     }
}).start();
int exitValue = p.waitFor();

 

 

需要注意的是，在waitFor执行完成之前，错误输出和标准要分开处理。

 

Apache commons-exec提供一些常用的方法用来执行外部进程，Apache commons exec库提供了监视狗Watchdog来设监视进程的执行超时，同时也还实现了同步和异步功能，Apache commonsexec涉及到多线程，比如新启动一个进程，Java中需要再开三个线程来处理进程的三个数据流，分别是标准输入，标准输出和错误输出。

 

需要使用该功能需要引入commons-exec-1.3.jar包，目前最新的版本为1.3版本。

 

Apache commons-exec的官方网站：http://commons.apache.org/proper/commons-exec/

 

其中就有相应的示例Example，来详解如何使用该工具来执行shell命令，我们执行命令的相关代码：

 

final Long executeSubTaskId = subTaskExecuteContext.getSubTaskExecuteId();
        final Long taskStatusId = subTaskExecuteContext.getTaskExecuteContext().getTaskExecuteId();

        ByteArrayOutputStream outputStream =
                new MzByteArrayOutputStream(executeSubTaskId, taskStatusNotifyCenter, true);
        ByteArrayOutputStream errorStream =
                new MzByteArrayOutputStream(executeSubTaskId, taskStatusNotifyCenter, false);
        PumpStreamHandler streamHandler = new PumpStreamHandler(outputStream, errorStream);
        taskThreadPoolExecutor.setStreamHandler(streamHandler);

        CommandLine commandLine = new CommandLine(new File(executeShellPath));

        final TaskProcessInfo taskProcessInfo = new TaskProcessInfo(subTaskExecuteContext.getTaskExecuteContext().getTaskId(),
                taskStatusId,
                executeSubTaskId);
        ProcessManagerDestroyer processManagerDestroyer =
                new ProcessManagerDestroyer(
                        taskProcessInfo, processInfoManager);
        taskThreadPoolExecutor.setProcessDestroyer(processManagerDestroyer);
        try {
            taskThreadPoolExecutor.execute(commandLine, new DefaultExecuteResultHandler() {
                @Override
                public void onProcessComplete(int exitValue) {
                    super.onProcessComplete(exitValue);
                    LOG.info(String.format("Task Process info: %s succeed!", taskProcessInfo));
                    taskStatusNotifyCenter.notifyEventChanged(new TaskStatusEvent(TaskStatusEventType.TASK_FINISHED,
                            new TaskEventObject(subTaskExecuteContext.getTaskExecuteContext().getTaskId(),
                                    subTaskExecuteContext.getTaskExecuteContext().getTaskExecuteId(),
                                    subTaskExecuteContext.getSubTaskExecuteId())));
                }

                @Override
                public void onProcessFailed(ExecuteException e) {
                    super.onProcessFailed(e);
                    LOG.error(e);
                    LOG.error(String.format("Task Process info: %s failed!", taskProcessInfo));
                    taskStatusNotifyCenter.notifyEventChanged(new TaskStatusEvent(TaskStatusEventType.TASK_FAILED,
                            new TaskEventObject(subTaskExecuteContext.getTaskExecuteContext().getTaskId(),
                                    subTaskExecuteContext.getTaskExecuteContext().getTaskExecuteId(),
                                    subTaskExecuteContext.getSubTaskExecuteId())));
                }
            });
        } catch (IOException e) {
            throw new BusinessException(e);
        }

 

 

 

新建执行Process需要new两个ByteArrayOutputStream，一个用来记录标准输出流，一个用来记录错误输出流。为了及时清理ByteArrayOutputStream中的内容，可以选择性地将该输出流重写：

@Override
public synchronized void write(byte[] b, int off, int len) {
    super.write(b, off, len);
    writeTimes++;
    writeLength += len;
    if (writeLength >= MAX_WRITE_LENGTH || writeTimes >= MAX_WRITE_TIMES) {
        updateStatus();
        this.buf = new byte[32];
        writeLength = 0;
        writeTimes = 0;
    }

}

@Override
public void flush() throws IOException {
    super.flush();
    updateStatus();
}

 

 

建立的ProcessManagerDestroyer用来任务创建或任务完成时，对任务的当前记录状态。

public class ProcessManagerDestroyer implements ProcessDestroyer {

    private final ProcessInfoManager processInfoManager;

    private final TaskProcessInfo taskProcessInfo;

    public ProcessManagerDestroyer(TaskProcessInfo taskProcessInfo, ProcessInfoManager processInfoManager) {
        this.taskProcessInfo = taskProcessInfo;
        this.processInfoManager = processInfoManager;
    }

    @Override
    public boolean add(Process process) {
        processInfoManager.addProcess(taskProcessInfo, process);
        return true;
    }

    @Override
    public boolean remove(Process process) {
        processInfoManager.removeProcess(taskProcessInfo);
        return true;
    }

    @Override
    public int size() {
        return processInfoManager.taskCount();
    }

 

 

在Destroyer中新建的Process可以保存，并在以后调用destroy方法将其kill掉：

 

process.destroy();

 

 

最后建立的DefaultExecuteResultHandler监听器用来在任务执行完成或出现错误时，提示对应的信息，并发送事件。

 

 

执行shell时遇到的问题，初步看来，没有执行的权限？

 

org.apache.commons.exec.ExecuteException: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "/Users/mazhiqiang/Downloads/1.sh" (in directory "."): error=13, Permission denied)
     at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:205)
     at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Cannot run program "/Users/xxx/Downloads/1.sh" (in directory "."): error=13, Permission denied
     at java.lang.ProcessBuilder.start(ProcessBuilder.java:1042)
     at java.lang.Runtime.exec(Runtime.java:620)
     at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:61)
     at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:279)
     at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:336)
     at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
     at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
     ... 1 more
Caused by: java.io.IOException: error=13, Permission denied
     at java.lang.UNIXProcess.forkAndExec(Native Method)
     at java.lang.UNIXProcess.<init>(UNIXProcess.java:185)
     at java.lang.ProcessImpl.start(ProcessImpl.java:134)
     at java.lang.ProcessBuilder.start(ProcessBuilder.java:1023)
     ... 7 more

 

 

最终发现生成的shell文件没有加上可执行文件的executable属性，以及shell命令文件头：

 

file.executable()
!/bin/bash

 

 

应用过程中后面还有很多坑，等着我们去填......