刚刚的项目中，有大量的场景需要Java 进程调用 Linux的bash shell 脚本实现相关功能。
从之前的项目中拷贝的相关模块和网上的例子来看，有个别的“陷阱”造成调用shell 脚本在某些特殊的场景下，有一些奇奇怪怪的bug。
大家且听我一一道来。
先看看网上搜索到的例子：
package someTest;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
public class ShellTest {
public static void main(String[] args) {
InputStreamReader stdISR = null;
InputStreamReader errISR = null;
Process process = null;
String command = "/home/Lance/workspace/someTest/testbash.sh";
try {
process = Runtime.getRuntime().exec(command);
int exitValue = process.waitFor();
String line = null;
stdISR = new InputStreamReader(process.getInputStream());
BufferedReader stdBR = new BufferedReader(stdISR);
while ((line = stdBR.readLine()) != null) {
System.out.println("STD line:" + line);
}
errISR = new InputStreamReader(process.getErrorStream());
BufferedReader errBR = new BufferedReader(errISR);
while ((line = errBR.readLine()) != null) {
System.out.println("ERR line:" + line);
}
} catch (IOException | InterruptedException e) {
e.printStackTrace();
} finally {
try {
if (stdISR != null) {
stdISR.close();
}
if (errISR != null) {
errISR.close();
}
if (process != null) {
process.destroy();
}
} catch (IOException e) {
System.out.println("正式执行命令：" + command + "有IO异常");
}
}
}
}
#!/bin/bash
echo pwd
输出结果为：
STD line:/home/Lance/workspace/someTest
Java在执行Runtime.getRuntime().exec(command)之后，Linux会创建一个进程，该进程与JVM进程建立三个管道连接，标准输入流、标准输出流、标准错误流。
上述代码，依次读取标准输出流和标准错误流，在shell给出“退出信号”后，做了相应的清理工作。
对于一般场景来说，这段代码可以凑合用了。但是，在实际场景中，会有以下几个“陷阱”。
一. 当标准输出流或标准错误流非常庞大的时候，会出现调用waitFor方法卡死的bug。
真实的环境中，当标准输出在10000行左右的时候，就会出现卡死的情况。
原因分析：假设linux进程不断向标准输出流和标准错误流写数据，而JVM却不读取，数据会暂存在linux缓存区，当缓存区存满之后导致该进程无法继续写数据，会僵死，导致java进程会卡死在waitFor()处，永远无法结束。
解决方式：由于标准输出和错误输出都会向Linux缓存区写数据，而脚本如何输出这两种流是Java端不能确定的。为了不让shell脚本的子进程卡死，这两种输出需要分别读取，而且不能互相影响。所以必须新开两个线程来进行读取。
我开始的实现如下：
package someTest;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.LinkedList;
import java.util.List;
public class CommandStreamGobbler extends Thread {
private InputStream is;
private String command;
private String prefix = "";
private boolean readFinish = false;
private boolean ready = false;
private List<String> infoList = new LinkedList<String>();
CommandStreamGobbler(InputStream is, String command, String prefix) {
this.is = is;
this.command = command;
this.prefix = prefix;
}
public void run() {
InputStreamReader isr = null;
try {
isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line = null;
ready = true;
while ((line = br.readLine()) != null) {
infoList.add(line);
System.out.println(prefix + " line: " + line);
}
} catch (IOException ioe) {
System.out.println("正式执行命令：" + command + "有IO异常");
} finally {
try {
if (isr != null) {
isr.close();
}
} catch (IOException ioe) {
System.out.println("正式执行命令：" + command + "有IO异常");
}
readFinish = true;
}
}
public InputStream getIs() {
return is;
}
public String getCommand() {
return command;
}
public boolean isReadFinish() {
return readFinish;
}
public boolean isReady() {
return ready;
}
public List<String> getInfoList() {
return infoList;
}
}
package someTest;
import java.io.IOException;
import java.io.InputStreamReader;
public class ShellTest {
public static void main(String[] args) {
InputStreamReader stdISR = null;
InputStreamReader errISR = null;
Process process = null;
String command = "/home/Lance/workspace/someTest/testbash.sh";
try {
process = Runtime.getRuntime().exec(command);
CommandStreamGobbler errorGobbler = new CommandStreamGobbler(process.getErrorStream(), command, "ERR");
CommandStreamGobbler outputGobbler = new CommandStreamGobbler(process.getInputStream(), command, "STD");
errorGobbler.start();
// 必须先等待错误输出ready再建立标准输出
while (!errorGobbler.isReady()) {
Thread.sleep(10);
}
outputGobbler.start();
while (!outputGobbler.isReady()) {
Thread.sleep(10);
}
int exitValue = process.waitFor();
} catch (IOException | InterruptedException e) {
e.printStackTrace();
} finally {
try {
if (stdISR != null) {
stdISR.close();
}
if (errISR != null) {
errISR.close();
}
if (process != null) {
process.destroy();
}
} catch (IOException e) {
System.out.println("正式执行命令：" + command + "有IO异常");
}
}
}
}
到此为止，解决了Java卡死shell脚本的情况。再说说，第二种可能。
二. 由于shell脚本的编写问题，当其自身出现僵死的情况，上述代码出现Java代码被僵死的Shell脚本阻塞住的情况。
原因分析：由于shell脚本也是人写的，难免会出现失误。在Java调用shell脚本时，无论是Debug场景还是生产环境，都发生过shell脚本意外僵死反过来卡死Java相关线程的情况。典型的表现为：shell脚本长时间运行，标准输出和错误输出没有任何输出（包括结束符），操作系统显示shell脚本在正常运行或僵死，没有退出信号。
解决方式：上述代码中，至少有三处会导致线程阻塞，包括标准输出和错误输出这线程的BufferedReader的readline方法，以及Process的waitFor方法。解决这个问题的核心有两个，1.避免任何Java线程被阻塞住，因为一旦被IO阻塞住，线程将处于内核态，主线程没有任何办法强制结束相关子线程。2.添加一个简单的超时机制，超时后回收相应的线程资源，并结束调用过程。
演示代码中，我改写了testshell.sh，写一个没有任何输出的死循环模拟shell卡死的情况。
#!/bin/bash
while true;do
a=1
sleep 0.1
done
package someTest;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.LinkedList;
import java.util.List;
public class CommandStreamGobbler extends Thread {
private InputStream is;
private String command;
private String prefix = "";
private boolean readFinish = false;
private boolean ready = false;
// 命令执行结果,0:执行中 1:超时 2:执行完成
private int commandResult = 0;
private List<String> infoList = new LinkedList<String>();
CommandStreamGobbler(InputStream is, String command, String prefix) {
this.is = is;
this.command = command;
this.prefix = prefix;
}
public void run() {
InputStreamReader isr = null;
BufferedReader br = null;
try {
isr = new InputStreamReader(is);
br = new BufferedReader(isr);
String line = null;
ready = true;
while (commandResult != 1) {
if (br.ready() || commandResult == 2) {
if ((line = br.readLine()) != null) {
infoList.add(line);
} else {
break;
}
} else {
Thread.sleep(100);
}
}
} catch (IOException | InterruptedException ioe) {
System.out.println("正式执行命令：" + command + "有IO异常");
} finally {
try {
if (br != null) {
br.close();
}
if (isr != null) {
isr.close();
}
} catch (IOException ioe) {
System.out.println("正式执行命令：" + command + "有IO异常");
}
readFinish = true;
}
}
public InputStream getIs() {
return is;
}
public String getCommand() {
return command;
}
public boolean isReadFinish() {
return readFinish;
}
public boolean isReady() {
return ready;
}
public List<String> getInfoList() {
return infoList;
}
public void setTimeout(int timeout) {
this.commandResult = timeout;
}
}
package someTest;
public class CommandWaitForThread extends Thread {
private Process process;
private boolean finish = false;
private int exitValue = -1;
public CommandWaitForThread(Process process) {
this.process = process;
}
public void run() {
try {
this.exitValue = process.waitFor();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
finish = true;
}
}
public boolean isFinish() {
return finish;
}
public void setFinish(boolean finish) {
this.finish = finish;
}
public int getExitValue() {
return exitValue;
}
}
package someTest;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.Date;
public class ShellTest {
public static void main(String[] args) {
InputStreamReader stdISR = null;
InputStreamReader errISR = null;
Process process = null;
String command = "/home/Lance/workspace/someTest/testbash.sh";
long timeout = 10 * 1000;
try {
process = Runtime.getRuntime().exec(command);
CommandStreamGobbler errorGobbler = new CommandStreamGobbler(process.getErrorStream(), command, "ERR");
CommandStreamGobbler outputGobbler = new CommandStreamGobbler(process.getInputStream(), command, "STD");
errorGobbler.start();
// 必须先等待错误输出ready再建立标准输出
while (!errorGobbler.isReady()) {
Thread.sleep(10);
}
outputGobbler.start();
while (!outputGobbler.isReady()) {
Thread.sleep(10);
}
CommandWaitForThread commandThread = new CommandWaitForThread(process);
commandThread.start();
long commandTime = new Date().getTime();
long nowTime = new Date().getTime();
boolean timeoutFlag = false;
while (!commandIsFinish(commandThread, errorGobbler, outputGobbler)) {
if (nowTime - commandTime > timeout) {
timeoutFlag = true;
break;
} else {
Thread.sleep(100);
nowTime = new Date().getTime();
}
}
if (timeoutFlag) {
// 命令超时
errorGobbler.setTimeout(1);
outputGobbler.setTimeout(1);
System.out.println("正式执行命令：" + command + "超时");
}else {
// 命令执行完成
errorGobbler.setTimeout(2);
outputGobbler.setTimeout(2);
}
while (true) {
if (errorGobbler.isReadFinish() && outputGobbler.isReadFinish()) {
break;
}
Thread.sleep(10);
}
} catch (IOException | InterruptedException e) {
e.printStackTrace();
} finally {
if (process != null) {
process.destroy();
}
}
}
private boolean commandIsFinish(CommandWaitForThread commandThread, CommandStreamGobbler errorGobbler, CommandStreamGobbler outputGobbler) {
if (commandThread != null) {
return commandThread.isFinish();
} else {
return (errorGobbler.isReadFinish() && outputGobbler.isReadFinish());
}
}
}
在以上的代码中，为了防止线程被阻塞，要点如下：
- 在CommandStreamGobbler里，bufferedReader在readLine()之前，先用ready()看一下当前缓冲区的情况，请特别注意ready()描述，这个方法是非阻塞的。
boolean java.io.BufferedReader.ready() throws IOException
Tells whether this stream is ready to be read. A buffered character stream is ready if the buffer is not empty, or if the underlying character stream is ready.
Returns:
True if the next read() is guaranteed not to block for input, false otherwise. Note that returning false does not guarantee that the next read will block.
2.在一个新线程commandThread中，调用process对象的waitFor()从而避免主线程卡死，主线程的最后会执行finally块中的process.destory()保证commandThread正常退出。
以上的两点改进，保证了Java在调用shell脚本过程互不被对方卡死的机制。
三.在执行shell脚本过程中，可能会添加参数，通常在终端中，我们使用“ ”（空格）把参数隔开。
为了区分空格是作为参数分隔符，还是参数的一部分。调用exec方法有特别的注意事项。
String command = “/home/Lance/workspace/someTest/testbash.sh ‘hello world’”;
process = Runtime.getRuntime().exec(command);
等价于
List commandList = new LinkedList();
commandList.add("/home/Lance/workspace/someTest/testbash.sh");
commandList.add(“hello world”);
String[] commands = new String[commandList.size()];
for (int i = 0; i < commandList.size(); i++) {
commands[i] = commandList.get(i);
}
process = Runtime.getRuntime().exec(commands);
好了，今天介绍到这里。
祝默默努力的人儿都成功！！！
8336
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
