目录
一、概述
前几天，Apache Log4j2 爆出远程代码执行漏洞，攻击者可利用该漏洞构造特殊的数据请求包，最终触发远程代码执行，对于JAVA项目来说，Log4j几乎是必备的组件，漏洞的爆出直接覆盖了一大堆java项目。
首先需要找出哪些项目可能会被本漏洞影响，才能完成漏洞修复，这便是个比较麻烦的问题，因为项目依赖中存在嵌套的关系，很难从表面分辨哪些项目存在log4j2依赖包。
二、具体实现
本方案通过gitlab api获取项目信息，对pom.xml依赖进行分析，产出依赖关系tree，可以很清晰的找出哪些项目依赖了哪些包，对项目的依赖关系tree进行分析和整理可以批量获取存在漏洞的项目。
1.产出依赖结果树
服务依赖：
<dependency>
<groupId>com.messners</groupId>
<artifactId>gitlab-api</artifactId>
<version>3.0.1</version>
</dependency>
<dependency>
<groupId>org.apache.maven.shared</groupId>
<artifactId>maven-invoker</artifactId>
<version>3.0.1</version>
</dependency>
<dependency>
<groupId>org.whitesource</groupId>
<artifactId>maven-dependency-tree-parser</artifactId>
<version>1.0.5</version>
</dependency>(1)gitlab项目信息获取
需要先获取一个管理员账号token，使用api拉取项目信息。
由于gitlab存在fork机制，故只需要获取具备group的project即可。
核心源码：
List<Project> projectList = new ArrayList<>();
List<Group> groupList = gitlabApiService.getAllGroups(gitlabHost, privateToken);
List<Integer> groupIdList = groupList.parallelStream().map(Group::getId).collect(Collectors.toList());
groupIdList.parallelStream().forEach(id -> {
try {
Group group = api.getGroupApi().getGroup(id);
List<Project> projects = group.getProjects();
projectList.addAll(projects);
} catch (GitLabApiException e) {
e.printStackTrace();
}
});
projectList即为gitlab上所有项目信息
(2) 分析pom.xml文件
解析项目文件，生成pom.xml文件写入本地
public boolean copyPomFileFromGitlab(GitLabApi api, Integer projectId, String projectName) throws Exception {
boolean rootPomExists = false;
log.info("解析项目{}文件开始", projectName);
List<TreeItem> itemList = api.getRepositoryApi().getTree(projectId);
if (!CollectionUtils.isEmpty(itemList)) {
for (TreeItem treeItem : itemList) {
if (treeItem.getType() == TreeItem.Type.BLOB && treeItem.getName().equals("pom.xml")) {
rootPomExists = true;
String outputPath=baseDir + "/" + projectName + "/pom.xml";
String fromPath="pom.xml";
outputFile(outputPath,fromPath,api,projectId);
} else if (treeItem.getType() == TreeItem.Type.TREE) {
List<TreeItem> treeItemList = api.getRepositoryApi().getTree(projectId, treeItem.getName(), "master");
for (TreeItem item : treeItemList) {
if (item.getType() == TreeItem.Type.BLOB && item.getName().equals("pom.xml")) {
rootPomExists = true;
String outputPath=baseDir + "/" + projectName + "/" + treeItem.getName() + "/pom.xml";
String fromPath="/" + treeItem.getName() + "/" + "pom.xml";
outputFile(outputPath,fromPath,api,projectId);
}
}
}
}
}
return rootPomExists;
}
(3)执行依赖分析，生成依赖结果Tree
获取项目关键信息，定义关键tree名：
if (copyPomFileFromGitlab(api, project.getId(), project.getName())) {
String prefix = project.getPathWithNamespace().replace("/","_")+"-";
String pomPath = String.format("%s/%s/pom.xml", baseDir, project.getName());
String outputFileTree = String.format("%s/%s/%stree.txt", baseDir, project.getName(),prefix);
//执行maven dependency
List<DependencyInfo> dependencyListOfJava = MavenUtil.getDependenncyList(pomPath, outputFileTree, mavenHomePath);
}
产出项目依赖关系tree，落到本机文件：
public static List<DependencyInfo> getDependenncyList(String pomPath, String outputFilePath, String mavenHomePath) {
List<DependencyInfo> dependencyInfoList = new ArrayList<>();
InvocationRequest request = new DefaultInvocationRequest();
request.setPomFile(new File(pomPath));
String commond = String.format("dependency:tree -DoutputFile=%s -DoutputType=text", outputFilePath);
request.setGoals(Collections.singletonList(commond));
Invoker invoker = new DefaultInvoker();
invoker.setMavenHome(new File(mavenHomePath));
try {
InvocationResult invocationResult = invoker.execute(request);
if (invocationResult.getExecutionException() == null) {
log.info("依赖树文件生成成功:{}", outputFilePath);
}
File f = new File(outputFilePath);
if (!f.exists()) {
f.getParentFile().mkdir();
f.createNewFile();
}
FileInputStream fis = new FileInputStream(f);
Reader reader = new BufferedReader(new InputStreamReader(fis));
InputType type = InputType.TEXT;
Parser parser = type.newParser();
Node node = parser.parse(reader);
LinkedList<Node> nodeLinkedList = new LinkedList<>();
getAllNodeList(node, nodeLinkedList);
dependencyInfoList = nodeLinkedList.parallelStream().map(n -> {
DependencyInfo dependencyInfo = new DependencyInfo();
dependencyInfo.setGroupId(n.getGroupId());
dependencyInfo.setArtifactId(n.getArtifactId());
dependencyInfo.setPackaging(n.getPackaging());
dependencyInfo.setVersion(n.getVersion());
dependencyInfo.setSourcePath("pom.xml");
return dependencyInfo;
}).collect(Collectors.toList());
} catch (Exception e) {
e.printStackTrace();
}
return dependencyInfoList;
}
public static void getAllNodeList(Node node, LinkedList<Node> nodeLinkedList) {
nodeLinkedList.add(node);
if (node.getChildNodes().size() != 0) {
node.getChildNodes().parallelStream().forEach(n -> getAllNodeList(n, nodeLinkedList));
}
}最终生成的文件结构信息：
打开tree文件，可以很清楚的看到有使用到log4j2相关依赖：
2.解析依赖结果树
实现方案：识别txt文件，获取存在字符串为"log4j-api:jar:2."的项目信息(具体查找哪个包可自行调整输入参数)
核心源码：
public static void writeLog4jInfo(String containsStr){
Map<String,String> treeMap = buildLog4jInfoByTree(baseDir,containsStr);
log.info("writeLog4jInfo-treeMap.size:"+treeMap.size());
FileUtil.write(RESULT_PATH,treeMap);
log.info("writeLog4jInfo-success!");
}
private static Map<String,String> buildLog4jInfoByTree(String path,String containsStr){
Map<String,String> relMap = new HashMap<>();
Map<String,String> map = getLog4J2Files(path,containsStr);
if(!CollectionUtils.isEmpty(map)){
map.keySet().stream().forEach(k->{
try {
String name = k.substring(k.lastIndexOf("/")+1);
name = name.substring(0,name.indexOf(TREE_SUFFIX)-1);
relMap.put(name,map.get(k).trim().replace("+-","")
.replace("|","").replace(" ","")
.replace("\\-",""));
} catch (Exception e) {
e.printStackTrace();
}
});
}
return relMap;
}
private static Map<String,String> getLog4J2Files(String path,String containsStr) {
Map<String,String> log4j2Files = new HashMap<>();
List<String> list = new ArrayList<>();
listTxt(path, list);
list.stream().forEach(k -> {
String findStr = findStringInFile(k, containsStr);
if (findStr!=null) {
log4j2Files.put(k,findStr);
}
});
return log4j2Files;
}
private static void listTxt(String path, List<String> list) {
if (list == null) {
list = new ArrayList<>();
}
File file = new File(path);
File[] array = file.listFiles();
for (int i = 0; i < array.length; i++) {
if (array[i].isFile()) {
if (array[i].getName().endsWith("txt")) {
list.add(array[i].getPath());
}
} else if (array[i].isDirectory()) {
listTxt(array[i].getPath(), list);
}
}
}
private static String findStringInFile(String path, String containsStr) {
InputStreamReader read = null;
BufferedReader bufferedReader = null;
String result = null;
try {
File file = new File(path);
read = new InputStreamReader(new FileInputStream(file), "UTF-8");
bufferedReader = new BufferedReader(read);
String line = null;
while ((line = bufferedReader.readLine()) != null) {
if (line.contains(containsStr)) {
result = line.trim();
break;
}
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (bufferedReader != null) {
bufferedReader.close();
}
if (read != null) {
read.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
return result;
}最终效果展示(自行处理下log4j2-info.txt文件)：
三、附言
1.项目指导人：咖啡男孩之SRE之路_牛麦康纳_CSDN博客-互联网,Spinnaker,Python领域博主
此技术方案由项目指导人指导，博主为部分代码实现者
2.代码不够规范，主要以实现功能为主
3.目前只适配使用maven pom.xml构建的java项目，gradle也可以以同样方式实现
4.gitlab需要根据版本适配对应的api
对您有作用，还请点个赞
404
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
