SGFuzz配置
SGFuzz的git地址SGFuzz
其中打开live555的示例，使用docker配置回容易一些，直接：
sudo docker build . -t live555
sudo docker run -it --privileged live555 /bin/bash
cd experiments/live555-sgfuzz/testProgs/
ASAN_OPTIONS=alloc_dealloc_mismatch=0 ./testOnDemandRTSPServer -close_fd_mask=3 -detect_leaks=0 -dict=${WORKDIR}/rtsp.dict -only_ascii=1 ${WORKDIR}/in-rtsp/
其中第一行构建了一个名为live555的镜像，第二行以特权启动该容器-it是docker的参数，在新容器内指定一个终端，以及允许队容器内的标准输入stdin进行交互。
进入容器后，切换到指定的目录：
最后的一行的命令是进行测试，其中，不同参数的含义是：
- ASAN_OPTIONS=alloc_dealloc_mismatch=0 为ASAN的错误检查选项关闭情况，关闭分配 API 与解除分配 API 之间不匹配检查。
- -close_fd_mask=3 libfuzzer的参数
可以看到取值为3禁用了输出，如果去掉盖参数，fuzz过程中会出现：
不知道原因 - -detect_leaks=0 libfuzzer的参数如果为1（默认值），并且启用了LeakSanitizer，则尝试在模糊处理期间检测内存泄漏（即不仅在关闭时）
- -dict=${WORKDIR}/rtsp.dict libfuzzer的参数，提供字典：
- -only_ascii=1 libfuzzer的参数，指定输入必须为ASCII格式
- ${WORKDIR}/in-rtsp/初始种子语料库目录
Libfuzzer文档
Libfuzzer git使用说明
运行后，遇到crash会停止，并把crash记录下来：
./测试程序 crash-XXX即可复现概crash，但是仿佛sgfuzz的crash都是有状态的，重新执行最后这一个输入，无事发生，不会造成崩溃。
profuzzbench配置
首先把路径导入
cd profuzzbench
export PFBENCH=$(pwd)
export PATH=$PATH:$PFBENCH/scripts/execution:$PFBENCH/scripts/analysis
在启动aflnet时要添加-m none 参数，否则模糊器无法启动
profuzzbench中的命令：
cd $PFBENCH
mkdir results-lightftp
profuzzbench_exec_common.sh lightftp 4 results-lightftp aflnet out-lightftp-aflnet "-P FTP -D 10000 -q 3 -s 3 -E -K" 3600 5 &
profuzzbench_exec_common.sh lightftp 4 results-lightftp aflnwe out-lightftp-aflnwe "-D 10000 -K" 3600 5
正确的：
cd $PFBENCH
mkdir results-lightftp
profuzzbench_exec_common.sh lightftp 4 results-lightftp aflnet out-lightftp-aflnet "-m none -P FTP -D 10000 -q 3 -s 3 -E -K" 3600 5 &
profuzzbench_exec_common.sh lightftp 4 results-lightftp aflnwe out-lightftp-aflnwe "-m none -D 10000 -K" 3600 5
profuzzbench地址，docker配置很简单，仍以live555为例，dockerfile中有几条信息解释：
patch -p1 < $WORKDIR/fuzzing.patch
#补丁，fuzzingpatch是已经生成的补丁文件，其中-p1参数是从当前目录的上一级目录开始把补丁中对应的文件修改，因为补丁中不止有一个文件，而是某一目录下的所有代码变化都写入到了这个补丁当中
对gcov也添加了对应的补丁，主要是添加exit，这样在程序非正常终止的时候，仍然可以可以生成覆盖率信息。
CFLAGES是gcov的参数，profuzzbench目前支持的模糊器都是afl一族，所以覆盖统计都是gcov，但是gcov在正常调用exit退出程序时，才会统计覆盖数据，没有exit不会统计，所以对其进行了部分修改。
patch -p1 < $WORKDIR/gcov.patch && \
./genMakefiles linux && \
make CFLAGS="" CPPFLAGS="-fprofile-arcs -ftest-coverage" CXXFLAGS="-fprofile-arcs -ftest-coverage" LDFLAGS="-fprofile-arcs -ftest-coverage" clean
运行
采用下面的命令运行
profuzzbench_exec_common.sh lightftp 4 results-lightftp aflnet out-lightftp-aflnet "-P FTP -D 10000 -q 3 -s 3 -E -K" 3600 5 ``&
参数含义：
1st argument (DOCIMAGE) : name of the docker image
2nd argument (RUNS) : number of runs, one isolated Docker container is spawned for each run可以同时运行多个相同实验
3rd argument (SAVETO) : path to a folder keeping the results
4th argument (FUZZER) : fuzzer name (e.g., aflnet) – this name must match the name of the fuzzer folder inside the Docker container (e.g., /home/ubuntu/aflnet)stateafl是后面添加的，要单独再补个dockerfile
5th argument (OUTDIR) : name of the output folder created inside the docker container
6th argument (OPTIONS) : all options needed for fuzzing in addition to the standard options written in the target-specific run.sh script针对每个模糊器单独的参数
7th argument (TIMEOUT) : time for fuzzing in seconds 运行时间，以秒计数
8th argument (SKIPCOUNT): used for calculating coverage over time. e.g., SKIPCOUNT=5 means we run gcovr after every 5 test cases because gcovr takes time and we do not want to run it after every single test case规定每几个测试用例后统计覆盖数据
profuzzbench脚本：
#!/bin/bash
DOCIMAGE=$1 #name of the docker image
RUNS=$2 #number of runs
SAVETO=$3 #path to folder keeping the results
FUZZER=$4 #fuzzer name (e.g., aflnet) -- this name must match the name of the fuzzer folder inside the Docker container
OUTDIR=$5 #name of the output folder created inside the docker container
OPTIONS=$6 #all configured options for fuzzing
TIMEOUT=$7 #time for fuzzing
SKIPCOUNT=$8 #used for calculating coverage over time. e.g., SKIPCOUNT=5 means we run gcovr after every 5 test cases
DELETE=$9
WORKDIR="/home/ubuntu/experiments"
#keep all container ids
cids=()
#create one container for each run 调用run.sh脚本运行
for i in $(seq 1 $RUNS); do
id=$(docker run --cpus=1 -d -it $DOCIMAGE /bin/bash -c "cd ${WORKDIR} && run ${FUZZER} ${OUTDIR} '${OPTIONS}' ${TIMEOUT} ${SKIPCOUNT}")
cids+=(${id::12}) #store only the first 12 characters of a container ID
done
dlist="" #docker list
for id in ${cids[@]}; do
dlist+=" ${id}"
done
#wait until all these dockers are stopped
printf "\n${FUZZER^^}: Fuzzing in progress ..."
printf "\n${FUZZER^^}: Waiting for the following containers to stop: ${dlist}"
docker wait ${dlist} > /dev/null
wait
#collect the fuzzing results from the containers
printf "\n${FUZZER^^}: Collecting results and save them to ${SAVETO}"
index=1
for id in ${cids[@]}; do
printf "\n${FUZZER^^}: Collecting results from container ${id}"
#将容器中运行结果的文件复制到本地
docker cp ${id}:/home/ubuntu/experiments/${OUTDIR}.tar.gz ${SAVETO}/${OUTDIR}_${index}.tar.gz > /dev/null
if [ ! -z $DELETE ]; then
printf "\nDeleting ${id}"
docker rm ${id} # Remove container now that we don't need it
fi
index=$((index+1))
done
printf "\n${FUZZER^^}: I am done!\n"
run.sh脚本
FUZZER=$1 #fuzzer name (e.g., aflnet) -- this name must match the name of the fuzzer folder inside the Docker container
OUTDIR=$2 #name of the output folder
OPTIONS=$3 #all configured options -- to make it flexible, we only fix some options (e.g., -i, -o, -N) in this script
TIMEOUT=$4 #time for fuzzing
SKIPCOUNT=$5 #used for calculating cov over time. e.g., SKIPCOUNT=5 means we run gcovr after every 5 test cases
strstr() {
[ "${1#*$2*}" = "$1" ] && return 1
return 0
}
#Commands for afl-based fuzzers (e.g., aflnet, aflnwe)
if $(strstr $FUZZER "afl"); then
TARGET_DIR=${TARGET_DIR:-"live555"}
INPUTS=${WORKDIR}/in-rtsp
# Run fuzzer-specific commands (if any)执行特殊的命令，比如run-stateafl.sh
if [ -e ${WORKDIR}/run-${FUZZER} ]; then
source ${WORKDIR}/run-${FUZZER}
fi
#Step-1. Do Fuzzing
#Move to fuzzing folder timeout规定了进程执行的时间
cd $WORKDIR/${TARGET_DIR}/testProgs
timeout -k 0 --preserve-status $TIMEOUT /home/ubuntu/${FUZZER}/afl-fuzz -d -i ${INPUTS} -x ${WORKDIR}/rtsp.dict -o $OUTDIR -N tcp://127.0.0.1/8554 $OPTIONS ./testOnDemandRTSPServer 8554
STATUS=$?
#Step-2. Collect code coverage over time
#Move to gcov folder
cd $WORKDIR/live555-cov/testProgs
#The last argument passed to cov_script should be 0 if the fuzzer is afl/nwe and it should be 1 if the fuzzer is based on aflnet
#0: the test case is a concatenated message sequence -- there is no message boundary
#1: the test case is a structured file keeping several request messages
if [ $FUZZER == "aflnwe" ]; then
cov_script ${WORKDIR}/${TARGET_DIR}/testProgs/${OUTDIR}/ 8554 ${SKIPCOUNT} ${WORKDIR}/${TARGET_DIR}/testProgs/${OUTDIR}/cov_over_time.csv 0
else
cov_script ${WORKDIR}/${TARGET_DIR}/testProgs/${OUTDIR}/ 8554 ${SKIPCOUNT} ${WORKDIR}/${TARGET_DIR}/testProgs/${OUTDIR}/cov_over_time.csv 1
fi
cd $WORKDIR/live555-cov
#copy .hh files since gcovr could not detect them
for f in BasicUsageEnvironment liveMedia groupsock UsageEnvironment; do
echo $f
cp $f/include/*.hh $f/
done
cd testProgs
gcovr -r .. --html --html-details -o index.html
mkdir ${WORKDIR}/${TARGET_DIR}/testProgs/${OUTDIR}/cov_html/
cp *.html ${WORKDIR}/live555/testProgs/${OUTDIR}/cov_html/
#Step-3. Save the result to the ${WORKDIR} folder
#Tar all results to a file
cd ${WORKDIR}/${TARGET_DIR}/testProgs
tar -zcvf ${WORKDIR}/${OUTDIR}.tar.gz ${OUTDIR}
exit $STATUS
fi
将SGFuzz集成到profuzzbench当中
1.将SGFuzz自身的Dockerfile修改一下
#在已构建的profuzzbench的镜像当中，构建SGFuzz
FROM live555-yang-stateafl
ENV HFND_TCP_PORT=8554
USER root
RUN chmod 777 /usr/lib
# Enable LLVM-10.0 in ubuntu 16.04
RUN apt-get update \
&& apt-get install -y --no-install-recommends curl ca-certificates gnupg2 apt-transport-https software-properties-common \
&& echo 'deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-10 main' >> /etc/apt/sources.list.d/llvm10.list \
&& echo 'deb-src http://apt.llvm.org/xenial/ llvm-toolchain-xenial-10 main' >> /etc/apt/sources.list.d/llvm10.list \
&& curl https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add -
# Install common dependencies
RUN apt-get -y update && \
apt-get -y install sudo \
apt-utils \
build-essential \
openssl \
clang \
graphviz-dev \
git \
wget \
libgnutls28-dev \
python3-pip \
nano \
net-tools \
vim \
python-jinja2 \
gdb \
psmisc
# Add a new user ubuntu, pass: ubuntu
#RUN groupadd ubuntu && \
# useradd -rm -d /home/ubuntu -s /bin/bash -g ubuntu -G sudo -u 1000 ubuntu -p "$(openssl passwd -1 ubuntu)"
# Use ubuntu as default username
USER ubuntu
WORKDIR /home/ubuntu
RUN git clone https://ghproxy.com/https://github.com/google/honggfuzz.git && \
cd honggfuzz && \
git checkout 6f89ccc9c43c6c1d9f938c81a47b72cd5ada61ba && \
#sed -i "s/-D_HF_ARCH_\${ARCH}//g" Makefile && \
#sed -i "s/INFO;/ERROR;/g" libhfcommon/log.c && \
CC=clang-10 CFLAGS="-g -fsanitize=fuzzer-no-link -fsanitize=address" make libhfcommon/libhfcommon.a && \
CC=clang-10 CFLAGS="-g -fsanitize=fuzzer-no-link -fsanitize=address -DHFND_RECVTIME=1" make libhfnetdriver/libhfnetdriver.a && \
mv libhfcommon/libhfcommon.a /usr/lib/libhfcommon.a && \
mv libhfnetdriver/libhfnetdriver.a /usr/lib/libhfnetdriver.a
# Set up SGFuzz
RUN git clone https://ghproxy.com/https://github.com/bajinsheng/SGFuzz.git && \
cd SGFuzz && \
./build.sh && \
cp libsfuzzer.a /usr/lib/libsFuzzer.a
ENV WORKDIR="/home/ubuntu/experiments"
COPY --chown=ubuntu:ubuntu fuzzing.patch ${WORKDIR}/fuzzing.patch
COPY --chown=ubuntu:ubuntu in-rtsp ${WORKDIR}/in-rtsp
COPY --chown=ubuntu:ubuntu rtsp.dict ${WORKDIR}/rtsp.dict
COPY --chown=ubuntu:ubuntu live.2021.08.13.tar.gz ${WORKDIR}/
COPY --chown=ubuntu:ubuntu sample_media_sources ${WORKDIR}/sample_media_sources
COPY --chown=ubuntu:ubuntu blocked_variables.txt ${WORKDIR}/blocked_variables.txt
# Download Live555
RUN cd $WORKDIR && \
tar -zxvf live.2021.08.13.tar.gz && \
chmod 777 -R live
# Download and compile Live555 for SGFuzz
RUN cd $WORKDIR && \
cp -r live live555-sgfuzz && \
cd live555-sgfuzz && \
patch -p1 < $WORKDIR/fuzzing.patch && \
#使用"s/int main(/extern \"C\"取代 int HonggfuzzNetDriver_main(/g" ，libfuzzer需要测试的入口函数
sed -i "s/int main(/extern \"C\" int HonggfuzzNetDriver_main(/g" testProgs/testOnDemandRTSPServer.cpp && \
python3 /home/ubuntu/SGFuzz/sanitizer/State_machine_instrument.py ${WORKDIR}/live555-sgfuzz/ -b ${WORKDIR}/blocked_variables.txt && \
./genMakefiles linux-no-openssl && \
#make C_COMPILER=clang-10 CPLUSPLUS_COMPILER=clang++-10 CFLAGS="-g -fsanitize=fuzzer-no-link -fsanitize=address" CXXFLAGS="-g -fsanitize=fuzzer-no-link -fsanitize=address" LINK="clang++-10 -fsanitize=fuzzer-no-link -fsanitize=address -o " all || cd testProgs; clang++-10 -fsanitize=fuzzer-no-link -fsanitize=address -o testOnDemandRTSPServer -L. testOnDemandRTSPServer.o announceURL.o ../liveMedia/libliveMedia.a ../groupsock/libgroupsock.a ../BasicUsageEnvironment/libBasicUsageEnvironment.a ../UsageEnvironment/libUsageEnvironment.a -lsFuzzer -lhfnetdriver -lhfcommon
#nolink 选项为后手动对一些库进行链接，-fsanitize=address开启ASAN地址检查
make C_COMPILER=clang-10 CPLUSPLUS_COMPILER=clang++-10 CFLAGS="-fsanitize=fuzzer-no-link -fsanitize=address" CXXFLAGS="-g -fsanitize=fuzzer-no-link -fsanitize=address" LINK="clang++-10 -fsanitize=fuzzer-no-link -fsanitize=address -o " all || cd testProgs; clang++-10 -fprofile-instr-generate -fcoverage-mapping -fsanitize=fuzzer-no-link -fsanitize=address -o testOnDemandRTSPServer -L. testOnDemandRTSPServer.o announceURL.o ../liveMedia/libliveMedia.a ../groupsock/libgroupsock.a ../BasicUsageEnvironment/libBasicUsageEnvironment.a ../UsageEnvironment/libUsageEnvironment.a -lsFuzzer -lhfnetdriver -lhfcommon
# Set up Live555
RUN cp ${WORKDIR}/sample_media_sources/*.* ${WORKDIR}/live555-sgfuzz/testProgs/
随后进入容器下的testProgs目录运行命令执行
目前的疑惑
- 在profuzzbench上不基于afl的模糊器使用gcov统计覆盖率如何实现
- clang编译的文件支持gcov是否也同样需要对gcov打补丁
- libfuzzer运行遇到crash回停止，如何测试运行指定长时间，中途停止后重新手动运行吗
- libfuzzer的cov应该是branch或者说edge覆盖情况，如何生成随时间变化的路径覆盖信息
159
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
