honggfuzz

Table of Contents

• 1. TODO
• 2. 总结
  • 2.1. 常用参数
  • 2.2. 关于coverage计算模式
    • 2.2.1. 1. SanitizerCoverage 模式
    • 2.2.2. 2. perf模式
  • 2.3. 关于fuzz运行三种状态
    • 2.3.1. 1. socket模式
    • 2.3.2. 2. persistent模式(常规模式)
    • 2.3.3. 3. -x 静态模式
  • 2.4. 覆盖率报告生成
  • 2.5. 整个fuzz函数流
  • 2.6. trace跟踪子进程
• 3. xcode 调试
• 4. 源码分析
  • 4.1. honggfuzz-clang编译器
  • 4.2. 结构体说明
• 5. 编译
  • 5.1. 1. 依赖库
  • 5.2. 3. clang
  • 5.3. 4. 编译
• 6. fuzz libxml2
  • 6.1. 1. 依赖库
  • 6.2. 2. 编译libxml2
  • 6.3. 编译测试样例
  • 6.4. 执行
  • 6.5. issue

1. TODO

2. 总结

2.1. 常用参数

-n 指定最大线程数

-f 原始输入种子文件目录

-W 有效变异种子文件输出目录

-e 指定输入文件扩展名 默认为*.fuzz

-d Debug info

--mutate_cmd|-c 扩展变异规则以代替原有变异方式, 当你通过-f提供输入样本目录后，在fuzzing时，随机提取的文件会直接传递给-c参数指定的扩展命令作变异

input_prepareExternalFile or input_postProcessFile:

bool input_prepareExternalFile(run_t* run) {
  // ...
  const char* const argv[] = {run->global->exe.externalCommand, fname, NULL};
  if (subproc_System(run, argv) != 0) {
    LOG_E("Subprocess '%s' returned abnormally", run->global->exe.externalCommand);
    return false;
  }
  // ...
}

uint8_t subproc_System(run_t* run, const char* const argv[]) {
  // ...
  execv(argv[0], (char* const*)&argv[0]);
  // ...
}

假如 run->global->exe.externalCommand = extcmd.py
则会执行 extcmd.py fname
在实现中可以通过open(argv[1])打开文件对数据进行处理

--mutations_per_run|-r 每个样例编译的最大次数 默认6

--pprocess_cmd 在原有的文件变异后再作处理

-P persistent模式

-t timeout

-F Max File Size

-V verifier 如果开启，则会验证崩溃样本(最大运行5次)

--exit_upon_crash 有崩溃直接退出fuzz

-R 生成报告文件

-E ENV参数，如增加sanitizers: ASAN_OPTIONS=coverage=1

-S sanitizer模式

参数放在 hfzz.exe.envs[]
hfuzz->sanitizer.enable
如果启用：

snprintf(buf, buflen, "%s=%s:%s:%s%s/%s", env, kASAN_OPTS, abortFlag, kSANLOGDIR,
            hfuzz->io.workDir, kLOGPREFIX);

kASAN_OPTS = kASAN_COMMON_OPTS:

"allow_user_segv_handler=1:" \
    "handle_segv=0:"             \
    "allocator_may_return_null=1:" kSAN_COMMON ":exitcode=" HF_XSTR(HF_SAN_EXIT_CODE)

#define kSAN_COMMON "symbolize=0"
/* Exit code is common for all sanitizers */
#define HF_SAN_EXIT_CODE 103

即：ASAN_OPTIONS=
"allow_user_segv_handler=1:handle_segv=0:allocator_may_return_null=1:symbolize=0:exitcode=103:abort_on_error=1:log_path=workDir/HF.sanitizer.log"

否则：

snprintf(buf, buflen, "%s=%s", env, kSAN_REGULAR);

/* If no sanitzer support was requested, simply make it use abort() on errors */
#define kSAN_REGULAR                                                 \
    "abort_on_error=1:handle_segv=0:handle_sigbus=0:handle_abort=0:" \
    "handle_sigill=0:handle_sigfpe=0:allocator_may_return_null=1:"   \
    "symbolize=1:detect_leaks=0:disable_coredump=0:"                 \
    "detect_odr_violation=0"

-x static mode(_HF_DYNFILE_NONE) 不使用任何feedback.

--instrument|-z 使用编译时的feedback 默认 _HF_DYNFILE_SOFT

-w 字典，针对特殊格式的解析，如xml: https://github.com/rc0r/afl-fuzz/blob/master/dictionaries/xml.dict

实际变异中，命中字典格式变异存在随机性，如果需要只指定字典变异，需要修改源码

mangle_Dictionary -> mangle_DictionaryNoCheck
mangle_DictionaryInsert -> mangle_DictionaryInsertNoCheck
取随机数N,遍历标签取到内容,覆写/插入随机位置。

__FILE__ 相当于AFL中的@@, 在实际的运行中会被input_dir中的输入所替换，替换为文件句柄，可以通过open(fd)打开进行操作，参考honggfuzz/examples/badcode/targets/badcode1.c

bool arch_launchChild(run_t* run) {
    //...
    int x = 0;
    for (x = 0; x < ARGS_MAX && x < run->global->exe.argc; x++) {
        if (run->global->exe.persistent || run->global->exe.fuzzStdin) {
            args[x] = run->global->exe.cmdline[x];
        } else if (!strcmp(run->global->exe.cmdline[x], _HF_FILE_PLACEHOLDER)) {
            args[x] = inputFile;
        } else if (strstr(run->global->exe.cmdline[x], _HF_FILE_PLACEHOLDER)) {
            const char* off = strstr(run->global->exe.cmdline[x], _HF_FILE_PLACEHOLDER);
            snprintf(argData, sizeof(argData), "%.*s%s", (int)(off - run->global->exe.cmdline[x]),
                run->global->exe.cmdline[x], inputFile);
            args[x] = argData;
        } else {
            args[x] = run->global->exe.cmdline[x];
        }
    }
    // ...
}

___FILE___:

必须在非persistent模式(-P)下，且没有指定stdin(-s)的情况下使用，

-s: run->global->exe.fuzzStdin
-P: run->global->exe.persistent

比如 ./badcode1 这个程序的 第1个参数是要处理的文件的名称，那么相应的 fuzz 的命令就是:
/honggfuzz -S -W ./outputs -f ./corpus -- ./badcode1 __FILE__
实际fuzz过程中，会将__FILE__替换成文件句柄传入，作为第一个参数, 即argv[1]=file_fd

--sancov|-C 使用ASAN_OPTIONS="coverage=1", clang-4使用，慢，已经废除(commit: 8267c77c0bfee82a528290c5e86c297291922dc6)。

--monitor_sigabrt: 默认是true，在=-S=模式下会设置ASAN标志 abort_on_error=1， 发生错误会调 abort() 返回错误代码，替代_exit()函数

2.2. 关于coverage计算模式

run->global->feedback.dynFileMethod字段记录着Trace方式

typedef enum {
    _HF_DYNFILE_NONE = 0x0,                             // -x 命令指定 static mode
    _HF_DYNFILE_INSTR_COUNT = 0x1,                      // --linux_perf_instr
    _HF_DYNFILE_BRANCH_COUNT = 0x2,                     // --linux_perf_branch
    _HF_DYNFILE_BTS_EDGE = 0x10,                        // --linux_perf_bts_edge
    _HF_DYNFILE_IPT_BLOCK = 0x20,                       // --linux_perf_ipt_block
    _HF_DYNFILE_SOFT = 0x40,                            // -z 编译时模式，初始化默认 -fsanitize-coverage
} dynFileMethod_t;

2.2.1. 1. SanitizerCoverage 模式

需要编译源码

编译添加CFLAGS: -fsanitize-coverage=trace-pc,trace-pc-guard,trace-cmp,trace-div,indirect-calls

默认方式_HF_DYNFILE_SOFT，通过重定义 SanitizerCoverage 中的函数来实现累加

Pc: __sanitizer_cov_trace_pc_guard

Edge: __sanitizer_cov_trace_pc_guard

Cmp: __sanitizer_cov_trace_cmpN, __sanitizer_cov_trace_switch, __sanitizer_cov_trace_divN, N=(1,2,4,8)

参考:

https://bcain-llvm.readthedocs.io/projects/clang/en/release_39/SanitizerCoverage/

https://github.com/google/sanitizers/wiki/SanitizerCommonFlags

2.2.2. 2. perf模式

不需要编译源码

fuzz时候通过添加参数实现

arch_perfCreate 创建 fd

perf_event_open(&pe, pid, -1, -1, PERF_FLAG_FD_CLOEXEC);

dynFileMethod_t             perf config                                                 fd

_HF_DYNFILE_INSTR_COUNT:    pe.config = PERF_COUNT_HW_INSTRUCTIONS                      &run->linux.cpuInstrFd
_HF_DYNFILE_BRANCH_COUNT:   pe.config = PERF_COUNT_HW_BRANCH_INSTRUCTIONS               &run->linux.cpuBranchFd
_HF_DYNFILE_BTS_EDGE:       pe.type = /sys/bus/event_source/devices/intel_bts/type      &run->linux.cpuIptBtsFd
_HF_DYNFILE_IPT_BLOCK:      pe.type = /sys/bus/event_source/devices/intel_pt/type,
                            pe.config = RTIT_CTL_DISRETC;                               &run->linux.cpuIptBtsFd

PERF_COUNT_HW_INSTRUCTIONS 完整执行的指令数
PERF_COUNT_HW_BRANCH_INSTRUCTIONS 完整执行的分支数

该模式需要硬件支持，简单验证：

在host运行perf命令即可得倒相应数据

./perf stat pwd
/home/secret/work

 Performance counter stats for 'pwd':

          0.277524      task-clock (msec)         #    0.017 CPUs utilized
                 1      context-switches          #    0.004 M/sec
                 0      cpu-migrations            #    0.000 K/sec
                56      page-faults               #    0.202 M/sec
           989,977      cycles                    #    3.567 GHz
   <not supported>      stalled-cycles-frontend
   <not supported>      stalled-cycles-backend
           656,001      instructions              #    0.66  insns per cycle
           135,672      branches                  #  488.866 M/sec
             7,481      branch-misses             #    5.51% of all branches

       0.016769350 seconds time elapsed

instructions 记录着指令数

参考：

http://www.man7.org/linux/man-pages/man2/perf_event_open.2.html

https://software.intel.com/en-us/vtune-amplifier-help-instructions-retired-event

2.3. 关于fuzz运行三种状态

hfuzz->feedback.state

typedef enum {
    _HF_STATE_UNSET = 0,                                // 初始化默认值
    _HF_STATE_STATIC = 1,                               // hfuzz->feedback.dynFileMethod == _HF_DYNFILE_NONE
    _HF_STATE_DYNAMIC_DRY_RUN = 2,                      // hfuzz->feedback.dynFileMethod != _HF_DYNFILE_NONE
    _HF_STATE_DYNAMIC_MAIN = 3,                         // hfuzz->socketFuzzer.enabled == true
} fuzzState_t;

2.3.1. 1. socket模式

hfuzz->feedback.state = _HF_STATE_DYNAMIC_MAIN

该模式用于fuzz socket服务

2.3.2. 2. persistent模式(常规模式)

正常设置语料库模式下，状态为 hfuzz->feedback.state = _HF_STATE_DYNAMIC_DRY_RUN, 执行所有语料库文件且不变异(run->mutationsPerRun = 0 不变异);

之后通过函数 fuzz_setDynamicMainState() 设置状态 hfuzz->feedback.state = _HF_STATE_DYNAMIC_MAIN (run->mutationsPerRun = -r指定变异次数，默认6)，进入正常fuzz流程。

2.3.3. 3. -x 静态模式

hfuzz->feedback.state = _HF_STATE_STATIC

该模式会设置 hfuzz->feedback.dynFileMethod = _HF_DYNFILE_NONE, 不会使用perf模式，不会产生feedback,主要用于验证样本

如复现某个崩溃样本

./honggfuzz -f crash_dir -x -R ./bug_report --exit_upon_crash -- ./honggfuzz-example/bin/persistent-bin

2.4. 覆盖率报告生成

clang-9.0

https://clang.llvm.org/docs/SourceBasedCodeCoverage.html#creating-coverage-reports

2.5. 整个fuzz函数流

main ->
fuzz_threadsStart ->
fuzz_threadNew ->
fuzz_fuzzLoop ->
subproc_Run ->
subproc_New ->
arch_launchChild(子进程中执行)

bool subproc_Run(run_t* run) {
    run->timeStartedMillis = util_timeNowMillis();
    /*
        fork子进程，通过ptrace attach
        打开perf，用于记录完整执行的指令数
        execve执行fuzz实例
    */
    if (!subproc_New(run)) {
        LOG_E("subproc_New()");
        return false;
    }
    /*
        arch_perfEnable 启用perf，记录执行的指令数
    */
    arch_prepareParent(run);
    /*
        arch_checkWait() -> arch_traceAnalyze()/arch_perfAnalyze() ptrace/perf结果分析
        崩溃记录，记录 cpuInstrCnt 和 cpuBranchCnt，关闭perf
    */
    arch_reapChild(run);

    return true;
}

static bool subproc_New(run_t* run) {
    // ...
    run->pid = arch_fork(run);
    if (run->pid == -1) {
        PLOG_E("Couldn't fork");
        run->pid = 0;
        return false;
    }
    /* The child process */
    if (!run->pid) {
        // ...
        /*
            系统资源和文件初始化 setrlimit
            -E fsanitizer参数放入环境变量 putenv
        */
        if (!subproc_PrepareExecv(run)) {
            LOG_E("subproc_PrepareExecv() failed");
            exit(EXIT_FAILURE);
        }
        /*
            子进程执行
            1. prctl(PR_SET_DUMPABLE), 设置可以被attach
            2. 关闭ASLR
            3. 格式化fuzz实例参数，等待父进程attach，然后调用execve(args[0], (char* const*)args, environ);
        */
        if (!arch_launchChild(run)) {
            LOG_E("Error launching child process");
            kill(run->global->threads.mainPid, SIGTERM);
            _exit(1);
        }
        abort();
    }

    // ...

    /*
        1. fcntl(%d, F_SETOWN_EX)，开启perf，用于记录完整执行的指令数
        2. arch_attachToNewPid():
            ptrace attatch fork子进程 (PTRACE_SEIZE方式,具体实现arch_traceAttach), 并通过 ptrace(PTRACE_CONT) Restart the stopped tracee process
    */
    arch_prepareParentAfterFork(run);
    // ...
    return true;
}

2.6. trace跟踪子进程

arch_reapChild() -> arch_checkWait() -> arch_traceAnalyze()

通过ptrace挂载子进程，捕获WIFSTOPPED状态，该状态表明目标进程退出，

之后通过 PTRACE_GETEVENTMSG 标志获取退出状态，进行退出状态码判断：

run->mainWorker=true(默认模式下): arch_traceSaveData
run->mainWorker=false(Verifier模式下): arch_traceAnalyzeData
HF_SAN_EXIT_CODE: arch_traceExitAnalyze

相关API:

long int ptrace(enum __ptrace_request request, pid_t pid, void * addr, void * data)
request决定ptrace做什么,pid是被跟踪进程的ID,data存储从进程空间偏移量为addr的地方开始将被读取/写入的数据.

pid_t wait4(pid_t pid,int *status,int options,struct rusage *rusage);
获得指定子进程的资源使用信息，通过参数 rusage 获得

PTRACE_SEIZE(since Linux 3.4) Attach to the process specified in pid
相对于 PTRACE_ATTACH, 该方式不会停止目标进程

PTRACE_CONT 让子进程继续运行

PTRACE_GETEVENTMSG (since Linux 2.5.46) 取回数据

arch_traceAnalyze()

3. xcode 调试

需要修改honggfuzz源码目录下的Makefile文件添加调试选项

diff --git a/Makefile b/Makefile
index fc5ea74..c6e1fdc 100644
--- a/Makefile
+++ b/Makefile
@@ -29,7 +29,8 @@ HFUZZ_CC_SRCS := hfuzz_cc/hfuzz-cc.c
 COMMON_CFLAGS := -D_GNU_SOURCE -Wall -Werror -Wno-format-truncation -I.
 COMMON_LDFLAGS := -lm libhfcommon/libhfcommon.a
 COMMON_SRCS := $(sort $(wildcard *.c))
-CFLAGS ?= -O3 -mtune=native
+# CFLAGS ?= -O3 -mtune=native
+CFLAGS ?= -g -mtune=native
 LDFLAGS ?=
 LIBS_CFLAGS ?= -fPIC -fno-stack-protector
 GREP_COLOR ?=

之后参考文章 https://blog.csdn.net/kubibo/article/details/25902703

4. 源码分析

4.1. honggfuzz-clang编译器

honggfuzz/hfuzz_cc/hfuzz-cc.c

通过编译器名字(hfuzz-clang, hfuzz-clang++, hfuzz-g++, hfuzz-gcc)来判断使用哪种编译器编译。

hfuzz-clang         -> clang
hfuzz-clang++       -> clang++
hfuzz-g++           -> g++
hfuzz-gcc           -> gcc

之后是参数拼接，主要添加的优化参数列表(clang):

"-Wno-unused-command-line-argument";
"-fsanitize-coverage=trace-pc-guard,trace-cmp,trace-div,indirect-calls";
"-mllvm";
"-sanitizer-coverage-prune-blocks=0";
"-mllvm";
"-sanitizer-coverage-level=3";
/*
 * Make the execution flow more explicit, allowing for more code blocks
 * (and better code coverage estimates)
 */
"-fno-inline";
"-fno-builtin";
"-fno-omit-frame-pointer";
"-D__NO_STRING_INLINES";
 /* Make it possible to use the libhfnetdriver */
"-DHFND_FUZZING_ENTRY_FUNCTION_CXX(x,y)="
                   "extern \"C\" int HonggfuzzNetDriver_main(x,y);"
                   "extern const char* LIBHFNETDRIVER_module_netdriver;"
                   "const char** LIBHFNETDRIVER_module_main = &LIBHFNETDRIVER_module_netdriver;"
                   "int HonggfuzzNetDriver_main(x,y)";
"-DHFND_FUZZING_ENTRY_FUNCTION(x,y)="
                   "int HonggfuzzNetDriver_main(x,y);"
                   "extern const char* LIBHFNETDRIVER_module_netdriver;"
                   "const char** LIBHFNETDRIVER_module_main = &LIBHFNETDRIVER_module_netdriver;"
                   "int HonggfuzzNetDriver_main(x,y)";
/* Intercept common *cmp functions */
"-Wl,--wrap=strcmp";
"-Wl,--wrap=strcasecmp";
"-Wl,--wrap=strncmp";
"-Wl,--wrap=strncasecmp";
"-Wl,--wrap=strstr";
"-Wl,--wrap=strcasestr";
"-Wl,--wrap=memcmp";
"-Wl,--wrap=bcmp";
"-Wl,--wrap=memmem";
"-Wl,--wrap=strcpy";
/* Apache's httpd mem/str cmp functions */
"-Wl,--wrap=ap_cstr_casecmp";
"-Wl,--wrap=ap_cstr_casecmpn";
"-Wl,--wrap=ap_strcasestr";
"-Wl,--wrap=apr_cstr_casecmp";
"-Wl,--wrap=apr_cstr_casecmpn";
/* Frequently used time-constant *SSL functions */
"-Wl,--wrap=CRYPTO_memcmp";
"-Wl,--wrap=OPENSSL_memcmp";
"-Wl,--wrap=OPENSSL_strcasecmp";
"-Wl,--wrap=OPENSSL_strncasecmp";
"-Wl,--wrap=memcmpct";
/* Frequently used libXML2 functions */
"-Wl,--wrap=xmlStrncmp";
"-Wl,--wrap=xmlStrcmp";
"-Wl,--wrap=xmlStrEqual";
"-Wl,--wrap=xmlStrcasecmp";
"-Wl,--wrap=xmlStrncasecmp";
"-Wl,--wrap=xmlStrstr";
"-Wl,--wrap=xmlStrcasestr";
/* Some Samba functions */
"-Wl,--wrap=memcmp_const_time";
"-Wl,--wrap=strcsequal";
/* Pull modules defining the following symbols (if they exist) */
args[j++] = "-Wl,-u,LIBHFNETDRIVER_module_main",
args[j++] = "-Wl,-u,LIBHFUZZ_module_instrument";
args[j++] = "-Wl,-u,LIBHFUZZ_module_memorycmp";
"/tmp/libhfnetdriver.uid.crc64.a";  // eg: /tmp/libhfnetdriver.1000.9a0f6dce36be32e.a, libhfnetdriver/libhfnetdriver.a
"/tmp/libhfuzz.uid.crc64.a"  // eg: /tmp/libhfuzz.1000.cb422fcd8679a683.a, libhfuzz/libhfuzz.a

-fsanitize参数

ASAN(AddressSanitizer) -fsanitize=address: 打开asan内存错误检查
-fno-omit-frame-pointer: 保留函数调用的帧信息，以便分析函数调用关系
UBSAN(UndefinedBehaviorSanitizer) -fsanitize=undefined: 未定义行为检测

4.2. 结构体说明

typedef struct {
    struct {
        size_t threadsMax;                                  // -n 指定最大线程数
        size_t threadsFinished;
        uint32_t threadsActiveCnt;
        pthread_t mainThread;
        pid_t mainPid;
        pthread_t threads[_HF_THREAD_MAX];
    } threads;
    struct {
        const char* inputDir;                               // -f 原始种子文件目录
        DIR* inputDirPtr;                                   // fdopendir(dir_fd) 打开的原始种子目录指针
        size_t fileCnt;                                     // 原始种子文件个数
        const char* fileExtn;                               // 文件扩展名, 不能包含 /
        bool fileCntDone;
        const char* workDir;                                // -W 有效变异种子文件输出目录
        const char* crashDir;
        const char* covDirAll;                              // 默认值为-f指定目录
        const char* covDirNew;
        bool saveUnique;
        size_t dynfileqCnt;
        pthread_rwlock_t dynfileq_mutex;
        TAILQ_HEAD(dyns_t, dynfile_t) dynfileq;
    } io;
    struct {
        int argc;
        const char* const* cmdline;                         // -- 后边跟的fuzz样例
        bool nullifyStdio;
        bool fuzzStdin;
        const char* externalCommand;                        // -c fuzz样例参数
        const char* postExternalCommand;
        bool netDriver;
        bool persistent;                                    // -P persistent模式
        uint64_t asLimit;
        uint64_t rssLimit;
        uint64_t dataLimit;
        uint64_t coreLimit;
        bool clearEnv;
        char* envs[128];                                    // -E ENV参数，如sanitizers
        sigset_t waitSigSet;
    } exe;
    struct {
        time_t timeStart;
        time_t runEndTime;
        time_t tmOut;                                       // -t timeout
        time_t lastCovUpdate;
        bool tmoutVTALRM;
    } timing;
    struct {
        const char* dictionaryFile;
        TAILQ_HEAD(strq_t, strings_t) dictq;
        size_t dictionaryCnt;
        size_t mutationsMax;
        unsigned mutationsPerRun;
        size_t maxFileSz;                                   // -F Max File Size
    } mutate;
    struct {
        bool useScreen;
        char cmdline_txt[65];
        int64_t lastDisplayMillis;
    } display;
    struct {
        bool useVerifier;                                   // -V verifier 如果开启，则会验证崩溃样本(最大运行5次)
        bool exitUponCrash;                                 // --exit_upon_crash 有崩溃直接退出fuzz
        const char* reportFile;                             // -R 生成报告文件
        pthread_mutex_t report_mutex;
        bool monitorSIGABRT;
        size_t dynFileIterExpire;
        bool only_printable;
    } cfg;
    struct {
        bool enable;                                          // -S sanitizer
    } sanitizer;
    struct {
        fuzzState_t state;
        feedback_t* feedbackMap;
        int bbFd;
        pthread_mutex_t feedback_mutex;
        const char* blacklistFile;
        uint64_t* blacklist;
        size_t blacklistCnt;
        bool skipFeedbackOnTimeout;
        dynFileMethod_t dynFileMethod;                      // -x static mode, feedback 默认 _HF_DYNFILE_SOFT
    } feedback;
    struct {
        size_t mutationsCnt;
        size_t crashesCnt;
        size_t uniqueCrashesCnt;
        size_t verifiedCrashesCnt;
        size_t blCrashesCnt;
        size_t timeoutedCnt;
    } cnts;
    struct {
        bool enabled;
        int serverSocket;
        int clientSocket;
    } socketFuzzer;
    /* For the Linux code */
    struct {
        int exeFd;
        hwcnt_t hwCnts;                             // 记录所有fuzz实例的代码覆盖数总和 edge pc cmp
        uint64_t dynamicCutOffAddr;
        bool disableRandomization;
        void* ignoreAddr;
        size_t numMajorFrames;
        const char* symsBlFile;
        char** symsBl;
        size_t symsBlCnt;
        const char* symsWlFile;
        char** symsWl;
        size_t symsWlCnt;
        uintptr_t cloneFlags;
        bool kernelOnly;
        bool useClone;
    } linux;
    /* For the NetBSD code */
    struct {
        void* ignoreAddr;
        size_t numMajorFrames;
        const char* symsBlFile;
        char** symsBl;
        size_t symsBlCnt;
        const char* symsWlFile;
        char** symsWl;
        size_t symsWlCnt;
    } netbsd;
} honggfuzz_t;


fuzzing thread:

typedef struct {
    honggfuzz_t* global;                // hfuzz实例
    pid_t pid;                          // 当前fuzz种子线程pid
    int64_t timeStartedMillis;
    char origFileName[PATH_MAX];
    char crashFileName[PATH_MAX];
    uint64_t pc;
    uint64_t backtrace;
    uint64_t access;
    int exception;
    char report[_HF_REPORT_SIZE];
    bool mainWorker;
    unsigned mutationsPerRun;
    struct dynfile_t* dynfileqCurrent;  // {data: 种子文件内容，size: 长度, pointers: 文件队列}
    uint8_t* dynamicFile;               // 指向name=hfuzz-input的buf, 该buff接受mangle变异的数据
    size_t dynamicFileSz;               // 种子文件size
    int dynamicFileFd;                  // 通过memfd_create创建name=hfuzz-input的匿名文件句柄
    int dynamicFileCopyFd;              // fuzz种子句柄，执行时会用过/dev/fd/%d方式打开并获取内容
    uint32_t fuzzNo;                    // 编号
    int persistentSock;
    bool waitingForReady;
    runState_t runState;
    bool tmOutSignaled;
#if !defined(_HF_ARCH_DARWIN)
    timer_t timerId;
#endif  // !defined(_HF_ARCH_DARWIN)

    struct {
        /* For Linux code */
        uint8_t* perfMmapBuf;
        uint8_t* perfMmapAux;
        hwcnt_t hwCnts;                 // 记录当前fuzz实例的代码覆盖数 edge pc cmp
        int cpuInstrFd;
        int cpuBranchFd;
        int cpuIptBtsFd;
    } linux;

    struct {
        /* For NetBSD code */
        uint8_t* perfMmapBuf;
        uint8_t* perfMmapAux;
        hwcnt_t hwCnts;
        int cpuInstrFd;
        int cpuBranchFd;
        int cpuIptBtsFd;
    } netbsd;
} run_t;

(lldb) print *hfuzz
(honggfuzz_t) $5 = {
  threads = {
    threadsMax = 2
    threadsFinished = 0
    threadsActiveCnt = 0
    mainThread = 0x000000013f19f380
    mainPid = 48586
    threads = {
      [0] = 0x0000000000000000
      [1] = 0x0000000000000000
      [2] = 0x0000000000000000
      [3] = 0x0000000000000000
      ...
      [255] = 0x0000000000000000
      ...
    }
  }
  io = {
    inputDir = 0x00007ffeefbffa3d "/Users/idhyt/Work/sec.ret/gitlab/opensourcefuzz/honggfuzz-example/bin/corpus_2019-02-28_13-48-20/"
    inputDirPtr = 0x000000013f601af0
    fileCnt = 5
    fileExtn = 0x000000010002cffb "fuzz"
    fileCntDone = false
    workDir = 0x00007ffeefbff9d8 "/Users/idhyt/Work/sec.ret/gitlab/opensourcefuzz/honggfuzz-example/bin/outputs_2019-02-28_13-48-20"
    crashDir = 0x00007ffeefbff9d8 "/Users/idhyt/Work/sec.ret/gitlab/opensourcefuzz/honggfuzz-example/bin/outputs_2019-02-28_13-48-20"
    covDirAll = 0x00007ffeefbffa3d "/Users/idhyt/Work/sec.ret/gitlab/opensourcefuzz/honggfuzz-example/bin/corpus_2019-02-28_13-48-20/"
    covDirNew = 0x0000000000000000 <no value available>
    saveUnique = true
    dynfileqCnt = 0
    dynfileq_mutex = (__sig = 766030772, __opaque = char [192] @ 0x00007fb4a533fa88)
    dynfileq = {
      tqh_first = 0x0000000000000000
      tqh_last = 0x0000000100128c68
    }
  }
  exe = {
    argc = 1
    cmdline = 0x000000013f302e68
    nullifyStdio = true
    fuzzStdin = false
    externalCommand = 0x0000000000000000 <no value available>
    postExternalCommand = 0x0000000000000000 <no value available>
    netDriver = false
    persistent = true
    asLimit = 0
    rssLimit = 0
    dataLimit = 0
    coreLimit = 0
    clearEnv = false
    envs = {
      [0] = 0x000000010005f5f0 "ASAN_OPTIONS=abort_on_error=1:handle_segv=0:handle_sigbus=0:handle_abort=0:handle_sigill=0:handle_sigfpe=0:allocator_may_return_null=1:symbolize=1:detect_leaks=0:disable_coredump=0:detect_odr_violation=0"
      [1] = 0x00000001000605f0 "UBSAN_OPTIONS=abort_on_error=1:handle_segv=0:handle_sigbus=0:handle_abort=0:handle_sigill=0:handle_sigfpe=0:allocator_may_return_null=1:symbolize=1:detect_leaks=0:disable_coredump=0:detect_odr_violation=0"
      [2] = 0x00000001000615f0 "MSAN_OPTIONS=abort_on_error=1:handle_segv=0:handle_sigbus=0:handle_abort=0:handle_sigill=0:handle_sigfpe=0:allocator_may_return_null=1:symbolize=1:detect_leaks=0:disable_coredump=0:detect_odr_violation=0"
      [3] = 0x00000001000625f0 "LSAN_OPTIONS=abort_on_error=1:handle_segv=0:handle_sigbus=0:handle_abort=0:handle_sigill=0:handle_sigfpe=0:allocator_may_return_null=1:symbolize=1:detect_leaks=0:disable_coredump=0:detect_odr_violation=0"
      [4] = 0x0000000000000000 <no value available>
      [5] = 0x0000000000000000 <no value available>
      ...
      [127] = 0x0000000000000000 <no value available>
    }
    waitSigSet = 541065216
  }
  timing = (timeStart = 1551337933, runEndTime = 0, tmOut = 10, lastCovUpdate = 1551337933, tmoutVTALRM = false)
  mutate = {
    dictionaryFile = 0x0000000000000000 <no value available>
    dictq = {
      tqh_first = 0x0000000000000000
      tqh_last = 0x0000000100129108
    }
    dictionaryCnt = 0
    mutationsMax = 0
    mutationsPerRun = 6
    maxFileSz = 8192
  }
  display = (useScreen = true, cmdline_txt = char [65] @ 0x00007fb4a5340019, lastDisplayMillis = 1551337933633)
  cfg = {
    useVerifier = false
    exitUponCrash = false
    reportFile = 0x0000000000000000 <no value available>
    report_mutex = (__sig = 850045863, __opaque = char [56] @ 0x00007fb4a5340080)
    monitorSIGABRT = true
    dynFileIterExpire = 0
    only_printable = false
  }
  sanitizer = (enable = false)
  feedback = {
    state = _HF_STATE_UNSET
    feedbackMap = 0x0000000142180000
    bbFd = 5
    feedback_mutex = (__sig = 850045863, __opaque = char [56] @ 0x00007fb4a53400f8)
    blacklistFile = 0x0000000000000000 <no value available>
    blacklist = 0x0000000000000000
    blacklistCnt = 0
    skipFeedbackOnTimeout = false
    dynFileMethod = _HF_DYNFILE_SOFT
  }
  cnts = {
    mutationsCnt = 0
    crashesCnt = 0
    uniqueCrashesCnt = 0
    verifiedCrashesCnt = 0
    blCrashesCnt = 0
    timeoutedCnt = 0
  }
  socketFuzzer = (enabled = false, serverSocket = -1, clientSocket = -1)
  linux = {
    exeFd = -1
    hwCnts = {
      cpuInstrCnt = 0
      cpuBranchCnt = 0
      bbCnt = 0
      newBBCnt = 0
      softCntPc = 0
      softCntEdge = 0
      softCntCmp = 0
    }
    dynamicCutOffAddr = 18446744073709551615
    disableRandomization = true
    ignoreAddr = 0x0000000000000000
    numMajorFrames = 7
    symsBlFile = 0x0000000000000000 <no value available>
    symsBl = 0x0000000000000000
    symsBlCnt = 0
    symsWlFile = 0x0000000000000000 <no value available>
    symsWl = 0x0000000000000000
    symsWlCnt = 0
    cloneFlags = 0
    kernelOnly = false
    useClone = true
  }
  netbsd = {
    ignoreAddr = 0x0000000000000000
    numMajorFrames = 7
    symsBlFile = 0x0000000000000000 <no value available>
    symsBl = 0x0000000000000000
    symsBlCnt = 0
    symsWlFile = 0x0000000000000000 <no value available>
    symsWl = 0x0000000000000000
    symsWlCnt = 0
  }
}

5. 编译

5.1. 1. 依赖库

sudo apt-get install libbfd-dev libunwind8-dev

### 2. GCC

sudo add-apt-repository ppa:jonathonf/gcc-7.1
sudo apt-get update
sudo apt-get install gcc-7 g++-7
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 70

5.2. 3. clang

要求 5.0+

sudo apt install clang-6.0

最新版

wget https://raw.githubusercontent.com/Dor1s/libfuzzer-workshop/master/checkout_build_install_llvm.sh
./checkout_build_install_llvm.sh

5.3. 4. 编译

git clone https://github.com/google/honggfuzz.git
cd honggfuzz
make

6. fuzz libxml2

6.1. 1. 依赖库

sudo apt install autoconf libtool liblzma-dev python-dev

6.2. 2. 编译libxml2

git clone https://github.com/GNOME/libxml2.git
cd libxml2/
git checkout f8a8c1f59db355b46962577e7b74f1a1e8149dc6
git branch libxml2-2.9.9
git checkout libxml2-2.9.9

./autogen.sh

hfuzz_cc_dir=/path/to/honggfuzz/hfuzz_cc
CFLAGS="-g -fno-omit-frame-pointer -fsanitize=address,undefined -fsanitize-coverage=trace-pc-guard"
CC="$hfuzz_cc_dir/hfuzz-clang $CFLAGS" CXX="$hfuzz_cc_dir/hfuzz-clang++ $CFLAGS" CCLD="$hfuzz_cc_dir/hfuzz-clang $CFLAGS" ./configure

make -j4

ls .libs/libxml2.a

6.3. 编译测试样例

测试样例: persistent-xml2

cd /path/to/honggfuzz/examples/libxml2

CFLAGS="-g -fno-omit-frame-pointer -fsanitize=address,undefined -fsanitize-coverage=trace-pc-guard"

./path/to/honggfuzz/hfuzz_cc/hfuzz-clang persistent-xml2.c $CFLAGS -I/path/to/libxml2/include -I/path/to/libxml2 /path/to/libxml2/.libs/libxml2.a /path/to/honggfuzz/libhfuzz/libhfuzz.a -o persistent-xml2

6.4. 执行

mkdir output_dir
mkdir corpus_dir
echo "test" > ./corpus_dir/origin

./path/to/honggfuzz/honggfuzz -P -S -W ./output_dir -f ./corpus_dir -- ./persistent-xml2

使用字典:
./path/to/honggfuzz/honggfuzz -P -S -w ./xml.dict -W ./output_dir -f ./corpus_dir -- ./persistent-xml2

6.5. issue

1. Did you mean '-sanitizer-coverage-level=0'…

clang-5.0

2. ASAN

==84373==LeakSanitizer has encountered a fatal error.
==84373==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
==84373==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)

fuzz去掉 -S 参数

3. ubuntu 16.04 apt

Reading package lists... Done
E: Problem executing scripts APT::Update::Post-Invoke-Success
'if /usr/bin/test -w /var/cache/app-info -a -e /usr/bin/appstreamcli;
 then appstreamcli refresh > /dev/null;
 fi'
E: Sub-process returned an error code

sudo pkill -KILL appstreamcli
wget -P /tmp https://launchpad.net/ubuntu/+archive/primary/+files/appstream_0.9.4-1ubuntu1_amd64.deb https://launchpad.net/ubuntu/+archive/primary/+files/libappstream3_0.9.4-1ubuntu1_amd64.deb
sudo dpkg -i /tmp/appstream_0.9.4-1ubuntu1_amd64.deb /tmp/libappstream3_0.9.4-1ubuntu1_amd64.deb

Author: idhyt

Created: 2017-03-02 Thu 15:53

Validate