LLVM 11#85
Conversation
|
Just a quick update: I haven't forgotten this pull request, I'm just busy with other stuff. |
``` ./distance.py_:17: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working if not isinstance(args, collections.Hashable): ```
Hmm. That's weird. afl-gcc doesn't use the LLVM pass. Does it work with vanilla AFLGo? |
Make tests using ./test-instr use paths that are less likely to be optimized out.
|
Yes, the same happens with mirrorer/afl, however it doesn't happen with google/AFL. I found that the problem was patched here. The Makefile in |
|
I did some brief tests, and it seems to work as expected :) |
mboehme
left a comment
There was a problem hiding this comment.
Thanks @Liblor! This is awesome. I'll tag the current version as LLVM-4.0.
I've left a few minor comments. Apart from that could I ask you to make the error messages a bit more expressive? You are welcome to accomodate here or or raise another PR for this.
For instance, it would be good if we got the stderr from distance_calculator here:
aflgo/scripts/gen_distance_fast.py
Line 136 in cbf091f
In distance_calculator, under open_file can you add which file could not be opened and add a newline:
aflgo/distance_calculator/main.cpp
Line 203 in cbf091f
|
|
||
| # Build clang & LLVM | ||
| LLVM_DEP_PACKAGES="build-essential make cmake ninja-build git subversion python2.7 binutils-gold binutils-dev curl wget" | ||
| LLVM_DEP_PACKAGES="build-essential make cmake ninja-build git binutils-gold binutils-dev curl wget python3-distutils" |
There was a problem hiding this comment.
Is python3-distutils really needed? On Ubuntu 16.04 and earlier, you need to first add the apt-repo ppa:deadsnakes/ppa before you can install python3-distutils. On my machine, it seemed to build fine without.
There was a problem hiding this comment.
I've only tried it with Ubuntu 20.04 and 18.04. I didn't check on 16.04, but I remember that there was an issue, if I didn't add python3-distutils. And indeed, I retried it and got the following error during the build process of llvm when the package is not installed.
Traceback (most recent call last):
File "/root/build/llvm_tools/llvm-11.0.0.src/projects/libcxx/utils/merge_archives.py", line 12, in <module>
import distutils.spawn
ModuleNotFoundError: No module named 'distutils.spawn'
it probably works for 16.04 because of an older python version, but that's just a guess. If you tell me that it works without this package on Ubuntu 16.04, I will add a check such that it is only installed when the Ubuntu version is >= 18.04.
| prefix="$TMPDIR/dot-files/$(basename $binary)" | ||
| while ! opt -dot-callgraph $binary.0.0.*.bc -callgraph-dot-filename-prefix $prefix >/dev/null 2> $TMPDIR/step${STEP}.log ; do |
There was a problem hiding this comment.
No, it doesn't. The output directory and name of the created file have changed for the -dot-callgraph flag, the -callgraph-dot-filename-prefix flag makes sure, that the callgraph dot file still lands in the dot-files directory.
I'd have an idea how to address #87, but currently I don't have time. I'll pitch the idea in the issue report later on
|
Awesome! Thanks @Liblor. LGTM. |
Work in progress: Port AFLGo to LLVM 11.
TODO:
afl-llvm-pass.so.cccompilegen_distance_fast.pygenDistance.shafl-gccwhich isn't used anyway (afaik). Building withmake clean && AFL_NO_X86=1 makeworks