Skip to content

Add libfuzzer fuzzer for continuous fuzzing#448

Closed
AdamKorcz wants to merge 2 commits into
uber:masterfrom
AdamKorcz:fuzz1
Closed

Add libfuzzer fuzzer for continuous fuzzing#448
AdamKorcz wants to merge 2 commits into
uber:masterfrom
AdamKorcz:fuzz1

Conversation

@AdamKorcz

@AdamKorcz AdamKorcz commented Apr 14, 2021

Copy link
Copy Markdown

Hi all, this is Adam of AdaLogics. I work on securing open source software and have worked on setting up continuous fuzzing of h3 through OSS-fuzz.

Since we cannot use the existing AFL fuzzers for the OSS-fuzz integration, I have added a libfuzzer fuzzer in this PR. Once this one is merged, the build on the OSS-fuzz side (google/oss-fuzz#5616) will pass.

Since each target of h3 in general is pretty small I have added them all in a single fuzzer. For now I recommend following this until it becomes stricly necessary to have more fuzzers, since there is little point in having dedicated fuzzers for 100-200 lines of code in terms of maintenance.

I will be happy to add more targets to the fuzzer. This will become easier once h3 is fuzzed by OSS-fuzz, since OSS-fuzz offers some useful diagnostic tools and coverage reports.

On the OSS-fuzz side a maintainers email address is needed for bug reports.

@CLAassistant

CLAassistant commented Apr 14, 2021

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@coveralls

coveralls commented Apr 14, 2021

Copy link
Copy Markdown

Coverage Status

Coverage remained the same at 99.317% when pulling 71e2354 on AdamKorcz:fuzz1 into a461213 on uber:master.

@dfellis

dfellis commented Apr 14, 2021

Copy link
Copy Markdown
Collaborator

So I don't think this will work as-is, because in here you call the file libfuzzer.c and in the other PR you call it h3_fuzzer.c which doesn't exist.

I would also note that there is no explanatory comments in this PR about what the purpose of this file is, so it could get accidentally deleted. I would definitely include comments with a link to the https://github.com/google/oss-fuzz project to make this clear to future developers.

Finally, I don't personally care for the compile rules for this file all living in the other repo. It would be relatively easy to break things if the code here is refactored. Better to have it in cmake as a new make rule that is simply invoked in the other repo. Ideally also with a clear way to run this fuzzing locally for debugging with gdb attached? I'm not sure what kind of report, if any, oss-fuzz provides on failures beyond pass/fail.

@nrabinowitz

Copy link
Copy Markdown
Collaborator

This looks great! We have some fuzzers in the lib, but being able to run continuous fuzzing would be great. Some thoughts here:

  • It looks like you added a custom build target for the new fuzzer file in your oss-fuzz PR. We should probably add a Cmake target for this instead; that would reduce what the oss-fuzz code has to know about the internals of the H3 repo, and it would make it easier to update the build if we add additional files, etc.
  • We should probably run the fuzzer you have here with known-good input as part of the standard CI test suite. That would ensure that it compiles and runs (for example, we just landed a significant renaming PR, which will be the basis for a 4.0.0-alpha-1 release, and I'd be surprised if this compiles at present).

@isaacbrodsky isaacbrodsky left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR! This is a great contribution and it's exciting to get more tooling and analysis looking at the library.

I agree with David and Nick that having a CMake target for this file would help us compile & run it as part of our CI, and thus prevent accidentally breaking this. I'd suggest adding something like the following, above line 640 in CMakeLists.txt:

    if(ENABLE_REQUIRES_ALL_SYMBOLS)
        add_library(libfuzzer src/apps/fuzzers/libfuzzer.c)
        target_link_libraries(libfuzzer PUBLIC h3)
        target_include_directories(libfuzzer  PUBLIC
          $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/src/apps/applib/include>)
        target_compile_options(libfuzzer PRIVATE ${H3_COMPILE_FLAGS})
        target_link_libraries(libfuzzer PRIVATE ${H3_LINK_FLAGS})
    endif()

Comment thread src/apps/fuzzers/libfuzzer.c Outdated
Comment thread src/apps/fuzzers/libfuzzer.c
Comment thread src/apps/fuzzers/libfuzzer.c
@isaacbrodsky

Copy link
Copy Markdown
Collaborator

On the OSS-fuzz side a maintainers email address is needed for bug reports.

Please let me know where a good place to provide this would be - here or on the OSS-fuzz PR or over Slack?

@AdamKorcz

Copy link
Copy Markdown
Author

Please let me know where a good place to provide this would be - here or on the OSS-fuzz PR or over Slack?

The OSS-fuzz PR would be great.

@AdamKorcz

Copy link
Copy Markdown
Author

@dfellis @nrabinowitz

So I don't think this will work as-is, because in here you call the file libfuzzer.c and in the other PR you call it h3_fuzzer.c which doesn't exist.

You are right. The build script will require a few modifications to use the fuzzer from this PR. I will take care of that.

I would also note that there is no explanatory comments in this PR about what the purpose of this file is, so it could get accidentally deleted. I would definitely include comments with a link to the https://github.com/google/oss-fuzz project to make this clear to future developers.

Absolutely! I will add that, however I would suggest we wait with that until H3 is integrated into OSS-fuzz. There may be other things that will need commenting in the process.

Regarding adding a rule in the build system, thank you very much for the great feedback, and thank you for the suggestion @isaacbrodsky. I will be glad to arrange a PR for that.
I suggest that this gets done after H3 is integrated into OSS-fuzz. Once the fuzzer is merged here, and the build script is merged upstream, we will have a complete picture of what needs to be added to the build script. At the moment there may still be a few adjustments, and I would prefer to have and end-to-end prototype of the integration before making the prototype easier manageable by setting up a make rule.
Once H3 is integrated into OSS-fuzz, I can look at it immediately afterwards. It should maximum take me a week for a PR with the added logic in the makefile.

What are your thoughts on the above?

@AdamKorcz

Copy link
Copy Markdown
Author

@isaacbrodsky Kind ping. Do you have an email address for bug reports?

@isaacbrodsky

Copy link
Copy Markdown
Collaborator

@isaacbrodsky Kind ping. Do you have an email address for bug reports?

Oops, seems like this got dropped. I need to confirm with @ajfriend if we have a distribution group we can use for this. In the mean time you can use my email, isaac@isaacbrodsky.com for reports.

@ajfriend

Copy link
Copy Markdown
Collaborator

@AdamKorcz, you can send bug reports to h3-dev@googlegroups.com.

The (eventual) group history can be seen here: https://groups.google.com/g/h3-dev/

@isaacbrodsky

Copy link
Copy Markdown
Collaborator

I think this has been superseded by new fuzzers in uber/h3 - if we still want to use custom fuzzers in OSS-Fuzz please let me know

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants