Evaluating synthetic bugs

J Bundt, A Fasano, B Dolan-Gavitt… - Proceedings of the …, 2021 - dl.acm.org
… We find that recent updates to bug injection systems have made synthetic bugs more … to
find than organic bugs in our target programs. Finally, this study identifies flaws in bug injection …

Bug synthesis: Challenging bug-finding tools with deep faults

S Roy, A Pandey, B Dolan-Gavitt, Y Hu - … of the 2018 26th ACM Joint …, 2018 - dl.acm.org
… a few program inputs) bugs in large software code bases. In our evaluation, we inject bugs
into thirty Coreutils programs as well as the TCAS test suite. We find that bugs synthesized by …

Characterizing and improving bug-finders with synthetic bugs

Y Hu, Z Shen, B Dolan-Gavitt - 2022 IEEE International …, 2022 - ieeexplore.ieee.org
bug injection and other synthetic bug corpora can be used to identify issues in mature bug-…
, synthetic programs we evaluate in the first part, we also examine larger programs using …

On the Effectiveness of Synthetic Benchmarks for Evaluating Directed Grey-box Fuzzers

H Lee, HD Yang, SG Ji, SK Cha - 2023 30th Asia-Pacific …, 2023 - ieeexplore.ieee.org
… undirected fuzzers [10], [23], [31], [38], there has been no prior work for evaluatingbug
synthesis techniques can be used for evaluating DGF, and those evaluation results with synthetic

Synthetic Students: A Comparative Study of Bug Distribution Between Large Language Models and Computing Students

S MacNeil, M Rogalska, J Leinonen, P Denny… - Proceedings of the …, 2024 - dl.acm.org
… To evaluate our research questions, we conducted two comprehensive studies. In the first …
-of-the-art LLM, to generate synthetic bugs. To address a lack of information about the precise …

On distribution shift in learning-based bug detectors

J He, L Beurer-Kellner… - … conference on machine …, 2022 - proceedings.mlr.press
bug detector in two phases, first on a synthetic bug distribution to adapt the model to the bug
detection domain, and then on a real bug … Note that we do not evaluate on synthetic bugs as …

BegBunch: benchmarking for C bug detection tools

C Cifuentes, C Hoermann, N Keynes, L Li… - Proceedings of the 2nd …, 2009 - dl.acm.org
… harness by evaluating it with our own Parfait and three publicly available bug detection tools
… For completeness, we collated the synthetic bug benchmarks from Kratkiewicz [6] and the …

Bug mapping and fitness testing of chemically synthesized chromosome X

Y Wu, BZ Li, M Zhao, LA Mitchell, ZX Xie, QH Lin… - Science, 2017 - science.org
… to identify bugs during genome synthesis, generalizable to any watermarked synthetic
chromosome… qPCR to evaluate the copy number of genes located in each of the triplicated regions …

Further field evaluation of synthetic herbivore-induced plan volatiles as attractants for beneficial insects

DG James - Journal of chemical ecology, 2005 - Springer
bug, Orius tristicolor, was attracted to traps baited with MeSA, He, Be, and octyl aldehyde (Oa),
and the bigeyed bug… The minute pirate bug, O. tristicolor was attracted to traps baited with …

Deepdebug: Fixing python bugs using stack traces, backtranslation, and code skeletons

D Drain, CB Clement, G Serrato… - arXiv preprint arXiv …, 2021 - arxiv.org
… for the purpose of generating synthetic bugs. We apply these synthetic bugs toward two ends.
First, … We can evaluate such a model directly on bug-fixing, or finetune it on more filtered …