anti-android-virus (aav) — Android malware static detection engine

aav (short for Anti-Android-Virus) is a static (non-emulating) detection engine for Android malware. It parses DEX bytecode (and the classes.dex inside an APK) and matches it against a compact, multi-dimensional signature database — class-path signatures, opcode-sequence and string/operand-sequence CRCs, an opcode bitmap pre-filter, and AND/OR/XOR/NOT logic combinations — to decide whether a file is malicious and, if so, which family it belongs to.

Its detection method comes from the Master of Engineering thesis "An Efficient Android Malware Static Detection System" — docs/Thesis.md gives the full method, algorithms, architecture and evaluation, and docs/Architecture.md shows how the design maps onto the code.

Status: research / educational. The bundled sample database only detects a synthetic sample produced by sigtool; it is not a real-world malware feed.

file or directory
  │  FileId: DEX, ZIP/APK, or unknown?  (from magic)
  ▼
┌── ZIP/APK ─► ApkScanner: unpack classes*.dex (miniz), scan each ─┐
│                                                                  │
└── DEX ───────────────────────────────────────────► DexScanner ◄─┘
                                                          │
   DexFile   parse header, id tables, classes, methods, code items
                                                          │
   DexCode   per method: opcode buffer + operand/string buffer (+ CRC32s)
                                                          │
   opcode bitmap pre-filter ─► DexSigMgr: path / opcode-CRC / operand-CRC
                               + AND/OR/XOR/NOT logic ─► matched sig IDs
                                                          │
                                                          ▼
                     ScanReport (path, is_malware, sig IDs + names)

The engine takes a file or an in-memory image, identifies it, and — for a DEX or the classes*.dex inside an APK — reduces each method to compact features (class path, opcode sequence, referenced strings) that it matches against the signature database. That matching is engineered to be cheap: an Aho–Corasick trie over path-segment CRCs, binary search over sorted code-CRC tables, and an opcode bitmap that skips methods no signature could match keep a whole-file scan close to O(n · log S) (for n methods and S signatures) instead of the O(n · S) of linear matchers — the source of the speed reported in docs/Benchmarks.md. Full walkthrough: docs/Architecture.md; the interface hierarchy that makes files and memory interchangeable is in docs/ObjectModel.md.

• DEX static analysis — a hardened, bounds-checked DEX parser (fuzzed under ASan/UBSan) that walks classes, methods and code items. Handles DEX versions 035–040 (Android 1.0 through 10+), including the modern method-handle / invoke-custom opcodes (DEX 038/039).
• Multi-dimensional matching
  • class-path signatures via an Aho–Corasick trie over per-segment CRC32s;
  • an opcode bitmap pre-filter to skip irrelevant methods cheaply;
  • opcode-sequence and operand(string)-sequence CRC signatures;
  • logic combinations (AND/OR/XOR/NOT) over code CRCs to confirm a hit.
• APK support — extracts every multidex member (classes.dex, classes2.dex, …) from the zip container (via vendored miniz), scans them all and merges the hits.
• Self-contained tooling — sigtool synthesizes a consistent sample DEX and a matching encrypted/compressed signature DB, so the whole pipeline runs and tests end-to-end with no external assets.
• Modern C++17 — RAII ownership (aav::ObjPtr), factory functions, no hand-rolled reference counting, cross-platform on Linux, macOS, and Android.

• CMake ≥ 3.21
• A C++17 compiler (GCC ≥ 9, Clang ≥ 10, or Apple Clang)
• zlib (zlib1g-dev on Debian/Ubuntu; preinstalled on macOS)
• (optional) Clang with the libFuzzer + sanitizer runtimes, to build the fuzzers

APK/zip support (miniz) is vendored under third_party/ — no extra dependency.

Install everything (compilers, CMake, zlib, clang-format, gcovr, LLVM) in one step: scripts/install-deps-ubuntu.sh (Debian/Ubuntu) or scripts/install-deps-macos.sh (macOS).

# 1. Configure + build (Debug, with the unit/e2e tests enabled)
cmake --preset debug -DAAV_BUILD_TESTS=ON
cmake --build ../anti-android-virus-build/debug -j

# 2. Generate a self-consistent sample DEX + signature DB
../anti-android-virus-build/debug/bin/sigtool gen-sample samples

# 3. Scan the sample: <signature-db> <apk|dex file|dir>
../anti-android-virus-build/debug/bin/aavscan samples/sample.sig samples/sample.dex

Expected output:

file: samples/sample.dex
  isMalware: 1  isWhite: 0
  sigID: 1002  Trojan!SampleFam.a@Android.Dex
  sigID: 1001  Trojan!SampleFam.a@Android.Dex
scanned 1 file(s), 1 flagged, 0.000s

aavscan accepts a single file or a directory (it walks the directory and scans every *.apk / *.dex it finds), plus a few optional runtime flags:

• --debug — verbose engine diagnostics (stderr on host, logcat on Android), off by default so real-time scanning stays fast.
• --analysis — after scanning, print every method's opcode/operand CRC32s and referenced strings. This is the data used to author new signatures; off by default because it records every method (no bitmap pre-filter), so it is slower.
• --mt <threads> — scan a directory across <threads> worker threads (default 1 = sequential). Only directory scans are parallelized; a single file is always scanned on the calling thread. Reports are still delivered one at a time, so output matches a sequential scan apart from ordering.

Full usage: aavscan [--debug] [--analysis] [--mt <threads>] <signature-db> <apk|dex file|dir>.

The engine is driven through one small facade, aav/engine_interface.h (plus aav/object_interface.h, the IObject::Destroy() base). This is the entire public API, and it is deliberately ABI-clean: only PODs, C strings and a callback cross the boundary — no std:: containers or smart pointers — so a prebuilt libaav.so stays compatible across compiler/stdlib versions. File identification, signature-DB loading, scanner selection, APK unpacking and directory walking are all hidden behind it:

#include "aav/engine_interface.h"

static void on_report(const aav::ScanReport* r, void* user) {
  if (r->is_malware) {
    // r->path, r->sig_ids[0..sig_count), r->names[...], and r->classes[...] when
    // analysis is enabled -- all engine-owned, valid only during this call.
  }
}

aav::IEngine* engine = aav::MakeEngine();
aav::EngineConfig config;  // scan_apk / scan_dex / recurse_dirs / verbose /
                           // analysis / scan_threads (>1 parallelizes dir scans)
engine->Init("samples/sample.sig", &config);

engine->Scan("path/to/file-or-dir", on_report, nullptr);  // file OR dir; APKs unpacked
// ...or scan an image already in RAM (no file on disk); apk/dex auto-detected:
engine->ScanBuffer(bytes, size, "app.apk", on_report, nullptr);

engine->Destroy();  // release the engine (never `delete` it)

Link against the engine with target_link_libraries(app PRIVATE aav::aav) (static) or aav::shared (shared). See Install / SDK to consume it as a packaged SDK via find_package(aav).

Prefer one-liners? scripts/ wraps the commands above:

scripts/build.sh     # configure + build (engine, aavscan, sigtool)
scripts/test.sh      # build + unit + e2e tests
scripts/run.sh       # generate a sample and scan it end-to-end
scripts/asan.sh      # sanitizer build + tests
scripts/format.sh    # clang-format check (--fix to apply)
scripts/tidy.sh      # clang-tidy check   (--fix to apply)
scripts/docs.sh      # Doxygen API reference (--open shows it)

The last three are each a CI job that fails the build, so running them before a pull request is what keeps it green. See scripts/README.md for the full set (fuzzing, Android, clean).

Configure presets (CMakePresets.json): debug, release, asan, coverage, fuzz.

The engine ships as an SDK — public headers plus a static and a shared library — installable to any prefix:

cmake --preset release && cmake --build ../anti-android-virus-build/release -j
cmake --install ../anti-android-virus-build/release --prefix /path/to/sdk

which lays out:

/path/to/sdk/
├── include/aav/      # public headers: engine_interface.h, object_interface.h, aav.h
└── lib/
    ├── libaav.a                   # static
    ├── libaav.so / libaav.dylib   # shared (self-contained: miniz built in)
    └── cmake/aav/                 # find_package(aav) config

Consume it from another CMake project:

find_package(aav REQUIRED)
target_link_libraries(myapp PRIVATE aav::aav)     # static
# or:  target_link_libraries(myapp PRIVATE aav::shared)

# Unit + end-to-end tests
cmake --preset debug -DAAV_BUILD_TESTS=ON && cmake --build ../anti-android-virus-build/debug -j
ctest --test-dir ../anti-android-virus-build/debug --output-on-failure

# AddressSanitizer + UndefinedBehaviorSanitizer
cmake --preset asan -DAAV_BUILD_TESTS=ON && cmake --build ../anti-android-virus-build/asan -j
ctest --test-dir ../anti-android-virus-build/asan --output-on-failure

Fuzzing the DEX parser (requires a Clang toolchain with libFuzzer):

FUZZ_TIME=60 scripts/fuzz.sh          # build and run
scripts/fuzz.sh --build-only          # just check the harness still compiles

CI (GitHub Actions) is split by cadence rather than by topic, one workflow per kind of failure:

Every step is a preset or a script, so a red job reproduces locally with the one command it ran — ctest --preset release, scripts/tsan.sh — rather than by transcribing flags out of a workflow file. Every check has a fixed name — the build workflow reports one per platform and iterates the compilers and build types inside it — so branch protection names them directly:

Lint / clang-format          Sanitizers / ASan + UBSan    Analysis / Coverage
Lint / clang-tidy            Sanitizers / TSan            Analysis / Fuzz build
Build / Ubuntu               Android / engine (arm64-v8a) Docs / build
Build / macOS                Android / app (debug APK)

Docs / deploy is the one job that does not gate a pull request: it only publishes from main, and is the only job anywhere that holds pages: write.

CI builds the fuzz harness but does not run it: a seedless run explores different input every time, so gating a pull request on it would fail changes for findings they did not introduce. What does gate is that the harness still compiles against the engine API. Running it is a developer's call — scripts/fuzz.sh, or FUZZ_TIME=900 scripts/fuzz.sh for a longer session, which writes any crashing input under the fuzz build tree.

.
├── CMakeLists.txt            # root: options, deps, install; delegates to subdirs
├── CMakePresets.json         # debug / release / asan / coverage / fuzz presets
├── include/aav/              # public SDK: engine_interface.h + object_interface.h + aav.h
├── src/
│   ├── engine/               # the IEngine facade implementation
│   ├── api/aav/              # internal object API (interfaces, factories, data records) — not exported
│   ├── platform/             # file/memory primitives (FileStream, FileTarget, FileMap, MemTarget)
│   ├── utils/                # crc32, leb128, blowfish, gzip inflate, logger
│   ├── sig/                  # signature-DB load/decrypt/decompress, format
│   ├── dex/                  # DEX parser + path/opcode/operand/logic matchers
│   └── scan/                 # file-type id (FileId) + APK (zip) white-list check
├── apps/
│   ├── aavscan/              # CLI scanner (thin facade consumer)
│   └── sigtool/              # sample DEX + signature-DB generator
├── android/                  # Gradle + NDK app; JNI bridge over the engine
├── tests/
│   ├── unit/                 # doctest white-box unit tests (one binary)
│   └── e2e/                  # generate-and-scan end-to-end CTest drivers
├── fuzz/                     # libFuzzer harness for the DEX parser
├── third_party/              # vendored: miniz (zip), doctest
├── scripts/                  # build.sh, test.sh, asan.sh, coverage.sh,
│                             # format.sh, tidy.sh, docs.sh, fuzz.sh, android*.sh
└── docs/                     # guides — see docs/README.md for the index
    └── doxygen/              # Doxyfile + landing page for the API reference

Full index: docs/README.md.

The generated API reference renders the same comments the interface headers carry, adding the one thing they cannot show: the inheritance diagrams for the IScanObject → IStream/ITarget hierarchy that makes files and memory interchangeable. scripts/docs.sh builds it locally.

The core engine (libaav) and the aavscan CLI cross-compile for Android via the NDK; CI builds the arm64-v8a (ARM64) ABI on every change:

cmake -S . -B ../anti-android-virus-build/android \
  -DCMAKE_TOOLCHAIN_FILE="$ANDROID_NDK_HOME/build/cmake/android.toolchain.cmake" \
  -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-24 \
  -DAAV_BUILD_CLI=OFF -DAAV_BUILD_TOOLS=OFF -DAAV_BUILD_TESTS=OFF
cmake --build ../anti-android-virus-build/android -j

A demo Android app (com.av.aav) that drives the engine on-device via JNI lives under android/; its JNI bridge is a consumer of the public IEngine SDK facade and statically links libaav (no separate engine .so). Build it with cd android && ./gradlew :app:assembleDebug (SDK 36 / NDK 29 / JDK 17); CI assembles the debug APK on every change.

Contributions are welcome — parser hardening, new tests, docs, opcode/version support, and engine features. Please read CONTRIBUTING.md for the build / test / coding-style workflow, and run scripts/test.sh and scripts/asan.sh before opening a pull request.

Licensed under the MIT License — see LICENSE.