PEAK is an LD_PRELOAD-based profiler for HPC applications. It profiles selected CPU functions, optional CUDA kernels, memory allocation activity, and JIT-published code without requiring application recompilation.

PEAK is designed for long-running and MPI applications where profiler overhead, safe attach and detach behavior, and reliable final reports matter.

• Profile named functions instead of instrumenting the whole application.
• Preload PEAK around existing Linux binaries without rebuilding them.
• Control profiling overhead by detaching and reattaching selected targets.
• Produce human-readable and CSV reports, including MPI-aware aggregation.
• Optionally profile CUDA kernels, memory activity, and JIT-published symbols.

mkdir -p build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
cmake --build .

PEAK_TARGET=my_function \
LD_PRELOAD="$PWD/src/libpeak.so" \
../target_application

PEAK writes a report to stderr and a CSV profile log using the default prefix ./peak_statslog.

• Linux for the primary LD_PRELOAD runtime path
• CMake 3.9 or newer
• C, C++, and Fortran compilers
• POSIX threads and standard Linux runtime libraries

MPI and CUDA are optional. CUDA profiling requires CUDA Toolkit 11.2 or newer. CMake validates or obtains the required Frida Gum and OTF2 dependencies during configuration.

The following commands intentionally remain compatible with CMake 3.9:

mkdir -p build
cd build
cmake \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX="$HOME/.local" \
  ..
cmake --build .
cmake --build . --target install

Common CMake options:

After installation:

PEAK_TARGET=my_function \
LD_PRELOAD="$HOME/.local/lib/libpeak.so" \
./target_application

Select one or more CPU functions:

PEAK_TARGET=dgemm_,dgemv_ \
LD_PRELOAD=/path/to/libpeak.so \
./target_application

Run under MPI:

mpirun -n 4 env \
  PEAK_TARGET=my_mpi_work \
  LD_PRELOAD=/path/to/libpeak.so \
  ./mpi_application

Keep the final report but suppress routine warnings:

PEAK_TARGET=my_function \
PEAK_VERBOSITY=quiet \
LD_PRELOAD=/path/to/libpeak.so \
./target_application

Set PEAK_TEXT_OUTPUT=0 for per-function CSV-only output. Heartbeat profile/control/risk diagnostics are currently text-only. PEAK_TARGET, PEAK_TARGET_GROUP, and PEAK_TARGET_FILE are merged into one CPU target list.

PEAK is configured through environment variables. The tables below are a compact index; the linked design documents describe the safety-sensitive paths in detail.

MPI finalization and aggregation are deliberately bounded and locally fail-closed. A rank that observes a collective error or timeout stops issuing later MPI teardown calls. After every rank completes its report or transport responsibility, a separate bounded all-rank release gate propagates publication success. Healthy runtimes then hand off to the real PMPI_Finalize(). Intel MPI 2019 skips that handoff by default because its hwloc teardown can crash after PEAK instrumentation; this compatibility path is non-conforming and must be validated with the target launcher. A gate error or timeout permanently disables later teardown MPI calls, even when PEAK_MPI_REAL_FINALIZE=1 was requested. Aggregate CSVs retain base-pPID.csv; multi-rank local fallback files use base-pPID-rRANK.csv (or a sanitized hostname suffix when rank metadata is unavailable) to avoid cross-node PID collisions. In both CSV and text reports, count is the exact total call count, per_thread is the ceiling over active threads, and per_rank/avg/rank is the non-truncated arithmetic mean over all ranks represented by the report. See Physical detach controller for the full output and teardown behavior.

For runtime behavior, accounting, and tuning, see Heartbeat mechanism and runtime policy.

Enabling either startup option may enter PEAK's rich exec handling, which can inspect environments, allocate, trace, and checkpoint and is not async-signal-safe. With both options disabled, array-based wrappers bypass directly to primed libc functions; variadic adapters only perform their bounded stack-based argument scan before calling the corresponding primed libc function.

Before constructor publication on Linux x86_64/aarch64, direct-path exec APIs issue raw kernel calls, while PATH-search APIs perform a bounded stack search whose candidates also use raw kernel calls. Unsupported architectures preserve native exec semantics through libc resolution in this window, so the resolver-free async-signal-safe guarantee does not apply there. Pre-constructor posix_spawn* retains libc resolution on every architecture to preserve native spawn semantics and is not async-signal-safe.

See Exec-chain profiling for supported API families, child-environment precedence, fork safety, and limitations.

See Physical detach controller and Patched Frida Gum before changing safety controls.

See JIT profiling for provider guarantees and code lifetime requirements.

See Runtime dlopen profiling for dynamic target discovery, FFTW first-call behavior, exact exported-entry attribution, queue semantics, module lifetime, and supported boundaries.

PEAK_PAUSE_TIMEOUT and PEAK_SIG_CONT_TIMEOUT configure the legacy cooperative pause and continue path. Strict physical detach does not rely on that path.

Configure and run the local suite with CMake 3.9-compatible commands:

mkdir -p build
cd build
cmake -DBUILD_TESTING=ON ..
cmake --build .
ctest --output-on-failure

MPI, CUDA, strict-backend, and real-runtime JIT coverage depends on the toolchains and host capabilities detected during configuration.

• Heartbeat mechanism and runtime policy: implementation behavior, overhead accounting, and tuning.
• Exec-chain profiling: child-environment behavior, supported exec and spawn APIs, and fork safety limits.
• Physical detach controller: strict transition safety, MPI output, and shutdown behavior.
• Runtime dlopen profiling: synchronous FFTW first-call attachment, asynchronous dynamic targets, and module lifetime.
• JIT profiling: provider guarantees, retry behavior, and metadata lifetime limits.
• Patched Frida Gum: PEAK-specific Gum APIs and patched relocation support.

• Lowercase Fortran procedure names commonly require a trailing underscore, for example fortran_procedure_name_.
• PEAK_TARGET, PEAK_TARGET_GROUP, and PEAK_TARGET_FILE are merged; duplicate entries are handled automatically but are best avoided.
• CUDA profiling includes first-use initialization and kernel warm-up effects. CUDA graphs may be reported as graph execution or as captured launches, depending on how the graph was created.
• JIT targets require runtime metadata; PEAK does not infer names or boundaries from anonymous executable pages.
• Strict detach backend availability depends on platform signal support and, for the helper backend, host ptrace policy.
• Heartbeat control uses process-local measurements. Local-rank scaling is a reattach policy input and reporting diagnostic, and is conservative only when launcher metadata reports the ranks per node correctly. It is not an MPI-wide overhead proof; measured A/B overhead remains authoritative. The current 60-second cooldown is provisional under the linked validation standard.
• MPI output and finalization behavior is runtime-sensitive. MPI aggregation is the default. Reporting uses an all-rank post-publication release gate; non-Intel-MPI-2019 jobs normally return to the real finalizer, while Intel MPI 2019 uses the documented compatibility skip unless explicitly overridden.

If you use PEAK in your research, please cite:

@inproceedings{10.1145/3624062.3624143,
  author = {Wang, Yinzhi and Li, Junjie},
  title = {PEAK: a Light-Weight Profiler for HPC Systems},
  year = {2023},
  isbn = {9798400707858},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3624062.3624143},
  doi = {10.1145/3624062.3624143},
  booktitle = {Proceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis},
  pages = {677--680},
  numpages = {4},
  keywords = {application performance, profiling, system tools},
  location = {Denver, CO, USA},
  series = {SC-W '23}
}

Contributions are welcome through GitHub issues and pull requests. Before opening a pull request, build PEAK locally, run the relevant CTest coverage, update user-facing documentation when behavior changes, and keep the change focused enough to review independently.

PEAK is distributed under the BSD 3-Clause License.