benchmark-cache

A pip-installable Python package that benchmarks function calls across parameter combinations and stores results in a local SQLite cache file.

pip install -e .

from benchmark_cache import Benchmark


def add(a, b):
    return a + b

bench = Benchmark("cache.db")
bench.run(
    func=add,
    parameters={"a": [1, 2, 3], "b": [4, 5, 6]},
    n_iterations=10,
    parallel=False,
)

# Process results using Pandas
df = bench.dataframe()
print(df.head())

If the process crashes and you run it again with the same function + parameters + iteration count, already completed executions are skipped.

Each benchmark_runs row contains:

• function_name
• params_json
• param_<param_name> for each parameter when the value is a string/number (otherwise null)
• result_json
• result
• duration_seconds
• iteration
• started_at_utc
• success
• error_type
• error_message