Skip to content

Add benchmark scripts to scrape timing information from log files #319

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
May 13, 2024
Merged

Add benchmark scripts to scrape timing information from log files #319

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
db38f26
Initial commit: Add script to scrape GEOS-Chem timers
yantosca Apr 30, 2024
c41a33f
Add function "replace_whitespace" in util.py
yantosca May 3, 2024
b258b39
Add benchmark script to scrape GEOS-Chem Classic timing information
yantosca May 3, 2024
4bb4a95
Add script to parse GCHP benchmark timing information
yantosca May 3, 2024
91617b5
Now also scrape GCHPchem timers as well as summary timers
yantosca May 6, 2024
3827302
run_1yr_fullchem_benchmark.py now produces timing table output
yantosca May 7, 2024
6f8e049
Add percent difference column to timing tables
yantosca May 10, 2024
efafa95
Update benchmark scripts & config files to add timing table output
yantosca May 10, 2024
e8e13c0
Add sevderal fixes for benchmark timing tables
yantosca May 10, 2024
d10b09a
More minor fixes for benchmark timing scripts
yantosca May 10, 2024
e0bd4a7
Now use "GC.log" as the GCClassic file name for 1mo_benchmark.yml
yantosca May 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Function `copy_file_to_dir` in `gcpy/util.py`. This is a wrapper for `shutil.copyfile`.
- GitHub Action config file `.github/workflows/stale.yml`, which replaces StaleBot
- Example script `gcpy/examples/working_with_files/make_mask_file.py`

- Convenience function `replace_whitespace` in `gcpy/util.py`
- Benchmark script `gcpy/benchmark/modules/benchmark_scrape_gcclassic_timers.py`
- Benchmark script `gcpy/benchmark/modules/benchmark_scrape_gchp_timers.py`
- 1-year benchmark scripts now produce GCC vs GCC and GCHP vs GCHP timing tables
- Functions `gcc_vs_gcc_dirs`, `gchp_vs_gcc_dirs`, `gchp_vs_gchp_dirs`, and `get_log_filepaths` in `gcpy/benchmark/modules/benchmark_utils.py`

### Changed
- Bump pip from 23.2.1 to 23.3 (dependabot suggested this)
- Bump pypdf from 3.16.1 to 3.17.0 (dependabot suggested this)
Expand All @@ -42,6 +47,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Create radionuclide, STE flux, and mass conservation tables for Ref and Dev versions in TransportTracers benchmarks
- Use new function `copy_file_to_dir` to copy the benchmark script and configuration file to the benchmark results folders
- Updated GitHub stalebot config file `stale.yml` with new issue/PR labels that should not go stale
- Updated benchmark driver scripts and config files to print GCClassic & GCHP timing information

### Fixed
- CS inquiry functions in `gcpy/cstools.py` now work properly for `xr.Dataset` and `xr.DataArray` objects
Expand Down
9 changes: 9 additions & 0 deletions gcpy/benchmark/cloud/template.1hr_benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,13 +47,17 @@ data:
dir: ref-gcc/run-directory
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: .
logs_template: runlog.txt
bmk_start: "2019-07-01T00:00:00"
bmk_end: "2019-07-01T01:00:00"
gchp:
version: ${GEOSCHEM_BENCHMARK_REF_PRIMARY_KEY}
dir: ref-gchp/run-directory
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: .
logs_template: runlog.txt
bmk_start: "2019-07-01T00:00:00"
bmk_end: "2019-07-01T01:00:00"
is_pre_14.0: False
Expand All @@ -64,13 +68,17 @@ data:
dir: dev-gcc/run-directory
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: .
logs_template: runlog.txt
bmk_start: "2019-07-01T00:00:00"
bmk_end: "2019-07-01T01:00:00"
gchp:
version: ${GEOSCHEM_BENCHMARK_DEV_PRIMARY_KEY}
dir: dev-gchp/run-directory
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: .
logs_template: runlog.txt
bmk_start: "2019-07-01T00:00:00"
bmk_end: "2019-07-01T01:00:00"
is_pre_14.0: False
Expand Down Expand Up @@ -117,6 +125,7 @@ options:
ops_budget_table: False
OH_metrics: True
ste_table: True # GCC only
timing_table: True
summary_table: True
plot_options:
by_spc_cat: True
Expand Down
9 changes: 9 additions & 0 deletions gcpy/benchmark/cloud/template.1mo_benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,13 +47,17 @@ data:
dir: ref-gcc/run-directory
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: .
logs_template: runlog.txt
bmk_start: "2019-07-01T00:00:00"
bmk_end: "2019-08-01T00:00:00"
gchp:
version: ${GEOSCHEM_BENCHMARK_REF_PRIMARY_KEY}
dir: ref-gchp/run-directory
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: .
logs_template: runlog.txt
bmk_start: "2019-07-01T00:00:00"
bmk_end: "2019-08-01T00:00:00"
is_pre_14.0: False
Expand All @@ -64,13 +68,17 @@ data:
dir: dev-gcc/run-directory
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: .
logs_template: runlog.txt
bmk_start: "2019-07-01T00:00:00"
bmk_end: "2019-08-01T00:00:00"
gchp:
version: ${GEOSCHEM_BENCHMARK_DEV_PRIMARY_KEY}
dir: dev-gchp/run-directory
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: .
logs_template: runlog.txt
bmk_start: "2019-07-01T00:00:00"
bmk_end: "2019-08-01T00:00:00"
is_pre_14.0: False
Expand Down Expand Up @@ -117,6 +125,7 @@ options:
ops_budget_table: False
OH_metrics: True
ste_table: True # GCC only
timing_table: True
summary_table: True
plot_options:
by_spc_cat: True
Expand Down
9 changes: 9 additions & 0 deletions gcpy/benchmark/config/1mo_benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,13 +47,17 @@ data:
dir: GCC_ref
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: .
logs_template: "GC.log"
bmk_start: "2019-07-01T00:00:00"
bmk_end: "2019-08-01T00:00:00"
gchp:
version: GCHP_ref
dir: GCHP_ref
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: .
logs_template: "gchp.%Y%m%d_0000z.log"
bmk_start: "2019-07-01T00:00:00"
bmk_end: "2019-08-01T00:00:00"
is_pre_14.0: False
Expand All @@ -64,13 +68,17 @@ data:
dir: GCC_dev
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: .
logs_template: "GC.log"
bmk_start: "2019-07-01T00:00:00"
bmk_end: "2019-08-01T00:00:00"
gchp:
version: GCHP_dev
dir: GCHP_dev
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: Logs
logs_template: "gchp.%Y%m%d_0000z.log"
bmk_start: "2019-07-01T00:00:00"
bmk_end: "2019-08-01T00:00:00"
is_pre_14.0: False
Expand Down Expand Up @@ -117,6 +125,7 @@ options:
ops_budget_table: False
OH_metrics: True
ste_table: True # GCC only
timing_table: True
summary_table: True
plot_options:
by_spc_cat: True
Expand Down
9 changes: 9 additions & 0 deletions gcpy/benchmark/config/1yr_fullchem_benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,13 +59,17 @@ data:
dir: GCC_ref
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: Logs
logs_template: "log.%Y%m%d"
bmk_start: "2019-01-01T00:00:00"
bmk_end: "2020-01-01T00:00:00"
gchp:
version: GCHP_ref
dir: GCHP_ref
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: Logs
logs_template: "gchp.%Y%m%d_0000z.log"
bmk_start: "2019-01-01T00:00:00"
bmk_end: "2020-01-01T00:00:00"
is_pre_14.0: False
Expand All @@ -76,13 +80,17 @@ data:
dir: GCC_dev
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: Logs
logs_template: "log.%Y%m%d"
bmk_start: "2019-01-01T00:00:00"
bmk_end: "2020-01-01T00:00:00"
gchp:
version: GCHP_dev
dir: GCHP_dev
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: Logs
logs_template: "gchp.%Y%m%d_0000z.log"
bmk_start: "2019-01-01T00:00:00"
bmk_end: "2020-01-01T00:00:00"
is_pre_14.0: False
Expand Down Expand Up @@ -130,6 +138,7 @@ options:
Ox_budget_table: True
ste_table: True # GCC only
OH_metrics: True
timing_table: True
plot_models_vs_obs: True
plot_options:
by_spc_cat: True
Expand Down
9 changes: 9 additions & 0 deletions gcpy/benchmark/config/1yr_tt_benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,13 +48,17 @@ data:
dir: GCC_ref
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: Logs
logs_template: "log.%Y%m%d"
bmk_start: "2019-01-01T00:00:00"
bmk_end: "2020-01-01T00:00:00"
gchp:
version: GCHP_ref
dir: GCHP_ref
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: Logs
logs_template: "gchp.%Y%m%d_0000z.log"
bmk_start: "2019-01-01T00:00:00"
bmk_end: "2020-01-01T00:00:00"
is_pre_14.0: False
Expand All @@ -65,13 +69,17 @@ data:
dir: GCC_dev
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: Logs
logs_template: "log.%Y%m%d"
bmk_start: "2019-01-01T00:00:00"
bmk_end: "2020-01-01T00:00:00"
gchp:
version: GCHP_dev
dir: GCHP_dev
outputs_subdir: OutputDir
restarts_subdir: Restarts
logs_subdir: Logs
logs_template: "gchp.%Y%m%d_0000z.log"
bmk_start: "2019-01-01T00:00:00"
bmk_end: "2020-01-01T00:00:00"
is_pre_14.0: False
Expand Down Expand Up @@ -115,6 +123,7 @@ options:
mass_table: True
ste_table: True
cons_table: True
timing_table: False
#
# n_cores: Specify the number of cores to use.
# -1: Use $OMP_NUM_THREADS cores
Expand Down
Loading