Skip to content

Conversation

yantosca
Copy link
Contributor

Name and Institution (Required)

Name: Bob Yantosca
Institution: Harvard + GCST

Confirm you have reviewed the following documentation

Describe the update

This is the companion PR to #285. This fixes the issue described in #285 by making sure that we append function output to the results variable instead of overwriting it. For example, in code blocks such as this:

    # --------------------------------------------
    # Create the plots in parallel
    # Turn off parallelization if n_job=1
    if n_job != 1:
        results = Parallel(n_jobs=n_job)(
            delayed(createplots)(filecat)
            for _, filecat in enumerate(catdict)
        )
    else:
        for _, filecat in enumerate(catdict):
            results = createplots(filecat)
    # --------------------------------------------

we have updated the else block so that the output from the function call when parallelization is turned off is appended to a list rather than ovewritten:

    else:
        results = []
        for _, filecat in enumerate(catdict):
            results.append(createplots(filecat))

Expected changes

This will prevent benchmark plots from failing when n_cores: 1 is specified in the YAML config file.

Related Github Issue(s)

This commit fixes the issue reported by @lizziel in #285.  The "results"
variable was being overwritten instead of appended to when plots are
generated sequentially (i.e. with "n_cores: 1" in the YAML input).

gcpy/benchmark/modules/run_1yr_*_fullchem.py
gcpy/benchmark_funcs.py
gcpy/plot/compare_*.py
- For the case when parallelization is off:
    1. Declare "results" as an empty list
    2. Append the output of the routine being called into "results"
  This will prevent a dictionary key error as described in #285.

Signed-off-by: Bob Yantosca <yantosca@seas.harvard.edu>
gcpy/benchmark_funcs.py
- Removed a leftover print statement and quit() statement
  from debugging

Signed-off-by: Bob Yantosca <yantosca@seas.harvard.edu>
@yantosca yantosca added topic: Benchmark Plots and Tables Issues pertaining to generating plots/tables from benchmark output category: Bug Fix Fixes a bug that was previously reported labels Jan 25, 2024
@yantosca yantosca requested a review from lizziel January 25, 2024 21:20
@yantosca yantosca self-assigned this Jan 25, 2024
@yantosca yantosca linked an issue Jan 25, 2024 that may be closed by this pull request
Copy link
Contributor

@lizziel lizziel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to merge!

Copy link
Contributor Author

Thanks @lizziel!

@yantosca yantosca merged commit a8c8b7d into dev Jan 26, 2024
@yantosca yantosca deleted the bugfix/parallel-plotting branch January 26, 2024 15:30
yantosca added a commit that referenced this pull request Jan 26, 2024
This is the release of GCPy 1.4.2, which includes:

- PR #286 (Github Action for GCPy environment)
- PR #287 (Fix dictionary key error for benchmark plotting)
- PR #289 (Use pegged versions in environment.yml; update GitHub actions

Signed-off-by: Bob Yantosca <yantosca@seas.harvard.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: Bug Fix Fixes a bug that was previously reported topic: Benchmark Plots and Tables Issues pertaining to generating plots/tables from benchmark output
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Plot parallelization off failing in GCPy 1.4.1
2 participants