Filter and combine result columns#1278
Draft
gnn wants to merge 23 commits into
Draft
Conversation
When calling `results.to_df(variable)`, look for a `filters` attribute on `results`, then look up `results.filters[variable]`, which should be a function, and use that function to filter the `columns` of the `variable`'s results `DataFrame` before returning it. More specifically, the function should accept entries of the `DataFrame`'s `columns` attribute and return something that evaluates to `True` if the column should be included in the results and to `False` if the column should be excluded. Retain all columns, if `results.filters` does not contain the key `variable` and provide a `filters` class attribute on `Results` in order to provide default filters. Add a `filters` argument to the `Results` constructor to allow setting the `filters` attribute on construction.
Don't overwrite the `variables` argument to `Results.to_df` within the method body with another computed value.
Instead of using a `filters` attribute to filter result `DataFrame` columns, use an additional `filters` argument to `to_df` to achieve the same result. Replace `functools.cache` on `to_df` with only caching unfiltered results manually because `functools.cache` would interact badly with the new `filters` argument. This new implementation allows easy usage of different custom filters without instantiating multiple `Results` objects and without constantly recomputing the results `DataFrame`. Note that default filtering is still done via a `filters` attribute.
Create as special `dict` subclass for `Filters` for `Results`. The
subclass as a special `updating` method, which is functionally
equivalent to `dict`'s `update` method, except that it doesn't update in
place, but creates a shallow clone and returns the `update`d clone.
The idea is that this enables combining the default `Results.filters`
easily with custom ones by doing e.g.:
```python
def nosinks(column):
return not isinstance(column[1], Sink)
results.filters = Results.filters.updating({"flow": nosinks})
flows = results["flow"]
```
Note that the code making `Results.filters` a `Filters` instance instead
of a regular `dict` is not properly formatted yet. This is intentional,
in order to make it easier to spot that the code didn't change
semantically beyond adding the `Filters` call. Formatting will be done
in the next commit.
No semantic changes. Just formatting.
Not applying any filtering at all as a default is a special feature of the `Filters` class, so the implementation should be placed there, instead of making it the call site's responsibility to supply that default.
| ) | ||
|
|
||
|
|
||
| def main(optimize=True): |
Check notice
Code scanning / CodeQL
Explicit returns mixed with implicit (fall through) returns Note
|
|
||
| # -------------- Investment Costs --------------------------- | ||
|
|
||
| invest = results.get("invest") |
Check notice
Code scanning / CodeQL
Unused local variable Note
And manually shorten some overly long lines, too.
Quench a `flake8` complaint.
Helps when comparing them between script runs.
The `Filters` class is intended to be a special dictionary holding filter functions as values, so it's `__getitem__` function should return such functions and, consequently, have an appropriate return type.
I added a function "hide_and_rename" to the examples to show what I need. For me it is already fine to have this function but we should think about a way to make the functionality of this function available for others in a simple way (callbacks, filters etc.)
Member
|
I added a function "hide_and_rename" to the examples to show what I need. For me it is already fine to have this function but we should think about a way to make the functionality of this function available for others in a simple way (callbacks, filters etc.) |
| ) | ||
| return filtered | ||
|
|
||
| res_flow = results["flow"] |
Check notice
Code scanning / CodeQL
Unused local variable Note
Since the function is analogous to Python's builtin `sorted`, this naming is more consistent.
Using `DataFrame.loc` instead of `[]` squelches the `SettingWithCopyWarning`s. It's IMHO also a lot more readable than making the reader figure out which of the many behaviours of `DataFrame`'s or `Series`'s overloaded `[]` implementation actually gets used.
It's consistent with `Callbacks.noop` and uses context to avoid redundant information. It should be clear that a `noop` in the context of `Filters` means using no filter, so there's no need to repeat the context in the name of the function.
Cast `Results.get`'s `filters` and `callbacks` arguments to `Filters` and `Callbacks` before using them, thus allowing normal dictionaries to be used when supplying values for these arguments. Among other things, this allows the usage of empty `dict`s as an easy way to disable defaults.
At least in one example. Leave the other one be for now, to illustrate the difference.
This method could be defined (just `pass` inside the method) also with the Node class of oemof.network
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implement a proposal for an API that allows filtering the columns of
DataFrames inoemof'sResultsobjects. Having a stable API would also allow predefining filters enabled by default or filters which are expected to be used often.Also implement a proposal for an API that allows specifying callback functions that manipulate the
DataFrames in different ways than just filtering.