-
Notifications
You must be signed in to change notification settings - Fork 213
Error stepper action reorder #2862
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error stepper action reorder #2862
Conversation
| } | ||
|
|
||
| if constexpr (Metavariables::local_time_stepping and | ||
| not SelfStart::is_self_start_procedure<ActionList>) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't love the is_self_start_pro... because subcell will likely wreak a lot of havoc in self starting (this was the case in my previous implementation where I never got it far enough to actually fully work, though I'm hopeful I can simplify it a lot this time around). Couldn't this check be done via runtime information from either the time stepper and/or the slab? E.g. I thought all slabs during self-start are before (in the sense of the direction that time flows) the initial time. That seems a lot less likely to break and causes less subtle coupling between pieces of code that might check if they are in self-start. I also don't think you can do LTS until a certain amount of time after self-start because the history is non-monotonic, but @wthrowe will need to say what the restrictions are there.
(I haven't looked at the rest of the code, I just wanted to see the changes in ComputeTimeDerivative)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The sanctioned method of determining you are in self-start is checking if the slab number is negative, but I'm not sure why you want to conditionally take a step anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here, the point is that we can only make the stepping a subroutine of the ComputeTimeDerivative during the main evolution and not during self-start, because self-start needs to replace the RecordTimeStepperData with it's own looping mechanism. During self-start, the action list explicitly contains the UpdateU and RecordTimeStepperData actions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also don't like that this adds another level of nested conditionals into the Metavariables. I'd generally prefer going the other direction. Maybe we can discuss this at the call today (Wed. Feb. 10) if both @wthrowe and @moxcodes can make it. DG-subcell needs even more jumping around in the action list (roughly), and making the action list conditional on self-starting will probably mean I have to rip out the error adjustment to get DG-subcell to work.
wthrowe
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've only skimmed most of this. Please explain what you are trying to do.
| step_actions, | ||
| tmpl::conditional_t< | ||
| local_time_stepping, | ||
| Actions::ChangeStepSize<step_choosers>, tmpl::list<>>, | ||
| step_actions, Actions::AdvanceTime>>>>>; | ||
| Actions::AdvanceTime>>>>>; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't this break the communication of the step size?
| /// Whether a change in the slab size is permitted -- this is to be evaluated | ||
| /// immediately after the time step id has been advanced. | ||
| template <typename Vars, typename DerivVars> | ||
| bool can_change_slab_size( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should never need to forbid a slab size change (except during self-start, but that's special). What are you trying to do here? The current behavior is required for observations without dense output.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here I'm just trying to preserve the current behavior in ChangeSlabSize -- because in my suggested strategy the ChangeStepSize must now happen after the step is evaluated by update_u (and the slab is still changed before the update), the time step id is now different for the case when the step can change as compared to when the slab can change.
edit: I suppose that, because the determination in ChangeSlabSize is exclusively for determining whether or not we're in self-start, I could entirely remove the can_change_slab_size in favor of a runtime function independent of stepper that determines whether the time steps are ordered (so, sufficiently far after self-start).
src/Time/Actions/ChangeSlabSize.hpp
Outdated
| // matches the previous step value | ||
| const auto& step_controller = db::get<Tags::StepController>(box); | ||
| new_step = step_controller.choose_step(new_time_step_id.step_time(), | ||
| current_step.value()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is susceptible to roundoff error causing step-size fluctuations. I suggest preventing the slab size from increasing rapidly instead of trying to deal with the fallout when it does. (There is already code to do this: add the "Increase" StepChooser to your input file.)
src/Time/TakeStep.hpp
Outdated
| typename DbTags, typename Metavariables> | ||
| void take_step(const gsl::not_null<db::DataBox<DbTags>*> box, | ||
| const Parallel::GlobalCache<Metavariables>& cache, | ||
| const VariablesTag /*meta*/ = VariablesTag{}) noexcept { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do you need this as an argument? Doesn't the template parameter work on its own?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're right -- it's a better interface to just always specify the template argument, I'll omit the extra argument.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ping
| } | ||
|
|
||
| if constexpr (Metavariables::local_time_stepping and | ||
| not SelfStart::is_self_start_procedure<ActionList>) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The sanctioned method of determining you are in self-start is checking if the slab number is negative, but I'm not sure why you want to conditionally take a step anyway.
|
So the main goal here is to get the step change to be between the update u and the advance time. Perhaps I have made the changes more confusing than they need to be in the process of trying to chunk them out into reasonably intuitive commits:
Does that make my intent more clear? Or is there something else I'm missing that still seems poorly motivated? |
7e617ae to
f32a6d7
Compare
|
This has been updated now that the other time-stepper utilities have been merged. It no longer makes a distinction between the self-start procedure and the main evolution. |
src/Time/StepChoosers/Cfl.hpp
Outdated
| // Reject the step if the CFL condition is violated. In the case of CFL | ||
| // violation, ensure that the step diminishes slightly, to avoid the | ||
| // possibility of roundoff-scale fluctuations causing a retry with the same | ||
| // rejected step-size. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand this. The rejection condition is step_size < last_step_magnitude, so how could there be a retry with the same last_step_magnitude?
src/Time/Actions/ChangeStepSize.hpp
Outdated
| : std::numeric_limits<double>::infinity(); | ||
| const double last_step_size = history.size() > 0 | ||
| ? db::get<Tags::TimeStep>(*box).value() | ||
| : std::numeric_limits<double>::infinity(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's reasonable to always use db::get<Tags::TimeStep>(*box).value(), unless it might not be set to something sane on the first step.
| const evolution_less_equal<Time> less_equal{time_id.time_runs_forward()}; | ||
| return history.size() == 0 or | ||
| (less(history.back(), time_id.step_time()) and | ||
| (less_equal(history.back(), time_id.step_time()) and |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is this change about?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the case of a length 1 history, the step size change should be permitted when the time step id is the same as the last in the history, because the step change now occurs before the advance time. The alternative, I suppose, would be to keep the same comparison operator but check the next time id instead of the current time id.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This check is supposed to determine the "local direction of time flow". If the two times are equal then something is wrong. The goal of the check is to reject the case where, e.g., the previous steps were at times 1, 2, 3 and the next step will be at 0 (because the is_sorted check will pass there).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. I think the way to keep this condition consistent with previous cases is then to just pass in the next time step (Tags::Next<Tags::TimeStepId>) instead of the current one (Tags::TimeStepId) and then I can revert the changes to the contents of the check.
| id = stepper.next_time_id(id, slab.duration() / 2); | ||
| if (id.substep() != 0) { | ||
| history.insert(id, 0.0, 0.0); | ||
| CHECK(not stepper.can_change_step_size(id, history)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is this change about? This is testing the behavior of substep integrators, but you're not changing any of the substep integrators.
| const bool forward_in_time, const Time& initial_time, | ||
| const TimeDelta& initial_time_step, const size_t order, | ||
| const double initial_value) noexcept { | ||
| const TimeDelta& initial_time_step, const TimeDelta& initial_next_time_step, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does it ever make sense for these to be different?
src/Time/TakeStep.hpp
Outdated
| typename DbTags, typename Metavariables> | ||
| void take_step(const gsl::not_null<db::DataBox<DbTags>*> box, | ||
| const Parallel::GlobalCache<Metavariables>& cache, | ||
| const VariablesTag /*meta*/ = VariablesTag{}) noexcept { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ping
| tmpl::list<::dg::Tags::Formulation, evolution::Tags::BoundaryCorrection< | ||
| typename Metavariables::system>>, | ||
| tmpl::list<::dg::Tags::Formulation>>; | ||
| tmpl::list<evolution::dg::Tags::BoundaryCorrectionAndGhostCellsInbox< |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Something went wrong with the formatting here. I don't know what the "correct" format is, but this line and the previous should line up.
| tmpl::conditional_t< | ||
| local_time_stepping, | ||
| tmpl::list<Actions::MutateApply<boundary_scheme>>, | ||
| tmpl::list<Actions::MutateApply<boundary_scheme>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Move the common action outside the conditional. Also for all the other executables.
tests/Unit/Time/Test_TakeStep.cpp
Outdated
| const TimeDelta time_step = slab.duration() / 2; | ||
| std::unique_ptr<TimeSequence<double>> times{ | ||
| std::make_unique<TimeSequences::Specified<double>>( | ||
| std::vector<double>{0.005, 0.0075, 0.00875, 0.01})}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand why this test passes. I think the floating point error in these values should cause the step to drop compared to the analytic value. (Because 0.0075 - 0.005 < 0.01 / 4)
|
fixups posted. |
wthrowe
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The fixups look fine, but I still don't understand the test.
| take_step<step_chooser_list>(make_not_null(&box), cache); | ||
| // check that the state is as expected | ||
| CHECK(db::get<Tags::TimeStepId>(box).substep_time().value() == 0.005); | ||
| CHECK(db::get<Tags::Next<Tags::TimeStep>>(box) == TimeDelta{slab, {1, 8}}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand why this is the outcome. We've just taken a 1/4 step from 0.005 to 0.0075, and I think this line means we will next take a 1/8 step, which will put us at 0.00875. But one of the goals was 0.00751, which we've then missed.
1e436a1 to
48a3f5d
Compare
|
Thanks for pointing that out, @wthrowe -- Yes, it turns out the |
|
ach, nevermind, now it's broken for slabs... |
48a3f5d to
41e4795
Compare
|
Okay, I've added a warning to |
wthrowe
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test looks good now. Go ahead and squash.
| /// changing immediately is inefficient, it may be best to use | ||
| /// triggers to only activate this check near (within a few slabs of) | ||
| /// the desired time. | ||
| /// \warning This step chooser should be used only to choose slabs, not steps in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good warning. We generally prevent this in the metavariables, but having the reason in the documentation is good.
|
|
||
| # Executable: EvolveBurgersStep | ||
| # Check: parse;execute | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Drop change.
|
squashed (and erroneous change dropped). |
7c237ec to
174e489
Compare
tests/Unit/Time/Test_TakeStep.cpp
Outdated
| CHECK(db::get<Tags::Next<Tags::TimeStep>>(box) == TimeDelta{slab, {1, 2}}); | ||
| CHECK(db::get<EvolvedVariable>(box) == initial_values * exp(0.005)); | ||
| CHECK(db::get<Tags::dt<EvolvedVariable>>(box) == | ||
| 1.0e-2 * initial_values * exp(0.0025)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CHECK_ITERABLE_APPROX to avoid trouble with #2984.
174e489 to
569f5f9
Compare
|
Iterable approx squashed in. |
nilsdeppe
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another that I wasn't sure where to comment on that wasn't clear: is the CFL chooser now choosing based only on the volume step? Normally the CFL condition depends on the char speeds at t^n, and here it looks like it's chosen based on the volume vars at t^{n+1}. I don't know how much this matters, or if my understanding of the code is correct.
src/Time/Actions/UpdateU.hpp
Outdated
| // IWYU pragma: no_forward_declare db::DataBox | ||
| /// \endcond | ||
|
|
||
| /// Perform variable updates for one substep |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should this be (sub)step? Since I guess for LMM steppers it updates for one step?
src/Time/Actions/ChangeStepSize.hpp
Outdated
| const auto new_step = | ||
| step_controller.choose_step(time_id.step_time(), desired_step); | ||
| const auto new_step = step_controller.choose_step( | ||
| db::get<Tags::Next<Tags::TimeStepId>>(*box).step_time(), desired_step); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not use the next_time_id variable you defined on line 47 above?
| detail::vars_to_save<System>, ::Tags::TimeStep>>::simple_tags; | ||
| using simple_tags = typename StoreInitialValues< | ||
| tmpl::push_back<detail::vars_to_save<System>, ::Tags::TimeStep, | ||
| ::Tags::Next<::Tags::TimeStep>>>::simple_tags; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you update the doxygen that now Next<TimeStep> is also saved and why it's saved?
| HEADERS | ||
| BoundaryHistory.hpp | ||
| EvolutionOrdering.hpp | ||
| History.hpp |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Commit message "use in outside actions" remove in or something. I'm having trouble parsing this because it makes "outside Actions" sound like a noun, and I don't know what those are
| /// updating the evolved variables and step size. | ||
| /// | ||
| /// This function is used to encapsulate any needed logic for updating the | ||
| /// system, and in the case for which step parameters may need to be rejected |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand where the step unwinding is happening. I was expecting a loop inside the take_step function surrounding the update_u and change_step_size calls, but that appears to be absent. What am I missing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, yes, good catch -- I hadn't added the loop yet because in the initial version, none of the steppers were able to reject a step, but now CFL can, so this should be updated.
|
Regarding the CFL condition -- strictly it's now actually checked twice. That might be a somewhat stronger condition than necessary, I'm not certain whether the 'before and after' condition should be any more reliable, but it does certainly prevent drama at slab changes. |
|
Okay, that sounds reasonable. We can adjust later if necessary, but that sounds like a good way to start. Thanks! |
4ccd26a to
2966cb8
Compare
2966cb8 to
709cc00
Compare
|
Posted fixups as well as an additional commit |
nilsdeppe
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, one dox request you can squash in immediately :)
| template <typename StepChooserRegistrars, typename DbTags, | ||
| typename Metavariables> | ||
| void change_step_size( | ||
| bool change_step_size( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
while squashing, could you add to the dox what the returned bool means?
This permits the stepper procedure to be called as a subroutine of other actions or functions
It's useful for testing other time utilities that need a prepared history for AdamsBashforthN
For ease of use in actions which need to invoke the stepper as a subroutine.
This is needed to support step-unwinding as a subroutine of `ComputeTimeDerivative`. The ordering of the time-stepper process ensures that the step unwinding must happen after the volume time-derivative terms are computed, but before the fluxes are sent to neighbors, so must happen as a subroutine of `ComputeTimeDerivative`.
1beb71d to
f9f6182
Compare
|
Squashed. |
wthrowe
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Squash directly.
src/Time/Actions/ChangeStepSize.hpp
Outdated
| tmpl::index_if< | ||
| ActionList, | ||
| tt::is_a_lambda<Actions::UpdateU, tmpl::_1>>, | ||
| tmpl::no_such_type_>) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tmpl::none<ActionList, tt::is_a_lambda<Actions::UpdateU, tmpl::_1>> or store the result of the index_if and use it again below.
f9f6182 to
b618efe
Compare
|
Squashed in the suggested change. Cheers! |
src/Time/Actions/ChangeStepSize.hpp
Outdated
| tmpl::_1>>::value) { | ||
| ERROR( | ||
| "Step not successful, and there is no UpdateU action to return " | ||
| "to."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One last question for my understanding: What is this case supposed to handle? If the run is going to die because the chosen step size is a bit off, should it at least give some information about what is wrong? Do we really want to abort in this case instead of continuing with a warning or something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is intended to flag a malformed action list - effectively, what I'm trying to communicate is that if the ChangeStepSize action is used, the UpdateU action must also be used to permit the action-list control flow (instead of stepping as a subroutine e.g. in ComputeTimeDerivatives)
But, now that you point it out, I see that this would be better handled with a static assert, so I'll squash that in too.
When a step is rejected and using `take_step`, the step is retried with new step values until the step is accepted.
b618efe to
7d5c23b
Compare
|
Squashed in converting the action check to a static_assert, and minor changes to the corresponding test to pass the static assert. |
Proposed changes
This factors time stepper routines into free functions so that they can be performed as a subroutine of
ComputeTimeDerivative, and perform the alteration of the order of operations in the local-time-stepping case to support step unwinding -- to be able to reject and re-try a step based on information from a step chooser, the step chooser must run after the time-stepperupdate_uand before the component sends flux information to the neighbors, so the entire loop must be placed as a subroutine ofComputeTimeDerivative.Upgrade instructions
Be sure to update any local time-stepping action lists to match the control flow in the
EvolveScalarWave.hppin this PR.Note that when using local time-stepping and not in self-start, the
ComputeTimeDerivativeaction now performs the steps previously done byRecordTimeStepperData,UpdateU, andChangeStepSize.Those actions are still available for cases in which either local time-stepping is not used or
ComputeTimeDerivativeis not used.Code review checklist
make docto generate the documentation locally intoBUILD_DIR/docs/html.Then open
index.html.code review guide.
bugfixormajor new featureif appropriate.