@Lewisrexwood gave a nice talk at EPOC that has me thinking.
Recording my stream-of-consciousness thoughts for posterity. A bit messy. May come back to this, or may close it.
- Assume you have a model
train_sddp(parameter), where we train an SDDP model that depends on some exogenous parameters -- demand distributions, costs, system outages, etc.
- If we have a new out-of-sample parameter, what do we do?
- At the moment we need to train from scratch
- We don't have a way of warm-starting SDDP
- Lewis' solution is to train a bunch of models with different input parameters, and then nearest-neighbour in parameter-space to get an out-of-sample policy to evaluate
- You could do something cleverer, like linearly interpolate between neighbors
- You could add the parameters as state variables, with a "stage-0" that randomised across the distributions of those parameters.
- Training that model lets you interpolate cuts between different parameters, which is more efficient than training a separate model for each parameter
- But parameters as states works only if the parameters appear linearly in the subproblems (RHS, variable bounds)
- We could do Regan's concave state trick for parameters that appear in the objective function
- But not parameters in the A matrix, or if parameters are things about the sample space or probability mass of the random variable
- TODO: we should add a tutorial on this
- Okay, but what else could we do.
- The main issue for re-training is that we spend a lot of time in passes with a bad value function, so we go places that we never end up needing
- We could just seed the training with historical forward pass trajectories...
- One thing I would like to do is explore something completely different that is hopefully much cheaper to solve (approximately)
- Instead of the full value function, we want the value function to be separable in the state variables so we just need to train a uni-variate function for each state
- And perhaps we could parameterise that uni-variate function so we could just training the parameters directly
- Warren will be happy because this would make it a cost-function approximation type algorithm
- We could also just ask the user for fixed water values, $1. $10, $20, $50, etc, and then just learn the offsets of each cut.
- We really want to exploit the structure of the water values. We know they are convex, monotonically decreasing, with nice piecewise linear functions that make water value decisions easy to interpret.
- We don't really care about solving SDDP to "optimality" because there are so many other approximations in it.
@Lewisrexwood here are some links to follow up on:
@Lewisrexwood gave a nice talk at EPOC that has me thinking.
Recording my stream-of-consciousness thoughts for posterity. A bit messy. May come back to this, or may close it.
train_sddp(parameter), where we train an SDDP model that depends on some exogenous parameters -- demand distributions, costs, system outages, etc.@Lewisrexwood here are some links to follow up on: