Feature request

Is your feature request related to a problem? Please describe.
I am trying to fine-tune a model with the --upstream-trainable flag, but it seems like there is no way to make the upstream model train with a lower learning rate compared to the downstream expert.

Describe the solution you'd like
In the optimizer settings, we should have an "upstream_lr" option. If this is set to a value, then when defining the optimizer parameters, I would save the upstream layers with their own learning rate, as shown in the per-parameter options of the torch optimizers docs.
A quick implementation I've written is here

Describe alternatives you've considered
None

Additional context
If you think this is a good idea, I can make a PR for it.