v0.20.1: 2025/06/12 Trainer.AccumulateGradients (when the batch doesn't fit memory); VNN fixes; Numpy improvements. #184
janpfeifer
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
train:SetLossNoRegularizationandGetLossNoRegularization.Trainer.AccumulateGradients(n)to accumulate n steps of gradients before applying them. This is useful ifthe desired batch size doesn't fit in memory, so it accumulates the gradients until the virtual batch size gradient
is calculated.
optimizers:train.OptimizeWithGradientsinterface, to support gradient accumulators.StochasticGradientDescentAPI. Added option to disable decay for testing.vnn:Config.Scalerto add a scaler operator just after the linear projection of a layer. It allows the VNNto operate on magnitude independent vectors.
LayerNormalization, to make it more stable in backprop.Relu: added support for non-shared non-linearities and a "leak" parameter ("vnn_relu_negative_slope").VNN().ActivationFn()to allow setting arbitrary activation functions.types/tensors/numpy:tensors:And if it has, it panics with a meaningful error message.
This discussion was created from the release v0.20.1: 2025/06/12 Trainer.AccumulateGradients (when the batch doesn't fit memory); VNN fixes; Numpy improvements..
All reactions