Repository navigation
Tags
Tags
Release 1.3 CHANGES: * Build: do not inherit OpenMP flags from the pkg-config files of OpenBLAS, CBLAS or LAPACKE unless `OWL_ENABLE_OPENMP=1`. An OpenBLAS built with OpenMP (Homebrew, FreeBSD) made Owl compile its `#pragma omp` code without linking an OpenMP runtime (undefined `__kmpc_*` symbols, #695) or fail with compilers that reject `-fopenmp` (#684). * Build: work around Apple Clang 21 (Xcode 26) crashing on arm64 while vectorising the `modf` loops of `owl_ndarray_maths_stub.c`: with Clang, `modf` is called through a function that is not inlined. * Build: keep the optional optimisation flags of the default configuration only if the C compiler accepts them without a warning, so that GCC-only flags are dropped with Clang and `-march=native` with compilers that reject it. Flags with GCC are unchanged. * Fix resuming a compiled network (`Owl_neural_compiler.train ~state`): the weights were always initialised again and the optimiser state (gradient history, momentum, Adam's moments) started from zero. `train` and `compile_deep` take `?init_model` (default `true`, as in `Graph.train`); pass `~init_model:false` with `~state` to resume. The optimiser state is now saved in the returned state and restored from it. Training an already compiled network a second time no longer fails in `Graph.init`. * Fix the learning rate of compiled networks (`Owl_neural_compiler.train`): the static graph used the rate of step 999 for every step, so Adam's bias correction was wrong and `Decay`, `Exp_decay` and `Schedule` were constant; Adam also still divided by the gradient. The step-dependent factor is now a scalar graph input set before every step (`assign_learning_rate`), using the new `Learning_Rate.scale` and `Learning_Rate.unscaled_update`. Compiled training now matches eager training step for step. * Fix NaN in cross-entropy when a prediction underflows to 0, e.g. from a saturated softmax: `Algodiff.Maths.cross_entropy` (and so `Loss.Cross_entropy`) and `Ndarray.cross_entropy'` raise predictions below the smallest positive normal float to that value. Results for all other predictions are unchanged bit for bit; NaN predictions still propagate. * Change `Regularisation.L2norm a` to `a * ||w||^2` (it was `a * ||w||`), and the L2 term of `Regularisation.Elastic_net` likewise. The plain norm has a NaN gradient at `w = 0`, so training a network with L2 regularisation turned every parameter into NaN, starting from the zero-initialised biases. `Regression.svm` uses the squared penalty as a result. * Fix `~keep_dims:true` without `~axis` in `sum`, `prod`, `min`, `max`, `minmax`, `mean`, `median`, `var`, `std`, `sem`, `l1norm`, `l2norm`, `l2norm_sqr` and `vecnorm` (and `sum`, `min`, `max` in owl-base and `Algodiff.Maths.sum`): it now keeps every dimension with size 1 instead of returning shape [|1|] (#621). Calls that do not pass `keep_dims` still return [|1|], so default results are unchanged. * Fix `Linalg.det` and `Linalg.logdet` on singular matrices: they return 0 and `neg_infinity` instead of raising `Failure "LAPACKE: i"` (#249). The pure-OCaml `Owl_base_linalg_generic.det` returned wrong values (e.g. -4 for [[1, 2], [2, 4]]) because the LU pivot search reused a stale row when a column was zero; it now returns 0, also for matrices with a zero row. * Fix `Stopping.Const`: `Checkpoint.run` overwrote the stop flag set by the stopping criterion, so training never ended early. It now stops at the first batch whose loss is below the threshold, in `minimise_weight`, `minimise_fun`, `minimise_network` and `minimise_compiled_network`. * Fix Adam: the update no longer divides by the gradient, so elements with a zero gradient keep moving with their first moment and tiny gradients no longer give infinite steps. Applies to `minimise_weight`, `minimise_fun`, `minimise_network` and `Stepper`; other learning-rate strategies are unchanged bit for bit. Adam results with non-zero gradients may differ in the last ulp. New `Learning_Rate.update` returns the weight update. * Fix `Regression`: `ols` and `ridge` are solved directly via the SVD, `lasso` and `elastic_net` by coordinate descent, so results are exact rather than the outcome of 1000 Adagrad steps (#340). Penalties no longer apply to the intercept, and the L2 penalty of `ridge` and `elastic_net` is now the squared norm. `logistic` now models `sigmoid (x w + b)` with targets in [0, 1] and is solved by Newton's method; it used to take the log of the raw linear output. `exponential` uses variable projection and fits growing exponentials (#650). `svm` no longer prints its training log. Documentation now states the actual defaults (`i = false`, `alpha = 0.001`). * Add standalone single-step optimiser `Optimise.Stepper` * Add unit tests for `Optimise` and `Regression` * Fix `exponpow_rvs`: it called the standard Gaussian sampler with the scale as its only argument; it now samples a Gaussian with that scale (#694, @maxkuttner) * Fix the documentation of the `gumbel2` distributions: the second parameter is a shape, not a location (#696, @tristan-f-r) * Publish the documentation from the `gh-pages` branch; the generated HTML is no longer committed in `docs/` * Run the tests in CI on Ubuntu and on macOS (arm64 and Intel)
Release 1.2 CHANGES: * Revise automatic installation test via GitHub Action * Revise dockerfiles; add docker automatic build and deployment * Fix compatibility on MacOS and Arm64 architecture * Update compilation to OCaml 5.2.0 * Revise examples and tests * Update inline document and tutorials * FFT module revamp (@gabyfle) * Changed def of ssqr_diff' to not modify inputs (@patrick-nicodemus) * Avoid calling log(0) when generating gaussian random variables (@edwintorok)
PreviousNext