Hi @middleyuan,

I've been thinking about this as well, but since the main goal of RLtools is being optimized/fast, most of the hyperparameters are compile-time constants. Non-structural parameters (i.e. that do not impact the sizes of Tensors or loops), e.g. the learning rate, could be a compile-time variables that are loaded from a config file at runtime. But leaving even those as compile-time constants brings the additional benefit that for GPU kernels the parameters are compiled into the stream of instructions and don't require global memory allocations / access, which is slow.

So keeping these compile-time constants and still using config files would involve somehow passing the parameters through CMake or using something like Menuconfig. Another concern I have is that many things are not easily described by a flat parameter, e.g. the network architecture. Trying to fit this into a flat config file can make things quite a bit more complicated.

I think your best bet would be to use the Python bindings rl-tools/python-interface. In this case you can pass the parameters as kwargs and the training code will be just-in-time compiled based on the provided parameter setting. This works by rendering the parameters into templates like this

The only downside is that the flexibility is a bit more limited (due to the mentioned issue wrt. e.g. network architectures)

You can even use C++ environments with the Python interface, in which case the performance is the same as when using the native library.

I hope this helps you and let me know how it goes!

Configuration of hyperparameters #21

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions