Description

The current version of SMAC does not support a trust region supported search (BOing/TuRBO). In previous versions it seems that this function was available, but has been discontinued.

Specific Application

In my specific application, I am dealing with a structural design optimization, in which first a configuration has to be chosen (e.g., type of materials). Only once the configuration has been chosen a set of continuous parameters need to be optimized (e.g. area). The objective function is expensive to compute, and thus the evaluation budget is limited.

Next to the issue of conditional parameters, the response function will be non-smooth due to non-linearity within the structure (e.g. crash structures). Therefore, the evaluation will be an expensive black-box function with a hierarchical design space. Due to the hierarchical design space and non-smooth response functions standard GP's struggle, and thus an RF surrogate is used in this case.
In current tests that I have run, it seems that SMAC, with RF as surrogate, has difficulty to exploit promising regions within the continuous variables.

Proposed idea

Update the old version of the BOing/TuRBO system to align with the current implementation of SMAC. Add the functionality to maintain multiple trust regions to explore different categories simultanously.

• A global RF (Random Forest) model is maintained to propose the parameters of the parents.
• $m$ trust-regions are maintained on the children (assuming the children are purely continuous variables).

I would be happy to try to contribute to this feature, if someone familiar with the code setup pointing me in the right direction.

References

[1] Searching in the Forest for Local Bayesian Optimization

[2] Old Boing implementation

[3] TurBO