Skip to content

Documentation: User's configspace defaults are not executed in initial design. #975

@timruhkopf

Description

@timruhkopf

According to #533, if a user makes an effort to specify default choices for the hyperparameters, they are not executed in the initial design. Marius had some reservations about making this the default behaviour, as it might change the benchmarks by user priors (?). Albeit having found an easy workaround in 2.0,

A warning should be marked in the documentation, as it may seem counterintuitive to a user. Also the fix below could be added to this as documentation.

simply add to the <facade>.get_initial_design call (e.g. in this SVM example) the default config as an additional_configs like this:

classifier = SVM()

[....]

 initial_design = HyperparameterOptimizationFacade.get_initial_design(
        scenario,
        n_configs=5,
        additional_configs=[  # to get the default configuration in the initial design
            classifier.configspace.get_default_configuration()
        ]
    )

This will result in SMAC executing the n_configs from the initial design and, subsequently, the default config you specified.

Originally posted by @timruhkopf in #533 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationDocumentation is needed/added.

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions