Jump to text
On this page
Copy and source
Published source confirmed at last check

Source changed 2026-11-18 21:11:39 UTC · snapshot created 2026-11-18 21:12:35 UTC · last check 2026-11-18 22:15:20 UTC

G.5:11 - SoTA-Echoing — return what the receiving use can actually select

Practice question. What should a dispatcher hand over when several candidates remain useful but the declared comparison does not justify one winner? G.5 returns the admissible alternatives with their ordering status and basis. A single-winner procedure chooses one best candidate under a declared objective and returns it ready for use. That procedure is efficient when the objective settles the choice; it loses a needed alternative when an undeclared scalar objective is substituted for an unresolved trade-off.

The scikit-learn GridSearchCV documentation, refit and best_estimator_, supplies the concrete default. It can choose one estimator by a scorer or a custom rule over the evaluation results; multi-metric evaluation still needs a specified refit choice. Adopt that explicit-choice requirement when G.5 emits an ordered result. Reject reading that best-estimator interface as a warrant to invent the missing comparator. The library supports custom choices and exposes other candidate results; it does not require G.5 to discard them.

PS-AAS, Kostovska et al. (2023) supplies a substantive alternative-selection line: form a complementary portfolio for later algorithm selection, balancing coverage with the cost of a larger choice problem. Its experiments concern specified CMA-ES variants and BBOB tasks, not arbitrary Methods. Adapt the distinction between forming an eligible pool and selecting a member to Shortlist and RankedShortlist in §4.4b, S3 and the pump case in §0.5. This use of PS-AAS concerns result declaration. Its experiments do not establish a best portfolio algorithm for every task.

A different receiving use combines members. Split-Ensemble, Chen et al. (2024) supplies a concrete counterexample to treating every retained set as alternatives: its submodels serve complementary subtasks in one ensemble. Adapt that use distinction to JointUseSet and G.5-6 DeclareSetResult. G.5 states all-member inclusion; the paper’s trained ensemble does not establish compatibility or effective combined use for an arbitrary set of framework editions. Show 4 therefore keeps those claims with their own patterns.

In §0.5 both grounded pump Methods meet the task constraints, but no admitted comparator orders them. With the same candidate evidence, returning the pair as an unordered Shortlist preserves the receiving choice at the cost of one further decision. Naming the spectral Method “best” would require an additional criterion and its supporting comparison. If that criterion is supplied and actually orders the candidates, a ranked result is available. If the receiving use includes every exact member, declare that different membership meaning; a ranking does not establish it. These distinctions govern the outcome declarations and the positive, ranked, no-survivor and joint-use cases in §5.

Reopen when new evidence changes eligibility, a comparator supplies or defeats an ordering, the downstream use changes from alternatives to joint inclusion, or keeping the retained set costs more than the receiving decision can bear. A bounded handoff or abstain outcome remains available. None of these sources establishes actual selection Work, a Method’s identity, public availability or assurance merely from the emitted result.