Desired-balance warn threshold logging should accumulate across restarts

Today we emit periodic INFO logs about an ongoing desired balance computation which has not converged after some amount of time or some number of iterations:

elasticsearch/server/src/main/java/org/elasticsearch/cluster/routing/allocation/allocator/DesiredBalanceComputer.java

Lines 321 to 329 in 18f960c

    
           logger.log( 
        
               reportByIterationCount || reportByTime ? Level.INFO : i % 100 == 0 ? Level.DEBUG : Level.TRACE, 
        
               () -> Strings.format( 
        
                   "Desired balance computation for [%d] is still not converged after [%s] and [%d] iterations", 
        
                   desiredBalanceInput.index(), 
        
                   TimeValue.timeValueMillis(currentTime - computationStartedTime).toString(), 
        
                   iterations 
        
               ) 
        
           );

However these numbers reset if a new cluster state is received, so it's possible for a steady stream of cluster states to prevent the computation for converging without ever seeing any warnings. IMO we should let these numbers accumulate until the computation fully converges, and report the number of restarts since convergence in the log message too.

Pinging @elastic/es-distributed (Team:Distributed)

Related to #91386

Pinging @elastic/es-distributed-obsolete (Team:Distributed (Obsolete))

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)

Accumulate compute() calls and iterations between convergences.

Accumulate compute() calls and iterations between convergences in Des…

allocation: time compute rounds by start time over last convergence

allocation: time compute rounds by start time over last convergence (#…

allocation: time compute rounds by start time over last convergence (e…

allocation: time compute rounds by start time over last convergence (#…

Navigation Menu

Repository navigation

Desired-balance warn threshold logging should accumulate across restarts #100850

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	logger.log(
	reportByIterationCount \|\| reportByTime ? Level.INFO : i % 100 == 0 ? Level.DEBUG : Level.TRACE,
	() -> Strings.format(
	"Desired balance computation for [%d] is still not converged after [%s] and [%d] iterations",
	desiredBalanceInput.index(),
	TimeValue.timeValueMillis(currentTime - computationStartedTime).toString(),
	iterations
	)
	);

Desired-balance warn threshold logging should accumulate across restarts #100850

Description

Activity

elasticsearchmachine commented on Oct 13, 2023

idegtiarenko commented on Nov 3, 2023

elasticsearchmachine commented on Mar 26, 2025

elasticsearchmachine commented on Mar 26, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions