Maestro: LayerLiveMem core right-sizing should rely on persisted MaxRss, not in-memory host reports #2564

Summary

LayerLiveMem is currently the only mechanism in the scheduler that right-sizes a layer's core requirement from previous executions, and its design is not sound for a multi-Cuebot deployment. It should be redesigned to derive layer sizing from the persisted MaxRss evidence rather than from in-memory accumulation of HostReport payloads.

Current behavior

Maestro.resizeFromLiveMem() (cuebot/src/main/java/com/imageworks/spcue/dispatcher/Maestro.java) sizes threadable layers before placement by reading LayerLiveMem.typicalRssKb(layerId):

• LayerLiveMem is a plain in-JVM ConcurrentHashMap (median of the last 32 frames' rss peaks, MIN_SAMPLES = 4, 30-minute expiry).

• It is fed exclusively from HostReportHandler.handleHostReport():

  if (layerLiveMem != null)
      layerLiveMem.record(report.getFramesList());

• The resize result drives cores = round(rss / memPerCoreKb), mem = max(declared, rss), and the rssProven probe gate that throttles unsampled 1-core layers to PROBE_FRAMES.

Why this is not ideal

1. No co-locality guarantee between reports and the planner. RQD host reports are spread across all Cuebot instances, while Maestro planning runs only on the advisory-lock leader (doTick: "Runs on every Cuebot; only the leader plans"). The leader therefore sizes layers from only the fraction of reports that happened to land on it. The same applies to frame-complete reports.

2. Non-durable. All evidence lives in heap. A Cuebot restart, a deploy, or a leadership failover resets every layer to "no evidence", re-arming the probe gate and re-throttling layers the farm has already characterized.

3. Sampling bias, not just sampling noise. Because the sample set is whatever subset of reports reached this JVM, the median is computed over a biased, instance-dependent population — it is not a median of the layer's frames. Two Cuebots can legitimately disagree about the same layer's size.

4. Wrong source of truth. Running-frame rss/maxRss snapshots from host reports are a lossy, mid-flight view. OpenCue already persists authoritative per-frame and per-layer high-water marks (frame.int_max_rss, layer_mem.int_max_rss, int_max_pss), maintained by DispatchSupportService on frame completion and by HostReportHandler.updateLayerMemoryUsage(). The scheduler should consume that, not rebuild a parallel, weaker view in memory.

5. Inconsistent with how memory sizing already works. The memory path is DB-persisted and instance-independent, reaching Maestro through layer.int_mem_min in SELECT_CANDIDATES_FOR_GROUP:

   • HostReportHandler.updateLayerMemoryUsage() → LayerDao.increaseLayerMinMemory() / updateLayerMaxRSS() / updateLayerMaxPSS()
   • FrameCompleteHandler → JobManager.optimizeLayer() → LayerDaoJdbc.balanceLayerMinMemory() (layer_mem.int_max_rss + 256MB, gated by b_optimize)
   • OOM retry → increaseLayerMemoryRequirement() (plus the in-process per-frame OomMemoryTracker bump)

   Core sizing is the one dimension left without a durable, shared backing store.

Proposed direction

• Source layer sizing from persisted MaxRss evidence (layer_mem.int_max_rss / int_max_pss, or a percentile over frame.int_max_rss for succeeded frames) instead of an in-memory fold of HostReport frame lists.
• Fold that evidence into the existing per-group candidate query so every Cuebot — leader or standby, before or after a restart — computes the identical layer shape.
• If a "typical" (median/percentile) figure is preferred over a high-water mark, persist the aggregate, for example as a per-layer statistic maintained on frame completion, so it survives restarts and is not a function of which instance received the report.
• Base the rssProven / probe gate on the same persisted evidence, so a restart cannot re-throttle an already-characterized layer.

Related cleanup

LayerDao.isOptimizable() and LayerDao.getThreadStats() (the legacy core/thread optimizer) have no scheduler callers — JobManagerService.optimizeLayer() ignores its cores and runTime arguments and only calls balanceLayerMinMemory(). Worth removing or re-wiring as part of this work.

Affected files

• cuebot/src/main/java/com/imageworks/spcue/dispatcher/LayerLiveMem.java
• cuebot/src/main/java/com/imageworks/spcue/dispatcher/Maestro.java
• cuebot/src/main/java/com/imageworks/spcue/dispatcher/HostReportHandler.java
• cuebot/src/main/java/com/imageworks/spcue/dispatcher/CoreUnitDispatcher.java

Written by Claude on behalf of Diego Tavares.