SUMMARY

The execution nodes are getting stuck at values of capacity=1 with cpu=0 and memory=0 permanently if the initial health check fails. The remediation condition to re-run the health check is never triggered because the if statement only checks for if capacity=0. The calculations for capacity lead to a situation where the capacity value has a floor of 1, so the remediation condition is never met.

The max(1, ...) floor in capacity formulas and the capacity == 0 remediation check are contradictory:

get_cpu_effective_capacity(0) → max(1, int(0 * 4)) → 1
get_mem_effective_capacity(0) → max(1, (0 - 2GB) // 100MB) → 1
set_capacity_value() → min(1, 1) + (max(1, 1) - min(1, 1)) * 1.0 → 1

Since capacity is 1 (not 0), the remediation condition at line 636 never matches.

I updated the remediation condition in awx/main/tasks/system.py:636 to also rerun the health check when instance.cpu == 0 and instance.memory == 0). This would make it so that the health check would run even if capacity = 1, if the cpu and memory values are set to 0.

ISSUE TYPE

• Bug, Docs Fix or other nominal change

COMPONENT NAME

• API

STEPS TO REPRODUCE AND EXTRA INFO

1. Deploy a containerized AAP cluster with 2 control nodes and 2 execution nodes
2. During initial setup, if the receptor mesh is not fully established when the first execution_node_health_check runs, the health check may return partial/empty data
3. The execution nodes end up with cpu=0, memory=0, version=ansible-runner-???, cpu_capacity=1, mem_capacity=1, capacity=1, node_state=ready
4. Wait any amount of time - the nodes remain stuck at capacity=1 indefinitely with no automatic remediation

Expected Result: Remediation logic should detect that the CPU and Memory values are at 0 and retrigger a health check eventually.

Actual Result: Nodes are stuck without retriggering a health check, because the capacity value is stuck at 1.

# Before
elif instance.capacity == 0 and instance.enabled:

# After
elif (instance.capacity == 0 or (instance.cpu == 0 and instance.memory == 0)) and instance.enabled: