-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
Arq's default health check interval is 3600 seconds (https://github.com/samuelcolvin/arq/blob/9109c2e59d2b13fa59d246da03d19d7844a6fa19/arq/worker.py#LL208C30-L208C30)
This causes spurious warn logs from time to time for me, which say:
'WARNING::arq.prometheus:159 [arq_prometheus] Health key could not be read, value is `None`.
Possible causes:
- health key has not been initialized by the worker yet
- `health_check_key` or `queue_name` settings may be wrong
This might be caused by an issue with redis persistence in my cluster, but in any case this led me to realise that arq-prometheus relies on this health check frequency for producing up-to-date metrics
Recommend adding health_check_interval to the Readme for arq worker settings.
Here is the patch that I used to resolve the warning spam:
# Run arq health checks at double the frequency of our arq-prometheus monitoring:
health_check_interval=datetime.timedelta(seconds=7.5),
Metadata
Metadata
Assignees
Labels
No labels