Skip to content

AAP-60452 Remove the dynamic log level filter for the dispatcherd main process #16200

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
AlanCoding wants to merge 6 commits into
base: devel
Choose a base branch
from
Open

AAP-60452 Remove the dynamic log level filter for the dispatcherd main process #16200

AlanCoding wants to merge 6 commits into from
+101 −2

Conversation

@AlanCoding
Copy link
Member

@AlanCoding AlanCoding commented Dec 9, 2025 •
edited by cursor bot
Loading

SUMMARY

Right now, it looks like logs will respect the dynamic_level_filter, and this is a bad thing, and was never intended. Because doing this means talking to postgres, and that has no async version, which means that it will likely create a new synchronous connection in asyncio code, which is very bad, although it might be kind of working for now.

This sets the dynamic log level only on startup, and then just to be sure, closes the connection.

I will put up a separate proposal for on-the-fly adjustments of the log level which I argue should be done by memory modification via a task (formally a control task), which the dispatcherd process, specifically, has facilities to allow.

ISSUE TYPE
  • Bug, Docs Fix or other nominal change
COMPONENT NAME
  • API

Note

Configures dispatcherd main-process logging to bypass the dynamic level filter, set level from LOG_AGGREGATOR_LEVEL at startup, close sync resources before async startup, propagate log-level changes via control, add tests, and bump dispatcherd version.

  • Logging / Dispatcher:
    • Remove dynamic_level_filter from parent-process handlers and set dispatcherd logger level from settings.LOG_AGGREGATOR_LEVEL in run_dispatcher.Command.configure_dispatcher_logging.
    • Before starting dispatcherd, close Django DB connection and cache to avoid sync usage in async startup.
  • Runtime log-level updates:
    • In clear_setting_cache, when LOG_AGGREGATOR_LEVEL changes, send set_log_level control to dispatcherd on get_task_queuename().
  • Tests:
    • Add tests for clear_setting_cache sending set_log_level and for configure_dispatcher_logging updating the logger level.
  • Dependencies:
    • Bump dispatcherd[pg_notify] to 2025.12.12.

Written by Cursor Bugbot for commit 9d24c5c. This will update automatically on new commits. Configure here.

@AlanCoding AlanCoding mentioned this pull request Dec 9, 2025
Merged
@AlanCoding AlanCoding changed the title Remove the dynamic log level filter for the dispatcherd main process AAP-60452 Remove the dynamic log level filter for the dispatcherd main process Dec 10, 2025
@AlanCoding AlanCoding marked this pull request as ready for review December 10, 2025 16:33
@AlanCoding AlanCoding marked this pull request as draft December 10, 2025 16:35
@AlanCoding
Copy link
Member Author

AlanCoding commented Dec 10, 2025

TODO:

After the dispatcherd patch merges, we need to have a special case in the task for processing a settings change to run the new control command.

cursor[bot]
cursor bot reviewed Dec 10, 2025
View reviewed changes
@AlanCoding AlanCoding force-pushed the dispatcher_logging branch 2 times, most recently from 08e849d to 1ba8665 Compare December 16, 2025 17:09
@AlanCoding
Copy link
Member Author

AlanCoding commented Dec 18, 2025

Manual testing looking good, just need some actual tests now.

tools_awx_1       | 2025-12-18 01:45:37,458 DEBUG    [-] awx.main.utils.reload supervisorctl restart awx-rsyslogd succeeded
tools_awx_1       | 2025-12-18 01:47:27,767 INFO     [-] awx.main.tasks.system clear_setting_cache of keys ['LOG_AGGREGATOR_LEVEL']
tools_awx_1       | 2025-12-18 01:47:27,772 INFO     [-] dispatcherd.service.control_tasks Changed dispatcherd logger level from WARNING to INFO
tools_awx_1       | INFO:dispatcherd.service.control_tasks:Changed dispatcherd logger level from WARNING to INFO
tools_awx_1       | 2025-12-18 01:47:27,772 INFO     [-] dispatcherd.service.main Control action set_log_level returned <class 'dict'>, done
tools_awx_1       | INFO:dispatcherd.service.main:Control action set_log_level returned <class 'dict'>, done
tools_awx_1       | [pid: 335|app: 0|req: 6/41] 172.18.0.1 () {68 vars in 1762 bytes} [Thu Dec 18 01:47:27 2025] PATCH /api/v2/settings/logging/ => generated 29154 bytes in 47 msecs (HTTP/1.1 200) 17 headers in 735 bytes (1 switches on core 0)
tools_awx_1       | 2025-12-18 01:47:27,831 INFO waiting for awx-rsyslogd to stop
tools_awx_1       | 2025-12-18 01:47:27,833 INFO stopped: awx-rsyslogd (exit status 0)

@AlanCoding AlanCoding marked this pull request as ready for review December 19, 2025 04:23
@github-actions github-actions bot added the dependencies Pull requests that update a dependency file label Dec 19, 2025
@sonarqubecloud
Copy link

sonarqubecloud bot commented Dec 19, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
82.1% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cursor cursor[bot] cursor[bot] left review comments

@thedoubl3j thedoubl3j Awaiting requested review from thedoubl3j

@djyasin djyasin Awaiting requested review from djyasin

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

component:api dependencies Pull requests that update a dependency file

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@AlanCoding