Silence untrustworthy FIM schema-validation errors during shutdown - #37688
Merged
Merged
Conversation
rovogel
approved these changes
Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
On agent shutdown,
wazuh-syscheckdcan emit a misleadingERRORreporting that a valid event failed schema validation because of an "invalid" ISO8601 date that is in fact perfectly valid:Both timestamps are valid ISO8601, and the same event validates correctly at runtime — the error only appears during shutdown.
Root cause (shutdown race on process-static state): the FIM scan runs on the main thread (
fim_scan) and validates each stateful event viavalidate_and_persist_fim_event()→schema_validator_validate()→isValidISO8601Date(), which uses a function-localstatic const std::regex. When a termination signal arrives,fim_shutdown_waiter()runs the teardown and callsHandleSIG(), which callsexit(1)(shared/src/sig_op.c).exit()destroys process-static objects — including that regex inlibschema_validator— while the scan thread may still be validating.std::regex_match()on a destroyed regex is undefined behavior and, in practice, returns no-match, so every date check fails and a valid event is reported as invalid. Because the scan holdsfim_scan_mutex, the waiter deliberately skips the FIM DB teardown but still proceeds toexit(), so the scan and static destruction run concurrently — this is why it only reproduces during shutdown.Closes #37654
Proposed Changes
In
validate_and_persist_fim_event()(src/syscheckd/src/run_check.c), when a schema-validation failure is observed while a shutdown is in progress (fim_shutdown_process_on()), treat the result as untrustworthy:debuginstead oferror(no misleadingERRORduring shutdown).trueso the caller releases its resources without leaking. The agent is stopping and the event is re-evaluated on the next start.Handling it at the point where the failure is detected (rather than skipping validation up front) also covers the case where the stop happens inside the validator, i.e.
std::regex_matchracing the static destruction mid-call.Note on why deletion matters: in the scan path
mark_for_deletionis set forINSERTED/MODIFIEDfiles (src/syscheckd/src/file/file.c), so acting on a spurious shutdown-time failure could delete a valid entry from the FIM database (only to be re-reported on the next start). Returning early avoids that.Results and Evidence
Before (from #37654, agent shutdown; the reported valid event is rejected):
After (with this change): no
ERRORis emitted for a validation failure that happens during shutdown. When debug logging is enabled, a single debug line is logged instead, and the untrustworthy result is not acted on:Genuine, non-shutdown validation failures are unchanged: they still log at
ERRORwith the failure reason and still skip persistence / mark for deletion.Artifacts Affected
wazuh-syscheckd(agent). No changes to the schema validator library or its embedded schemas.Configuration Changes
None.
Documentation Updates
None required. Recommendation: add the (now debug-level) shutdown message to the known-messages list of the epic (#37410) if needed.
Tests Introduced
No new unit tests: the behavior is a shutdown-timing race that is not exercised by the existing FIM unit suite, and no existing test asserts on the affected
ERRORmessages (verified). The change is a guarded log-level/early-return withinvalidate_and_persist_fim_event(); the non-shutdown path is unchanged.Review Checklist