Skip to content

Change read mode to immediately stop consuming lines when shutting down - #322

Merged
edmocosta merged 2 commits into
logstash-plugins:mainfrom
edmocosta:change-read-mode-shutdown
Dec 13, 2023
Merged

edmocosta merged 2 commits into
logstash-plugins:mainfrom
edmocosta:change-read-mode-shutdown

Conversation

@edmocosta

@edmocosta edmocosta commented Dec 5, 2023 •

Copy link
Copy Markdown
Contributor

This PR changes the read mode to immediately stop consuming buffered lines when the pipeline shutdown is requested.


Closes: #323

@edmocosta
edmocosta force-pushed the change-read-mode-shutdown branch from ce0b13c to 087b401 Compare December 5, 2023 14:36
@edmocosta edmocosta changed the title [WIP][Test] Change read mode to immediately stop consuming lines when shutting down [WIP] Change read mode to immediately stop consuming lines when shutting down Dec 5, 2023
@edmocosta edmocosta changed the title [WIP] Change read mode to immediately stop consuming lines when shutting down Change read mode to immediately stop consuming lines when shutting down Dec 7, 2023
@edmocosta
edmocosta marked this pull request as ready for review December 7, 2023 09:11
@edmocosta
edmocosta requested a review from jsvd December 12, 2023 09:56

@jsvd jsvd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
I've confirmed that this change allows Logstash to terminate much sooner, on each line instead on each loop (which can be hundreds of lines).
I've also tested with multiple reads and multiple shutdowns/restarts that no data was lost or duplicated.

@edmocosta
edmocosta merged commit 70aadf2 into logstash-plugins:main Dec 13, 2023
@edmocosta
edmocosta deleted the change-read-mode-shutdown branch December 13, 2023 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Read mode is not immediately stop consuming lines when shutting down is requested

3 participants