Skip to content

Set FluentBit self logs to buffer in memory instead of filesystem #631

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Jul 5, 2022
Merged

Set FluentBit self logs to buffer in memory instead of filesystem #631

merged 8 commits into from
Jul 5, 2022

Conversation

@avilevy18
Copy link
Contributor

@avilevy18 avilevy18 commented May 26, 2022 •
edited
Loading

Parameterized storage.type config with variable BufferInMemory. Edit component to always use in memory buffering for self logs

Following: http://b/226668416, this PR changes self logs to buffer in memory as opposed to the filesystem.

Soak Test:
image

@google-cla
Copy link

google-cla bot commented May 26, 2022

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@avilevy18 avilevy18 force-pushed the avilevy-self-logs-memory branch 2 times, most recently from 33bef92 to 989db22 Compare May 26, 2022 16:12
@avilevy18 avilevy18 requested review from a team and hsmatulis and removed request for a team May 26, 2022 16:18
@avilevy18 avilevy18 force-pushed the avilevy-self-logs-memory branch from fb594fb to cb9339e Compare May 26, 2022 17:22
qingling128
qingling128 approved these changes Jun 3, 2022
View reviewed changes
Copy link
Contributor

@qingling128 qingling128 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qingling128
Copy link
Contributor

qingling128 commented Jun 3, 2022

SLES 12 was failing, but it's not related to this PR.

quentinmit
quentinmit reviewed Jun 3, 2022
View reviewed changes
Copy link
Member

@quentinmit quentinmit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a doc somewhere I can read on why we want to do this?

(Also, presumably fluent-bit is still writing state somewhere on disk so it doesn't reprocess the same log lines, right?)

@avilevy18 avilevy18 merged commit 3d487fe into master Jul 5, 2022
@igorpeshansky igorpeshansky deleted the avilevy-self-logs-memory branch July 10, 2023 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@quentinmit quentinmit quentinmit left review comments

@qingling128 qingling128 qingling128 approved these changes

@hsmatulis hsmatulis Awaiting requested review from hsmatulis hsmatulis was automatically assigned from GoogleCloudPlatform/stackdriver-reviewers

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@avilevy18 @qingling128 @quentinmit