Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement logs websocket #23016

Merged
merged 44 commits into from
Aug 29, 2024
Merged

Implement logs websocket #23016

merged 44 commits into from
Aug 29, 2024

Conversation

licitdev
Copy link
Member

@licitdev licitdev commented Jul 15, 2024

Scope

What's changed:

  • Implemented logs websocket
  • New environment variables were added
    WEBSOCKETS_LOGS_ENABLED="true"
    WEBSOCKETS_LOGS_STYLE="raw" // Defaults to pretty printing
    WEBSOCKETS_LOGS_LEVEL="trace"

Potential Risks / Drawbacks

  • Potential logs exposure so the access is only available for admin

Review Notes / Questions

  • Existing logs logic should remain the same

TODO


Fixes #22482

Copy link

changeset-bot bot commented Jul 15, 2024

🦋 Changeset detected

Latest commit: 6863bc7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages
Name Type
@directus/env Minor
@directus/api Minor
docs Patch
@directus/sdk Patch
directus Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link
Member

@br41nslug br41nslug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of initial thoughts looking at this PR :)

Since this WS endpoint will only have a single purpose we could simplify the handler a bit:

  • require authentication mode strict and admin only
  • instead of recreating a subscription model the connection itself can be a subscription where a successful connection is "subscribed" and a closed connection "unsubscribed"

And as extra thought, do we want to buffer some logs so there is some backlog to be fetched/rendered? Or do we want to keep it fully ephemeral

@licitdev licitdev marked this pull request as ready for review August 19, 2024 10:24
@br41nslug br41nslug self-requested a review August 26, 2024 14:02
api/src/websocket/handlers/logs.ts Show resolved Hide resolved
api/src/logger/logs-stream.ts Outdated Show resolved Hide resolved
@rijkvanzanten rijkvanzanten merged commit 6c6977c into main Aug 29, 2024
5 checks passed
@rijkvanzanten rijkvanzanten deleted the logs-ws branch August 29, 2024 14:01
@github-actions github-actions bot added this to the Next Release milestone Aug 29, 2024
@licitdev licitdev restored the logs-ws branch August 29, 2024 14:05
@licitdev licitdev deleted the logs-ws branch August 29, 2024 14:07
u12206050 pushed a commit to u12206050/directus that referenced this pull request Sep 5, 2024
* Implement logs websocket

* Update docs

* Remove commented test code

* fixed controller shared logic

* fixed controller type

* Added shutdown callback

* Expose allowed log levels

* Use a different event

* Add log level filtering

* Return log_level when subscribed

* Remove unused import

* Limit logs websocket to `strict` auth mode (directus#23023)

* simplify logging handler

* enforce strict mode

* Update logs handler

* Update docs

---------

Co-authored-by: ian <licitdev@gmail.com>

* Add unique nodeId as uid

* Create logStream only when required

* Add admin requirement check

* Extract isValidLogLevel as util

* Remove authentication and path from server info

* Remove hostname and pid from logs if not raw

* Fix nodeId implementation

* Support custom log levels

* Fix build

* Fix test

* Remove unused constants and util

* Display websocket logs details in server info for admins only

* Expose log level value in server info

* Stream raw logs to websocket

* Retain hostname and pid field in raw logs

* Add separate env var for logs streaming

* Reset attempts after reconnection failure

* Remove obsolete accountability refreshing

* Run handler only when enabled

* Add changeset

* Allow wss protocol

* Rename to logs-stream

* Add unit tests

* Add explicit check for the remaining client

* Updated json stringification in log-streaming

* prettier

---------

Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Logs Streaming
3 participants