Conversation
Studio records every OpenAI-compatible API request into an in-memory
ApiMonitor (the "API Monitor" activity view). Users who run Studio purely
as an inference API and handle logging/telemetry with an external gateway
have no use for it and want to turn it off.
Add an opt-in UNSLOTH_STUDIO_DISABLE_API_MONITOR env var (house
{"1","true","yes","on"} truthy parsing). When set, ApiMonitor.start()
returns an empty id so every downstream mutator short-circuits on its
existing `if not entry_id` guard and no entry is ever recorded; snapshot(),
active_count() and get() return empty/zero/None naturally. Off by default,
so existing behaviour is unchanged and no call sites need to change.
Fixes unslothai#7036
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Code Review
This pull request introduces an opt-in kill switch for the in-memory API monitor via the UNSLOTH_STUDIO_DISABLE_API_MONITOR environment variable. When disabled, the ApiMonitor class acts as a no-op, returning a falsy ID on start and preventing any telemetry or logging from being recorded. Comprehensive unit tests have been added to verify the default behavior, the disabled state, and the environment variable parsing. There are no review comments, and I have no additional feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
|
@codex review |
|
Codex Review: Didn't find any major issues. Swish! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Thanks @gaurav0107. This is a sensible opt-in for headless Studio deployments like the one described in #7036, where an external gateway already owns logging and telemetry. Keeping prompt/reply previews and model lifecycle events in Studio is then duplicate work and unnecessary in-memory retention. The default behavior and HTTP access logs remain unchanged. I verified disabled mode across requests and lifecycle events. Merging. |
main's unslothai#7156 landed the same feature as UNSLOTH_STUDIO_DISABLE_API_MONITOR, guarding start() and record_lifecycle() both. Resolve the conflict by taking that implementation wholesale and dropping this branch's duplicate UNSLOTH_STUDIO_DISABLE_API_MONITOR_LOGS, so only the part main lacks remains: surfacing the disabled state in the console. Also fixes a pre-existing bug the flag exposes: the external-provider proxy read [DONE] out of the monitor helper, which returns None for every line once recording is off, so it appended a second [DONE] after the provider's own.
Summary
Studio records every OpenAI-compatible API request into an in-memory
ApiMonitor(the "API Monitor" activity view). A user running Studio purely asan inference API, with logging/telemetry handled by an external gateway (e.g.
LiteLLM on a separate host), asked for a way to turn this off (#7036).
This adds an opt-in
UNSLOTH_STUDIO_DISABLE_API_MONITORenvironment variable,parsed with the same truthy set (
{"1","true","yes","on"}) the backend alreadyuses elsewhere (
transformers_latest.py,llama_admission.py, ...). When set:ApiMonitor.start()returns an empty (falsy) id, so every downstream mutator(
append_reply/set_reply/set_usage/finish/fail) short-circuitson its existing
if not entry_idguard and no entry is ever recorded.snapshot(),active_count()andget()return empty / zero /Nonenaturally, so the Monitor view stays empty.
It is off by default, so existing behaviour is unchanged and no call sites
needed to change — the singleton is simply constructed as
ApiMonitor(enabled = not _api_monitor_disabled()).Scope note: "API monitor" here means the in-memory OpenAI-compatible API
activity monitor in
core/inference/api_monitor.py, not the HTTP access log(
LoggingMiddlewareinloggers/handlers.py). Happy to extend to the accesslog too if that was the intent.
Validation
python -m pytest tests/test_api_monitor.py-> 17 passed (5 new tests:disabled-is-noop, enabled-by-default-records, and env-var truthy/falsy/unset
parsing).
ruff check studio/backend/core/inference/api_monitor.py studio/backend/tests/test_api_monitor.py-> clean.python -m compileallon both touched files -> clean.UNSLOTH_STUDIO_DISABLE_API_MONITOR=1->start()returns
""andsnapshot()is[]; unset -> records normally.api_monitor.start()call sites inroutes/inference.pyassign to an
Optional[str]monitor_idthat is consumed only through theguarded mutators, so returning
""when disabled is safe.Fixes #7036