Skip to content

Add Request Logging to MDEMG Service#9

Merged
reh3376 merged 5 commits into
mainfrom
auto-claude/018-012-add-request-logging
Jan 16, 2026
Merged

Add Request Logging to MDEMG Service#9
reh3376 merged 5 commits into
mainfrom
auto-claude/018-012-add-request-logging

Conversation

@reh3376

@reh3376 reh3376 commented Jan 16, 2026

Copy link
Copy Markdown
Owner

This feature adds HTTP request logging middleware to the MDEMG Go service. Currently, the service uses basic log.Printf statements for startup messages and warnings but lacks comprehensive request-level logging for observability. Request logging will capture method, path, status code, duration, and optional request IDs for each HTTP request, enabling debugging, performance monitoring, and audit capabilities.

Summary by CodeRabbit

Release Notes

  • New Features

    • Added HTTP request logging with configurable format (text or JSON).
    • Auto-generated request IDs for tracking individual requests.
    • Request duration measurement and logging.
    • Optional health endpoint logging exclusion via configuration.
    • New environment variables for logging control: LOG_FORMAT and LOG_SKIP_HEALTH.
  • Tests

    • Added comprehensive test coverage for logging functionality.

✏️ Tip: You can customize this high-level summary in your review settings.

rhenley1958 and others added 5 commits January 16, 2026 17:56
…ment var

Added two new configuration options for request logging:
- LOG_FORMAT: "text" (default) or "json" for structured logging
- LOG_SKIP_HEALTH: boolean to skip logging /healthz and /readyz endpoints

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…rapper

Add HTTP request logging middleware implementation:
- responseWriter wrapper to capture status codes
- Request ID generation using crypto/rand
- LogConfig struct with Format and SkipHealth options
- LoggingMiddleware function supporting text and JSON formats
- Health endpoint exclusion option (/healthz, /readyz)
- Duration timing with time.Since()

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Updated server.go Routes() method to wrap the ServeMux with
LoggingMiddleware. Creates LogConfig from s.cfg.LogFormat and
s.cfg.LogSkipHealth, then returns the wrapped handler instead
of the bare mux.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…n tests

Added comprehensive table-driven tests for middleware functionality:
- TestIsHealthEndpoint: tests health endpoint detection
- TestGenerateRequestID: tests unique request ID generation
- TestResponseWriter_WriteHeader: tests status code capture
- TestResponseWriter_Write: tests implicit 200 OK behavior
- TestLoggingMiddleware: tests full middleware flow
- TestLoggingMiddleware_RequestIDUniqueness: tests ID uniqueness
- TestLoggingMiddleware_PassesRequestThrough: tests request passthrough
- TestLogConfig: tests config struct
- TestLogEntry: tests JSON marshaling

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fix pre-existing bug in cache_test.go where test expected incorrect LRU
eviction behavior. The test creates a cache of size 2, and when text1 is
re-added after being evicted, text2 becomes the LRU and is also evicted.
The test incorrectly expected text2 to still be in cache.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jan 16, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Introduces HTTP request logging middleware for a Go service with configurable text/json formats and optional health endpoint filtering. Adds request ID generation, integrates middleware into the server router, and updates configuration to support logging parameters. Includes comprehensive test coverage and adjusts a cache test expectation.

Changes

Cohort / File(s) Summary
Core Middleware Implementation
mdemg_build/service/internal/api/middleware.go
Adds LoggingMiddleware function that wraps HTTP handlers with request logging. Implements LogConfig struct (Format, SkipHealth), generateRequestID using crypto/rand, responseWriter wrapper to capture status codes, isHealthEndpoint helper, and logEntry struct for JSON formatting.
Middleware Tests
mdemg_build/service/internal/api/middleware_test.go
Comprehensive test suite covering isHealthEndpoint validation, generateRequestID uniqueness and hex format, responseWriter wrapper behavior, LoggingMiddleware status capture and X-Request-ID generation, text/json format handling, health skip rules, and LogEntry JSON marshaling.
Server Integration
mdemg_build/service/internal/api/server.go
Routes method now wraps HTTP mux with LoggingMiddleware, using LogConfig constructed from server configuration.
Configuration
mdemg_build/service/internal/config/config.go
Adds LogFormat (string) and LogSkipHealth (bool) fields to Config struct. Updates FromEnv to validate LOG_FORMAT ("text" or "json") and read LOG_SKIP_HEALTH boolean flag.
Cache Test Adjustment
mdemg_build/service/internal/embeddings/cache_test.go
Updates LRU cache eviction test expectation: increases mock call count from 4 to 5 and adjusts comment to reflect cache miss when re-embedding after eviction.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant LoggingMiddleware
    participant HTTPHandler
    participant ResponseWriter
    participant Logger

    Client->>LoggingMiddleware: HTTP Request
    LoggingMiddleware->>LoggingMiddleware: Generate Request ID
    LoggingMiddleware->>LoggingMiddleware: Start Timer
    LoggingMiddleware->>ResponseWriter: Wrap Original ResponseWriter
    LoggingMiddleware->>HTTPHandler: Pass Request + Wrapped ResponseWriter
    HTTPHandler->>ResponseWriter: Write Status Code
    HTTPHandler->>ResponseWriter: Write Headers & Body
    HTTPHandler-->>LoggingMiddleware: Handler Complete
    LoggingMiddleware->>LoggingMiddleware: Calculate Duration
    LoggingMiddleware->>Logger: Format Log Entry (Text/JSON)
    LoggingMiddleware->>Logger: Write Log
    LoggingMiddleware-->>Client: Response
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Poem

🐰 A logger hops into the fold,
Capturing requests, brave and bold,
With IDs unique, times so fleet,
Health checks skip their rabbit's beat,
Middleware magic, logs complete! 📝✨


Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

Comment @coderabbitai help to get the list of available commands and usage tips.

@reh3376 reh3376 merged commit 25086d4 into main Jan 16, 2026
1 check passed
reh3376 pushed a commit that referenced this pull request Jan 16, 2026
- Update status: 14 PRs merged (#1-12)
- Add PRs 9-12 to task table (Tasks 012-015)
- Document new features:
  - Request Logging Middleware (PR #9)
  - Configurable Scoring Hyperparameters (PR #10)
  - Memory Stats Endpoint (PR #11)
  - Memory Archive/Delete Endpoints (PR #12)
- Add new environment variables (SCORING_*, LOG_*)
- Update Key Files Reference with new files
- Mark Tasks 016, 017 as in-progress

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@reh3376 reh3376 deleted the auto-claude/018-012-add-request-logging branch January 28, 2026 01:03
reh3376 pushed a commit that referenced this pull request Jan 28, 2026
Gap Interview System (Task #8):
- Add GapInterviewer for generating interview prompts from capability gaps
- Create type-specific prompts for data_source, reasoning, query_pattern gaps
- Add RunWeeklyInterview() APE job with configurable scheduling
- Add API endpoints: GET/POST /v1/system/gap-interviews
- Add prompt answer/skip tracking with Neo4j persistence
- Add V0010 migration for InterviewPrompt schema
- Wire StartWeeklyGapInterviews() background job into server

CMS Integration Tests (Task #9):
- Add integration_test.go with Neo4j test fixtures
- Test visibility filtering (private/team/global)
- Test Context Cooler graduation and decay
- Test Jiminy rationale generation
- Test REFERS_TO cross-module linking
- Test surprise detection for corrections
- Test end-to-end conversation flow

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
reh3376 pushed a commit that referenced this pull request Mar 23, 2026
Close three feedback loop gaps in the Jiminy guidance system to create
a self-improving guidance pipeline where chronically ignored guidance
decays automatically and consistently followed guidance strengthens.

Gap 1 — Dispatcher gap (3 new executors):
- review_guidance_effectiveness: diagnostic executor that queries
  per-constraint effectiveness and categorizes items as high/low/insufficient
- adjust_guidance_confidence: mutating executor that boosts items with
  effectiveness >= 0.7, decays items < 0.1 (min 5 surfaces), archives stale
- archive_ineffective_constraints: delegates to ArchiveStaleConstraints

Gap 2 — Confidence scope gap:
- ConfidenceUpdater now fires for ALL guidance types (constraints,
  corrections, patterns, decisions), not just GuidanceConstraint
- Archive guard added: only constraint nodes get archived when confidence
  drops below threshold — corrections/learnings/patterns retain status

Gap 3 — SignalLearner isolation:
- Jiminy Guide() now records Hebbian emissions for each surfaced item
- RecordOutcome() records responses for followed/partial-compliance outcomes
- SignalLearnerProvider interface avoids circular import between jiminy/ape

New reflection pattern #15 (guidance_confidence_drift):
- Triggers adjust_guidance_confidence when GuidanceHealth is between 0 and 0.7
- Complements pattern #9 (diagnostic at 0.5) with corrective action at 0.7

Architecture:
- GuidanceCalibrationProvider interface in ape/types_rsic.go
- rsicGuidanceCalibrationAdapter in api/rsic_adapters.go bridges ape→jiminy
- 5 new RSIC_GUIDANCE_* config variables with sensible defaults
- Wired in server.go: SetGuidanceCalibrator + SetSignalLearner

Tests: 10 new tests across 4 test files
Docs: feature doc, UATS spec, cli-reference, .env.example, AGENT_HANDOFF

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
reh3376 pushed a commit that referenced this pull request May 4, 2026
…lag-off) + Phase 13 Epic 6 V0017 audit-writer fix + Phase 11+ feature-doc backfill (narrow close)

Narrow close per operator approval after Epic 0+1+2 produced design questions
that warrant dedicated follow-up sprints. Note 05 deferred to Phase 14.2;
Note 06 default flip deferred to Phase 14.1.

What landed
-----------

* Phase 13 Epic 6 V0017 audit-writer fix (in-flight discovery)
  - tsdb/retrieval_audit_writer.go (new, ~165 LOC; buffered + 30s flush via CopyFrom)
  - retrievalAuditAdapter in api/server.go (cycle-safe translation)
  - V0017 was empty since Phase 13 because SetRetrievalAuditWriter had no
    callers; now writes per retrieve when RETRIEVAL_AUDIT_ENABLED=true.
  - Live verification: 279 audit rows accumulated in 4h since fix landed.

* Note 06 sparse activation gate (flag-off)
  - retrieval/gate.go (~190 LOC) + 9 Tier 1 unit tests, all green
  - Wired post-aggregation, pre-rerank in service.go
  - 4 config knobs (SPARSE_*); default off, percentile 0.95, min 3, max 20
  - Per-request override via ?sparse=true|false and ?sparse_percentile=N
  - debug.sparse_gate_* + debug.below_threshold_* (when JiminyEnabled)
  - 3 Prometheus histograms

* TSDB V0019 sparse_gate_metrics
  - migrations/019_sparse_gate_metrics.sql (hypertable, 7-day chunks)
  - tsdb/sparse_gate_writer.go (~165 LOC)
  - sparseGateRecorderAdapter in api/server.go (always wired so per-request
    overrides record even when default off)
  - TSDB_REQUIRED_SCHEMA_VERSION 18 -> 19

* Epic 0 forensic doc — phase_14_score_distribution_analysis.md
  - Defaults derived from llm_interactions.retrieval_scores (99k+50k score
    points across consulting.classify + retrieval.rerank_cross)
  - Heavy-tail confirmed (p98/p50 ~ 4-5x); within-call clamp dominates
    percentile choice in dominant K=20-50 regime
  - Note 05 catalog redesign needed for whk-wms (0 distinct symbols, 0
    distinct roles) — flagged for Phase 14.2

* A/B verdicts captured
  - 16q quick at MIN=3 / p95,p98,p99: all FAIL (q69 boundary)
  - 16q quick at MIN=10 / p95: PASS (mean +0.019, 0 regressions, 3 improvements)
  - 120q full at MIN=10 / p95: FAIL per-question (mean parity 0.413=0.413,
    7 boundary regressions across 4 categories, 3 of 7 in
    architecture_structure)
  - Per sprint plan §10 risk #1: ship flag-off; Phase 14.1 will retune.

* Phase 11+ feature-doc backfill (operator request 2026-05-04)
  - new: docs/features/{mlx-watchdog,uvts-validation,column-voting-retrieval,
         local-llm-runtime,sparse-retrieval}.md
  - extended: docs/features/service-resilience.md (Phase 11.6.x additions)
  - Standing rule saved as memory feedback_per_feature_docs_required.md

* Follow-up sprint stubs scoped
  - sprint_plan_phase_14_1_adaptive_per_category_gate.md (~3 days, ~$15)
  - sprint_plan_phase_14_2_note_05_sparse_fingerprints.md (~7 days, ~$25)

Decision-fork outcomes
----------------------

| Fork | Provisional | Outcome |
|---|---|---|
| #2 percentile default | 0.98 | 0.95 (Epic 0 data) |
| #5 catalog bit policy | static 64/64/64/64 | adaptive (deferred Phase 14.2) |
| #8 gate ordering | pre-rerank | pre-rerank (confirmed) |
| #9 default flip | per-Note conditional | flag-off (Phase 14.1 will flip) |

OpenAI spend (actual): ~$13. Well under sprint $25-50 budget.

Tests + lint
------------

* go test -race ./internal/{retrieval,config,metrics,tsdb}: all green
* golangci-lint run on affected packages: 0 issues
* Live smoke: /healthz green, retrieve returns 20 (gate off), 279 V0017
  audit rows in 4h (Phase 13 Epic 6 fix verified in production)

Memory observations
-------------------

* rw0mzergwcqct8abpw0dli9x — Phase 14 Epic 8 doc-backfill scope
* sc4iwy3of9ndn5kowja1i14i — Epic 0 forensic + audit-writer gap
* omr2rs5jppqrvee2k0l1xtd1 — Epic 1 gate code complete
* re4k7rpd3hjt5a52l8qwx8fp — Epic 2 verdict + Phase 14.1 scope

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants