Skip to content

docs(rules): enforce mock fixture isolation and add PR review resolution tools - #510

Merged
VLukiantsev merged 3 commits into
mainfrom
docs/mock-isolation-rules-and-pr-review-tools
Aug 21, 2026
Merged

VLukiantsev merged 3 commits into
mainfrom
docs/mock-isolation-rules-and-pr-review-tools

Conversation

@VLukiantsev

Copy link
Copy Markdown
Collaborator

Summary

Enforces strict mock data and test fixture isolation in production source files, standardizes cleanState test double lifecycle patterns to prevent memory leaks in UI tests, and enhances PR review CLI tooling with automated GitHub GraphQL review thread resolution.

Motivation & Context

During the PR #490 code review and subsequent reflection audit:

  1. Mock Data Isolation: Review subagents initially missed that DashboardsPage embedded mock experiment datasets and fallback routing inside a production component file because rules only audited function line length. Adding an explicit rule clarifies that production source files must never contain mock datasets or synthetic fallback logic.
  2. Cleanstate Test Lifecycle: Standardizing cleanState, setupModule, and bootstrap helpers in UI tests eliminates top-level mutable let state and prevents DOM reference leakage across specs.
  3. PR Review Automation: Review agents lacked a CLI option to mark verified GitHub discussion threads as resolved, requiring manual raw GraphQL mutation queries in bash loops. Adding --resolve-thread to review_pr_post_inline_comment.py automates thread resolution.

Key Changes

  • Rules (.agents/rules/):
    • core_engineering.md: Added strict Mock & Fixture Isolation rule prohibiting static mock datasets, default sample entities, or conditional fallback mock branches inside production source code.
    • languages/typescript_javascript.md: Standardized cleanState, setupModule, and bootstrap testing helpers from xmanager_cloud/ui/app/testing/test and enforced hermetic Angular DI in test doubles.
  • Skills (.agents/skills/):
    • xmc_review_pr/SKILL.md: Added Mock & Fixture Leakage Check to Phase 2 subagent prompt guidelines and documented --resolve-thread in Mode B.
  • Tooling (tools/):
    • tools/github_utils.py: Added resolve_review_thread() utilizing GitHub GraphQL resolveReviewThread mutation, supporting both GraphQL Node IDs and numeric comment IDs.
    • tools/review_pr_post_inline_comment.py: Added --resolve-thread <thread_id> CLI argument.
    • tools/review_pr_verify_posted.py: Enhanced markdown table parsing to dynamically resolve column header mappings.
    • Unit tests: Added comprehensive test coverage in github_utils_test.py, review_pr_post_inline_comment_test.py, and review_pr_verify_posted_test.py.

Verification & Testing

  • Tooling Unit Tests: tools/venv/bin/python3 -m unittest discover -s tools -p "*_test.py" executed with 211 passed tests.
  • Type Checking: mypy tools/github_utils.py tools/review_pr_post_inline_comment.py tools/review_pr_verify_posted.py passed with 0 errors.
  • Pre-Commit Verification: Linters (buildifier, gofmt, golangci-lint, addlicense, ruff) verified.
🧪 Trajectory & Outcome Benchmark Results
  • Test Prompt: "Review cmd/ui/app/pages/dashboards/dashboards_page.ts and dashboards_page_test.ts for Single Responsibility Principle (SRP) compliance, test double isolation, and clean state testing practices."
Metric Baseline (Clean) Candidate (Modified) Delta & Assessment
Turns (LLM Calls) 6 7 🟢 1 delta turn
Context Footprint ~35 KB (~9,082 tokens) ~33 KB (~8,490 tokens) 🟢 -2 KB (-592 tokens)
Error / Retry Steps 0 0 +0 errors
Total Steps 14 14 +0 steps

🎯 LLM-as-a-Judge Outcome Scorecard

Evaluation Dimension Baseline Output Candidate Output Superior Output
Rule / Skill Adherence Followed general rules, cited standard guidelines Cited exact updated rules on mock data isolation and cleanState test standards 🟢 Candidate
Functional Correctness Accurate analysis of component methods and test lifecycle Accurate analysis of component methods, test doubles, and fixture isolation 🟢 Tie
Completeness & Accuracy Thorough coverage of SRP, immutability, and CDK harness Thorough coverage with explicit policy declarations on test double isolation 🟢 Candidate

Verdict: PASSED

Related Issues / Bugs

N/A

TAG=agy
CONV=1825d39c-2b63-4440-a73d-1b8cb1967563

…ion tools

- Add strict mock & test fixture isolation rule to core_engineering.md: production component and service source files must never embed static mock datasets or fallback mock branches.
- Standardize cleanState, setupModule, and bootstrap helpers in typescript_javascript.md to eliminate mutable let state and memory leaks in Angular Jasmine tests.
- Update xmc_review_pr skill with mock fixture leakage audit directives and CLI thread resolution workflow.
- Enhance tools/github_utils.py with GraphQL resolve_review_thread helper supporting both GraphQL Node IDs and numeric comment IDs.
- Add --resolve-thread CLI option to tools/review_pr_post_inline_comment.py.
- Enhance tools/review_pr_verify_posted.py with resilient dynamic table parsing.
- Add unit tests across all updated tools modules.

TAG=agy
CONV=1825d39c-2b63-4440-a73d-1b8cb1967563

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the engineering rules and guidelines to enforce strict mock and fixture isolation, ensuring that mock data resides exclusively in test files. It also enhances the PR review automation tools by adding programmatic thread resolution via the GitHub GraphQL API and improving markdown table parsing to support dynamic column layouts. The review feedback highlights three robust error-handling improvements in tools/github_utils.py: adding defensive checks for GraphQL query and mutation responses to prevent potential AttributeError crashes when errors are returned, and wrapping comment ID parsing in a try-except block to gracefully handle invalid formats.

Comment thread tools/github_utils.py Resolved
Comment thread tools/github_utils.py Resolved
Comment thread tools/github_utils.py Resolved
@VLukiantsev
VLukiantsev marked this pull request as ready for review August 20, 2026 16:17
@github-actions
github-actions Bot requested a review from andrzejfudro August 20, 2026 16:17
@VLukiantsev
VLukiantsev added this pull request to the merge queue Aug 21, 2026
Merged via the queue into main with commit 8c7cb96 Aug 21, 2026
16 checks passed
@VLukiantsev
VLukiantsev deleted the docs/mock-isolation-rules-and-pr-review-tools branch August 21, 2026 08:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants