Remove docker layer from fuzz tests workflow#4663
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughCI workflows now prepare and run the backend on the GitHub runner via a reusable setup action (Poetry venv cache, migrations, Gunicorn startup, Postgres restore). E2E and fuzz jobs call that action; fuzz tests run in-job, and the fuzz entrypoint quotes TEST_FILE when invoking pytest. ChangesBackend execution migration from Docker to runner
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
PR validation failed: No linked issue and no valid closing issue reference in PR description |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## feature/ci-cd-optimization #4663 +/- ##
===========================================================
Coverage 98.92% 98.92%
===========================================================
Files 528 528
Lines 16968 16968
Branches 2416 2364 -52
===========================================================
Hits 16785 16785
Misses 98 98
Partials 85 85
Flags with carried forward coverage won't be shown. Click here to find out more. Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/setup-backend-environment/action.yaml (1)
3-5:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winUpdate the action description to reflect removed Docker build step.
The description still mentions "builds the backend test image," but the Docker build step has been removed in this PR. The action now only sets up the backend environment and dependencies.
📝 Proposed fix
description: >- Fetches nest.dump from S3 using the same Poetry environment as local Make targets, waits for - Postgres, installs the PostgreSQL client, and builds the backend test image. + Postgres, and installs the PostgreSQL client.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/setup-backend-environment/action.yaml around lines 3 - 5, Update the description field in the GitHub Action YAML (the description: block) to remove the phrase about "builds the backend test image" and instead state that the action sets up the backend environment, fetches nest.dump, waits for Postgres, and installs the PostgreSQL client — i.e., edit the description value to reflect the removed Docker build step and the current responsibilities of the action.
♻️ Duplicate comments (1)
.github/workflows/run-fuzz-tests.yaml (1)
54-61: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick winVerify background process error handling.
Similar to the E2E workflow, this step backgrounds gunicorn with
&. If gunicorn fails to start, the step may still succeed, but the health check in lines 63-71 should catch this.Consider the same refactoring suggestion as the E2E workflow to improve observability.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/run-fuzz-tests.yaml around lines 54 - 61, The backgrounded gunicorn launch ("poetry run gunicorn wsgi:application --bind 0.0.0.0:9500 &" in the "Run backend with fuzz environment variables" step) can silently fail; change the step to capture the gunicorn PID after backgrounding, immediately verify the process is alive (e.g., using kill -0 or checking the PID) and if it exited, fail the step (exit non‑zero) before proceeding to the existing health check (the block that follows the current gunicorn start). Ensure the fix references the exact command string "poetry run gunicorn wsgi:application --bind 0.0.0.0:9500 &" and the health check block so the script both starts gunicorn and verifies it didn't exit prematurely.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/run-ci-cd.yaml:
- Around line 136-142: The CI step currently runs migrations and backgrounds
gunicorn in one shell line (cd backend && poetry run python manage.py migrate &&
poetry run gunicorn wsgi:application --bind 0.0.0.0:9000 &), which can hide
gunicorn startup failures; split this into two steps or add explicit error
checks: run poetry run python manage.py migrate in its own step, then start
gunicorn in a separate step (or start it and then verify it started by checking
the process or probing the port), and ensure failures from poetry run gunicorn
wsgi:application or a health check cause the job to fail; reference the
manage.py migrate invocation and the gunicorn wsgi:application startup when
making the change.
In @.github/workflows/setup-backend-environment/action.yaml:
- Around line 29-34: Update the GitHub Action step with id "cache-poetry-venv"
(named "Cache Poetry virtual environment") to use the v5 release of
actions/cache instead of actions/cache@v3; replace the uses value
"actions/cache@v3" with either the v5 tag "actions/cache@v5" or the commit pin
"actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae" to match other
workflows and ensure consistency and security.
---
Outside diff comments:
In @.github/workflows/setup-backend-environment/action.yaml:
- Around line 3-5: Update the description field in the GitHub Action YAML (the
description: block) to remove the phrase about "builds the backend test image"
and instead state that the action sets up the backend environment, fetches
nest.dump, waits for Postgres, and installs the PostgreSQL client — i.e., edit
the description value to reflect the removed Docker build step and the current
responsibilities of the action.
---
Duplicate comments:
In @.github/workflows/run-fuzz-tests.yaml:
- Around line 54-61: The backgrounded gunicorn launch ("poetry run gunicorn
wsgi:application --bind 0.0.0.0:9500 &" in the "Run backend with fuzz
environment variables" step) can silently fail; change the step to capture the
gunicorn PID after backgrounding, immediately verify the process is alive (e.g.,
using kill -0 or checking the PID) and if it exited, fail the step (exit
non‑zero) before proceeding to the existing health check (the block that follows
the current gunicorn start). Ensure the fix references the exact command string
"poetry run gunicorn wsgi:application --bind 0.0.0.0:9500 &" and the health
check block so the script both starts gunicorn and verifies it didn't exit
prematurely.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 334a6043-133e-49d2-a172-054cdbf6d828
📒 Files selected for processing (3)
.github/workflows/run-ci-cd.yaml.github/workflows/run-fuzz-tests.yaml.github/workflows/setup-backend-environment/action.yaml
There was a problem hiding this comment.
3 issues found across 4 files
Confidence score: 3/5
- There is meaningful merge risk from CI/workflow behavior: in
.github/workflows/run-ci-cd.yaml, backgroundingcd && migrate && gunicorncan let migration failures avoid failing the step, which can mask real deployment problems. - In
.github/workflows/setup-backend-environment/action.yaml, usingactions/cache@v3instead of a pinned commit SHA introduces supply-chain drift risk because upstream action code can change without review. - Also in
.github/workflows/setup-backend-environment/action.yaml, omitting Python version from the venv cache key can restore incompatible environments and cause flaky or misleading CI results. - Pay close attention to
.github/workflows/run-ci-cd.yamland.github/workflows/setup-backend-environment/action.yaml- failure signaling and cache/action pinning need tightening to avoid hidden CI regressions.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name=".github/workflows/setup-backend-environment/action.yaml">
<violation number="1" location=".github/workflows/setup-backend-environment/action.yaml:31">
P1: Pin `actions/cache` to an immutable commit SHA instead of the mutable `@v3` tag to avoid unreviewed action code changes in CI.</violation>
<violation number="2" location=".github/workflows/setup-backend-environment/action.yaml:34">
P2: Include Python version in the venv cache key to prevent restoring virtualenvs built with a different interpreter.</violation>
</file>
<file name=".github/workflows/run-ci-cd.yaml">
<violation number="1" location=".github/workflows/run-ci-cd.yaml:142">
P2: The trailing `&` backgrounds the entire `cd && migrate && gunicorn` chain, so migration failures won’t fail this step directly. Run migration in the foreground, then background only Gunicorn.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/run-fuzz-tests.yaml:
- Around line 57-67: Remove the redundant and risky activation step in the "Run
fuzz tests" job: delete the `eval $(poetry env activate)` invocation so the
workflow simply changes directory to backend and runs the entrypoint script;
keep the rest of the environment setup (TEST_FILE/REST_URL/BASE_URL/CI) intact
because backend/entrypoint.fuzz.sh already uses `poetry run pytest` to execute
tests in the Poetry-managed environment.
In @.github/workflows/setup-backend-environment/action.yaml:
- Around line 39-44: The cache key for the "cache-poetry-venv" step doesn't
include the Python version, so add the Python version to the key (e.g. change
key to include either ${{ matrix.python-version }} or ${{
steps.setup-python.outputs.python-version }}), ensuring the cache step id
"cache-poetry-venv" still targets path "backend/.venv" and keeps the poetry.lock
hash; update the key expression from poetry-venv-${{ runner.os }}-${{
hashFiles('backend/poetry.lock') }} to something like poetry-venv-${{ runner.os
}}-${{ steps.setup-python.outputs.python-version }}-${{
hashFiles('backend/poetry.lock') }} (or use your matrix variable) so venvs are
not reused across different Python versions.
In `@backend/entrypoint.fuzz.sh`:
- Around line 45-49: The shell invocation uses an unquoted variable ${TEST_FILE}
which can cause word splitting or globbing; update both pytest calls to use the
quoted variable "${TEST_FILE}" (i.e., change usages in the if/else blocks where
poetry run pytest ./tests/${TEST_FILE} and poetry run pytest
--log-cli-level=INFO -s ./tests/${TEST_FILE} appear) so pytest receives the
intended single path even if TEST_FILE contains spaces or glob characters.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: d386fc05-b575-48c7-8245-e55e1a6098d5
📒 Files selected for processing (4)
.github/workflows/run-ci-cd.yaml.github/workflows/run-fuzz-tests.yaml.github/workflows/setup-backend-environment/action.yamlbackend/entrypoint.fuzz.sh
There was a problem hiding this comment.
1 issue found across 3 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name=".github/workflows/setup-backend-environment/action.yaml">
<violation number="1" location=".github/workflows/setup-backend-environment/action.yaml:77">
P3: Remove the unused `BACKEND_PORT` environment variable from the migration step.</violation>
</file>
Tip: Review your code locally with the cubic CLI to iterate faster.
There was a problem hiding this comment.
1 issue found across 4 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name=".github/workflows/setup-backend-environment/action.yaml">
<violation number="1" location=".github/workflows/setup-backend-environment/action.yaml:45">
P1: The cache key uses `$${{ hashFiles(...) }}` instead of `${{ hashFiles(...) }}`, which prevents the lockfile hash from being evaluated and can cause stale dependency caches.</violation>
</file>
Tip: Review your code locally with the cubic CLI to iterate faster.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/run-fuzz-tests.yaml:
- Around line 57-70: Replace the manual virtualenv activation and direct script
invocation with Poetry's runner: remove the "source ./.venv/bin/activate" step
and invoke the fuzz entrypoint via "poetry run" so the job uses Poetry's managed
venv; ensure environment variables (TEST_FILE, REST_URL, BASE_URL, CI,
PYTEST_ADDOPTS) remain exported before calling "poetry run sh
./entrypoint.fuzz.sh" (or "poetry run ./entrypoint.fuzz.sh" if executable) so
the same runtime behavior is preserved while avoiding a hardcoded ./.venv path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 7758dede-ccd8-43ed-842c-e473ffc0c005
📒 Files selected for processing (5)
.github/workflows/run-backend-tests.yaml.github/workflows/run-ci-cd.yaml.github/workflows/run-fuzz-tests.yaml.github/workflows/setup-backend-environment/action.yamlbackend/entrypoint.fuzz.sh
There was a problem hiding this comment.
♻️ Duplicate comments (1)
.github/workflows/run-ci-cd.yaml (1)
135-136: 🧹 Nitpick | 🔵 Trivial | 💤 Low valueParameter order: place
db_usernamebeforedb_password.The conventional parameter ordering places username before password. This issue was previously flagged but remains unaddressed.
♻️ Proposed reordering
backend_port: 9000 db_name: nest_db_e2e - db_password: nest_user_e2e_password db_username: nest_user_e2e + db_password: nest_user_e2e_password env_file: .env.e2e.example🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/run-ci-cd.yaml around lines 135 - 136, Swap the two parameters so db_username appears before db_password in the workflow mapping; locate the entries named db_username and db_password in the run-ci-cd.yaml job step and reorder them to place the db_username: nest_user_e2e line immediately above db_password: nest_user_e2e_password.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In @.github/workflows/run-ci-cd.yaml:
- Around line 135-136: Swap the two parameters so db_username appears before
db_password in the workflow mapping; locate the entries named db_username and
db_password in the run-ci-cd.yaml job step and reorder them to place the
db_username: nest_user_e2e line immediately above db_password:
nest_user_e2e_password.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 20ae799c-47dc-46ca-bb0b-a56bc63ec1b6
📒 Files selected for processing (1)
.github/workflows/run-ci-cd.yaml
|
You're iterating quickly on this pull request. To help protect your rate limits, cubic has paused automatic reviews on new pushes for now—when you're ready for another review, comment |
|
Proposed change
Resolves #4636
setup-backend-environmentactionChecklist
make check-testlocally: all warnings addressed, tests passed