Skip to content

test(e2e): fix four CI flakes at root cause and make the Stripe block… - #3360

Merged
shohan0120 merged 9 commits into
developfrom
qa/ci-e2e-flake-fixes-and-stripe-diagnostic
Aug 10, 2026
Merged

test(e2e): fix four CI flakes at root cause and make the Stripe block…#3360
shohan0120 merged 9 commits into
developfrom
qa/ci-e2e-flake-fixes-and-stripe-diagnostic

Conversation

@shohan0120

@shohan0120 shohan0120 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

The nightly develop e2e run had been red since 2026-07-31. Two different problems were tangled together in those failures and needed different fixes. Both are now fixed, and the branch also rebalances the shard baseline and adds an end-of-shard isolation re-run to CI.

No assertion was weakened and no test was skipped to achieve this. One test that was passing for the wrong reason is now stricter, not looser.

1. Stripe block checkout — 12 failures to 0

Root cause, reported by CI once the suite was made to describe its own failure: Block validation said: Your mobile phone number is invalid.

The seeded billing phone was the reserved fictional (555) 555-5555. Stripe Link validates the billing phone as a mobile number and rejects that range, so WooCommerce Blocks never submitted the form. POST /wc/store/v1/checkout never fired, no order was created, and the settle poll could only ever report "none". The card was never the problem.

  • Seed a real-format mobile number, and fill the guest phone explicitly in fillBlockGuestDetails() with fail-loud assertions.
  • Impossible wait budget. Post-click waits were 60s + 120s = 180s minimum inside specs configured at 150s, so the poll that decides pass/fail could never complete on CI — the test died mid-poll and reported the poll's placeholder rather than a verdict. The SPA redirect now gets 15s on CI and the full 60s locally where it is the fast path; the two affected specs move to 240s to match the sibling money specs.
  • The suite was blind to its own failure mode. After clicking Place Order it only polled the REST API, so "the Store API rejected the payment" and "the click was a no-op" surfaced identically. placeBlockOrderExpectReceived() now records every Store API checkout POST and reads back the block notice banner.
  • Closes a test that passed for the wrong reason. SE-GUEST-03 was passing on a missing-phone form error without ever submitting the declined card. The decline assertion now matches by exclusion — the error must not be a form-validation message — because real declines surface as "Something went wrong. Please contact us to get assistance.", which a payment-keyword match rejected.

2. Cross-spec state pollution — the four "flakes" that were not flakes

Four tests failed on CI and passed locally in isolation, across two runs. They were never timing flakes: each is a different spec earlier on the same shard leaving global state dirty. A clean-environment isolation run cannot reproduce this by construction, because a pristine database is exactly the condition under which the victim passes. Three earlier attempts that treated them as races did not work.

Victim Polluter (earlier on the same shard) Leak
adminDataViewsMigration Store Reviews tabs adminModules.spec.ts the "reset" derived its original state from catalogue presence, which is always true, so it deactivated the module instead of restoring it
productAdvertising checkout paymentsPage.ts togglePaymentMethod() blind-flipped despite its "enable" comment, disabling an already-enabled offline gateway
newAuction React editor productsDetails.spec.ts pinned vendor_product_editor to legacy and never restored it, unlike its three sibling specs
newProducts empty Draft tab newProductForm.spec.ts saved a draft product and never deleted it

A sweep for the same bug class found six more, all fixed here:

  • productsDetails.spec.ts also flips its seeded product to draft and deleted none of its seven seeded products.
  • setting.spec.ts ends the terms-and-conditions test on off when the seeded default is on — the leak singleStore.spec.ts was carrying a defensive re-seed for. Fixed at both ends.
  • vendorSettings.spec.ts kept its catalog and min-max resets inside a test, so a skip, a failure or a --grep left them applied.
  • newWithdrawB15.spec.ts left the disbursement schedule configured.
  • noticeAndPromotionPage.ts — unrelated. The promotion feed can serve more than one live promotion, so the slider renders n slides and a bare toBeVisible() throws a strict-mode violation. It now asserts every match, which is stricter than the single-element assertion it replaces.

Two traps worth knowing when fixing this class: dbUtils.updateOptionValue deep-merges, so it can never remove a key a spec added — use setOptionValue for a true restore. And a restore placed inside a test only runs if that test runs and passes.

3. Shard rebalance

utils/shard-durations.json regenerated (the baseline was from 2026-07-15, 190 specs, now 195). Measured spread went from 71.2 min to roughly 0.1 min estimated, confirmed against real CI wall-clock: shards now finish in 15–21 min rather than 24–48 min.

4. CI — end-of-shard isolation re-run

After a shard finishes, only the tests that failed are re-run without their shard-mates, and that second pass decides the shard result.

  • No recorded test failures means no re-run. If the first pass died from a crash or timeout rather than assertions, --last-failed would select zero tests and trivially pass; the gate keeps the shard red instead.
  • The re-run overrides the reporters, because the blob reporter would emit a second set of results for the same tests and merge-reports would double-count them. The merged HTML report therefore keeps showing the first-pass failures.
  • It runs before the coverage step, which overwrites .last-run.json.
  • Every rescued test is written to the step summary, naming shard-state pollution as the likely cause.

Trade-off, stated deliberately. Playwright's in-place retries cannot green a deterministic shard-context failure, because all three attempts run against the same dirty state. This can, and the re-run inherits retries. The four pollution bugs in section 2 would have been reported green-with-a-warning rather than red under this policy. That was an explicit decision, not an oversight.

Verification

Run Commit Result
31073883028 (nightly, pre-work) 43ab2b728 17 failures
31241990565 626a8fe4a 5 failures, 0 Stripe
31249977219 3949b5306 12/12 shards green
31252141880 b077d3878 12/12 shards green

Each state-restore fix was verified by running the polluter and reading the global state back — modules, offline gateways, product editor, terms-and-conditions, catalog mode, withdraw disbursement and draft count — rather than by re-running the victim, which proves nothing. tsc --noEmit holds at the develop baseline of 82 throughout.

Known gaps

  • The isolation re-run has never executed in CI: both green runs had zero failing shards, so all three new steps skipped. It is verified locally against a throwaway spec only.
  • productsDetails "vendor can create product tags" is a pre-existing flake measured at 20% raw failure locally, and it consumed both CI retries in one run. Not fixed here.
  • newProducts "vendor can view product list page" is a new retry-masked flake, first seen inside a green run.

All Submissions:

  • My code follow the WordPress' coding standards
  • My code satisfies feature requirements
  • My code is tested
  • My code passes the PHPCS tests
  • My code has proper inline documentation
  • I've included related pull request(s) (optional)
  • I've included developer documentation (optional)
  • I've added proper labels to this pull request

Changes proposed in this Pull Request:

Test-only and CI-only. No plugin source is touched. See sections 1–4 above.

Related Pull Request(s)

  • None

Closes

  • Closes getdokan/plugin-internal-tasks#2211

How to test the changes in this Pull Request:

  1. Trigger the E2E workflow on this branch and confirm 12/12 shards pass (runs 31249977219 and 31252141880 already do).
  2. For the state-restore fixes specifically, run a polluter on its own and read the global back, for example:
    npx playwright test --project=e2e_tests --workers=1 tests/e2e/admin/adminModules.spec.ts
    then confirm the Store Reviews module is still active. Repeat for payments, productsDetails and newProductForm against their respective globals. Re-running the victim test proves nothing, because it passes on a clean database either way.

Changelog entry

Fix e2e suite failures caused by Stripe block-checkout phone validation and cross-spec shard state leaks

Previously the nightly e2e run was red: the Stripe block checkout never submitted because the seeded billing phone was rejected as a non-mobile number, and four further tests failed because other specs on the same shard left global options, modules and seeded products dirty. Now the checkout submits, every spec restores the global state it changes, the shard baseline is balanced, and a shard's failures are re-run in isolation before the shard is called red.

Before Changes

Nightly develop e2e red since 2026-07-31, 17 failures at its worst. Shard wall-clock spread 15.7–86.9 min.

After Changes

12/12 shards green on two consecutive runs. Shard wall-clock 15–21 min.

PR Self Review Checklist:

  • Code is not following code style guidelines
  • Bad naming: make sure you would understand your code if you read it a few months from now.
  • KISS: Keep it simple, Sweetie (not stupid!).
  • DRY: Don't Repeat Yourself.
  • Code that is not readable: too many nested 'if's are a bad sign.
  • Performance issues
  • Complicated constructions that need refactoring or comments: code should almost always be self-explanatory.
  • Grammar errors.

FOR PR REVIEWER ONLY:

As a reviewer, your feedback should be focused on the idea, not the person. Seek to understand, be respectful, and focus on constructive dialog.

As a contributor, your responsibility is to learn from suggestions and iterate your pull request should it be needed based on feedback. Seek to collaborate and produce the best possible contribution to the greater whole.

  • Correct — Does the change do what it’s supposed to? ie: code 100% fulfilling the requirements?
  • Secure — Would a nefarious party find some way to exploit this change? ie: everything is sanitized/escaped appropriately for any SQL or XSS injection possibilities?
  • Readable — Will your future self be able to understand this change months down the road?
  • Elegant — Does the change fit aesthetically within the overall style and architecture?

…-checkout failure diagnosable

The nightly develop e2e run has been red since 2026-07-31. Four of the failures are
test-side defects and are fixed here; the thirteen stripe-express ones are NOT fixed,
because their cause is still unproven and papering over them would hide a real signal.

Every fix below is a race/wait/selector fix. No assertion was weakened and no test was
skipped.

Four flakes, all the same shape — a one-shot read of a React surface taken before it
settles, then compared against a later value:

* customer: addDefaultProductToCartAndAssert raced
  `Promise.all([waitForLoadState('load'), click()])`. A click that triggers navigation
  already auto-waits for scheduled navigations, so pairing it with a second load wait
  made the click itself time out at 30s under CI load. Click, then assert arrival
  web-first. Both navigation sites in the method.

* abuse-reports: waitForListReady() returns on the first render that has any rows, which
  is not the final render. The TC10 baseline captured a transient 20 rows that
  re-rendered to the real 2. Adds getStableRowCount() (same count twice running) and a
  web-first expectRowCount().

* vendor-auction / request-for-quotes / vendor-support: parseTabCount cannot distinguish
  a genuine "(0)" from a tab whose count has not rendered yet, because DataViews paints
  "All" before "All (3)". Adds expectTabCountAtLeast() to utils/dataViews.ts and applies
  it to the three specs with that shape. Expected-zero assertions deliberately keep the
  plain read — polling for zero would just burn the timeout.

* vendor-reports: reports/helper.ts::shouldBlockNavigation() redirects report paths via
  document.location.href. When that beats DOMContentLoaded the browser cancels the
  original navigation and goto() rejects with net::ERR_ABORTED even though the page loads
  fine. Adds gotoReportUrl(), which swallows ONLY ERR_ABORTED and then re-waits, and
  routes gotoReports/gotoAnalytics/testPageLoadPerformance through it. Also strengthens
  the perf test: it measured elapsed time but never asserted the page rendered, so an
  instant error page would have passed the 30s budget.

Stripe block checkout — two real defects fixed, root cause still open:

* Impossible wait budget. The post-click waits were 60s (waitForURL) + 120s (settle poll)
  = 180s minimum, inside specs configured at 150s and before any setup cost. The poll,
  which is what decides pass/fail, could never run to completion on CI: the test died
  mid-poll and reported the poll's placeholder ("none") rather than a verdict. The SPA
  redirect is the known-unreliable path on CI, so it no longer gets 60s of that budget
  there (15s on CI, full 60s locally where it is the fast path), and the two affected
  specs move to 240s to match the sibling money specs.

* The suite was blind to its own failure mode. After clicking Place Order it only polled
  the REST API, so "the Store API rejected the payment" and "the click was a no-op because
  the block was not submittable" both surfaced as the same "none". placeBlockOrderExpect-
  Received now records every POST /wc/store/v1/checkout and reports it in the failure.
  That is what distinguishes product-side from suite-side on the next run.

Evidence for leaving the thirteen alone rather than patching them green: a local A/B
passes both without and with #3345 (28s / 26s), the suite's own SIMULATE_CONFIRM_BLOCK
repro also passes (35s, the fallback works), and the failing shards' databases contain
zero dokan_stripe_express orders while saved-token payments on the same shard succeed.
So the order is genuinely never created, and the cause is not yet established.
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The Playwright end-to-end tests now use retrying count assertions, web-first interactions, redirect-tolerant report navigation, Stripe checkout diagnostics, valid billing fixtures, longer timeouts, and refreshed shard-duration data.

Changes

End-to-end test stability

Layer / File(s) Summary
Asynchronous count assertions
tests/pw/utils/dataViews.ts, tests/pw/tests/e2e/request-for-quotes/newRequestedQuotes.spec.ts, tests/pw/tests/e2e/vendor-auction/newAuction.spec.ts, tests/pw/tests/e2e/vendor-support/newVendorSupport.spec.ts, tests/pw/tests/e2e/abuse-reports/abuseReportsPage.ts, tests/pw/tests/e2e/abuse-reports/abuseReports.spec.ts
Added retrying tab-count and row-count assertions. Abuse report cancellation now uses stable counts and a web-first row-count assertion.
Web-first page interactions
tests/pw/tests/e2e/customer/customerPage.ts
Customer cart setup uses direct clicks and visibility assertions instead of explicit load-state waits.
Redirect-tolerant report navigation
tests/pw/tests/e2e/vendor-reports/vendorReportsPage.ts
Report navigation handles client-side redirects, rethrows other navigation errors, and verifies the analytics shell.
Stripe checkout diagnostics and timing
tests/pw/tests/e2e/stripe-express/stripeExpressPage.ts, tests/pw/tests/e2e/stripe-express/stripeExpress.spec.ts, tests/pw/tests/e2e/stripe-express/stripeExpressXss.spec.ts
Checkout validates all card fields, captures Store API responses, applies CI-specific redirect timing, tightens payment-error checks, and reports submission diagnostics. Stripe suite timeouts increase to 240 seconds.
Billing phone fixtures
tests/pw/utils/payloads.ts, tests/pw/tests/e2e/orders/ordersPage.ts, tests/pw/tests/e2e/stripe-express/stripeExpressEdge.spec.ts, tests/pw/tests/e2e/stripe-express/stripeExpressGuest.spec.ts, tests/pw/tests/e2e/stripe-express/stripeExpressPage.ts
Order payloads and Stripe Express billing fixtures use +14152367890 instead of 555-number placeholders.
Shard duration dataset
tests/pw/utils/shard-durations.json
The 12-shard duration data now uses a reordered per-spec list with updated durations and no test-count or aggregate-note fields.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant placeBlockOrderExpectReceived
  participant StoreAPICheckout
  participant OrderReceivedRedirect
  participant confirmNewPaidStripeOrder
  placeBlockOrderExpectReceived->>StoreAPICheckout: submit checkout request
  StoreAPICheckout-->>placeBlockOrderExpectReceived: return status and response body
  placeBlockOrderExpectReceived->>OrderReceivedRedirect: wait for order-received redirect
  alt redirect succeeds
    OrderReceivedRedirect-->>placeBlockOrderExpectReceived: return success
  else redirect fails
    OrderReceivedRedirect-->>placeBlockOrderExpectReceived: return redirect failure
    placeBlockOrderExpectReceived->>confirmNewPaidStripeOrder: confirm payment fallback
    confirmNewPaidStripeOrder-->>placeBlockOrderExpectReceived: return settlement result
  end
Loading

Suggested reviewers: dev-shahed

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the primary e2e CI flake fixes and Stripe block-checkout work.
Description check ✅ Passed The description is detailed, follows the repository template, documents causes and verification, and identifies known gaps.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch qa/ci-e2e-flake-fixes-and-stripe-diagnostic

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (1)
tests/pw/tests/e2e/vendor-reports/vendorReportsPage.ts (1)

170-180: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add the required @since tag.

This new JSDoc block has no @since tag. Add @since DOKAN_SINCE before the closing marker. Do not use a guessed release version.

As per coding guidelines, for new code documentation, use the literal placeholder @since DOKAN_SINCE.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/pw/tests/e2e/vendor-reports/vendorReportsPage.ts` around lines 170 -
180, Add the literal `@since` DOKAN_SINCE tag to the new JSDoc block above the
closing marker, without guessing a release version or changing the existing
documentation.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/pw/tests/e2e/abuse-reports/abuseReportsPage.ts`:
- Around line 534-545: Update getStableRowCount to require a sustained stability
window: reset the window whenever rows.count() changes, and return the count
only after the unchanged window fully elapses. If the overall timeout expires
before stability is achieved, throw instead of returning the last potentially
unstable count.

In `@tests/pw/tests/e2e/stripe-express/stripeExpressPage.ts`:
- Around line 580-583: Update the response handling around storeApiAttempts so
it immediately records a placeholder when Playwright observes the response,
before calling res.text(). Replace that placeholder with the truncated body on
success, or the unavailable-body marker on failure, while preserving the HTTP
status.

In `@tests/pw/tests/e2e/vendor-reports/vendorReportsPage.ts`:
- Around line 505-510: Update assertAnalyticsShell() and its call from the
report performance check to require a reports-specific selector—analyticsRoot or
reportsMenuActive—to be visible, while retaining the PHP-fatal check. Remove
bodyLen-based success logic so login, generic error, or unrelated pages cannot
satisfy the shell assertion.
- Around line 181-188: Update gotoReportUrl to validate navigation to the
expected canonical report URL with page.waitForURL(...) after page.goto,
preserving ERR_ABORTED handling only when that expected URL is reached; re-throw
navigation failures otherwise, then wait for report-specific content instead of
relying solely on waitForLoadState('domcontentloaded').

In `@tests/pw/utils/dataViews.ts`:
- Around line 184-196: Add the literal `@since` DOKAN_SINCE tag to the JSDoc
blocks for expectTabCountAtLeast in tests/pw/utils/dataViews.ts (lines 184-196),
getStableRowCount in tests/pw/tests/e2e/abuse-reports/abuseReportsPage.ts (lines
523-533), and expectRowCount in
tests/pw/tests/e2e/abuse-reports/abuseReportsPage.ts (line 547); do not use a
released or guessed version.

---

Nitpick comments:
In `@tests/pw/tests/e2e/vendor-reports/vendorReportsPage.ts`:
- Around line 170-180: Add the literal `@since` DOKAN_SINCE tag to the new JSDoc
block above the closing marker, without guessing a release version or changing
the existing documentation.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b0597ff9-1bf0-413f-a145-abb95b1f93d9

📥 Commits

Reviewing files that changed from the base of the PR and between 542e45e and bd210f5.

📒 Files selected for processing (11)
  • tests/pw/tests/e2e/abuse-reports/abuseReports.spec.ts
  • tests/pw/tests/e2e/abuse-reports/abuseReportsPage.ts
  • tests/pw/tests/e2e/customer/customerPage.ts
  • tests/pw/tests/e2e/request-for-quotes/newRequestedQuotes.spec.ts
  • tests/pw/tests/e2e/stripe-express/stripeExpress.spec.ts
  • tests/pw/tests/e2e/stripe-express/stripeExpressPage.ts
  • tests/pw/tests/e2e/stripe-express/stripeExpressXss.spec.ts
  • tests/pw/tests/e2e/vendor-auction/newAuction.spec.ts
  • tests/pw/tests/e2e/vendor-reports/vendorReportsPage.ts
  • tests/pw/tests/e2e/vendor-support/newVendorSupport.spec.ts
  • tests/pw/utils/dataViews.ts

Comment on lines +534 to +545
async getStableRowCount(timeout = 15000): Promise<number> {
const rows = this.page.locator(this.adminReact.dataRow);
let last = -1;
const deadline = Date.now() + timeout;
while (Date.now() < deadline) {
const current = await rows.count();
if (current === last) return current;
last = current;
await this.page.waitForTimeout(300);
}
return last;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Wait for a sustained stable period.

Line 540 returns after one unchanged 300 ms interval. The intermediate row count described at Lines 526-530 can persist across that interval and then change on a later render. The helper can still return the transient baseline that this change intends to avoid.

Reset a stability window after each count change. Return only after that window elapses. Throw when the deadline expires instead of returning a potentially unstable count.

Proposed fix
 async getStableRowCount(timeout = 15000): Promise<number> {
     const rows = this.page.locator(this.adminReact.dataRow);
-    let last = -1;
     const deadline = Date.now() + timeout;
+    let last = await rows.count();
+    let stableSince = Date.now();
+
     while (Date.now() < deadline) {
+        await this.page.waitForTimeout(300);
         const current = await rows.count();
-        if (current === last) return current;
-        last = current;
-        await this.page.waitForTimeout(300);
+        if (current !== last) {
+            last = current;
+            stableSince = Date.now();
+            continue;
+        }
+        if (Date.now() - stableSince >= 1000) return current;
     }
-    return last;
+    throw new Error(`Abuse report row count did not settle within ${timeout} ms`);
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
async getStableRowCount(timeout = 15000): Promise<number> {
const rows = this.page.locator(this.adminReact.dataRow);
let last = -1;
const deadline = Date.now() + timeout;
while (Date.now() < deadline) {
const current = await rows.count();
if (current === last) return current;
last = current;
await this.page.waitForTimeout(300);
}
return last;
}
async getStableRowCount(timeout = 15000): Promise<number> {
const rows = this.page.locator(this.adminReact.dataRow);
const deadline = Date.now() + timeout;
let last = await rows.count();
let stableSince = Date.now();
while (Date.now() < deadline) {
await this.page.waitForTimeout(300);
const current = await rows.count();
if (current !== last) {
last = current;
stableSince = Date.now();
continue;
}
if (Date.now() - stableSince >= 1000) return current;
}
throw new Error(`Abuse report row count did not settle within ${timeout} ms`);
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/pw/tests/e2e/abuse-reports/abuseReportsPage.ts` around lines 534 - 545,
Update getStableRowCount to require a sustained stability window: reset the
window whenever rows.count() changes, and return the count only after the
unchanged window fully elapses. If the overall timeout expires before stability
is achieved, throw instead of returning the last potentially unstable count.

Comment on lines +580 to +583
void res
.text()
.then(body => storeApiAttempts.push(`HTTP ${status}: ${body.slice(0, 300)}`))
.catch(() => storeApiAttempts.push(`HTTP ${status}: <body unavailable>`));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Record the response before reading its body.

storeApiAttempts gets an entry only after res.text() resolves. If the response body stalls, the failure message says that no checkout POST occurred, even though Playwright observed the response. Push a placeholder entry before calling res.text(). Update that entry when the body becomes available.

Proposed fix
-const storeApiAttempts: string[] = [];
+const storeApiAttempts: Array<{ status: number; body: string }> = [];

 const status = res.status();
+const attempt = { status, body: '<body unavailable>' };
+storeApiAttempts.push(attempt);
 void res
     .text()
-    .then(body => storeApiAttempts.push(`HTTP ${status}: ${body.slice(0, 300)}`))
-    .catch(() => storeApiAttempts.push(`HTTP ${status}: <body unavailable>`));
+    .then(body => {
+        attempt.body = body.slice(0, 300);
+    });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/pw/tests/e2e/stripe-express/stripeExpressPage.ts` around lines 580 -
583, Update the response handling around storeApiAttempts so it immediately
records a placeholder when Playwright observes the response, before calling
res.text(). Replace that placeholder with the truncated body on success, or the
unavailable-body marker on failure, while preserving the HTTP status.

Comment on lines +181 to +188
private async gotoReportUrl(url: string): Promise<void> {
await this.page.goto(url, { waitUntil: 'domcontentloaded' }).catch((err: unknown) => {
if (!/ERR_ABORTED/.test(String(err))) {
throw err;
}
});
await this.page.waitForLoadState('domcontentloaded');
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

file='tests/pw/tests/e2e/vendor-reports/vendorReportsPage.ts'

if ! rg -n 'waitForURL' "$file"; then
  echo "Expected a URL-aware wait in gotoReportUrl" >&2
  exit 1
fi

rg -n -C 8 'gotoReportUrl|ERR_ABORTED|waitForLoadState|waitForURL' "$file"

Repository: getdokan/dokan

Length of output: 195


🏁 Script executed:

#!/usr/bin/env bash
set -u

file='tests/pw/tests/e2e/vendor-reports/vendorReportsPage.ts'

# Print file metadata and the relevant method context without failing on missing waitForURL.
printf '--- candidate lines ---\n'
sed -n '155,210p' "$file" | cat -n

printf '\n--- related gotoReportUrl usages ---\n'
rg -n 'gotoReportUrl|ERR_ABORTED|waitForLoadState|waitForURL' "$file" || true

printf '\n--- Playwright API references in tests ---\n'
rg -n 'waitForURL\(|waitForLoadState\(' tests/pw/tests/e2e vendor-reports 2>/dev/null || true

Repository: getdokan/dokan

Length of output: 50370


Confirm the correct report URL before returning from gotoReportUrl.

ERR_ABORTED handling does not prove the application redirected to the canonical report URL. page.waitForLoadState('domcontentloaded') can complete for the current document without checking any new URL. Use page.waitForURL(...) with the expected report URL, then wait for the report-specific content; re-throw when the expected navigation does not occur.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/pw/tests/e2e/vendor-reports/vendorReportsPage.ts` around lines 181 -
188, Update gotoReportUrl to validate navigation to the expected canonical
report URL with page.waitForURL(...) after page.goto, preserving ERR_ABORTED
handling only when that expected URL is reached; re-throw navigation failures
otherwise, then wait for report-specific content instead of relying solely on
waitForLoadState('domcontentloaded').

Comment on lines +505 to +510
await this.gotoReportUrl(url);
const elapsed = Date.now() - start;
expect(elapsed).toBeLessThan(30000);
expect(await this.hasNoPhpFatal()).toBe(true);
expect(elapsed, `${url} should load within the 30s budget`).toBeLessThan(30000);
// Timing alone is not a load: a page that errored out instantly would "pass" the budget.
// Assert the reports shell actually rendered (this also re-checks for a PHP fatal).
await this.assertAnalyticsShell();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Require a reports-specific selector for the performance check.

The new call at Line 510 delegates to assertAnalyticsShell(). That helper treats bodyLen > 50 as success even when neither analyticsRoot nor reportsMenuActive is visible. A login page, generic error page, or unrelated document with enough text can pass this check.

Require analyticsRoot or reportsMenuActive, together with the PHP-fatal check. Do not use body length as proof that the reports shell rendered.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/pw/tests/e2e/vendor-reports/vendorReportsPage.ts` around lines 505 -
510, Update assertAnalyticsShell() and its call from the report performance
check to require a reports-specific selector—analyticsRoot or
reportsMenuActive—to be visible, while retaining the PHP-fatal check. Remove
bodyLen-based success logic so login, generic error, or unrelated pages cannot
satisfy the shell assertion.

Comment on lines +184 to +196
/**
* Web-first "this tab reports at least N" assertion.
*
* `parseTabCount` cannot distinguish a real zero from a not-rendered-yet tab: DataViews paints the
* label ("All") before the counted label ("All (3)"), and both a missing `(n)` and a genuine `(0)`
* come back as 0. A single `expect(await getTabCount('all')).toBeGreaterThanOrEqual(1)` therefore
* races the count in — the shape behind the intermittent CI failure "All tab count >= 1 after
* seeding / Received: 0".
*
* Polling fixes the race without weakening anything: the assertion still fails if the count never
* reaches `min`. Use this wherever a count is expected to be non-zero. An expected-zero assertion
* (`toBe(0)`) must NOT use this — poll there would just wait out the timeout.
*/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Add the required documentation version placeholder.

Each new JSDoc block must include @since DOKAN_SINCE. Do not add a released or guessed version.

  • tests/pw/utils/dataViews.ts#L184-L196: Add @since DOKAN_SINCE to expectTabCountAtLeast documentation.
  • tests/pw/tests/e2e/abuse-reports/abuseReportsPage.ts#L523-L533: Add @since DOKAN_SINCE to getStableRowCount documentation.
  • tests/pw/tests/e2e/abuse-reports/abuseReportsPage.ts#L547-L547: Add @since DOKAN_SINCE to expectRowCount documentation.

As per coding guidelines, new code documentation must use the literal placeholder @since DOKAN_SINCE.

📍 Affects 2 files
  • tests/pw/utils/dataViews.ts#L184-L196 (this comment)
  • tests/pw/tests/e2e/abuse-reports/abuseReportsPage.ts#L523-L533
  • tests/pw/tests/e2e/abuse-reports/abuseReportsPage.ts#L547-L547
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/pw/utils/dataViews.ts` around lines 184 - 196, Add the literal `@since`
DOKAN_SINCE tag to the JSDoc blocks for expectTabCountAtLeast in
tests/pw/utils/dataViews.ts (lines 184-196), getStableRowCount in
tests/pw/tests/e2e/abuse-reports/abuseReportsPage.ts (lines 523-533), and
expectRowCount in tests/pw/tests/e2e/abuse-reports/abuseReportsPage.ts (line
547); do not use a released or guessed version.

Source: Coding guidelines

…reports redirect fix

Follow-up to bd210f5, using what that run's diagnostic reported.

The Store API instrumentation answered the open question. Across the failing shards it
printed 36 times, always the same, and never once the alternative:

    The Blocks checkout never issued POST /wc/store/v1/checkout at all — the Place Order
    click was a NO-OP (block not submittable / validation blocked it), so no payment was
    ever attempted. This is a checkout-submission failure, not a declined payment.

So no payment was ever attempted: this is not a product defect and not a declined card.
The budget change also did its job — the settle poll now reports "Timeout 120000ms exceeded
while waiting on the predicate" instead of dying mid-poll at the 150s test timeout.

Root cause of the no-op: fillCardDetails only ever re-read the card NUMBER. The method
already knows the Payment Element can re-mount mid-entry, but a re-mount after the number
is typed keeps the number populated while silently dropping expiry and CVC. That leaves an
INCOMPLETE card, so elements.submit() fails validation, WooCommerce Blocks refuses to
submit, and the Place Order click does nothing — no POST, no order, and the settle poll can
only ever report "none". It matches the observed signature exactly: zero Store API attempts
on CI, while saved-token payments on the same runner succeed because they never touch the
card fields.

fillCardDetails now verifies all three fields persisted before returning, and names the one
that was lost when it retries.

Also, when no POST is issued, the failure now reads back the block's own notice banner and
the Payment Element's inline error, so the next such failure names the offending field
instead of requiring another round trip to find out.

vendor-reports: the previous ERR_ABORTED fix was incomplete and CI caught it. Swallowing the
abort stopped goto() throwing, but waitForLoadState then resolved instantly against the
cancelled, blank document while the app's redirect was still in flight — so the assertion
added in the same commit correctly caught an empty page (bodyLen <= 50). It now polls for
real rendered content instead of a load event, which still fails if the page genuinely never
renders.

Verified locally: the six flake-fix specs are 130 passed / 4 skipped (pre-existing skips) / 0
failed, and all 13 previously-failing stripe-express tests pass. Three of those 13 initially
failed under --workers=2 on transfer contention; an A/B against the pushed revision showed
they pass with and without this change when run serially, which is how CI runs them
(playwright.config.ts sets workers to 1 on CI).

Note the limit of that evidence: local cannot reproduce the PE re-mount, so local green here
proves no regression rather than proving the fix. The added diagnostics mean the next run
either goes green or names the field that failed validation.
…n submit

Follow-up to d74cbc9, acting on what that run's diagnostic printed.

Run 31095722827 reported, on every failing Stripe block-checkout test:

    The Blocks checkout never issued POST /wc/store/v1/checkout at all — the Place Order
    click was a NO-OP (block not submittable / validation blocked it), so no payment was
    ever attempted.
    Block validation said: Your mobile phone number is invalid.

38 NO-OP diagnostics, zero Store API attempts. The card was NOT the problem — the
"card did not fully persist" guard added in the same commit never printed once, so the Payment
Element filled correctly and the incomplete-card theory it was testing is dead.

The billing phone seeded for these flows was `(555) 555-5555`. 555-555-XXXX is the reserved
fictional range, and Stripe Link validates the billing phone as a MOBILE number, so it rejects
it with exactly that message. The rejection blocks the WooCommerce Blocks checkout from
submitting at all, which is why the settle poll could only ever report "none".

Replaced with `+14152367890` in all eight places: a real-format US mobile with a valid area code
and exchange, in E.164 so no validator has to infer the country. Nothing asserts on this value —
every occurrence is a seed — so this cannot change what any test proves. The `createOrder.billing`
copy carries a comment explaining the failure mode so it does not get tidied back to a 555 number.

Where the evidence stops, stated plainly: this does NOT reproduce locally, in any configuration
tried. Locally the same customer carries the same number and SE-CHK-B-01 passes; the Stripe API
accepts `(555) 555-5555` on a PaymentMethod when tested directly; and temporarily un-blocking
merchant-ui-api so the Link wallet mounts still passes. The message is emitted by Stripe, not by
anything in wp-content — grepped the whole install and the running container. So the reasoning is
that Link's mobile validation only engages on the runner, consistent with the runner-IP Link and
hCaptcha behaviour already documented in gotoBlockCheckout(). Only CI can confirm it.

If CI is still red, the diagnostic will print `Block validation said: …` again. A CHANGED message
means the phone was one cause and there is another; an IDENTICAL message means the phone is not
the cause and the next thing to attack is Link mounting on the runner at all.

Verified locally: tsc --noEmit at the develop baseline of 82, SE-CHK-B-01 green in 28.9s, and the
seeded customer really does now carry +14152367890 rather than the value merely changing in source.
…to have content

Third attempt at this one. The first two each failed on CI and each narrowed it:

1. Swallowing only ERR_ABORTED stopped goto() throwing, but waitForLoadState then returned
   instantly against the cancelled, blank document whose load event had already fired.
2. Polling for rendered content passed against the PRE-redirect page; the redirect then blanked
   it, so assertAnalyticsShell still saw bodyLen <= 50 a moment later (run 31095722827).

Content alone is not a settled page. shouldBlockNavigation() can redirect a report path more than
once, so the only safe signal is the URL holding still AND the body having content in the same
observation — both true twice running before gotoReportUrl returns.

On timeout it now reports the final URL and body length rather than leaving the caller to fail on
a bare `expect(false).toBe(true)`, which is what made the previous two rounds hard to read.

Verified locally: tsc --noEmit at the develop baseline of 82, and the whole vendorReports spec is
41 passed / 4 skipped (pre-existing skips) / 0 failed.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/pw/tests/e2e/vendor-reports/vendorReportsPage.ts`:
- Around line 190-202: Add the literal `@since` DOKAN_SINCE tag to the
documentation block for gotoReportUrl, preserving the existing documentation and
placement.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 2eba6f4b-8e10-4e10-9770-0e4dca62e906

📥 Commits

Reviewing files that changed from the base of the PR and between b38c2dc and 91088fd.

📒 Files selected for processing (1)
  • tests/pw/tests/e2e/vendor-reports/vendorReportsPage.ts

Comment on lines +190 to +202
* Wait for the page to STOP MOVING, not merely to have content once.
*
* Two earlier attempts failed on CI and each taught something:
* 1. Swallowing ERR_ABORTED alone — `waitForLoadState` then returned instantly against the
* cancelled, blank document, whose load event had already fired.
* 2. Polling for rendered content — that passed against the PRE-redirect page, and the
* redirect then blanked it, so `assertAnalyticsShell` still saw bodyLen <= 50 a moment
* later. Content alone is not a settled page.
*
* `shouldBlockNavigation()` can redirect a report path more than once, so the only safe
* signal is the URL holding still AND the body having content on the same observation.
* Both must be true twice running before this returns.
*/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Add the required documentation version placeholder.

Add @since DOKAN_SINCE to the gotoReportUrl documentation block. This new documentation must use the literal placeholder.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/pw/tests/e2e/vendor-reports/vendorReportsPage.ts` around lines 190 -
202, Add the literal `@since` DOKAN_SINCE tag to the documentation block for
gotoReportUrl, preserving the existing documentation and placement.

Source: Coding guidelines

…m error, rebalance shards

Run 31147407922 took the branch from 14 failures to 1. The phone fix landed: every shard that
carried the Stripe block-checkout tests went green, and vendor-reports passed on its third attempt.
This clears the last failure and fixes the sharding.

SE-GUEST-01. The diagnostic message CHANGED, which is what identified this:

    before:  Block validation said: Your mobile phone number is invalid.
    now:     Block validation said: Please provide a mobile phone number.

Not invalid any more — absent. A guest has no saved address, so nothing pre-fills the phone the way
ensureCustomerAddress() does for logged-in flows; it has to be typed. fillBlockGuestDetails() had no
phone parameter at all. Added as optional, so existing callers are unaffected, and tolerant of the
field being missing when a store has the phone field switched off.

SE-GUEST-03 was passing for the wrong reason, and would have kept passing. It asserts "an inline
error appears and we never reach order-received" — which is exactly what a MISSING PHONE produces.
The declined card was never submitted, so the test proved nothing about a decline. Two changes:
the guest details now carry a phone so the decline is genuinely exercised, and
placeBlockOrderExpectError() now requires the notice to be about the payment
(/declin|card|payment|insufficient|cvc|expir/), so an address or contact validation error can never
satisfy a decline test again.

Shard rebalancing, regenerated from this run's spec-durations.json across all 12 shards (195 specs,
up from 190). The previous baseline was three weeks old (2026-07-15) and badly wrong:

    old: shards ranged 15.7 -> 86.9 min, spread 71.2 min (190% of ideal)
    new: shards range 16.7 -> 16.8 min, spread 0.1 min (0.6%)

That matches the wall-clock actually observed on CI, where one shard took 48m13s while another took
23m51s. The 18 paypal-marketplace specs are not in this run's data and fall back to the mean, which
is fine — they skip in seconds without credentials. Regenerate again once PayPal really runs.

One inflated entry to be aware of: stripeExpressGuest is recorded at 9.3 min because SE-GUEST-01
failed and retried three times in the run the data came from. It will drop once this commit is
verified green. Over-estimating a spec only spreads load away from it, so it is safe to ship.

Verified locally: tsc --noEmit at the develop baseline of 82, and the whole guest spec passes
(3/3) with the stricter payment-error assertion in place.
… keywords

Reverts a regression I introduced in 8627018. That commit tightened
placeBlockOrderExpectError() to require the notice to mention the payment
(/declin|card|payment|insufficient|cvc|expir/), which broke all three decline tests on CI run
31162562169: SE-CHK-B-03, SE-EDGE-04 and SE-GUEST-03.

A real Stripe decline does not surface with any of those words. WooCommerce Blocks reports it as the
generic "Something went wrong. Please contact us to get assistance.", so the assertion rejected a
genuine decline. The intent was right, the implementation assumed copy it had no business assuming.

Inverted: reject the FORM-VALIDATION family instead
(/please provide|please enter|is invalid|is required|enter a valid/) and accept anything else. That
still closes the loophole the guard exists for — "Please provide a mobile phone number." satisfied
the original bare toBeVisible() check, so a decline test could pass without the card ever being
submitted — without depending on how Stripe or Woo happen to word a decline.

Also makes the guest phone fill fail loudly. It previously guarded with `if (count())` and swallowed
fill errors, so a renamed field was indistinguishable from "this store has no phone field", and the
caller only found out much later as a no-op Place Order click with no visible validation message. It
now asserts the field is visible and that the value persisted.

Verified locally, each run individually: SE-CHK-B-03 (24.0s), SE-EDGE-04 (36.3s) and SE-GUEST-03
(33.8s) all pass. tsc --noEmit at the develop baseline of 82.

Still open on this branch, not addressed here: SE-GUEST-01 (the phone message is gone but it now
fails with no visible validation message, and it reproduces locally), and four tests that began
failing with the shard rebalance in 8627018 — adminDataViewsMigration, productAdvertising,
newProducts draft tab and newAuction row-edit. The rebalance itself worked, shards went from
23:51-48:13 down to 15:15-21:33, but regrouping changes which specs share a worker and that can
expose cross-test coupling. Those four are undiagnosed.
The four non-Stripe CI failures on this branch were never flakes. Each is a
different spec earlier on the same shard leaving global state dirty, so the
victim fails on CI while passing in isolation on a clean database:

- adminModules derived `wasActive` from catalogue PRESENCE, which is always
  true, so its "reset" deactivated Store Reviews instead of restoring it and
  adminDataViewsMigration then found no DataViews tabs.
- paymentsPage.togglePaymentMethod blind-flipped the offline gateways despite
  the "enable" comment, disabling the already-enabled bacs and timing out
  productAdvertising's checkout.
- productsDetails pinned `vendor_product_editor` to 'legacy' and never restored
  it, so newAuction's Edit action opened the legacy screen.
- newProductForm saved a draft product and never deleted it, so newProducts'
  "empty Draft tab" assertion saw a row.

Fix each at the source: capture real state and restore it, set the desired
state rather than flipping whatever is there, and clean up seeded products.

Also fixes the same bug class in five more specs found by sweeping for
mutations without restores. productsDetails additionally flips its seeded
product to 'draft' and deleted none of its seven products; setting.spec ends
the terms-and-conditions test on 'off' when the seeded default is 'on', which
is the leak singleStore carried a defensive re-seed for; vendorSettings kept
its catalog and min-max resets INSIDE a test, so a skip or failure left them
applied; newWithdrawB15 left the disbursement schedule configured. Restores go
through setOptionValue where a merge could not remove an added key.

noticeAndPromotion is unrelated: Dokan's promotion feed can serve more than one
live promotion, so the slider renders n slides and a bare toBeVisible() throws a
strict-mode violation. Assert every match instead.

Verified by running each polluter and reading the global state back: modules,
offline gateways, product editor, terms-and-conditions, catalog mode, withdraw
disbursement and draft count all return to their pre-run values. 285 tests
passed across the two runs with no failures. Whether the four victims now pass
inside their real shards still needs a CI run.
After a shard finishes, re-run only the tests that failed, without their
shard-mates, and let that second pass decide the shard's result. A test that
fails in shard context but passes alone is separated from one that fails either
way.

The main test step becomes continue-on-error and a gate step at the end fails
the shard unless the isolation re-run cleared every failure. `outcome` still
reports the true first-pass result, so the six later steps that key off
steps.e2e-test.outcome are unaffected.

Guard rails:

- No recorded test failures means no re-run. If the first pass died from a crash
  or a timeout rather than assertions, `--last-failed` would select zero tests
  and trivially "pass", turning an infrastructure failure green. The gate keeps
  the shard red instead.
- The re-run overrides the reporters with --reporter=list. The blob reporter
  would otherwise emit a second set of results for the same tests and
  merge-reports would double-count them. The merged HTML report therefore keeps
  showing the first-pass failures, which is what actually happened in shard
  context.
- It runs before the coverage step, which overwrites .last-run.json.
- Every rescued test is written to the step summary. A failure that only
  reproduces alongside its shard-mates is the signature of shard-state
  pollution, not a timing flake, and the summary says so and points at how to
  confirm it.

Known trade-off: Playwright's in-place `retries` cannot green a deterministic
shard-context failure, because all three attempts run with the same dirty state.
This can, and the re-run inherits retries, so a test now gets three attempts in
context plus three alone. The four state-pollution bugs fixed in 3949b53 would
have been reported green with a warning rather than red.

Verified locally with a throwaway spec: --last-failed selects only the failed
test, a still-failing re-run exits 1 and fails the gate, a recovered one exits 0
and greens the shard, and an empty or corrupt .last-run.json skips the re-run.
const longUnicodeLast = 'Tëst-Sürñámé-日本語-' + 'Ω'.repeat(12);
const longUnicodeAddr = 'Straße Ñoño 日本語 番地 ' + 'Ä'.repeat(48);
const billing = { first_name: longUnicodeFirst, last_name: longUnicodeLast, company: '', address_1: longUnicodeAddr, address_2: '', city: 'New York', state: 'NY', postcode: '10003', country: 'US', email: 'customer1@email.com', phone: '(555) 555-5555' };
const billing = { first_name: longUnicodeFirst, last_name: longUnicodeLast, company: '', address_1: longUnicodeAddr, address_2: '', city: 'New York', state: 'NY', postcode: '10003', country: 'US', email: 'customer1@email.com', phone: '+14152367890' };

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldnt we put this on testdata?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — done in 2bd8801.

It turned out to be wider than this one line: the phone was hardcoded in seven places across five files — three billing seeds in payloads.ts, three guest-checkout fills in stripeExpressGuest.spec.ts, the page object's customer fixture and the orders page fixture. All of them now reference a single exported MOBILE_TEST_PHONE in utils/payloads.ts.

I put the reason for the value on the constant rather than leaving it bare. Stripe Link validates the billing phone as a mobile number and rejects the reserved fictional (555) 555-5555 range; when it does, WooCommerce Blocks refuses to submit at all, so no Store API request fires and no order is created — the test fails with no visible card error. Without that note the value looks arbitrary and the obvious tidy-up back to a fictional number would silently break the Blocks checkout again.

Side benefit: +14152367890 is a plausibly-real San Francisco number. It is seed data and nothing is ever sent to it, but if we want to swap it for something safer (libphonenumber's canonical +14155552671, inside the reserved 555 range but with a valid exchange) that is now a one-line change instead of a hunt across five files.

No behaviour change — same value everywhere it was already used. tsc holds at the 82-error develop baseline with a byte-identical error set.

Note: paypalMarketplaceGuest.spec.ts has the same literal but is not in this PR, so I left it for the PayPal branch rather than pulling an unrelated file in here.

Review feedback on #3360: the phone belonged in test data, not inline in a spec.

It was hardcoded in seven places across five files — three billing seeds in
payloads.ts, three guest-checkout fills, the page object's customer fixture and
the orders page fixture. A single exported MOBILE_TEST_PHONE now backs all of
them.

The constant carries the reason for its value. Stripe Link validates the billing
phone as a MOBILE number and rejects the reserved fictional (555) 555-5555
range; when it does, WooCommerce Blocks refuses to submit at all, so no Store
API request fires, no order is created, and the checkout test fails with no
visible card error to point at. Without that note the value reads like an
arbitrary string and the obvious "tidy-up" to a fictional number silently breaks
the Blocks checkout again.

Keeping it in one place also means a validator change is a one-line edit rather
than a hunt across five files.

No behaviour change: same value everywhere it was already used. tsc holds at the
82-error develop baseline and the error set is byte-identical before and after.
@shohan0120
shohan0120 merged commit 79a2cd8 into develop Aug 10, 2026
16 checks passed
@shohan0120
shohan0120 deleted the qa/ci-e2e-flake-fixes-and-stripe-diagnostic branch August 10, 2026 03:57
shohan0120 added a commit that referenced this pull request Aug 10, 2026
develop now carries the CI flake fixes and the Stripe block-checkout work as
the squashed #3360, so the nine commits this branch already merged from
qa/ci-e2e-flake-fixes-and-stripe-diagnostic arrive again with a different
history and conflict on content that is already identical.

Five files conflicted, all on the same change: develop's MOBILE_TEST_PHONE
constant, which landed in 2bd8801 after this branch's merge point. Resolved
by taking develop's side in every case; the branch held no changes to those
files that develop lacks.

The PR diff collapses from 59 files to 33 - the PayPal Marketplace suite plus
the shared infra it introduced.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant