Skip to content

mach: Add --update-expectations flag to test-wpt - #45521

Merged
TimvdLippe merged 2 commits into
servo:mainfrom
TimvdLippe:add-update-expectations-flag
Jun 10, 2026
Merged

mach: Add --update-expectations flag to test-wpt#45521
TimvdLippe merged 2 commits into
servo:mainfrom
TimvdLippe:add-update-expectations-flag

Conversation

@TimvdLippe

Copy link
Copy Markdown
Contributor

With this new flag, it automatically runs update-wpt at the end, without the need to specify a --log-raw argument.

Fixes #45476

Testing: Local runs with the various combinations of flags

@servo-highfive servo-highfive added the S-awaiting-review There is new code that needs to be reviewed. label Jun 9, 2026
@TimvdLippe
TimvdLippe force-pushed the add-update-expectations-flag branch from 52ab26d to 6738820 Compare June 9, 2026 19:22
@lumiscosity

Copy link
Copy Markdown
Member

Might be nice to have a book PR for this as well!

@TimvdLippe
TimvdLippe force-pushed the add-update-expectations-flag branch from 6738820 to fecbab2 Compare June 9, 2026 19:25
@TimvdLippe

Copy link
Copy Markdown
Contributor Author

Might be nice to have a book PR for this as well!

Yup will do once this merges

@TimvdLippe
TimvdLippe force-pushed the add-update-expectations-flag branch from fecbab2 to 881535d Compare June 9, 2026 19:27
With this new flag, it automatically runs `update-wpt` at the
end, without the need to specify a `--log-raw` argument.

Fixes servo#45476

Testing: Local runs with the various combinations of flags

Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
@TimvdLippe
TimvdLippe force-pushed the add-update-expectations-flag branch from 881535d to cc52be0 Compare June 9, 2026 19:32
def test_wpt(self, servo_binary: str, multiprocess: bool, update_expectations: bool, **kwargs: Any) -> int:
if update_expectations:
if kwargs["log_raw"]:
print("Do not specify --log-raw when updating tests directly")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While it seems a bit weird to use both, would there really be a problem?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it would break second time parsing. Initially I allowed for both, but then paraing it for update would fail since it expects a string.

Comment thread python/servo/testing_commands.py Outdated
@CommandArgument("--multiprocess", "-M", default=False, action="store_true", help="Run in multiprocess mode")
@CommandArgument(
"--update-expectations",
"-U",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any chance of lowercase -u? Partly because uppercase short flags conventionally mean NOT. Partly because almost all the git commands have a really useful flag under -u and it would match my general pattern of "that useful option I use all the time is -u" (not the most rational justification I admit)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, done

@Taym95 Taym95 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ✅ , really helpful thanks!

@servo-highfive servo-highfive removed the S-awaiting-review There is new code that needs to be reviewed. label Jun 9, 2026
Signed-off-by: Tim van der Lippe <TimvdLippe@users.noreply.github.com>
@servo-highfive servo-highfive added the S-awaiting-review There is new code that needs to be reviewed. label Jun 10, 2026
@TimvdLippe
TimvdLippe enabled auto-merge June 10, 2026 04:58
@TimvdLippe
TimvdLippe added this pull request to the merge queue Jun 10, 2026
@servo-highfive servo-highfive added the S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. label Jun 10, 2026
Merged via the queue into servo:main with commit 433024d Jun 10, 2026
33 checks passed
@TimvdLippe
TimvdLippe deleted the add-update-expectations-flag branch June 10, 2026 06:45
@servo-highfive servo-highfive removed the S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. label Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-awaiting-review There is new code that needs to be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make updating web platform test baselines more intuitive

7 participants