Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(test runner): improve sharding algorithm to better spread similar tests among shards #30962

Closed
wants to merge 61 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
54bbba8
sharding algorithm to better spread similar tests among shards
muhqu May 22, 2024
ee64b15
improve shard algorithm by sorting test groups by number of tests
muhqu May 22, 2024
6543baf
adjust shard.spec.ts to new algorithm
muhqu May 22, 2024
59c625e
fix reporter-json.spec.ts due to sharding
muhqu May 22, 2024
d3b4fad
fix reporter-junit.spec.ts due to sharding
muhqu May 22, 2024
4bcfc78
empty commit to trigger ci
muhqu May 22, 2024
061f559
add test durations to last run info
muhqu May 27, 2024
14ca30a
allow .last-run.json to be generated via merge-reports
muhqu May 27, 2024
ce09f88
Add shardingMode configuration
muhqu May 27, 2024
d42c499
revert json/junit test changes
muhqu May 27, 2024
b5b8174
empty commit to trigger ci
muhqu May 27, 2024
389e571
fix(merge-reports) only change test ids when needed
muhqu May 29, 2024
6884fd7
remove special handling of merged test ids
muhqu May 29, 2024
ef2d35f
Merge branch 'main' into sharding-algorithm
muhqu May 29, 2024
d33e8da
fix(merge-reports) only change test ids when needed
muhqu May 29, 2024
6b051cb
fix(runner) don't write last run info when listing tests
muhqu May 29, 2024
32169b7
optimize: use single global set
muhqu May 30, 2024
fd90424
Feedback use outputLines
muhqu May 30, 2024
c7b58d7
Revert "fix(merge-reports) only change test ids when needed"
muhqu May 30, 2024
6cf1217
Merge branch 'fix-merge-reports-test-ids' into sharding-algorithm
muhqu May 30, 2024
7dd7115
empty commit to trigger ci
muhqu May 30, 2024
cabfa74
lint fix
muhqu May 30, 2024
899c068
Merge branch 'fix-merge-reports-test-ids' into sharding-algorithm
muhqu May 30, 2024
8aee7df
Merge branch 'main' into sharding-algorithm
muhqu May 30, 2024
843d629
Merge branch 'main' into sharding-algorithm
muhqu Jun 24, 2024
2536da7
fix test.d.ts
muhqu Jun 24, 2024
087a57f
add --last-run-file CLI parameter
muhqu Jun 25, 2024
7aa2d95
Adjust since to v1.46
muhqu Jul 2, 2024
29f67f3
Merge branch 'main' into sharding-algorithm
muhqu Jul 2, 2024
7b5f7c6
Add documentation on sharding modes
muhqu Jul 2, 2024
65a2b55
linting
muhqu Jul 2, 2024
df6d05d
Add into text to more complex round-robin scenario
muhqu Jul 2, 2024
3fed470
no ts highlight for sharding illustration code blocks
muhqu Jul 2, 2024
050e837
use yaml highlight for sharding illustration code blocks as NO highli…
muhqu Jul 2, 2024
f483ac6
revert / reorder import changes
muhqu Jul 2, 2024
7ea1506
Merge branch 'main' into sharding-algorithm
muhqu Sep 9, 2024
8d485f5
feedback
muhqu Sep 9, 2024
e7273de
feedback - remove redundant tests
muhqu Sep 9, 2024
1eb11ff
always use lastrun reporter and keep em internal
muhqu Sep 9, 2024
0503672
validate --sharding-mode CLI parameter
muhqu Sep 9, 2024
4b3d754
export type ShardingMode
muhqu Sep 9, 2024
89458db
fix .last-run.json location
muhqu Sep 9, 2024
7dab70c
fix .last-run.json location in reporter-lastrun spec
muhqu Sep 9, 2024
b0e5745
use lastRunFile from config when defined
muhqu Sep 9, 2024
5e3fa62
Merge branch 'main' into sharding-algorithm
muhqu Sep 9, 2024
4f758a1
Merge branch 'main' into sharding-algorithm
muhqu Sep 10, 2024
eb25f3c
validate config.shardingMode
muhqu Sep 11, 2024
16b39b8
fix --last-run-file parameter
muhqu Sep 11, 2024
ea72517
Merge branch 'main' into sharding-algorithm
muhqu Sep 11, 2024
b78b88f
adapt LastRunReporter
muhqu Sep 11, 2024
f62651c
use separate LastRunReporter instances
muhqu Sep 11, 2024
07914f7
allow lastRun.ts to be imported by merge.ts
muhqu Sep 11, 2024
e7d07bc
Merge branch 'main' into sharding-algorithm
muhqu Sep 12, 2024
412e05b
Add test.skip for Node.js bug in 'should not transform external' babe…
muhqu Sep 12, 2024
171c5e1
Merge branch 'main' into sharding-algorithm
muhqu Sep 13, 2024
ad6af69
Revert "Add test.skip for Node.js bug in 'should not transform extern…
muhqu Sep 13, 2024
b3b568b
Merge branch 'main' into sharding-algorithm
muhqu Sep 16, 2024
36433d0
Merge branch 'main' into sharding-algorithm
muhqu Sep 17, 2024
ae34689
Merge branch 'main' into sharding-algorithm
muhqu Sep 17, 2024
4dd2842
empty commit to trigger CI
muhqu Sep 17, 2024
ab3ab83
Merge branch 'main' into sharding-algorithm
muhqu Sep 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
remove special handling of merged test ids
  • Loading branch information
muhqu committed May 29, 2024
commit 6884fd715f344d6949074284a85c2f7445c47fa5
11 changes: 2 additions & 9 deletions packages/playwright/src/reporters/lastrun.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,6 @@ type LastRunOptions = {
_mode: 'list' | 'test' | 'merge',
};

// The blob reporter adds a suffix to test ids to avoid collisions. But we need
// to remove that suffix to match the test ids from the last run.
function unsaltTestId(testId: string): string {
return testId.split('-').map(s => s.substring(0, 20)).join('-');
}

class LastRunReporter extends BaseReporter {

Expand All @@ -40,12 +35,10 @@ class LastRunReporter extends BaseReporter {
};

private resolvedOutputFile: string | undefined;
private testId: (testId: string) => string;

constructor(options: LastRunOptions) {
super();
this.resolvedOutputFile = resolveOutputFile('LASTRUN', { fileName: '.last-run.json', ...options })?.outputFile;
this.testId = options._mode === 'merge' ? unsaltTestId : (testId: string) => testId;
}

override printsToStdio() {
Expand All @@ -54,9 +47,9 @@ class LastRunReporter extends BaseReporter {

override onTestEnd(test: TestCase, result: TestResult): void {
super.onTestEnd(test, result);
this.lastRun.testDurations![this.testId(test.id)] = result.duration;
this.lastRun.testDurations![test.id] = result.duration;
if (result.status === 'failed')
this.lastRun.failedTests.push(this.testId(test.id));
this.lastRun.failedTests.push(test.id);
}

override async onEnd(result: FullResult) {
Expand Down