Improve defaults generation with logging, stats, and token validation #128244

guidezpl · 2023-06-05T14:28:30Z

Description

This improves defaults generation with logging, stats, and token validation.

This PR includes these changes:

introduce TokenLogger, with a verbose mode
- prints versions and tokens usage to the console
- outputs generated/used_tokens.csv, a list of all used tokens, for use by Google
find token files in data automatically
hide tokens Map
- tokens can be obtained using existing resolvers (e.g. color, shape), or directly through getToken.
- tokens can be checked for existence with tokenAvailable
remove version from template, since the tokens are aggregated and multiple versions are possible (as is the case currently), it does not make sense to attribute a single version
improve documentation

Related Issues

Fixes Provide list of used Material Design tokens #122602

Tests

Added tests for TokenLogger
Regenerated tokens, no-op except version removal

Future work

A future PR should replace or remove the following invalid tokens usages

Pre-launch Checklist

I read the Contributor Guide and followed the process outlined there for submitting PRs.
I read the Tree Hygiene wiki page, which explains my responsibilities.
I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
I signed the CLA.
I listed at least one issue that this PR fixes in the description above.
I updated/added relevant documentation (doc comments with ///).
I added new tests to check the change I am making, or this PR is test-exempt.
All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

QuncCccccc

This looks great! Just left some questions below:)

dev/tools/gen_defaults/generated/used_tokens.csv

QuncCccccc · 2023-06-08T06:16:53Z

dev/tools/gen_defaults/lib/color_scheme_template.dart


 class ColorSchemeTemplate extends TokenTemplate {
-  ColorSchemeTemplate(super.blockName, super.fileName, super.tokens);
+  ColorSchemeTemplate(this._colorTokensLight, this._colorTokensDark, super.blockName, super.fileName, super.tokens);


Just for my education, why do we use private properties in constructor?

It's no different than the following

ColorSchemeTemplate(
Map<String, dynamic> colorTokensLight,
...
) : _colorTokensLight = colorTokensLight;

QuncCccccc · 2023-06-08T06:27:17Z

dev/tools/gen_defaults/lib/template.dart

+  bool tokenAvailable(String tokenName) => _tokens.containsKey(tokenName);
+
+  /// Resolve a token while logging its usage.
+  dynamic getToken(dynamic tokenName) {


Should these two dynamic just be String?

Yes for tokenName, no for the return type, which can be other types (e.g. double)

QuncCccccc

LGTM:)!!

auto-submit · 2023-06-09T09:13:09Z

auto label is removed for flutter/flutter, pr: 128244, due to - The status or check suite Google testing has failed. Please fix the issues identified (or deflake) before re-applying this label.

…lidation (flutter/flutter#128244)

flutter/flutter@6e254a3...da127f1 2023-06-09 hans.muller@gmail.com Updated material button theme tests for Material3 (flutter/flutter#128543) 2023-06-09 engine-flutter-autoroll@skia.org Roll Flutter Engine from cb93477008d6 to 93afba901b3b (2 revisions) (flutter/flutter#128573) 2023-06-09 6655696+guidezpl@users.noreply.github.com Improve defaults generation with logging, stats, and token validation (flutter/flutter#128244) 2023-06-09 whesse@google.com [testing] Make the FLUTTER_STORAGE_BASE_URL warning non-fatal (flutter/flutter#128335) 2023-06-09 danny@tuppeny.com [flutter_tools] [DAP] Don't try to restart/reload if app hasn't started yet (flutter/flutter#128267) 2023-06-09 engine-flutter-autoroll@skia.org Roll Flutter Engine from 8f9e608d39ab to cb93477008d6 (3 revisions) (flutter/flutter#128568) 2023-06-09 tessertaha@gmail.com Replace `MaterialButton` from test classes (flutter/flutter#128466) 2023-06-09 tessertaha@gmail.com Fix `showBottomSheet` doesn't remove scrim when draggable sheet is dismissed (flutter/flutter#128455) 2023-06-09 engine-flutter-autoroll@skia.org Manual roll Flutter Engine from a5f7d5d75ff2 to 8f9e608d39ab (31 revisions) (flutter/flutter#128554) 2023-06-09 ychris@google.com Revert "test owners: cyanglaz -> vashworth" (flutter/flutter#128462) 2023-06-09 43054281+camsim99@users.noreply.github.com [Android] Bump integration tests using `compileSdkVersion` 31 to 33 (flutter/flutter#128072) 2023-06-09 dkwingsmt@users.noreply.github.com Remove single view assumption from MouseTracker, and unify its hit testing code flow (flutter/flutter#127060) 2023-06-09 christopherfujino@gmail.com [flutter_tools] Precache after channel switch (flutter/flutter#118129) 2023-06-08 leigha.jarett@gmail.com Adding migration guide for Material 3 colors (flutter/flutter#128429) 2023-06-08 gspencergoog@users.noreply.github.com Add `AppLifecycleListener`, with support for application exit handling (flutter/flutter#123274) 2023-06-08 thkim1011@users.noreply.github.com Sliver Main Axis Group (flutter/flutter#126596) 2023-06-08 31859944+LongCatIsLooong@users.noreply.github.com Reduce `_DoubleClampVisitor` false positives (flutter/flutter#128539) 2023-06-08 leigha.jarett@gmail.com Advise developers to use OverflowBar instead of ButtonBar (flutter/flutter#128437) 2023-06-08 jacksongardner@google.com Reland "Migrate benchmarks to package:web" (flutter/flutter#128266) 2023-06-08 53684884+mhbdev@users.noreply.github.com Navigator.pop before PopupMenuItem onTap call (flutter/flutter#127446) 2023-06-08 leroux_bruno@yahoo.fr Fix navigation rail with long labels misplaced highlights (flutter/flutter#128324) 2023-06-08 tessertaha@gmail.com Update `chip.dart` to use set of `MaterialState` (flutter/flutter#128507) 2023-06-08 jcollins@google.com Update flutter to dartdoc 6.3.0 and hide Icons implementation from doc pages (flutter/flutter#128442) 2023-06-08 31859944+LongCatIsLooong@users.noreply.github.com Disable blinking cursor when `EditableText.showCursor` is false (flutter/flutter#127562) 2023-06-08 41930132+hellohuanlin@users.noreply.github.com [floating_cursor_selection]add more comments on the tricky part (flutter/flutter#127227) 2023-06-08 goderbauer@google.com Move RenderObjectElement.updateChildren to Element (flutter/flutter#128458) 2023-06-08 goderbauer@google.com Fix PointerEventConverter doc (flutter/flutter#128452) 2023-06-08 engine-flutter-autoroll@skia.org Roll Packages from a84b2c2 to e13b8c4 (9 revisions) (flutter/flutter#128508) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC rmistry@google.com,stuartmorgan@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

…lidation (flutter/flutter#128244)

guidezpl added 8 commits June 5, 2023 15:03

update

84b0450

create TokenLogger and tests

52da031

Update README.md

4213043

move args

7da28c2

use getToken

24cb66b

add verbose flag and remove manual file listing

eabcd96

add logging

b5027ec

refactor

ec86e49

github-actions bot removed the f: material design flutter/packages/flutter/material repository. label Jun 5, 2023

flutter-dashboard bot added the f: material design flutter/packages/flutter/material repository. label Jun 5, 2023

guidezpl added 3 commits June 6, 2023 01:26

x

c429fcf

fix analysis

a152c3d

update TokenLogger docs

c1dbdaa

guidezpl requested a review from QuncCccccc June 6, 2023 00:14

guidezpl marked this pull request as ready for review June 6, 2023 08:11

QuncCccccc reviewed Jun 8, 2023

View reviewed changes

address feedback

ad7d830

QuncCccccc approved these changes Jun 8, 2023

View reviewed changes

guidezpl and others added 4 commits June 8, 2023 13:08

Merge branch 'master' into tokens-cleanup

01ce882

Merge branch 'master' into tokens-cleanup

4933049

Merge remote-tracking branch 'upstream/master' into tokens-cleanup

3ec27e3

Update pubspec.yaml

f5f8715

guidezpl added the autosubmit Merge PR when tree becomes green via auto submit App label Jun 9, 2023

auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jun 9, 2023

guidezpl added the autosubmit Merge PR when tree becomes green via auto submit App label Jun 9, 2023

auto-submit bot merged commit 66cda59 into flutter:master Jun 9, 2023

guidezpl deleted the tokens-cleanup branch June 9, 2023 13:39

guidezpl mentioned this pull request Jun 9, 2023

Defaults generator uses invalid tokens #128578

Closed

engine-flutter-autoroll mentioned this pull request Jun 9, 2023

Roll Flutter from 6e254a3f9fb7 to da127f15ad54 (28 revisions) flutter/packages#4170

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jun 9, 2023

66cda59 Improve defaults generation with logging, stats, and token va…

b638a30

…lidation (flutter/flutter#128244)

HansMuller mentioned this pull request Jun 9, 2023

Flutter Framework Team: Notable Commits #121415

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 16, 2023

66cda59 Improve defaults generation with logging, stats, and token va…

69436fe

…lidation (flutter/flutter#128244)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023

66cda59 Improve defaults generation with logging, stats, and token va…

c04bcf8

…lidation (flutter/flutter#128244)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023

66cda59 Improve defaults generation with logging, stats, and token va…

9cccb85

…lidation (flutter/flutter#128244)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023

66cda59 Improve defaults generation with logging, stats, and token va…

56022bc

…lidation (flutter/flutter#128244)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve defaults generation with logging, stats, and token validation #128244

Improve defaults generation with logging, stats, and token validation #128244

Uh oh!

guidezpl commented Jun 5, 2023 •

edited

Loading

Uh oh!

QuncCccccc left a comment

Uh oh!

Uh oh!

QuncCccccc Jun 8, 2023

Uh oh!

guidezpl Jun 8, 2023

Uh oh!

QuncCccccc Jun 8, 2023

Uh oh!

guidezpl Jun 8, 2023

Uh oh!

QuncCccccc left a comment

Uh oh!

auto-submit bot commented Jun 9, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Improve defaults generation with logging, stats, and token validation #128244

Improve defaults generation with logging, stats, and token validation #128244

Uh oh!

Conversation

guidezpl commented Jun 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Tests

Future work

Pre-launch Checklist

Uh oh!

QuncCccccc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

QuncCccccc Jun 8, 2023

Choose a reason for hiding this comment

Uh oh!

guidezpl Jun 8, 2023

Choose a reason for hiding this comment

Uh oh!

QuncCccccc Jun 8, 2023

Choose a reason for hiding this comment

Uh oh!

guidezpl Jun 8, 2023

Choose a reason for hiding this comment

Uh oh!

QuncCccccc left a comment

Choose a reason for hiding this comment

Uh oh!

auto-submit bot commented Jun 9, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

guidezpl commented Jun 5, 2023 •

edited

Loading