Cleanup DDC files copied to the released SDK for Dart 3.0

@mit-mit

I want to highlight and coordinate some technically observable breaking changes to the files packaged with the SDK that I want to make for Dart 3.0. My intention is that no user (using a supported DDC build) will notice any changes. The intent is to only change the files copied to the packaged SDK that we ship externally, not the files that we build on our test bots or in google3.

With the removal of weak null safety we no longer need to ship the precompiled weak .dill files for the SDK libraries. In the process I would also like to cleanup the names of the sound null safe .dill files shipped for DDC. This will make the names consistent across dart2js, dart2wasm, and ddc.

@mit-mit

I'm not sure exactly when we consider it safe to break compiling with --no-sound-null-safety.

@domesticmouse

Do we still publish a version of "DDC DartPad" that supports weak null safety? Where are the scripts that compile the sound null safe modules for "DDC DartPad" and what .dill files do they use for the SDK?

@annagrin

I don't believe the Flutter web builds rely on these .dill files but could you help me confirm? Also what about the debugger?

@jakemac53

build_web_compilers will need to update the .dill file paths with this change lands.

DDC SDK .dill Files

Description	Current path/file	Proposed path/file
Sound null safety outline	dart-sdk/lib/_internal/ddc_outline_sound.dill	dart-sdk/lib/_internal/ddc_outline.dill
Sound null safety full	dart-sdk/lib/_internal/ddc_platform_sound.dill	dart-sdk/lib/_internal/ddc_platform.dill
Weak null safety outline	dart-sdk/lib/_internal/ddc_sdk.dill	DELETED
Weak null safety full	dart-sdk/lib/_internal/ddc_platform.dill	DELETED

I also want to remove the precompiled SDK JavaScript files. We never shipped the sound null safe versions and instead updated all the integrations of DDC to compile the SDK as needed. We only left the files in the SDK for backwards compatibility which we are breaking with Dart 3.0 anyway. AFAIK we don't use any of the files I want to delete in any supported use case.

@domesticmouse @jakemac53 @annagrin

The one file I think we need to keep shipping is the module loading library require.js. AFAIK all of our external uses (webdev, flutter web, DartPad) all rely on a copy of that file. I'm proposing to remove a pointless directory from the path but I'm open to your thoughts on this one.

DDC JavaScript Files

Description	Current path/file	Proposed path/file
Weak null safety compiled SDK (amd format)	dart-sdk/lib/dev_compiler/kernel/amd/dart_sdk.js	DELETED
SDK source map	dart-sdk/lib/dev_compiler/kernel/amd/dart_sdk.js.map	DELETED
Require.js module loader	dart-sdk/lib/dev_compiler/kernel/amd/require.js	dart-sdk/lib/dev_compiler/amd/require.js
Weak null safety compiled SDK (common format)	dart-sdk/lib/dev_compiler/kernel/common/dart_sdk.js	DELETED
SDK source map	dart-sdk/lib/dev_compiler/kernel/common/dart_sdk.js.map	DELETED
Common module loader	dart-sdk/lib/dev_compiler/kernel/common/run.js	DELETED
Weak null safety compiled SDK (es6 format)	dart-sdk/lib/dev_compiler/kernel/es6/dart_sdk.js	DELETED
SDK source map	dart-sdk/lib/dev_compiler/kernel/es6/dart_sdk.js.map	DELETED

@sigmundch

cc @sigmundch

There is no null-unsafe version of DartPad at this point. We just run null safe versions of stable, old (stable - 1), beta, and master. (There is a hidden one that has the name dev, which is a clone of stable with different package config)

For the code that Dart Services uses to build DDC artifacts with the dill files, please see https://github.com/dart-lang/dart-services/blob/master/tool/grind.dart#L240

@johnpryan

On the require.js front, please chat with @johnpryan as that is a concern for the DartPad front end, which is something John knows a lot better than I do.

I think this very reasonable, and if the only changes are behind the legacy flag, then I do not consider it formally breaking.

My only comment would be it might be best to avoid re-using any pre-existing file paths. The errors are likely to be more confusing for the users that are affected, than if we used a totally new path so the old file doesn't exist at all.

@christopherfujino

/cc @christopherfujino may be able to help answer some of your questions regarding flutter web

@mit-mit

@mit-mit

I think this very reasonable, and if the only changes are behind the legacy flag, then I do not consider it formally breaking.

I just want make sure I know what you mean by "the legacy flag". Do you mean --no-sound-null-safety? I'm essentially removing the files that will support that flag in DDC from the released SDK.

I should have said "related to the legacy flag". I have no problem with removing that flag, and the associated support from DDC.

Update: This work is in progress but we have found tests for package:dwds that expect to run with weak null safety. They need to continue running for now to prevent any regressions. The deletion of the unsound .dill files from the packaged SDK will not happen until after we move those tests to run in a different environment. For now they will be renamed in preparation for the eventual deletion:

Description	Current path/file	Proposed path/file
Sound null safety outline	dart-sdk/lib/_internal/ddc_outline_sound.dill	dart-sdk/lib/_internal/ddc_outline.dill
Sound null safety full	dart-sdk/lib/_internal/ddc_platform_sound.dill	dart-sdk/lib/_internal/ddc_platform.dill
Weak null safety outline	dart-sdk/lib/_internal/ddc_sdk.dill	dart-sdk/lib/_internal/ddc_outline_unsound.dill
Weak null safety full	dart-sdk/lib/_internal/ddc_platform.dill	dart-sdk/lib/_internal/ddc_platform_unsound.dill

For now they will be renamed in preparation for the eventual deletion

When is this scheduled to happen? This change will break dart-lang/dart-services I believe. Given Dart Services supports everything from master to stable - 1 this is going to be an interesting thing to manage through the waterfall.

I'd like to land this change as soon as possible, but I don't think this will actually break dart-lang/dart-services. Looking at this line it looks like the .dill file being used comes from the Flutter SDK which is built separately.
https://github.com/dart-lang/dart-services/blob/53ace61b517e689fe2bd0d17e2afc24fc58f29d4/lib/src/compiler.dart#L191

I don't see any .dill file being passed that comes from the Dart SDK which is what I am changing.

@johnpryan

@johnpryan Do you know where require.js is being pulled from to load DDC compiled code? That is another file in the Dart SDK that I'm planning on moving in another change.

Discussed via chat, DartPad has its own copy of require.js checked in so my change should not be a problem.
https://github.com/dart-lang/dart-pad/blob/master/web/scripts/require.js

@nshahan

@nshahan Summarizing our discussion offline: we don't want to break webdev tests that are currently running in weak mode, so the incremental plan is in dart-lang/webdev#1878:

dwds tests fixtures: Use special branch of build_web_compilers that supports weak null safety
webdev, dwds test infrastructure: Do not pass --(no)-sound-null-safety flags to build_web_compilers
dwds test infrastructure (using build_web_compilers): Generate js and dill files from sources for weak version of SDK
dwds test infrastructure (using build_web_compilers): Handle old and new names for the SDK assets
SDK: Then we can remove the weak sdk files from the sdk releases.

[ddc] Cleanup JavaScript files in packaged SDK

Drop support for unsound null safety in Dart 3 (#3424)

@nshahan

@nshahan I think these changes may have broken our basic webdev workflow? Try these with a recent master (I'm using dart from flutter master):

dart create -t web webapp
cd webapp
dart pub global activate webdev
dart pub global run webdev serve

=>

[SEVERE] build_web_compilers:sdk_js on asset:build_web_compilers/$package$: Error compiling dartdevc module:build_web_compilers|lib/src/dev_compiler/dart_sdk.sound.js

Unhandled exception:
PathNotFoundException: Cannot open file, path = '/Users/mit/dev/flutter/bin/cache/dart-sdk/lib/_internal/ddc_platform_sound.dill' (OS Error: No such file or directory, errno = 2)

mit-macbookpro7:webapp1 mit$ dart pub global run webdev --version
2.7.12

@mit-mit

@mit-mit you must be on the latest build_runner/build_web_compilers/build_modules for things to work.

@mit-mit

@mit-mit You are right, the dart create -t web webapp command is currently broken. I have an update to the new project template prepared but I've been waiting for all versions of the necessary packages webdev, dwds, build_web_compilers to be published before I updated the template. Those are done or very close but I believe some are blocked on a release of the Dart SDK at or after 3.0.0-134.0.dev which is the first version to include my changes.

@mit-mit

@mit-mit I opened a new issue to track the progress: #51106

Navigation Menu

Repository navigation

Cleanup DDC files copied to the released SDK for Dart 3.0 #50700

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Cleanup DDC files copied to the released SDK for Dart 3.0 #50700

Description

Activity

nshahan commented on Dec 13, 2022

domesticmouse commented on Dec 13, 2022

domesticmouse commented on Dec 13, 2022

mit-mit commented on Dec 13, 2022

jakemac53 commented on Dec 13, 2022

sigmundch commented on Dec 13, 2022

nshahan commented on Dec 16, 2022

mit-mit commented on Dec 19, 2022

nshahan commented on Jan 4, 2023

domesticmouse commented on Jan 4, 2023

nshahan commented on Jan 5, 2023

nshahan commented on Jan 5, 2023

nshahan commented on Jan 6, 2023

annagrin commented on Jan 10, 2023

mit-mit commented on Jan 22, 2023

jakemac53 commented on Jan 23, 2023

nshahan commented on Jan 23, 2023

nshahan commented on Jan 23, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions