Skip to content

Investigate using native streams #20

@MattiasBuelens

Description

@MattiasBuelens

Inspired by this tweet from @surma:

@MattiasBuelens Is it possible to offer up the polyfills for Readable, Writable and Transform individually? Most browsers have Readable, so ideally I’d only load Writable and Transform.

I've thought about this previously. Back then, I decided that it was not feasible because readable byte streams are not supported by any browser. A full polyfill would always need to provide its own ReadableStream implementation that supports byte streams. By extension, it would also need to provide its own implementations for WritableStream (that works with its ReadableStream.pipeTo()) and TransformStream (that uses its readable and writable streams).

Looking at this again, I think we can do better. If you don't need readable byte streams, then the native ReadableStream should be good enough as a starting point for the polyfill. From there, the polyfill could add any missing methods (pipeTo, pipeThrough, getIterator,...) and implement them using the native reader from getReader().

This approach can never be fully spec-compliant though, since the spec explicitly forbids these methods to use the public API. For example, pipeTo() must use AcquireReadableStreamDefaultReader() instead of ReadableStream.getReader(), so it cannot be affected by user-land JavaScript code making modifications to ReadableStream.prototype. I don't think that has to be a problem though: we are already a user-land polyfill written in JavaScript that modifies those prototypes, it would be silly for the polyfill to try and guard itself against other JavaScript code making similar modifications.

Steps in the spec that require inspecting the internal state of the stream or call into internal methods will need to be replaced by something that emulates the behavior using solely the public API.

  • Often, this will be easy: e.g. ReadableStreamDefaultControllerEnqueue() becomes controller.enqueue().

  • Sometimes, we have to be a bit more lenient. ReadableStreamPipeTo()'s error propagation says:

    if source.[[state]] is or becomes "errored"

    We can check if it becomes errored by waiting for the source.closed promise to become rejected. However, we can't synchronously check if it is already errored.

  • In rare cases, this may turn out to be impossible. TransformStreamDefaultSinkWriteAlgorithm specifies:

    If state is "erroring", throw writable.[[storedError]].

    Usually, the writable stream starts erroring because the writable controller has errored, which the transform stream's implementation controls. However, it could also be triggered by WritableStream.abort(), which is out of the control of the transform stream implementation. In this case, the controller is only made aware of it after the writable stream finishes erroring (state becomes "errored") through its abort() algorithm, which is already too late.

Of course, we can't just flat-out remove byte stream support from the polyfill, just for the sake of using native streams more. The default should still be a full polyfill, but we might want to give users the option to select which features they want polyfilled (as @surma suggested in another tweet).

Anyway, I still want to give this a try. It might fail catastrophically, but then at least I'll have a better answer on why we use so little from the native streams implementation. 😅

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions