Repository navigation
Make decodeAudioData more precise #1563
Description
It would be nice if https://webaudio.github.io/web-audio-api/#dom-baseaudiocontext-decodeaudiodata invoked an algorithm in MIME Sniffing rather than just referencing the whole standard. It would also be good to say which return values are accepted/rejected.
Are there tests for this method?
Metadata
Metadata
Assignees
Labels
No labels
Activity
padenot commentedon Apr 12, 2018
We're in the process of upstreaming tests from the various UAs into web-platform-tests, I don't now where we're at right now, but there certainly will be tests.
rtoy commentedon Apr 12, 2018
Chrome has basically skipped upstreaming the decoder tests. I think the only thing that might really be cross-platform would be decoding 16-bit PCM WAV files. We could probably move the couple of tests that we have for this.
rtoy commentedon May 18, 2018
Do you mean we should probably reference https://mimesniff.spec.whatwg.org/#audio-or-video-type-pattern-matching-algorithm ?
annevk commentedon May 19, 2018
Yeah, I guess. Whatever matches implementations. (That's also why I'm interested in tests, I wonder if this is specified properly.)
rtoy commentedon May 21, 2018
Chrome lets ffmpeg do all the sniffing. I don't know how that relates to the mimesniff algorithm, but it must be somewhat similar.
Chrome has no explicit tests on sniffing; we only check that the decoded file matches what we thought it should be.
agnivade commentedon Aug 15, 2018
@rtoy - Does that mean a web server cannot reliably sniff audio content without calling into ffmpeg ?
Because without any algorithm it is not possible to reliably do mime sniffing.
Context:
I would like to implement mime sniffing for mp3s in Go's http server. But that is blocked on whatwg/mimesniff#70. Which is blocked on this issue. Would appreciate any leads on this.
rtoy commentedon Aug 15, 2018
@agnivade: Don't have anything really to add. Mime sniffing is just a heuristic anyway, right?
agnivade commentedon Aug 16, 2018
Sure, that maybe. But there are major differences between the mime sniffing algorithm in https://mimesniff.spec.whatwg.org/#signature-for-mp3-without-id3 and the actual algorithm used by mozilla
https://dxr.mozilla.org/mozilla-central/source/toolkit/components/mediasniffer/mp3sniff.c.
We should fix that. I tried to implement what was in mimesniff and it failed for every possible mp3 out there. I don't believe that should be the case.
annevk commentedon Aug 16, 2018
It's not exactly a heuristic. It determines whether or not a byte stream can be played back and needs to be interoperable across implementations.
Make decodeAudioData more precise.
Make decodeAudioData more precise. (#1781)
Make decodeAudioData more precise. (WebAudio#1781)
guest271314 commentedon Feb 2, 2020
GPHemsley commentedon Sep 9, 2021
I'm not sure that #1781 sufficiently addressed this.
For starters, it's pointing to the wrong mimesniff algorithm. The rules for sniffing audio and video specifically are what is intended to be used for sniffing in an audio or video context (such as
<audio>or<video>). This is distinct from the algorithm identifying audio or video in a browsing context, which is meant to be much more restrictive and geared towards balancing security and legacy content.Secondly, the spec for decodeAudioData() still contains:
Which I believe is what @annevk was referring to as vague.
guest271314 commentedon Sep 10, 2021
padenot commentedon Sep 15, 2021
I agree, it's not enough. I've filed #2378 as a follow-up and will address the issues you describe there. Thanks for reporting!