Skip to content

Latest commit

 

History

History
79 lines (67 loc) · 15.9 KB

INSPIRATION.md

File metadata and controls

79 lines (67 loc) · 15.9 KB

Inspiration

Why

I wanted something smaller and faster, written natively in Rust, for the PITO MD family of apps and my games. One library each app can carry, with no external ffmpeg to download, bundle or keep in step, and the GPU used first.

It is my own Rust, built for speed, that I can change and release on my own schedule, with fast iterations, instead of waiting on ffmpeg's and other projects' releases. Everything taken in is permissively licensed, so pito-codecs stays MIT.

Origins

The codecs here are written in Rust. Parts are adapted from permissively licensed projects, each named below with the tag or commit it was taken from; their copyright and licence texts are kept in NOTICE.

Origin Licence taken Tag and commit What came from it
FilmCraft MIT (FilmCraft is MIT OR Apache-2.0) v0.6.0, 2ff3f5e266838700c292a35aaf762e3d4dcef980 See the module lines in NOTICE.
libva headers MIT (Intel) libva 2.23 (VA-API 1.23) The VA-API declarations in src/gpu/vaapi/ffi.rs.
NVIDIA Video Codec SDK header nvEncodeAPI.h MIT (NVIDIA) API 12.1 The NVENC declarations in src/gpu/nvenc/ffi.rs.
RFC 6386 reference decoder (Section 20) BSD-3-Clause (Google) RFC 6386 (November 2011) VP8 loop filter, intra prediction, token and transform details in src/webp/vp8/.
libwebp BSD-3-Clause (Google) v1.6.0, 4fa21912338357f89e4fd51cf2368325b59e9bd9 YUV and RGB conversion, the fancy upsampler, the SSE2 loop filter layout, forward transforms and the quality scale in src/webp/vp8/.
libwebm BSD-3-Clause (Google) libwebm-1.0.0.32, f2a982d748b80586ae53b89a2e6ebbc305848b8c The BlockGroup and BlockAdditions layout of a VP9 alpha frame (mkvmuxer's WriteBlock element order) and the AlphaMode and MaxBlockAdditionID track elements in src/mkv/; the alpha encoder and decoder pairing in src/vp9/alpha.rs is new.
libwebp BSD-3-Clause (Google) v1.6.0, 4fa21912338357f89e4fd51cf2368325b59e9bd9 YUV and RGB conversion, the fancy upsampler, the SSE2 loop filter layout, forward transforms and the quality scale in src/webp/vp8/; the lossy animation sub-frame rules of anim_encode.c in src/webp/anim.rs.
oxideav-ac3 MIT (Karpelès Lab Inc.) 0.0.11, 99228831d2b29a35fe2b2035118a18f8cb0cae59 The AC-3 and E-AC-3 decoder in src/ac3/: frame and BSI parsing, exponents, bit allocation, mantissas, coupling, rematrixing, spectral extension, AHT and the IMDCT structure.
Mesa VA-API frontend and video compositor MIT mesa-26.2.2 Read for behaviour, no code copied: how its video processor composes several pipeline buffers in one picture, blends premultiplied RGBA onto NV12 and P010, and sites chroma, which src/gpu/vaapi/va.rs follows to draw export overlays on the GPU.
minimp3 CC0 1.0 (public domain) ea99364f61c14656440e8d77e9c233ccf3124633 The MPEG-1, 2 and 2.5 audio decoder (layers I, II and III) in src/mp3/, with its tables and SSE and NEON layout, and minimp3_ex.h's VBR-tag reading, ID3 and APE skipping and gapless trim in src/mp3/file.rs; the container packet front, the channel matching and the granule-0 scfsi mask are new.
Apple ALAC Apache-2.0 (Apple Inc.) c38887c5c5e64a4b31108733bd79ca9b2496d987 The ALAC decoder in src/alac/: element parsing, adaptive Golomb decoding, the adaptive predictor with its order-4 and order-8 paths, stereo unmixing and the shifted low bytes, with their constants; the bounds checks, errors and the cookie lookup in MP4, MOV and Matroska are new. One change from the reference: the running mean clamps when the decoded value plus the zero-run carry passes 0xffff, as FFmpeg's ALAC encoder and decoder do, so streams from FFmpeg-based recorders decode where the reference refuses them.
zune-png MIT OR Apache-2.0 OR Zlib (Caleb Etemesi) 0.4.10, aaf40839ed2596fa83091ccfb49629eb5d8efd91 The scanline de-filters with their SSE2 and SSE4.1 kernels, the chunk checks, bit unpacking, palette and tRNS expansion and Adam7 geometry in src/png/; the RGBA row loop is new.
zune-inflate MIT OR Apache-2.0 OR Zlib (Caleb Etemesi), a port of libdeflate (MIT) 0.2.54, 69502ce83fdfecdd0beefd677e2abb3781b29d98 The whole-buffer zlib and deflate decoder in src/png/inflate.rs: decode tables, fast and slow loops, match copies.
fdeflate MIT OR Apache-2.0 (The image-rs Developers) 0.3.7, c365c7e6ffa81feb2e1fb762eed7299f05c9b0ca The PNG encoder's deflate in src/png/encode.rs: the fixed Huffman code tuned for filtered rows, its stored dynamic-block header, zero runs and the bit writer; the Paeth pass and chunk writer are new.
adler2 0BSD OR MIT OR Apache-2.0 2.0.1, 89a031a0f42eeff31c70dc598b398cbf31f1680f The four-lane Adler-32 in src/png/inflate.rs.
crc32fast MIT OR Apache-2.0 (Sam Rijs, Alex Crichton and contributors) 1.5.2, 7eb0b8a2c9b246d27dc4cb5a53d3fc3b43a4bb83 The slice-by-16 CRC-32 in src/png/crc.rs that the PNG encoder writes and the decoder checks.
jpeg-encoder MIT (jpeg-encoder is (MIT OR Apache-2.0) AND IJG, Volker Ströbel); its DCT under the IJG licence, its AVX2 DCT under libjpeg-turbo's zlib licence 0.7.1, baf40191ae25a32e56a884fa6e001c4fe938d451 The baseline JPEG encoder in src/jpeg/: the interleaved 4:2:0 scan, the quality-scaled Annex K quantisation and Huffman tables, the reciprocal quantiser, the bit writer and markers, the integer RGB to YCbCr conversion and the islow forward DCT with their AVX2 forms; the RGBA load of the AVX2 colour conversion and the fixed pass-2 rounding of the AVX2 DCT are new.
zune-jpeg MIT (zune-jpeg is MIT OR Apache-2.0 OR Zlib, the zune-image developers) 0.5.15, 31d81fed7551c8ccea456d9d8e2b1fd8bebb6995 The JPEG decoder in src/jpeg/decode.rs and src/jpeg/decode/: Huffman tables with lookahead and fast AC, the bit reader, baseline and progressive (spectral selection and successive approximation) decode, the integer IDCT, fancy upsampling and YCbCr to RGB; the progressive AC refinement loop follows IJG libjpeg's jdphuff.c; EXIF orientation and the CMYK and YCCK refusals are new.
stb_image MIT or public domain (Sean Barrett), through zune-jpeg The stb_image.h code zune-jpeg 0.5.15 ported The ultimate origin of the decoder's integer IDCT, its fast AC table and the 14-bit YCbCr to RGB constants in src/jpeg/decode/.
cros-codecs BSD-3-Clause (The ChromiumOS Authors) 5ff6d693ffae0b36935b8fc13092c733b4c2646f The AV1 OBU, sequence, frame header and tile group parser in src/av1/ and the VA-API AV1 picture and tile parameter fill in src/gpu/vaapi/av1.rs.
libva va_dec_av1.h MIT (Intel) libva 2.23 The VA-API AV1 decode structures in src/gpu/vaapi/av1.rs.
matplotlib inferno colour map CC0 1.0 (public domain) _cm_listed.py, unchanged since matplotlib 1.5 Ten colour stops of inferno, interpolated for the spectrogram in src/filter/spectrum.rs.
sha2 MIT (sha2 is MIT OR Apache-2.0, RustCrypto Developers) 0.10.9, 82c36a428f8d6f05f3bfccdedb243e9d1f85359d The SHA-256 compression functions (portable, x86-64 SHA-NI and AArch64 SHA2) and round constants in src/bin/pcod/sha256.rs; the streaming buffer and dispatch are new.
pconv, pcod's predecessor, folded into this repository MIT (Catalin Ilinca) v1.0.1, 8879614 The progress line in src/bin/pcod/progress.rs, the probe JSON keys in src/bin/pcod/probe_json.rs, the --json manifest shape in src/bin/pcod/manifest.rs, and the command line, its checks, sizes, profiles and refusals in src/bin/pcod/{args,plan,size,profile,convert}.rs.
rav1d BSD-2-Clause (VideoLAN and dav1d authors, Internet Security Research Group) 0c851fa046d6a36430570459283e5886fbdd8a66 The AV1 decoder in src/av1/cpu/: OBU and header parsing, the entropy decoder and CDFs, reconstruction (intra, inter, transforms), loop filter, CDEF, loop restoration, film grain, reference MVs and frame and tile threading, with the Rust fallback kernels; the frame front is new.
zimg WTFPL v2 (sekrit-twc) 67e0603271c080e22c8429856dd4a8a56587e61e The resampling filter construction, Lanczos weights and BT.1886, BT.709 and sRGB transfer functions behind the linear-light resize in src/filter/linear.rs; the LUTs, banding and colour conversions are new.
fast_image_resize MIT OR Apache-2.0 (Kirill Kuzminykh) 6.1.0, ce36aeb20ed4a33805baa3c0ff4caff7cf163e19 The interleaved RGBA f32 convolution layout with tap pairs per AVX2 register in src/filter/linear.rs, accumulating in f32 with FMA.
libFLAC BSD-3-Clause (Josh Coalson, Xiph.Org Foundation) 1.5.0, 1507800de4b70e21be71f38caa0d9079d0bc6e45 The FLAC frame header, subframe, Rice residual (escape partitions included), fixed and LPC restore rules, the 32-bit or 64-bit restore choice and the CRC-8 and slice-by-8 CRC-16 in src/flac/.
Claxon Apache-2.0 (Ruud van Asseldonk) v0.4.3, 2f053855d581cfe3b9e7cef67fed5b0b0ccc45aa The FLAC decoder's module layout, the in-place residual and restore, the stereo decorrelation, the STREAMINFO layout and the unit test vectors in src/flac/; the 64-bit bit reader with its fast Rice path, the unrolled LPC, escape partitions, the file reader and its seeking are new.
Sun Microsystems g711.c Unrestricted use (Sun Microsystems, Inc.) the CCITT G.711 reference, unversioned The A-law and mu-law expansion (alaw2linear, ulaw2linear) behind the PCM tables in src/media/front/sound.rs; the Matroska A_MS/ACM and MOV alaw/ulaw mapping is new.
pconv, pcod's predecessor, folded into this repository MIT (Catalin Ilinca) v1.0.1, 8879614b928b8fdde500a16376b85a2c95551dc9 The join's copy test, its reasons and their wording, the letterbox fit and the colour rules in src/media/front/join.rs and src/media/front/shape.rs; the packet copy, timing and audio fill are new.
pconv, pcod's predecessor, folded into this repository MIT (Catalin Ilinca) v1.0.1, 8879614b928b8fdde500a16376b85a2c95551dc9 The loudness target, its limits and defaults, the YouTube preset and the silent-source refusal in src/media/front/level.rs; the two-pass levelling, limiter calibration and stream copy are new.
lewton MIT (lewton is MIT OR Apache-2.0, est31 and contributors) 0.10.2 plus 29 commits, bb2955b717094b40260902cf2f8dd9c5ea62a84a The Vorbis bit reader, Huffman codebook trees with their 8-bit lookup table, the identification, comment and setup headers (codebooks and VQ lookups, floors 0 and 1, residues, mappings, modes) and the window, twiddle and bark-map tables in src/vorbis/, and the audio packet decode (floors 0 and 1, residues 0 to 2, inverse coupling, inverse MDCT, windowing and overlap-add) in src/vorbis/audio.rs and src/vorbis/imdct.rs; the 128-bit reader window, the refusals in place of panics, the Ogg reader with its granule trimming, the Matroska CodecPrivate split and the primed lead-in for container timing are new.
stb_vorbis MIT (stb_vorbis is public domain or MIT, Sean Barrett) as translated in lewton bb2955b717094b40260902cf2f8dd9c5ea62a84a The fast inverse MDCT (the multirate filter-bank kernel with its merged steps) in src/vorbis/imdct.rs.
libvorbis BSD-3-Clause (Xiph.org Foundation) v1.3.7, 0657aee69dec8508a0011f47f3b69d7538e9d262 The codebook size checks of vorbis_staticbook_unpack, the floor 0 checks of floor0_unpack and the residue classbook checks of res0_unpack in src/vorbis/header.rs, and vorbisfile's granule rules (leading samples dropped by the first granule page, the end set by the last) in src/vorbis/ogg.rs.

Specifications

  • The ITU-T H.264 and H.265 Recommendations, and the ISO/IEC 14496 parts for AAC and the MP4 file format.
  • The VP9 Bitstream and Decoding Process Specification (v0.6, WebM project).
  • The AV1 Bitstream and Decoding Process Specification (AOMedia, v1.0.0 with Errata 1), against which the cros-codecs parser port in src/av1/ was checked.
  • The VP Codec ISO Media File Format Binding (v1.0, WebM project) for the vp09 sample entry and its vpcC box, and the WebM project's VP9 levels for the level it carries.
  • RFC 6716 and RFC 8251 (Opus), RFC 7845 (Ogg Opus), RFC 8794 and RFC 9559 (EBML and Matroska) and the WebM container guidelines.
  • The Vorbis I specification (Xiph.org Foundation), against which the lewton header port in src/vorbis/ was checked.
  • RFC 9649 (WebP: the container, VP8L lossless and ALPH, written from it alone) and RFC 6386 (VP8 lossy) for src/webp/.
  • CompuServe's GIF89a specification (1990), the source for src/gif/; the encoder's frame differencing (the changed-pixel tolerance, the settle limit near moving blocks, the whole-frame redraw on a slow drift), its clamped Sierra-2-4A error diffusion and its 7-bit nearest-colour table follow pconv v1.0.1 (8879614b928b8fdde500a16376b85a2c95551dc9, MIT, same author).
  • RFC 1950 and RFC 1951 (zlib and deflate) and the W3C PNG specification for src/png/; Microsoft's RIFF WAVE format notes, the only source for src/media/wav.rs.
  • ITU-T T.81 (ISO/IEC 10918-1, JPEG) and its Annex K tables, and JFIF 1.02, for src/jpeg/.
  • ATSC A/52 and ETSI TS 102 366 (AC-3 and E-AC-3), which oxideav-ac3 follows; the KBD window in src/ac3/ is computed from the A/52 definition.
  • RFC 9639 (FLAC) for src/flac/, the Matroska codec mapping for A_FLAC and the FLAC-in-ISOBMFF encapsulation (the fLaC sample entry and its dfLa box).
  • ITU-R BS.1770-4 and EBU Tech 3341 and 3342 (loudness), and ITU-R BT.601, BT.709 and BT.2020 (colour).
  • The paper "A novel macroblock-tree algorithm for high-performance optimization of dependent video coding in H.264/AVC" (2009), whose published description is the only source for the lookahead QP offsets in src/h264/enc/mbtree.rs.

Read, never copied

FFmpeg, x264, x265 and libde265 are GPL or LGPL. They are read for understanding only: no code from them is copied, translated line by line, linked or bundled. FFmpeg appears only as the test oracle and the benchmark reference.

What may come in

  • Code under MIT, Apache-2.0, BSD-2-Clause, BSD-3-Clause, ISC or zlib may be adapted. Each origin gets its copyright and licence text in NOTICE and a row above with its tag or commit. Apache-2.0-only code also carries its own NOTICE text, if it has one, and NOTICE says which modules were changed.
  • MPL-2.0 code is not taken.
  • GPL and LGPL code is never copied.
  • libvpx (BSD-3-Clause, with Google's additional patent grant for VP8 and VP9) and libopus (BSD-3-Clause, the RFC 6716 reference) may be ported, encoders included, with their copyright and licence text in NOTICE.
  • Cisco's openh264 (BSD-2-Clause), Kvazaar (BSD-3-Clause) and dav1d (BSD-2-Clause) may be adapted with their notices. openh264's patent licence covers only Cisco's own binaries, not code built from it.
  • Test media is generated by the tests or comes with a clear licence; none is committed with unclear terms.