draft-dutta-gcmf-00.txt   draft-dutta-gcmf-01.txt 
Independent Research S. Dutta Independent Research S. Dutta
Internet-Draft The ZSMC Co. Internet-Draft The ZSMC Co.
Intended status: Experimental 29 August 2026 Intended status: Experimental 29 August 2026
Expires: 2 March 2027 Expires: 2 March 2027
General-Purpose Compression via Mathematical Functions (GCMF) General-Purpose Compression via Mathematical Functions (GCMF)
draft-dutta-gcmf-00 draft-dutta-gcmf-01
Abstract Abstract
This document specifies General-Purpose Compression via Mathematical This document specifies General-Purpose Compression via Mathematical
Functions (GCMF), a lossless compression format that represents Functions (GCMF), a lossless compression format that represents
sequences of data using mathematical functions and associated sequences of data using mathematical functions and associated
parameters. parameters.
GCMF attempts to represent a sequence using a compact mathematical GCMF attempts to represent a sequence using a compact mathematical
representation rather than storing every value explicitly. This representation rather than storing every value explicitly. This
skipping to change at page 2, line 10 skipping to change at page 2, line 10
Provisions Relating to IETF Documents (https://trustee.ietf.org/ Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document. license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. and restrictions with respect to this document.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3
2. File Extension and MIME Type . . . . . . . . . . . . . . . . 3 2. File Extension and MIME Type . . . . . . . . . . . . . . . . 3
3. Function Representation Model . . . . . . . . . . . . . . . . 3 3. Function Representation Model . . . . . . . . . . . . . . . . 4
3.1. Source Size Comparison . . . . . . . . . . . . . . . . . 4 3.1. Source Size Comparison . . . . . . . . . . . . . . . . . 5
4. GCMF Binary Format Specification . . . . . . . . . . . . . . 5 4. GCMF Binary Format Specification . . . . . . . . . . . . . . 5
4.1. File Structure . . . . . . . . . . . . . . . . . . . . . 5 4.1. File Structure . . . . . . . . . . . . . . . . . . . . . 6
4.2. Magic Number . . . . . . . . . . . . . . . . . . . . . . 6 4.2. Magic Number . . . . . . . . . . . . . . . . . . . . . . 6
4.3. Version . . . . . . . . . . . . . . . . . . . . . . . . . 6 4.3. Version . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.4. Function Address . . . . . . . . . . . . . . . . . . . . 6 4.4. Function Address . . . . . . . . . . . . . . . . . . . . 7
4.4.1. Residual Flag . . . . . . . . . . . . . . . . . . . . 6 4.4.1. Residual Flag . . . . . . . . . . . . . . . . . . . . 7
4.4.2. Function Type Address . . . . . . . . . . . . . . . . 7 4.4.2. Function Type Address . . . . . . . . . . . . . . . . 7
4.5. Bit Packing . . . . . . . . . . . . . . . . . . . . . . . 7 4.5. Bit Packing . . . . . . . . . . . . . . . . . . . . . . . 7
4.6. Variable Length Integer Encoding . . . . . . . . . . . . 7 4.6. Variable Length Integer Encoding . . . . . . . . . . . . 8
4.7. Signed Integer Encoding . . . . . . . . . . . . . . . . . 8 4.7. Signed Integer Encoding . . . . . . . . . . . . . . . . . 8
4.8. CUSTOM Function . . . . . . . . . . . . . . . . . . . . . 8 4.8. CUSTOM Function . . . . . . . . . . . . . . . . . . . . . 8
4.9. Predefined Functions . . . . . . . . . . . . . . . . . . 9 4.9. Predefined Functions . . . . . . . . . . . . . . . . . . 10
4.9.1. Predefined Function Parameter Encoding . . . . . . . 10 4.9.1. Predefined Function Parameter Encoding . . . . . . . 11
4.10. Function Bytecode . . . . . . . . . . . . . . . . . . . . 11 4.10. Function Bytecode . . . . . . . . . . . . . . . . . . . . 12
4.11. Stack Machine Requirements . . . . . . . . . . . . . . . 11 4.11. Stack Machine Requirements . . . . . . . . . . . . . . . 12
4.11.1. Numerical Evaluation . . . . . . . . . . . . . . . . 12 4.11.1. Numerical Evaluation . . . . . . . . . . . . . . . . 13
4.12. Example Custom Function . . . . . . . . . . . . . . . . . 12 4.12. Example Custom Function . . . . . . . . . . . . . . . . . 13
4.13. X_start and X_end . . . . . . . . . . . . . . . . . . . . 12 4.13. X_start and X_end . . . . . . . . . . . . . . . . . . . . 13
4.14. Base Encoding . . . . . . . . . . . . . . . . . . . . . . 13 4.14. Base Encoding . . . . . . . . . . . . . . . . . . . . . . 14
4.15. Base Interpretation . . . . . . . . . . . . . . . . . . . 13 4.15. Base Interpretation . . . . . . . . . . . . . . . . . . . 14
4.16. Residual Representation . . . . . . . . . . . . . . . . . 14 4.16. Residual Representation . . . . . . . . . . . . . . . . . 15
4.17. Residual Function Container . . . . . . . . . . . . . . . 14 4.17. Residual Function Container . . . . . . . . . . . . . . . 15
4.18. Literal . . . . . . . . . . . . . . . . . . . . . . . . . 15 4.18. Literal . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.19. Physical End Of File . . . . . . . . . . . . . . . . . . 16 4.19. Physical End Of File . . . . . . . . . . . . . . . . . . 17
4.20. Complete Canonical Layout . . . . . . . . . . . . . . . . 16 4.20. Complete Canonical Layout . . . . . . . . . . . . . . . . 17
4.21. Encoding Example . . . . . . . . . . . . . . . . . . . . 17 4.21. Encoding Example . . . . . . . . . . . . . . . . . . . . 18
4.22. Encoder Selection Algorithm . . . . . . . . . . . . . . . 18 4.22. Encoder Selection Algorithm . . . . . . . . . . . . . . . 19
4.23. Literal Selection . . . . . . . . . . . . . . . . . . . . 18 4.23. Literal Selection . . . . . . . . . . . . . . . . . . . . 19
4.24. Decoder State Machine . . . . . . . . . . . . . . . . . . 18 4.24. Decoder State Machine . . . . . . . . . . . . . . . . . . 20
4.25. Decoder Validation . . . . . . . . . . . . . . . . . . . 19 4.25. Decoder Validation . . . . . . . . . . . . . . . . . . . 21
4.26. Canonical Encoding . . . . . . . . . . . . . . . . . . . 21 4.26. Canonical Encoding . . . . . . . . . . . . . . . . . . . 23
4.27. Versioning . . . . . . . . . . . . . . . . . . . . . . . 21 4.27. Versioning . . . . . . . . . . . . . . . . . . . . . . . 23
4.28. Reserved Space . . . . . . . . . . . . . . . . . . . . . 22 4.28. Reserved Space . . . . . . . . . . . . . . . . . . . . . 24
4.29. Losslessness Requirement . . . . . . . . . . . . . . . . 22 4.29. Losslessness Requirement . . . . . . . . . . . . . . . . 24
4.30. Summary of Binary Grammar . . . . . . . . . . . . . . . . 22 4.30. Summary of Binary Grammar . . . . . . . . . . . . . . . . 24
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26
6. Security Considerations . . . . . . . . . . . . . . . . . . . 24 6. Security Considerations . . . . . . . . . . . . . . . . . . . 26
7. Normative References . . . . . . . . . . . . . . . . . . . . 25 7. Normative References . . . . . . . . . . . . . . . . . . . . 27
8. Informative References . . . . . . . . . . . . . . . . . . . 25 8. Informative References . . . . . . . . . . . . . . . . . . . 27
Appendix A. Example GCMF Representation . . . . . . . . . . . . 25 Appendix A. Example GCMF Representation . . . . . . . . . . . . 27
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 26 Appendix B. Changes from draft-dutta-gcmf-00 . . . . . . . . . . 30
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 32
1. Introduction 1. Introduction
General-Purpose Compression via Mathematical Functions (GCMF) is a General-Purpose Compression via Mathematical Functions (GCMF) is a
lossless compression method that represents data using mathematical lossless compression method that represents data using mathematical
functions. functions.
Instead of storing every value in a sequence explicitly, GCMF may Instead of storing every value in a sequence explicitly, GCMF may
store a function and a range over which the function can reproduce store a function and a range over which the function can reproduce
the original values. the original values.
The objective of GCMF is to reduce the amount of information required The objective of GCMF is to reduce the amount of information required
to represent sequences that exhibit mathematical regularity, in the to represent sequences that exhibit mathematical regularity, in the
spirit of minimum-description-length approaches to modeling such as spirit of minimum-description-length approaches to modeling such as
[J_RISSANEN_1978]. [J_RISSANEN_1978].
A GCMF encoder MUST evaluate a set of valid candidate representations A GCMF encoder MUST evaluate a set of valid candidate representations
supported by the implementation and SHOULD select the candidate supported by the implementation and SHOULD select the candidate
having the minimum complete encoded length, within the closed having the minimum complete encoded length, as described in
interval [X_start, X_end], as described in Section 3. Section 3.
1.1. Terminology 1.1. Terminology
The key words MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
document are to be interpreted as described in [RFC2119]. "OPTIONAL" in this document are to be interpreted as described in
BCP 14 [RFC2119] when, and only when, they appear in all capitals, as
shown here.
This document also uses "OPTIONAL" as a structural label within
binary layout diagrams (Section 4) to mark a field that is absent
under some, but not all, values of a preceding discriminant field
(for example, a Function Address or a predefined-function identity).
A field so marked is still mandatory whenever the applicable
discriminant requires it; "OPTIONAL" in a diagram never means an
implementation may omit the field at its own discretion. Section 4.1
identifies, for each representation, which fields are conditional and
on what.
2. File Extension and MIME Type 2. File Extension and MIME Type
GCMF currently utilizes a working MIME type of application/gcmf. GCMF currently utilizes a working MIME type of application/gcmf.
This is not an officially registered MIME type under [RFC2046], or This is not an officially registered MIME type under [RFC2046], or
under any current standard or working group as of the publication of under any current standard or working group as of the publication of
this document. It is intended only to identify the file type to this document. It is intended only to identify the file type to
adopting implementations. adopting implementations.
GCMF also currently utilizes a working file extension of .gcmf. GCMF also currently utilizes a working file extension of .gcmf.
skipping to change at page 4, line 4 skipping to change at page 4, line 16
Until formal registration occurs, this extension SHOULD be treated Until formal registration occurs, this extension SHOULD be treated
with the same significance as the MIME type above. with the same significance as the MIME type above.
3. Function Representation Model 3. Function Representation Model
GCMF represents a numerical data sequence as the evaluation of a GCMF represents a numerical data sequence as the evaluation of a
mathematical function over a finite domain, optionally augmented by a mathematical function over a finite domain, optionally augmented by a
residual function. The original sequence MUST be: residual function. The original sequence MUST be:
D = d0, d1, ..., d(n-1) D = d0, d1, ..., d(n-1)
A GCMF encoder MUST attempt to determine a function (f), a domain A GCMF encoder MUST attempt to determine a function (f), a domain
sequence (X = x0, x1, ..., x(n-1)), and MAY also determine a residual sequence (X = x0, x1, ..., x(n-1)), and MAY also determine a residual
function. For a generalized representation: function. For a generalized representation:
d(i) = f(x(i)) d(i) = f(x(i))
For an exact representation: For an exact representation:
d(i) = f(x(i)) + r(x(i)) d(i) = f(x(i)) + r(x(i))
The exact representation is therefore a special case of the The exact representation is therefore a special case of the
generalized representation for which: generalized representation for which:
r(x) = 0 r(x) = 0
Let C be the theoretical set of all valid GCMF candidate Let C be the theoretical set of all valid GCMF candidate
representations and let C_I be the subset of candidates generated by representations, and let C_I be the subset of candidates that a
a particular implementation, where C_I is a subset of C. The encoder particular implementation is capable of generating, where C_I is a
optimization objective is: subset of C. Because a real implementation can only search C_I, its
encoder optimization objective is:
C*_I = arg min L(C), for C in C_I C*_I = arg min L(C), for C in C_I
where L(C) is the complete encoded length of candidate C in bytes. where L(C) is the complete encoded length of candidate C in bytes, as
defined in Section 3.1.
An implementation MAY use heuristic, statistical, symbolic, or other An implementation MAY use heuristic, statistical, symbolic, or other
candidate-discovery methods. An implementation is not required to candidate-discovery methods. An implementation is not required to
discover every mathematically valid representation. discover every mathematically valid representation; consequently an
implementation is not required to find C*, the theoretical global
optimum over C, and is only required to find C*_I, the best candidate
it is able to construct.
A candidate is considered compressive only when its complete encoded A candidate is considered compressive only when its complete encoded
representation is smaller than the original input under the source- representation is smaller than the original input under the source-
size comparison defined in Section 3.1. The global optimization size comparison defined in Section 3.1.
objective is:
C* = arg min L(C), for C in C
where C is the set of all valid GCMF candidate representations and
L(C) is the number of bytes required to encode candidate C.
3.1. Source Size Comparison 3.1. Source Size Comparison
For the purpose of determining whether a GCMF representation is For the purpose of determining whether a GCMF representation is
compressive, the original input is considered as a sequence of N compressive, the original input is considered as a sequence of N
bytes and has a source size of N bytes. bytes and has a source size of N bytes.
A GCMF representation is compressive if and only if: A GCMF representation is compressive if and only if:
L(C) < N L(C) < N
skipping to change at page 5, line 26 skipping to change at page 5, line 37
This section defines the binary representation of GCMF. A conforming This section defines the binary representation of GCMF. A conforming
GCMF decoder MUST be capable of parsing all mandatory structure GCMF decoder MUST be capable of parsing all mandatory structure
defined in this specification. Unless otherwise stated, all bit defined in this specification. Unless otherwise stated, all bit
fields are packed from the most-significant bit to the least- fields are packed from the most-significant bit to the least-
significant bit. significant bit.
GCMF is a lossless representation format. A valid GCMF stream MUST GCMF is a lossless representation format. A valid GCMF stream MUST
decode to exactly the byte sequence represented by the encoded data. decode to exactly the byte sequence represented by the encoded data.
Every GCMF representation is one of exactly two kinds: a LITERAL
representation (Section 4.18) or a mathematical representation
(Section 4.8 through Section 4.17). A mathematical representation is
further one of exactly two kinds: a CUSTOM function representation,
in which the function is defined inline by bytecode, or a PREDEFINED
function representation, in which the function is identified by
Function Address alone and looked up in the registry of Section 4.9.
These branches are mutually exclusive: a single representation never
contains both a CUSTOM bytecode body and a PREDEFINED parameter body,
and never contains both a mathematical body and a LITERAL payload.
The field tables in Section 4.8, Section 4.9, and Section 4.18 give
the exact, non-overlapping field sequence for each of these three
branches; the combined view in Section 4.20 shows them side by side
for comparison only and is not itself a fourth, independent layout.
4.1. File Structure 4.1. File Structure
A GCMF file consists of a fixed header followed by a representation- A GCMF file consists of a fixed header followed by a representation-
specific payload. The general structure is: specific payload. The general structure is:
+--------------------------------+ +--------------------------------+
| Magic | 4 bytes | | Magic | 4 bytes |
+--------------------------------+ +--------------------------------+
| Version | 1 byte | | Version | 1 byte |
+--------------------------------+ +--------------------------------+
| Function Address | 5 bits | | Function Address | 5 bits |
+--------------------------------+ +--------------------------------+
| Representation Data | variable| | Representation Data | variable, see Sections 4.8, 4.9, 4.18 |
+--------------------------------+ +--------------------------------+
| Physical EOF | | Physical EOF |
+--------------------------------+ +--------------------------------+
The Function Address determines the interpretation of Representation The Function Address determines the interpretation of Representation
Data. For mathematical representations, Representation Data consists Data. For mathematical representations, Representation Data consists
of the function definition or predefined-function parameters, of the function body (either inline CUSTOM bytecode or PREDEFINED
X_start, X_end, BASE, and, when present, the residual representation. parameters), X_start, X_end, BASE, and, when the residual flag is
set, the residual representation. For LITERAL, Representation Data
For LITERAL, Representation Data consists entirely of the original consists entirely of the original source bytes after the required
source bytes after the required alignment padding. alignment padding.
Physical EOF is not encoded as a field or byte. The end of the Physical EOF is not encoded as a field or byte; it is the point at
physical GCMF stream terminates the final representation. which the physical GCMF stream ends, and it terminates the final
representation. There is exactly one Magic field and one Physical
EOF per file: unlike some of the illustrative field tables later in
this section, the file as a whole never repeats the Magic field or
contains an in-band EOF marker (see Section 4.19).
The exact fields present after the function address depend on the The exact fields present after the Function Address depend on the
selected function type. The LITERAL function type is an exception selected function type. The LITERAL function type is an exception
and terminates the structured portion of the stream. and terminates the structured portion of the stream after three
padding bits.
4.2. Magic Number 4.2. Magic Number
Every GCMF file MUST begin with the following 4 bytes: 0x47 0x43 0x4D Every GCMF file MUST begin with the following 4 bytes: 0x47 0x43 0x4D
0x46. These bytes correspond to the ASCII string "GCMF". A decoder 0x46. These bytes correspond to the ASCII string "GCMF". A decoder
MUST reject a file whose first 4 bytes do not equal 0x47 0x43 0x4D MUST reject a file whose first 4 bytes do not equal 0x47 0x43 0x4D
0x46. 0x46.
4.3. Version 4.3. Version
skipping to change at page 6, line 49 skipping to change at page 7, line 35
where R is the residual flag and AAAA is the function type address. where R is the residual flag and AAAA is the function type address.
4.4.1. Residual Flag 4.4.1. Residual Flag
The most-significant bit MUST be the residual flag, where 0 indicates The most-significant bit MUST be the residual flag, where 0 indicates
no residual function is present and 1 indicates a residual function no residual function is present and 1 indicates a residual function
is present. is present.
If R = 1, the residual representation immediately follows the primary If R = 1, the residual representation immediately follows the primary
mathematical representation according to the residual encoding rules mathematical representation according to the residual encoding rules
in Section 4.16. in Section 4.16. R MUST be 0 for LITERAL (Section 4.18).
4.4.2. Function Type Address 4.4.2. Function Type Address
LITERAL + RESIDUAL is reserved and MUST be rejected by a conforming The four-bit function type address AAAA selects one of the sixteen
function types listed in Table 1. LITERAL with the residual flag set
(address 10001) is reserved and MUST be rejected by a conforming
decoder unless a future GCMF version explicitly defines semantics for decoder unless a future GCMF version explicitly defines semantics for
it. it.
4.5. Bit Packing 4.5. Bit Packing
The Function Address occupies exactly five bits and MUST NOT The Function Address occupies exactly five bits and MUST NOT
automatically consume a complete byte. automatically consume a complete byte.
The bit immediately following the Function Address is the first bit The bit immediately following the Function Address is the first bit
of the next field unless alignment is explicitly required by the of the next field unless alignment is explicitly required by the
skipping to change at page 8, line 27 skipping to change at page 9, line 11
4.8. CUSTOM Function 4.8. CUSTOM Function
A CUSTOM function has Function Type Address: A CUSTOM function has Function Type Address:
0000 0000
or, when residual data is present: or, when residual data is present:
10000 10000
The structure is: A CUSTOM representation always carries an inline function body; the
Function Length and Function Bytecode fields below are therefore
mandatory whenever Function Address selects CUSTOM, and are never
present under any other function type. The structure is:
+--------------------------------------------------------------+
| Magic | 4 bytes (0x47434D46) |
+--------------------------------------------------------------+
| Version | 1 byte (0x00) |
+--------------------------------------------------------------+
| Function Address | 5 bits |
+--------------------------------------------------------------+
| Function Length | variable VLI |
+--------------------------------------------------------------+
| Function Bytecode | Function Length bits |
+--------------------------------------------------------------+
| X_start | variable VLI |
+--------------------------------------------------------------+
| X_end | variable VLI |
+--------------------------------------------------------------+
| Base | variable VLI |
+--------------------------------------------------------------+
| [Residual] | present iff R = 1, see Section 4.17 |
+--------------------------------------------------------------+
Physical EOF follows the final field; per Section 4.19 no additional
Magic or EOF field is encoded at the end of the representation.
+-------------------------------------------+
| Magic | 4 bytes (0x47434D46) |
+-------------------------------------------+
| Version | 1 byte (0x00) |
+-------------------------------------------+
| Function Address | 5 bits |
+-------------------------------------------+
| Function Length | OPTIONAL VLI |
+-------------------------------------------+
| Function Bytecode | OPTIONAL, Function |
| | Length bits |
+-------------------------------------------+
| X_start | variable VLI |
+-------------------------------------------+
| X_end | variable VLI |
+-------------------------------------------+
| Base | variable VLI |
+-------------------------------------------+
| Magic | 4 bytes (0x47434D46)|
+-------------------------------------------+
| EOF |
+-------------------------------------------+
The Function Length specifies the number of bits occupied by the The Function Length specifies the number of bits occupied by the
function bytecode, including opcode bits, operand bits, and the END function bytecode, including opcode bits, operand bits, and the END
instruction. instruction.
The Function Length MUST be sufficient to contain a complete bytecode The Function Length MUST be sufficient to contain a complete bytecode
sequence ending in an END instruction. sequence ending in an END instruction.
The decoder MUST NOT read beyond the declared function length while The decoder MUST NOT read beyond the declared function length while
executing the function. executing the function.
skipping to change at page 9, line 28 skipping to change at page 10, line 15
The function bytecode MUST contain a valid END instruction within the The function bytecode MUST contain a valid END instruction within the
specified function length. specified function length.
4.9. Predefined Functions 4.9. Predefined Functions
A predefined function does not contain its mathematical definition in A predefined function does not contain its mathematical definition in
the file. The Function Address identifies the mathematical function the file. The Function Address identifies the mathematical function
through the GCMF predefined-function registry. The decoder obtains through the GCMF predefined-function registry. The decoder obtains
the function definition from the version-specific registry. the function definition from the version-specific registry.
A predefined function therefore has the general structure: A predefined representation never carries a Function Length or
Function Bytecode field; those fields are exclusive to CUSTOM
(Section 4.8). Depending on which predefined function is selected,
the Function Parameters field itself may be empty: Section 4.9.1
defines five predefined functions (FIBONACCI, FACTORIAL, TRIANGULAR,
SQUARE, CUBE) that take no parameters, in which case this field is
zero-length and X_start immediately follows the Function Address. A
predefined function therefore has the general structure:
+-------------------------------------------+ +--------------------------------------------------------------+
| Magic | 4 bytes (0x47434D46) | | Magic | 4 bytes (0x47434D46) |
+-------------------------------------------+ +--------------------------------------------------------------+
| Version | 1 byte (0x00) | | Version | 1 byte (0x00) |
+-------------------------------------------+ +--------------------------------------------------------------+
| Function Address | 5 bits | | Function Address | 5 bits |
+-------------------------------------------+ +--------------------------------------------------------------+
| Function Parameters | variable | | [Function Parameters] | OPTIONAL, variable -- empty for the |
+-------------------------------------------+ | | zero-parameter functions listed |
| X_start | variable VLI | | | above, otherwise per Section 4.9.1 |
+-------------------------------------------+ +--------------------------------------------------------------+
| X_end | variable VLI | | X_start | variable VLI |
+-------------------------------------------+ +--------------------------------------------------------------+
| Base | variable VLI | | X_end | variable VLI |
+-------------------------------------------+ +--------------------------------------------------------------+
| Magic | 4 bytes (0x47434D46)| | Base | variable VLI |
+-------------------------------------------+ +--------------------------------------------------------------+
| EOF | | [Residual] | present iff R = 1, see Section 4.17 |
+-------------------------------------------+ +--------------------------------------------------------------+
Physical EOF follows the final field; per Section 4.19 no additional
Magic or EOF field is encoded at the end of the representation.
Each predefined function MUST have a precisely defined parameter Each predefined function MUST have a precisely defined parameter
encoding. encoding.
4.9.1. Predefined Function Parameter Encoding 4.9.1. Predefined Function Parameter Encoding
Unless otherwise specified, all integer parameters of predefined Unless otherwise specified, all integer parameters of predefined
functions are encoded using GCMF-VLI. Parameters that may be functions are encoded using GCMF-VLI. Parameters that may be
negative MUST use ZigZag encoding before GCMF-VLI encoding. negative MUST use ZigZag encoding before GCMF-VLI encoding.
skipping to change at page 14, line 28 skipping to change at page 15, line 28
output. For every value: output. For every value:
r(i) = d(i) - f(x(i)) r(i) = d(i) - f(x(i))
The original datum is reconstructed using: The original datum is reconstructed using:
d(i) = f(x(i)) + r(i) d(i) = f(x(i)) + r(i)
The residual representation MUST itself be encoded using a valid GCMF The residual representation MUST itself be encoded using a valid GCMF
function representation. Therefore a residual MAY be a predefined function representation. Therefore a residual MAY be a predefined
function, a custom function, or another mathematically represented function or a custom function, subject to the restrictions imposed by
sequence, subject to the restrictions imposed by the GCMF version. this version of GCMF (see Section 4.17, which prohibits a residual
from itself carrying a further nested residual).
The residual representation does not independently require a second The residual representation does not independently require a second
Function Address byte unless specified by the residual container Function Address byte; it uses the four-bit Residual Function Type
structure. defined in Section 4.17 instead.
4.17. Residual Function Container 4.17. Residual Function Container
When R = 1, the residual representation immediately follows the When R = 1, the residual representation immediately follows the
complete primary mathematical representation. The residual section complete primary mathematical representation. The residual section
is encoded as: is encoded as:
+-----------------------------+ +----------------------------------------------------------+
| Residual Function Type | 4 bits | | Residual Function Type | 4 bits |
+-----------------------------+ +----------------------------------------------------------+
| Residual Function Data | variable | | Residual Function Data | variable, mandatory -- see below |
+-----------------------------+ +----------------------------------------------------------+
The Residual Function Type uses the same function-type address table The Residual Function Type uses the same function-type address table
as the primary function, excluding the residual flag. The residual as the primary function (Table 1), excluding the residual flag. The
function therefore contains exactly four function-type bits. residual function therefore contains exactly four function-type bits,
and Residual Function Data is always present: a custom residual's
data is its Function Length and Function Bytecode, and a predefined
residual's data is its parameter fields, which may be zero-length for
the parameterless functions of Section 4.9.1 but the field itself is
never omitted outright.
For a custom residual, 0000 is used as the Residual Function Type. A For a custom residual, 0000 is used as the Residual Function Type. A
custom residual MUST contain its Function Length followed by its custom residual MUST contain its Function Length followed by its
Function Bytecode according to the CUSTOM function encoding rules. Function Bytecode according to the CUSTOM function encoding rules.
A predefined residual MUST contain the parameter fields specified for A predefined residual MUST contain the parameter fields specified for
its function type. its function type.
The Residual Function Type MUST NOT encode LITERAL (0001): a residual
is always a mathematical representation, never a literal payload.
The residual function MUST NOT contain a residual flag and MUST NOT The residual function MUST NOT contain a residual flag and MUST NOT
contain another residual representation. Recursive residual nesting contain another residual representation. Recursive residual nesting
is therefore prohibited in GCMF v1. is therefore prohibited in GCMF v1.
The residual representation MUST generate exactly the same number of The residual representation MUST generate exactly the same number of
values as the primary representation. values as the primary representation.
4.18. Literal 4.18. Literal
The LITERAL function has Function Type Address: The LITERAL function has Function Type Address:
skipping to change at page 15, line 36 skipping to change at page 17, line 5
Address is therefore: Address is therefore:
00001 00001
LITERAL is a terminal representation. Immediately following the LITERAL is a terminal representation. Immediately following the
Function Address, the decoder MUST consume three zero-valued padding Function Address, the decoder MUST consume three zero-valued padding
bits. These three padding bits align the stream to the next byte bits. These three padding bits align the stream to the next byte
boundary. The remainder of the GCMF stream is then interpreted as boundary. The remainder of the GCMF stream is then interpreted as
raw source bytes. raw source bytes.
+----------------------+ +--------------------------+
| Magic | | Magic |
+----------------------+ +--------------------------+
| Version | | Version |
+----------------------+ +--------------------------+
| Function Address | | Function Address |
+----------------------+ +--------------------------+
| Raw source bytes | | Zero padding (3 bits) |
| Raw source bytes | +--------------------------+
| Raw source bytes | | Raw source bytes |
| ... | +--------------------------+
+----------------------+ | Physical EOF |
| EOF | +--------------------------+
+----------------------+
No function length is stored. No X_start is stored. No X_end is No function length is stored. No X_start is stored. No X_end is
stored. No BASE field is stored. No residual representation is stored. No BASE field is stored. No residual representation is
stored. stored. There is no trailing Magic field or in-band EOF field; per
Section 4.19, the physical end of the stream is the only termination
condition.
The decoder MUST copy the literal payload directly to the output The decoder MUST copy the literal payload directly to the output
without interpretation or transformation. without interpretation or transformation.
4.19. Physical End Of File 4.19. Physical End Of File
The physical end of the GCMF file is the termination condition for The physical end of the GCMF file is the termination condition for
the final field. No EOF byte or other in-band EOF marker is defined the final field. No EOF byte or other in-band EOF marker is defined
by GCMF. by GCMF, and the Magic field defined in Section 4.2 is never repeated
within a representation.
This is particularly important for LITERAL, because every byte value This is particularly important for LITERAL, because every byte value
from 0x00 through 0xFF is valid source data. Consequently, no byte from 0x00 through 0xFF is valid source data. Consequently, no byte
value MAY be reserved as an EOF marker. value MAY be reserved as an EOF marker.
For LITERAL, EOF is the physical end of the GCMF stream. The final For LITERAL, EOF is the physical end of the GCMF stream. The final
byte of the file is therefore the final literal source byte. byte of the file is therefore the final literal source byte.
A transport or container system embedding GCMF MUST preserve the A transport or container system embedding GCMF MUST preserve the
exact file length. exact file length.
4.20. Complete Canonical Layout 4.20. Complete Canonical Layout
For a normal mathematical representation without residual data: This section restates, side by side and for reference only, the three
mutually exclusive field sequences already given in full and without
ambiguity in Section 4.8 (CUSTOM), Section 4.9 (PREDEFINED), and
Section 4.18 (LITERAL). A single GCMF representation always matches
exactly one of these three sequences; the bracketed items below are
conditional exactly as specified in the section cited, not
independently optional.
CUSTOM mathematical representation (Section 4.8):
GCMF GCMF
VERSION VERSION
FUNCTION ADDRESS FUNCTION ADDRESS
[FUNCTION LENGTH] FUNCTION LENGTH
[FUNCTION BYTECODE] FUNCTION BYTECODE
[FUNCTION PARAMETERS]
X_START X_START
X_END X_END
BASE BASE
[RESIDUAL FUNCTION TYPE, RESIDUAL FUNCTION DATA] -- iff R = 1
EOF EOF
For a representation containing a residual: PREDEFINED mathematical representation (Section 4.9):
GCMF GCMF
VERSION VERSION
FUNCTION ADDRESS FUNCTION ADDRESS
[FUNCTION LENGTH] [FUNCTION PARAMETERS] -- zero-length for the parameterless
[FUNCTION BYTECODE] functions listed in Section 4.9.1
[FUNCTION PARAMETERS]
X_START X_START
X_END X_END
BASE BASE
RESIDUAL FUNCTION [RESIDUAL FUNCTION TYPE, RESIDUAL FUNCTION DATA] -- iff R = 1
RESIDUAL DATA
EOF EOF
For LITERAL: LITERAL representation (Section 4.18):
GCMF GCMF
VERSION VERSION
00001 00001
ZERO PADDING (3 BITS)
RAW DATA RAW DATA
EOF EOF
4.21. Encoding Example 4.21. Encoding Example
Consider the sequence: Consider the sequence:
10, 13, 16, 19, 22, 25 10, 13, 16, 19, 22, 25
It can be represented by the linear function f(x) = 3x + 10 over the It can be represented by the linear function f(x) = 3x + 10 over the
domain X = 0, 1, 2, 3, 4, 5. The logical GCMF representation is domain X = 0, 1, 2, 3, 4, 5. This is a PREDEFINED representation
therefore: (LINEAR), so it follows the PREDEFINED field sequence of Section 4.9
with Function Parameters = (a=3, b=10) and no residual. The logical
GCMF representation is therefore:
MAGIC MAGIC
VERSION VERSION
FUNCTION ADDRESS = LINEAR FUNCTION ADDRESS = LINEAR, R = 0
a = 3 a = 3
b = 10 b = 10
X_START = 0 X_START = 0
X_END = 5 X_END = 5
BASE BASE
PHYSICAL EOF PHYSICAL EOF
The decoder evaluates: The decoder evaluates:
f(0) = 10 f(0) = 10
skipping to change at page 19, line 16 skipping to change at page 21, line 16
| |
v v
READ MAGIC READ MAGIC
| |
v v
READ VERSION READ VERSION
| |
v v
READ FUNCTION ADDRESS READ FUNCTION ADDRESS
| |
+-----> LITERAL -----> COPY REMAINDER -----> END +-----> LITERAL --> SKIP 3 PADDING BITS --> COPY REMAINDER --> END
| |
v v
READ FUNCTION [not LITERAL]
| |
v v
READ PARAMETERS READ FUNCTION (CUSTOM: length + bytecode; PREDEFINED: parameters)
| |
v v
READ X_start READ X_start
| |
v v
READ X_end READ X_end
| |
v v
READ BASE READ BASE
| |
+-----> R = 0 -----> EVALUATE +-----> R = 0 -----> EVALUATE -----> END
| |
+-----> R = 1 -----> READ RESIDUAL +-----> R = 1 -----> READ RESIDUAL
| |
v v
EVALUATE RESIDUAL EVALUATE RESIDUAL
| |
v v
RECONSTRUCT RECONSTRUCT
| |
v v
END END
4.25. Decoder Validation 4.25. Decoder Validation
A decoder MUST reject a GCMF stream if any of the following occurs: A decoder MUST reject a GCMF stream if any of the following occurs:
* The magic number is invalid. * The magic number is invalid.
* The version is unsupported. * The version is unsupported.
* The Function Address is invalid. * The Function Address is invalid, including LITERAL with the
residual flag set (Section 4.4.2).
* A required VLI is truncated. * A required VLI is truncated.
* A VLI is malformed. * A VLI is malformed.
* A custom function exceeds its declared length. * A custom function exceeds its declared length.
* A custom function lacks a valid END. * A custom function lacks a valid END.
* A function causes stack underflow. * A function causes stack underflow.
* A function terminates with an invalid stack state. * A function terminates with an invalid stack state.
* X_end is less than X_start. * X_end is less than X_start.
* BASE is less than 2. * BASE is less than 2.
* A required function parameter is missing. * A required function parameter is missing.
* A residual representation is malformed. * A residual representation is malformed, including a Residual
Function Type of LITERAL (Section 4.17).
* A prohibited recursive residual is encountered. * A prohibited recursive residual is encountered.
* The mathematical evaluation produces a value that cannot be * The mathematical evaluation produces a value that cannot be
represented by the selected numerical representation. represented by the selected numerical representation.
* The stream terminates before all mandatory fields are read. * The stream terminates before all mandatory fields are read.
A decoder SHOULD also impose implementation-defined limits on: A decoder SHOULD also impose implementation-defined limits on:
skipping to change at page 23, line 36 skipping to change at page 25, line 28
FUNCTION := FUNCTION :=
CUSTOM_FUNCTION CUSTOM_FUNCTION
| PREDEFINED_FUNCTION | PREDEFINED_FUNCTION
CUSTOM_FUNCTION := CUSTOM_FUNCTION :=
FUNCTION_LENGTH FUNCTION_LENGTH
FUNCTION_BYTECODE FUNCTION_BYTECODE
PREDEFINED_FUNCTION := PREDEFINED_FUNCTION :=
FUNCTION_PARAMETERS [FUNCTION_PARAMETERS]
RESIDUAL := RESIDUAL :=
RESIDUAL_FUNCTION_TYPE RESIDUAL_FUNCTION_TYPE
RESIDUAL_FUNCTION_DATA RESIDUAL_FUNCTION_DATA
CUSTOM_RESIDUAL := RESIDUAL_FUNCTION_DATA :=
0000 CUSTOM_RESIDUAL_DATA
| PREDEFINED_RESIDUAL_DATA
CUSTOM_RESIDUAL_DATA :=
FUNCTION_LENGTH FUNCTION_LENGTH
FUNCTION_BYTECODE FUNCTION_BYTECODE
PREDEFINED_RESIDUAL_DATA :=
[FUNCTION_PARAMETERS]
PHYSICAL_EOF := PHYSICAL_EOF :=
END_OF_PHYSICAL_GCMF_STREAM END_OF_PHYSICAL_GCMF_STREAM
The fundamental mathematical decoding equation is: The fundamental mathematical decoding equation is:
d(i) = f(x(i)) + r(x(i)) d(i) = f(x(i)) + r(x(i))
where r(x(i)) = 0 for representations without residuals. For where r(x(i)) = 0 for representations without residuals. For
LITERAL, Decode(G) = LiteralPayload(G). No additional EOF field is LITERAL, Decode(G) = LiteralPayload(G). No additional EOF field is
encoded. encoded.
skipping to change at page 24, line 22 skipping to change at page 26, line 22
registration action from IANA at this time. Should GCMF advance registration action from IANA at this time. Should GCMF advance
toward broader adoption, a future document MAY request registration toward broader adoption, a future document MAY request registration
of application/gcmf in accordance with the procedures of [RFC2046] of application/gcmf in accordance with the procedures of [RFC2046]
and BCP 13. and BCP 13.
6. Security Considerations 6. Security Considerations
GCMF CUSTOM function representations embed executable stack-based GCMF CUSTOM function representations embed executable stack-based
bytecode within the file format, as described in Section 4.10. A bytecode within the file format, as described in Section 4.10. A
decoder that evaluates untrusted GCMF input therefore executes decoder that evaluates untrusted GCMF input therefore executes
untrusted, attacker- supplied instructions. Implementations MUST untrusted, attacker-supplied instructions. Implementations MUST
treat GCMF decoding of CUSTOM and MODULAR functions as execution of treat GCMF decoding of CUSTOM and MODULAR functions as execution of
untrusted code and SHOULD apply appropriate sandboxing. untrusted code and SHOULD apply appropriate sandboxing.
Because GCMF functions are evaluated over a domain that can be Because GCMF functions are evaluated over a domain that can be
specified independently of the resulting output size, a maliciously specified independently of the resulting output size, a maliciously
crafted file can specify an extremely large [X_start, X_end] domain, crafted file can specify an extremely large [X_start, X_end] domain,
or a MODULAR or POLYNOMIAL function with a large nesting or degree, or a MODULAR or POLYNOMIAL function with a large nesting or degree,
in order to force a decoder to perform an excessive amount of in order to force a decoder to perform an excessive amount of
computation or produce an excessive volume of output from a very computation or produce an excessive volume of output from a very
small input file. This is a form of decompression-bomb risk small input file. This is a form of decompression-bomb risk
skipping to change at page 25, line 22 skipping to change at page 27, line 22
environments SHOULD apply them at another layer. environments SHOULD apply them at another layer.
7. Normative References 7. Normative References
[RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet Mail [RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet Mail
Extensions (MIME) Part Two: Media Types", RFC 2046, Extensions (MIME) Part Two: Media Types", RFC 2046,
DOI 10.17487/RFC2046, November 1996, DOI 10.17487/RFC2046, November 1996,
<https://www.rfc-editor.org/rfc/rfc2046>. <https://www.rfc-editor.org/rfc/rfc2046>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", RFC 2119, DOI 10.17487/RFC2119, March Requirement Levels", BCP 14, RFC 2119,
1997, <https://www.rfc-editor.org/rfc/rfc2119>. DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/rfc/rfc2119>.
8. Informative References 8. Informative References
[J_RISSANEN_1978] [J_RISSANEN_1978]
Rissanen, J., "Modeling by shortest data description", Rissanen, J., "Modeling by shortest data description",
Automatica, vol. 14, no. 5, pp. 465-471, 1978. Automatica vol. 14, no. 5, pp. 465-471, 1978.
Appendix A. Example GCMF Representation Appendix A. Example GCMF Representation
This appendix reproduces, for reference, the worked example of This appendix reproduces, for reference, the worked example of
Section 4.21 in a single consolidated view. Section 4.21 in a single consolidated view.
Given the source sequence 10, 13, 16, 19, 22, 25 (six 8-bit unsigned Given the source sequence 10, 13, 16, 19, 22, 25 (six 8-bit unsigned
values, an 6-byte source), an encoder implementing this specification values, a 6-byte source), an encoder implementing this specification
could produce a LINEAR representation with a = 3, b = 10, X_start = could produce a LINEAR representation with a = 3, b = 10, X_start =
0, X_end = 5, discovering that: 0, X_end = 5, discovering that:
f(x) = 3x + 10 f(x) = 3x + 10
f(0) = 10 f(0) = 10
f(1) = 13 f(1) = 13
f(2) = 16 f(2) = 16
f(3) = 19 f(3) = 19
f(4) = 22 f(4) = 22
skipping to change at page 25, line 49 skipping to change at page 28, line 4
0, X_end = 5, discovering that: 0, X_end = 5, discovering that:
f(x) = 3x + 10 f(x) = 3x + 10
f(0) = 10 f(0) = 10
f(1) = 13 f(1) = 13
f(2) = 16 f(2) = 16
f(3) = 19 f(3) = 19
f(4) = 22 f(4) = 22
f(5) = 25 f(5) = 25
which exactly reproduces the six-value source sequence with an exact which exactly reproduces the six-value source sequence with an exact
(residual-free) representation, per Section 3. Per Section 4.22, the (residual-free) representation, per Section 3. Per Section 4.22, the
encoder would compare the total encoded length of this LINEAR encoder would compare the total encoded length of this LINEAR
representation, including all structural overhead, against the representation, including all structural overhead, against the
LITERAL representation of the same six bytes, and select whichever LITERAL representation of the same six bytes, and select whichever
candidate yields the smaller value of L(C). candidate yields the smaller value of L(C).
+======+=============+==========================================+
| AAAA | Function | Formula |
+======+=============+==========================================+
| 0000 | CUSTOM | User-defined |
+------+-------------+------------------------------------------+
| 0001 | LITERAL | N/A -- raw bytes, see Section 4.18 |
+------+-------------+------------------------------------------+
| 0010 | CONSTANT | F(x) = c |
+------+-------------+------------------------------------------+
| 0011 | LINEAR | F(x) = ax + b |
+------+-------------+------------------------------------------+
| 0100 | ARITHMETIC | F(x) = a + xd |
+------+-------------+------------------------------------------+
| 0101 | GEOMETRIC | F(x) = a r^x |
+------+-------------+------------------------------------------+
| 0110 | POLYNOMIAL | F(x) = sum_(i=0)^n a_i x^i |
+------+-------------+------------------------------------------+
| 0111 | EXPONENTIAL | F(x) = a b^x |
+------+-------------+------------------------------------------+
| 1000 | LOGARITHMIC | F(x) = a log_b(x) + c |
+------+-------------+------------------------------------------+
| 1001 | POWER | F(x) = a x^b + c |
+------+-------------+------------------------------------------+
| 1010 | MODULAR | F(x) = G(x) mod m |
+------+-------------+------------------------------------------+
| 1011 | FIBONACCI | F(x)=F(x-1)+F(x-2), x>=2, F(0)=0, F(1)=1 |
+------+-------------+------------------------------------------+
| 1100 | FACTORIAL | F(x) = x! |
+------+-------------+------------------------------------------+
| 1101 | TRIANGULAR | F(x) = x(x+1)/2 |
+------+-------------+------------------------------------------+
| 1110 | SQUARE | F(x) = x^2 |
+------+-------------+------------------------------------------+
| 1111 | CUBE | F(x) = x^3 |
+------+-------------+------------------------------------------+
Table 1: Function Type Addresses
NOTE: the LITERAL row was previously listed with the formula "F(x) =
x", which incorrectly implied LITERAL evaluates a function. Per
Section 4.18, LITERAL stores no function, domain, or base; the
decoded output is the raw payload copied verbatim (Section 4.29).
+========+==========+===========================+
| OPCODE | MNEMONIC | STACK OPERATION |
+========+==========+===========================+
| 00000 | CONST | Push constant |
+--------+----------+---------------------------+
| 00001 | X | Push current x |
+--------+----------+---------------------------+
| 00010 | ADD | a + b |
+--------+----------+---------------------------+
| 00011 | SUB | a - b |
+--------+----------+---------------------------+
| 00100 | MUL | a * b |
+--------+----------+---------------------------+
| 00101 | DIV | a / b |
+--------+----------+---------------------------+
| 00110 | MOD | a mod b |
+--------+----------+---------------------------+
| 00111 | POW | a ^ b |
+--------+----------+---------------------------+
| 01000 | NEG | -a |
+--------+----------+---------------------------+
| 01001 | ABS | |a| |
+--------+----------+---------------------------+
| 01010 | SQRT | sqrt(a) |
+--------+----------+---------------------------+
| 01011 | LOG | ln(a) |
+--------+----------+---------------------------+
| 01100 | LOG10 | log10(a) |
+--------+----------+---------------------------+
| 01101 | EXP | e^a |
+--------+----------+---------------------------+
| 01110 | SIN | sin(a) |
+--------+----------+---------------------------+
| 01111 | COS | cos(a) |
+--------+----------+---------------------------+
| 10000 | TAN | tan(a) |
+--------+----------+---------------------------+
| 10001 | ASIN | arcsin(a) |
+--------+----------+---------------------------+
| 10010 | ACOS | arccos(a) |
+--------+----------+---------------------------+
| 10011 | ATAN | arctan(a) |
+--------+----------+---------------------------+
| 10100 | FLOOR | floor(a) |
+--------+----------+---------------------------+
| 10101 | CEIL | ceil(a) |
+--------+----------+---------------------------+
| 10110 | ROUND | round(a) |
+--------+----------+---------------------------+
| 10111 | MIN | min(a, b) |
+--------+----------+---------------------------+
| 11000 | MAX | max(a, b) |
+--------+----------+---------------------------+
| 11001 | AND | a & b |
+--------+----------+---------------------------+
| 11010 | OR | a | b |
+--------+----------+---------------------------+
| 11011 | XOR | a xor b (bitwise) |
+--------+----------+---------------------------+
| 11100 | SHL | a << b |
+--------+----------+---------------------------+
| 11101 | SHR | a >> b |
+--------+----------+---------------------------+
| 11110 | DUP | Duplicate top stack value |
+--------+----------+---------------------------+
| 11111 | END | End function |
+--------+----------+---------------------------+
Table 2: Function Bytecode Opcodes
Appendix B. Changes from draft-dutta-gcmf-00
This appendix is non-normative and summarizes the corrections made
relative to draft-dutta-gcmf-00; it will be removed before this
document advances.
* Section 4.8 (CUSTOM Function): the layout table previously marked
Function Length and Function Bytecode as OPTIONAL, which
contradicted the surrounding prose (Function Length "MUST be
sufficient", bytecode "MUST contain a valid END"). These fields
are mandatory for every CUSTOM representation; the table has been
corrected and an explanatory sentence added.
* Section 4.9 (Predefined Functions): the layout table previously
showed Function Parameters as an unmarked, unconditionally present
field, even though Section 4.9.1 defines five predefined functions
that take no parameters. The field is now correctly marked
OPTIONAL/zero-length for those functions, with a cross-reference
to Section 4.9.1.
* Section 4.8 and Section 4.9: the layout tables previously repeated
a trailing "Magic | 4 bytes" and "EOF" field at the end of each
representation. This contradicted Section 4.1 (one Magic field
per file) and Section 4.19 (no in-band EOF marker is ever
encoded). The trailing Magic/EOF rows have been removed and
replaced with an explicit forward reference to Section 4.19.
* Section 4.20 (Complete Canonical Layout): previously presented a
single combined diagram with FUNCTION LENGTH, FUNCTION BYTECODE,
and FUNCTION PARAMETERS all bracketed as independently optional,
which incorrectly implied a single representation could mix CUSTOM
and PREDEFINED fields, or omit them freely. The section now
presents the CUSTOM, PREDEFINED, and LITERAL field sequences as
three separate, mutually exclusive layouts, consistent with the
grammar in Section 4.30.
* Section 4.18 (Literal): the layout table previously omitted the
3-bit zero-padding field shown in the surrounding prose, and
retained a trailing bare "EOF" row. The table now shows the
padding field explicitly and labels the terminal row "Physical
EOF" for consistency with Section 4.19.
* Section 4.1: added an explicit statement that the file contains
exactly one Magic field and no in-band EOF marker, to prevent the
per-representation layout tables from being read as implying
otherwise.
* Section 4.4.2: clarified that AAAA selects among the sixteen
entries of Table 1, and gave the reserved LITERAL+RESIDUAL
combination its explicit 5-bit encoding (10001) rather than
leaving it as a bare 4-bit function-type value inconsistent with
the 5-bit Function Address field defined in Section 4.4.
* Section 4.17: clarified that Residual Function Data is always
present when R = 1 (it is never itself optional), that it may be
zero-length only for a parameterless predefined function, and
added an explicit prohibition on a Residual Function Type of
LITERAL, consistent with Section 4.16's statement that a residual
is always a mathematical representation.
* Section 4.24 (Decoder State Machine): the LITERAL branch
previously omitted the padding-skip step shown in Section 4.18's
prose, and the non-LITERAL path had no explicit branch label.
Both have been added, and an END state was added after EVALUATE to
match the END state already present on the residual path.
* Section 4.30 (Summary of Binary Grammar): PREDEFINED_FUNCTION
previously expanded to an unconditional FUNCTION_PARAMETERS,
inconsistent with Section 4.9.1. It is now bracketed as optional.
RESIDUAL_FUNCTION_DATA was added as an explicit production
distinguishing CUSTOM_RESIDUAL_DATA from PREDEFINED_RESIDUAL_DATA,
since Section 4.17 defines both but the previous grammar only gave
a production for the custom case.
* Section 1.1 (Terminology): added a paragraph defining how the
structural label "OPTIONAL" is used inside binary layout diagrams
in Section 4, as distinct from the RFC 2119 keyword OPTIONAL, to
prevent the ambiguity that produced the defects above.
* Table 1 (Function Type Addresses): the LITERAL row previously
listed a formula, "F(x) = x", which is incorrect: Section 4.18
states that LITERAL stores no function, domain, or base, and
Decode(G) for LITERAL is defined in Section 4.29 as the raw
payload, not a function evaluation. The row has been corrected to
read "N/A -- raw bytes, see Section 4.18".
* Section 3: removed a duplicated statement of the global
optimization objective (arg min over C) that repeated, almost
verbatim, the implementation-level objective (arg min over C_I)
given immediately before it, and clarified in prose why the two
objectives differ (C_I is what an implementation can search; C is
the theoretical optimum it is not required to find).
Author's Address Author's Address
Srijal Dutta Srijal Dutta
The ZSMC Co. The ZSMC Co.
Gurugram Gurugram
Haryana Haryana
India India
Email: srijal@thezsmc.co Email: srijal@thezsmc.co
 End of changes. 61 change blocks. 
169 lines changed or deleted 453 lines changed or added

This html diff was produced by rfcdiff 1.49. The latest version is available from https://github.com/ietf-tools/rfcdiff