Mid-stream trailers #313

Activity

1. mnot

   added

   semantics

   on Feb 5, 2020
2. mnot

   assigned

   royfielding

   on Feb 6, 2020
3. ioggstream commented on Feb 10, 2020

   

   ioggstream

   on Feb 10, 2020

   Contributor

   More actions

   Some preliminary considerations.

   Potential use cases for mid-stream trailers

   • Incremental checksums eg. MICE
   • Service management (eg. convey the ETA of the transfer, ...)
   • Sending signatures of partial body (in mid-stream) has some complexities respect to sending a signature of the whole message in the header/trailer section, and requires a security assessment.

   Combining

   • mid-stream trailers, when used for integrity can't be dropped without informing the user agent
   • They probably need to be defined as 1#field-item because mid-stream trailers will be sent multiple times (eg. one per chunk)
   • Whenever they are tied to the partial transmitted representation, they will eventually be unuseful when merged, unless they have some associative property

   Relationship with payload body

   • They will be especially useful when tied to a portion of the body
   • They will be significantly less useful when not tied to a portion of the body
4. wenbozhu commented on Feb 10, 2020

   

   wenbozhu

   on Feb 10, 2020

   More actions

   Potential use cases

   • mostly limited to long-lived streaming traffic, i.e. long-lived "uploading/downloading"
   • in-band signaling support such as load reporting, app-level flow-control, authn update etc

   Semantics

   • trailers are a special case of mid-stream metadata, e.g. only trailers are able to convey the status of the entire transaction or the entire resource (body)
   • our experience is that in-order delivery is useful and often assumed, since the underlying transport (for HTTP body) is in-order and reliable. Out-of-order metadata is likely a use case for connection-level metadata, IMO
   • with the above assumption on in-order delivery, you really only need one API for delivering in-stream or trailer metadata

   Multi-part

   • without in-stream metadata, you would have to use multipart (with all the overhead)
   • for structured data, the original content-type can be preserved, e.g. JSON stream; and for media data, the content-type may have to be application/octet-stream
5. davidben commented on Mar 17, 2020

   

   davidben

   on Mar 17, 2020

   More actions

   Copying over the questions from @vasilvv over in httpwg/http-extensions#986, as they are good ones:

   1. What does "semantic model of being equivalent to trailers" mean here?
   2. Would the proposed frames require a new form of flow control?
   3. Are the proposed frames supposed to be associated with a specific position in HTTP response body?
   4. How would this interact with caching?
6. royfielding commented on Jun 11, 2020

   

   royfielding

   on Jun 11, 2020

   Member

   More actions

   1. The semantic model of trailers is that they provide additional metadata that may (or may not) be useful to recipients but was not available prior to the initial body data sent. How the are useful and what they might contain would be defined by the field name. A recipient is free to drop the trailer metadata.
   2. A mid-stream trailer frame would have the same role in flow control as a HEADERS frame, which is none at the current time.
   3. The proposed frames would not have any specific position, but the values inside trailer fields might identify specific positions (on their own, regardless of framing).
   4. A cache would either drop the values or store them separately as trailer fields in the order received.
7. LPardue commented on Jun 11, 2020

   

   LPardue

   on Jun 11, 2020

   Contributor

   More actions

   A mid-stream trailer frame would have the same role in flow control as a HEADERS frame, which is none at the current time.

   HTTP/2's flow control only cares about DATA frames. But all frames are subject to TCP's flow control. In HTTP/3, all frames are sent on streams which is subject to QUIC's flow control.

   Responding to the original question, I don't know what a "new form of flow control" would look like and I don't see how one would improve things.
8. mnot commented on Jun 24, 2020

   

   mnot

   on Jun 24, 2020

   MemberAuthor

   More actions

   What's the minimal change that would need to happen to semantics to enable this in a future version of HTTP (or an extension to an existing one)?
9. MikeBishop commented on Jul 8, 2020

   

   MikeBishop

   on Jul 8, 2020

   Contributor

   More actions

   I think you would need to make "trailer part" less special, and simply say that a version of HTTP can define a way to carry additional field sections during or after the body. H1-H3 currently define a way to carry only one such block, but an extension could provide more.

   👍React with 👍1Reacted by Mark Nottingham
10. mnot

    added this to the pre-reorg milestone on Aug 20, 2020
11. royfielding

    added

    has-proposal

    on Aug 21, 2020
12. reschke

    closed this as completedin #439on Aug 24, 2020