Repository navigation
Trailers #16
Description
7230 says:
The trailer fields are identical to header fields, except they are sent in a chunked trailer instead of the message's header section.
and:
When a chunked message containing a non-empty trailer is received, the recipient MAY process the fields (aside from those forbidden above) as if they were appended to the message's header section.
As discussed at the 2016 HTTP Workshop as part of @annevk's Fetch presentation, this isn't necessarily sensible.
Activity
mnot commentedon Aug 6, 2017
I wonder whether it would be best to add something like:
... with an appropriate column in the header field registry to record this. I don't think this would break any existing implementations, because trailers are so seldom used right now, and I think there's a strong argument that this is the case anyway; the consuming application has to expect something in trailers to act upon it.
annevk commentedon Aug 7, 2017
Can we differentiate between what is processed and what is exposed to APIs? I think we should expose the headers to APIs still and let consumers of those make their own choice.
mnot commentedon Aug 7, 2017
Yep, I was thinking that too. It's really hard for a generic implementation to know the semantics of individual headers. For ones that the implementation does look at, however (e.g., caching headers), it should pay attention.
This is really just clarifying current behaviour, I think.
royfielding commentedon Sep 25, 2017
I don't think that is helpful. First, nobody reads header field definitions. Second, servers will not differentiate the processing of extension fields according to their extension semantics, since they don't know the semantics.
In any case, we can't assume the client is a browser and the server is a website; that's not the scope of HTTP. When a use case is more limited, an API, media type, or specific application of HTTP may choose to exclude or otherwise narrow the use of trailers. This doesn't prevent them from being used in other contexts.
mnot commentedon Feb 8, 2018
Roy, what we're trying to avoid is the assumption that headers and trailers are semantically equivalent (which 7230 heavily implies), mostly for the benefit of senders. The requirements proposed above aren't intended to be targeted at generic HTTP software.
How about:
reschke commentedon Oct 10, 2018
WFM.
royfielding commentedon Oct 10, 2018
The phrasing is important and shouldn't diverge from our existing forms.
annevk commentedon Oct 11, 2018
What is a recipient?
In browsers at least there's the low-level HTTP library that's the initial recipient, then there's something akin to what Fetch describes, which is what most features would use, and then there's various APIs such as
<img>orfetch()which is what web applications would end up using.Currently
fetch()is described in such a way as to expose trailers as a separate field onResponseobjects, assuming none of them get discarded at a lower layer. Nobody has thus far implemented that so we will likely drop it, but I'd still like to know what the architecture should be if it ever came back.carl-mastrangelo commentedon Oct 11, 2018
This is really a shame, and was one of the features I was hoping to use. I can't speak for what the standard should say, but my own usage (gRPC) would treat trailers as a key-value list not treated specially by the browser (or any other intermediaries). That means they only have meaning (from the browser POV) to the JS applications, not anything else.
royfielding commentedon Oct 11, 2018
A recipient is the role associated with anyone/anything receiving the message (as opposed to sending a message). Which API is involved cannot matter for a messaging protocol. However, the reason I narrow the scope to the convoluted "that processes a trailer field for conversion to a corresponding header field" is because this requirement only matters for code that takes a trailer field and merges it with a set of header fields. It would not apply to code that keeps headers and trailers as separate tables.
13 remaining items
annevk commentedon Nov 30, 2018
Unless I'm missing something that still leaves the issue unaddressed. Are implementations allowed to maintain trailers separately from headers? (Note that per @mcmanus's comment they already do.) Can something "downstream" from HTTP require that?
mnot commentedon Dec 2, 2018
I think implementations are free to consider them separate, but they have to understand that somewhere in the chain, an intermediary may have collapsed trailers into headers -- so if something depends on a particular field showing up in headers, they're going to be disappointed.
annevk commentedon Dec 4, 2018
Hmm, I guess as long as that's true it's better to not support them in clients at all, to prevent that kind of thing from messing with application security.
mnot commentedon Dec 7, 2018
How do you get there, Anne? How would this "mess with application security"?
annevk commentedon Dec 7, 2018
Right, for what I'm worried about it doesn't really matter if browsers implement it or not, it's mostly intermediaries that might do such combining naïvely. E.g., if they put a
Content-Typein a trailer block in the header block. Still, browsers not supporting them might make it more likely that this feature goes away.mnot commentedon Dec 7, 2018
If someone designs a field that has to be in the trailer, and is somehow insecure if it's promoted into a header by an intermediary, they're going to have a bad time (because intermediaries are allowed to fold all trailers into headers).
More generally, if someone designs a field that is allowed to be in a trailer and they depend on it being available there end-to-end, they're going to have a bad time.
TEmight give them some help here, though (see #18).If someone puts a header that doesn't explicitly say it's allowed in trailers into trailers, they're going to have a bad time (because if it arrives at the other end, someone may not be looking for it there -- i.e., trailers are not always automatically folded into headers).
AIUI Firefox already supports
Server-Timingin trailers, and that works well for its use case. It doesn't have any of the bad interactions above.There are other specific cases where a browser could make a decision about whether or not it supports trailers for a header that it processes -- e.g.,
ETagis asked for once in a while (so that a server can compute an identifier based upon the body without buffering it).I think that browsers can also expose trailers in an API safely, provided that they're not automatically collapsed into headers. That said, the constraints above still apply to the people using them -- and we definitely need to do a better job documenting them.
annevk commentedon Dec 8, 2018
Thanks, I think given this the API is rather tricky to expose, as you cannot depend on there not being intermediaries in the wild, so your application might end up failing for a small number of users without that being very visible, but I'll follow-up in #18.
Clarify when trailers can be generated
royfielding commentedon Aug 7, 2019
Going back to the core issue of trailers overriding headers, we should first decide whether we continue to recommend/allow trailer fields to be merged with headers. If not, we should actually specify that! None of the changes in #219 are necessary if merging is not allowed.
annevk commentedon Aug 8, 2019
It'd be amazing if we could agree that merging is not allowed and they're essentially a separate member of the response to be processed independently. I think that would increase robustness of the whole setup quite a bit.
mnot commentedon Sep 2, 2019
I think we'd still want to say:
... and probably add that older implementations might still merge them, so recipients SHOULD disregard headers that are only defined to appear in trailers.
As to whether we should do it -- I doubt that merging is widely implemented in pure intermediaries, since it would require buffering the message. However, I wonder whether it is in proxy/caches; I'll do some testing.