Description
Time series data streams (TSDS) index documents based on their @timestamp and not the ingestion time because their value depends on this @timestamp.
A user can configure lifecycle actions such as downsampling or searchable snapshots that convert indices to read-only. As a consequence, documents with @timestamps that fall within these indices will be rejected. Effectively, a TSDS has a write window within which it should be able to accept documents that fall within this window.
This is not fully supported yet, because new TSDS cannot only go back till index.look_back_time, we are working on improving this, but before we do that we need a way to retrieve the write window of a TSDS regardless if the backing indices exist or not.
Implementation notes:
To define the write-window we need to:
- Determine which lifecycle feature is currently managing this data stream (ILM or data stream lifecycle).
- Determine how does each feature influence the write window, for example, which read-only actions in ILM.
3. Expose it via the DataStream or something more appropriate.
Description
Time series data streams (TSDS) index documents based on their
@timestampand not the ingestion time because their value depends on this@timestamp.A user can configure lifecycle actions such as downsampling or searchable snapshots that convert indices to read-only. As a consequence, documents with
@timestamps that fall within these indices will be rejected. Effectively, a TSDS has a write window within which it should be able to accept documents that fall within this window.This is not fully supported yet, because new TSDS cannot only go back till
index.look_back_time, we are working on improving this, but before we do that we need a way to retrieve the write window of a TSDS regardless if the backing indices exist or not.Implementation notes:
To define the write-window we need to:
3. Expose it via theDataStreamor something more appropriate.