Skip to content

Fix to tdigest implicit eval - #140690

Merged
not-napoleon merged 15 commits into
elastic:mainfrom
not-napoleon:fix-to-tdigest-implicit-eval
Jan 20, 2026
Merged

not-napoleon merged 15 commits into
elastic:mainfrom
not-napoleon:fix-to-tdigest-implicit-eval

Conversation

@not-napoleon

@not-napoleon not-napoleon commented Jan 14, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #140670

This addresses the linked bug, but it's not at all clear to me that the solution is correct. I'm pushing this up as a draft to discuss. If we agree that this is the right way to fix it, I can write some more tests for it and get the PR merged.

Also, it turns out we didn't have any TS mode CSV tests for histogram data. I've added a new CSV data source and one test for that case. We need to replicate all the other tests in TS mode too, I think.

@not-napoleon not-napoleon added >bug WIP :StorageEngine/ES|QL Timeseries / metrics / logsdb capabilities in ES|QL v9.4.0 labels Jan 14, 2026

@felixbarny felixbarny left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also test convert functions on non-histogram fields and also nested convert functions.


// -----------------------------------------

Core supported t-digest aggs on histogram field, without grouping (TS mode)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add more TS mode tests here, but this is kind of the minimum to fix this specific bug.

@not-napoleon
not-napoleon marked this pull request as ready for review January 15, 2026 21:30
* @return true iff the given predicate is true for all nodes
*/
public boolean allMatch(Predicate<? super T> predicate) {
return anyMatch(Predicate.not(predicate)) == false;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alex-spies I discussed this briefly with you, but tagging you here in case you want to see the implementation.

@not-napoleon

Copy link
Copy Markdown
Contributor Author

Okay, BWC tests are failing trying to load the histogram time series data, because the histogram metric type did not exist prior to 9.3. This probably needs some way to gate loading that data set for older versions; CsvTestsDataLoader#availableDataSetsForEs does something similar, might be able to use that but I need to look a little deeper at it. Also that method is a mess, and needs to be refactored badly. I'll try to kludge together something tomorrow to get BWC happy here, and we should plan to clean up this mess soon.

@JonasKunz JonasKunz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after applying Felix' suggestion.

@not-napoleon not-napoleon removed the WIP label Jan 16, 2026
@elasticsearchmachine

Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@elasticsearchmachine

Copy link
Copy Markdown
Collaborator

Hi @not-napoleon, I've created a changelog YAML for you.

@not-napoleon
not-napoleon merged commit e8f71f8 into elastic:main Jan 20, 2026
35 checks passed
spinscale pushed a commit to spinscale/elasticsearch that referenced this pull request Jan 21, 2026
Fixes elastic#140670

This addresses the linked bug, but it's not at all clear to me that the solution is correct. I'm pushing this up as a draft to discuss. If we agree that this is the right way to fix it, I can write some more tests for it and get the PR merged.

Also, it turns out we didn't have any TS mode CSV tests for histogram data. I've added a new CSV data source and one test for that case. We need to replicate all the other tests in TS mode too, I think.

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>bug :StorageEngine/ES|QL Timeseries / metrics / logsdb capabilities in ES|QL Team:StorageEngine v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Histogram to TDigest cast doesn't work in TS mode

4 participants