Repository navigation
Implement PromQL set operator: top-level or (UNION) - #151486
Merged
Merged
Conversation
Translates a top-level PromQL `or` chain into a UnionAll + TopNBy plan. Left-branch precedence is enforced via a synthetic _branch ordering column. and/unless remain unsupported and are rejected by the verifier.
Collaborator
|
Hi @felixbarny, I've created a changelog YAML for you. |
# Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/promql/TranslatePromqlToEsqlPlan.java
…omql-set-operator
Contributor
🔍 Preview links for changed docs |
Contributor
ℹ️ Important: Docs version tagging👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version. We use applies_to tags to mark version-specific features and changes. Expand for a quick overviewWhen to use applies_to tags:✅ At the page level to indicate which products/deployments the content applies to (mandatory) What NOT to do:❌ Don't remove or replace information that applies to an older version 🤔 Need help?
|
The new set_operator_union_* CSV-spec cases omitted the promql_command_v0 capability, so GenerativeForkIT did not skip them and failed with "FORK after subquery is not supported".
# Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java
Distinguish the three set-operator failures with specific messages: scalar operands are illegal in PromQL itself (align wording with Prometheus), while unsupported and/unless and non-top-level or remain current implementation limitations. Also reuse VectorBinarySet#output dedup logic for the union branch output in translation.
Explain why top-level `or` is special-cased and where `and`/`unless` will translate. Replace the redundant branch-limit throw with an assert, since PromqlCommand.verify already enforces it.
Collaborator
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
Top-level `or` between instant vectors is now supported; only `and`/`unless` and non-top-level/scalar `or` remain unsupported.
Scalar/instant-vector operand rules are standard PromQL behavior, not Elasticsearch-specific; only the top-level restriction remains.
sidosera
approved these changes
Jun 18, 2026
Contributor
|
|
szybia
added a commit
to szybia/elasticsearch
that referenced
this pull request
Jun 18, 2026
* upstream/main: (83 commits) [Inference API] Refactor Llama service settings to use ObjectParser (elastic#151225) Implement PromQL set operator: top-level `or` (UNION) (elastic#151486) Mute org.elasticsearch.gradle.internal.precommit.PomValidationPrecommitPluginFuncTest detects missing POM elements and reports structured problems elastic#151636 Fix native library setup for JMH benchmark forks (elastic#149866) [Test] Fix SearchCommitPrefetcherIT.testSearchNodePrefetchesOnlyLatestGenerationOnFirstCommitNotification (elastic#151424) Refactor bulk byte vector implementations to not split by parts (elastic#151518) Add missing release notes for bundled JDK 26 and 26.0.1 upgrades (elastic#151616) ES|QL: Load flattened mapped-root from _source (elastic#151415) [ML-CPS] Default project_routing to _alias:_origin on first-time UIAM datafeed migration (elastic#146651) Clarify default index type for byte and bit vectors (elastic#151557) Fix issue with CacheRegion and PinnedWindowEvictionPolicyTests (elastic#151620) [ML] Fix DatafeedConfigTests mutation no-op (elastic#151484) Mute org.elasticsearch.xpack.stateless.cluster.coordination.BlobStoreSyncDirectoryTests testCloseCancelsOnGoingUploads elastic#151622 ESQL:DS: Add in all dataset-supported params (elastic#151551) Make an expensive views test faster by reducing index count (elastic#151555) Mute org.elasticsearch.xpack.stateless.StatelessMergePreWarmingIT testMergePreWarmingFailureDoesNotFailTheEngine elastic#151618 Add timestamp to CacheFileRegion (elastic#151240) Both PanamaESVectorUtilSupport and DefaultESVectorUtilSupport computed the inverse norm as a float before multiplying each byte element: [slices] Add Yaml tests (elastic#151435) Disable BCC slow upload hot threads logging (elastic#151612) ...
Kubik42
pushed a commit
to Kubik42/elasticsearch
that referenced
this pull request
Jun 18, 2026
Translates a top-level PromQL `or` chain into a UnionAll + TopNBy plan. Left-branch precedence is enforced via a synthetic _branch ordering column. and/unless remain unsupported and are rejected by the verifier.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the PromQL
or(UNION) set operator for top-level instant-vector expressions.oroperators (e.g.a or b or c) is flattened into a singleUnionAll+TopNByplan. The synthetic_branchordering column ensures left-branch precedence: a series from the left operand shadows any matching series from the right, exactly matching PromQL semantics.VectorBinarySet.output()now returns the union of both operands' label sets (left first), and thedropMetricNameflag isfalsefor set operators so__name__is preserved.orbetween instant vectors and rejectsand/unless(still unsupported), nested unions, scalar operands, and chains exceedingFork.MAX_BRANCHES(8 operands).PROMQL_SET_OPERATOR_UNIONcapability gate added.applyNullOutputFilternow locates the value column by name rather than by position, which was incorrect in the multi-branch path where the value column appears at the end of each branch's output before projection.New tests:
PromqlVerifierTests: verifier acceptance/rejection coverage for the new supported and still-unsupported shapes.PromqlPlanSetOperatorTests: golden-plan tests asserting theUnionAll+TopNBystructure.PrometheusSetOperatorRestIT: end-to-end REST tests via the native Prometheus query endpoints.vector(0)fallback,rate or irateidiom, heterogeneous groupings, and a three-branch chain.