fix(api): treat OpenSearch's flat_object as an alias for flattened - #86

Reviewed for v1.0.0-rc.9. This is the only one of #86–#89 with no blocker — the alias is faithful to flattened at every site it touches and introduces no regression. I'm still holding it out of rc.9, for a reason that's about completeness rather than safety, and I'd rather say why than merge it quietly.

It only half-fixes the case it exists for

The commit names OpenSearch's sample dashboards/UBI data as the motivation. OpenSearch Dashboards discovers index-pattern fields through _field_caps, not GET _mapping — and the alias doesn't reach _field_caps.

Only GET /{index}/_mapping round-trips it, because get_mapping (es_compat.rs:1846-1859) returns the stored JSON verbatim. The other three introspection surfaces derive the type from the native schema, not the stored mapping:

• field_caps → native_type_to_es_str(&field.field_type) (:17070)
• global_field_caps → same (:25448)
• get_mapping_field → schema_to_es_properties (:25511, :25526-25529)

es_type_to_native sends both flattened and flat_object through the _ => FieldType::Object catch-all (:12995), and native_type_to_es_str maps FieldType::Object => "object" (:17207). collect_multi_fields only emits fields/properties children, never the root, so nothing overrides it.

Net effect: index creation now succeeds where it previously 400'd — a real improvement — but the field surfaces to the dashboard as {"type": "object", "searchable": true, "aggregatable": true}. The failure mode moved from a loud mapper_parsing_exception at create time to a silently mistyped field at visualisation time, which is arguably harder to debug.

To be fair: this is identical pre-existing behaviour for ES flattened, so it's not a regression this PR introduces. It's the reason the PR doesn't yet accomplish what it sets out to.

No test exercises any of the new arms

The diff is 17 insertions in one file and zero test changes; grep -rl flat_object engine/tests/ returns nothing. The repo has several flattened fixtures — search/340_flattened.yml, 600_flattened_ignore_above.yml, aggregations/terms_flattened_field.yml, indices.create/20_synthetic_source.yml and more — and none has a flat_object counterpart. So all six new match arms (:1037, :4429, :9117, :9476, :9513, :11734) execute zero times in CI. Green checks here aren't weak evidence, they're no evidence.

Cloning two of those fixtures with type: flat_object would cover the synthetic-source collapse, the fields-expansion roots and the ignore_above pruning in one commit — and would also have surfaced the _field_caps gap above.

Smaller things

• The shared validator was widened beyond the mapping path, so runtime: {x: {type: flat_object}} now returns 200.
• The commit message describes highlighting handling that isn't in the code.
• One of the six changed sites is unreachable.

Suggestion

Add a FieldType::Flattened (or map both names to a shared native type that native_type_to_es_str can render back correctly), so _field_caps reports the field as flattened/flat_object rather than object — that fixes the alias and the pre-existing flattened gap in one go — plus the two cloned fixtures. With those, this is a clean merge for rc.10.

Holding it out of rc.9 only because that release is already merged, tagged-pending and fully verified, and adding an untested change at the end for a partial fix means re-running the whole gate without gaining the thing the PR is for.

Thanks — the _field_caps gap was the right thing to hold this on. Fixed the main issue plus two of the smaller ones; being upfront about the two I couldn't close.

• _field_caps reports object instead of the real type: added declared_flattened_type, which consults the stored mapping (already available for the multi-fields case _field_caps handles) and overrides the generic object string with the real declared flattened/flat_object — applied to field_caps, global_field_caps, and GET /{index}/_mapping/field/{field}. Scoped as a per-field override, not a wholesale swap of the properties source, so a dynamically-inferred field with no stored-mapping entry still falls back to the schema-derived type exactly as before. Verified live: curl .../_field_caps?fields=attrs now returns {"attrs":{"flat_object":{...}}} instead of {"attrs":{"object":{...}}}.
• Runtime-field validator leak: confirmed — is_supported_field_type is shared between the mapping path and validate_runtime_fields, so widening it for flat_object let it through for runtime fields too, where real ES doesn't allow it. Added an explicit exclusion scoped to validate_runtime_fields only.
• Test coverage: added a YAML fixture cloned from 340_flattened.yml/600_flattened_ignore_above.yml covering exists-query, fields-option/synthetic-source, and ignore_above pruning with type: flat_object — ran it against a live built instance rather than just adding it untested (caught one broken case in the process: a field_caps: action, which turns out the project's YAML runner doesn't implement as a verb at all — dropped from the fixture, covered instead with a proper Rust end-to-end test that goes through the real HTTP route, including one that fails without the fix and passes with it).

Two things I did not resolve, flagging rather than guessing:

• The commit message on the original PR does describe highlighting handling that isn't in the diff — didn't rewrite the pushed/reviewed commit over it, noting it here instead.
• I couldn't conclusively identify which of the six original match arms you flagged as unreachable — traced each one I could match to current line numbers and each looked live to me, but I didn't have a precise enough reference into a ~28k-line function to be confident. Happy to take another pass if you can point at the specific one.

Full suite green: cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test -p xerj-engine -p xerj-query -p xerj-api --lib (only the 2 pre-existing, unrelated snapshot_path_security_tests failures).

🤖 Generated with Claude Code

Housekeeping, not a review comment.

We've just added a Contributor License Agreement to the project (#93). Until now contributions here were covered only by Apache-2.0 §5's inbound=outbound clause, which gives no explicit patent grant — worth tightening for a project this size.

Once #93 merges, a cla-bot check will appear on this PR and it will be red until you're covered. Signing is one small pull request, once per contributor — not per PR:

1. Read CLA.md
2. Open a PR adding your GitHub username to .contributors
3. Comment @cla-bot check back here and it turns green

That PR is the signature — it comes from your own account, so the commit history is the record.

To be explicit about something: we did not add anyone to the signed list on their behalf, including you. That file asserts a person has signed, and that's not ours to assert for someone else — hence the ask rather than a quiet edit.

Sorry for the extra step on work that's already in flight. Thanks for the contributions.