Update semantic text to use BFLOAT16 by default by Mikep86 · Pull Request #144236 · elastic/elasticsearch

Mikep86 · 2026-03-13T16:57:55Z

Updates the semantic_text field to use the BFLOAT16 element type by default for all inference services that use FLOAT.

This change minorly impacts knn query scoring due to the reduced precision of BFLOAT16. This won't matter for 99% of users, but since this may be a problem for some, this PR also adds a way to override the element type used so that these users can continue using FLOAT. An element_type parameter has been added to the dense vector index options:

PUT semantic-embeddings
{
  "mappings": {
    "properties": {
      "content": {
        "type": "semantic_text",
        "inference_id": "my-inference-endpoint",
        "index_options": {
          "dense_vector": {
            "element_type": "float",
            "type": "bbq_disk"
          }
        }
      }
    }
  }
}

Since element_type is not part of standard dense vector index options (and it doesn't make sense to add it there, since element_type is a top-level param for dense_vector), this param was added by extending dense vector index options only for semantic_text. This creates some new scenarios for us to handle:

Allowing the user to set only the element_type without setting type
Requiring type only when params other than element_type are set
Throwing an error when the element_type set in index_options is incompatible with the model element type
Serializing element_type when include_defaults is true and we defaulted to BFLOAT16

…lement type, dimensions, and similarity match what is expected

elasticsearchmachine · 2026-03-13T16:58:24Z

Hi @Mikep86, I've created a changelog YAML for you.

Mikep86 · 2026-03-13T17:00:09Z

@elasticmachine update branch

Mikep86 · 2026-03-27T21:19:29Z

I found a couple bugs while working on this:

Cannot get semantic_text field mapping with defaults when inference service does not exist - Fixed this for 9.4 in this PR, since I was already modifying the code that would need to change anyways. It should be easy to apply the necessary changes to earlier branches via a manual backport.

Unhandled edge cases when setting semantic_text index options - I left the failure modes the same so we can address this in a separate PR.

Mikep86 · 2026-03-27T21:22:41Z

+                    if (includeDefaults || isConfigured()) {
+                        if (value == null) {
+                            // Default value, serialize resolved defaults
+                            MinimalServiceSettings resolvedModelSettings = getResolvedModelSettings(null, false);


This is the fix for #145136. getResolvedModelSettings must return null when the inference service can't be found.

Mikep86 · 2026-03-27T21:26:03Z

-            if (resolved != null && settings.canMergeWith(resolved) == false) {
-                throw new IllegalArgumentException(
-                    "Mismatch between provided and registered inference model settings. "
-                        + "Provided: ["
-                        + settings
-                        + "], Expected: ["
-                        + resolved
-                        + "]."
+                        + modelSettings.taskType().name()
                );


@jimczi I removed this because this check was ineffective. It was only ever called when both settings and resolved the same instance, set via modelSettings.get().

Mikep86 · 2026-03-27T21:27:45Z

+    public void testGetDefaultIndexOptionsBeforeInferenceServiceExists() throws Exception {
+        final String inferenceId = randomIdentifier();
+        final String inferenceFieldName = "inference_field";
+
+        // Create the index before the inference endpoint exists. Default index options cannot be determined yet.
+        assertAcked(safeGet(prepareCreate(INDEX_NAME).setMapping(generateMapping(inferenceFieldName, inferenceId, null)).execute()));
+        Map<String, Object> actualFieldMappings = getFieldMappings(inferenceFieldName, true);
+
+        Map<String, Object> inferenceFieldMappings = XContentMapValues.nodeMapValue(
+            actualFieldMappings.get(inferenceFieldName),
+            inferenceFieldName
+        );
+        assertThat(inferenceFieldMappings.containsKey("index_options"), is(true));
+        assertThat(inferenceFieldMappings.get("index_options"), nullValue());


This is the test that validates the fix for #145136

elasticsearchmachine · 2026-03-27T21:31:20Z

Pinging @elastic/es-search-relevance (Team:Search Relevance)

elasticsearchmachine · 2026-03-27T21:31:20Z

Pinging @elastic/es-search-foundations (Team:Search Foundations)

…e often

kkharbas

LGTM! Except one nit comment

Updates the semantic_text field to use the BFLOAT16 element type by default for all inference services that use FLOAT. This change minorly impacts knn query scoring due to the reduced precision of BFLOAT16. This won't matter for 99% of users, but since this may be a problem for some, this PR also adds a way to override the element type used so that these users can continue using FLOAT. An element_type parameter has been added to the dense vector index options.

Mikep86 added 18 commits March 10, 2026 09:58

Default to bfloat16 when the model uses float element type

44e36ed

Merge branch 'main' into semantic-text_default-to-bfloat16

4c19af9

Merge branch 'main' into semantic-text_default-to-bfloat16

76dbe26

Added ExtendedDenseVectorIndexOptions

5021169

Implement equals and hashCode

636b893

Apply element type set in ExtendedDenseVectorIndexOptions

832f840

Merge branch 'main' into semantic-text_default-to-bfloat16

c0acfe2

Updated SemanticTextFieldMapperTests to check that the dense vector e…

977f885

…lement type, dimensions, and similarity match what is expected

Serialize index options when bfloat16 is used by default

651d69d

Don't require type param

d8f1675

Fix SemanticTextIndexOptionsIT test

f4e8684

Require type field when specifying more than element type

ca10bb7

Add cluster feature

e3d6e5f

Fix YAML test

cf1f22a

Fix YAML tests

e067d7a

Test setting element type to BYTE when using default BBQ index options

c629bf0

Validate element type override

60af7a1

Update integration tests

a7ac594

Mikep86 requested review from a team, jimczi and tballison March 13, 2026 16:57

Mikep86 added >enhancement :Search Foundations/Mapping Index mappings, including merging and defining field types :Search Relevance/Vectors Vector search v9.4.0 labels Mar 13, 2026

Update docs/changelog/144236.yaml

59f2ce9

Update changelog

c8aea83

Merge branch 'main' into semantic-text_default-to-bfloat16

5f40519

Always expect inner index options to be non-null

a286aab

Mikep86 commented Mar 27, 2026

View reviewed changes

Mikep86 marked this pull request as ready for review March 27, 2026 21:30

elasticsearchmachine added Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch labels Mar 27, 2026

benwtrent approved these changes Mar 30, 2026

View reviewed changes

Increase randomized iteration count and use current index version mor…

ad955e5

…e often

kkharbas reviewed Mar 30, 2026

View reviewed changes

Comment thread ...nference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java

kkharbas approved these changes Mar 30, 2026

View reviewed changes

kkharbas and others added 5 commits March 30, 2026 14:02

Merge branch 'main' into semantic-text_default-to-bfloat16

d4154c6

Merge branch 'main' into semantic-text_default-to-bfloat16

1e4f512

Added element type update tests

db4d575

Adjusted & added default index option tests

1786b75

Merge branch 'main' into semantic-text_default-to-bfloat16

f34cd34

Mikep86 enabled auto-merge (squash) April 1, 2026 15:02

Fix highlighter tests

3e5c919

Mikep86 merged commit dbc9098 into elastic:main Apr 1, 2026
31 of 35 checks passed

This was referenced Apr 3, 2026

Fix semantic_text inference ID update tests that default to BFLOAT16 #145679

Merged

Document semantic_text field defaulting to bfloat16 #145697

Merged

Cannot get semantic_text field mapping with defaults when inference service does not exist #145136

Open

Mikep86 mentioned this pull request Jul 6, 2026

Inconsistent unit-length vectors when using semantic_text with some inference providers #153028

Open

Navigation Menu

Repository navigation

Update semantic text to use BFLOAT16 by default - #144236

Update semantic text to use BFLOAT16 by default#144236
Mikep86 merged 49 commits into
elastic:mainfrom
Mikep86:semantic-text_default-to-bfloat16

Mikep86 commented Mar 13, 2026 •

edited

Loading

Uh oh!

elasticsearchmachine commented Mar 13, 2026

Uh oh!

Mikep86 commented Mar 13, 2026

Uh oh!

Mikep86 commented Mar 27, 2026

Uh oh!

Mikep86 Mar 27, 2026

Uh oh!

Mikep86 Mar 27, 2026

Uh oh!

Mikep86 Mar 27, 2026

Uh oh!

elasticsearchmachine commented Mar 27, 2026

Uh oh!

elasticsearchmachine commented Mar 27, 2026

Uh oh!

Uh oh!

kkharbas left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

Mikep86 commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticsearchmachine commented Mar 13, 2026

Uh oh!

Mikep86 commented Mar 13, 2026

Uh oh!

Mikep86 commented Mar 27, 2026

Uh oh!

Mikep86 Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

Mikep86 Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

Mikep86 Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

elasticsearchmachine commented Mar 27, 2026

Uh oh!

elasticsearchmachine commented Mar 27, 2026

Uh oh!

Uh oh!

kkharbas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Mikep86 commented Mar 13, 2026 •

edited

Loading