Skip to content

perf(search): drop schema text from the phrase test - #2559

Merged
mystic123 merged 1 commit into
mainfrom
search/phrase-without-schema-text
Oct 21, 2026
Merged

mystic123 merged 1 commit into
mainfrom
search/phrase-without-schema-text

Conversation

@mystic123

@mystic123 mystic123 commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Intent

Closes #2533. Follows #2547.

Phrase points are the first sort key on both backends. After #2547, an Operation's searchable_text holds each word once, at its first occurrence. The phrase test still gives schema text 1 point, or 2 for the whole phrase. So two query words that happen to sit next to each other there lift a row above rows with a better score.

Change

  • phrase_points reads the name, the title, the description and the Provider name. Schema text gives no phrase points. It still scores: weight 0.25 on Postgres, 1 on SQLite.
  • The Postgres candidates CTE no longer carries searchable_text. Nothing after the CTE reads it.

No change to indexed text, so INDEXER_REVISION stays at 1.

Measurements

Ship rule, set before the runs: on each backend, the replay MRR falls by no more than 0.005, and the #2453 eval MRR does not fall on Graph or GitHub. If either fails, this PR closes and #2533 keeps the phrase test as it is.

Universal Search replay, Postgres

coral-benchmarks tools/universal-search, frozen run 2026-10-20-graph-pg: 2,385 cases (one_writer 1,188, two_writers 1,197), 100 targets per Provider (graph, github, openai, datadog_v1, linear, notion). main 8d77893 against this branch at 1680137: fresh release builds on an M4 Pro, Postgres 17, each runtime with its own copy of the same database, re-indexed by its own build. Both replays ran at the same time, 4 cases at once each. 0 failed searches on both.

hit@1, hit@10 and MRR at limit 10; hit@50 at limit 50. main → branch.

Subset hit@1 hit@10 hit@50 MRR Top 10 hits gained / lost
one_writer 0.45 → 0.45 0.75 → 0.75 0.87 → 0.87 0.550 → 0.551 0 / 0
two_writers 0.30 → 0.30 0.59 → 0.59 0.73 → 0.73 0.389 → 0.389 0 / 0

MRR at limit 10, hit@10 in brackets. main → branch.

Provider one_writer two_writers
graph 0.207 → 0.207 (0.42 → 0.42) 0.103 → 0.103 (0.28 → 0.28)
linear 0.594 → 0.597 (0.80 → 0.80) 0.439 → 0.439 (0.64 → 0.64)
notion 0.793 → 0.793 (0.88 → 0.88) 0.561 → 0.561 (0.67 → 0.67)
github 0.596 → 0.596 (0.85 → 0.85) 0.494 → 0.494 (0.82 → 0.82)
openai 0.526 → 0.526 (0.77 → 0.77) 0.329 → 0.329 (0.51 → 0.51)
datadog_v1 0.586 → 0.586 (0.81 → 0.81) 0.404 → 0.404 (0.61 → 0.61)

One case of 2,385 moves: s0186-unnamed, "create customer", where linear.saveCustomer goes from rank 3 to rank 1 (a Graph Operation held rank 1 on main).

The replay can show little of this change. The phrase is the whole query, so a field gets phrase points only when it holds every query word in order. Replay queries have a median of 7 words, and 6% have 3 words or fewer; the #2453 eval queries have a median of 4, and 34% to 39% have 3 or fewer.

Search time in the same replays, limit 10:

main branch
Median 2.85 s 2.49 s
p95 4.47 s 3.97 s

The branch was faster in 1,897 of 2,385 paired cases (median difference −289 ms; Graph targets −396 ms). Other work shared the machine, but both replays ran at the same time against the same Postgres, so the two columns compare.

#2453 eval

Fresh release builds of main 8d77893 and of this branch, one run each. MRR at limit 10.

SQLite Postgres
GitHub, 31 queries 0.668 → 0.668 0.671 → 0.671
Microsoft Graph, 38 queries 0.355 → 0.377 0.458 → 0.480

The GitHub queries do not change rank. The Graph gain matches the run before the rebase.

Search time on one search

#2453 eval on Microsoft Graph, 38 queries, before the rebase: the #2547 head 4d9110b against the same head with this commit. There were three runs of each build per backend, in the order base, branch, branch, base, base, branch. Another benchmark ran on the machine at the same time (load average 9 to 34), so the times are higher than #2547's quiet 2.0 s and 0.6 s. The pairs compare.

One search, median of 3 #2547 branch
Postgres 2.41 s 1.95 s
SQLite 0.85 s 0.89 s

The branch was faster on Postgres in each of the three pairs (12.6, 20.3 and 31.8 s faster for all 38 queries). On SQLite there is no difference above the noise.

Validation

  • cargo test -p coral-search on SQLite and on Postgres, after the rebase onto main 8d77893: 119 passed on each.
  • cargo clippy -p coral-search --all-targets -- -D warnings: clean.
  • The new zorble quint row of the phrase test fails with schema text in the phrase test and passes without it.
  • Not run on SQLite: the Universal Search replay. The frozen run is Postgres only.

@mystic123
mystic123 added this pull request to stack #2560 October 20, 2026 12:49
Base automatically changed from search/index-each-word-once to main October 20, 2026 16:44
Schema text holds each word once, at its first occurrence, so two query words in order there are chance. Phrase points now come from the name, the title, the description and the Provider name. The Postgres candidates CTE no longer carries searchable_text, which nothing after it reads.
@mystic123
mystic123 force-pushed the search/phrase-without-schema-text branch from 060ebe2 to 1680137 Compare October 21, 2026 07:34
@mystic123
mystic123 marked this pull request as ready for review October 21, 2026 09:39

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Re-trigger cubic

@mystic123
mystic123 merged commit cfdbafa into main Oct 21, 2026
27 checks passed
@mystic123
mystic123 deleted the search/phrase-without-schema-text branch October 21, 2026 11:11
simonwhitaker pushed a commit that referenced this pull request Oct 22, 2026
🤖 I have created a release *beep* *boop*
---


## [1.2.0](v1.1.0...v1.2.0)
(2026-10-22)


### Features

* **activity:** record authored source locations for calls and outputs
([#2535](#2535))
([2d735ca](2d735ca))
* **activity:** show persistent Program source beside selected records
([#2569](#2569))
([f403819](f403819))
* add the Claude recipe and read gzipped OpenAPI documents
([#2513](#2513))
([07c27a2](07c27a2))
* **catalog:** share named TypeScript types across a Provider
([#2482](#2482))
([256e37d](256e37d))
* **desktop:** run the runtime on a Postgres database named in
desktop.json ([#2485](#2485))
([96465a0](96465a0))
* **functions:** type coral.providers, coral.workspace and
coral.functions at Saved Function save
([#2526](#2526))
([a465850](a465850))
* **provider:** create the Coral Built-in Provider at coral.workspace in
every new Workspace
([#2349](#2349))
([06e65ca](06e65ca))
* **provider:** store an agent's createProvider request on its Setup
([#2593](#2593))
([92847d2](92847d2))
* **provider:** update every Workspace's Built-in Provider catalog at
boot ([#2355](#2355))
([24615d9](24615d9))
* **reef:** add NextDNS provider recipe
([#2509](#2509))
([e44880e](e44880e))
* **reef:** calmer providers table and a flush app shell
([#2462](#2462))
([ffe279c](ffe279c))
* **reef:** improve the UI on small screens
([#2531](#2531))
([75e020a](75e020a))
* **reef:** make the providers rail vertical tabs with an All providers
entry ([#2552](#2552))
([08d007a](08d007a))
* **reef:** providers page UI tweaks
([#2562](#2562))
([7ca218e](7ca218e))
* **reef:** put the list pages' main action back in the top bar as a
primary button ([#2546](#2546))
([761ea6d](761ea6d))
* **reef:** read ^, $ and * in every filter field
([#2520](#2520))
([d322356](d322356))
* **reef:** refuse a provider display name another provider uses
([#2518](#2518))
([fdcd473](fdcd473))
* **reef:** review provider configuration changes in a dialog
([#2579](#2579))
([24417e9](24417e9))
* **reef:** show search indexing failures on a Provider and its
Operations ([#2504](#2504))
([6cffed0](6cffed0))
* **reef:** show your recent activity summary on the workspace overview
([#2574](#2574))
([0d1dba5](0d1dba5))
* **reef:** title list pages in their toolbar, with the main action
beside it ([#2525](#2525))
([fd44b72](fd44b72))
* **reef:** UI tweaks to provider's page
([#2566](#2566))
([62204ca](62204ca))
* **reef:** unify Activity detail panels and action reviews
([#2544](#2544))
([409d062](409d062))
* return only declared successes from Operation calls
([#2517](#2517))
([0270bf1](0270bf1))
* **run:** enforce the Fan-out Limit inside the Run Owner
([#2571](#2571))
([cb53419](cb53419))
* **run:** introduce the Run Owner for Program requests
([#2524](#2524))
([245d19d](245d19d))
* **run:** let coral.workspace.createProvider configure and publish the
Provider itself
([#2352](#2352))
([088faaf](088faaf))
* **run:** let createProvider configure sign-in and link to Reef's
sign-in step ([#2353](#2353))
([8d5495c](8d5495c))
* **run:** list a workspace's providers with
coral.workspace.listProviders
([#2374](#2374))
([179c43a](179c43a))
* **run:** make the Run Owner survive uncertain commits and lost leases
([#2541](#2541))
([3a1fad1](3a1fad1))
* **run:** name Tasks and Runs with four-word identifiers
([#2558](#2558))
([c82259a](c82259a))
* **run:** prepare a Provider from a Program with
coral.workspace.createProvider
([#2351](#2351))
([e7e56fd](e7e56fd))
* **run:** reshape the Dispatch Fence into one batched check per chunk
commit ([#2550](#2550))
([120c9d3](120c9d3))
* **run:** route approval decisions and cancellation through the Run
Owner ([#2577](#2577))
([77bcbf5](77bcbf5))
* **run:** summarize your recent activity in a Workspace
([#2573](#2573))
([bf4c75e](bf4c75e))
* **search:** show a Saved Function's result fields in its signature
line ([#2522](#2522))
([41365f6](41365f6))
* **search:** trace each reconcile pass and its steps
([#2456](#2456))
([0594094](0594094))
* **ui:** add option to create a correctly-configured Slack app during
Slack provider setup
([#2585](#2585))
([8c3a883](8c3a883))
* **ui:** move Add provider to the providers toolbar
([#2591](#2591))
([e1ee4cb](e1ee4cb))


### Bug Fixes

* **app:** sweep device authorization leases every 15 s
([#2452](#2452))
([1a4e4ae](1a4e4ae))
* **auth:** drop the unverified client name note from the approval page
([#2578](#2578))
([ddf5aa4](ddf5aa4))
* **catalog:** derive each Built-in Operation's contract from the types
Coral parses ([#2582](#2582))
([534e636](534e636))
* **catalog:** import OpenAPI Providers whose Operations request
different OAuth scopes
([#2401](#2401))
([3acfe55](3acfe55))
* **catalog:** keep keyword names in Operation Call Paths
([#2506](#2506))
([b20795b](b20795b))
* **catalog:** never make an Operation unavailable for search material
([#2465](#2465))
([f83c909](f83c909))
* **catalog:** stop rejecting createProvider request headers by name
([#2589](#2589))
([39e54b9](39e54b9))
* **provider:** ask for an OAuth client ID when Coral cannot register
one ([#2580](#2580))
([baefe9e](baefe9e))
* **provider:** bind each Setup to its starter and hide Coral from
Members ([#2480](#2480))
([6011a64](6011a64))
* **reef:** align the catalog table with the first rail entry
([#2484](#2484))
([fb59758](fb59758))
* **reef:** calm the Activity badge and keep it while the stream
reconnects ([#2468](#2468))
([1b66639](1b66639))
* **reef:** frame the providers table again and lower the top bar
([#2472](#2472))
([87e41ee](87e41ee))
* **reef:** pin third-party recipe OpenAPI documents to a commit
([#2510](#2510))
([b4e9280](b4e9280))
* **reef:** replace loose help text with info tips and banners
([#2461](#2461))
([695500a](695500a))
* **reef:** round the page's corners beside the sidebar
([#2503](#2503))
([a1fef64](a1fef64))
* **reef:** shorten OAuth scope chips that share a URL prefix
([#2455](#2455))
([5ae67ec](5ae67ec))
* **reef:** suggest a provider's namespace from its document name, not
its host ([#2565](#2565))
([8d77893](8d77893))
* **run:** name every argument to correct in an invalid_input failure
([#2354](#2354))
([07bd727](07bd727))
* **run:** reject a credential beside createProvider's bearer method and
report unreachable sources as upstream_incomplete
([#2542](#2542))
([590eab7](590eab7))
* **run:** report an MCP JSON-RPC rejection as invalid_input or
upstream_rejected
([#2402](#2402))
([749770b](749770b))
* **search:** load each Generation's schema graph once per search
([#2448](#2448))
([445d945](445d945))
* **search:** retry a failed Saved Function only after one interval
([#2460](#2460))
([a61ab3a](a61ab3a))
* **search:** return no hits for a query without a letter or digit
([#2561](#2561))
([bb0a4f0](bb0a4f0))
* **search:** start the write limit at the lock and build rows before
the write ([#2575](#2575))
([06ad55d](06ad55d))
* **search:** stop a self-referential Promise result from overflowing
the stack ([#2543](#2543))
([1d4f257](1d4f257))
* **search:** stop retrying an index write that fails the same way every
time ([#2548](#2548))
([e3b6d15](e3b6d15))


### Performance Improvements

* **catalog:** dedup credential destinations once per build, not per
Operation ([#2555](#2555))
([0afbcf5](0afbcf5))
* **catalog:** project each named TypeScript type once per Provider
([#2443](#2443))
([153c273](153c273))
* **search:** drop schema text from the phrase test
([#2559](#2559))
([cfdbafa](cfdbafa))
* **search:** index each word of schema text once
([#2547](#2547))
([d1123fb](d1123fb))
* **search:** prepare search entries in the indexer after commit
([#2467](#2467))
([1ab6765](1ab6765))
* **search:** recount only Workspaces whose rows changed
([#2457](#2457))
([777cfb3](777cfb3))
* **search:** skip unchanged rows when an index write runs again
([#2459](#2459))
([43b5188](43b5188))
* **search:** write only the statistics rows that changed
([#2458](#2458))
([8e2832c](8e2832c))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: coral-release-bot[bot] <269256696+coral-release-bot[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <release-please[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Decide phrase points in schema text after each word is indexed once

1 participant