Dual search box, vector and lexical, plus hybrid slider#4744
Dual search box, vector and lexical, plus hybrid slider#4744joelochlann wants to merge 4 commits into
Conversation
ellenmuller
left a comment
There was a problem hiding this comment.
Just the same question/comment three times, otherwise looks good from what I can see!
| // TODO: avoid this initial search (two API calls to init!) | ||
| const isAiSearch = !!$stateParams.useAISearch; | ||
| // AI/KNN search activates when an aiQuery is present. | ||
| const isAiSearch = !!($stateParams.aiQuery && $stateParams.aiQuery.trim()); |
There was a problem hiding this comment.
Why do we need to trim?
There was a problem hiding this comment.
I guess in case there were accidental leading or trailing spaces typed in by the user? But not sure if this is the right place to do that. Or if we do it here, why not for everything
|
|
||
| ctrl.loadRange = function(start, end) { | ||
| if ($stateParams.useAISearch) { | ||
| if ($stateParams.aiQuery && $stateParams.aiQuery.trim()) { |
| // Polling for new images is meaningless for AI search — results are | ||
| // ranked by vector similarity, not upload time. | ||
| if ($stateParams.useAISearch) { return; } | ||
| if ($stateParams.aiQuery && $stateParams.aiQuery.trim()) { return; } |
ellenmuller
left a comment
There was a problem hiding this comment.
This is a nice idea and I think we should have a conversation about it with the data scientists too. I'd want to think through the implications of filtering by lexical and then ranking by knn... One of the implications would be that we wouldn't be able to surface images without any metadata/description etc right?
|
Cool! Alternative pattern might be to introduce it as a field, e.g. 'liverpool looks-like:"city at dusk"' Feels much less discoverable for what is (I assume!) going to be on a hot UI path, though! |
+1. There are arguments for both pre-filtering and post-filtering, I think. Just hard to think of UX that will allow both… But given that we only have 200 results to post-filter, I think pre-filtering this PR proposes is a stronger use case? Maybe? Would love to chat. My head hurts trying to come up with UX/UI ideas given all the possibilities (add sorting and |
Nope. If AI search is empty, it's just a metadata search. If metadata search is empty, is just an AI search. |
It seems like the hybrid search perform that well in cases like:
I was curious whether manually separating out the specific bits ("Hamburg", "Liverpool") from generic ("in the rain", "at dusk") might improve matters.
To that end, I've added a new AI search mode, hidden behind a feature switch & UI toggle, that provides a dual search box:
This is currently intended to be used by us for evaluation purposes, not necessarily directly by normal Grid users.
The idea is that if, when we evaluate hybrid search, we find cases where it struggles regardless of the mix, we have another approach that we can manually experiment with to see if better results are possible in principle given current functionality.
If we confirm that a manual hybrid does work the best, we would need to decide whether to give that control directly to users, or perhaps have a lightweight model doing the splitting up of the query on behalf of users and allowing power users to modify the dual query manually if they wish.
I've also added a slider for hybrid search weight in the UI, to make our evaluation process a bit easier.
Dual search box
The dual searchbox is gated from users in two ways:
1. You must enable a feature switch to access the feature (in addition to the AI feature switch toggle)
Without this switch on, it looks just like the existing search.
2. You must toggle hybrid => manual
Slider
This is accessible, but hidden behind an unobtrusive toggle icon, regardless of whether the new feature switch is on or not
Other screenshots