Page MenuHomePhabricator
Create Task
Maniphest T178851

Use label & description index for fulltext search
Closed, ResolvedPublic
Actions

Description

Use label index and description index from T176903 to implement better fulltext search in Wikidata.

Details

SubjectRepoBranchLines +/-
Actually enable the query builder hookmediawiki/extensions/Wikibasemaster+11 -1
Add fulltext search support for Wikidata labelsmediawiki/extensions/Wikibasemaster+3 K -338
Extract link formatting function into separate classmediawiki/extensions/Wikibasemaster+128 -106
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 Resolved• Wikidata-bugsT77898 query taking 10s: TermSqlIndex::getMatchingIDs
 DeclinedNoneT141813 Add full-text search support to Query Service
 OpenNoneT46529 Wikidata search problems (tracking)
 ResolvedaudeT110648 [Bug] high-ranking items seemed to have dropped significantly in Special:Search results for wikidata
 ResolvedSmalyshevT119067 Adjust rescoring config for Wikidata to consider sitelink count
 ResolvedaudeT119066 Add sitelink count to search index for Wikidata
 ResolvedNoneT120217 [Task] Add CirrusSearch and Elastica as dependencies of Wikibase on Jenkins
 InvalidNoneT161334 Stemming for item suggestions, e.g. "the" vs. no "the" on Wikidata
 ResolvedSmalyshevT143706 Implement TermLookup based on Elastic
 ResolvedSmalyshevT78157 [Story] Use ElasticSearch for entity search on wikidata.org
 ResolvedLydia_PintscherT88535 [Task] Investigate preconditions & constraints for using Elastic directly
 DuplicateSmalyshevT117520 Index Wikidata labels, aliases and descriptions as separate fields in ElasticSearch
 ResolvedSmalyshevT125500 [Epic] Index Wikidata labels and descriptions as separate fields in ElasticSearch
 ResolvedSmalyshevT178851 Use label & description index for fulltext search
 ResolvedSmalyshevT176903 Index wikidata descriptions
 ResolvedSmalyshevT180169 Make list of languages where using stemmed analyzer for Wikibase is beneficial
 ResolvedSmalyshevT182271 Wikidata fulltext search should handle search syntax properly
 ResolveddcausseT182293 Tune wikidata fulltext search similarity parameters
 DuplicateNoneT117522 Create standard "completion suggestion" API for Search
 DeclinedNoneT120089 Add an internal completion or suggestions API to core SearchEngine
 DeclinedNoneT170392 Create gadget that enables the use of the elastic search backend for the entity selector
 ResolveddanielT170400 Define metrics for search result quality for the entity selector widget on wikidata.
 ResolvedLydia_PintscherT170405 Manually evaluate cirrus based entity search on test system
 OpenNoneT170547 Metrics to evaluate new search for item suggestor
 DeclinedNoneT170549 Provide A/B test for item suggestor
ResolvedSmalyshevT162292 Reindex wikidata to pick up labels/descriptions mappings
 ResolveddcausseT160926 Make noop script be able replace whole fields with nested subfields
 ResolveddcausseT166589 Update wikidata code to take advantage of nested fields noop script
 ResolvedSmalyshevT175199 Index certain statements for Wikidata items

Event Timeline

Smalyshev created this task.Oct 23 2017, 11:19 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 23 2017, 11:19 PM
Smalyshev claimed this task.Oct 23 2017, 11:19 PM
Smalyshev triaged this task as Medium priority.
Smalyshev added a parent task: T176903: Index wikidata descriptions.
Smalyshev moved this task from needs triage to Current work on the Discovery-Search board.
Smalyshev edited projects, added Discovery-Search (Current work); removed Discovery-Search.
Smalyshev moved this task from Incoming to not in use - please delete on the Discovery-Search (Current work) board.
Smalyshev added a project: Discovery-ARCHIVED.
Smalyshev updated the task description. (Show Details)Oct 23 2017, 11:21 PM
Smalyshev removed a parent task: T176903: Index wikidata descriptions.
Smalyshev added a subtask: T176903: Index wikidata descriptions.
Smalyshev mentioned this in T125500: [Epic] Index Wikidata labels and descriptions as separate fields in ElasticSearch.
Smalyshev added subscribers: dcausse, EBernhardson, daniel.
Smalyshev added a parent task: T125500: [Epic] Index Wikidata labels and descriptions as separate fields in ElasticSearch.Oct 23 2017, 11:23 PM
gerritbot subscribed.Oct 23 2017, 11:27 PM

Change 380895 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[mediawiki/extensions/Wikibase@master] [WIP] Add fulltext search support for Wikidata labels

https://gerrit.wikimedia.org/r/380895

gerritbot added a project: Patch-For-Review.Oct 23 2017, 11:27 PM
Smalyshev added a project: User-Smalyshev.Oct 25 2017, 7:38 PM
Smalyshev moved this task from Backlog to Doing on the User-Smalyshev board.
debt subscribed.Oct 25 2017, 8:54 PM
Smalyshev moved this task from not in use - please delete to Needs review on the Discovery-Search (Current work) board.Nov 16 2017, 1:25 AM
Smalyshev closed subtask T176903: Index wikidata descriptions as Resolved.Nov 29 2017, 12:46 AM
Smalyshev moved this task from Needs review to not in use - please delete on the Discovery-Search (Current work) board.Dec 5 2017, 6:13 PM
Smalyshev added a parent task: T110648: [Bug] high-ranking items seemed to have dropped significantly in Special:Search results for wikidata.Dec 13 2017, 12:53 AM
Lydia_Pintscher moved this task from incoming to monitoring on the Wikidata board.Dec 18 2017, 2:58 PM
Smalyshev moved this task from not in use - please delete to Needs review on the Discovery-Search (Current work) board.Dec 19 2017, 6:21 PM
Smalyshev moved this task from Needs review to not in use - please delete on the Discovery-Search (Current work) board.Jan 2 2018, 6:13 PM
Realworldobject subscribed.Jan 4 2018, 5:22 PM
Smalyshev moved this task from not in use - please delete to Needs review on the Discovery-Search (Current work) board.Jan 5 2018, 9:18 AM
debt closed subtask T182293: Tune wikidata fulltext search similarity parameters as Resolved.Jan 8 2018, 2:31 PM
Smalyshev moved this task from Doing to In review on the User-Smalyshev board.Jan 17 2018, 6:49 PM
Lydia_Pintscher moved this task from monitoring to consider for next sprint on the Wikidata board.Jan 26 2018, 12:51 AM
Cparle mentioned this in T187441: Investigate adding MediaInfo data to search index for File page.Feb 15 2018, 12:54 PM
Abit subscribed.Mar 6 2018, 2:58 AM
gerritbot added a comment.Mar 7 2018, 7:43 PM

Change 417011 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[mediawiki/extensions/Wikibase@master] Extract link formatting function into separate class

https://gerrit.wikimedia.org/r/417011

gerritbot added a comment.Mar 16 2018, 9:53 AM

Change 417011 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Extract link formatting function into separate class

https://gerrit.wikimedia.org/r/417011

ReleaseTaggerBot added a project: MW-1.31-release-notes (WMF-deploy-2018-03-20 (1.31.0-wmf.26)).Mar 16 2018, 10:00 AM
gerritbot added a comment.Mar 16 2018, 11:31 AM

Change 380895 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Add fulltext search support for Wikidata labels

https://gerrit.wikimedia.org/r/380895

Smalyshev moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Mar 16 2018, 8:01 PM
Smalyshev closed subtask T182271: Wikidata fulltext search should handle search syntax properly as Resolved.Mar 20 2018, 4:58 AM
gerritbot added a comment.Mar 20 2018, 9:12 PM

Change 420855 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[mediawiki/extensions/Wikibase@master] Actually enable the query builder hook

https://gerrit.wikimedia.org/r/420855

gerritbot added a comment.Mar 22 2018, 8:11 AM

Change 420855 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Actually enable the query builder hook

https://gerrit.wikimedia.org/r/420855

ReleaseTaggerBot edited projects, added MW-1.31-release-notes (WMF-deploy-2018-03-27 (1.31.0-wmf.27)); removed MW-1.31-release-notes (WMF-deploy-2018-03-20 (1.31.0-wmf.26)).Mar 22 2018, 9:00 AM
Smalyshev closed this task as Resolved.Apr 3 2018, 12:52 AM

This is now done.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits