Update ElasticSearch and Elastica to 6.x#1880
Merged
Merged
Conversation
9f05a24 to
1e9114f
Compare
1e9114f to
e6a7473
Compare
7bdf937 to
0842eb9
Compare
jraddaoui
requested changes
Oct 29, 2024
Update ElasticSearch to 6.8.23 in the Dockerfile and update Elastica to 6.x in composer.
- 'inline' scripts are deprecated, changed to 'source'
Update arElasticSearchPlugin to use multiple indices instead of multiple types since ES 6.x removed being able to add multiple types. Also update mapping to remove include_in_all and add that to the _all field using copy_to instead since include_in_all was also removed in ES 6.x
Update autocompleteAction to use multiple indices
Add the appropriate index types for ES requests that did not need to specify explicit types in ES 5.x
Add missing document type to partialUpdate in arElasticSearchPlugin
Update condition in arElasticSearchPluginUtil to only get string fields included in _all if copy_to is explicitly set to _all in the mappping.
abb1768 to
0259ed0
Compare
Renamed arElasticSearchIndexDecorator as arElasticSearchMultiIndexWrapper for clarity and added a few comments about changes needed for ElasticSearch/Elastica 7.x.
Rename getType from arElasticSearchMultiIndexWrapper to getIndex for clarity.
bbf4bb1 to
0e0d87d
Compare
a37039a to
4b52458
Compare
Add a dummy ElasticSearch index type since one is still required for ES 6.x, instead of having a seperate type name for each of the multiple indices.
4b52458 to
bc54543
Compare
jraddaoui
reviewed
Nov 20, 2024
Update any references to ElasticSearch index type to index name. Also add getType as an alias for getIndex for backwards compatibility with custom themes that still might be making references to ElasticSearch's getType.
Remove the index prefix from arElasticSearchMultiIndexWrapper and use AtoM class names internally instead of using prefix to refer to them.
jraddaoui
reviewed
Nov 22, 2024
jraddaoui
reviewed
Nov 22, 2024
jraddaoui
reviewed
Nov 22, 2024
Fix bug in search autocompleteAction which was showing duplicate results for actors under repositories. Also address some of the CR feedback and update/add methods for add, update, and deleteById in the arElasticSearchMultiIndexWrapper.
Remove unnecessary foreach loop in arElasticSearchPlugin. Also move code for adding filter for stripping md tags into a seperate method to make the initialization code more readable.
c5bbe4a to
ef14555
Compare
jraddaoui
reviewed
Nov 25, 2024
jraddaoui
reviewed
Nov 25, 2024
Update the index refresh method to accept an optional name param that only refreshes that particular index in arElasticSearchMultiIndexWrapper. Also address some small CR feedback.
a94554a to
02b3c4b
Compare
Check setting and update config global variable there.
jraddaoui
approved these changes
Dec 4, 2024
miceno
pushed a commit
to miceno/atom
that referenced
this pull request
Feb 18, 2026
* Update ElasticSearch and Elastica dependencies Update ElasticSearch to 6.8.23 in the Dockerfile and update Elastica to 6.x in composer. * Remove arElasticSearchIndexDecorator. - 'inline' scripts are deprecated, changed to 'source' * Update index for ES 6.x Update arElasticSearchPlugin to use multiple indices instead of multiple types since ES 6.x removed being able to add multiple types. Also update mapping to remove include_in_all and add that to the _all field using copy_to instead since include_in_all was also removed in ES 6.x * Fix autocompleteAction for ES 6.x Update autocompleteAction to use multiple indices * Add index types for ES Queries without a type Add the appropriate index types for ES requests that did not need to specify explicit types in ES 5.x * Fix updateByQuery calls for ElasticSearch 6.x * Fix broken publication status update Add missing document type to partialUpdate in arElasticSearchPlugin * Fix linting errors * Switch elasticsearch to the oss docker image * Update mapping condition for _all fields Update condition in arElasticSearchPluginUtil to only get string fields included in _all if copy_to is explicitly set to _all in the mappping. * Update ES mapping to copy relevant fields to _all * Rename arElasticSearchIndexDecorator Renamed arElasticSearchIndexDecorator as arElasticSearchMultiIndexWrapper for clarity and added a few comments about changes needed for ElasticSearch/Elastica 7.x. * Rename getType function from ESMultiIndexWrapper Rename getType from arElasticSearchMultiIndexWrapper to getIndex for clarity. * Add dummy type for ES index Add a dummy ElasticSearch index type since one is still required for ES 6.x, instead of having a seperate type name for each of the multiple indices. * Change references to index type for ElasticSearch Update any references to ElasticSearch index type to index name. Also add getType as an alias for getIndex for backwards compatibility with custom themes that still might be making references to ElasticSearch's getType. * Fix copy to _all fields for ES mapping Removed copy_to _all from all fields that previously had include_in_all set to false. * Remove unnecessary ElasticSearch type variables Remove unnecessary $type ElasticSearch variables in QubitLftSyncer, arElasticSearchPlugin, and updatePublicationStatusTask * Remove ES index prefix from MultiIndexWrapper Remove the index prefix from arElasticSearchMultiIndexWrapper and use AtoM class names internally instead of using prefix to refer to them. * Fix autocompleteAction bug, address CR feedback Fix bug in search autocompleteAction which was showing duplicate results for actors under repositories. Also address some of the CR feedback and update/add methods for add, update, and deleteById in the arElasticSearchMultiIndexWrapper. * Remove non text fields from _all ES mapping * Remove unnecessary loop in arElasticSearchPlugin Remove unnecessary foreach loop in arElasticSearchPlugin. Also move code for adding filter for stripping md tags into a seperate method to make the initialization code more readable. * Update ES index refresh to only refresh one index Update the index refresh method to accept an optional name param that only refreshes that particular index in arElasticSearchMultiIndexWrapper. Also address some small CR feedback. * Refactor ES initialize and populate methods (WIP) * Tweak index recreation * Initialize on populate * Tweak flush/refresh operation * Only ignore 404 errors when deleting indexes * Prevent flush if current index name is not set * Make loadDiacriticsMappings a private function * Improve populate logging * Configure filters once before indices creation * Normalize loadDiacriticsMappings function Check setting and update config global variable there. * Create indices on update if they don't exist * Trust recreate option on index creation * Remove unused function --------- Co-authored-by: melaniekung <mkung@artefactual.com> Co-authored-by: José Raddaoui Marín <raddaouimarin@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Update ElasticSearch and Elastica and address issues introduced by breaking changes in ElasticSearch 6.x.
The most notable changes are:
include_in_allmapping parameter is disallowed and the_allfield is disabled by default.Changes in Elastica: