Skip to content

201411PluggableFilters

jonescc edited this page Nov 20, 2014 · 4 revisions
Date 20 Nov 2011 Contacts Craig Jones
Status Design Release Future release
Resources Integrated Marine observing System Ticket # TBD
Source code TBD
Funding Integrated Marine observing System

Overview

Background

We use GeoNetwork to catalogue our available data collections. The metadata for each collection records the location of OGC (and other) services that can be used to subset/download the collection in the online resources section of the metadata.

We currently maintain all our metadata in a single instance of GeoNetwork, but we will be moving to an harvesting arrangement where we will have one GeoNetwork instance that harvests records of this nature from a number of providers (including ourselves).

We make all of the collections publicly available through a portal application which uses the published services for each collection to provide download access to each collection.

The services for the collections may be unstable and we'd like to be able to restrict access to collections for which downloads are available (according to the last check) or flag ones which aren't.

Proposal

We'd like to add support for plugging in custom filters into GeoNetwork.

The custom filtering support will

  • support filtering of search requests based on search request parameters supported by that filter
  • allow filter values associated with each metadata record to be included in the search response

We will use this custom filter support to plug in a resource availability filter which we will develop for our own use case, but which may be useful for others.

Technical Details:

Custom filters to implement the org.fao.geonet.kernel.search.filter.Filter interface which will require implementation of the following methods:

public List<String> getAcceptedFilters();

public boolean isFilteringRequired(Element params);

public org.apache.lucene.search.Filter getLuceneFilter(Element params);

public String getFilterValue(String uuid, String filterName);

GeoNetwork to be modified to:

  • Load all filters implementing org.fao.geonet.kernel.search.filter.Filter using spring plugin so that spring - dependency injection and configuration can be used to instantiate plugins. This will load all plugins - implementing this interface found on the classpath with all spring managed beans available for dependency injection
  • Ignore all search parameters in the request returned by getAcceptedFilters for each loaded filter (refer org.fao.kernel.search.LuceneSearcher computeQuery method)
  • For each filter returning true for isFilteringRequired(), call getFilter and apply the filter to the lucene query in addition to any other filters applied (refer org.fao.kernel.search.LuceneSearcher computeQuery method)
  • Include any filters configured for return in the search response (lucene-config) using getAcceptedFilters to identify the filter to use and getFilterValue to get the filter value (refer org.fao.kernel.search.LuceneSearche getMetadataFromIndex method)

Note that filter plugins need to be able to filter results quickly so as not to impact search performance adversely, for example by caching filtering information in memory or making use of indexes.

Resource Availability Filter

For example, we will then be able to implement a resource availability filter as follows:

Method Action
getAcceptedFilters return 'resourcesAvailable'
isFilteringRequired return true if params include 'resourcesAvailable' with a value of 'true'
getLuceneFilter return an instance of org.apache.lucene.search.Filter which will filter collections using resource availability information (refer Lucene in Action 6.4 Custom filters for guidance/example implementation - lucene uuid field in GeoNetwork is '_uuid')
getFilterValue return the availability of resources for the metadata record passed using resource availability information as a string

The implementation details of the resource availability filter are currently being fleshed out at:

https://github.com/aodn/internal_discussions/wiki/Geonetwork-Layer-Monitor

Proposal Type:

  • Type:
  • Module:

Voting History

  • Vote Proposed: TBA

Participants

  • All
Clone this wiki locally