Apache Unomi 1.X - Documentation
Apache Unomi 1.X - Documentation
X -
DOCUMENTATION
Apache Software Foundation
TABLE OF CONTENTS
1. Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2. Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3. Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4. Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.3.1. ActionType . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3.2. ConditionType . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3.3. Persona . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3.4. PropertyMergeStrategyType. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3.5. PropertyType . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3.6. Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3.7. Scoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3.8. Segments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3.9. Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3.10. ValueType . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.4.1. UserList . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.4.2. Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.4.3. Campaign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3. Quick start . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1. Building . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1.2. Building . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2.1. Prerequisites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.3. Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.4. Important ! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4. Sample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.1. Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.2.1. Warning ! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.3.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.3.3. Retrieving context information from Unomi using the context servlet . . . . . . . . . . . . . . . . . . 24
4.4. Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.4.2. Javascript . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.6. Annex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5. Connectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.1. Connectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
6. Cluster setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
7. Custom extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
8. Consent API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
These properties are further defined by the Item’s type definition which explicits the Item’s structure
and semantics. By defining new types, users specify which properties (including the type of values they
accept) are available to items of that specific type.
Unomi defines default value types: date, email, integer and string, all pretty self-explanatory. While you
can think of these value types as "primitive" types, it is possible to extend Unomi by providing additional
value types.
Additionally, most items are also associated to a scope, which is a concept that Unomi uses to group
together related items. A given scope is represented in Unomi by a simple string identifier and usually
represents an application or set of applications from which Unomi gathers data, depending on the
desired analysis granularity. In the context of web sites, a scope could, for example, represent a site or
family of related sites being analyzed. Scopes allow clients accessing the context server to filter data to
only see relevant data.
{
"itemType": <type of the item>,
"scope": <scope>,
"itemId": <item identifier>,
"properties": <optional properties>
}
Some types can be dynamically defined at runtime by calling to the REST API while other extensions are
done via Unomi plugins. Part of extending Unomi, therefore, is a matter of defining new types and
specifying which kind of Unomi entity (e.g. profiles) they can be affected to. For example, the following
JSON document can be passed to Unomi to declare a new property type identified (and named) tweetNb,
tagged with the social tag, targeting profiles and using the integer value type.
{
"itemId": "tweetNb",
"itemType": "propertyType",
"metadata": {
"id": "tweetNb",
"name": "tweetNb",
"systemTags": ["social"]
},
"target": "profiles",
"type": "integer"
}
Unomi defines a built-in scope (called systemscope) that clients can use to
share data across scopes.
1.2. EVENTS
Users' actions are conveyed from clients to the context server using events. Of course, the required
information depends on what is collected and users' interactions with the observed systems but events
minimally provide a type, a scope and source and target items. Additionally, events are timestamped.
Conceptually, an event can be seen as a sentence, the event’s type being the verb, the source the subject
and the target the object.
Event structure:
{
"eventType": <type of the event>,
"scope": <scope of the event>,
"source": <Item>,
"target": <Item>,
"properties": <optional properties>
}
Source and target can be any Unomi item but are not limited to them. In particular, as long as they can
be described using properties and Unomi’s type mechanism and can be processed either natively or via
extension plugins, source and target can represent just about anything. Events can also be triggered as
part of Unomi’s internal processes for example when a rule is triggered.
Events are sent to Unomi from client applications using the JSON format and a typical page view event
from a web site could look something like the following:
1.3. PROFILES
By processing events, Unomi progressively builds a picture of who the user is and how they behave. This
knowledge is embedded in Profile object. A profile is an Item with any number of properties and
optional segments and scores. Unomi provides default properties to cover common data (name, last
name, age, email, etc.) as well as default segments to categorize users. Unomi users are, however, free
and even encouraged to create additional properties and segments to better suit their needs.
Contrary to other Unomi items, profiles are not part of a scope since we want to be able to track the
associated user across applications. For this reason, data collected for a given profile in a specific scope
is still available to any scoped item that accesses the profile information.
It is interesting to note that there is not necessarily a one to one mapping between users and profiles as
users can be captured across applications and different observation contexts. As identifying information
might not be available in all contexts in which data is collected, resolving profiles to a single physical
user can become complex because physical users are not observed directly. Rather, their portrait is
progressively patched together and made clearer as Unomi captures more and more traces of their
actions. Unomi will merge related profiles as soon as collected data permits positive association between
distinct profiles, usually as a result of the user performing some identifying action in a context where
the user hadn’t already been positively identified.
Blueprint is used to declare what the plugin provides and inject any required dependency. The Blueprint
file is located, as usual, at OSGI-INF/blueprint/blueprint.xml in the bundle JAR file.
The plugin otherwise follows a regular maven project layout and should depend on the Unomi API
maven artifact:
Some plugins consists only of JSON definitions that are used to instantiate the appropriate structures at
runtime while some more involved plugins provide code that extends Unomi in deeper ways.
In both cases, plugins can provide more that one type of extension. For example, a plugin could provide
both `ActionType`s and `ConditionType`s.
2.3.1. ACTIONTYPE
ActionType`s define new actions that can be used as consequences of Rules being triggered. When a rule
triggers, it creates new actions based on the event data and the rule internal processes, providing values
for parameters defined in the associated `ActionType. Example actions include: “Set user property x to
value y” or “Send a message to service x”.
2.3.2. CONDITIONTYPE
`ConditionType`s define new conditions that can be applied to items (for example to decide whether a
rule needs to be triggered or if a profile is considered as taking part in a campaign) or to perform
queries against the stored Unomi data. They may be implemented in Java when attempting to define a
particularly complex test or one that can better be optimized by coding it. They may also be defined as
combination of other conditions. A simple condition could be: “User is male”, while a more generic
condition with parameters may test whether a given property has a specific value: “User property x has
value y”.
2.3.3. PERSONA
A persona is a "virtual" profile used to represent categories of profiles, and may also be used to test how
a personalized experience would look like using this virtual profile. A persona can define predefined
properties and sessions. Persona definition make it possible to “emulate” a certain type of profile, e.g :
US visitor, non-US visitor, etc.
2.3.4. PROPERTYMERGESTRATEGYTYPE
2.3.5. PROPERTYTYPE
Definition for a profile or session property, specifying how possible values are constrained, if the value
is multi-valued (a vector of values as opposed to a scalar value). `PropertyType`s can also be
categorized using systemTags or file system structure, using sub-directories to organize definition files.
`Rule`s are conditional sets of actions to be executed in response to incoming events. Triggering of rules
is guarded by a condition: the rule is only triggered if the associated condition is satisfied. That condition
can test the event itself, but also the profile or the session. Once a rule triggers, a list of actions can be
performed as consequences. Also, when rules trigger, a specific event is raised so that other parts of
Unomi can react accordingly.
2.3.7. SCORING
`Scoring`s are set of conditions associated with a value to assign to profiles when matching so that the
associated users can be scored along that dimension. Each scoring element is evaluated and matching
profiles' scores are incremented with the associated value.
2.3.8. SEGMENTS
`Segment`s represent dynamically evaluated groups of similar profiles in order to categorize the
associated users. To be considered part of a given segment, users must satisfies the segment’s condition.
If they match, users are automatically added to the segment. Similarly, if at any given point during, they
cease to satisfy the segment’s condition, they are automatically removed from it.
2.3.9. TAG
`Tag`s are simple labels that are used to classify all other objects inside Unomi.
2.3.10. VALUETYPE
2.4.1. USERLIST
User list are simple static lists of users. The associated profile stores the lists it belongs to in a specific
property.
2.4.2. GOAL
Goals represent tracked activities / actions that can be accomplished by site (or more precisely scope)
visitors. These are tracked in general because they relate to specific business objectives or are relevant
to measure site/scope performance.
Goals can be defined at the scope level or in the context of a particular Campaign. Either types of goals
behave exactly the same way with the exception of two notable differences: - duration: scope-level goals
are considered until removed while campaign-level goals are only considered for the campaign duration
- audience filtering: any visitor is considered for scope-level goals while campaign-level goals only
consider visitors who match the campaign’s conditions
A goal-oriented, time-limited marketing operation that needs to be evaluated for return on investment
performance by tracking the ratio of visits to conversions.
3. QUICK START
3.1. BUILDING
1) Install J2SE 8.0 SDK (or later), which can be downloaded from
http://www.oracle.com/technetwork/java/javase/downloads/index.html
2) Make sure that your JAVA_HOME environment variable is set to the newly installed JDK location, and
that your PATH includes %JAVA_HOME%\bin (windows) or $JAVA_HOME$/bin (unix).
3.1.2. BUILDING
1) Change to the top level directory of Apache Unomi source distribution. 2) Run
This will compile Apache Unomi and run all of the tests in the Apache Unomi source distribution.
Alternatively, you can run
This will compile Apache Unomi without running the tests and takes less time to build.
Starting with version 1.2, Apache Unomi no longer embeds an ElasticSearch server as this is no longer
supported by the developers of ElasticSearch. Therefore you will need to install a standalone
ElasticSearch using the following steps:
Download an ElasticSearch version. Here’s the version you will need depending on your version of
cluster.name: contextElasticSearch
Check that the ElasticSearch is up and running by accessing the following URL :
http://localhost:9200
The "package" sub-project generates a pre-configured Apache Karaf installation that is the simplest way
to get started. Simply uncompress the package/target/unomi-VERSION.tar.gz (for Linux or Mac OS X) or
package/target/unomi-VERSION.zip (for Windows) archive into the directory of your choice.
You can then start the server simply by using the command on UNIX/Linux/MacOS X :
./bin/karaf
or on Windows shell :
bin\karaf.bat
You will then need to launch (only on the first Karaf start) the Apache Unomi packages using the
following Apache Karaf shell command:
This is only needed if you didn’t use the generated package. Also, this is the preferred way to install a
development environment if you intend to re-deploy the context server KAR iteratively.
Before deploying, make sure that you have Apache Karaf properly installed. You will also have to
increase the default maximum memory size and perm gen size by adjusting the following environment
values in the bin/setenv(.bat) files (at the end of the file):
MY_DIRNAME=`dirname $0`
MY_KARAF_HOME=`cd "$MY_DIRNAME/.."; pwd`
export JAVA_MAX_MEM=3G
export JAVA_MAX_PERM_MEM=384M
Install the WAR support, CXF and Karaf Cellar into Karaf by doing the following in the Karaf command
line:
Create a new $MY_KARAF_HOME/etc/org.apache.cxf.osgi.cfg file and put the following property inside :
org.apache.cxf.servlet.context=/cxs
If all went smoothly, you should be able to access the context script here :
http://localhost:8181/cxs/cluster . You should be able to login with karaf / karaf and see basic server
information. If not something went wrong during the install.
You might need to select the JDK to run the tests in the itests subproject. In order to do so you can list the
installed JDKs with the following command :
/usr/libexec/java_home -V
java -version
The integration tests are not executed by default to make build time minimal, but it is recommended to
run the integration tests at least once before using the server to make sure that everything is ok in the
build. Another way to use these tests is to run them from a continuous integration server such as
Jenkins, Apache Gump, Atlassian Bamboo or others.
Performance tests are based on Gatling. You need to have a running context server or cluster of servers
before executing the tests.
export GATLING_CONF=<path>/performance-tests/src/test/resources
gatling.sh
http://localhost:8181/index.html
This test page will trigger the loading of the /context.js script, which will try to retrieving the user
context or create a new one if it doesn’t exist yet. It also contains an experimental integration with
Facebook Login, but it doesn’t yet save the context back to the context server.
Simply reference the context script in your HTML as in the following example:
<script type="text/javascript">
(function(){ var u=(("https:" ==== document.location.protocol) ? "https://localhost:8181/" :
"http://localhost:8181/");
var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
g.type='text/javascript'; g.defer=true; g.async=true; g.src=u+'context.js';
s.parentNode.insertBefore(g,s); })();
</script>
This document assumes that you are already familiar with Unomi’s concepts. On the technical side, we
also assume working knowledge of git to be able to retrieve the code for Unomi and the example.
Additionnally, you will require a working Java 7 or above install. Refer to
http://www.oracle.com/technetwork/java/javase/ for details on how to download and install Java SE 7 or
greater.
BUILDING UNOMI
START UNOMI
Start Unomi according to the instructions. Once you have Karaf running, you should wait until you see
the following messages on the Karaf console:
This indicates that all the Unomi services are started and ready to react to requests. You can then open a
browser and go to http://localhost:8181/cxs to see the list of available RESTful services or retrieve an
initial context at http://localhost:8181/context.json (which isn’t very useful at this point).
REQUEST EXAMPLES
curl http://localhost:8181/context.js?sessionId=1234
This will retrieve a JavaScript script that contains a cxs object that contains the context with the current
user profile, segments, scores as well as functions that makes it easier to perform further requests (such
as collecting events using the cxs.collectEvents() function).
If you prefer to retrieve a pure JSON object, you can simply use a request formed like this:
curl http://localhost:8181/context.json?sessionId=1234
By default, in order to optimize the amount of data sent over the network, Apache Unomi will not send
the content of the profile or session properties. If you need this data, you must send a JSON object to
configure the resulting output of the context.js(on) servlet.
Here is an example that will retrieve all the session and profile properties.
At the same time as you are retrieving the context, you can also directly send events in the
ContextRequest object as illustrated in the following example:
Upon received events, Apache Unomi will execute all the rules that match the current context, and
return an updated context. This way of sending events is usually used upon first loading of a page. If you
want to send events after the page has finished loading you could either do a second call and get an
updating context, or if you don’t need the context and want to send events in a network optimal way you
can use the eventcollector servlet (see below).
If you only need to send events without retrieving a context, you should use the eventcollector servlet
that is optimized respond quickly and minimize network traffic. Here is an example of using this servlet:
Note that the eventcollector executes the rules but does not return a context. If is generally used after a
page is loaded to send additional events.
• Read the Twitter samples documentation that contains a detailed example of how to integrate with
Apache Unomi.
3.3. CONFIGURATION
If you want to change the default configuration, you can perform any modification you want in the
$MY_KARAF_HOME/etc directory.
contextserver.publicAddress=https://localhost:9443
contextserver.internalAddress=http://127.0.0.1:8181
If you need to specify an Elasticsearch cluster name, or a host and port that are different than the
default, it is recommended to do this BEFORE you start the server for the first time, or you will loose all
the data you have stored previously.
$MY_KARAF_HOME/etc/org.apache.unomi.persistence.elasticsearch.cfg
cluster.name=contextElasticSearch
# The elasticSearchAddresses may be a comma seperated list of host names and ports such as
# hostA:9300,hostB:9300
# Note: the port number must be repeated for each host.
elasticSearchAddresses=localhost:9300
index.name=context
Unomi secures some events by default. You can find the default configuration in the following file
(created after the first server startup):
$MY_KARAF_HOME/etc/org.apache.unomi.thirdparty.cfg
Ususally, login events, which operate on profiles and do merge on protected properties, must be secured.
For each trusted third party server, you need to add these 3 lines :
thirdparty.provider1.key=secret-key
thirdparty.provider1.ipAddresses=127.0.0.1,::1
thirdparty.provider1.allowedEvents=login,updateProperties
The events set in allowedEvents will be secured and will only be accepted if the call comes from the
specified IP address, and if the secret-key is passed in the X-Unomi-Peer header.
The Context Server requires an IP database in order to resolve IP addresses to user location. The
GeoLite2 database can be downloaded from MaxMind here :
http://dev.maxmind.com/geoip/geoip2/geolite2/
In order to use it, you need to install the Geonames database into . Get the "allCountries.zip" database
from here : http://download.geonames.org/export/dump/
Download it and put it in the "etc" directory, without unzipping it. Edit
$MY_KARAF_HOME/etc/org.apache.unomi.geonames.cfg and set
request.geonamesDatabase.forceImport to true, import should start right away. Otherwise, import
should start at the next startup. Import runs in background, but can take about 15 minutes. At the end,
you should have about 4 million entries in the geonames index.
The Context Server REST API is protected using JAAS authentication and using Basic or Digest HTTP
auth. By default, the login/password for the REST API full administrative access is "karaf/karaf".
The generated package is also configured with a default SSL certificate. You can change it by following
these steps :
http://wiki.eclipse.org/Jetty/Howto/Configure_SSL
org.osgi.service.http.secure.enabled = true
org.ops4j.pax.web.ssl.keystore=${karaf.etc}/keystore
org.ops4j.pax.web.ssl.password=changeme
org.ops4j.pax.web.ssl.keypassword=changeme
org.osgi.service.http.port.secure=9443
You should now have SSL setup on Karaf with your certificate, and you can test it by trying to access it
on port 9443.
1. Changing the default Karaf password can be done by modifying the etc/users.properties file
The context server is capable of merging profiles based on a common property value. In order to use
this, you must add the MergeProfileOnPropertyAction to a rule (such as a login rule for example), and
configure it with the name of the property that will be used to identify the profiles to be merged. An
example could be the "email" property, meaning that if two (or more) profiles are found to have the
same value for the "email" property they will be merged by this action.
Upon merge, the old profiles are marked with a "mergedWith" property that will be used on next profile
access to delete the original profile and replace it with the merged profile (aka "master" profile). Once
this is done, all cookie tracking will use the merged profile.
To test, simply configure the action in the "login" or "facebookLogin" rules and set it up on the "email"
property. Upon sending one of the events, all matching profiles will be merged.
Before going live with a project, you should absolutely read the following section that will help you setup
a proper secure environment for running your context server.
You should setup a firewall around your cluster of context servers and/or Elasticsearch nodes. If you
have an application-level firewall you should only allow the following connections open to the whole
world :
• http://localhost:8181/eventcollector
For your Context Server client applications (such as the Jahia CMS), you will need to make the following
ports accessible :
The context server actually requires HTTP Basic Auth for access to the Context Server administration
REST API, so it is highly recommended that you design your client applications to use the HTTPS port for
accessing the REST API.
The user accounts to access the REST API are actually routed through Karaf’s JAAS support, which you
may find the documentation for here :
• http://karaf.apache.org/manual/latest/users-guide/security.html
karaf/karaf
You should really change this default username/password as soon as possible. To do so, simply modify
the following file :
$MY_KARAF_HOME/etc/users.properties
For your context servers, and for any standalone Elasticsearch nodes you will need to open the following
ports for proper node-to-node communication : 9200 (Elasticsearch REST API), 9300 (Elasticsearch TCP
transport)
Of course any ports listed here are the default ports configured in each server, you may adjust them if
needed.
• https://www.elastic.co/blog/found-elasticsearch-security
• https://www.elastic.co/blog/scripting-security
If you want to setup an Apache HTTP web server in from of Apache Unomi, here is an example
configuration using mod_proxy.
contextserver.publicAddress=https://unomi.apache.org/
contextserver.internalAddress=http://192.168.1.1:8181
and you will also need to change the contextserver.domain in the /etc/org.apache.unomi.web.cfg file
contextserver.domain=apache.org
<VirtualHost *:80>
Include /var/www/vhosts/unomi.apache.org/conf/common.conf
</VirtualHost>
<IfModule mod_ssl.c>
<VirtualHost *:443>
Include /var/www/vhosts/unomi.apache.org/conf/common.conf
SSLEngine on
SSLCertificateFile /var/www/vhosts/unomi.apache.org/conf/ssl/24d5b9691e96eafa.crt
SSLCertificateKeyFile /var/www/vhosts/unomi.apache.org/conf/ssl/apache.org.key
SSLCertificateChainFile /var/www/vhosts/unomi.apache.org/conf/ssl/gd_bundle-g2-g1.crt
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
<Directory /usr/lib/cgi-bin>
SSLOptions +StdEnvVars
</Directory>
BrowserMatch "MSIE [2-6]" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown
</VirtualHost>
</IfModule>
common.conf:
DocumentRoot /var/www/vhosts/unomi.apache.org/html
CustomLog /var/log/apache2/access-unomi.apache.org.log combined
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /var/www/vhosts/unomi.apache.org/html>
Options FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
</Directory>
<Location /cxs>
Order deny,allow
deny from all
allow from 88.198.26.2
allow from www.apache.org
</Location>
RewriteEngine On
RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK)
RewriteRule .* - [F]
ProxyPreserveHost On
ProxyPass /server-status !
ProxyPass /robots.txt !
When performing localhost requests to Apache Unomi, a default location will be used to insert values
into the session to make the location-based personalization still work. You can find the default location
settings in the file :
org.apache.unomi.plugins.request.cfg
You might want to change these for testing or for demonstration purposes.
The Apache Karaf SSH console is available inside Apache Unomi, but the port has been changed from the
default value of 8101 to 8102 to avoid conflicts with other Karaf-based products. So to connect to the SSH
console you should use:
or the user/password you have setup to protect the system if you have changed it.
It is now possible to use X-Pack to connect to ElasticSearch. However, for licensing reasons this is not
provided out of the box. Here is the procedure to install X-Pack with Apache Unomi:
3.4. IMPORTANT !
Do not start Unomi directly with unomi:start, perform the following steps below first !
2. Download https://artifacts.elastic.co/maven/org/elasticsearch/client/x-pack-transport/5.6.3/x-pack-
transport-5.6.3.jar to XPACK_JARS_DIRECTORY
3. Download https://artifacts.elastic.co/maven/org/elasticsearch/plugin/x-pack-api/5.6.3/x-pack-api-
5.6.3.jar to XPACK_JARS_DIRECTORY
4. Download http://central.maven.org/maven2/com/unboundid/unboundid-ldapsdk/3.2.0/unboundid-
ldapsdk-3.2.0.jar to XPACK_JARS_DIRECTORY
5. Download http://central.maven.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.55/bcpkix-jdk15on-
1.55.jar to XPACK_JARS_DIRECTORY
6. Download http://central.maven.org/maven2/org/bouncycastle/bcprov-jdk15on/1.55/bcprov-jdk15on-
1.55.jar to XPACK_JARS_DIRECTORY
transportClientClassName=org.elasticsearch.xpack.client.PreBuiltXPackTransportClient
transportClientJarDirectory=XPACK_JARS_DIRECTORY
transportClientProperties=xpack.security.user=elastic:changeme
You can setup more properties (for example for SSL/TLS support) by seperating the properties with
commas, as in the following example:
transportClientProperties=xpack.security.user=elastic:changeme,xpack.ssl.key=/home/user/elasticse
arch-5.6.3/config/x-pack/localhost/localhost.key,xpack.ssl.certificate=/home/user/elasticsearch-
5.6.3/config/x-pack/localhost/localhost.crt,xpack.ssl.certificate_authorities=/home/user/elasticsearch-
5.6.3/config/x-pack/ca/ca.crt,xpack.security.transport.ssl.enabled=true
Launch Karaf and launch unomi using the command from the shell :
unomi:start
Alternatively you could edit the configuration directly from the Karaf shell using the following
commands:
config:edit org.apache.unomi.persistence.elasticsearch
config:property-set transportClientClassName
org.elasticsearch.xpack.client.PreBuiltXPackTransportClient
config:property-set transportClientJarDirectory XPACK_JARS_DIRECTORY
config:property-set transportClientProperties xpack.security.user=elastic:changeme
config:update
unomi:start
You can setup more properties (for example for SSL/TLS support) by seperating the properties with
commas, as in the following example:
config:property-set transportClientProperties
xpack.security.user=elastic:changeme,xpack.ssl.key=/home/user/elasticsearch-5.6.3/config/x-
pack/localhost/localhost.key,xpack.ssl.certificate=/home/user/elasticsearch-5.6.3/config/x-
pack/localhost/localhost.crt,xpack.ssl.certificate_authorities=/home/user/elasticsearch-5.6.3/config/x-
pack/ca/ca.crt,xpack.security.transport.ssl.enabled=true
• Twitter integration
• Login integration
4.2.1. WARNING !
The example code uses client-side Javascript code to send the login event. This is only done this way for
the sake of samples simplicity but if should NEVER BE DONE THIS WAY in real cases.
The login event should always be sent from the server performing the actual login since it must only be
sent if the user has authenticated properly, and only the authentication server can validate this.
Login into the Unomi Karaf SSH shell using something like this :
bundle:install mvn:org.apache.unomi/login-integration-samples/${project.version}
when the bundle is successfully install you will get an bundle ID back we will call it BUNDLE_ID.
bundle:start BUNDLE_ID
If all went well you can access the login samples HTML page here :
http://localhost:8181/login/index.html
You can fill in the form to test it. Note that the hardcoded password is:
test1234
4.3.1. OVERVIEW
We will examine how a simple HTML page can interact with Unomi to enrich a user’s profile. The use
case we will follow is a rather simple one: we use a Twitter button to record the number of times the
visitor tweeted (as a tweetNb profile integer property) as well as the URLs they tweeted from (as a
tweetedFrom multi-valued string profile property). A javascript script will use the Twitter API to react to
clicks on this button and update the user profile using a ContextServlet request triggering a custom
event. This event will, in turn, trigger a Unomi action on the server implemented using a Unomi plugin,
a standard extension point for the server.
cd samples/tweet-button-plugin
mvn clean install
This will compile and create the OSGi bundle that can be deployed on Unomi to extend it.
In standard Karaf fashion, you will need to copy the samples bundle to your Karaf deploy directory.
If you are using the packaged version of Unomi (as opposed to deploying it to your own Karaf version),
you can simply run, assuming your current directory is samples/tweet-button-plugin and that you
uncompressed the archive in the directory it was created:
cp target/tweet-button-plugin-1.0.0-incubating-SNAPSHOT.jar ../../package/target/unomi-1.0.0-
incubating-SNAPSHOT/deploy
You can now go to http://localhost:8181/index.html to test the samples code. The page is very simple, you
will see a Twitter button, which, once clicked, will open a new window to tweet about the current page.
The original page should be updated with the new values of the properties coming from Unomi.
Additionnally, the raw JSON response is displayed.
We will now explain in greater details some concepts and see how the example works.
There are essentially two modalities to interact with the context server, reflecting different types of
Unomi users: context server clients and context server integrators.
Context server clients are usually web applications or content management systems. They interact
with Unomi by providing raw, uninterpreted contextual data in the form of events and associated
metadata. That contextual data is then processed by the context server to be fed to clients once
actionable. In that sense context server clients are both consumers and producers of contextual data.
Context server clients will mostly interact with Unomi using a single entry point called the
ContextServlet, requesting context for the current user and providing any triggered events along the
way.
On the other hand, context server integrators provide ways to feed more structured data to the context
server either to integrate with third party services or to provide analysis of the uninterpreted data
provided by context server clients. Such integration will mostly be done using Unomi’s API either
directly using Unomi plugins or via the provided REST APIs. However, access to REST APIs is restricted
due for security reasons, requiring privileged access to the Unomi server, making things a little more
complex to set up.
For simplicity’s sake, this document will focus solely on the first use case and will interact only with the
context servlet.
Unomi provides two ways to retrieve context: either as a pure JSON object containing strictly context
information or as a couple of JSON objects augmented with javascript functions that can be used to
interact with the Unomi server using the <context server base URL>/context.json or <context server base
URL>/context.js URLs, respectively.
Below is an example of asynchronously loading the initial context using the javascript version, assuming
a default Unomi install running on http://localhost:8181:
This initial context results in a javascript file providing some functions to interact with the context
server from javascript along with two objects: a cxs object containing information about the context for
the current user and a digitalData object that is injected into the browser’s window object (leveraging
the Customer Experience Digital Data Layer standard). Note that this last object is not under control of
the context server and clients are free to use it or not. Our example will not make use of it.
On the other hand, the cxs top level object contains interesting contextual information about the current
user:
{
"profileId":<identifier of the profile associated with the current user>,
"sessionId":<identifier of the current user session>,
"profileProperties":<requested profile properties, if any>,
"sessionProperties":<requested session properties, if any>,
"profileSegments":<segments the profile is part of if requested>,
"filteringResults":<result of the evaluation of personalization filters>,
"trackedConditions":<tracked conditions in the source page, if any>
}
We will look at the details of the context request and response later.
4.4. EXAMPLE
The code for the HTML page with our Tweet button can be found at
https://github.com/apache/incubator-unomi/blob/master/wab/src/main/webapp/index.html.
This HTML page is fairly straightforward: we create a tweet button using the Twitter API while a
Javascript script performs the actual logic.
4.4.2. JAVASCRIPT
Globally, the script loads both the twitter widget and the initial context asynchronously (as shown
previously). This is accomplished using fairly standard javascript code and we won’t look at it here.
Using the Twitter API, we react to the tweet event and call the Unomi server to update the user’s profile
with the required information, triggering a custom tweetEvent event. This is accomplished using a
contextRequest function which is an extended version of a classic AJAX request:
• If we specify a payload, it is expected to use the JSON format so we stringify it and encode it if
passed as a URL parameter in a GET request.
• We need to make a CORS request since the Unomi server is most likely not running on the same
host than the one from which the request originates. The specific details are fairly standard and we
will not explain them here.
• We need to either retrieve (from the initial context we retrieved previously using cxs.sessionId) or
generate a session identifier for our request since Unomi currently requires one.
• We don’t need to provide any authentication at all to interact with this part of Unomi since we only
have access to read-only data (as well as providing events as we shall see later on). If we had been
using the REST API, we would have needed to provide authentication information as well.
The interesting part, though, is the payload. This is where we provide Unomi with contextual
information as well as ask for data in return. This allows clients to specify which type of information
they are interested in getting from the context server as well as specify incoming events or content
filtering or property/segment overrides for personalization or impersonation. This conditions what the
context server will return with its response.
{
source: <Item source of the context request>,
events: <optional array of triggered events>,
requiredProfileProperties: <optional array of property identifiers>,
requiredSessionProperties: <optional array of property identifiers>,
filters: <optional array of filters to evaluate>,
profileOverrides: <optional profile containing segments,scores or profile properties to override>,
- segments: <optional array of segment identifiers>,
- profileProperties: <optional map of property name / value pairs>,
- scores: <optional map of score id / value pairs>
sessionPropertiesOverrides: <optional map of property name / value pairs>,
requireSegments: <boolean, whether to return the associated segments>
}
SOURCE
A context request payload needs to at least specify some information about the source of the request in
the form of an Item (meaning identifier, type and scope plus any additional properties we might have to
provide), via the source property of the payload. Of course the more information can be provided about
the source, the better.
A client wishing to perform content personalization might also specify filtering conditions to be
evaluated by the context server so that it can tell the client whether the content associated with the filter
should be activated for this profile/session. This is accomplished by providing a list of filter definitions to
be evaluated by the context server via the filters field of the payload. If provided, the evaluation results
will be provided in the filteringResults field of the resulting cxs object the context server will send.
OVERRIDES
It is also possible for clients wishing to perform user impersonation to specify properties or segments to
override the proper ones so as to emulate a specific profile, in which case the overridden value will
temporarily replace the proper values so that all rules will be evaluated with these values instead of the
proper ones. The segments (array of segment identifiers), profileProperties (maps of property name and
associated object value) and scores (maps of score id and value) all wrapped in a profileOverrides object
and the sessionPropertiesOverrides (maps of property name and associated object value) fields allow to
provide such information. Providing such overrides will, of course, impact content filtering results and
segments matching for this specific request.
The clients can also specify which information to include in the response by setting the requireSegments
property to true if segments the current profile matches should be returned or provide an array of
property identifiers for requiredProfileProperties or requiredSessionProperties fields to ask the context
server to return the values for the specified profile or session properties, respectively. This information
is provided by the profileProperties, sessionProperties and profileSegments fields of the context server
response.
Additionally, the context server will also returns any tracked conditions associated with the source of
the context request. Upon evaluating the incoming request, the context server will determine if there are
any rules marked with the trackedCondition tag and which source condition matches the source of the
incoming request and return these tracked conditions to the client. The client can use these tracked
conditions to learn that the context server can react to events matching the tracked condition and
coming from that source. This is, in particular, used to implement form mapping (a solution that allows
clients to update user profiles based on values provided when a form is submitted).
EVENTS
Finally, the client can specify any events triggered by the user actions, so that the context server can
process them, via the events field of the context request.
DEFAULT RESPONSE
If no payload is specified, the context server will simply return the minimal information deemed
necessary for client applications to properly function: profile identifier, session identifier and any
tracked conditions that might exist for the source of the request.
Now that we’ve seen the structure of the request and what we can expect from the context response,
let’s examine the request our component is doing.
In our case, our source item looks as follows: we specify a scope for our application (unomi-tweet-
button-samples), specify that the item type (i.e. the kind of element that is the source of our event) is a
page (which corresponds, as would be expected, to a web page), provide an identifier (in our case, a
Base-64 encoded version of the page’s URL) and finally, specify extra properties (here, simply a url
property corresponding to the page’s URL that will be used when we process our event in our Unomi
extension).
We also specify that we want the context server to return the values of the tweetNb and tweetedFrom
profile properties in its response. Finally, we provide a custom event of type tweetEvent with associated
scope and source information, which matches the source of our context request in this case.
var contextPayload = {
source: source,
events: [
{
eventType: 'tweetEvent',
scope: scope,
source: source
}
],
requiredProfileProperties: [
'tweetNb',
'tweetedFrom'
]
};
The tweetEvent event type is not defined by default in Unomi. This is where our Unomi plugin comes
into play since we need to tell Unomi how to react when it encounters such events.
In order to react to tweetEvent events, we will define a new Unomi rule since this is exactly what Unomi
rules are supposed to do. Rules are guarded by conditions and if these conditions match, the associated
set of actions will be executed. In our case, we want our new incrementTweetNumber rule to only react
RULE DEFINITION
{
"metadata": {
"id": "smp:incrementTweetNumber",
"name": "Increment tweet number",
"description": "Increments the number of times a user has tweeted after they click on a tweet
button"
},
"raiseEventOnlyOnceForSession": false,
"condition": {
"type": "eventTypeCondition",
"parameterValues": {
"eventTypeId": "tweetEvent"
}
},
"actions": [
{
"type": "incrementTweetNumberAction",
"parameterValues": {}
}
]
}
Rules define a metadata section where we specify the rule name, identifier and description.
When rules trigger, a specific event is raised so that other parts of Unomi can react to it accordingly. We
can control how that event should be raised. Here we specify that the event should be raised each time
the rule triggers and not only once per session by setting raiseEventOnlyOnceForSession to false, which
is not strictly required since that is the default. A similar setting (raiseEventOnlyOnceForProfile) can be
used to specify that the event should only be raised once per profile if needed.
We could also specify a priority for our rule in case it needs to be executed before other ones when
similar conditions match. This is accomplished using the priority property. We’re using the default
priority here since we don’t have other rules triggering on `tweetEvent`s and don’t need any special
ordering.
We then tell Unomi which condition should trigger the rule via the condition property. Here, we specify
that we want our rule to trigger on an eventTypeCondition condition. Unomi can be extended by adding
new condition types that can enrich how matching or querying is performed. The condition type
definition file specifies which parameters are expected for our condition to be complete. In our case, we
use the built-in event type condition that will match if Unomi receives an event of the type specified in
the condition’s eventTypeId parameter value: tweetEvent here.
ACTION DEFINITION
{
"id": "incrementTweetNumberAction",
"actionExecutor": "incrementTweetNumber",
"systemTags": [
"event"
],
"parameters": []
}
We specify the identifier for the action type, a list of systemTags if needed: here we say that our action is
a consequence of events using the event tag. Our actions does not require any parameters so we don’t
define any.
The action executor references the actual implementation of the action as defined in our blueprint
definition:
<blueprint xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v1.0.0
http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd">
In standard Blueprint fashion, we specify that we will need the profileService defined by Unomi and
then define a service of our own to be exported for Unomi to use. Our service specifies one property:
Our action executor definition specifies that the bean providing the service is implemented in the
org.apache.unomi.samples.tweet_button_plugin.actions .IncrementTweetNumberAction class. This class
implements the Unomi ActionExecutor interface which provides a single int execute(Action action,
Event event) method: the executor gets the action instance to execute along with the event that triggered
it, performs its work and returns an integer status corresponding to what happened as defined by public
constants of the EventService interface of Unomi: NO_CHANGE, SESSION_UPDATED or
PROFILE_UPDATED.
tweetNb = 0;
tweetedFrom = new ArrayList<>();
}
return EventService.PROFILE_UPDATED;
It is fairly straightforward: we retrieve the profile associated with the event that triggered the rule and
check whether it already has the properties we are interested in. If not, we create the associated
property types and initialize the property values.
Once this is done, we update our profile with the new property values based on the previous values and
the metadata extracted from the event using the extractSourceURL method which uses our url property
that we’ve specified for our event source. We then return that the profile was updated as a result of our
action and Unomi will properly save it for us when appropriate. That’s it!
return null;
}
4.5. CONCLUSION
We have seen a simple example how to interact with Unomi using a combination of client-side code and
Unomi plugin. Hopefully, this provided an introduction to the power of what Unomi can do and how it
can be extended to suit your needs.
4.6. ANNEX
Here is an overview of how Unomi processes incoming requests to the ContextServlet. image:
images/unomi-request.png[Unomi request overview]
5. CONNECTORS
5.1. CONNECTORS
Apache Unomi provides the following connectors:
We are looking for help with the development of additional connectors. Any contribution (large or
small) is more than welcome. Feel free to discuss this in our mailing list.
https://developer.salesforce.com/signup
Create a new Connected App, by going into Setup -> App Manager and click "Create Connected App"
Make sure you retrieve the following information once you have created the app in the API (Enable
OAuth Settings):
Consumer key
Consumer secret (click to see it)
You must also retrieve your user’s security token, or create it if you don’t have one already. To do this
simply click on your user at the top right, select "Settings", the click on "Reset my security token". You
will receive an email with the security token.
sfdc.user.username=YOUR_USER_NAME
sfdc.user.password=YOUR_PASSWORD
sfdc.user.securityToken=YOUR_USER_SECURITY_TOKEN
sfdc.consumer.key=CONNECTED_APP_CONSUMER_KEY
sfdc.consumer.secret=CONNECTED_APP_SECRET
Deploy into Apache Unomi using the following commands from the Apache Karaf shell:
feature:repo-add mvn:org.apache.unomi/unomi-salesforce-connectors-karaf-
kar/${project.version}/xml/features
feature:install unomi-salesforce-connectors-karaf-kar
You can then test the connection to Salesforce by accessing the following URLs:
https://localhost:9443/cxs/sfdc/version
https://localhost:9443/cxs/sfdc/limits
The first URL will give you information about the version of the connectors, so this makes it easy to
check that the plugin is properly deployed, started and the correct version. The second URL will actually
make a request to the Salesforce REST API to retrieve the limits of the Salesforce API.
Both URLs are password protected by the Apache Unomi (Karaf) password. You can find this user and
password information in the etc/users.properties file.
You can now use the connectors’s defined actions in rules to push or pull data to/from the Salesforce
CRM. You can find more information about rules in the Concepts and the Getting Started pages.
If you followed all the steps in the Getting Started section, you can upgrade the Salesforce connectors by
cd extensions/salesforce-connectors
mvn clean install
feature:repo-refresh
feature:uninstall unomi-salesforce-connectors-karaf-feature
feature:install unomi-salesforce-connectors-karaf-feature
You can then check that the new version is properly deployed by accessing the following URL and
checking the build date:
https://localhost:9443/cxs/sfdc/version
The Salesforce Workbench contains a REST API Explorer that is very useful to test requests. You may
find it here :
https://workbench.developerforce.com/restExplorer.php
Using the Salesforce Workbench, you can setting streaming push queries (Queries->Streaming push
topics) such as the following example:
Before running the tests, make sure you have completed all the steps above, including the streaming
push queries setup.
By default the unit tests will not run as they need proper Salesforce credentials to run. To set this up
create a properties file like the following one:
test.properties
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
sfdc.user.username=YOUR_USER_NAME
sfdc.user.password=YOUR_PASSWORD
sfdc.user.securityToken=YOUR_USER_SECURITY_TOKEN
sfdc.consumer.key=CONNECTED_APP_CONSUMER_KEY
sfdc.consumer.secret=CONNECTED_APP_SECRET
and then use the following command line to reference the file:
cd extensions/salesforce-connectors
mvn clean install -DsfdcProperties=../test.properties
(in case you’re wondering the ../ is because the test is located in the services sub-directory)
6. CLUSTER SETUP
6.1. CLUSTER SETUP
Apache Karaf relies on Apache Karaf Cellar, which in turn uses Hazelcast to discover and configure its
etc/hazelcast.xml
All nodes on the same network, sharing the same cluster name will be part of the same cluster.
For the actual ElasticSearch configuration however, this must be done using the following file:
etc/org.apache.unomi.persistence.elasticsearch.cfg
Depending on the cluster size, you will want to adjust the following parameters to make sure your setup
is optimal in terms of performance and safety.
Node A :
numberOfReplicas=0
monthlyIndex.numberOfReplicas=0
Node B :
numberOfReplicas=0
monthlyIndex.numberOfReplicas=0
One node dedicated to context server, 2 nodes for elasticsearch storage with fault-tolerance
Node A :
numberOfReplicas=1
monthlyIndex.numberOfReplicas=1
Node B :
numberOfReplicas=1
monthlyIndex.numberOfReplicas=1
Node C :
7. CUSTOM EXTENSIONS
7.1. CUSTOM EXTENSIONS
Apache Unomi is a pluggeable server that may be extended in many ways. This document assumes you
are familiar with the Apache Unomi concepts . This document is mostly a reference document on the
different things that may be used inside an extension. If you are looking for complete samples, please
see the samples page.
An extension is simply a Maven project, with a Maven pom that looks like this:
<modelVersion>4.0.0</modelVersion>
<artifactId>unomi-extension-example</artifactId>
<name>Apache Unomi :: Extensions :: Example</name>
<description>Service implementation for the Apache Unomi Context Server extension that
integrates with the Geonames database</description>
<version>${project.version}</version>
<packaging>bundle</packaging>
<dependencies>
<!-- This dependency is not required but generally used in extensions -->
<dependency>
<groupId>org.apache.unomi</groupId>
<artifactId>unomi-api</artifactId>
<version>${project.version}</version>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<instructions>
<Embed-Dependency>*;scope=compile|runtime</Embed-Dependency>
<Import-Package>
sun.misc;resolution:=optional,
*
</Import-Package>
</instructions>
</configuration>
</plugin>
</plugins>
</build>
</project>
An extension may contain many different kinds of Apache Unomi objects, as well as custom OSGi
services or anything that is needed to build your application.
When you deploy a custom bundle with a custom definition (see "Predefined xxx" chapters under) for
the first time, the definition will automatically be deployed at your bundle start event if it does not
exist, after that if you redeploy the same bundle there are two cases: 1. Your bundle is a SNAPSHOT
then every time you redeploy it the definition will be redeployed 2. Your bundle is NOT a SNAPSHOT
then the definition will not be redeployed, but you can redeploy it manually using the command
unomi:deploy-definition <bundleId> <fileName>
You may provide pre-defined segments by simply adding a JSON file in the src/main/resources/META-
INF/cxs/segments directory of your Maven project. Here is an example of a pre-defined segment:
{
"metadata": {
"id": "leads",
"name": "Leads",
"scope": "systemscope",
"description": "You can customize the list below by editing the leads segment.",
"readOnly":true
},
"condition": {
"parameterValues": {
"subConditions": [
{
"parameterValues": {
"propertyName": "properties.leadAssignedTo",
"comparisonOperator": "exists"
},
"type": "profilePropertyCondition"
}
],
"operator" : "and"
},
"type": "booleanCondition"
}
}
Basically this segment uses a condition to test if the profile has a property leadAssignedTo that exists. All
profiles that match this condition will be part of the pre-defined segment.
You may provide pre-defined rules by simply adding a JSON file in the src/main/resources/META-
INF/cxs/rules directory of your Maven project. Here is an example of a pre-defined rule:
"condition" : {
"type": "profileUpdatedEventCondition",
"parameterValues": {
}
},
"actions" : [
{
"type": "evaluateProfileSegmentsAction",
"parameterValues": {
}
}
]
In this example we provide a rule that will execute when a predefined composed condition of type
"profileUpdatedEventCondition" is received. See below to see how predefined composed conditions are
declared. Once the condition is matched, the actions will be executed in sequence. In this example there
is only a single action of type "evaluateProfileSegmentsAction" that is defined so it will be executed by
Apache Unomi’s rule engine. You can also see below how custom actions may be defined.
By default Apache Unomi comes with a set of pre-defined properties, but in many cases it is useful to add
additional predefined property definitions. You can create property definitions for session or profile
properties by creating them in different directories.
For session properties you must create a JSON file in the following directory in your Maven project:
src/main/resources/META-INF/cxs/properties/sessions
For profile properties you must create the JSON file inside the directory in your Maven project:
src/main/resources/META-INF/cxs/properties/profiles
You can define new predefined conditions that are actually conditions inheriting from a parent
condition and setting pre-defined parameter values. You can do this by creating a JSON file in:
src/main/resources/META-INF/cxs/conditions
Here is an example of a JSON file that defines a profileUpdateEventCondition that inherits from a parent
condition of type eventTypeCondition.
{
"metadata": {
"id": "profileUpdatedEventCondition",
"name": "profileUpdatedEventCondition",
"description": "",
"systemTags": [
"event",
"eventCondition"
],
"readOnly": true
},
"parentCondition": {
"type": "eventTypeCondition",
"parameterValues": {
"eventTypeId": "profileUpdated"
}
},
"parameters": [
]
}
Personas may also be pre-defined by creating JSON files in the following directory:
{
"persona": {
"itemId": "usVisitor",
"properties": {
"description": "Represents a visitor browsing from inside the continental US",
"firstName": "U.S.",
"lastName": "Visitor"
},
"segments": []
},
"sessions": [
{
"itemId": "aa3b04bd-8f4d-4a07-8e96-d33ffa04d3d9",
"profileId": "usVisitor",
"properties": {
"operatingSystemName": "OS X 10.9 Mavericks",
"sessionCountryName": "United States",
"location": {
"lat":37.422,
"lon":-122.084058
},
"userAgentVersion": "37.0.2062.120",
"sessionCountryCode": "US",
"deviceCategory": "Personal computer",
"operatingSystemFamily": "OS X",
"userAgentName": "Chrome",
"sessionCity": "Mountain View"
},
"timeStamp": "2014-09-18T11:40:54Z",
"lastEventDate": "2014-09-18T11:40:59Z",
"duration": 4790
}
]
}
You can see that it’s also possible to define sessions for personas.
Custom actions are a powerful way to integrate with external systems by being able to define custom
logic that will be executed by an Apache Unomi rule. An action is defined by a JSON file created in the
following directory:
src/main/resources/META-INF/cxs/actions
The actionExecutor identifier refers to a service property that is defined in the OSGi Blueprint service
registration. Note that any OSGi service registration may be used, but in these examples we use OSGi
Blueprint. The definition for the above JSON file will be found in a file called src/main/resources/OSGI-
INF/blueprint/blueprint.xml with the following content:
<service interface="org.apache.unomi.api.actions.ActionExecutor">
<service-properties>
<entry key="actionExecutorId" value="addToLists"/>
</service-properties>
<bean class="org.apache.unomi.lists.actions.AddToListsAction">
<property name="profileService" ref="profileService"/>
<property name="eventService" ref="eventService"/>
</bean>
</service>
</blueprint>
You can note here the actionExecutorId that corresponds to the actionExecutor in the JSON file.
Custom conditions are different from predefined child conditions because they implement their logic
using Java classes. They are also declared by adding a JSON file into the conditions directory:
src/main/resources/META-INF/cxs/conditions
{
"metadata": {
"id": "matchAllCondition",
"name": "matchAllCondition",
"description": "",
"systemTags": [
"logical",
"profileCondition",
"eventCondition",
"sessionCondition",
"sourceEventCondition"
],
"readOnly": true
},
"conditionEvaluator": "matchAllConditionEvaluator",
"queryBuilder": "matchAllConditionESQueryBuilder",
"parameters": [
]
}
Note the conditionEvaluator and the queryBuilder values. These reference OSGi service properties that
are declared in an OSGi Blueprint configuration file (other service definitions may also be used such as
Declarative Services or even Java registered services). Here is an example of an OSGi Blueprint
definition corresponding to the above JSON condition definition file.
<blueprint xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v1.0.0
http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd">
<service
interface="org.apache.unomi.persistence.elasticsearch.conditions.ConditionESQueryBuilder">
<service-properties>
<entry key="queryBuilderId" value="matchAllConditionESQueryBuilder"/>
</service-properties>
<bean
class="org.apache.unomi.plugins.baseplugin.conditions.MatchAllConditionESQueryBuilder"/>
</service>
<service interface="org.apache.unomi.persistence.elasticsearch.conditions.ConditionEvaluator">
<service-properties>
<entry key="conditionEvaluatorId" value="matchAllConditionEvaluator"/>
</service-properties>
<bean class="org.apache.unomi.plugins.baseplugin.conditions.MatchAllConditionEvaluator"/>
</service>
</blueprint>
• org.apache.unomi.plugins.baseplugin.conditions.MatchAllConditionESQueryBuilder
• org.apache.unomi.plugins.baseplugin.conditions.MatchAllConditionEvaluator
8. CONSENT API
8.1. CONSENT API
Starting with Apache Unomi 1.3 (still in development), a new API for consent management is now
available. This API is designed to be able to store/retrieve/update visitor consents in order to comply
with new privacy regulations such as the GDPR.
Visitor profiles now contain a new Consent object that contains the following information:
• a scope
• a type identifier for the consent. This can be any key to reference a consent. Note that Unomi does
not manage consent definitions, it only stores/retrieves consents for each profile based on this type
• a revocation date, in order to comply with GDPR this is usually set at two years
{
"profileId": "8cbe380f-57bb-419d-97bf-24bf30178550",
"sessionId": "0d755f4e-154a-45c8-9169-e852e1d706d9",
"consents": {
"example/newsletter": {
"scope": "example",
"typeIdentifier": "newsletter",
"status": "GRANTED",
"statusDate": "2018-05-22T09:44:33Z",
"revokeDate": "2020-05-21T09:44:33Z"
}
}
}
It is of course possible to have multiple consents defined for a single visitor profile.
Apache Unomi does not manage consent definitions, it leaves that to an external system (for example a
CMS) so that it can handle user-facing UIs to create, update, internationalize and present consent
definitions to end users.
The only thing that is import to Apache Unomi to manage visitor consents is a globally unique key, that
is called the consent type.
A new built-in event type called "modifyConsent" can be sent to Apache Unomi to update a consent for
the current profile.
Upon receiving this event, Apache Unomi will trigger the modifyAnyConsent rule that has the following
definition:
"condition" : {
"type": "modifyAnyConsentEventCondition",
"parameterValues": {
}
},
"actions" : [
{
"type": "modifyConsentAction",
"parameterValues": {
}
}
]
As we can see this rule is pretty simple it will simply execute the modifyConsentAction that is
implemented by the ModifyConsentAction Java class
This class will update the current visitor profile to add/update/revoke any consents that are included in
the event.