Redisson tomcat session manager performance advice/questions/issues #6054

stefan-campiomate · 2024-07-25T05:38:36Z

stefan-campiomate
Jul 25, 2024

TL;DR: SessionManager is very slow. What am I doing wrong?

While implementing a multi instance webapp, we came across the redisson session manager and tried it out.
The webapp and redis cache are both hosted on azure, using (for development and testing purposes) the lowest plan possible (I think this will become important later, as the redis cache is marketed as 'low bandwidth')

The immediate and very noticeable issue was a serious performance drop in response time. Request times increased from milliseconds to couple seconds and maybe more.

So naturally, we thought there's a config problem. Could not find anything that improved the situation.

Next was to look at the redisson-tomcat code to try and figure out what we were doing wrong. And we came across some pieces of code that raised some questions. I'll just include a couple here so that the post gets not too long, but I'll be happy to discuss each one in detail if needed.

redisson-tomcat/redisson-tomcat-9/src/main/java/org/redisson/tomcat/RedissonSessionManager.java:173

Why call access() and endAccess() in sequence ? According to RedissonSession.java:214, access() will set THIS_ACCESSED_TIME_ATTR then the expiration, and endAccess() will do almost the same thing all over again then broadcast the update... Not to mention that a store() is also called later which includes more or less the same updates.

Not to mention that access() is also called internally by tomcat, so the call in findSession() seems redundant.

According to our profiling and our infrastructure, both take some 150 to 250ms to complete which amount to at least 350-500 ms for the findSession() call... for EVERY call. First page load (I'm saying 'first' because later on there may be some caching involved for some ressources) will call the findSession() several times which will add up to a hell of a response time.

Which brings us to the next one ...

This only one example, but it's all over; redisson/src/main/java/org/redisson/RedissonMap.java:1315

public boolean fastPut(K key, V value) {
        return get(fastPutAsync(key, value));
}

The call to get() will wait for the Future to finish and kind of serialize in time all calls. This begs the question : can this be done differently ? especially if the return value is not needed (like in RedissonSession.java:214).

We have timed a redis operation to about 50ms (this is where the azure low bandwidth plan comes into play), so the response time from calls from the previous access() and endAccess() gets explained. Probably, with a faster redis response time these issues go unnoticed ...

Overall, the impression is that there is a lot of overhead that is not necessary or could be improved. It seems that there is a lot of network traffic between tomcat and the redis cache that can be avoided.
Or maybe we are trying to use the session manager the wrong way or with the wrong configuration.

Any advice/comment/criticism/help/solution is welcome. Thanks in advance.

mrniko · 2024-07-25T06:49:03Z

mrniko
Jul 25, 2024
Maintainer

Thanks for you feedback.

Why call access() and endAccess() in sequence ? According to RedissonSession.java:214, access() will set THIS_ACCESSED_TIME_ATTR then the expiration, and endAccess() will do almost the same thing all over again then broadcast the update... Not to mention that a store() is also called later which includes more or less the same updates.

Don't forget that the internal state of Tomcat session also should be in correct state not only state stored in Redis. This is why these methods should be called in such way.

This begs the question : can this be done differently ? especially if the return value is not needed (like in RedissonSession.java:214).

What do you suggest? response has boolean type and it's synchronous.

Overall, the impression is that there is a lot of overhead that is not necessary or could be improved. It seems that there is a lot of network traffic between tomcat and the redis cache that can be avoided.

There is always room for improvement.

2 replies

mrniko Jul 25, 2024
Maintainer

I think new methods can be introduced which groups:

access() and endAccess() methods
access() and endAccess() and save() methods

Can you create an issue regarding this?

stefan-campiomate Jul 25, 2024
Author

I think new methods can be introduced which groups:
Can you create an issue regarding this?

Sure thing, but let me share my detailed findings first; It'll take a moment, because I've tried some local changes and need to revert them.

stefan-campiomate · 2024-07-25T08:08:56Z

stefan-campiomate
Jul 25, 2024
Author

Don't forget that the internal state of Tomcat session also should be in correct state not only state stored in Redis. This is why these methods should be called in such way.

Totally agree, but I think they're called too many times.

Here's a sample of a page load; there are two calls for findSession() and one for store(), as all the rest is cached on the browser side.
Each 'level' sums the delays from the indented lines before it, and every call is between the '---' boundaries.
For instance, the first call to findSession() will spend 0ms on super.findSession(), 101ms on access() then 105 on endAccess() so the total will be 206ms overall.

findSession(ADEE376F0127FA25096E4CA55784F170) ---------------------------------------------------
	0 ms : super.findSession(ADEE376F0127FA25096E4CA55784F170) : RedissonSession
		0 ms : super.access(ADEE376F0127FA25096E4CA55784F170)
			52 ms : m.fastPut(session:thisAccessedTime, 1721894299066), RedissonMap
			52 ms : fastPut done
		52 ms : fastPut()
			49 ms : fastExpire(1860)
		101 ms : expireSession(ADEE376F0127FA25096E4CA55784F170)
	101 ms : result.access()
		0 ms : super.endAccess()
		52 ms : m.putAll(newMap)
			53 ms : fastExpire(1860)
		105 ms : expireSession(ADEE376F0127FA25096E4CA55784F170)
		105 ms : endAccess(ADEE376F0127FA25096E4CA55784F170) done
	206 ms : result.endAccess()
------- 206 ms : exit findSession(ADEE376F0127FA25096E4CA55784F170)

		0 ms : super.access(ADEE376F0127FA25096E4CA55784F170)
			51 ms : m.fastPut(session:thisAccessedTime, 1721894299272), RedissonMap
			51 ms : fastPut done
		51 ms : fastPut()
			51 ms : fastExpire(1860)
		102 ms : expireSession(ADEE376F0127FA25096E4CA55784F170)

findSession(ADEE376F0127FA25096E4CA55784F170) ---------------------------------------------------
	0 ms : super.findSession(ADEE376F0127FA25096E4CA55784F170) : RedissonSession
		0 ms : super.access(ADEE376F0127FA25096E4CA55784F170)
			50 ms : m.fastPut(session:thisAccessedTime, 1721894299375), RedissonMap
			51 ms : fastPut done
		51 ms : fastPut()
			51 ms : fastExpire(1860)
		102 ms : expireSession(ADEE376F0127FA25096E4CA55784F170)
	102 ms : result.access()
		0 ms : super.endAccess()
		51 ms : m.putAll(newMap)
			52 ms : fastExpire(1860)
		103 ms : expireSession(ADEE376F0127FA25096E4CA55784F170)
		103 ms : endAccess(ADEE376F0127FA25096E4CA55784F170) done
	205 ms : result.endAccess()
------- 205 ms : exit findSession(ADEE376F0127FA25096E4CA55784F170)

store(ADEE376F0127FA25096E4CA55784F170) ---------------------------------------------------
		0 ms : super.access(ADEE376F0127FA25096E4CA55784F170)
			50 ms : m.fastPut(session:thisAccessedTime, 1721894299619), RedissonMap
			50 ms : fastPut done
		50 ms : fastPut()
			51 ms : fastExpire(1860)
		101 ms : expireSession(ADEE376F0127FA25096E4CA55784F170)
	101 ms : session.access()
		0 ms : super.endAccess()
		53 ms : m.putAll(newMap)
			50 ms : fastExpire(1860)
		103 ms : expireSession(ADEE376F0127FA25096E4CA55784F170)
		103 ms : endAccess(ADEE376F0127FA25096E4CA55784F170) done
	204 ms : session.endAccess()
		558 ms : m.putAll(newMap)
		609 ms : m.remove(newMap)
			52 ms : fastExpire(1860)
		661 ms : expireSession(ADEE376F0127FA25096E4CA55784F170)
	865 ms : session.save()
------- 865 ms : exit store(ADEE376F0127FA25096E4CA55784F170)

		0 ms : super.endAccess()
		52 ms : m.putAll(newMap)
			53 ms : fastExpire(1860)
		105 ms : expireSession(ADEE376F0127FA25096E4CA55784F170)
		105 ms : endAccess(ADEE376F0127FA25096E4CA55784F170) done

What's intriguing is that there is a call to access() from tomcat between the two findSession() and one to endAccess() after the store()

This kind of makes the previous explicit calls to access() and endAccess() redundant, doesn't it ?

4 replies

mrniko Jul 25, 2024
Maintainer

Which profiler do you use?

This kind of makes the previous explicit calls to access() and endAccess() redundant, doesn't it ?

I wouldn't rely on that since that may change in the next Tomcat version.

stefan-campiomate Jul 25, 2024
Author

Which profiler do you use?

As a matter of fact, none; these are just System.outs like so :

@Override
public Session findSession(String id) throws IOException {
    Long lStart = System.currentTimeMillis();
    System.out.println("findSession(" + id + ") ---------------------------------------------------");
    Session session = findSession(id, true);
    System.out.println("------- " + (System.currentTimeMillis() - lStart) + " ms : exit findSession(" + id + ")");
    return session;
}

stefan-campiomate Jul 25, 2024
Author

I wouldn't rely on that since that may change in the next Tomcat version.

I agree, but adding such an overhead only because tomcat developers may someday change this (quite improbable, I'd say).
Look at my timings... ~50ms per redis request adds up to quite some amount of time.

And in production environments, the tomcat version does not change that often either ...

mrniko Jul 26, 2024
Maintainer

Did some optimizations 67f03e2

stefan-campiomate · 2024-07-26T12:45:50Z

stefan-campiomate
Jul 26, 2024
Author

Just trying to see what happens, I tried something myself. Tried two things :

added some new methods to the RMap and RExpirable
add a new parameter/attribute named 'fastRedis' to the Manager tag in the context.xml file; I'll describe below what it does

For the sake of keeping this comment short, I'll post only one of the methods, since they all follow the same pattern :

// original method
boolean fastPut(K key, V value);

// new method
default void _fastPut(K key, V value) {
    fastPut(key, value);
}

Obviously, the default implementation will call the original method to avoid making too many changes; for these new methods the return type is always void.
Note : I know they don't follow your naming rules, but they are only intended as a POC, not a permanent addition

The implementation in the RedissonMap goes like this :

// original method
public boolean fastPut(K key, V value) {
    return get(fastPutAsync(key, value));
}

// new method
public void _fastPut(K key, V value) {
    fastPutAsync(key, value);
}

Basically, I created asynchronous versions without caring for the timing or the return value for two reasons : it's never used and it serializes the operations in time.

Then, whenever the original method was used :

if (fastRedis) {
    m._fastPut(name, value);
} else {
    m.fastPut(name, value);
}

In addition to that, I wrapped the access() and endAccess() in the findSession() and a couple more places like so :

if (!fastRedis) {
    session.access();
    session.endAccess();
}

Simply put, if I set fastRedis to false all works like the original code; when fastRedis is true, all the duplicate redis requests are removed.

The result : every call to findSession() goes down to a negligeable amount, as well as the final store(), and response times are down again to almost the same as using the standard session manager.

But it created a problem : being asynchronous, when the session changes id following a login request, there is a session object left in redis with no expiration set; tricky to understand why, but I won't go in such detail here.

Which was solved by triggering a delayed expiration on the previous session id after a session id change.

Currently I plan to test this against an azure configuration to see if I can get rid of the sticky sessions I have previously used.

2 replies

mrniko Jul 27, 2024
Maintainer

Instead of introducing the new method you could just call RMapAsync.fastPutAsync() method. I would name fastMode with a notice in the description.

stefan-campiomate Jul 28, 2024
Author

I did not want to disrupt anything else, since I'm not at all familiar with the rest of the code; just wanted to try and see if this approach works; hence the new methods and their default to the existing code.

Anyway, the idea was that, since there is no action taken on the update of the values in redis or failure of the operation , why wait for the response ?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Redisson tomcat session manager performance advice/questions/issues #6054

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 3 comments 8 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Redisson tomcat session manager performance advice/questions/issues #6054

stefan-campiomate Jul 25, 2024

Replies: 3 comments · 8 replies

mrniko Jul 25, 2024 Maintainer

mrniko Jul 25, 2024 Maintainer

stefan-campiomate Jul 25, 2024 Author

stefan-campiomate Jul 25, 2024 Author

mrniko Jul 25, 2024 Maintainer

stefan-campiomate Jul 25, 2024 Author

stefan-campiomate Jul 25, 2024 Author

mrniko Jul 26, 2024 Maintainer

stefan-campiomate Jul 26, 2024 Author

mrniko Jul 27, 2024 Maintainer

stefan-campiomate Jul 28, 2024 Author

stefan-campiomate
Jul 25, 2024

Replies: 3 comments 8 replies

mrniko
Jul 25, 2024
Maintainer

mrniko Jul 25, 2024
Maintainer

stefan-campiomate Jul 25, 2024
Author

stefan-campiomate
Jul 25, 2024
Author

mrniko Jul 25, 2024
Maintainer

stefan-campiomate Jul 25, 2024
Author

stefan-campiomate Jul 25, 2024
Author

mrniko Jul 26, 2024
Maintainer

stefan-campiomate
Jul 26, 2024
Author

mrniko Jul 27, 2024
Maintainer

stefan-campiomate Jul 28, 2024
Author