Incessant timeouts; slow performance overall

You can watch any page on xwiki.org, go to the bell on the top right corner and enable it for this page and then make sure you have a configured “Notifier” in the section Watchlist of your user profile.

crw, I’d also recommend installing javamelody so you can easily monitor your system and get a good idea of what your normal performance is.

The overhead is light, and the insights are invaluable. This tool will help you identify problems and also help you figure out where those problems might be.

Thank you, I looked for the notification ability but didn’t find it (I am default subscribed to all changes everywhere on the wikis I run, so it isn’t something I think about often).

edit: and, I wasn’t logged in so I couldn’t see the Bell icon; I made the assumption that functionality was turned off on that wiki somehow. As a UX thing, it might be good to have that greyed out with a message to log in to configure to be alerted to changes.

Thank you, I’ll definitely check this out!

You should try glowroot (https://glowroot.org/) which provides even better information that javamelody. FYI we were using javamelody on xwiki.org before and we switched to glowroot.

@tmortagne I think we forgot to update the doc https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Monitoring It would be nice to document glowroot there.

For your use case maybe but not generically. XWiki can be used for lots of use cases, including a web site use case and it doesn’t make sense to show a greyed out icon for this use case where users are not supposed to register/have a login.

Cool! Glad to hear that :slight_smile:

Could you remind us what “new” recommendations you’re talking about?

Thx

On Ubuntu, my /etc/defaults/tomcat8 changed from:

JAVA_OPTS="-Djava.awt.headless=true -Xmx512m -XX:MaxPermSize=196m -XX:+UseParallelGC -XX:MaxGCPauseMillis=100 -Dfile.encoding=utf-8"

To:

JAVA_OPTS="-server -Xms800m -Xmx1600m -XX:MaxPermSize=222m -Dfile.encoding=utf-8 -Djava.awt.headless=true -XX:+UseParallelGC -XX:MaxGCPauseMillis=100

Those changes seem to have been the difference between crashing / hanging all the time and not. So, relatively important changes.

Ok I misunderstood you then. You mentioned “new recommendations” but these are very old one (since the beginning of XWiki actually, unless we’re talking about different things). So I don’t understand the part about “it would be nice if there were posts here when they are updated, or it would be nice if I could subscribe to that page to get notifications when it is changed.”. Do you mean this generically and not specifically for the memory recommendations?

And you’re right the memory settings are important. They’re so important that they are part of our installation prerequisites: https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/#HHardwareandSoftwarerequirements

Maybe it will be easier to see the changes if I re-order things (old first, new second):

JAVA_OPTS="-Xmx512m -XX:MaxPermSize=196m -XX:MaxGCPauseMillis=100 -XX:+UseParallelGC -Djava.awt.headless=true -Dfile.encoding=utf-8"

JAVA_OPTS=" -server -Xms800m -Xmx1600m -XX:MaxPermSize=222m -XX:MaxGCPauseMillis=100 -XX:+UseParallelGC -Djava.awt.headless=true -Dfile.encoding=utf-8

Please take a moment to look closely at the two recommendations. Two directives were added to the beginning, and two values were changed. That is a change. You changed your recommendation. Thus becoming a new recommendation.

You may not consider it a very dramatic change, and yet it was the difference between a functioning server.

@crw Could you please provide a link to where you see this and a URL with the history diff so that I can understand what you mean? Thanks!

For the diff you go here for example: https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Performances/#History and you pick from and to values and then click “compare selected values”. This will give you a URL that you can paste here.

Thx

Done.

Thanks Thomas! I’ve resized it a bit to follow our best practices from https://dev.xwiki.org/xwiki/bin/view/Community/DocGuide#HScreenshots2FImages (was very large)! :wink:

Two examples:

https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Performances/?viewer=changes&rev1=96.1&rev2=97.1

https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Performances/?viewer=changes&rev1=53.1&rev2=54.1

This one is not about the recommended settings. It’s just what we use on xwiki.org.

Same here.

FTR I had checked before asking you and couldn’t find any recent change of the recommendations which is why I was surprised you mentioned it changed. You said you applied the new settings and I still don’t know what this means. I wanted to make sure I didn’t miss anything.

If you could clarify that would be great.

Thx

Let’s go back to the beginning.

  1. xwiki was cpu spiking and not returning requests.
  2. I changed the CATALINA_OPTS parameters on my server.
  3. Things started working.

The old CATALINA_OPTS I was using came from how you run xwiki.org, from years ago. The new CATALINA_OPTS I am now using come from how you are running xwiki.org today.

That’s it.

Thanks. The xwiki.org settings are actually not recommendations.

So from what you said above at https://forum.xwiki.org/t/incessant-timeouts-slow-performance-overall/3595/17?u=vmassol you just changed the max memory from 512M to 1600M. You also changed the perm size from 196M to 222M. All the other parameters were unchanged)

If you check the recommendations they say:

Small and medium installs: A minimum of 1024MB (-Xmx1024m)
Large installs: 2048MB or beyond (-Xmx2048m).

So you were simply not providing enough memory since the minimum XWiki needs is 1024MB and that’s already part of the recommendations (and FTR this hasn’t changed for about 8-9 years).

So it seems we’re good on the doc side, do you agree?

Thanks and have a great weekend.

BTW if you’re on java8, the recommendation also say you don’t need the perm size…

Since XWiki requires java8 since a few years now, you’re probably on java8.

PS: We need to edit the xwiki.org settings since they’re no longer current for the same reason.

This is actually not correct. I’ve just noticed that the old recommendation for XWiki < 8.1 for small install was 512MB. I guess we increased the memory need after 8.1.

Done.

Just gave glowroot a try - it’s even easier to install than javamelody. Thanks for the tip!