Since 13.2, we have had a performance issue. Twice now I’ve found tomcat9.exe process using more than 50% CPU on this server. Right now, it’s using >97% CPU continuously. The only thing that has changed is the upgrade to 13.2 (actually now we are on 13.3). If I restart the service, the CPU usage returns to normal, but then it starts again at some point. I am going to try to add some monitoring to determine the timing of this issue.
Ugh, still happening and I’m guessing nobody has any feedback on what could be the cause here.
Today, even after a restart of the service, still happening. This will make the system unusable for us - that servers is not dedicated to XWiki and we can’t have it overloaded like this!
Taking resource right after a restart is expected since there is various things to initialize or to cache during first requests.
It would be interesting to take a thread dump when this happens long after a restart when you did not really start something you know to be resource consuming. It should at least tell us what’s running which might give us a clue of what might cause this.
Hi Keith, all, Beside the thread dump mentioned by Thomas, inspecting the ongoing requests with Glowroot can also be very handy. Also, you might find it helpful to possibly install the Active Jobs Extension for checking which jobs are running in the background?
HTH / cheers
Thanks, and I’m aware of the high usage right after the restart, however, I’ve watched this process enough (I’m kind of obsessive that way) to know it was longer than normal (but can’t give you a number) - however, in this case, I found out after I posted the above that the service had not restarted…was stuck in “Stopping”. When restarting, takes a very long time for the service to stop, then restart.
Also, prior to this becoming an issue, it NEVER took 98% to do anything. Maybe 405-50%, but at least twice I’ve seen it stuck at 98% for 10 minutes (probably longer, that is when I took a look at it).
The problem with any of these solutions is if the CPU usage is at 98%, the server is unusable for its other purposes, and most of the time that is not something we can wait to fix. Happened just now and users were having trouble getting their tasks done that use it, so I had to kill it and restart.
Well taking a thread dump before killing it should not take long.
Ok, but although I probably have taken a thread dump at some point in my career, I don’t know how to do that. I can look it up, but if you have any pointers, I would appreciate it.
As described here?
Yes that’s it. As an alternative if you have Glowroot you can access it in its UI which might be a bit easier unless the JVM is really unusable.
Well, these instructions don’t work for Apache Tomcat, as there is no Java process…unless I can do it with the Tomcat9.exe process, I guess I’ll try that.
I don’t know that this is going to be doable. While Tomcat is using 98% of CPU (which it is right now) very operation that normally takes seconds, takes minutes, completely locks up the UI until it is complete. What should take a few minutes, to get this thread dump, will probably take an hour or two. Maybe if I Was adept at what I am doing here, I could cut that down some, but if I can’t get it in the next 10-15 minutes, I will have to kill it as they access to this server for patients.
Well, another problem doing this - we use the server version of Java, does not have jstack.
Tried to load the tools from a JRE, as the docs say I can use jcmd instead of the JRE, but that doesn’t work:
C:\Temp>psexec -s c:\java\bin\jcmd 5276 Thread.print>thread_dump_001.log
PsExec v2.33 - Execute processes remotely
Copyright (C) 2001-2021 Mark Russinovich
Sysinternals - www.sysinternals.com
Starting c:\java\bin\jcmd on SERVER06...06...
Error: Could not find or load main class sun.tools.jcmd.JCmd
c:\java\bin\jcmd exited on SERVER06 with error code 1.
All the time I could give it, had to kill the process.
We have something similar lately. Our Wiki 12.10.7 LTS on Apache Tomcat/8.5.39 (Ubuntu), with java 11.0.11+9-Ubuntu-0ubuntu2.18.04 stops working randomly. The Tomcat process is at 99% and we have to restart the tomcat8 process.
I’ll reinstall glowroot sometime soon…
It died just right after an hour 
Trying to install Glowroot, but I don’t understand how.
Download and unzip glowroot-0.13.6-dist.zip, add
-javaagent:path/to/glowroot.jarto your application’s JVM args
Using Apache Tomcat on Windows, I don’t see how that is possible.
Hi. I’ve just googled for apache tomcat windows jvm args and the first link I got led to
Add JVM options in Tomcat - Stack Overflow which has instructions. I’m sure you can also simply go to the tomcat user guide for windows.
Hope it helps