Applications /Macros are not executed after update form 11.5 to 11.6 (SOLVED)

Jan · August 5, 2019, 9:11am

Hello everybody,
we use Xwiki as replacement for Confluence and are very excited. It’s really a lot of fun to work with it.
Thank you very much for all the effort!

Unfortunately we have some strange problems after an update from 11.5 to 11.6:

The setup wizard was not started after Administrator-Login
It seems that the applications and macros are not executed. For example, the menu on the left is empty and the administrator menu shows only names of variable but no content.
Executing macros also fails ( The “…” macro is not in the list of registered macros … )

The web pages within the wiki still exist and can be accessed via direct links. But the menu panels are not available or empty.l

Maybe the Logfile helps:
catalina.out.log (14.2 KB)

The XWiki runs on Ubuntu Server 18.04 LTS . The Installation / Upgrade was done via apt.

Does anyone have any idea what the problem might be?

Thanks a lot!

ThomasFa · August 5, 2019, 6:33pm

We have the same issue. Hope that we will find a solution quickly

ClemensRobbenhaar · August 5, 2019, 6:56pm

The error in your log is because of a missing “xwiki-locales.txt”.
This should be present in /usr/lib/xwiki/WEB-INF/ and if it is the standard Debian package, then it should be a symbolic link to /etc/xwiki/xwiki-locales.txt

I have looked as my production instance (still on 10.11.9) and the symbolic link is missing for me, too.
I also get the same exception in the log when I visit the “Localization” section in the Wiki admin. However so far I never noticed it because it has no further effects.

The following repairs the problem:

 cd /usr/lib/xwiki/WEB-INF/
 ln -s /etc/xwiki/xwiki-locales.txt .
 systemctl restart tomcat8

Afterwards the stack trace no longer shows up in the log.

However I doubt this is the source of the problems you have.

The update wizard should have shown up after the restart; otherwise you have an outdated UI.
You can still try to update in the Wiki Admin via “Extensions” -> “Updater”. However as you might restart the server, maybe you can also:

move away the /var/lib/xwiki/data/jobs/status/distribution/ directory (you can put it back later if the setup wizard does still not show up)
after restarting the server look for any error messages that happen immediately after startup (and after XWiki initializes with the first request)

There is one error message that you can ignore:

     05-Aug-2019 20:45:49.229 SEVERE [localhost-startStop-1] org.apache.catalina.session.StandardManager.startInternal Exception loading sessions from persistent storage
     java.lang.ClassCastException: java.io.ObjectStreamClass cannot be cast to java.lang.String

That one is “ok” - the persisted session cannot be loaded, but that is not really a problem.

Every other error message would be very interesting.

ThomasFa · August 5, 2019, 7:34pm

The link to xwiki-locales is there now, but as you assumed, it will not work:

2019-08-05 21:31:41,397 [XWiki Scheduler initialization] ERROR c.x.x.p.s.SchedulerPlugin      - Failed to restore job with in document [xwiki:Scheduler.WatchListHourlyNotifier] and wiki [xwiki] 
com.xpn.xwiki.plugin.scheduler.SchedulerPluginException: Error number 90006 in 5: Exception in plugin [com.xpn.xwiki.plugin.scheduler.SchedulerPlugin]: Error while loading job class for job : WatchList hourly notifier
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.scheduleJob(SchedulerPlugin.java:426)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.register(SchedulerPlugin.java:316)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.register(SchedulerPlugin.java:309)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.restoreExistingJobs(SchedulerPlugin.java:294)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.initAsync(SchedulerPlugin.java:168)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.access$000(SchedulerPlugin.java:80)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin$1.run(SchedulerPlugin.java:127)
	at org.xwiki.context.concurrent.ExecutionContextRunnable.run(ExecutionContextRunnable.java:70)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.xwiki.watchlist.internal.job.WatchListJob
	at org.xwiki.classloader.URIClassLoader$1.run(URIClassLoader.java:193)
	at org.xwiki.classloader.URIClassLoader$1.run(URIClassLoader.java:180)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.xwiki.classloader.URIClassLoader.findClass(URIClassLoader.java:179)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at org.xwiki.classloader.xwiki.internal.ContextNamespaceURLClassLoader.loadClass(ContextNamespaceURLClassLoader.java:176)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.scheduleJob(SchedulerPlugin.java:361)
	... 8 common frames omitted
2019-08-05 21:31:41,892 [XWiki Scheduler initialization] ERROR c.x.x.p.s.SchedulerPlugin      - Failed to restore job with in document [xwiki:Scheduler.WatchListWeeklyNotifier] and wiki [xwiki] 
com.xpn.xwiki.plugin.scheduler.SchedulerPluginException: Error number 90006 in 5: Exception in plugin [com.xpn.xwiki.plugin.scheduler.SchedulerPlugin]: Error while loading job class for job : WatchList weekly notifier
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.scheduleJob(SchedulerPlugin.java:426)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.register(SchedulerPlugin.java:316)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.register(SchedulerPlugin.java:309)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.restoreExistingJobs(SchedulerPlugin.java:294)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.initAsync(SchedulerPlugin.java:168)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.access$000(SchedulerPlugin.java:80)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin$1.run(SchedulerPlugin.java:127)
	at org.xwiki.context.concurrent.ExecutionContextRunnable.run(ExecutionContextRunnable.java:70)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.xwiki.watchlist.internal.job.WatchListJob
	at org.xwiki.classloader.URIClassLoader$1.run(URIClassLoader.java:193)
	at org.xwiki.classloader.URIClassLoader$1.run(URIClassLoader.java:180)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.xwiki.classloader.URIClassLoader.findClass(URIClassLoader.java:179)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at org.xwiki.classloader.xwiki.internal.ContextNamespaceURLClassLoader.loadClass(ContextNamespaceURLClassLoader.java:176)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.scheduleJob(SchedulerPlugin.java:361)
	... 8 common frames omitted
2019-08-05 21:31:41,918 [XWiki Scheduler initialization] ERROR c.x.x.p.s.SchedulerPlugin      - Failed to restore job with in document [xwiki:Scheduler.WatchListDailyNotifier] and wiki [xwiki] 
com.xpn.xwiki.plugin.scheduler.SchedulerPluginException: Error number 90006 in 5: Exception in plugin [com.xpn.xwiki.plugin.scheduler.SchedulerPlugin]: Error while loading job class for job : WatchList daily notifier
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.scheduleJob(SchedulerPlugin.java:426)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.register(SchedulerPlugin.java:316)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.register(SchedulerPlugin.java:309)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.restoreExistingJobs(SchedulerPlugin.java:294)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.initAsync(SchedulerPlugin.java:168)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.access$000(SchedulerPlugin.java:80)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin$1.run(SchedulerPlugin.java:127)
	at org.xwiki.context.concurrent.ExecutionContextRunnable.run(ExecutionContextRunnable.java:70)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.xwiki.watchlist.internal.job.WatchListJob
	at org.xwiki.classloader.URIClassLoader$1.run(URIClassLoader.java:193)
	at org.xwiki.classloader.URIClassLoader$1.run(URIClassLoader.java:180)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.xwiki.classloader.URIClassLoader.findClass(URIClassLoader.java:179)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at org.xwiki.classloader.xwiki.internal.ContextNamespaceURLClassLoader.loadClass(ContextNamespaceURLClassLoader.java:176)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at com.xpn.xwiki.plugin.scheduler.SchedulerPlugin.scheduleJob(SchedulerPlugin.java:361)
	... 8 common frames omitted

ClemensRobbenhaar · August 5, 2019, 8:05pm

Ouch. This looks like several important classes are missing. I actually wonder why XWiki starts up at all.

The class org.xwiki.watchlist.internal.job.WatchListJob is a rather old class of the ‘watchlist’ feature that has been replaced by notifications quite some time ago. If you still have a scheduler that uses this class, you likely have a very old UI.

Did the Distribution wizard show up the first time you logged in as an Admin to the updated Wiki Instance?
Can you go to the Wiki Admin, and search for “XWiki Standard Flavor” in the “Installed Extensions” and see which version you have there?

ThomasFa · August 6, 2019, 8:00am

The XWiki Menu is not available also. I use apt package manager. Is there any convenient way to replace the XWiki version?

tmortagne · August 6, 2019, 8:29am

This is supposed to be fixed by a migration which delete those old schedulers which require a class that does not exist anymore. In practice it does not have any more impact that this log but unless those pages were restored some way I don’t see why you still have them after the upgrade. Maybe you installed/repaired the watchlist UI ?

ClemensRobbenhaar · August 6, 2019, 10:11am

You should be able to get to the wiki Admin by manually typing in the URL to .../xwiki/bin/view/XWiki/ and follow the link shown on that page. If that does not work, try .../xwiki/bin/admin/XWiki/XWikiPreferences.

Alternatively I think you can try to downgrade the XWiki version on the file system to a version which is compatible with the UI, but I do not know the apt tool good enough to give advice how to downgrade a package.

Jan · August 6, 2019, 10:18am

Please excuse the late reply and thank you so much for the effort !

I created a symbolic link to /etc/xwiki/xwiki-locales.txt and restartet the Tomcat server.
Beyond that I tried to update the Extensions and the Standard Flavor via “Extensions”->“Updater”.
It did actually update a lot of Extensions and as far as I can tell all Extensions and the Standard Flavor are now installed in Version 11.6.
Unfortunately -as you suspected- it didn’t solve the main problem. The Macros and the Applications are still not executed by the system, therefore menus und panels are still empty.

This is the new logfile, which was saved after restarting the tomcat8 server:
catalina.out.log (392.2 KB)

It shows a lot of different errors which sounds not good

ClemensRobbenhaar · August 6, 2019, 10:58am

There are some errors related to solr. If you move away the /var/lib/xwiki/data/solr directory, these errors should go away.
It seems to me, that the com.xpn.xwiki.plugin.watchlist.WatchListPlugin is still in teh list of plugins in the /etc/xwiki/xwiki.cfg . This plugin is gone. Can you please remove it from the list of plugins and try again?

I have to admit that I do not see the actual cause of the problem from the logs, maybe because the errors from solr and the WatchListPlugin fill the logs and I am overlooking them.

ClemensRobbenhaar · August 6, 2019, 11:56am

I should really have read the logs bettter the last line says:

 2019-08-06 10:47:35,481 [http://localhost:8080/bin/get/Main/WebHome?outputSyntax=plain&sheet=XWiki.DocumentTree&showTranslations=false&limit=10&data=children&id=%23] WARN  o.x.i.t.i.n.WikiTreeNode       - Failed to retrieve the children of [wiki:xwiki]. Root cause [IllegalArgumentException: No query defined for that name [nonTerminalPagesOrderedByTitle]].

I am currently abroad and cannot check where the named query nonTerminalPagesOrderedByTitle is defined, but it seems something is missing.

tmortagne · August 6, 2019, 12:15pm

This is dynamically registered in xwiki-platform-index-tree-api jar (in QueryRegistrationHandler class). It’s also where it’s used.

Jan · August 6, 2019, 1:37pm

Hello again,
I removed the WatchListPlugin from /etc/xwiki/xwiki.cfg and moved away /var/lib/xwiki/data/solr.
The restart of the Tomcat server produced this logfile:
catalina.out.log (440.6 KB)

What strikes me is the message:

06-Aug-2019 13:46:22.353 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/var/lib/tomcat8/common/classes], exists: [false], isDirectory: [false], canRead: [false]
06-Aug-2019 13:46:22.360 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/var/lib/tomcat8/common], exists: [false], isDirectory: [false], canRead: [false]
06-Aug-2019 13:46:22.360 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/usr/share/tomcat8/common/classes], exists: [false], isDirectory: [false], canRead: [false]
06-Aug-2019 13:46:22.361 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/usr/share/tomcat8/common], exists: [false], isDirectory: [false], canRead: [false]
06-Aug-2019 13:46:22.362 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/var/lib/tomcat8/server/classes], exists: [false], isDirectory: [false], canRead: [false]
06-Aug-2019 13:46:22.362 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/var/lib/tomcat8/server], exists: [false], isDirectory: [false], canRead: [false]
06-Aug-2019 13:46:22.363 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/usr/share/tomcat8/server/classes], exists: [false], isDirectory: [false], canRead: [false]
06-Aug-2019 13:46:22.363 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/usr/share/tomcat8/server], exists: [false], isDirectory: [false], canRead: [false]
06-Aug-2019 13:46:22.364 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/var/lib/tomcat8/shared/classes], exists: [false], isDirectory: [false], canRead: [false]
06-Aug-2019 13:46:22.364 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/var/lib/tomcat8/shared], exists: [false], isDirectory: [false], canRead: [false]
06-Aug-2019 13:46:22.364 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/usr/share/tomcat8/shared/classes], exists: [false], isDirectory: [false], canRead: [false]
06-Aug-2019 13:46:22.370 WARNING [main] org.apache.catalina.startup.ClassLoaderFactory.validateFile Problem with directory [/usr/share/tomcat8/shared], exists: [false], isDirectory: [false], canRead: [false]

Is there maybe a problem with our Tomcat 8 installation?

ClemensRobbenhaar · August 6, 2019, 2:03pm

These directories are missing in my installation, too. I am not sure what they are supposed to contain, but this seems notto be the source of the problem for me.

For me it looks like you have XWiki installed as ROOT-webapp in /etc/tomcat8/Catalina/localhost/xwiki.xml and in /etc/tomcat8/Catalina/localhost/ROOT.xml. They seem to interfere with each other, especially as both try to recreate the solr index at the same time. The second version the fails to start properly. Maybe it helps to disable one of these context files, either the xwiki.xml (which is likely to be restored by the next update, as it is part of the Debian package) or the ROOT.xml.

Jan · August 7, 2019, 2:14pm

Hello,

closer inspection showed, that both files are symbolic links to the same file:

$ ls -l /etc/tomcat8/Catalina/localhost/xwiki.xml /etc/tomcat8/Catalina/localhost/ROOT.xml
[…] 28 Jun 25 12:12 /etc/tomcat8/Catalina/localhost/ROOT.xml -> /etc/xwiki/xwiki-tomcat8.xml
[…] 28 Jul 31 17:11 /etc/tomcat8/Catalina/localhost/xwiki.xml -> /etc/xwiki/xwiki-tomcat8.xml

I removed the latest Link and restarted Tomcat.

What can I say, it works! All menus, applications, and macros back!

Thanks again for all the help and effort!!!

ClemensRobbenhaar · August 7, 2019, 5:46pm

Glad to hear that all the stuff is back!

The problem is that the symbolic link

 /etc/tomcat8/Catalina/localhost/xwiki.xml -> /etc/xwiki/xwiki-tomcat8.xml

is created by the Debian installer, so after the next update it will be back.

Can you try if things still work if you remove the other link? Otherwise you need to remember to remove that extra link every time you update XWiki.

Did the Distribution Wizard also show up in the mean time? Otherwise you might have a slightly outdated user interface and might experience a few malfunction because of that (though I think with the current update you just might miss out on new features).