Error starting xwiki after restoring from backup (please help!)

I am trying to restore my xwiki installation from a backup. I first reinstalled postgresql, installed xwiki-common xwiki-pgsql-common xwiki-tomcat8-common xwiki-tomcat8-psql, re-read the save of the database back into psql, restored all files under the xwiki and tomcat8 directories I could find:

/etc/tomcat8
/var/lib/tomcat8
/var/lib/xwiki
/usr/share/tomcat8
/usr/share/tomcat8-root
/usr/share/xwiki
/etc/xwiki
/usr/lib/xwiki
...

Now, I’m getting the following and I’ve spent 2 days looking around at comparisons of my backup with the new installation, making sure all permissions and ownerships are the same. I also made sure all “.jar” files on the system are not corrupted. I don’t see enough information from this error message to really tell what’s wrong. Do anyone have a clue? I’m quickly running out of things to try!

Any help would be greatly appreciated!

25-Aug-2018 20:27:44.321 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDescriptor Deploying configuration descriptor /etc/tomcat8/Catalina/localhost/xwiki.xml
25-Aug-2018 20:27:44.599 SEVERE [localhost-startStop-1] org.apache.catalina.core.ContainerBase.addChildInternal ContainerBase.addChild: start:
 org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/xwiki]]
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:153)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:725)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:701)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:585)
        at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1794)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.catalina.LifecycleException: Failed to start component [org.apache.catalina.webresources.StandardRoot@31d6cbd5]
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:153)
        at org.apache.catalina.core.StandardContext.resourcesStart(StandardContext.java:4937)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5067)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
        ... 10 more
Caused by: org.apache.catalina.LifecycleException: Failed to initialize component [org.apache.catalina.webresources.JarResourceSet@15e905ff]
        at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:136)
        at org.apache.catalina.webresources.StandardRoot.startInternal(StandardRoot.java:699)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
        ... 13 more
Caused by: java.lang.IllegalArgumentException: java.util.zip.ZipException: error in opening zip file
        at org.apache.catalina.webresources.JarResourceSet.initInternal(JarResourceSet.java:96)
        at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
        ... 16 more
Caused by: java.util.zip.ZipException: error in opening zip file
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:225)
        at java.util.zip.ZipFile.<init>(ZipFile.java:155)
        at java.util.jar.JarFile.<init>(JarFile.java:166)
        at java.util.jar.JarFile.<init>(JarFile.java:103)
        at org.apache.catalina.webresources.JarResourceSet.initInternal(JarResourceSet.java:88)
        ... 17 more

25-Aug-2018 20:27:44.613 SEVERE [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDescriptor Error deploying configuration descriptor /etc/tomcat8/Catalina/localhost/xwiki.xml
 java.lang.IllegalStateException: ContainerBase.addChild: start: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/xwiki]]
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:729)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:701)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:585)
        at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1794)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

25-Aug-2018 20:27:44.615 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDescriptor Deployment of configuration descriptor /etc/tomcat8/Catalina/localhost/xwiki.xml has finished in 294 ms
25-Aug-2018 20:27:44.617 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory /var/lib/tomcat8/webapps/ROOT
25-Aug-2018 20:27:45.447 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.

This is what seems to be the error. In other words your restore has some invalid JAR files. You need to find which ones by checking their validity with, for example, zip -Tv.

That’s what I thought, but I ran “find / -name *.jar -ls” to find every jar file on the system, then ran “zip -T” on each one and every one verified as fine. I was wondering if maybe the error may have to do with the directory into which the zip is unzipping the file. Maybe that directory doesn’t have correct permissions, but I don’t know what directory that would be.

I’ll re-verify the jar files yet again… done. no bad .jar files found.

Ok, I thought maybe these “dot-underscore” files were the cause of the problem, but they’re not really .jar or .zip files. They just contain text. So… at this point I see no .jar files which don’t begin with “dot-underscore” having problems. I’m reinstalling xwiki on another system and it works there. I’ll try to verify all ownerships and permissions, etc., between the two.

I have now reinstalled xwiki with postgresql on a new server and after installing it, I updated the xwiki DB with the original data and copied everything under /var/lib/xwiki/data/storage from the original system. I don’t even see the wiki’s which I had originally! How do I restore xwiki files and database to a newly installed system?

I read restored everything under /var/lib/xwiki/data, restored the xwiki postgress database, restored under /usr/lib/xwiki:
hibernate.cfg.xml
xwiki.cfg
xwiki.properties
observation/*

I now see some errors loading extensions which I had previously but have not loaded now. I would expect to be able to see my original wiki’s in the “wiki index”, but I see no mention of them. Perhaps I really MUST have all of the extensions loaded or they fail anyway? I’ll try loading them.

I also see mentions of my previous wiki pages in the history but when I click on them it says it can’t find them!

Ok… So after days of learning about the internals of xwiki, the java code, the file layout, and the postgres db, I can finally answer my own question!

The hint was the mention of “dot-underscore” files.

When you save files (with rsync, in my case, to a MacOS filesystem, MacOS creates “resource forks” for all of the files. This is for saving MacOS resource information; something which unix doesn’t have and thus there’s no place to put it in a unix style filesystem. Thus, MacOS creates files beginning with dot and underscore, and otherwise named the same as the original files.

When these files are tar’ed up and copied back to a unix filesystem, you end up with a bunch of hidden files beginning with dot and underscore. When Java tries to load all of the “.jar” files in a directory, it also ends up trying to load the dot underscore files. These are not java files, however. They’re resource fork files.

The solution was to look for all files in the restored filesystem which begin with dot underscore and remove them. You have to use “rm -rf” because some are directories.

Xwiki now works just fine! :slight_smile:

Glad it works and thanks for posting back!