Issue with attachments after recovery from a backup

Hello

I could really use some help restoring Xwiki from a backup. In my best case approach (there were many) I got the wiki up and running, all attachments listed, but not displayed due to broken links. I seem to do things out of order, but don’t know how to fix it.

I am aware some of those steps below make no sense but I have tried many different combinations and this gave me at least something.

  1. Set up a fresh system (Ubuntu 22.04 LTS)
  2. Installed Java (apt-get install default-jdk gnupg2 -y)
  3. Installed Xwiki with Tomcat and MySQL (apt install xwiki-tomcat9-common xwiki-tomcat9-mariadb -y)
  4. Opened wiki in a browser (http://127.0.0.1:8080/xwiki)
  5. Registerd admin account (I provided the same info as in the original wiki)
  6. Installed Standard Flavor 14.10.2 with Distribution Wizard
  7. Restored database and rebooted system
mysql -e "SET FOREIGN_KEY_CHECKS=0;"
mysql -e "DROP DATABASE xwiki;"
mysql -e "CREATE DATABASE IF NOT EXISTS xwiki DEFAULT CHARACTER SET utf8;"
mysql xwiki --user=root -p < xwiki_db.sql
mysql -e "SET FOREIGN_KEY_CHECKS=1;"

At this point wiki works when it comes to pages but there are NO attachments.

  1. Removed data dir from /var/lib/xwiki/ (simple overriding was ending with error 404 later on)
  2. Make empty data dir in /var/lib/xwiki/
  3. Move data dir copied from the original xwiki to /var/lib/xwiki/ (reboot system afterwards)
  4. Went to the page: greeted by Distribution Wizard
  5. Repeat point 6 but this time there are conflicts. I choose to use auto resolve after selecting to go by the newest first, then by merge if newest is not an option

At this point wiki has pages and attachments but they are not displayed.
I also remembered I was supposed to copy WEB-INF files

  1. Copy hibernate.cfg.xml, xwiki.cfg, xwiki.properties, logback.xml, observation to /etc/xwiki/ (The originals were in /usr/lib/xwiki/WEB-INF/ but now it contains only symbolic links to files in /etc/xwiki/)

At this point wiki still has all the pages but I lost the attachments.

Please, help.

That’s not where attachment files are stored.

That’s where attachment files are stored.

You might want to take a look at https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Backup.

I know.

I know. That is why I have tried to restore the data dir in points 8-10

I did. Many times in fact. This page does not describe the process in details beside the restoration of the database. Other files are only labeled as “simply copy them at the same location where you backed them up from”, which is what I did.

I have managed to restore all the pages with attachments but they are not displayed correctly. Restoring other files (“XWiki WAR”) revered the process (attachments disappeared). Obviously the order at which restoration is done is relevant, yet not mentioned in the page you have linked.

Are you sure you have the same permanent directory configured in xwiki.properties ?

Yes. Both new and old (backed up) xwiki.properties have set environment.permanentDirectory to /var/lib/xwiki/data

It’s not very clear if the original wiki is 14.10.2 too or a much older version.

Sometimes there are things to migrate in the permanent directory (for example, the path of filesystem attachments completely changed in 11.0). The migration version handling is done with a value stored in the database and from what I understand from your step, your migrations are already executed when you restore the permanent directory.

According to my notes it was 14.3 at the installation time. I update everything whenever it is possible so I do not know what was the version at the time of backup.

From your description I am guessing the order should be to: install, move permanent data, move RAW, restore database, only then open it in a browser. Is that correct?

Ideally, the order should be:

  • move the exact same version to the new server
  • make sure it works as expected
  • upgrade XWiki

If Xwiki upgrades with simple apt upgrade then it should be 14.3 as I made the backup only few days ago. In such a case: is the order I mentioned correct?

apt upgrade is the standard way to upgrade XWiki when you are using Debian package, but I’m not sure how this is related.

If the versions are exactly the same then yes the order looks generally OK but:

  • you should stop XWiki after you installed it (the installation automatically starts it)
  • not really sure what you mean by “RAW”

I have tried once more. This time I have moved all the files before restoring the database. I still got some conflicts (although I have noticed the version is already 14.10.3). The result was a page with no attachments. I’ll try again this time stopping the service after installation.

I used the term “RAW” from Backup/Restore (XWiki.org) to call the config files ( hibernate.cfg.xml , xwiki.cfg , xwiki.properties , logback.xml , observation). Now I see I used the term incorrectly. Sorry, for the confusion.

Unfortunately stoping services did not help. Any other things I could try?

When you try to access an attachment with a missing file you should see exactly where it’s trying to access it in an error message, it should help understand where is the problem.

When I try to access an attachment it just says “The attachment does not exist.”. The overall path to the attachment’s download is http://<ip>:8080/xwiki/bin/download/Main/mypage/mysubpage/WebHome/file.jpg which is not telling much. How else can I bite this?

I don’t understand, this would suggest there is no reference to this attachment at all, are you sure the attachment is listed on the page attachments section at the bottom ?

Yes. The panel contains list of all attachments with appropriate info: file size, date, who posted it. There is no preview, downloading leads to “The attachment does not exist.”

And no error in the log ? A broken attachment usually produces a detailed error on why it’s broken.

Just opening the page cases an issue due to user avatar being an attachment too. Here is what I found in /var/log/tomcat9/catalina.out

[2023-01-13 12:53:24] [info] 2023-01-13 12:53:24,576 [http-nio-8080-exec-7 - http://192.168.50.50:8080/xwiki/bin/download/XWiki/admin/admin_avatar.jpg?width=50&height=50&keepAspectRatio=true&rev=1.1] WARN  c.x.x.p.i.ImagePlugin          - Failed to transform image attachment.
[2023-01-13 12:53:24] [info] com.xpn.xwiki.XWikiException: Error number 3002 in 3: The attachment [Attachment xwiki:XWiki.admin@admin_avatar.jpg] (file /var/cache/tomcat9/Catalina/localhost/xwiki/store/file/xwiki/a/a/ef4cf8ad02ede0bb3157f1596afc14/attachments/8/2/5edb6ddab010efcb1c23c5aed4974a/f.jpg) could not be found in the filesystem attachment store.

So that’s where it expects to find attachment files.

Note that this looks very much like the temporary directory and not the permanent one, which would suggest that either you did not configure one or it’s not readable/writable by the system user which runs Tomcat (it which case you should find a clear warning in the startup log).

  • When restarting tomcat9 the only warning I got is this:
The path attribute with value [/xwiki] in deployment descriptor [/etc/tomcat9/Catalina/localhost/xwiki.xml] has been ignored
  • /var/lib/xwiki/data had permission 755. I have stopped tomcat9 and changed to 777, After restarting tomcat9 the status showed:
jan 13 13:51:24 thehub tomcat9[2454]: The path attribute with value [/xwiki] in deployment descriptor [/etc/tomcat9/Catalina/localhost/xwiki.xml] has been ignored
jan 13 13:51:27 thehub tomcat9[2454]: 2023-01-13 13:51:27,880 [main] INFO  iPropertiesConfigurationSource - Loading [xwiki.properties] from default location [/etc/xwiki/xwiki.properties]
jan 13 13:51:28 thehub tomcat9[2454]: 2023-01-13 13:51:28,055 [main] INFO  o.x.e.i.ServletEnvironment     - Using permanent directory [/var/lib/xwiki/data]

but resulted in HTTP Status 500. Restoring to 755 and reset fixes the issue

  • In /etc/xwiki/xwiki.properties I find environment.permanentDirectory = /var/lib/xwiki/data

At no step I have selected the location for permanent directory, always going with the defaults. Is there some other place I configure the location of the permanent directory?