Upgrading from 15.10.2 to 16.9.0 - using large amounts of disk space

Hi everyone! We are in the process of upgrading from XWiki 15.10.2 to 16.9.0. We have tried running the migration both locally and on a VM, and in both cases we are finding that the directory we’ve assigned to the environment.permanentDirectory property in the xwiki.properties file is growing to a huge size during the migration. During the last attempt at the migration, we started with an empty directory and let XWiki recreate everything. The directory grew to 35GB+. Specifcally, the /jobs folder took up the majority of the space.

Can someone help me determine if we’re doing something wrong? Is there a configuration that we can turn off to prevent this folder from growing to be so huge? I’m at the point where I need to get an external hard drive to run this on my local computer, as our VMs do not come equipped with this kind of storage space.

Thank you!

This folder is mostly containing background jobs log. Do you have mode details on what exactly takes all that space in there (i.e. which job is going nuts) ?

Hi tmortagne,

The /jobs/status/solr/indexer/log.xml file is the file that is growing to be huge. I left the XWiki migration running overnight and that log.xml file took up an entire 128GB hard drive. Do you know if there is a way I can decrease the number of logs or turn logging off during the migration? Or do you know why this may be happening?

Thank you!

This file is really not supposed to be that huge. For me, it only contains one log summarizing the documents missing in the index, etc. and 2 logs indicating the beginning/end of the job. It’s also not supposed to be any different from the one in 15.10.2.

It would be interesting if you could extract from there a few <org.xwiki.logging.event.LogEvent> entries which are not one of those 3 logs I described above (be cafefull to remove anything sentitive from there). Is it possible that you enabled some debug log ?