Confluence Migration: attachments show filesize of 0 bytes after import

Hello everyone.

We are currently testing the migration of our wiki from Confluence to XWiki.

  • XWiki Version 17.6.0
  • Confluence XML 9.88.3
  • Confluence Migrator Application (Pro) 1.35.5

We experience a problem with attachments. After migrating a confluence space, the majority of attachments (but not all, even on the same document) show up with a filesize of “0 bytes” and no mime type in the attachment section:

The attachment itself is present on disk and thumbnails, viewing, downloading, … works just fine. From a functionality standpoint, everything works as expected. The migration logs also do not show anything suspicious at first glance. Therefore, I thought this was just a display bug and looked into the xwikiattachment database table:

All columns look good, but the filesize and mime type fields are actually empty for the effected attachments - but only for the current revisions. All older revisions do have mime types and file sizes:

Given that

  • not all attachments in a confluence space are effected
  • only the newest revision of attachments is effected
  • everything works fine with the effected attachments
  • no obvious errors are logged
  • I did not find any other posts/issue regarding this topic

I suspect this to be a subtle bug during the migration. I guess you would not be able to immediatly present a solution, so I would be happy if you could point me to where I could dig more into the problem. Maybe someone knows where in the code the missing data is supposed to be read and subsequently written to the database so I could debug that? I already tried debugging but the code for the migration is rather complex so some tips would be appreciated.

I guess a good starting point would be confluence/confluence-xml/src/main/java/org/xwiki/contrib/confluence/filter/internal/input/ConfluenceInputFilterStream.java at master · xwiki-contrib/confluence · GitHub (which is in charge of finding the actual file corresponding to a given attachment found in a page content).

You might also want to check the log of the import for any related warning/error.

Hi @grimmksg,

Another thing to check would be:

  • if the attachments are okay in Confluence
  • if the attachments are okay in your export package

Hello @tmortagne and @rjakse

I did check the files in the zipped confluence export and after extraction during the migration process, everything looks fine. The log does not show any warning or error that I could point to the effected documents or attachments.

I debugged into fillAttachment in XWikiAttachmentOutputFilterStream.java where the attachment information is processed and subsequently attached to the document. The filesize is present and correct at this point, so the issue appears not to be within the reading of the information.

I then tried some other things and realized something: Refreshing the attachments table does not change anything, but when I attach a new file to a document, the filesize sometimes suddenly starts to show up in the UI. It does not matter if it is a new file or a new revision of one of the already present files - after the upload the correct file size is showing for all attachments. It also (sometimes) works when you delete an attachment from the document. However, the longsize database field is still empty for these attachments?!

My questions would be:

  1. Where is the file size for migrated attachments actually coming from if not from the database?
  2. Could this issue actually be a “display problem” caused by caching/refreshing rather than an technical issue with the attachment logic? If so, why are there cases where it works for some attachments and does not work for other attachments on the same document?

Hi @grimmksg,

When importing a Confluence package, we read the size of the attachments from the confluence XML package if present (otherwise we deduce it from the file itself).

Maybe something is wrong with this mechanism and we end up sending the wrong attachment size?

If you want to further debug:

That happens in method getAttachmentInfo of ConfluenceInputFilterStream.

        long attachmentSize = attachmentContentProperties.getLong(ConfluenceXMLPackage.KEY_ATTACHMENT_CONTENT_FILESIZE,
            -1);
        if (attachmentSize == -1 && contentFile != null) {
            attachmentSize = contentFile.length();
        }

BTW, I’m starting to wonder if we shouldn’t just use the actual file size and completely ignore the value from the Confluence package, or issue a warning in case of mismatch with the actual file. Nothing good can be brought by a potential mismatch.

And then, we send it to the output filter stream in the readAttachments method

                proxyFilter.onWikiAttachment(attachmentName, fis, a.size, a.parameters);

or the readAttachmentRevision method:

        proxyFilter.beginWikiAttachmentRevision(Long.toString(a.revision), fis, a.size, a.parameters);
        proxyFilter.endWikiAttachmentRevision(Long.toString(a.revision), fis, a.size, a.parameters);

If you manage to reproduce something wrong in this code, a minimal working example would be nice.

Another place that could be interesting to debug, on “write side” this time, is xwiki-platform/xwiki-platform-core/xwiki-platform-oldcore/src/main/java/com/xpn/xwiki/internal/filter/output/XWikiAttachmentOutputFilterStream.java at master · xwiki/xwiki-platform · GitHub.

Most probably. Since the Confluence “export package” is mostly a database dump, I assume the point of the size stored in there is to allow query this information and not really to force a size which is different from the actual file.

Another place that could be interesting to debug, on “write side” this time, is xwiki-platform/xwiki-platform-core/xwiki-platform-oldcore/src/main/java/com/xpn/xwiki/internal/filter/output/XWikiAttachmentOutputFilterStream.java at master · xwiki/xwiki-platform · GitHub .

I already debugged both the InputFilterStream and OutputFilterStream (see my post before). attachment.setLongSize(size) is indeed called with the correct size present. I did not find anything suspicious during debugging or any difference to attachments where filesize works in the UI.

Given the observation that the filesize for the other attachments sometimes suddently appears in the UI when uploading/deleting an attachment, I suspect the issue to be caused by something else.

Ha, sorry, I read too quickly. What is sure is that whatever is the problem is on XWiki Standard side if you get the right size in there.

This definitely looks like a bug to me. That table column should never be empty. Don’t hesitate to create a bug issue on Loading... if you find a simple way to reproduce.

Alright. I have done lots of testing today but I am unable to reproduce the issue reliably :expressionless_face: I looked for a document that did have the attachment issue: 1 file shows a size in UI and database, 2 files show a size in UI but the database field is empty (?!?), and 3 files show no filesize.

I placed a breakpoint in XWikiHibernateStore.java where the attachment is actually written to the database and of course, it magically does work on every attempt :woozy_face:

Sure, will do if I manage to reproduce it reliably…