Upgrade 10.1 -> 10.4 - failed to store deleted attachment

Help / Discuss
1

Hi all

We’ve upgraded xwiki from 10.1 to 10.4 recently, and we ran into problems with the attachments. Some details:

  • Everything is stored in the DB. Everything. Attachments too.
  • A fair number of pages and their attachments are pushed from our system to XWIKI using the REST API
  • In order to avoid our DB to get flooded by attachments, I disabled versioning and run a DELETE before I PUT
  • After the upgrade, the attachments disappeared (only the pushed ones)

Investigating on the issue I figured that there were changes in xwiki.cfg. My current (non-working) config looks like this:

xwiki.store.recyclebin.content.hint=hibernate
xwiki.store.attachment.hint=hibernate
xwiki.store.attachment.versioning.hint=hibernate ## was set to void before
xwiki.store.attachment.recyclebin.content.hint=hibernate
xwiki.store.attachment.recyclebin.hint=hibernate
storage.attachment.recyclebin=1
xwiki.store.attachment.versioning=0

I noticed a few things:

  • storage.attachment.recyclebin doesn’t follow the naming conventions
  • There seem to be two spots to disable versioning:
    • Setting the versioning hint to void
    • Setting attachment versioning to 0

Now, what happens is a bit strange. We first noticed that attachments didn’t get uploaded. I checked the logs and saw an error 500 in response to the DELETE call:

Response: com.xpn.xwiki.XWikiException: Error number 0 in 3: Failed to store
  deleted attachment [Attachment xwiki:105 Module Descriptions.LCR.LCR     
  Views.WebHome@A01240_00423.jpg]

However, the attachment was uploaded correctly. While playing around with the cfg I found that the attachment disappear after I touch web.xml to bounce the Wiki. They’re gone completely, vanished. When I push the upload again, they reappear and stay there until I bounce the web app the next time.

Simply put: I’d like to keep everything in the DB and turn off versioning for the attachments. What’s wrong with my settings?

Update
I tried to delete one of the disappeared attachments manually, just to see whether the exception comes up, too. It didn’t. I started the automatic upload again and it worked without error message. All subsequent automatic uploads worked now.

It seems that there are attachments which aren’t real attachments, sort of. Here’s a screenshot:

2018-05-25_16h10_09

The first one (A01241_00415.jpg) is the one that survives a restart and a re-upload. The others first complain with the 500 error, then appear correctly until I bounce the application.

How do I find attachments which are in such an invalid state?

Thanks a ton in advance!

André

2

You mixed several versions of the configuration here. In the standard xwiki.cfg you have two sections:

  • properties that can be set to either hibernate (old default) or file (new default)
  • properties which only support hibernate in XWiki Standard which you usually don’t touch (unless of course you provide a different implementation for them)

xwiki.store.attachment.recyclebin.hint is always hibernate now even when using filesystem attachments (the metadata are always stored in the database).

storage.attachment.recyclebin is enabled by default already so no need to set it.

History I guess. In practice they have the same effect or to be more accurate xwiki.store.attachment.versioning=0 automatically set the versioning hint to void which is what really cancel versioning (since VoidAttachmentVersioningStore simply does not do anything).

The complete error would be nice to understand why it could not store it.

It’s not very clear to me what this means exactly.

The settings looks OK.

Hard to tell without knowing what is this invalid state. Without a way to reproduce this to debug it I’m afraid I don’t have much idea.

3

Hi Thomas

Thanks for that fast response.

Hard to tell without knowing what is this invalid state. Without a way to reproduce this to debug it I’m afraid I don’t have much idea.

The “invalid” attachments look like this:

2018-05-28_09h17_35

I believe I will find them by going through XWIKIATTACHMENT and XWIKIATTACHMENT_CONTENT. That particular one exists in the first table but doesn’t have anything in the content BLOB.

What I tried now is the following (still having the same config as in my original post):

  1. Get the XWA_ID of that particular attachment:

     select * from xwikiattachment a where XWA_FILENAME = 'A01245_00426.jpg';

  2. I suppose this should match the XWA_ID of XWIKIATTACHMENT_CONTENT:

     select count(*) from xwikiattachment_content where XWA_ID = 1224816789;

    That one returns 0, which explains the dummy jpg.

  3. I re-upload the file using REST and tadaa! It reappears:

    2018-05-28_09h35_01
    2018-05-28_09h35_01716×271 56.9 KB

  4. In the DB, I would expect it to be there, too, but it’s not. The above count(*) still returns 0, which could explain the outcome of the next step.

  5. In Linux, I visit the web applications WEB-INF directory and type touch web.xml, which is a soft way to tell tomcat to reload the web application. After that, the jpg is gone again and the placeholder shows up.

To reproduce, I think you could:

  1. Delete one XWIKIATTACHMENT_CONTENT record to get the status quo I have

  2. Send a “DELETE” request via REST, which will probably show something like this:

    Response: com.xpn.xwiki.XWikiException: Error number 0 in 3: Failed to store deleted attachment [Attachment xwiki:105 Module Descriptions.LCR.LCR Views.Code.WebHome@A01245_00426.jpg]

  3. Upload the attachment using REST

  4. Check if it’s there (it is, here at least), and check the DB

  5. Restart the web application

  6. Check again

Would you think it’s safe to delete (SQL wise) and re-upload all attachments which have no BLOB in the content table? I’ll try that right now for that single jpg.

Cheers

André

4

Hi again

It seems that something got messed up here. XDD_CONTENTSTORE is a wild mix of NULL, hibernate and file. I’ll clean up and do some re-uploading first. I think I will close that thread afterwards.

Thanks for your help! The confirmation of my config is already worth a lot!

Cheers

André

5

This is what I was about to ask you. Having a mix is technically OK (XWiki now support multi storage for attachments as well as deleted documents) but from what I understand about your use case you are supposed to only have hibernate or empty which are the same thing. Maybe you upgraded with the default (so some attachment ended up in the file store) and then changes the configuration.

Note that this is not a great idea with XWiki since there is still a few things that require a full restart and you might end up with unexpected issues when doing just a reload.

1 Like
6

I believe this is the reason. I need to be more careful while upgrading …

PEBKAC :wink:

Cheers

André