Move installed/local extensions to blob storage

[MichaelHamann]

@vmassol

Does it support all the S3 API we use? See S3 Compatibility status | Garage HQ

Has someone tested XWiki on it?

Motivated by this discussion, I’ve just tested it and the answer is unfortunately, no, it doesn’t support conditional writes which we’re currently using. I would need to check how much we’re actually relying on this, though. Additionally, there is a need for another configuration option, but that’s not really a blocker.

[tmortagne]

@vmassol

Then restart XWiki. So what’s the difference with bundling it by default?

With a XIP, it’s still an installed extension with the exact same constraints, you just skipped the download step (step which never been mentioned as a problem here).

@vmassol

If S3 is not installed, why would you need to have S3 to scan existing extensions?

I don’t understand the question. If you don’t configure s3, then sure, you don’t need s3, but that’s not the problem we are discussing here.

@vmassol

if you install S3 and enable it then a migration could move the extension metadata stored on the FS to S3.

OK, let’s test that then, I guess it will be more clear.
You decided that you wanted to use s3 as blob storage:

• it’s not in the WAR, so you go to the admin → Extensions and install the extension xwiki-commons-store-blob-s3
• you stop XWiki
• in xwiki.properties you configure store.blobStoreType and switch it to s3
• you restart XWiki
• InstalledExtensionManager component initializes and ask BlobStoreManager for the BlobStore with name “extension”
• BlobStoreManager ask the ComponentManager for the component BlobStoreFactory with the configured hint (s3)
  → Can't find descriptor for the component with type [org.xwiki.store.blob.BlobStoreFactory] and hint [s3]

That’s because that component is provided by an installed extension, which you did not load yet, because you are still trying to access the list of installed extensions, which, according to the configuration, is stored in s3. Not sure why you focus so much in the migration, it’s just something that happen when the blob store notice that there seems to still be stuff in the previous store, and move them to the new one. This is a simple cross dependency problem, and you cannot really do much about it conceptually.

[vmassol]

Thanks for the steps, that helps understand the problem.

@tmortagne

That’s because that component is provided by an installed extension, which you did not load yet, because you are still trying to access the list of installed extensions, which, according to the configuration, is stored in s3.

ok I understand now. The issue is that the list of installed extensions is located in the configured blob store.

I can imagine several ways to work around this conceptually:

1. Provide a solution to manually install a core extension in XWiki.
2. (Similar to 1) but simpler for users) Provide a way in the EM to install an extension as a core extension.
3. Put the list of installed extensions in a blob store that is always available (e.g. FS) so that the xwiki-commons-store-blob-s3 extension can be found as installed, and if we need to distribute that list of installed extensions, have a process to sync what is in the local blob store to the configured blob store so that it’s distributed.

You have the same problem when you develop a microkernel. If we were to recode XWiki from scratch and have a microkernel containing only a headless Extension Manager, and the rest of the features would be extensions, we would need to develop some strategies like the ones I’ve mentioned above.

BTW I find the reliance on the S3 blobstore fragile. Imagine that the network has a problem (or a firewall config changes or you’re on a trip with no internet). Suddenly your local instance doesn’t work anymore (even if you’re the only XWiki node in the topology) because the list of installed extensions cannot be retrieved. I’d find more stable to have that list located in several places (in the configured blob store + on the local FS for ex), so that XWiki can fallback to the safe local store in case of problem. And have some sync between the two.

I understand it’s more complex to implement.

That’s why option 1) (or 2), a bit more complex) could also be good enough for now seen the number of users who’ll need to configure XWiki with a S3 blobstore.

I don’t want to block this, so +0 fro me, but I don’t like to include by default technologies that very few users will need, especially when we’re trying to reduce the size and scope of XWiki Standard.

Thanks

[tmortagne]

@vmassol

Provide a solution to manually install a core extension in XWiki.

It’s not very clear what you mean here. Create a cli tool to download an extension JAR and its dependencies and put them in the WEB-INF/lib folder ?

@vmassol

(Similar to 1) but simpler for users) Provide a way in the EM to install an extension as a core extension.

Having write right in that area (and provided the WAR is expanded) would imply a very bad setup from security point of view.

[vmassol]

@tmortagne

It’s not very clear what you mean here. Create a cli tool to download an extension JAR and its dependencies and put them in the WEB-INF/lib folder ?

Yes. Either download or even more manually take a XIP as input and copy its content to the right places. Ofc it would need to be a XIP for a given version of XWiki to avoid dependency version conflicts.

@tmortagne

Having write right in that area (and provided the WAR is expanded) would imply a very bad setup from security point of view.

Yes, it wouldn’t be inside the WAR since that’s a no go. It would be somewhere on the filesystem.

[tmortagne]

@vmassol

Yes, it wouldn’t be inside the WAR since that’s a no go. It would be somewhere on the filesystem.

So what you suggest is having two types of installed extensions basically. But then you get back the same problem in a cluster that the local type is not synchronized/backuped properly.