Data migration from xWiki 12.9 on host to 14.7 in docker

mariusbmg · September 15, 2022, 10:42am

Hello everybody!

I have to migrate the data from an old X-Wiki 12.9 instance installed on linux host to new instance version 14.7 in docker ;
For that I was running all these steps, gathered from all kind of sources, it went well till Phase 1 (same version in docker as software installed on host) but, it failed with bellow errors when tried to upgrade through latest docker push;
Pls guide me through missing, wrong, unnecessary or incorrectly ordered steps:

Right order of data migration from source into target server:
#############################################################

Deploy the empty dockers for xwiki-web & xwiki-db having the exact version as the source:
Source server:
- XWIKI_VERSION=12.9
- JAVA_VERSION=1.8.0_131
- TOMCAT_VERSION=8.0.14
- MARIADB_VERSION=10.5.15

Phase1. Target (same versions as source in dockers):
- XWIKI_VERSION=12.9
- JAVA_VERSION=jdk-11.0.9
- TOMCAT_VERSION=8.5.60
- MARIADB_VERSION=10.5.16
- MYSQL_JDBC_VERSION=8.0.21

Phase2. Target (final version dockers):
- XWIKI_VERSION=14.7
- JAVA_VERSION=jdk-11.0.16.1+1
- TOMCAT_VERSION=9.0.65
- MARIADB_VERSION=10.9.2
- MARIADB_JDBC_VERSION=3.0.7

On new docker, install all extensions\flavors\skins, grant admin rights for required users, existent in old env:

##ERR:
2022-09-07 19:21:09,632 [XWiki initialization] INFO  a.i.BridgeAuthorizationManager - [admin] access has been denied for user [xwiki:XWiki.hagbeaca] on [xwiki]: security checkpoint
2022-09-07 19:21:09,633 [XWiki initialization] ERROR DocumentTranslationBundleFactory - Failed to register translation bundle from document [xwiki:IconThemesCode.Translations]
org.xwiki.security.authorization.AccessDeniedException: Access denied when checking [admin] access to [Wiki xwiki] for user [xwiki:XWiki.hagbeaca]
        at org.xwiki.security.authorization.DefaultAuthorizationManager.checkAccess(DefaultAuthorizationManager.java:104)
        at org.xwiki.security.authorization.internal.BridgeAuthorizationManager.checkAccess(BridgeAuthorizationManager.java:53)
        at org.xwiki.localization.wiki.internal.DocumentTranslationBundleFactory.checkRegistrationAuthorization(DocumentTranslationBundleFactory.java:442)
        at org.xwiki.localization.wiki.internal.DocumentTranslationBundleFactory.registerTranslationBundle(DocumentTranslationBundleFactory.java:418)
        at org.xwiki.localization.wiki.internal.DocumentTranslationBundleFactory.loadTranslations(DocumentTranslationBundleFactory.java:233)
        at org.xwiki.localization.wiki.internal.DocumentTranslationBundleFactory.initialize(DocumentTranslationBundleFactory.java:207)
        at org.xwiki.component.embed.InitializableLifecycleHandler.handle(InitializableLifecycleHandler.java:39)
        at org.xwiki.component.embed.EmbeddableComponentManager.createInstance(EmbeddableComponentManager.java:365)
        at org.xwiki.component.embed.EmbeddableComponentManager.getComponentInstance(EmbeddableComponentManager.java:451)
        at org.xwiki.component.embed.EmbeddableComponentManager.getInstance(EmbeddableComponentManager.java:201)
        at org.xwiki.component.embed.GenericProvider.getInstance(GenericProvider.java:111)
        at org.xwiki.component.embed.GenericProvider.get(GenericProvider.java:100)
        at org.xwiki.localization.wiki.internal.DocumentTranslationBundleInitializer.onEvent(DocumentTranslationBundleInitializer.java:82)
        at org.xwiki.observation.internal.DefaultObservationManager.notify(DefaultObservationManager.java:320)
        at org.xwiki.observation.internal.DefaultObservationManager.notify(DefaultObservationManager.java:285)
        at com.xpn.xwiki.internal.XWikiInitializerJob.runInternal(XWikiInitializerJob.java:131)
        at org.xwiki.job.AbstractJob.runInContext(AbstractJob.java:242)
        at org.xwiki.job.AbstractJob.run(AbstractJob.java:219)
        at java.base/java.lang.Thread.run(Thread.java:834)

Transfer\adjust all variables from old env to match the target env from:

- xwiki.cfg, xwiki.properties, hibernate.cfg.xml, logback.xml

$ vi xwiki.properties |replace environment.permanentDirectory value with new docker store path

 environment.permanentDirectory = /var/lib/xwiki/data <-- /usr/local/xwiki/data 
 
 Using Solr home directory: [/usr/local/xwiki/data/store/solr] 

- Go to http://<yourwiki>/xwiki/bin/edit/XWiki/XWikiPreferences?editor=object
- Click on the line that says XWikiPreferences 0 (right below the line that says Objects of type XWiki.XWikiPreferences (1))
- Scroll down to the field that says Maximum Upload Size and change the number to whatever size you want (it is expressed in bytes)
- Scroll to the bottom and click "Save"

$ vi xwiki.cfg
---------------
#-# LDAP authentication service
xwiki.authentication.authclass=org.xwiki.contrib.ldap.XWikiLDAPAuthServiceImpl

#-# Turn LDAP authentication on - otherwise only XWiki authentication
#-# - 0: disable
#-# - 1: enable
#-# The default is 0
xwiki.authentication.ldap=1

Import mariadb sql dump & change the server name:
- Change the server name:
  The “table.column” I have identified are :
  
  activitystream_events.ase_url
  xwikidoc.xwd_content
  xwikilargestrings.xwl_value
  xwikilinks.xwl_link
  xwikipreferences.xwp_admin_email
  xwikircs.xwr_patch
  xwikirecyclebin.xdd_fullname
  xwikistrings.xws_value
If new pages are edited since last mariadb sql dump then, export all xwiki modified objects as .xar & Import .xar archive in target server:
Migrate old content folder:
Migrating/archive old content from previous wiki - #6 by vmassol
- Check Database Attachment Store:
  
  The default before 10.5.
  
  This attachment storage mechanism stores your attachments in database entries in the xwikiattachment_content, xwikiattachment_archive and xwikiattrecyclebin tables.
  This system allows for easy backup of your attachments by dumping the database and keeping all of your data together, but attachment size is memory constrained since the attachment content and archive must all be held in memory. As a general rule, attachments larger than 30MB are not possible.
  Switch to database attachment store
  
  These settings should read as follows:
  xwiki.store.attachment.hint = hibernate
  xwiki.store.attachment.versioning.hint = hibernate
  xwiki.store.attachment.recyclebin.content.hint=hibernate
  
  If you need to use database store for the attachment it’s probably true for deleted attachments
  
  xwiki.store.recyclebin.content.hint = hibernate
  
  When using this attachment store with a MySQL database, you must set the max_allowed_packet to about 3 times the size of your largest attachment since the attachment and its version history must be saved. See the MySQL Installation guide for more information.
  
  root@hagux013 /etc/xwiki# grep -r xwiki.store.attachment ./*
  ./xwiki.cfg:# xwiki.store.attachment.hint=hibernate
  ./xwiki.cfg:# xwiki.store.attachment.versioning.hint=hibernate
  ./xwiki.cfg:# xwiki.store.attachment.recyclebin.content.hint=hibernate
  ./xwiki.cfg:# xwiki.store.attachment.recyclebin.hint=hibernate
  ./xwiki.cfg:# xwiki.store.attachment.versioning=1
  ./xwiki.cfg.ucf-dist:# xwiki.store.attachment.hint=file
  ./xwiki.cfg.ucf-dist:# xwiki.store.attachment.versioning.hint=file
  ./xwiki.cfg.ucf-dist:# xwiki.store.attachment.recyclebin.content.hint=file
  ./xwiki.cfg.ucf-dist:# xwiki.store.attachment.recyclebin.hint=hibernate
  ./xwiki.cfg.ucf-dist:# xwiki.store.attachment.versioning=1
  
  ##ERR:
  2022-09-13 08:11:58,759 [XWiki Solr index thread] ERROR .DocumentSolrMetadataExtractor - Failed to retrieve the content of attachment [Attachment xwiki:Sandbox.TestPage1@image-20220825151226-1.jpeg]
  com.xpn.xwiki.XWikiException: Error number 3002 in 3: The attachment [Attachment xwiki:Sandbox.TestPage1@image-20220825151226-1.jpeg] (file /usr/local/xwiki/data/store/file/xwiki/b/5/24e18aa4b9579149fb1597d4edeef9/attachments/d/6/ea3436c7ef78aa40c2e342fee89aee/f.jpeg) could not be found in the filesystem attachment store.
  at org.xwiki.store.legacy.store.internal.FilesystemAttachmentStore.loadAttachmentContent(FilesystemAttachmentStore.java:214)
  at com.xpn.xwiki.doc.XWikiAttachment.loadAttachmentContent(XWikiAttachment.java:1094)
  at com.xpn.xwiki.doc.XWikiAttachment.reloadAttachmentContent(XWikiAttachment.java:917)
  at com.xpn.xwiki.doc.XWikiAttachment.getContentInputStream(XWikiAttachment.java:895)
  at org.xwiki.search.solr.internal.metadata.AbstractSolrMetadataExtractor.getContentAsText(AbstractSolrMetadataExtractor.java:506)
  at org.xwiki.search.solr.internal.metadata.DocumentSolrMetadataExtractor.setAttachment(DocumentSolrMetadataExtractor.java:281)
  at org.xwiki.search.solr.internal.metadata.DocumentSolrMetadataExtractor.setAttachments(DocumentSolrMetadataExtractor.java:261)
  at org.xwiki.search.solr.internal.metadata.DocumentSolrMetadataExtractor.setExtras(DocumentSolrMetadataExtractor.java:187)
  at org.xwiki.search.solr.internal.metadata.DocumentSolrMetadataExtractor.setFieldsInternal(DocumentSolrMetadataExtractor.java:135)
  at org.xwiki.search.solr.internal.metadata.AbstractSolrMetadataExtractor.getSolrDocument(AbstractSolrMetadataExtractor.java:133)
  at org.xwiki.search.solr.internal.DefaultSolrIndexer.getSolrDocument(DefaultSolrIndexer.java:499)
  at org.xwiki.search.solr.internal.DefaultSolrIndexer.processBatch(DefaultSolrIndexer.java:408)
  at org.xwiki.search.solr.internal.DefaultSolrIndexer.run(DefaultSolrIndexer.java:376)
  at java.base/java.lang.Thread.run(Thread.java:834)
Run xwiki ver. 12.9 & mariadb 10.5 dockers :
Test & validate the environment by:
- using known list of operations done on environment
- check data availability

After receiving validation OK, the pilot docker env will be upgraded to latest, by running xwiki ver. 14.7 & mariadb 10.9 dockers; (tried with xwiki:13.10.9-mariadb-tomcat or xwiki:latest dockers but errors were the same) :

##ERR:
2022-09-13 12:47:40,171 [main] INFO  o.x.e.i.ServletEnvironment     - Using permanent directory [/usr/local/xwiki/data]
2022-09-13 12:47:43,250 [main] WARN  ltInstalledExtensionRepository - Invalid extension [org.xwiki.platform:xwiki-platform-distribution-flavor-mainwiki/12.9] on namespace [wiki:xwiki] (InvalidExtensionException: Dependency [org.xwiki.platform:xwiki-platform-oldcore-[12.9]] is incompatible with the core extension [org.xwiki.platform:xwiki-platform-legacy-oldcore/13.10.9])
2022-09-13 12:47:43,253 [main] WARN  ltInstalledExtensionRepository - Invalid extension [org.webjars.npm:fast-diff/1.2.0] (InvalidExtensionException: Extension [org.webjars.npm:fast-diff/1.2.0] already exists as a core extension)

2nd testing phase should be run, to validate the latest XWIKI_VERSION
Mounting the ssl ca-signed certificate and enabling the reverse-proxy to secure the traffic

Data migration from xWiki 12.9 on host to 14.7 in docker

If you need to use database store for the attachment it’s probably true for deleted attachments