Truely offline XIP package for xwiki flavors

Help / Discuss
,
#1

In the installation documentation, the Installing without internet connection / XIP package section is deceiving. It claims that the provided XIP file is an “offline extension repository with all the extensions making the standard flavor”. The downloadable XIP package is not exhaustive and some needed components still require an internet connection to be fetched.

Could a true “no internet needed” XIP package be built for each xWiki release ? Yes, I know, with maven it’s possible to attain the “really offline install” goal… but after having read a lot of documentation. It’s not intuitive at all and time consuming.

PS: I witnessed this with the 14.8 XIP package. It could have been corrected in 14.9 or 14.10 release. If so, great.

#2

That’s definitely a bug and many people were able to use it without internet connection AFAIK, could you detail a bit more the problem you have.

#3

Let me detail our context. To migrate a Debian (stable) server from an old 11.x version of xWiki to a current one, we chose to…

  1. backup our pages

  2. uninstall everything (data, mariadb, tomcat)

  3. reinstall from Debian package “xwiki-tomcat9-mariadb” (at this point it’s a completely fresh xwiki Debian install)

  4. customize Tomcat configuration files as specified in xWiki install documentation

  5. finalize the flavor install from the web interface and

  6. restore the previously saved data

This production server have a clone to test our upgrade procedures. We’re stuck on step 5 on this first test server. On another clone with unrestricted and speedy internet access, the flavor install works as expected. On our first test server, with a much slower but still unrestricted internet access, the flavor install seems to time out. Since the production server does not have any direct internet access, we thought about XIP package. The flavor install seems a little bit speedier, with 80% of the packages explicitely marked as installed from local source. Still, it hangs too.

Important detail : our test have been done with 14.8 Debian xwiki packages and the 14.8 XIP package to install the flavor.

To check if internet access is necessary, we monitored our actions with a TCPdump background process. We analyzed afterwards the resulting capture file with Wireshark. It appears that a lot of http/https requests are still made to xwiki main Maven repository and, because of package dependencies, to other repositories too.

To further our tests, we plan to re-test this install while blocking internet access with an active packet capture, to list what is queried on the net and to try to understand why the flavor install hangs.

#4

Do you have examples of those URLs ? To see what seems to be missing locally.

#5

To create a truely reproducible experiment and eliminate as much variables as I can, I created a new Debian VM from cloud official Debian images.

https://cloud.debian.org/images/cloud/

Because the 2Gb initial disk image can be filled nearly instantaneously by a new xwiki install, I extended disk initial size from 2 to 16 Gb and started it with one CPU and 4Gb of RAM. I added the Maven apt repository and installed the xwiki-tomcat9-mariadb meta-package. To keep this experimental protocol as short as possible, I did deliberately not customize any xwiki config file afterwards. Yet unoptimized, it still runs.

I then downloaded XIP package and unzipped it in /var/lib/xwiki/data/extension/repository. I changed to tomcat:tomcat all the files ownerships.

I blocked any outgoing traffic from the server with those iptables commands :

# First flush previous rules
iptables --flush
iptables -t nat --flush
iptables -t mangle --flush
# Simulate a blocked internet access
# - Keep everything established
iptables -A OUTPUT -m state --state ESTABLISHED -j ACCEPT
# - Allow everything on 127.0.0.1
iptables -A OUTPUT -o lo -j ACCEPT
# - Drop all the rest
iptables -A OUTPUT -j DROP

In doubt, I restarted tomcat9 service, then logged on http://serverip:8080/xwiki/

The wizard started. After the 1. Admin user step I arrived on 2. Flavor. Nothing is listed. Reloading the page or going back to it doesn’t change anything. The page remains empty of any flavor.

Your mission (should you choose to accept it :slight_smile:) is to find the needle that I probably forgot in this haystack. At least, this protocol seems easily reproducible.

#6

I reply to myself to add that I’ll probably also try “the manual route” by installing a Xwiki completely decoupled from the OS to test if the Debian integrated packages (so convenient, by the way) play a role in this “the XIP offline package won’t completely install or won’t even be found” weirdness. Still, the Debian packages maintainer(s) could be interested to reproduce my experiment, maybe to find a missing or incomplete config file somewhere that counteracts offline packages install.

#7

Excellent news here : while the 14.10 XIP package was problematic, the 14.10.3 brand new XIP package, after being uncompressed, is properly displayed as a possible choice by the installer when being “offline” (ie without internet access) and is installed without a hitch. :+1:

I consider this/my problem solved. I hope that I helped, even a little, to correct a possible minuscule bug. :blush: