Issues creating packages for offline repository

Help / Discuss
1

Hi,

I am running an offline xwiki that for security reasons is running in a shielded environment which has no internet connectivity whatsoever. The installation went fine (RHEL) using the settings for the offline repository. Now I want to add some extensions to my install, however I run into some issues. I hope someone here can point me in the right direction and shed some light on where I should be looking next.

So far I have never used Apache’s maven, so I am probably doing some obvious stuff wrong, but I really cannot find what it is that I am messing up.

What I did so far is (obviously) install maven and set the correct environment variables. The data that I’ll post here comes from a test machine that is online itself, I’m trying to create the packages on there.

[root@localhost xwiki-platform-export]# mvn --version
Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
Maven home: /opt/maven
Java version: 11.0.16, vendor: Red Hat, Inc., runtime: /usr/lib/jvm/java-11-openjdk-11.0.16.0.8-1.el7_9.x86_64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-1160.66.1.el7.x86_64", arch: "amd64", family: "unix"

I’ve cloned the repo with GIT

git clone https://github.com/xwiki/xwiki-platform.git
git checkout stable-14.6.x

Then from the xwiki-platform directory I tried to build the packages for test using ‘mvn package’. Here I would continuesly run into the following error:

[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for org.xwiki.platform:xwiki-platform-tools:14.6.1-SNAPSHOT: Could not find artifact org.xwiki.platform:xwiki-platform:pom:14.6.1-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 25, column 11

After some probing and testing and googling I decided to replace the version number as test as I could not find any reference to the .1-SNAPSHOT part.

grep -rli '14.6.1-SNAPSHOT' * | xargs -i@ sed -i 's/14.6.1-SNAPSHOT/14.6/g' @

This seemed to have worked as after that it started downloading all sorts of things and attempted to build stuff, until it ran into the following error:

[INFO] --- maven-compiler-plugin:3.10.1:compile (default-compile) @ xwiki-platform-tool-jetty-listener ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /home/tomcat/xwiki-platform/xwiki-platform-tools/xwiki-platform-tool-jetty/xwiki-platform-tool-jetty-listener/target/classes
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for XWiki Platform - Parent POM 14.6:
[INFO]
[INFO] XWiki Platform - Parent POM ........................ SUCCESS [  0.914 s]
[INFO] XWiki Platform - Tools - Parent POM ................ SUCCESS [  0.098 s]
[INFO] XWiki Platform - Tools - XWiki Configuration Resources SUCCESS [  9.947 s]
[INFO] XWiki Platform - Tools - Jetty - Parent POM ........ SUCCESS [  0.062 s]
[INFO] XWiki Platform - Tools - Jetty - Jetty Lifecycle Listener FAILURE [  0.535 s]
[INFO] XWiki Platform - Tools - Jetty - Jetty Configuration Resources SKIPPED
<rest of output omitted>
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:compile (default-compile) on project xwiki-platform-tool-jetty-listener: Fatal error compiling: error: release version 11 not supported -> [Help 1]

Because I really just want the PDF export extension I went into that sub-directory and tried a mvn package in there, but it would run into the same ‘version 11 not supported’ error.
This is where I get stuck and don’t know how to continue on my path. I googled all afternoon but all solutions to this error point to setting the correct JAVA_HOME env, but this is set correct at the system

[root@localhost xwiki-platform]# echo $JAVA_HOME
/usr/lib/jvm/jre
[root@localhost xwiki-platform]# ls -als /usr/lib/jvm/jre
0 lrwxrwxrwx. 1 root root 21 Jul 28 16:34 /usr/lib/jvm/jre -> /etc/alternatives/jre
[root@localhost xwiki-platform]# ls -als /etc/alternatives/jre
0 lrwxrwxrwx. 1 root root 55 Jul 28 16:34 /etc/alternatives/jre -> /usr/lib/jvm/java-11-openjdk-11.0.16.0.8-1.el7_9.x86_64

From here on my google skills appear insufficient as I cannot find anything that could get me past this error.

Is there someone who can point me into a new direction?

Cheers,

Tnias

2

This generally means you did not followed https://dev.xwiki.org/xwiki/bin/view/Community/Building/#HInstallingMaven and especially the Maven setting required to use SNAPSHOT XWiki artifacts.

3

HI,

Thanks for that. The link at https://www.xwiki.org/xwiki/bin/view/FAQ/How%20to%20generate%20a%20XIP%20file%20for%20an%20Extension%3F which describes the XIP file packaging doesn’t referenced the link you posted (as far as I can see). On my Google searches I had not ended up at the page you’ve posted. Using the linked article I did indeed solve the ‘could not find’ error. I am however still stuck at the ‘release version 11 not supported’ on the RHEL machine.

On a 3rd machine I installed Debian. Here I run into a new set of errors being a bit random trying to download a bunch of items from 0.0.0.0

[DEBUG] Using transporter WagonTransporter with priority -1.0 for http://0.0.0.0/
[DEBUG] Using connector BasicRepositoryConnector with priority 0.0 for http://0.0.0.0/
Downloading from maven-default-http-blocker: http://0.0.0.0/org/xwiki/commons/xwiki-commons-tool-enforcer-dependencies/14.7-SNAPSHOT/maven-metadata.xml

Other than that there seems to be progress in building. It does error out somewhere half-way, but that is something I’ll have to have a look at.

Cheers,

4

This suggests you indicated http URLs for the repositories in your settings.xml file, but recent versions of Maven require https by default. I think you should find Maven warnings about that before that error.

5

Ah right, it appears that for some reason its first trying http://0.0.0.0 to then attempt downloading from https.

I can now build individual extensions with packaging method ‘pom’. Now my adventure starts for packaging it as ‘xip’, which unfortunately is not just a case of changing the packaging method in the pom.xml file. I’ll report back later :slight_smile:

6

Note to produce a XIP package for an already released extension, it’s generally easier to look at https://dev.xwiki.org/xwiki/bin/view/Community/ExtensionPlugin/#HExample. What is described in https://www.xwiki.org/xwiki/bin/view/FAQ/How%20to%20generate%20a%20XIP%20file%20for%20an%20Extension%3F is more for adding a XIP package in an extension you are developing.

7

Thanks a lot! It’s a right pain in the *** to work out the correct formatting and details of a pom.xml for an existing package, but I just now have been able to successfully build a xip for ldap. I am fairly certain it is not entirely correct yet, but we’re making progress so that’s positive :slight_smile:

8

I could really do with some more pointers here, as the XIP that I currently have created does not lead to it being installable at my offline site after extracting it in the offline repo.
Currently I am trying to build a XIP for the LDAP authenticator. Am I using the right pom.xml (ldap/ldap-authenticator/pom.xml at master · xwiki-contrib/ldap · GitHub) here, or is this not the one I should be using?

9

Would be easier to tell you what’s wrong if you were pasting the pom you created to build the XIP package.

10

You are right, but as I am not even sure if my build attempt was done using the correct base I am trying to avoid asking questions due to my own stupidities :slight_smile:

I have attempted to build the XIP file of https://extensions.xwiki.org/xwiki/bin/view/Extension/LDAP/Authenticator/ . I have used the earlier linked https://dev.xwiki.org/xwiki/bin/view/Community/ExtensionPlugin/#HExample and attempted to port the pom.xml information from the authenticator to that example file. The resulting xip package I’ve uploaded and extracted to my offline server, but here its not showing the ldap authenticator in the available extensions.
build log.txt (42.9 KB)

pom.xml.log (2.8 KB)

I have attachted the build log and the pom file (renamed as .log due to upload restrictions). I could upload the resulting XIP package if required.

Thanks for your help!

11

Looks like you duplicate the LDAP pom, this is really not necessary. The point of the XIP package is to zip a set of dependencies, so you should just use the example indicated on https://dev.xwiki.org/xwiki/bin/view/Community/ExtensionPlugin/#HExample and only change what’s different in your use case:

  • the version of XWiki where you plan to use that (in <parent>)
  • the <dependencies> you want to put in your XIP
12

Thanks! I now have the following as pom.xml, this produces a XIP package which I can upload (found https://snippets.xwiki.org/xwiki/bin/view/Extension/XIPImporter/ which I hadn’t seen before), and it installs!

Few more questions here, does the pom.xml that I created now look good?
pom.xml.log (2.3 KB)

I had included the ‘org.xwiki.platform:xwiki-platform-oldcore 8.4’ dependency initially in the pom.xml config as it is listed at the dependency page but this created an error ‘There is already a core extension covering feature [org.xwiki.platform:xwiki-platform-oldcore]’, so I removed it.

I am left with this warning which I am not sure what to do with. Line 3 is set like in the example file, so I am not sure if I have to deal with the warning given.

[WARNING] Some problems were encountered while building the effective model for org.xwiki.contrib:ldap-xip:xip:9.7.4
[WARNING] 'parent.relativePath' of POM org.xwiki.contrib:ldap-xip:9.7.4 (/home/eve/ldap/ldap-authenticator/pom.xml) points at org.xwiki.platform:xwiki-platform-distribution:ldap instead of org.xwiki.platform:xwiki-platform-distribution, please verify your project structure @ line 3, column 11
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
13

You still have a lot of useless things in it which are not in https://dev.xwiki.org/xwiki/bin/view/Community/ExtensionPlugin/#HExample (like most of <properties>) also just put as dependency what you would have installed, and transitive dependencies are going to be resolved automatically (i.e. you don’t need to indicate the jldap dependency).

It’s probably because you created your XIP package pom as a sub-module of /home/eve/ldap/ but all that is not needed, you just need to put your pom in some empty folder (no clone, etc.).

14

Oh right, my bad. That was indeed clutter from all my attempts. So I think I finally cleaned it up properly, like the example. Sorry for this all, it was much more confusing task than I initially anticipated it would be. Could you please have another look at the pom.xml file as it is now? If everything is set as intended I’ll make a little write-up to clarify some things in case anyone else struggles with this and ends up at this topic.
pom.xml.log (1.7 KB)

In case the pom.xml is correct, could you perhaps clarify where the settings on line 9 through 11 are being used? Is this package information for the GUI?

15

Lines 9 and 10 could be any value, they don’t really matter in any way (it’s just that Maven requires a <groupid> and an <artifactid>). The only visible effect is that <artifactid> is used as name for the produced file.

Line 10 tells Maven that you want to create a XIP package, so it’s pretty much the most important line :slight_smile:

16

Cheers!
I have created the following quick 'n dirty how-to. Could you perhaps have a quick look at it, especially the comments added to the pom.xml part and see if there isn’t some utter nonsense written there? If its correct otherwise I’ll c/p it here for others to find in case they struggle with it like me.
maven instructions.txt (6.6 KB)

17

Looks good.

Note that on most Linux distributions, I think there is a package for Maven available in the standard repositories, so everything before the ~/.m2/settings.xml step is generally a one-liner (but I don’t know for Centos).

18

Thanks, I’ll update that when I post the instructions here later.

Following my own guide I tried to build a few random extensions which mostly seems to work. I do run into another error while trying to build the pdf export application (https://extensions.xwiki.org/xwiki/bin/view/Extension/PDF%20Export%20Application/) for xwiki 14.6 using the following pom.xml file.
pom.xml.log (1.7 KB)

The error thrown is

[ERROR] Failed to execute goal on project export-pdf-ui-xip: Could not resolve dependencies for project org.xwiki.platform:export-pdf-ui-xip:xip:14.6: Could not find artifact org.xwiki.platform:xwiki-platform-export-pdf-ui:jar:14.6 in xwiki-releases (https://nexus.xwiki.org/nexus/content/groups/public) -> [Help 1]

Any idea why this is still erroring out?

19

org.xwiki.platform:xwiki-platform-export-pdf-ui:jar

When you don’t indicate any <type> Maven assume it’s a jar but in this specific case (and usually is when the id ends with -ui) it’s actually a xar (in XWiki Extension Manager we actually load the descriptor associated to the id to find the default type which is why we don’t need to indicate the type, but Maven does not do this kind of cheating unfortunately).

So the fix is to have the following as dependency:

    <dependency>
      <groupId>org.xwiki.platform</groupId>
      <artifactId>xwiki-platform-export-pdf-ui</artifactId>
      <version>14.6</version>
      <type>xar</type>
    </dependency>
20

Thanks! Is there any way I could have known this, or is it by this point trial and error and assuming the -ui part you mentioned?