Dependency conflict in extension manager for docx4j-ImportXHTML

Help / Discuss
#9

<distributionManagement> just describe where artifacts are deployed, nothing to do with dependencies.

#10

Note that it’s easy to fix it for your instance (by applying what I fixed in https://jira.xwiki.org/browse/XWIKI-16988 or some other customization) but anyone else who use your extension will still have the issue until a new version of XWiki is released.

The idea is to tell Extension Manager that it already have the dependency, it’s just that it’s not the same id. For that you need to go to WEB-INF/lib and open bcmail xed file, you will find a bunch of “features” listed in there. The issue here is that the version scheme changed so you need to indicate it by adding the corresponding version (which means adding /140 next to each bouncycastle:* feature) as in:

<features>
<feature>bouncycastle:bcmail-jdk16/140</feature>
<feature>bouncycastle:bcmail-jdk15/140</feature>
<feature>bouncycastle:bcmail-jdk14/140</feature>
<feature>bouncycastle:bcmail-jdk13/140</feature>
<feature>bouncycastle:bcmail-jdk12/140</feature>
<feature>org.bouncycastle:bcmail-jdk16</feature>
<feature>org.bouncycastle:bcmail-jdk15</feature>
<feature>org.bouncycastle:bcmail-jdk14</feature>
<feature>org.bouncycastle:bcmail-jdk13</feature>
<feature>org.bouncycastle:bcmail-jdk12</feature>
</features>

Same for any other Bouncy Castle xed file.

#11

Great!

It definitely solves the problem and is much nicer than what I was imagining (build a list of dependant jars, edit it and then copy everything to WEB-INF/lib: maintenance nightmares guaranteed…)

In fact you are a bit cheating the Extension Manager, by telling it that a dependency is already present, which is close to ignoring it.

The good news here, is that as you have already ported it in future versions, I can tell the sysadmins that the patch may not be quite clean and neat, but that it will be automatically included in the future upgrades which save them from maintenance problems.

The downside, is that I will not be able to publish the extension until a next version of XWiki is realeased. Anyway there is still quit a good deal of work to have it in a publiable state…

Many thanks :slight_smile:

#12

Well the dependency is already present in a more recent version. It’s just that Maven is not very good at tracking id changes so we had to invent something on XWiki side.

#13

A last remark.

Simply adding /140 at the end of the lines as suggested by tmortagne did not help: the (red) conflict lines did not disappear.

I had to consistentlly change any reference to the 1.60 version in the xed files with a 140 (in fact I used 160 to only remove one single character). Then the extension manager accepted to say that a 160 version was installed while the itext extension required a 138. It looks like the XWiki instance still works even after all that black magick…

I shall report here for eventual future problems.

#14

This is very suprising. What is the exact version of XWiki on which you tested that ?

#15

I am testing on a 10.11.9 which is the same version as the one of the production instance.

#16

Indeed I indicated the wrong location. <features> is actually the very old way. Now it needs to be fixed in <extensionfeatures> and the version need to be changed to “140” instead of adding “/140” to the id.

#17

Hi Thomas,

Do you mean that I should add new lines with 140 along with the original ones with 1.60 and that I should not change the original <version>1.60</version>? As it currently works without me really knowing how, I dare not change anything…

#18

No I said the version need to changed but not “any reference to the 1.60 version in the xed files” (otherwise you are lying about the version of org.bouncycastle:bcmail-jdk15on which is installed and it might cause issues with some extensions), just those inside <extensionfeatures> and related to the old bouncycastle groupid. As in:

<extensionfeatures>
<feature>
<feature>bouncycastle:bcmail-jdk16</feature>
<version>140</version>
</feature>
<feature>
<feature>bouncycastle:bcmail-jdk15</feature>
<version>140</version>
</feature>
<feature>
<feature>bouncycastle:bcmail-jdk14</feature>
<version>140</version>
</feature>
<feature>
<feature>bouncycastle:bcmail-jdk13</feature>
<version>140</version>
</feature>
<feature>
<feature>bouncycastle:bcmail-jdk12</feature>
<version>140</version>
</feature>
<feature>
<feature>org.bouncycastle:bcmail-jdk16</feature>
<version>1.64</version>
</feature>
<feature>
<feature>org.bouncycastle:bcmail-jdk15</feature>
<version>1.64</version>
</feature>
<feature>
<feature>org.bouncycastle:bcmail-jdk14</feature>
<version>1.64</version>
</feature>
<feature>
<feature>org.bouncycastle:bcmail-jdk13</feature>
<version>1.64</version>
</feature>
<feature>
<feature>org.bouncycastle:bcmail-jdk12</feature>
<version>1.64</version>
</feature>
</extensionfeatures>
#19

Ok, more clear now. Thank you very much for your time and your explainations.

#20

Unfortunately, it does not work. If I do not change the master <version> value, the extension manager finds that its version is 1.60 and refuses any install depending on a 138 version.

I shall try to change those @#&*! version only during the install of my dependant extension and revert to a more correct numbering after…

#21

And that’s the case. But it’s supposed to compare to the feature version and not to the main extensions. Most probably a bug in 10.11.9 which was fixed since (I vaguely remember something like this), will try to reproduce when I find some time.

#22

Just for refence with 10.11.9, if I put back the correct version number in BouncyCastle extensions, iText extension (and all my dependant ones) refuses to load at start time with the following error message:

[main] WARN ltInstalledExtensionRepository - Invalid extension [com.lowagie:itext/2.1.7] on namespace [wiki:xwiki] (InvalidExtensionException: Dependency [bouncycastle:bcmail-jdk14-138] is incompatible with the core extension [org.bouncycastle:bcmail-jdk15on/1.60])

Looks like I will have no clean solution on 10.11.9 :frowning:

I will still continue testing with the odd version numbers on 10.11.9 but really think that upgrade time has arrived

For future readers, the installation way that seems to work is:

  1. before changing anything in xed files, install org.bouncycastle:bctsp-jdk14 version 1.46 on which iText 1.2.17 depends
  2. change the numbers in xed files for bcmail-jdk15on-1.60.xed and bcprov-jdk15on-1.60.xed for bouncycastle extensionfeatures and the main <version> bloc
  3. find the bctsp-jdk14.xed file under the /data/extensions/repository folder and add it an extensionfeature and a feature bloc mimicing the one from bcprov-jdk15on-1.60.xed. Do not forget to change the main <version> bloc too.
  4. at that moment, install normally the docx4j-ImportXHTML extension

Definitely ugly because from now on, you cannot install anything depending on BouncyCastle, because in the end the system will choke with the inconsistent version numbers

#23

The simplest in your case is most probably to put docx4j-ImportXHTML jar (and missing jar it needs like xhtmlrenderer from what I understood) in WEB-INF/lib, i.e. make them core extensions too. XWiki trust those and does not try to validate them (it does not have any control over them anyway, impossible to unload them for example).

#24

By the way, which version of docx4j-ImportXHTML are you trying to install ?

So I just tried the <extensionfeatures> fix I suggested but in 11.10.2 and it works as expected (that’s reassuring) for the problem we discussed but I actually hit a very different problem: itext 2.1.7 have a dependency on bctsp-jdk14 138… which does not exist (the pom exists but not the JAR). Looks like it was fixed in following versions. Since XWiki does not package bctsp (simply because it does not need it) it cannot automatically “fix” this nonsense with some <extensionfeatures> magic.

#25

My problem is that the magic of maven have driven me to be rather lazy, and I prefere when the system take cares of the dependencies :wink: . More exactly, there are many more dependencies listed in the dependency hierarchy but fortunately they are already present so you are true adding itext, xhtmlrenderer and bctsp-jdk14 in WEB-INF lib would certainly be enough, but I was not sure of that when I first tried. And for pushing it into a production system, I would have to set up a manual registration of the added jars and their versions as long the dependency tree, when the extension manager give it for free.

For docx-ImportXHTML I am using the last version which depends on slf4j 1.7.25, so the 6.1.0 and 6.1.2 for docx4j. I will change to the last 8 .x series when the production XWiki will upgrade to a 11.x version.

For bctsp-jdk14 it looks to be a maven trick I was not aware of: a <distributionManagement><relocation> bloc. Maven knows about it and automatically fetched the jar from the org.bouncycastle group. But the extension manager apparently does not processes that. IMHO it seems to be scarcely used, so I really wonder whether it should ever be implemented…

#26

Right, I did not even checked. I was expecting Maven Resolver (the library XWiki uses to resolve Maven artifacts) was taking care of it but looks like we need to do it on our side. Just fixed it in https://jira.xwiki.org/browse/XCOMMONS-1858.

#27

Yes the coverage of this use case is definitely not great right now. The problem is that, while it’s easy for a build system, taking care of the exclusion logic is a bit more complex for a runtime system. The good new is that we plan to work on that in 12.x (so somewhere in 2020).

#28

Thank you for this info. Knowing that a true solution should come in 12.x version, the current workaround is now fully acceptable.

PS: sorry for the delay to answer, I was on vacation…