Integrate the Link Normalizer in xwiki-platform and XWiki Standard

Hi XWiki devs,

I recently worked on stabilizing and improving a bit the Wiki Link URL Normalizer.

It should be working well now, and @vmassol (among others) suggested that it would make sense to have it in XWiki Standard.

We don’t want to add a contrib extension in XWiki Standard, so I’m here proposing to merge it into xwiki-platform. Of course, it’s not a simple copy/paste, so here is my more detailed proposal:

  • Since it’s actually more of a link normalizer than only a URL one (which is just one type of links it can normalize) I propose to put the main entry point in xwiki-platform-rendering-linknormalizer and the part related to URL manipulation somewhere in xwiki-platform-url (probably distributed in xwiki-platform-url-api and xwiki-platform-url-default).
  • we cannot keep the same Java package, but the current contrib application have a few APIs, so I propose to make sure to be able to have both the platform and contrib one working at the same time to be safe, which mean avoid any collision:
    • renamed the main package into org.xwiki.rendering.linknormalizer (contrib being org.xwiki.contrib.urlnormalizer)
    • introduce a rendering.linknormalizer ScriptService (contrib being urlnormalizer)
    • put the wiki pages in LinkNormalizer space (contrib being URLNormalizer)

Of course, as usual the name possibilities are infinite, so don’t hesitate if you have better ideas. It will also probably be refined through the comments of the pull request I will create for it.

I also prefer to make it disabled by default for now (there is a configuration in the admin to enable/disable it). The reason is that while it seems to work fine, I’m not yet convinced by the current design regarding performances (it parses, and potentially modify, each saved document in reaction to a DocumentCreating/UpdatingEvent) and I definitely won’t have the time to refactor that anytime soon.

Here is +1

+1 from me to have it in XS as I think that being able to convert URLs into wiki links is key for XWiki’s usability, and having it as an extension means having a lot of XWiki instances around the world not knowing about it and suffering a decreased usability because of it.

This is why I also believe we need to turn it on ASAP. Also, I’d turn it on in XS 15.9 so that we get the max feedback and so that it’s fully ready for XS 15.10.x (next LTS). In case there are problems, Admins can always turn it off. Right now we don’t know of any existing problem and thus I’m +1 to turn it on in 15.9. Regarding performances, it would only affect saving times. I guess it could be improved by storing some metadata indicating the last date/time the normalizer has executed on a given reference and compare it with the last modification date to decide if it should run again or not. @tmortagne if you think this is a good idea or if you have other ideas, I propose that we create a jira about this.

Also note that we should work on Loading... if it’s merged in platform, as the blocking/depending issue is fixed there.

Thanks!

+1, thanks!

I won’t really help, as the main cost is to parse all that to find the links. To me, what we need to do is making this normalization asynchronous through the TaskManager, which is used for example for mentions for similar reasons.

The main problem with this is that it’ll create a new revision which looks a bit magical for users (would need to be done by the system user), in addition to having some time frame when the links are wrong and if a rename/move is done during that time, you’ll miss the links.

Well, what is very magical right now is that what’s saved is not what you wrote. It would make things a lot more clear to see in the history an entry indicating that the links have been normalized, especially when there is a bug or a bad filtering configuration.

A future improvement is to have the WYSIWYG editor and wiki editor to update URLs on the spot (convert them to wiki links when local) in the editor before the user hits save.

I think find this better from a UI/UX POV than some async process. Which is both opaque (WTF feeling when you see a new revision) + it will lead to bugs (as mentioned in my previous reply).

Sure, in any case, none of that is going to be done as part of the move to xwiki-platform, so it does not really impact this proposal.

Actually I’m wondering if what should be in XS shouldn’t be this part:

A future improvement is to have the WYSIWYG editor and wiki editor to update URLs on the spot (convert them to wiki links when local) in the editor before the user hits save.

While the URL Normalizer would remain a contrib extensions, used mostly with a scheduler job to fix existing pages.

One reason is that once we have the feature in the editors, then I don’t think we need the normalizer anymore and it would be painful to move it back to contrib then. The other pro is that the features in the editors do not have the performance issue mentioned above.

WDYT?

The main problem I see is that ATM there’s no roadmap plan to introduce the feature in the editors, but I guess we could estimate it and decide to work on it, in between other stuff, if not too complex.

It sounds quite complex to me, and I doubt it’s going to beat the other stuff we have planned.