[Brainstorming] XWiki 11.x cycle roadmap

cycle
roadmap
brainstorming

#1

Hi everyone,

We’re getting close to a new XWiki cycle: XWiki 11.x.

We’re curious to hear from XWiki users what they’d like to see and what they think we should focus on. No promises but we’ll listen :slight_smile:

I’ll also post later some ideas that we have.

Thanks. Let’s start!

PS: For reference, the thread from last year about the 10.x cycle: https://forum.xwiki.org/t/brainstorming-xwiki-10-x-cycle-roadmap/2274


#2

Here’s a thread we need to review to see what’s been done/what’s left:
https://forum.xwiki.org/t/your-xwiki-usability-pain-points/440


#3

We, AutomationX, are using XWiki as an Intranet interally (meeting protocols, blog post news, internal information) as well as a knowledgebase / software documentation plattform for our products for both, external customers as well as our co-workers who use the software.

The following is a work in progess brainstorming :slight_smile:

(0) Make common tasks dummy-user friendly

You are already working on this :slight_smile: This means enabling WYSIWYG experience for most daily tasks (autocompletion of stuff like links etc. etc.) which has greatly improved recently. Time has shown us that most of the users are in no way willing to learn xwiki syntax or anything in that direction, everything has to be a kidsfriendly-applike-facebook-instagram difficulty :roll_eyes:


The following to things are the only BIG issues we face, which are still unresolved in XWiki 10.10 for us.
Our customers often demand “offline documentation”, like something they can print out (for real)

Issues (1) und (2) are somewhat mitigated by the greatly improved HTML Export, so maybe we should scrap PDF and focus on HTML completely (they can carry it on USB and save some trees)

(1) PDF Export looses formatting

If you export the Sandbox Page to PDF in latest xwiki 10.10, it looses lots of formatting. I don’t know if this is a limitation of PDF / OpenOffice? If I copy paste it from Firefox to M$ Word 2013, it has some of the same issues. Exporting has improved a lot since xwiki 8.4.3 though

  • Info / Warning / Error / Success Box Macro or Style is lost, this is vital information in the docs. The only thing thats preserved is Background Color via CKEditor Toolbar.
  • {{code}} Macro Syntaxhighlight does not use monspaced font in PDF
  • Tables have formatting problems if there is to much stuff
  • Tables look ugly in PDF (can probably be tweaked)

Is openoffice used for the pdf export? I had noticed Libre Office does a better job on importing office. Since the docker images contains openoffice, I haven’t used libre office in a while.

(2) Easy export of multiple pages (spaces) to PDF not possible

The was the greatest loss when we ditched mediawiki, since that had the export to book feature, which rendered a really nice pdf for several pages.

As far as I can tell, none of the available PDF export solutions for xwiki work satisfactory.
Maybe there is some browser extension which could help us.

(2.1) PDF Export Panel Application

Does the job and is easy to use, but if you export multiple (child-)pages, everything is crammed together:

  • Page Head on each page shows the Title of the page where wo startet the export, even if you use Page Breaks and the PDF page shows a Child page
  • Childpages have no title introducing them, you would have to use a Heading 1 yourself on every page.
  • The TOC at the beginning also shows all headings of all child pages on the same level

(2.2) PDF Export Collection Application

  • Saving of collections sometimes gets stuck at 0%
  • Exporting a collection to pdf renders a pdf with only exception callstacks:
org.xwiki.rendering.macro.MacroExecutionException: Failed to evaluate Velocity Macro for content

(3) Improving HTML Export

HTML Export of spaces is pretty great! It looks exactly like the live version, and retains relative page links so you can user links to navigate between offline documents.

(3.1) Render Macros in HTML Export (3)

We heavily rely on the {{children}} Macro for navigation, since manual hyperlinks are time-consuming. These are not rendered in HTML export, in contrast to e.g. the {{toc/}} macro.
It would be great to optionally execute all macro content during HTML export, and save the macro output in the export. This would be sufficient for our cases I guess.

The same goes for the Navigation Panel and Breadcrumb dropdown menu, but this is lower priority.

(3.2) Customize HTML Export Layout

In offline HTML the following elements should be excludeable / customizeable , to show only Pagecontent in the export:

  • Hide Left and/or Right Columns of Main Page Layout (online show content)
  • Hide Page Buttom Layout: shows Tabs for Comments; Attachments; History; Information: None of them are working.
  • Hide Actions: actions
  • Hide Global Site Header stuff: Search / Notifications Button / UserProfile / Hamburger Menu Button

(4) Page Auto Translation

We are currently writing all our content in German. Manually maintaining everything also in English is unmanageable for us. We use a custom Panel with a google translate widget, which works fine for viewing single pages.
But this is not usable when using PDF (1 & 2) or HTML (3) Offline Export. I’m not aware of any xwiki extension aiding in that. Maybe there is some browser extension which could help us.

[EDIT]: to clearify, the task is to either:

  • a) automatically translate content of several pages to english in the course of exporting them to PDF or HTML, which would have to be done on the server side.

  • b) Translate complete spaces of xwiki pages in a batch job and store the results in english translations of these pages. THEN export the pages to PDF or HTML using the English content.



Other points:

Improving Page Relations Application

This extension is great! In my opinion the best one to easily improve content navigation and visualisation. Tags are too easly overlooked and are meant for a different purpose.

  • In the Future section is says “Automated creation of relations when page links are detected in the content.” : This would be very helpful, but should be optional, since there may be lots of links, some of which you may not want to add as relation.
  • Since this extension relies on attached objects: does it scale? If there are > 1000 Pages and each of them has several relations?

Improving Search Experience

Repeatedly some of our users complain about the search function. Most of the time it is PEBKAC , like not using a * wildcard or using wrong key words in general. I haven’t deep dived into Apache SOLR configuration yet.

Maybe there can something be done to make search more “Google like” ?

  • Searching ServiceAccess does not match Pagename ServiceAccessMethod
  • Search Benutzer Verwaltung does not match Pagename Benutzerverwaltung

Improve Page Print Layout

Some users still want to actually print out pages. If you Print a xwiki page (to pdf or to paper), the layout contains some ugly elements, which are mostly undesired IMHO (could depend on user needs).
The printed layout looks better then the current Export to pdf result.

The default layout could be improved by the following points. Typically, the users wants to print CONTENT, and content only.

Example is XWiki Sandbox Page, PDF via Foxit PDF Printer:
print_top

  1. Page Titel is an the far top left side, und therefore cut off on some printers, this should be centered (?) in the header

  2. Top right side shows ugly url

  3. and 4.) All hyperlinks are rendered twice (probably intentional), label and url. If you have a deep hierarchy, the breadcrumbs can get huge, see following screenshot:
    print-breadcrumbs
    Breadcrumbs may print as label only, or be hidden by default.

  4. The page bottom shows TAGs and Comments, even if there are none. Show tags only if they exist, hide page comments by default.
    print_bottom


For the time being, thanks a lot.
BR Mario


#4

Depends on the browser. I know you have that out of the box in Chrome (my parents use that a lot) and I’m sure an extension exist for Firefox and other browsers too.


#5

Regarding which topic, content translation or PDF-book-generation of mulitple pages?
The crucial point is that our customers don’t have internet connection on some plants.


#6

I agree about the items you listed in this category and I think it’s relatively easy to do. That would be good things to improve. Now I’m happy that you’re valuing the HTML export because it’s not a feature I’ve seen used a lot so far (which explains why we’re lacking those features).

I find the use case of offline interesting. I hadn’t thought much about it as a use case for HTML export but it does make sense, and it’s much simpler to get it perfect than to get PDF export perfect.


#7

The docker image actually uses LibreOffice: https://github.com/xwiki-contrib/docker-xwiki/blob/master/template/Dockerfile#L38


#8

Actually I read it too quickly. I tough you were talking about auto translation of HTML content in the browser but seems you were talking about translating the content while doing the PDF export and indeed it won’t help for this use case since it’s done on server side.


#9

No it’s not. We use the architecture described in the diagram at https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Configuration/#HCustomizingthePDFexportLook26Feel (Apache FOP/XSLFO).

I don’t recall if we tried to use OO/LO to perform the PDF export and what result we got. We do use OO/LO to perform RTF exports for example. Maybe @mflorea or @tmortagne remember?


#10

It’s true that we haven’t worked on the PDF multipage export over the years and it’s the last type of exports that doesn’t not support multipage export. There used to be https://extensions.xwiki.org/xwiki/bin/view/Extension/PDF%20Export%20Collection%20Application meant for that but it’s an extension and it’s not been supported by the XWiki dev team and has probably decayed over the years (i.e. not been maintained). It might not be hard to make it work again though.

For reference @evalica has done some design pages with ideas a long time ago:

There were also some design pages by others:


#11

This is not easy since the trees are dynamically loaded (and doing this in JS, i.e. client side). So we would need to replace them by some static list fallbacks containing only what’s displayed on the screen for example. But then they are not going to be useful in the majority of cases… But better than displaying something empty for sure :wink:


#12

I’m not sure what you’re proposing here. To have some sort of auto translation help (using some external api like google translate) to automatically provide a translation when editing a page into a language that doesn’t have any content?

You mentioned something about PDF/HTML export but I didn’t get it. Do you mean choosing the language of the pages to export or something else?

Thx


#13

My guess is other xwiki users have already developed their own solutions or use some tool. Even some of our guys mentioned he had some tool lying around that crawls a target page to some depth and downlaods everything into files.

Or everybody is always online nowadays :smiley: but that can’t be the case in let’s say an nuclear power plant (exaggerated)


#14

I agree, this is asked very regularly. We do have some guide to explain how you can configure this I believe. I couldn’t find it on https://extensions.xwiki.org/xwiki/bin/view/Extension/Solr%20Search%20Application though, weird. @mflorea any idea where it could be? Thx


#15

@tmortagne got it right:

So all our content is in e.g. German. There is no English “Translation” stored within the xwiki pages.
We’d like to export a space to PDF or HTML and have the results in English.
I will update my original post to clearify.

Since Google Translate and so forth run client-side and the export on the xwiki server, thats a challenge.

Another possible approach would involve two steps:

  1. Use some tool to read all xwiki content pages (german), translate it to english using some translation service, and save the results to the corresponding xwiki pages, as English Translation.
    This would have the additional benefit of having english content without always live translating. Since the XWiki UI langage and the page content language are still one, english customers (english browser) would then have an english interface.
    This translation batch job could run once every month or once every quarter year.

  2. Then Export all content to PDF or HTML using the english content version, which is now available server side since it resides in the sql database.

WDYT?

PS: I just googled keywords like “web crawler” and “website downloader” and there is an vast amount of free software availble, maybe something is suitable.


#16

Urgent technical challenges for 11.x

  • Solve Jenkins stability (InterruptedException/ClosedChannelException). Rationale: stabilization times for releases, day to day dev work (too long to get feedback)
  • Upgrade Hibernate to latest version - Rationale: performance, better DB support, ease of future upgrades
  • Upgrade to Velocity 2.x - Rationale: performance, ease of future upgrades
  • Fix all WCAG failing tests and more generally move to WCAG 2.1 (https://www.w3.org/TR/WCAG21/) - Rationale: usability through accessibility, current failing test reducing trust in CI
  • Continue removing all modules from xwiki-commons/rendering/platform that are not bundled in XS - Rationale: smaller core repositories, faster CI builds, ability to install non-core features as extensions
  • Fix ALL flickering tests we have had for months and years. Get our CI jobs to be blue again. Rationale: current failing test reducing trust in CI, lost time for stabilization/releases, STAMP metric.
  • Performance improvements: make sure we go back to at least 8.4.x perfs and even do better. - Rationale: usability
  • Merge commons &rendering into one repo (and possibly also platform)

There are probably some more urgent ones. Note that I’ve listed only very urgent things (I didn’t list stuff like Rendering 3.0, “new” Model, etc). Feel free to add more urgent ones.


#17

Added the following in my original post above. I dunno if nobody prints anymore today :man_shrugging:


#18

Thx. For this I suggest you create JIRA issues fi they don’t already exist.


#19

I would add “Upgrade to jQuery 3.x” to “Urgent technical challenges for 11.x”.


#20

Other technical stuff:

  • Get rid of old Struts based actions system or upgrade Struts at worst
  • Add proper support of pull requests and features branches in Jenkins (mostly means make sure it’s built with a custom <version> to not mess with master and be careful about security regarding stuff injected trough a pull request)
  • Upgrade Solr