Backporting cache and macro performance improvements to the LTS branch

Development
1

Hi everyone,

I’ve recently fixed some performance issues. As these fixes seemed a bit dangerous (or even introduce a new feature), I didn’t backport them to the LTS version. This vote is about backporting three fixes to the 16.10.x LTS version:

Security and document cache

Under high load, the security cache can behave very badly due to the use of read/write locks. I’ve fixed this in XWIKI-22979. It seems that we also reproduce these problems on xwiki.org, causing slowness on xwiki.org. The fix removes some locking in the security cache and replaces the locks by lock-free invalidation counters. The fixes have been released in XWiki 17.2.0RC1 and deployed in production for some time without issues, nevertheless, all changes to locking have the potential for introducing regressions.

As part of this security cache fix, I’ve introduced a new stress test for the security cache. On our CI, this test uncovered a race condition in the loading of the document cache which can lead to outdated documents being loaded into the document cache. This is issue XWIKI-23055 that I’ve fixed in XWiki 17.3.0RC1 by introducing a cache loading helper (XCOMMONS-3300). First versions of this helper had problems with recursive loads, showing the dangerous nature of these changes. I’ve fixed those issues already before the release, and XWiki 17.3.0 has been deployed on myxwiki.org and possibly other instances and so far I haven’t heard of any issues. In order to avoid the test failures on CI and to also improve the consistency of the document cache, I suggest to also backport this patch.

Macro transformation

We noticed that rendering huge pages with thousands of macros is quite slow due to the quadratic running time of the macro transformation. For this reason, I’ve introduced the concept of isolated macro execution (see also the proposal and XRENDERING-778). This has been released as part of XWiki 17.3.0 and has been deployed on production systems. I’m not aware of any issues, but they could also be hard to catch (like some macros being executed in the wrong order). Still, I think it is an important performance improvement that we should backport to LTS.

I’m opening this vote about backporting the following three fixes until next Monday, May 26th, 10:00:

  1. Security cache (XWIKI-22979).
  2. Document cache (XWIKI-23055 and XCOMMONS-3300).
  3. Macro transformation (XRENDERING-778).

Thank you very much for your votes!

1 Like
2

+1 from me (as we could consider these important bugs for the LTS). I hope we’ve had enough time to uncover potential issues on the branches where they were fixed, but I’m willing to take the risk (and I know we’ll provide quick fixes if we find new issues).

Thank you for working on this.

3

+1

4

+1 thanks

5

I’m concluding this vote with 3 +1 votes, so the vote has been accepted. Thank you very much! I’ve backported all mentioned issues.

1 Like