Introduce methods to fetch a subset of revisions in XWikiVersioningStoreInterface

Development
#1

Hi everyone,

I have been working on fixing performance issues when browsing and editing documents with a lot of revisions, and in particular Loading... and Loading....
The main problem here is that operations that require access to the document history load the complete history from the database all the time. To be perfectly transparent, I tried to reproduce the first issue using the script provided by Manuel in the issue, but I couldn’t even reach the required number of revisions to generate a OOM on my instance since it becomes exponentially longer to generate new revisions (saving takes more than 1s after 250k revisions).
For the second issue, the history is even loaded completely twice (once to get the number of revisions for pagination, and a second time to get the actual revisions).

The main cause is an API limitation: VersioningStores do not provide any method to query a specific subset of revisions from the database, it’s either all or nothing. XWikiDocument.getRevisions(RevisionCriteria, XWikiContext) currently filters them afterwards.
As such, I would like to propose two additions to com.xpn.xwiki.store.XWikiVersioningStoreInterface:

/**
 * Return a subset of versions from the document history according to criteria.
 */
Version[] getXWikiDocVersions(XWikiDocument doc, RevisionCriteria criteria, XWikiContext context);

/**
 * Return the number of versions from the document history that respect some criteria.
 */
long getXWikiDocVersionsTotal(XWikiDocument doc, RevisionCriteria criteria, XWikiContext context);

These would be implemented in XWikiHibernateVersioningStore through database queries based on the provided criteria. When saving a document, we would then only fetch the last revision, and when displaying the history we would only fetch the result of a count operation and query only the revisions required for the pagination.

What do you think?

#2

Would it be possible to provide default implementations of these methods, e.g., just loading everything and filtering after loading? That way, the situation would stay the same for any other implementation of this interface instead of breaking them. Or is this filtering logic too complex to be put into a default method?

#3

It is definitely possible, I would probably extract the filtering currently in XWikiDocument.getRevisions(RevisionCriteria, XWikiContext) to do that since it would no longer be needed in XWikiDocument.

#4

Any reason to return an array here and not a collection?

#5

My only reason is that the existing getXWikiDocVersions (without filtering) also returns an array, so it was more consistent. I don’t mind either way, though.

#6

Those are probably old APIs, IMO it would be cleaner to return a Collection. That being said I’m +1 to modify the API provided that you provide the default implem in the interface to avoid the breaking compat’.

#7

We normally use count for APIs that return the number of something. So I suggest using:

long getXWikiDocVersionCount(XWikiDocument doc, RevisionCriteria criteria, XWikiContext context);

Looks good otherwise. +1

Thanks,
Marius

#8

I have created Loading....