I’m working on a new contrib macro that provides a new $services.stats service, allowing to give different reporting statistics on the wiki.
For now, there would only two functions (getting the number of created pages in the last 30 days, and getting the number of updated pages in the last 30 days), but the idea would be to have a generic service that could be later extended with any other stats useful for reporting.
So the new contrib extension would be named api-stats.
Could someone create the corresponding contrib repo and jira project for this?
Do you have a new store for stats? How do you gather stats? Just XWQL queries to the DB or something else?
I’m wondering if the name is not too generic but it depends what’s the goal of this module (hence my question above). If it has its own store, synced with the xwiki store, then everything is doable in term of stats I believe, but if it’s just querying the existing RDBMS or the existing Solr core, then it’s going to be quite limited in term of what’s doable. For example, how would you report unique visitors or visits?
From what I understood of the Statistic Application, there is an active monitoring of actions inside the wiki that needs to be enabled in order to get the data we need to do the statistics.
The idea here is the use the data we already have in order to produce the stats. For example, for the two functions described above, I am fetching the data from the solr events core.
Indeed, the name of the service is a good question, we discussed about it with @lucaa this morning:
if we are doing stats only from the Event Stream data, this service should be something like: $services.eventstream.stats
if we think data could come from other sources, like hql/xwql requests, then a more global “stats” name should be more appropriated (hence the $services.stats)
I would be more for the latter, so that we have all the reporting stats at the same place, instead of having to use different services depending on where the data comes from.
But indeed, if the goal is to have a generic place for reporting stats, this is stilll kind of a duplicate of the Statistic Application, even if they are not working the same.
This is not really in the scope of this services, the goal here is to do stats based on data we already have in xwiki, without having to generate any other data.
So if there is a way to indicate this in the name of the service instead of just “stats”, for sure this would be better. If you come up with such a name (that is usable as a service name), for sure let’s use it instead!
It’s actually not a macro, it’s just an API - for now at least we don’t plan to handle any presentation.
The main focus is to encapsulate the code that allows to extract the existing reporting data from the wiki.
@cdesableau please don’t forget to verify the scope of the service you publish and make sure that the extension settings are coherent. For example, since data is extracted globally, I am wondering whether it makes sense to allow it to be installed only on a subwiki. Since the thing is written as a scriptcompnent (the current version), we probably should avoid deploying this page on all subwikis if it needs to be used on all subwikis. So maybe we should make the service global and then if we do so, we should only allow to install it on the main wiki, we need to attentively check this to try to make it coherent with a farm usage.
As we discussed live this morning, both make sense, even if I also have a doubt about the fact that stats is bit too generic. If there is a more specific vocabulary that we could use to describe our service (“providing stats from the data that is already stored by the engine”), then I will definitely accept it to replace $services.stats .
The stats one is too generic for a contrib extension and there’s also an XWiki xwiki-platform-statistics module which may want a script service one day (it would probably be $services.statistics but still, that would be confusing IMO). Using stats would also be taking/reserving a very generic hint.
Stats already exists in XS, but we should probably move it to contrib. I wonder if the API that you proposed could not be just added to the existing stats API, in which case they could both go in the same contrib module. There’s one difference indeed, which is the current stats module stores data in the DB, but I really don’t see how you can do any serious stats without having a store to aggregate stats data over time (recomputing everything in realtime is just not going to work for any complex stats).
Note that moving the stats api/app outside of xS is not straightforward as the code is hardcoded in oldcore, but it’s doable and it should be done IMO, especially since the XS core devs have not worked on stats for years now and we’re not even sure we’d like to continue supporting this, vs Matomo for example (although Matomo would probably not be able to provide all types of stats a specific stats module for XWiki could implement - Right now most of the stats in the stats API can be provided by Matomo).
So, since we’re not going to move the stats module out of XS just now (would need someone to work on it), I think you should keep your contrib extension separate for now and we could merge them later if need be.
In view of this I’d name it something like api-statistics-alternate, api-statistics-extra, api-statistics-query (to show that it only does query and doesn’t store stats data), api-statistics-additional, api-statistics-query-solr, etc. (with script service hints of $services.alternateStatistics - or $services.alternateStats if you prefer -, $services.querySolrStatistics, etc).
