@vmassol, it looks like for every page I created, a filter was automatically created to notify me of any changes made to the pages and since I was the one setting up the wiki, that’s why I guess I had so many filters.
I guess the real solution is to make the default never notify anyone for anything and only turn it on if the user asks for it.
I actually don’t know how to script that, so yes, I’m doing it manually. If I’m not insane by now, I soon will be!
Something like this:
{{velocity}}
#if ($request.confirm == '1')
#set ($targetDoc = $xwiki.getDocument('XWiki.SomeUser'))
#set ($result = $targetDoc.removeObjects('XWiki.Notifications.Code.NotificationFilterPreferenceClass'))
#if ($result)
#set ($discard = $targetDoc.save('Removed all notification filter xobjects'))
XObjects removed!
#else
Error: XObjects not removed!
#end
#else
[[Remove XObjects>>||queryString='confirm=1']]
#end
{{velocity}}
FWIW I’ve put the snippet here: http://snippets.xwiki.org/xwiki/bin/view/Extension/Remove%20Objects/
Thank you!
I’m going to let this run overnight and see if it’s finished by the morning.
[edit] and done! it only took a few minutes to run!
I’ll test to see if I get any more spikes when editing/creating the pages and let you know.
Oh, the difference is night and day. Editing/saving is fast and responsive. No cpu spikes, no delays, no errors, no timeout messages.
Those notification filters were definitely having some kind of adverse effect.
Ok, very interesting, thanks.
@gdelhumeau We really have a big issue here that we need to fix somehow. This could actually make XWiki 9.11.x and 10.x not recommended/usable.
This looks like autowatch is turned on. @gdelhumeau is it on by default now? (it used to be off by default for the watchlist).
@pdwalker: Could you send me a XAR of the page in the state it was when you had the problem? I need to digg more to understand why you have this problem. I remember a bug at some point, when the same filter was saved a thousand of times because of some concurrency issues. Having ~700 filters is not supposed to happen in normal conditions, so I suspect a bug there.
All I had to do was make a new page to start seeing the performance issues. It affected everything i did. The problem completely disappeared once I deleted all the notification filters.
It feels like there may be some kind of O(n^2) algorithm running in there somewhere that’s not noticeable under normal circumstances.
The ~700 plus filters is likely my fault as I may have turned it on by default when i was first setting up the wiki. I’d need to do a brand new install to see if that was really the case.
There is one potential problem that I think actually does need fixing; when I was first setting things up, I created a subwiki called “test” for testing the confluence page importing. The pages I manually created there resulted in thr creation of some notification filters.
Later, I deleted the subwiki, but yesterday I noticed that all the notification filters remained behind (but without showing a hyperlink to the now deleted page). That definitely shouldn’t be. Those notification filters should be removed when when the subwiki is removed.
I have previously seen errors in Catalina.out referencing the deleted subwiki and I think that this might have been the source of those errors. I’ll know soon enough.
Lastly, just an observation (warning: I know absolutely nothing about xwiki internals so could be completely wrong here) - it seems weird to me that notifications for page changes are associated with user objects rather than the page objects. Is there a particular reason why it was designed this way? If the “who to notify” data were associated with the page data it seems that it’d be easier to clean up old notifications when a page was deleted.
Hi,
It’s actually true for my installation (10.4) that autosubscribe is on for newly created users (synced from LDAP).
To be honest I’m not a big fan of this because I had to answer a lot of these “why do I get these (notification) mails?”. And people actively creating content and pages collected hundreds of this notification filters too.
In combination with the filters not being cleaned up on page delete (as @pdwalker) mentioned this is a “time bomb”. People who are actively editing will collect enough filters over time to bring down the wiki.
It is the whole point of my wiki and the migration to xwiki to encourage more active contributions.
Please make this whole notification feature opt-in not opt-out! Also please add some kind of bulk delete for the filters!
Will check later how many people already have tons of notification filters, thanks for the script @vmassol.
Hello, I was worry about this issue myself then I writed this snippet to display a nice filtered table of users with one or more NotificationFilterPreferenceClass (or another class) objects
If that can can help 
{{velocity}}
#set($className='XWiki.Notifications.Code.NotificationFilterPreferenceClass')
**Users with 1 or more $className object:**
$xwiki.ssfx.use("js/xwiki/table/table.css")
$xwiki.jsfx.use("js/xwiki/table/tablefilterNsort.js", true)
{{html wiki='true'}}
<table id="tableid" class="grid sortable filterable doOddEven">
<tr class="sortHeader">
<th>User</th> <th>Object number</th>
</tr>
#set($hql = "select obj.name from BaseObject obj where obj.className='XWiki.XWikiUsers'")
## display 10 users
## set($results = $xwiki.search($hql, 10, 0))
#set($results = $xwiki.search($hql))
#foreach ($User in $results)
## * [[$User]]
## if you have more than one object on a page, you will have to loop over them and use "$doc.use"
#set($MyDoc = $xwiki.getDocument("$User"))
#set($class = $xwiki.getClass("$className"))
#set($count = 0)
#set ($displayUser="")
## loop over all objects
#foreach($obj in $MyDoc.getObjects("$className"))
#set($count = $velocityCount)
##set ($displayUser="* [[$User]]: $count")
#set ($displayUser='<tr><td>' + [[$User]] + '</td><td>' + $count + '</td></tr>')
#end
$!displayUser
#end
</table>
{{/html}}
{{/velocity}}
Pascal B
Thanks to @Pbas I had a nice way to check the other users on my system and sure enough it seems the other major editor has over 200 notification filters.
Our system also syncs the users with Windows AD, so it appears that enabling notifications is the default.
Another benefit I am seeing since I deleted all those notifications - the jvm memory usage is way down; roughly speaking from 2.5GB to 750 MB.
Hi.
On the XWiki’s side, we are going to:
It’s going to require some work so I hope we will have this for 10.7.
In the meantime, you could:
xwiki.properties: xwiki.plugin.watchlist.automaticwatch=none
(even if the watchlist is not used anymore, there is a fallback to this property)
That should help until we provide an official fix. Sorry for that inconvenience. We are working on it!
Guillaume
Edit: it’s
xwiki.plugin.watchlist.automaticwatch=none
I have created a new JIRA issue for the filter preferences problem:
https://jira.xwiki.org/browse/XWIKI-15445
For the SQL problem, the following fixed issue should already be an improvement:
https://jira.xwiki.org/browse/XWIKI-15440
Until this problem is properly fixed, you could disable the autowatch mode in XWiki UI’s in the next XWiki version:
Thanks,
Guillaume
You should definitely not end up with all those documents taken into account since the filter stream job produce BeginFoldEvent which tells the notification to not take into account any document event during this process.
You used standard Filter Stream app, right ? Not some custom script ?
I think the priority before trying to optimize the notification for such volume is to find how you ended up with such nonsense volume in the first place and fix that. In general we need to make sure the notification module properly take into account BeginFoldEvent marker and that this marker is used as much as possible when it make sense (for example you should not end with a preference for each document in a space you moved, either nothing or a single preference for the space itself).
Let me clarify.
The imported pages didn’t create notifications, only those pages I created manually.
The 700+ notification filters included some manually created pages from the test instance, and all the other pages from manually create pages in the main and one other sub wiki.
Does that clear things up?
Yep but I still find 700 very big for hand written pages.