Random missing pages with solr search engine

Help / Discuss
1

Hello,

I have some missing page when I try to find them with suggest search and Main.Search page.
I tried with title:MyPieceOfTitle and MyPieceOfTitle and it doesn’t work.
My page exist and I can find it with Ctrl+G: MyPieceOfTitle (I suppose Ctrl+G feature doesn’t use solr index)
I’m login with Admin account and can see hidden pages.

This fix this issue:

  • stop Tomcat/ remove .solr and restart tomcat :-/
  • save again MySpace.MyMissingpage (in this case other page missing but I don’t know which one of course)

Could you help me to debug this?
It is very annoying issue about the search engine of course and to add a link to an another page…

I use xwiki 8.4.5 (but I had same problem with 8.4.4 version: Solr search engine doesn't work anymore - #14 by mflorea)

This issue randomly appaear and haven’t some usefull information in catalina.out.

Maybe I must increase this in xwiki.properties?

#-# The maximum number of elements in the background queue of elements to index/delete
#-# The default is 10000.
# solr.indexer.queue.capacity=100000

or I must increase verbosity of solr logs in xwiki preferences but which recorder ? org.apache.solr ?

I have the same issue on my 2 servers (I can debuging on my developpment server).

ty for any help

Pascal B

2

Just out of curiosity (and I’m not familiar with xwiki 8.x) did you try rebuilding the index through the administrator tools?

Was this page that you cannot find recently created? Or is it an older, at least a day or so old?

3

Yes, but it doesn’t fix the issue.

It is an “old” page created the 2017/01/26 ’ and last edited the 2018/01/17.
The page name is “SMQ-P3-FEA-Concessions hydroélectriques” but others pages missing when this problem ocured.

4

No idea to help me?

I must add a cron to stop tomcat, remove ./solr subdirectory and start Tomcat then? (a little bit uggly, isn’t it?)

5

Still searching: maybe a trace about my solr corrupted index in my catalina.out (after catalina startup)

2018-01-18 19:02:10,447 [solr/indexer job group daemon thread - org.xwiki.search.solr.internal.job.IndexerJob@62f08f51] ER
ROR o.x.s.s.i.j.IndexerJob         - Exception thrown during job execution 
java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
        at java.util.LinkedList.checkElementIndex(LinkedList.java:555)
        at java.util.LinkedList.get(LinkedList.java:476)
        at java.util.Collections$UnmodifiableList.get(Collections.java:1309)
        at org.xwiki.search.solr.internal.job.DocumentReferenceComparator.compare(DocumentReferenceComparator.java:42)
        at org.xwiki.search.solr.internal.job.DiffDocumentIterator.next(DiffDocumentIterator.java:122)
        at org.xwiki.search.solr.internal.job.IndexerJob.updateSolrIndex(IndexerJob.java:151)
        at org.xwiki.search.solr.internal.job.IndexerJob.updateSolrIndex(IndexerJob.java:134)
        at org.xwiki.search.solr.internal.job.IndexerJob.runInternal(IndexerJob.java:110)
        at org.xwiki.job.AbstractJob.runInContext(AbstractJob.java:206)
        at org.xwiki.job.AbstractJob.run(AbstractJob.java:189)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
2018-01-18 19:02:10,451 [solr/indexer job group daemon thread - org.xwiki.search.solr.internal.job.IndexerJob@62f08f51] INFO  o.x.s.s.i.j.IndexerJob         - Finished job of type [solr.indexer] with identifier [[solr, indexer]]

(to myself: must try this: CheckIndex for the rescue – Solr.pl)

6

This one definitely looks like a bug on XWiki’s side worth reporting. Always hard without a way to reproduce but the stack trace might help understanding what might have append and bulletproof it.

In the meantime usually the safest is to:

  • stop XWik
  • delete the /solr/ folder
  • restart XWiki which will reindex everything (and won’t go trough the code that produced this error which is a code comparing what’s in the index and what’s in the database)
7

This bug seem already fixed … on xwiki 8.4.6 (I’m on xwiki 8.4.4) : Loading...

Then I must:

  • merge Marius piece of code (If I manage)
  • write a batch to watch my log and delete solr like you said
  • or upgrade and wait…
8

Note that there is some nasty bugs that have been fixed in 8.4.5 and 8.4.6 (which is the last version of this branch, LTS now moved to 9.11.x) so upgrading to 8.4.6 would really be a good thing (but right now the safest and fastest is to just delete the index IMO :slight_smile: ).

9

I’m on xwiki 8.4.5 (I made a typing mistake). I already delete solr subdirectory but like it is a randomly annoying bug, I need to upgrade. I must check if adapt of my customization take a lot of time on 8.4.6 or 9.11 version. Ty

10

If same problem occured, we can check solr index with:

/home/xxx/jdk1.8.0_121/bin/java -cp /home/xxxxx/xwiki/WEB-INF/lib/lucene-core-6.2.1.jar -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex /home/xxxx/data_xwiki/solr/xwiki/data/index/

but because lock files we must stop tomcat server before, then I agree remove solr is better. I just writing this tip to debug if this bug is still there

11

Note that this bug is only in the synchronization which happen at startup and which is here to ensure that no document were missed which is not supposed to happen unless you are in a cluster (and some node was down when a modification has been made on another).

12

Ok. I’m not on a cluster. Hope that bug is my issue then (I will quick upgrade to 8.4.6 and later on 9.11).