Solr search engine doesn't work anymore

Help / Discuss
#1

Hello,

On my xwiki 8.4.4, it seem that solr didn’t find some page anymore.
Pages starts with “SMQ-P1” name. My xwiki doesn’t found page named “SMQ-P1 : Présentation xxx”
I fix temporaly my issue by deleting /data_xwiki/solr and restart my tomcat server but this issue coming again after 4 days…
Reindexing from xwiki in XWiki Preferences didn’t fix my issue
And now quick search displayed these red popup messages: “Impossible d’extraire des suggestions: 403"”
Do you know how can I debug my issue?
Or maybe reinit solr index without restart Tomcat (with command line on debian server)?

Ty
Pascal B

#2

403 means “Forbidden”. Maybe you have an access rights issue. You can look at the AJAX requests that retrieve the search suggestions and check where they go and what’s their response, using the browser’s developer tools.

Hope this helps,
Marius

#3

In XwikiPreference I switch on database in default search engine.
I have “Impossible d’extraire des suggestions: 403” still but quick search doesn’t found my page but Main.Search found my missing page with database search engine

#4

I’m searching with Admin account and I can view my missing page.
I must watch ajax request like you said

#5

There’s no search suggest source (see http://extensions.xwiki.org/xwiki/bin/view/Extension/Search+Application#HSearchSuggest) defined for the “database” search engine so it’s normal that you don’t get any suggestion when using this engine.

#6

ok ty good to know.
Then it is my solr search engine the problem.
I wonder how I can reinit it and debug my issue (without tomcat restart)

With solr on Main.search:
title:“smq-p1” found 2 pages on 7 only (I cheched with Main.Alldoc live table) :confused:

#7

In catalina.out I have lot of these entries:

2017-10-04 18:36:33,912 [XWiki Solr index thread] ERROR .DocumentSolrMetadataExtractor - Failed to retrieve the content of attachment [Attachment xwiki:Encyclopédie.xxxx@xxxx.odt] 
org.apache.tika.exception.TikaException: Zip bomb detected!

odt file is open office files. I wonder if it is the cause of my corrupt solr index…

#8

Ok you are right about “Impossible d’extraire des suggestions: 403” message issue. xwikiAllgroup couldn’t have view right on WikiManager/WikisSuggestSolrService page.
I fix this point.

Now I try to fix my solr engine :slight_smile:I’m login with Admin account and solr didn’t found some pages:
title:“smq-p1” found 2 pages on 7 only (I cheched with Main.Alldoc live table)

#9

If the pages that don’t appear in the search results are those that have these attachments then yes, this could be the problem.

#10

No my missing pages haven’t attachments.
I though about corrupt solr index with “zip bomb”…

#11

“zip bomb” should only prevent the indexing of the document that holds the attachment. Try this:

  • Go to the administration and look for the search index queue. Wait until the queue is 0 (no pending documents)
  • Edit the document that doesn’t appear in search results and do some small change then save.
  • Go back to the search index queue in the administration and wait for it to be 0
  • Look at the server logs for any indexing error that may have occurred after the document was saved.
  • Search for the document you modified
#12

ok ty.
I must wait to test your way because I fix it like few days ago: I remove xwiki_data/solr, work and temp tomcat drectory and start again Tomcat.

Thxs for the tip, I will wait that my issue occured again to post here (I hope not anyway)

#13

Ok I had same problem on my dev server…

Then, I have:

  • “SMQ-P1 : Présentation du processus” missing in search result.
  • I modified this one
  • and now this doc is not missing anymore in search result…

(I have other SMQ-P1 xxx pages with redirection but not for “SMQ-P1 : Présentation du processus”)

I didn’t found interesting logs about it but maybe I must activate debug log in some solr stuff (which one?) ?

When I started my tomcat (without /data/solr directory), I have this error:

2017-10-05 13:34:37,886 [solr/indexer job group daemon thread - org.xwiki.search.solr.internal.job.IndexerJob@3a3d8e15] ERROR o.x
.s.s.i.j.IndexerJ
ob         - Exception thrown during job execution 
java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
        at java.util.LinkedList.checkElementIndex(LinkedList.java:555)
        at java.util.LinkedList.get(LinkedList.java:476)
        at java.util.Collections$UnmodifiableList.get(Collections.java:1309)
        at org.xwiki.search.solr.internal.job.DocumentReferenceComparator.compare(DocumentReferenceComparator.java:42)
        at org.xwiki.search.solr.internal.job.DiffDocumentIterator.next(DiffDocumentIterator.java:122)
        at org.xwiki.search.solr.internal.job.IndexerJob.updateSolrIndex(IndexerJob.java:151)
        at org.xwiki.search.solr.internal.job.IndexerJob.updateSolrIndex(IndexerJob.java:134)
        at org.xwiki.search.solr.internal.job.IndexerJob.runInternal(IndexerJob.java:110)
        at org.xwiki.job.AbstractJob.runInContext(AbstractJob.java:206)
        at org.xwiki.job.AbstractJob.run(AbstractJob.java:189)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
2017-10-05 13:34:37,886 [solr/indexer job group daemon thread - org.xwiki.search.solr.internal.job.IndexerJob@3a3d8e15] INFO  o.x.s.s.i.j.IndexerJob         - Finished job of type [solr.indexer] with identifier [[solr, indexer]]
#14

From the stack trace it looks like DocumentReferenceComparator doesn’t support nested pages (it assumes all document references have the same number of components). Can you report the issue please?

Thanks,
Marius

#15

https://jira.xwiki.org/browse/XWIKI-14766

#16

In my catalina.out, I found this logs to when my issue occured:

2017-10-05 11:07:03,910 [searcherExecutor-7-thread-1] DEBUG .a.s.c.CachingDirectoryFactory - Releasing directory: /home/opera/data_xwiki/solr/xwiki/
data/index 2 false 
2017-10-05 11:07:03,912 [XWiki Solr index thread] DEBUG .u.p.LogUpdateProcessorFactory - PRE_UPDATE FINISH {commit=true&softCommit=false&df=text&wai
tSearcher=true} 
2017-10-05 11:07:03,912 [XWiki Solr index thread] INFO  .u.p.LogUpdateProcessorFactory - [xwiki]  webapp=null path=/update params={commit=true&softC
ommit=false&waitSearcher=true}{commit=} 0 64 
2017-10-05 11:07:03,952 [Lucene Merge Thread #31] INFO  o.a.s.u.LoggingInfoStream      - [SM][Lucene Merge Thread #31]: 60 msec to merge postings [7
7 docs] 
2017-10-05 11:07:03,955 [Lucene Merge Thread #31] INFO  o.a.s.u.LoggingInfoStream      - [SM][Lucene Merge Thread #31]: 2 msec to merge doc values [
77 docs]

then in XWikiPreferences in Core extension I’m looking for Lucene installed extension and found … 18 Lucene xx extension (version 6.2.1). I’m using xwiki 8.4.4 but I had some upgrade from xwiki 6.2.xx

I need to remove these core extension? If yes how?

#17

Solr is based on Lucene and that is the Lucene version not the XWiki version. So no problem here.