Solr and Discussion forums

[Carlos del Castillo, modificado 15 Anos atrás.]

I have setup a test liferay environment and a Solr search engine. I installed the plugin according to the wiki and Solr is indexing all the content from Wiki, Blogs and forums.

When I do the search on wiki and blogs the results are correct, but when doing a search on the forums I get no results.

Running a sniffer between the Solr server and liferay I see the search query and Solr is returning valid results but for some reason the search function in the forums is not showing any results.

Is anybody else seeing this behavior? Anybody with some pointers on how to solve this or what other information will be useful to solve it or to open a ticket in liferay bug tracking tool?

Thank you very much.

[Bruno Farache, modificado 15 Anos atrás.]

Hi Carlos, which version of Solr are you using?

Can you test if Journal Content Search works? If it doesn't I might know what's going on...

[Carlos del Castillo, modificado 15 Anos atrás.]

Bruno, thanks for the quick reply. I tested the search on the journal content and didn't work. I reindexed the content manually, tested the search directly on solr admin page (content:query) and the content is indexed and returned by a simple query. But from the journal search portlet does not display any results.

Any ideas?

Thanks again!

[Bruno Farache, modificado 15 Anos atrás.]

Both searches are filtered by ThreadHits (forum) and ContentHits (journal), if both aren't working the problem is probably in these classes. I will test it locally soon and come back with a fix, hopefully...

[Carlos del Castillo, modificado 15 Anos atrás.]

Thank you so much Bruno, the Solr version I'm using is 1.2.0

[Bruno Farache, modificado 15 Anos atrás.]

Can you make a test very quick while I'm setting up Solr here?

Comment lines 104 and 106 at:

/html/portlet/message_boards/search.jsp

If it works we confirm that the problem is in ThreadHits

[Carlos del Castillo, modificado 15 Anos atrás.]

Bruno,
I commented out the following lines:

//ThreadHits threadHits = new ThreadHits();

//threadHits.recordHits(results, searchContainer.getStart(), searchContainer.getEnd());

But now it's seems that the JSP is broken on the logs I just see

23:10:47,446 ERROR [jsp:52]

And on the browser the portlet does not render any content, just displays the Search tab. So I guess we need those lines for something.

Thanks again!

[Carlos del Castillo, modificado 15 Anos atrás.]

Bruno, may this can help, here is the xml response that Solr is sending after doing a search on the forums:

<!--?xml version="1.0" encoding="UTF-8"?-->
<response>
  <lst name="responseHeader">
    <int name="status">0</int>
    <int name="QTime">1</int>
    <lst name="params">
      <str name="fl">score</str>
      <str name="q">
	+(+portletId:19 +groupId:10208 +(categoryId:0 categoryId:10301 
	categoryId:13505 categoryId:13281)) +(userName:test title:test 
	content:test tagsEntries:test)
      </str>
    </lst>
  </lst>
  <result name="response" numfound="4" start="0" maxscore="3.1238697">
    <doc>
      <float name="score">3.1238697</float>
      <str name="categoryId">13281</str>
      <str name="companyId">10109</str>
      <str name="content">new thread in test category</str>
      <str name="entryClassName">com.liferay.portlet.messageboards.model.MBMessage</str>
      <str name="entryClassPK">13295</str>
      <str name="groupId">10208</str>
      <str name="modified">20081001222519</str>
      <str name="portletId">19</str>
      <str name="threadId">13296</str>
      <str name="title">forum test</str>
      <str name="uid">19_PORTLET_13295</str>
      <str name="userName">Some User</str>
    </doc>
    <doc>
      <float name="score">3.0721412</float>
      <str name="categoryId">10301</str>
      <str name="companyId">10109</str>
      <str name="content">ANS testing...</str>
      <str name="entryClassName">com.liferay.portlet.messageboards.model.MBMessage</str>
      <str name="entryClassPK">12258</str>
      <str name="groupId">10208</str>
      <str name="modified">20081001222519</str>
      <str name="portletId">19</str>
      <str name="threadId">12259</str>
      <str name="title">Test ANS</str>
      <str name="uid">19_PORTLET_12258</str>
      <str name="userName">Some User</str>
    </doc>
    <doc>
      <float name="score">2.8801155</float>
      <str name="categoryId">10301</str>
      <str name="companyId">10109</str>
      <str name="content">testing</str>
      <str name="entryClassName">com.liferay.portlet.messageboards.model.MBMessage</str>
      <str name="entryClassPK">13013</str>
      <str name="groupId">10208</str>
      <str name="modified">20081001222519</str>
      <str name="portletId">19</str>
      <str name="threadId">13014</str>
      <str name="title">Leia test</str>
      <str name="uid">19_PORTLET_13013</str>
      <str name="userName">Some User</str>
    </doc>
    <doc>
      <float name="score">2.560073</float>
      <str name="categoryId">10301</str>
      <str name="companyId">10109</str>
      <str name="content">Cool</str>
      <str name="entryClassName">com.liferay.portlet.messageboards.model.MBMessage</str>
      <str name="entryClassPK">12941</str>
      <str name="groupId">10208</str>
      <str name="modified">20081001222519</str>
      <str name="portletId">19</str>
      <str name="threadId">13014</str>
      <str name="title">RE: Leia test</str>
      <str name="uid">19_PORTLET_12941</str>
      <str name="userName">Some User</str>
    </doc>
  </result>
</response>

[Bruno Farache, modificado 15 Anos atrás.]

Hi Carlos,

I tried to install Solr yesterday but something was going wrong, I guess I had a wrong build, I'm trying to do it now...

I remembered that a similar problem was already fixed, but I would like to double check if it's the same you are having now. The bug was affecting not only Solr but also Lucene.

Take a look and these jsps/java:

http://support.liferay.com/browse/LEP-6623?page=com.atlassian.jira.plugin.ext.subversion:subversion-commits-tabpanel

If your Liferay is older than 09/July you don't have this fix.

[Carlos del Castillo, modificado 15 Anos atrás.]

Bruno, I did the test with Lucene and at least the Dicussion Forum search works. The lifreay version I'm using is:

Liferay Portal 5.1.1 (Calvin / Build 5101 / August 11, 2008)

Thanks!

[Bruno Farache, modificado 15 Anos atrás.]

That's right Carlos, it's a different issue, I've finally tested it locally, debugging it right now...

[Bruno Farache, modificado 15 Anos atrás.]

Hey Carlos, fixed in:

http://support.liferay.com/browse/LEP-7724

Revision 21167 of plugins svn. You need to modify one file in solr-web and redeploy it, here are the changes:

http://lportal.svn.sourceforge.net/viewvc/lportal/plugins/trunk/webs/solr-web/docroot/WEB-INF/src/com/liferay/portal/search/solr/SolrIndexSearcherImpl.java?r1=20719&r2=21167

Thanks for bringing this up!

[Carlos del Castillo, modificado 15 Anos atrás.]

Awesome, thank you very much Bruno, I'll checkout the plugin from the svn repository and let you know.

[Bruno Farache, modificado 15 Anos atrás.]

I would recommend to download the source code of the plugin how it was in version 5.1.1 and not the current source code, then change only these lines that I pointed.

Some parts of the plugin may have changed from version 5.1.1 that won't be compatible with your Liferay at version 5.1.1. That is, the current plugin source code is only compatible with the current Liferay trunk.

Seems like journal content search is still not working, it's a different issue... debugging it...

[Carlos del Castillo, modificado 15 Anos atrás.]

May be this is stupid question, but how can I download the plugin source code of the specific version? I looked in Subversion but could not find any tags for the plugins, the only tags I saw were for Liferay Portal. Any suggestions?

[Bruno Farache, modificado 15 Anos atrás.]

Checkout revision 19295, it's the one you want.

To figure out this I used smartsvn to see all logs of changes to the solr-web folder since its creation. This plugin was released on 8/15/08 and the last change before it was on 8/14, which is revision 19295.

[Bruno Farache, modificado 15 Anos atrás.]

The journal content search bug was just fixed in:

http://support.liferay.com/browse/LEP-7731

revision 21186

[Carlos del Castillo, modificado 15 Anos atrás.]

Bruno, finally I was able to compile the plugin and apply the changes to the SolrIndexSearcherImpl.java file here is the diff output if somebody else needs it. I used the revision 19295 that you mention.

24a25
> import com.liferay.portal.kernel.dao.orm.QueryUtil;
102a104,108
>  
> 		if ((start == QueryUtil.ALL_POS) && (end == QueryUtil.ALL_POS)) {
> 			start = 0;
>                         end = length;
>                 }

Now I can search the discussions, the journal content can be searched after reindexing.

Thank you so much for all your help, I just have one more question for you, I don't know if you notice but after installing the solr plugin the Plugins Portlet does not work, this happened to me in two different environments. Here are the logs from catalina.out

Oct 2, 2008 9:30:52 PM org.apache.solr.core.SolrCore execute
INFO: /select fl=score&q=%2B(%2BportletId:PluginPackageIndexer)+%2B(type:portlet)+%2B(status:notInstalled+status:olderVersionInstalled) 0 1
21:30:52,760 ERROR [SolrIndexSearcherImpl:80] Error while sending request to Solr
java.lang.IndexOutOfBoundsException: Index: 10, Size: 10
	at java.util.ArrayList.RangeCheck(ArrayList.java:547)
	at java.util.ArrayList.get(ArrayList.java:322)
	at com.liferay.portal.search.solr.SolrIndexSearcherImpl.subset(SolrIndexSearcherImpl.java:128)
	at com.liferay.portal.search.solr.SolrIndexSearcherImpl.search(SolrIndexSearcherImpl.java:77)
	at com.liferay.portal.search.solr.SolrSearchEngineUtil.search(SolrSearchEngineUtil.java:78)
	at com.liferay.portal.search.solr.messaging.SolrReaderMessageListener.doCommandSearch(SolrReaderMessageListener.java:92)
	at com.liferay.portal.search.solr.messaging.SolrReaderMessageListener.doReceive(SolrReaderMessageListener.java:75)
	at com.liferay.portal.search.solr.messaging.SolrReaderMessageListener.receive(SolrReaderMessageListener.java:46)
	at com.liferay.portal.kernel.messaging.InvokerMessageListener.receive(InvokerMessageListener.java:69)
	at com.liferay.portal.kernel.messaging.ParallelDestination$1.run(ParallelDestination.java:59)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
	at java.lang.Thread.run(Thread.java:619)

There are some other errors after this one, let me know if you need them.

Thanks again for all your help and a really fast fix.

[Bruno Farache, modificado 15 Anos atrás.]

Yes, there was some problems with paginated results, this one fixes it:

http://support.liferay.com/browse/LEP-7740

Regarding the journal articles bug, you will always need to re-index so they can be searched unless you apply the fix I mentioned in my last post (LEP-7731).

[saurabh jain, modificado 13 Anos atrás.]

Hi Bruno,

I am using trunk for portal, plugins and apache solr 1.4.
I am refering http://www.liferay.com/web/guest/community/wiki/-/wiki/Main/Pluggable+Enterprise+Search+with+Solr
I am getting error.

log4j:ERROR The class "org.apache.log4j.Appender" was loaded by
log4j:ERROR [WebappClassLoader
  delegate: false
  repositories:
    /WEB-INF/classes/
----------> Parent Classloader:
org.apache.catalina.loader.StandardClassLoader@10bc49d
] whereas object of type
log4j:ERROR "org.apache.log4j.ConsoleAppender" was loaded by [WebappClassLoader
  delegate: false
  repositories:
    /WEB-INF/classes/
----------> Parent Classloader:
org.apache.catalina.loader.StandardClassLoader@10bc49d
].
log4j:ERROR Could not instantiate appender named "CONSOLE".
May 7, 2010 7:40:28 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory tunnel-web
07:40:28,964 ERROR [SolrIndexWriterImpl:52] org.apache.solr.common.SolrException
: ERRORunknown_field_uid

ERRORunknown_field_uid

request: http://localhost:8983/solr/update?wt=javabin&version=2.2
org.apache.solr.common.SolrException: ERRORunknown_field_uid

ERRORunknown_field_uid

request: http://localhost:8983/solr/update?wt=javabin&version=2.2
        at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(Commo
nsHttpSolrServer.java:343)
        at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(Commo
nsHttpSolrServer.java:183)
        at com.liferay.portal.search.solr.server.BasicAuthSolrServer.request(Bas
icAuthSolrServer.java:96)
        at org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequ
est.java:217)
        at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:63)
        at com.liferay.portal.search.solr.SolrIndexWriterImpl.addDocument(SolrIn
dexWriterImpl.java:45)
        at com.liferay.portal.search.solr.SolrIndexWriterImpl.updateDocument(Sol
rIndexWriterImpl.java:138)
        at com.liferay.portal.kernel.search.messaging.SearchWriterMessageListene
r.doReceive(SearchWriterMessageListener.java:88)
        at com.liferay.portal.kernel.search.messaging.SearchWriterMessageListene
r.receive(SearchWriterMessageListener.java:35)
        at com.liferay.portal.kernel.messaging.InvokerMessageListener.receive(In
vokerMessageListener.java:65)
        at com.liferay.portal.kernel.messaging.ParallelDestination$1.run(Paralle
lDestination.java:63)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
utor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:908)
        at java.lang.Thread.run(Thread.java:619)
07:40:28,980 ERROR [SearchWriterMessageListener:38] Unable to process message {d
estinationName=liferay/search_writer, responseDestinationName=null, responseId=n
ull, payload={companyId=0, document={uid=[PluginPackageIndexer_PORTLET_solr-web/
solr-web/unknown/war], moduleId=[solr-web/solr-web/unknown/war], installedVersio
n=[unknown], status=[newerVersionInstalled], tag=[], artifactId=[solr-web], osi-
approved-license=[false], type=[], version=[unknown], content=[solr-web null  ],
 groupId=[solr-web], title=[solr-web], repositoryURL=[LOCAL_URL], portletId=[Plu
ginPackageIndexer], license=[]}, documents=null, end=0, id=null, ids=null, query
=null, searchEngineCommand=UPDATE_DOCUMENT, sorts=null, start=0}, values=null}
com.liferay.portal.kernel.search.SearchException: ERRORunknown_field_uid

ERRORunknown_field_uid

request: http://localhost:8983/solr/update?wt=javabin&version=2.2
        at com.liferay.portal.search.solr.SolrIndexWriterImpl.addDocument(SolrIn
dexWriterImpl.java:54)
        at com.liferay.portal.search.solr.SolrIndexWriterImpl.updateDocument(Sol
rIndexWriterImpl.java:138)
        at com.liferay.portal.kernel.search.messaging.SearchWriterMessageListene
r.doReceive(SearchWriterMessageListener.java:88)
        at com.liferay.portal.kernel.search.messaging.SearchWriterMessageListene
r.receive(SearchWriterMessageListener.java:35)
        at com.liferay.portal.kernel.messaging.InvokerMessageListener.receive(In
vokerMessageListener.java:65)
        at com.liferay.portal.kernel.messaging.ParallelDestination$1.run(Paralle
lDestination.java:63)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
utor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:908)
        at java.lang.Thread.run(Thread.java:619)
May 7, 2010 7:40:30 AM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-8080
May 7, 2010 7:40:30 AM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:8009
May 7, 2010 7:40:30 AM org.apache.jk.server.JkMain start
INFO: Jk running ID=0 time=0/31  config=null
May 7, 2010 7:40:30 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 32243 ms
07:40:35,563 ERROR [SolrIndexWriterImpl:119] org.apache.solr.common.SolrExceptio
n: undefined_field_portletId

undefined_field_portletId

request: http://localhost:8983/solr/update?wt=javabin&version=2.2
org.apache.solr.common.SolrException: undefined_field_portletId

undefined_field_portletId

request: http://localhost:8983/solr/update?wt=javabin&version=2.2
        at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(Commo
nsHttpSolrServer.java:343)
        at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(Commo
nsHttpSolrServer.java:183)
        at com.liferay.portal.search.solr.server.BasicAuthSolrServer.request(Bas
icAuthSolrServer.java:96)
        at org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequ
est.java:217)
        at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java
:101)
        at com.liferay.portal.search.solr.SolrIndexWriterImpl.deletePortletDocum
ents(SolrIndexWriterImpl.java:111)
        at com.liferay.portal.kernel.search.messaging.SearchWriterMessageListene
r.doReceive(SearchWriterMessageListener.java:83)
        at com.liferay.portal.kernel.search.messaging.SearchWriterMessageListene
r.receive(SearchWriterMessageListener.java:35)
        at com.liferay.portal.kernel.messaging.InvokerMessageListener.receive(In
vokerMessageListener.java:65)
        at com.liferay.portal.kernel.messaging.ParallelDestination$1.run(Paralle
lDestination.java:63)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
utor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:908)
        at java.lang.Thread.run(Thread.java:619)
07:40:35,594 ERROR [SearchWriterMessageListener:38] Unable to process message {d
estinationName=liferay/search_writer, responseDestinationName=null, responseId=n
ull, payload={companyId=0, document=null, documents=null, end=0, id=PluginPackag
eIndexer, ids=null, query=null, searchEngineCommand=DELETE_PORTLET_DOCUMENTS, so
rts=null, start=0}, values=null}
com.liferay.portal.kernel.search.SearchException: undefined_field_portletId

undefined_field_portletId

request: http://localhost:8983/solr/update?wt=javabin&version=2.2
        at com.liferay.portal.search.solr.SolrIndexWriterImpl.deletePortletDocum
ents(SolrIndexWriterImpl.java:121)
        at com.liferay.portal.kernel.search.messaging.SearchWriterMessageListene
r.doReceive(SearchWriterMessageListener.java:83)
        at com.liferay.portal.kernel.search.messaging.SearchWriterMessageListene
r.receive(SearchWriterMessageListener.java:35)
        at com.liferay.portal.kernel.messaging.InvokerMessageListener.receive(In
vokerMessageListener.java:65)
        at com.liferay.portal.kernel.messaging.ParallelDestination$1.run(Paralle
lDestination.java:63)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
utor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:908)
        at java.lang.Thread.run(Thread.java:619)

please help me.

Thanks
saurabh

[François Le Quemener, modificado 13 Anos atrás.]

Hi,
have you resolved your problem? I have the same (ERRORunknown_field_uid)

Thanks

[Ranga Rao Bobbili, modificado 11 Anos atrás.]

Hi Bruno or friends,

I am facing issue with solr search engine and not able to retrieve results from solr. please find my following liferay therad link and if possible reply the same

http://www.liferay.com/community/forums/-/message_boards/message/21180581

Thanks in advance..................

Best Regards,
Ranga Rao Bobbili
Adaequare INC

[Ranga Rao Bobbili, modificado 11 Anos atrás.]

Thanks friends for your valuable support to resolve my solr indexing issue for custom entities.

I have resolved my issue using enterprise solr plugin.

Thanks & Regards,
Ranga Rao Bobbili
Adaequare INC