Foros de discusión

Liferay constantly triggers reindexing

Lubomir Georgiev, modificado hace 8 años.

Liferay constantly triggers reindexing

New Member Mensajes: 23 Fecha de incorporación: 29/10/13 Mensajes recientes
Hello,

We have migrated our Liferay 6.1 to 6.2 and we noticed one strange thing - Liferay 6.2 is automatically reindexing.

We have tried both lucene and SOLR but the result is one and the same.
Automatic reindexing is triggered when the server is started and on first access of home page.

The reindexing is triggered on each 30-40 mins and it takes about 15-20 mins.


In order to see what is happening with the indexing I am setting all the lucene* logs properties to ALL. This change produce a lot of records in catalina.out.

Do you have any idea what could be the reason for this?

Thanks in advance.
thumbnail
David H Nebinger, modificado hace 8 años.

RE: Liferay constantly triggers reindexing

Liferay Legend Mensajes: 14914 Fecha de incorporación: 2/09/06 Mensajes recientes
Do you have values set for these properties in portal-ext.properties?

index.on.startup=true
index.on.startup.delay=30
Lubomir Georgiev, modificado hace 8 años.

RE: Liferay constantly triggers reindexing

New Member Mensajes: 23 Fecha de incorporación: 29/10/13 Mensajes recientes
David H Nebinger:
Do you have values set for these properties in portal-ext.properties?

index.on.startup=true
index.on.startup.delay=30



Hi David,

No, we have set index.on.startup to false as the migration phase is not completed and in order to save time on server startup we have disable it.

Do you think that it is something relate to this properties?
thumbnail
David H Nebinger, modificado hace 8 años.

RE: Liferay constantly triggers reindexing

Liferay Legend Mensajes: 14914 Fecha de incorporación: 2/09/06 Mensajes recientes
Liferay requires the index, so if there is not one there at startup it will index anyway.
Lubomir Georgiev, modificado hace 8 años.

RE: Liferay constantly triggers reindexing

New Member Mensajes: 23 Fecha de incorporación: 29/10/13 Mensajes recientes
Update:

Today I used JProfiler to check what is going on in LR and it seems that the problem is related to the cluster configuration of our environment.

Here are more details about the problem:
1. The 2xLR servers are started up
2. On the server that is started first - go to login page, open home page, open control panel and click reindex. The reindexing is completed in 15 mins. On this server (the first one) there is no automatic reindexing.
3. On the second started server - repeat the steps from point 2. Reindexing is completed about 15 mins but the strange thing is that on interval of X mins the server trigers reindexing again and again.

I also checked data/lucene folder which is about 120 MB after full reindexing and when the automatic reindexing starts its size is about 80 MB.

Again the reindexing is trigered only on the second started server.


David - we also set the index.on.start to true but the result is the same.
thumbnail
David H Nebinger, modificado hace 8 años.

RE: Liferay constantly triggers reindexing

Liferay Legend Mensajes: 14914 Fecha de incorporación: 2/09/06 Mensajes recientes
When you click on "reindex all search indexes" the instance will broadcast a message to all nodes and they should also start reindexing. When using Lucene, all indexes are local to the node so each server maintains it's own index files. When using Solr, the indexing is handled by the external Solr server so each node will not individually manage indexing.

When you update an indexed entity, the node will broadcast a message to the other nodes to update the index entry for the specified entity.

If you have two separate nodes pointing to the same lucene directory, well this will cause you problems as each will attempt to lock the index and this will throw the other nodes out of whack.
Lubomir Georgiev, modificado hace 8 años.

RE: Liferay constantly triggers reindexing

New Member Mensajes: 23 Fecha de incorporación: 29/10/13 Mensajes recientes
Hi David,

We still have problem with the constant automatic reindex0ing of our cluster.

As you mentioned - we have 2 separate LR servers which are using lucene. They are not pointing to one and the same directory but to two different folders. Both servers are using common DB.

The behaviuor is very strange as the reindexing is trigered only to one of the servers ( depending on the start oreder of the servers)
We have the following repeated steps:
1. The first reindexing is started ~40 min. after the start of the setver.

2. Reindexing takes approximately one and the same time (about 20 min.)

3. There is pause interval between the last and next reindexing ( about 5 min)

I can see the server is reindexing by using jprofiler or changing the log level to ALL for lucene.

I have also noticed that this is also valid when i start only one of the servers.

Index.on.start is false.

This problem is also impacting the performance x2.
thumbnail
Juan Gonzalez, modificado hace 8 años.

RE: Liferay constantly triggers reindexing

Liferay Legend Mensajes: 3089 Fecha de incorporación: 28/10/08 Mensajes recientes
Hi Lubomir,

this happens when having a cluster and slave servers ask master to provide its index. Then each one will get that and reindex using that new index.

If your index is very big this can cause your slave servers to delay when starting up.

Maybe you can try using SolR, that should improve this.
Lubomir Georgiev, modificado hace 8 años.

RE: Liferay constantly triggers reindexing

New Member Mensajes: 23 Fecha de incorporación: 29/10/13 Mensajes recientes
Hi,

I think that you are right but this should happen only one time. For example for the last night (for period of 12 hours) we have about 24 automatic reindexings, whixh means 24 * ~20 min = 8 hours spent in reindexing and performance on 50% (I mean that during this time the portal pages are opened for 5-6 sec. instead of 2 sec)......

The console prints things like this :

09:05:00,747 DEBUG [liferay/search_writer/SYSTEM_ENGINE-66][IndexAccessorImpl:298] Indexing Document<stored,indexed<uid:125_portlet_5201163> stored,indexed<scopegroupid:10180> stored,indexed,tokenized<screenname:bk04065> stored,indexed<screenname_sortable:bk04065> stored,indexed<organizationcount:0> stored,indexed<usergroupids:5165177> stored,indexed<userid:5201163> stored,indexed<username:asen haj-mahmud> stored,indexed<portletid:125> stored,indexed,tokenized<firstname:asen> stored,indexed<roleid:10163> stored,indexed,tokenized<jobtitle:brokers> stored,indexed<jobtitle_sortable:brokers> stored,indexed<entryclassname:com.liferay.portal.model.user> stored,indexed,tokenized<lastname:haj-mahmud> stored,indexed<lastname_sortable:haj-mahmud> stored,indexed<status:0> stored,indexed,tokenized<emailaddress:5201163@no-emailaddress.com> stored,indexed<entryclasspk:5201163> stored,indexed<modified:20150520034214> stored,indexed<groupid:10180> stored,indexed<groupids:10180> stored,indexed<firstname_sortable:asen> stored,indexed,tokenized,omitNorms,indexOptions=DOCS_ONLY<modified_sortable:1432082534000> stored,indexed<companyid:10154> stored,indexed,tokenized<fullname:asen haj-mahmud> stored,indexed<roleids:10164> stored,indexed<roleids:10165>&gt;
09:05:00,752 DEBUG [liferay/search_writer/SYSTEM_ENGINE-66][LuceneIndexWriter:124] Updated document 125_PORTLET_5201163
</roleids:10165></roleids:10164></fullname:asen></companyid:10154></modified_sortable:1432082534000></firstname_sortable:asen></groupids:10180></groupid:10180></modified:20150520034214></entryclasspk:5201163></emailaddress:5201163@no-emailaddress.com></status:0></lastname_sortable:haj-mahmud></lastname:haj-mahmud></entryclassname:com.liferay.portal.model.user></jobtitle_sortable:brokers></jobtitle:brokers></roleid:10163></firstname:asen></portletid:125></username:asen></userid:5201163></usergroupids:5165177></organizationcount:0></screenname_sortable:bk04065></screenname:bk04065></scopegroupid:10180></stored,indexed<uid:125_portlet_5201163>



Is it possible LDAP sync to trigger such a reindexing?
thumbnail
David H Nebinger, modificado hace 8 años.

RE: Liferay constantly triggers reindexing

Liferay Legend Mensajes: 14914 Fecha de incorporación: 2/09/06 Mensajes recientes
Should not be a complete reindex, but the add/update of a user (from ldap import) will need indexed, but it should not be a complete one.
varshali c, modificado hace 7 años.

RE: Liferay constantly triggers reindexing

New Member Mensajes: 11 Fecha de incorporación: 21/06/16 Mensajes recientes
Hi David,

We are also facing same issue. We have 2 nodes in cluster with cluster link enabled. We have used asset publisher to display blogs dynamically. But each node starts loading indexing file automatically from another after every 2hrs and it takes 15 minutes to complete the process. During this time blogs disappears from asset publisher.

W e haven't set
index.on.startup=true. It is false only. Liferay version is 6.2 GA2 and lucene is present separately on both nodes with lucene.replicate.write=true.

Problem was started all of sudden after restart of application servers i.e. tomcat and now it is consistent. Tried full reindex but still no luck.

We have some restrictions in terms of server capabilities hence can't use SOLR.
Marcu Catalin, modificado hace 6 años.

RE: Liferay constantly triggers reindexing

New Member Mensajes: 2 Fecha de incorporación: 25/09/17 Mensajes recientes
Hi, Mr. David.
I'm using liferay 6.2 CE. Everything was fine, but by mistake, I pressed on 'Reindex all searching index' button. Then all Organisations and Users dissapeared from 'Users and Organisations'. I tried to solve it in many ways, but without result ... What happened and what can I do?
Please, help me.
Thanks advance!
thumbnail
Marcial Calvo Valenzuela, modificado hace 6 años.

RE: Liferay constantly triggers reindexing

New Member Mensajes: 18 Fecha de incorporación: 27/01/14 Mensajes recientes
Marcu Catalin:
Hi, Mr. David.
I'm using liferay 6.2 CE. Everything was fine, but by mistake, I pressed on 'Reindex all searching index' button. Then all Organisations and Users dissapeared from 'Users and Organisations'. I tried to solve it in many ways, but without result ... What happened and what can I do?
Please, help me.
Thanks advance!

Hi Marcu,

when you press the 'Reindex all searching index' button, reindex process is executing. For debug the process and show what are it doing, you can put the following classes to debug:
com.liferay.portal.kernel.search.SearchEngineUtil --&gt; DEBUG
com.liferay.portal.search.solr --&gt; (DEBUG if you are using solr indexer)


If you don't know how put this classes on DEBUG Mode, please, follow the next link:
https://dev.liferay.com/discover/deployment/-/knowledge_base/6-2/liferays-logging-system

While the process is running, you can't show any user and organization until the entity was indexed.

Remember return the correct logging level when the process has finished.

Regards.
Marcu Catalin, modificado hace 6 años.

RE: Liferay constantly triggers reindexing

New Member Mensajes: 2 Fecha de incorporación: 25/09/17 Mensajes recientes
Thanks! I've solve it! God bless you!
thumbnail
Jorge Díaz, modificado hace 6 años.

RE: Liferay constantly triggers reindexing

Liferay Master Mensajes: 753 Fecha de incorporación: 9/01/14 Mensajes recientes
Marcu Catalin:
Hi, Mr. David.
I'm using liferay 6.2 CE. Everything was fine, but by mistake, I pressed on 'Reindex all searching index' button. Then all Organisations and Users dissapeared from 'Users and Organisations'. I tried to solve it in many ways, but without result ... What happened and what can I do?
Please, help me.
Thanks advance!
Hi Marcu,
Try using my Index Checker portlet, see: https://web.liferay.com/marketplace/-/mp/application/70121999

With that portlet you can check your index status and make selective reindex operations.

Regards