ElasticSearch issue

[6年前 に Daniel G によって更新されました。]

Hi everyone.

I'm using liferay DXP and I have an issue with elasticSearch and I don'k know how to solve it, so I need any help you can give me.
I have this error when I start Liferay.

[localhost-startStop-1][com_liferay_portal_search:97] [com.liferay.portal.search.internal.SearchEngineHelperImpl(1537)] The addSearchEngineConfigurator method has thrown an exception 
java.lang.IllegalStateException: Unable to initialize Elasticsearch cluster: {_  "cluster_name" : "LiferayElasticsearchCluster",_  "status" : "red",_  "timed_out" : true,_  "number_of_nodes" : 1,_  "number_of_data_nodes" : 1,_  "active_primary_shards" : 1,_  "active_shards" : 1,_  "relocating_shards" : 0,_  "initializing_shards" : 1,_  "unassigned_shards" : 0,_  "delayed_unassigned_shards" : 0,_  "number_of_pending_tasks" : 0,_  "number_of_in_flight_fetch" : 0,_  "task_max_waiting_in_queue_millis" : 0,_  "active_shards_percent_as_number" : 50.0_} [Sanitized]
	at com.liferay.portal.search.elasticsearch.internal.ElasticsearchSearchEngine.waitForYellowStatus(ElasticsearchSearchEngine.java:344)
	at com.liferay.portal.search.elasticsearch.internal.ElasticsearchSearchEngine.initialize(ElasticsearchSearchEngine.java:112)
	at com.liferay.portal.kernel.search.SearchEngineProxyWrapper.initialize(SearchEngineProxyWrapper.java:95)
	at com.liferay.portal.kernel.search.AbstractSearchEngineConfigurator.setSearchEngine(AbstractSearchEngineConfigurator.java:436)
	at com.liferay.portal.kernel.search.AbstractSearchEngineConfigurator.initSearchEngine(AbstractSearchEngineConfigurator.java:366)
	at com.liferay.portal.kernel.search.AbstractSearchEngineConfigurator.initialize(AbstractSearchEngineConfigurator.java:293)
	at com.liferay.portal.search.elasticsearch.internal.ElasticsearchEngineConfigurator.initialize(ElasticsearchEngineConfigurator.java:91)
	at com.liferay.portal.kernel.search.AbstractSearchEngineConfigurator$1.dependenciesFulfilled(AbstractSearchEngineConfigurator.java:75)
	at com.liferay.registry.dependency.ServiceDependencyManager.verifyDependencies(ServiceDependencyManager.java:146)
	at com.liferay.registry.dependency.ServiceDependencyManager$ServiceDependencyServiceTrackerCustomizer.addingService(ServiceDependencyManager.java:242)
	at com.liferay.registry.internal.ServiceTrackerCustomizerAdapter.addingService(ServiceTrackerCustomizerAdapter.java:35)
	at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:941)
	at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:1)
	at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256)
	at org.osgi.util.tracker.AbstractTracked.trackInitial(AbstractTracked.java:183)
	at org.osgi.util.tracker.ServiceTracker.open(ServiceTracker.java:318)
	at com.liferay.registry.internal.ServiceTrackerWrapper.open(ServiceTrackerWrapper.java:191)
	at com.liferay.registry.dependency.ServiceDependency.open(ServiceDependency.java:73)
	at com.liferay.registry.dependency.ServiceDependencyManager.registerDependencies(ServiceDependencyManager.java:72)
	at com.liferay.portal.kernel.search.AbstractSearchEngineConfigurator.afterPropertiesSet(AbstractSearchEngineConfigurator.java:84)
	at com.liferay.portal.search.internal.SearchEngineHelperImpl.addSearchEngineConfigurator(SearchEngineHelperImpl.java:295)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.felix.scr.impl.inject.BaseMethod.invokeMethod(BaseMethod.java:224)
	at org.apache.felix.scr.impl.inject.BaseMethod.access$500(BaseMethod.java:39)
	at org.apache.felix.scr.impl.inject.BaseMethod$Resolved.invoke(BaseMethod.java:617)
	at org.apache.felix.scr.impl.inject.BaseMethod.invoke(BaseMethod.java:501)
	at org.apache.felix.scr.impl.inject.BindMethod.invoke(BindMethod.java:655)
	at org.apache.felix.scr.impl.manager.DependencyManager.doInvokeBindMethod(DependencyManager.java:1722)
	at org.apache.felix.scr.impl.manager.DependencyManager.invokeBindMethod(DependencyManager.java:1699)
	at org.apache.felix.scr.impl.manager.SingleComponentManager.invokeBindMethod(SingleComponentManager.java:371)
	at org.apache.felix.scr.impl.manager.DependencyManager$MultipleDynamicCustomizer.addedService(DependencyManager.java:320)
	at org.apache.felix.scr.impl.manager.DependencyManager$MultipleDynamicCustomizer.addedService(DependencyManager.java:291)
	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerAdded(ServiceTracker.java:1215)
	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerAdded(ServiceTracker.java:1136)
	at org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked.trackAdding(ServiceTracker.java:945)
	at org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked.track(ServiceTracker.java:881)
	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:1167)
	at org.apache.felix.scr.impl.BundleComponentActivator$ListenerInfo.serviceChanged(BundleComponentActivator.java:127)
	at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:109)
	at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:917)
	at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
	at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:862)
	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:801)
	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:127)
	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:225)
	at org.eclipse.osgi.internal.framework.BundleContextImpl.registerService(BundleContextImpl.java:464)
	at org.apache.felix.scr.impl.manager.AbstractComponentManager$3.register(AbstractComponentManager.java:886)
	at org.apache.felix.scr.impl.manager.AbstractComponentManager$3.register(AbstractComponentManager.java:873)
	at org.apache.felix.scr.impl.manager.RegistrationManager.changeRegistration(RegistrationManager.java:132)
	at org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:940)
	at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:740)
	at org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.addedService(DependencyManager.java:1012)
	at org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.addedService(DependencyManager.java:968)
	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerAdded(ServiceTracker.java:1215)
	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerAdded(ServiceTracker.java:1136)
	at org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked.trackAdding(ServiceTracker.java:945)
	at org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked.track(ServiceTracker.java:881)
	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:1167)
	at org.apache.felix.scr.impl.BundleComponentActivator$ListenerInfo.serviceChanged(BundleComponentActivator.java:127)
	at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:109)
	at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:917)
	at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
	at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:862)
	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:801)
	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:127)
	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:225)
	at org.eclipse.osgi.internal.framework.BundleContextImpl.registerService(BundleContextImpl.java:464)
	at com.liferay.portal.bootstrap.ModuleFrameworkImpl._registerService(ModuleFrameworkImpl.java:1092)
	at com.liferay.portal.bootstrap.ModuleFrameworkImpl._registerApplicationContext(ModuleFrameworkImpl.java:1058)
	at com.liferay.portal.bootstrap.ModuleFrameworkImpl.registerContext(ModuleFrameworkImpl.java:336)
	at com.liferay.portal.module.framework.ModuleFrameworkUtilAdapter.registerContext(ModuleFrameworkUtilAdapter.java:71)
	at com.liferay.portal.spring.context.PortalContextLoaderListener.contextInitialized(PortalContextLoaderListener.java:350)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4812)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5255)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:725)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:701)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717)
	at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:585)
	at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1794)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

and then I have this kind of messages all the time:

11:26:31,223 WARN  [elasticsearch[Paibo][generic][T#1]][shard:136] [Paibo] [liferay-20116][0] received shard failed for target shard [[liferay-20116][0], node[DmbonsY-RRmIYx8U1XPIKA], [P], v[117], s[INITIALIZING], a[id=KDP_ZuP4SCuoEuho3RQBCA], unassigned_info[[reason=ALLOCATION_FAILED], at[2017-11-21T10:22:58.335Z], details[engine failure, reason [lucene commit failed], failure AccessDeniedException[C:\liferay-dxp-digital-enterprise-7.0-sp2\data\elasticsearch\indices\LiferayElasticsearchCluster\nodes\0\indices\liferay-20116\0\index\_n9ds.cfs]]]], indexUUID [4-I3tGJ9Q0yNRkrgqEncBg], message [failed recovery], failure [IndexShardRecoveryException[failed to recovery from gateway]; nested: EngineCreationFailureException[failed to recover from translog]; nested: FlushFailedEngineException[Flush failed]; nested: AccessDeniedException[C:\liferay-dxp-digital-enterprise-7.0-sp2\data\elasticsearch\indices\LiferayElasticsearchCluster\nodes\0\indices\liferay-20116\0\index\_n9ds.cfs]; ]
[liferay-20116][[liferay-20116][0]] IndexShardRecoveryException[failed to recovery from gateway]; nested: EngineCreationFailureException[failed to recover from translog]; nested: FlushFailedEngineException[Flush failed]; nested: AccessDeniedException[C:\liferay-dxp-digital-enterprise-7.0-sp2\data\elasticsearch\indices\LiferayElasticsearchCluster\nodes\0\indices\liferay-20116\0\index\_n9ds.cfs];
	at org.elasticsearch.index.shard.StoreRecoveryService.recoverFromStore(StoreRecoveryService.java:250)
	at org.elasticsearch.index.shard.StoreRecoveryService.access$100(StoreRecoveryService.java:56)
	at org.elasticsearch.index.shard.StoreRecoveryService$1.run(StoreRecoveryService.java:129)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: [liferay-20116][[liferay-20116][0]] EngineCreationFailureException[failed to recover from translog]; nested: FlushFailedEngineException[Flush failed]; nested: AccessDeniedException[C:\liferay-dxp-digital-enterprise-7.0-sp2\data\elasticsearch\indices\LiferayElasticsearchCluster\nodes\0\indices\liferay-20116\0\index\_n9ds.cfs];
	at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:174)
	at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:25)
	at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:1509)
	at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:1493)
	at org.elasticsearch.index.shard.IndexShard.internalPerformTranslogRecovery(IndexShard.java:966)
	at org.elasticsearch.index.shard.IndexShard.performTranslogRecovery(IndexShard.java:938)
	at org.elasticsearch.index.shard.StoreRecoveryService.recoverFromStore(StoreRecoveryService.java:241)
	... 5 more
Caused by: [liferay-20116][[liferay-20116][0]] FlushFailedEngineException[Flush failed]; nested: AccessDeniedException[C:\liferay-dxp-digital-enterprise-7.0-sp2\data\elasticsearch\indices\LiferayElasticsearchCluster\nodes\0\indices\liferay-20116\0\index\_n9ds.cfs];
	at org.elasticsearch.index.engine.InternalEngine.flush(InternalEngine.java:762)
	at org.elasticsearch.index.engine.InternalEngine.recoverFromTranslog(InternalEngine.java:246)
	at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:171)
	... 11 more
Caused by: java.nio.file.AccessDeniedException: C:\liferay-dxp-digital-enterprise-7.0-sp2\data\elasticsearch\indices\LiferayElasticsearchCluster\nodes\0\indices\liferay-20116\0\index\_n9ds.cfs
	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
	at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115)
	at java.nio.channels.FileChannel.open(FileChannel.java:287)
	at java.nio.channels.FileChannel.open(FileChannel.java:335)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:391)
	at org.apache.lucene.store.FSDirectory.fsync(FSDirectory.java:288)
	at org.apache.lucene.store.FSDirectory.sync(FSDirectory.java:232)
	at org.apache.lucene.store.FilterDirectory.sync(FilterDirectory.java:78)
	at org.apache.lucene.store.FilterDirectory.sync(FilterDirectory.java:78)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.sync(LockValidatingDirectoryWrapper.java:62)
	at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4435)
	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2860)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2963)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2930)
	at org.elasticsearch.index.engine.InternalEngine.commitIndexWriter(InternalEngine.java:1256)
	at org.elasticsearch.index.engine.InternalEngine.commitIndexWriter(InternalEngine.java:1264)
	at org.elasticsearch.index.engine.InternalEngine.flush(InternalEngine.java:755)
	... 13 more</init></init>

Any idea? I can't solve this by myself.
It worked fine few days ago, but the portal was changed to another machine and it stops working fine.
Thanks in advance!!

[6年前 に David H Nebinger によって更新されました。]

Moved to another machine? Purge the data/elasticsearch directory and try again.

[6年前 に Daniel G によって更新されました。]

Thanks, it works!

I thought it may be a solution, but I was afraid that deleting that files could affect to other process.

Thanks again!

[4年前 に Herianto Saragi によって更新されました。]

How I to purge the data/elasticsearch directory?

[4年前 に David H Nebinger によって更新されました。]

Just delete it when the instance is stopped.

[4年前 に asif aftab によって更新されました。]

David H Nebinger:

Moved to another machine? Purge the data/elasticsearch directory and try again.

It works for me as I haven't change anything just started server next day then I faced this issue.Just deleted /data/elasticsearch6 folder in liferay-ce-portal directory.I m using liferay-ce-portal-7.2.1-ga2.