Kombinierte Ansicht Flache Ansicht Baumansicht
Threads [ Zurück | Nächste ]
toggle
Robert Revnic
Problems Synchronizing Nodes
29. März 2013 13:44
Antwort

Robert Revnic

Rang: New Member

Nachrichten: 11

Eintrittsdatum: 29. März 2013

Neue Beiträge

Hi All!

I'm having some problems with my Liferay cluster. The cluster consists of two Tomcat nodes which have Liferay 6.1.1 deployed to them. The problem is that the changes made in Liferay on node1 do not reflect on node2 for a significant amount of time (I believe until the ehcache gets invalidated). If I browse to the control panel and hit clear the content cached across the cluster, the changes appear correctly on both nodes. Is there a configuration change that I have overlooked? My portal-ext.properties file can be found below.

lucene.replicate.write=true


################################################################################################
################################################################################################
################################################################################################



##
## Hibernate
##

#
# Input a list of comma delimited Hibernate configurations.
#
hibernate.configs=\
META-INF/mail-hbm.xml,\
META-INF/portal-hbm.xml,\
META-INF/ext-hbm.xml

#
# Liferay will automatically detect the Hibernate dialect in
# com.liferay.portal.spring.PortalHibernateConfiguration. Set this property
# to manually override the automatically detected dialect.
#
#hibernate.dialect=

#
# Set the Hibernate connection release mode. You should not modify this
# unless you know what you're doing. The default setting works best for
# Spring managed transactions. See the method buildSessionFactory in class
# org.springframework.orm.hibernate3.LocalSessionFactoryBean and search for
# the phrase "on_close" to understand how this works.
#
#hibernate.connection.release_mode=on_close

#
# Set the Hibernate cache region factory. Ehcache is recommended in a
# clustered environment. See the property
# "net.sf.ehcache.configurationResourceName" for detailed configuration.
#
hibernate.cache.region.factory_class=com.liferay.portal.dao.orm.hibernate.region.SingletonLiferayEhcacheRegionFactory

#
# This property is used if Hibernate is configured to use Ehcache's region
# factory.
#
net.sf.ehcache.configurationResourceName=/myehcache/hibernate-clustered.xml

#
# Configure this property if using Ehcache's default replication method for

# Hibernate's region cache.
#
net.sf.ehcache.configurationResourceName.peerProviderProperties=peerDiscovery=automatic,multicastGroupAddress=${multicast.group.address["hibernate"]},multicastGroupPort=${multicast.group.port["hibernate"]},timeToLive=1

#
# Set other Hibernate cache settings.
#
hibernate.cache.use_query_cache=true
hibernate.cache.use_second_level_cache=true
hibernate.cache.use_minimal_puts=true
hibernate.cache.use_structured_entries=false


#
# Uncomment these properties to disable Hibernate caching.
#
#hibernate.cache.use_query_cache=false
#hibernate.cache.use_second_level_cache=false

#
# Set the JDBC batch size to improve performance.
#
# If you're using Hypersonic, you SHOULD set the batch size to 0 as a
# workaround for a logging bug in the Hypersonic driver. See LPS-5426 for
# more information.
#
# If you're using Oracle 9i, you MUST set the batch size to 0 as a
# workaround for a hanging bug in the Oracle driver. See LEP-1234 for more
# information.
#
hibernate.jdbc.batch_size=20
#hibernate.jdbc.batch_size=0

#
# Set other miscellaneous Hibernate properties.
#
hibernate.jdbc.use_scrollable_resultset=true
hibernate.bytecode.use_reflection_optimizer=true
hibernate.show_sql=false

#
# Use the classic query factory until WebLogic and Hibernate 3 can get
# along. See http://www.hibernate.org/250.html#A23 for more information.
#
hibernate.query.factory_class=org.hibernate.hql.classic.ClassicQueryTranslatorFactory

#
# Set this property to true to enable Hibernate cache monitoring. See
# LPS-2056 for more information.
#
hibernate.generate_statistics=false




################################################################################################
################################################################################################
################################################################################################

##
## Ehcache
##

#
# Set the classpath to the location of the Ehcache config file for internal
# caches. Edit the file specified in the property
# "ehcache.multi-vm.config.location" to enable clustered cache.
#
ehcache.single.vm.config.location=/ehcache/liferay-single-vm.xml
ehcache.multi.vm.config.location=/myehcache/liferay-multi-vm-clustered.xml

#
# Uncomment the following in a clustered environment.
#
ehcache.multi.vm.config.location.peerProviderProperties=peerDiscovery=automatic,multicastGroupAddress=${multicast.group.address["multi-vm"]},multicastGroupPort=${multicast.group.port["multi-vm"]},timeToLive=1

#
# Input a class name that extends
# net.sf.ehcache.bootstrap.BootstrapCacheLoaderFactory. This class is used
# by Ehcache to determine how caches communicate with each other in a
# clustered environment.
#
ehcache.bootstrap.cache.loader.factory=net.sf.ehcache.distribution.RMIBootstrapCacheLoaderFactory
#ehcache.bootstrap.cache.loader.factory=com.liferay.portal.cache.ehcache.JGroupsBootstrapCacheLoaderFactory

#
# Input a class name that extends
# net.sf.ehcache.event.CacheEventListenerFactory. This class is used by
# Ehcache to determine how caches communicate with each other in a clustered
# environment.
#
ehcache.cache.event.listener.factory=net.sf.ehcache.distribution.RMICacheReplicatorFactory
#ehcache.cache.event.listener.factory=net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory

#
# Input a class name that extends
# net.sf.ehcache.distribution.CacheManagerPeerProviderFactory. This class is
# used by Ehcache to determine how caches communicate with each other in a
# clustered environment.
#
ehcache.cache.manager.peer.provider.factory=net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory
#ehcache.cache.manager.peer.provider.factory=net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProviderFactory

#
# Set this to true to enable JMX integration in
# com.liferay.portal.cache.EhcachePortalCacheManager. Set this to false if
# the portal needs to be reloaded under WebSphere.
#
ehcache.portal.cache.manager.jmx.enabled=true


#
# Set this to true to allow Ehcache to use blocking caches. This improves
# performance significantly by locking on keys instead of the entire cache.
# The drawback is that threads can hang if the cache is not used properly.
# Make sure that all queries that return a miss also immediately populate
# the cache, or else other threads that are blocked on a query of that same
# key will continue to hang. Reference Ehcache's BlockingCache for more

# information. The blocking cache is no longer implemented by Ehcache's
# BlockingCache, but by Liferay's BlockingPortalCache for better safety and
# faster performance.
#
ehcache.blocking.cache.allowed=true

#
# Set this to true to enable Ehcache statistics.
#
ehcache.statistics.enabled=false



################################################################################################
################################################################################################
################################################################################################



cluster.link.enabled=true
ehcache.cluster.link.replication=true



Any tips/help on this is greatly appreciated.
Juan Gonzalez
RE: Problems Synchronizing Nodes
29. März 2013 14:57
Antwort

Juan Gonzalez

LIFERAY STAFF

Rang: Liferay Legend

Nachrichten: 2807

Eintrittsdatum: 28. Oktober 2008

Neue Beiträge

Hi Robert,

You only have to set this properties:


1cluster.link.enabled=true
2lucene.replicate.write=true



and then check if the default multicast IP are accesible in your network from one node to the others:


 1multicast.group.address["cluster-link-control"]=239.255.0.1
 2    multicast.group.port["cluster-link-control"]=23301
 3
 4    #
 5    # See the properties "cluster.link.channel.properties.transport.0" and
 6    # "cluster.link.channel.system.properties".
 7    #
 8    multicast.group.address["cluster-link-udp"]=239.255.0.2
 9    multicast.group.port["cluster-link-udp"]=23302
10
11    #
12    # See the property "cluster.link.channel.system.properties".
13    #
14    multicast.group.address["cluster-link-mping"]=239.255.0.3
15    multicast.group.port["cluster-link-mping"]=23303
16
17    #
18    # See the properties "net.sf.ehcache.configurationResourceName" and
19    # "net.sf.ehcache.configurationResourceName.peerProviderProperties".
20    #
21    multicast.group.address["hibernate"]=239.255.0.4
22    multicast.group.port["hibernate"]=23304
23
24    #
25    # See the properties "ehcache.multi.vm.config.location" and
26    # "ehcache.multi.vm.config.location.peerProviderProperties".
27    #
28    multicast.group.address["multi-vm"]=239.255.0.5
29    multicast.group.port["multi-vm"]=23305
Robert Revnic
RE: Problems Synchronizing Nodes
1. April 2013 07:58
Antwort

Robert Revnic

Rang: New Member

Nachrichten: 11

Eintrittsdatum: 29. März 2013

Neue Beiträge

Thanks for the support Juan.

When commenting out ehcache.cluster.link.replication.enabled=true, the following errors are thrown when the second node is started:

14:33:06,980 INFO [pool-3-thread-1][LiferayCacheManagerPeerProviderFactory:76] portalPropertyKey net.sf.ehcache.configurationResourceName.peerProviderProperties has value peerDiscovery=automatic,multicastGroupAddress=239.255.0.4,multicastGroupPort=23304,timeToLive=1
14:33:08,423 ERROR [pool-3-thread-1][ContextLoader:227] Context initialization failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.liferay.portal.spring.aop.ServiceBeanAutoProxyCreator#0' defined in class path resource [META-INF/base-spring.xml]: Cannot resolve reference to bean 'serviceAdvice' while setting bean property 'methodInterceptor'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'serviceAdvice' defined in class path resource [META-INF/base-spring.xml]: Cannot resolve reference to bean 'asyncAdvice' while setting bean property 'nextMethodInterceptor'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'asyncAdvice' defined in class path resource [META-INF/base-spring.xml]: Cannot resolve reference to bean 'threadLocalCacheAdvice' while setting bean property 'nextMethodInterceptor'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'threadLocalCacheAdvice' defined in class path resource [META-INF/base-spring.xml]: Cannot resolve reference to bean 'bufferedIncrementAdvice' while setting bean property 'nextMethodInterceptor'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'bufferedIncrementAdvice' defined in class path resource [META-INF/base-spring.xml]: Cannot resolve reference to bean 'indexableAdvice' while setting bean property 'nextMethodInterceptor'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'indexableAdvice' defined in class path resource [META-INF/base-spring.xml]: Cannot resolve reference to bean 'transactionAdvice' while setting bean property 'nextMethodInterceptor'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionAdvice' defined in class path resource [META-INF/base-spring.xml]: Cannot resolve reference to bean 'liferayTransactionManager' while setting bean property 'platformTransactionManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liferayTransactionManager' defined in class path resource [META-INF/hibernate-spring.xml]: Cannot resolve reference to bean 'liferayHibernateSessionFactory' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liferayHibernateSessionFactory' defined in class path resource [META-INF/hibernate-spring.xml]: Invocation of init method failed; nested exception is java.lang.OutOfMemoryError: unable to create new native thread

followed by several out of memory exceptions.

The nodes are running on the same physical box. Any insight on how I could proceed? Does this still seem like an issue with the multicast?


Thanks again,

Rob
Juan Gonzalez
RE: Problems Synchronizing Nodes
1. April 2013 08:01
Antwort

Juan Gonzalez

LIFERAY STAFF

Rang: Liferay Legend

Nachrichten: 2807

Eintrittsdatum: 28. Oktober 2008

Neue Beiträge

Can you try shutting down all nodes and then ensure there aren't any Liferay process running? (for example: ps -ef |grep java and kill all process that should be stopped) ?

After that, start one node, see logs, and then start second node and see its logs....
Robert Revnic
RE: Problems Synchronizing Nodes
1. April 2013 08:34
Antwort

Robert Revnic

Rang: New Member

Nachrichten: 11

Eintrittsdatum: 29. März 2013

Neue Beiträge

I should have mentioned that I also have multiple stand alone nodes running on the box (dev, qastg) in addition to the clustered nodes (qa prd). Shutting everything down and restarting one by one fixed the OOM issue. The clustered nodes are in sync now emoticon emoticon . Removing the ehcache.cluster.link.replication.enabled=true property worked. Thanks for the help.
Juan Gonzalez
RE: Problems Synchronizing Nodes
1. April 2013 08:43
Antwort

Juan Gonzalez

LIFERAY STAFF

Rang: Liferay Legend

Nachrichten: 2807

Eintrittsdatum: 28. Oktober 2008

Neue Beiträge

Glad to hear that.

The property

ehcache.cluster.link.replication.enabled=true

shouldn't be used unless you have EE license, and hence you can install ehcache cluster web plugin.