Combination View Flat View Tree View
Threads [ Previous | Next ]
toggle
Manuel Pinto
Cluster Environment Session Locks
January 24, 2013 3:08 PM
Answer

Manuel Pinto

Rank: New Member

Posts: 4

Join Date: August 20, 2012

Recent Posts

Hi,

I need some help to resolve a problem with session locks in a cluster environment.
We have two Liferay 6.1 over JBoss 7.1.1 in standalone-ha configuration (build 7.1.1 Final) with infinispan "web" cache-container, invalidation-cache and jdbc store. The load balancer is configured in non sticky session mode.

Problem: when a node processes requests that manipulate session's data, in some cases the node locks the session and never unlock it, preventing other node from processing requests for that session. The affected node never regain the locked session and keep throwing the following exception for all subsequent requests and only recover a session when other node shutdown:

Note: we also tried replication-cache and distributed-cache and all locking modes but without success.

Exception:

16:29:52,278 ERROR [org.apache.catalina.connector.CoyoteAdapter] (http--127.0.0.1-8080-2) An exception or error occurred in the container during the request processing: java.lang.RuntimeException: JBAS018060: Exception acquiring ownership of ruxDFIayi1vrsCejzwnVQJl3
at org.jboss.as.web.session.ClusteredSession.access(ClusteredSession.java:534) [jboss-as-web-7.1.1.Final-custom.jar:7.1.1.Final]
at org.apache.catalina.connector.Request.doGetSession(Request.java:2625) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.connector.Request.getSession(Request.java:2375) [jbossweb-7.0.13.Final.jar:]
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:81) [jboss-as-web-7.1.1.Final-custom.jar:7.1.1.Final]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.13.Final.jar:]
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) [jbossweb-7.0.13.Final.jar:]
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) [jbossweb-7.0.13.Final.jar:]
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.13.Final.jar:]
at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_31]
Caused by: org.jboss.as.clustering.lock.TimeoutException: JBAS010223: Cannot acquire lock //default-host//ruxDFIayi1vrsCejzwnVQJl3 from cluster
at org.jboss.as.clustering.lock.SharedLocalYieldingClusterLockManager.lock(SharedLocalYieldingClusterLockManager.java:439)
at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.acquireSessionOwnership(DistributedCacheManager.java:372)
at org.jboss.as.web.session.ClusteredSession.acquireSessionOwnership(ClusteredSession.java:560) [jboss-as-web-7.1.1.Final-custom.jar:7.1.1.Final]
at org.jboss.as.web.session.ClusteredSession.access(ClusteredSession.java:532) [jboss-as-web-7.1.1.Final-custom.jar:7.1.1.Final]
... 11 more


Infinispan modules:

Infinispan Core 5.1.6 Final and Infinispan Cachestore JDBC 5.1.6.FINAL


The standalone-ha.xml "web" cache-container config:

<cache-container name="web" default-cache="database">
<transport lock-timeout="60000"/>
<invalidation-cache name="database" mode="SYNC" batching="true">
<locking isolation="REPEATABLE_READ" />
<mixed-keyed-jdbc-store datasource="java:jboss/datasources/LPORTAL_DS" shared="true" preload="true" passivation="false" purge="true">
<property name="databaseType">
oracle
</property>
<string-keyed-table prefix="stringbased">
<id-column name="id" type="VARCHAR2(200)"/>
<data-column name="datum" type="BLOB"/>
<timestamp-column name="version" type="NUMBER"/>
</string-keyed-table>
<binary-keyed-table prefix="binarybased">
<id-column name="id" type="VARCHAR2(200)"/>
<data-column name="datum" type="BLOB"/>
<timestamp-column name="version" type="NUMBER"/>
</binary-keyed-table>
</mixed-keyed-jdbc-store>
</invalidation-cache>
</cache-container>

Thanks,
Manuel
Manuel Pinto
RE: Cluster Environment Session Locks
January 24, 2013 3:31 PM
Answer

Manuel Pinto

Rank: New Member

Posts: 4

Join Date: August 20, 2012

Recent Posts

The correct log is:

17:39:00,174 ERROR [org.apache.catalina.connector.CoyoteAdapter] (http--172.16.250.105-8080-4) An exception or error occurred in the container during the request processing: java.lang.RuntimeException: JBAS018060: Exception acquiring ownership of Cvn-K+r-cBGesIBoDrakJhrO
at org.jboss.as.web.session.ClusteredSession.acquireSessionOwnership(ClusteredSession.java:528) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.web.session.ClusteredSession.access(ClusteredSession.java:496) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
at org.apache.catalina.connector.Request.doGetSession(Request.java:2625) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.connector.Request.getSession(Request.java:2375) [jbossweb-7.0.13.Final.jar:]
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:81) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.13.Final.jar:]
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) [jbossweb-7.0.13.Final.jar:]
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) [jbossweb-7.0.13.Final.jar:]
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.13.Final.jar:]
at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_32]
Caused by: org.jboss.as.clustering.lock.TimeoutException: JBAS010223: Cannot acquire lock //default-host//Cvn-K+r-cBGesIBoDrakJhrO from cluster
at org.jboss.as.clustering.lock.SharedLocalYieldingClusterLockManager.lock(SharedLocalYieldingClusterLockManager.java:439)
at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.acquireSessionOwnership(DistributedCacheManager.java:372)
at org.jboss.as.web.session.ClusteredSession.acquireSessionOwnership(ClusteredSession.java:520) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
... 12 more