Foren

Kaleo Workflow fork only creates two out of three nodes

thumbnail
Willem Vermeer, geändert vor 11 Jahren.

Kaleo Workflow fork only creates two out of three nodes

Junior Member Beiträge: 32 Beitrittsdatum: 30.03.12 Neueste Beiträge
Hello,

We are experiencing a problem with the kaleo workflow engine when entering a fork task. The fork has three nodes and after completion of all three of them it proceeds to the join tasks. We are processing ~300 assets with this workflow in a sequential process. The fork happens right at the start of the workflow. So what happens is we get an asset, we start the workflow and we proceed on to the next asset.
Usually it works fine but sometimes only two of the nodes are created. It seems to be random which of the three doesn't make it. We find the following stacktrace in the log files.

Does anybody have any idea why this might be happening? We are on Liferay Portal Enterprise Edition 6.1 EE GA1 (Paton / Build 6110 / February 15, 2012)

It's quite a problematic situation for us as it's quite doubtful whether we will be able to move the asset onwards in the workflow if we can only finish two out of its three nodes.

Thanks,
Willem
14:40:51,149 ERROR [ParallelDestination:111] Unable to process message {destinationName=liferay/kaleo_graph_walker, response=null, responseDestinationName=null, responseId=null, payload=com.liferay.portal.workflow.kaleo.runtime.graph.PathElement@7526d02f, values={principalPassword=null, principalName=979459, companyId=10154}}
com.liferay.portal.kernel.messaging.MessageListenerException: java.util.ConcurrentModificationException
at com.liferay.portal.kernel.messaging.BaseMessageListener.receive(BaseMessageListener.java:31)
at com.liferay.portal.kernel.messaging.InvokerMessageListener.receive(InvokerMessageListener.java:63)
at com.liferay.portal.kernel.messaging.ParallelDestination$1.run(ParallelDestination.java:108)
at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask._runTask(ThreadPoolExecutor.java:669)
at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask.run(ThreadPoolExecutor.java:580)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$EntryIterator.next(HashMap.java:834)
at java.util.HashMap$EntryIterator.next(HashMap.java:832)
at org.jabsorb.serializer.impl.MapSerializer.marshall(MapSerializer.java:118)
at org.jabsorb.JSONSerializer.marshall(JSONSerializer.java:390)
at org.jabsorb.JSONSerializer.toJSON(JSONSerializer.java:528)
at com.liferay.portal.json.JSONFactoryImpl.serialize(JSONFactoryImpl.java:221)
at com.liferay.portal.kernel.json.JSONFactoryUtil.serialize(JSONFactoryUtil.java:123)
at com.liferay.portal.workflow.kaleo.util.WorkflowContextUtil.convert(WorkflowContextUtil.java:36)
at com.liferay.portal.workflow.kaleo.service.impl.KaleoLogLocalServiceImpl.addTaskAssignmentKaleoLog(KaleoLogLocalServiceImpl.java:185)
at sun.reflect.GeneratedMethodAccessor3132.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:112)
at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:71)
at com.liferay.portal.dao.jdbc.aop.DynamicDataSourceTransactionInterceptor.invoke(DynamicDataSourceTransactionInterceptor.java:44)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:108)
at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:71)
at com.liferay.portal.dao.jdbc.aop.DynamicDataSourceTransactionInterceptor.invoke(DynamicDataSourceTransactionInterceptor.java:44)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:108)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:59)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:108)
at com.liferay.portal.spring.aop.ServiceBeanAopProxy.invoke(ServiceBeanAopProxy.java:211)
at $Proxy740.addTaskAssignmentKaleoLog(Unknown Source)
at com.liferay.portal.workflow.kaleo.runtime.node.TaskNodeExecutor.doEnter(TaskNodeExecutor.java:165)
at com.liferay.portal.workflow.kaleo.runtime.node.BaseNodeExecutor.enter(BaseNodeExecutor.java:49)
at com.liferay.portal.workflow.kaleo.runtime.graph.DefaultGraphWalker.follow(DefaultGraphWalker.java:62)
at sun.reflect.GeneratedMethodAccessor3038.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:320)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:71)
at com.liferay.portal.dao.jdbc.aop.DynamicDataSourceTransactionInterceptor.invoke(DynamicDataSourceTransactionInterceptor.java:44)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy767.follow(Unknown Source)
at com.liferay.portal.workflow.kaleo.runtime.graph.messaging.PathElementMessageListener.doReceive(PathElementMessageListener.java:54)
at com.liferay.portal.kernel.messaging.BaseMessageListener.receive(BaseMessageListener.java:25)
... 5 more
thumbnail
Willem Vermeer, geändert vor 11 Jahren.

RE: Kaleo Workflow fork only creates two out of three nodes

Junior Member Beiträge: 32 Beitrittsdatum: 30.03.12 Neueste Beiträge
Some more information about the problem:
It appears to be a problem of the workflow engine not being thread safe. I've been able to reliably reproduce the problem with the following scenario:

1. create a new asset
2. start a new workflow for this asset by invoking WorkflowInstanceManagerUtil.startWorkflowInstance(..)
3. in the workflow definition the workflow automatically proceeds into a fork task with three nodes

When I repeat these three steps in a loop of 10 assets, it works fine. 10 assets are created as well as three fork nodes for each one of them.

When I repeat the three steps in a loop of 50 assets, two of them will have only 2 fork nodes instead of three and I see two occurrences of the stacktrace in the logs.

When I add a Thread.sleep(1000) after step 3 everything works OK for any number of assets.

So it seems that somewhere in the creation of the forked nodes a process is not thread-safe and collisions are happening.

Should I raise a bug for this problem and if yes, how?

Thanks,
Willem
All testing was performed on a MacBookPro with 2.2Ghz Intel Core i7 and 16GB memory on Liferay Portal Enterprise Edition 6.1 EE GA1 (Paton / Build 6110 / February 15, 2012)