Component: Bonita | Version: 7.9
What’s wrong?
The messages in the log files are appended at the end. We can see some of these keywords:
- ARJUNA016102: The transaction is not active!
- HibernateException: Logical connection is closed
- HibernateException: Current transaction is not in progress
- ARJUNA012095: Abort of action id 0:…e invoked while multiple threads active within it
Answer
These messages in the log files show that there are XA transactions which time out.
Explanation
A XA transaction can time out when:
- Transactions are waiting for locks, when there are deadlocks…
- There are shortage of connection in the pools associated to datasources.
- There are broken TCP/IP connections
- There are long transactions:
- The tables are becoming huge.
- A client application calls the Engine API or the REST API which triggers searches which need a long time to execute.
- The users are using the portal so that it triggers searches (via the REST API) which need a long time to execute.
- There are performance issues which slow down Java process:
- either an environmental problem impacts the CPU or Disk I/O
- or an extra Garbage Collector’s threads activity, triggered by a java memory shortage (-Xmx), leaves nothing to java application’s threads.
- The current XA transaction timeout limit is too small
1. Database server
Bonita requires specific database server settings.
If not implemented, there could be deadlocks, especially when MS Sql server is used.
Check that the settings listed in the Database creation and settings to work with Bonita documentation page have been implemented.
2. Simple configuration check with Tomcat
Narayana transaction manager ™ had been integrated into Tomcat in order to enable the use of the XA transactions.
Make sure that this option below is passed to the java command used to run the Tomcat, otherwise all the defaults will apply:
-Dcom.arjuna.ats.arjuna.common.propertiesFile=/a/path/to/jbossts-properties.xml
Make sure that the jbossts-properties.xml file exists.
3. Sizing
Analyze the configuration and better configure the connections’ pools and threads’ pools ; see the Sizing of the engine with examples.
Make sure that there will never be shortage of connections in the pools.
4. How to cope with broken TCP/IP connections
There is a page dedicated to this particular situation: Error: Bonita server not responding after an inactivity period
When the TCP/IP connections are broken: as there are not closed, the Tomcat’s pools may provide to the http and bonita threads such broken connections. The threads send SQL queries and wait endlessly to an answer, until the XA transaction times out.
The pools of connections and the jdbc drivers can be configured to be more robust.
When BonitaSubscription-2021.1-0811/start-bonita.sh (.bat) script is used to start the Tomcat, edit BonitaSubscription-2021.1-0811/setup/tomcat-templates/bonita.xml file.
These first settings, will make the idle (unused) connections being droped from the pool.
Make sure these properties below are used with bonitaDS, bonitaSequenceManagerDS, BusinessDataDS, and NotManagedBizDataDS resources
testWhileIdle="false"
softMinEvictableIdleTimeMillis="-1"
timeBetweenEvictionRunsMillis="60000"
minEvictableIdleTimeMillis="900000"
These other settings will make the jdbc driver create the TCP/IP sockets with keep alive activated.
Make sure this property below is used with both RawBonitaDS and RawBusinessDataDS resources.
tcpKeepAlive="true"
On linux system, use the command-line below to display the settings:
$ sudo sysctl net.ipv4.tcp_keepalive_intvl net.ipv4.tcp_keepalive_probes net.ipv4.tcp_keepalive_time
It is wise to reduce the default value of the net.ipv4.tcp_keepalive_time variable to 600.
“Keep alive mechanism” will make the system find the broken connections, so that the pools will drop them in turn.
5. Tables’ size
Analyze the tables’ sizes used with the engine and plan to delete the oldest data, using the API: better do that with 7.7.5 version or greater.
In particular, you may want to delete archived process instances with the API call below:
6. Performance issue investigation
Monitor both the CPU and the disks I/O utilization, plus the CPU consumption of the java process running the Tomcat.
If there are performance problems when there are the XA transactions time out, fix them.
If the Java itself is consuming all the CPU:
- if there is some RAM left (
availablememory), raise up the-Xmxvalue in order to provide more memory to the java process - when
Business Data Model(BDM) is used, with multiple objects linked withEAGERtype (orAlways load related objects) relationship, then change the relationship toLAZY(Only load related objects when needed). The forms and the processes using theBDMneed to be modified as well.
7. Raise up the XA default transaction timeout
At production sites, values from 300 to 2400 seconds are used.
Narayana is the transaction manager ™ used with Bonita.
It enables the use of XA transactions.
Tomcat
Edit the BonitaSubscription-7.9-Tomcat-8.5.34/server/conf/jbossts-properties.xml.
Raise up the value, in seconds, of the xml element listed below:
<entry key="com.arjuna.ats.arjuna.coordinator.defaultTimeout">600</entry>
Restart the Tomcat to activate the changes.
Wildfly
Edit the standalone.xml file
Add the coordinator-environment element in the subsystem section, as shown below:
<subsystem xmlns="urn:jboss:domain:transactions:3.0">
<core-environment node-identifier="bonita">
<process-id>
<uuid />
</process-id>
</core-environment>
<recovery-environment socket-binding="txn-recovery-environment" status-socket-binding="txn-status-manager" />
<coordinator-environment default-timeout="600" />
</subsystem>
Restart the Wildfly to activate the changes.
standalone.xml file location
start-bonita script
When the BonitaSubscription-7.9.0-wildfly-10.1.0.Final/start-bonita.sh (start-bonita.bat) script is used to start the application server, then edit the BonitaSubscription-7.9.0-wildfly-10.1.0.Final/setup/wildfly-templates/standalone.xml file.
Other situations
When Wildfly is started with the standard BonitaSubscription-7.9.0-wildfly-10.1.0.Final/server/bin/standalone.sh (standalone.bat) or as a Windows service, edit the BonitaSubscription-7.9.0-wildfly-10.1.0.Final/server/standalone/configuration/standalone.xml file.
Raw messages from the log files
Tomcat
There are below other examples from the server/logs/bonita*log file:
2018-08-17 16:05:37.780 +0200 SEVERE: org.bonitasoft.web.toolkit.server.servlet.ToolkitHttpServlet org.bonitasoft.web.toolkit.server.servlet.ToolkitHttpServlet catchAllExceptions SEVERE: org.bonitasoft.engine.exception.SearchException: USERNAME=kiki | org.bonitasoft.engine.persistence.SBonitaReadException: org.bonitasoft.engine.services.SPersistenceException: org.hibernate.HibernateException: Current transaction is not in progress org.bonitasoft.web.toolkit.client.common.exception.api.APIException: org.bonitasoft.engine.exception.SearchException: USERNAME=kiki | org.bonitasoft.engine.persistence.SBonitaReadException: org.bonitasoft.engine.services.SPersistenceException: org.hibernate.HibernateException: Current transaction is not in progress
... Caused by: org.bonitasoft.engine.services.SPersistenceException: org.hibernate.HibernateException: Current transaction is not in progress
... Caused by: org.hibernate.HibernateException: Current transaction is not in progress ...
2018-09-10 07:40:22,470 SEVERE [org.restlet.Component.BonitaSPRestletApplication] (default task-29) Exception or error caught in server resource: org.bonitasoft.engine.exception.BonitaRuntimeException: USERNAME=kiki | org.bonitasoft.engine.persistence.SRetryableException: javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed at org.bonitasoft.engine.api.impl.ServerAPIImpl.wrapThrowable(ServerAPIImpl.java:148)
... Caused by: javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed ... at org.bonitasoft.engine.business.data.impl.JPABusinessDataRepositoryImpl.findList(JPABusinessDataRepositoryImpl.java:318) at org.bonitasoft.engine.business.data.impl.JPABusinessDataRepositoryImpl.findListByNamedQuery(JPABusinessDataRepositoryImpl.java:295) ... 149 more Caused by: org.hibernate.HibernateException: Logical connection is closed at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:164) ...
2018-09-10 10:01:20,153 SEVERE [org.restlet.Component.BonitaSPRestletApplication] (default task-4) Exception or error caught in server resource: org.bonitasoft.engine.exception.BonitaRuntimeException: USERNAME=kiki | org.bonitasoft.engine.persistence.SRetryableException: javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed
... Caused by: org.bonitasoft.engine.persistence.SRetryableException: javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed at org.bonitasoft.engine.business.data.impl.JPABusinessDataRepositoryImpl.findById(JPABusinessDataRepositoryImpl.java:193)
... Caused by: javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed
... Caused by: org.hibernate.HibernateException: Logical connection is closed at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:164)
...
2020-09-28 00:08:00.537 -0400 FINE: org.bonitasoft.engine.transaction.JTATransactionServiceImpl THREAD_ID=106 | HOSTNAME=quantum-srv1 | Setting rollbackOnly on current transaction because callable 'org.bonitasoft.engine.execution.work.ExecuteConnectorWork$EvaluateParameterAndGetConnectorInstance@7570c2d1' has thrown an exception: javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed
org.bonitasoft.engine.commons.exceptions.SRetryableException: javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed
at org.bonitasoft.engine.business.data.impl.JPABusinessDataRepositoryImpl.findById(JPABusinessDataRepositoryImpl.java:199)
at org.bonitasoft.engine.business.data.BusinessDataRetriever.getSimpleBusinessData(BusinessDataRetriever.java:53)
at org.bonitasoft.engine.business.data.BusinessDataRetriever.getBusinessData(BusinessDataRetriever.java:95)
at org.bonitasoft.engine.expression.BusinessDataExpressionExecutorStrategy.evaluate(BusinessDataExpressionExecutorStrategy.java:68)
at org.bonitasoft.engine.expression.BusinessDataExpressionExecutorStrategy.evaluate(BusinessDataExpressionExecutorStrategy.java:84)
at org.bonitasoft.engine.expression.impl.ExpressionServiceImpl.evaluate(ExpressionServiceImpl.java:154)
at org.bonitasoft.engine.core.expression.control.api.impl.ExpressionResolverServiceImpl.evaluateExpressionsOfKind(ExpressionResolverServiceImpl.java:225)
at org.bonitasoft.engine.core.expression.control.api.impl.ExpressionResolverServiceImpl.evaluateAllExpressionsWithNoDependencies(ExpressionResolverServiceImpl.java:182)
at org.bonitasoft.engine.core.expression.control.api.impl.ExpressionResolverServiceImpl.evaluateExpressionsFlatten(ExpressionResolverServiceImpl.java:115)
at org.bonitasoft.engine.core.expression.control.api.impl.ExpressionResolverServiceImpl.evaluate(ExpressionResolverServiceImpl.java:83)
at org.bonitasoft.engine.core.connector.impl.ConnectorServiceImpl.evaluateInputParameters(ConnectorServiceImpl.java:314)
at org.bonitasoft.engine.connector.ConnectorServiceDecorator.evaluateInputParameters(ConnectorServiceDecorator.java:103)
at org.bonitasoft.engine.execution.work.ExecuteConnectorWork$EvaluateParameterAndGetConnectorInstance.call(ExecuteConnectorWork.java:257)
at org.bonitasoft.engine.execution.work.ExecuteConnectorWork$EvaluateParameterAndGetConnectorInstance.call(ExecuteConnectorWork.java:215)
at org.bonitasoft.engine.transaction.JTATransactionServiceImpl.executeInTransaction(JTATransactionServiceImpl.java:274)
at org.bonitasoft.engine.execution.work.ExecuteConnectorWork.work(ExecuteConnectorWork.java:148)
at org.bonitasoft.engine.execution.work.failurewrapping.TxInHandleFailureWrappingWork.work(TxInHandleFailureWrappingWork.java:41)
at org.bonitasoft.engine.execution.work.failurewrapping.TxInHandleFailureWrappingWork.work(TxInHandleFailureWrappingWork.java:41)
at org.bonitasoft.engine.execution.work.failurewrapping.TxInHandleFailureWrappingWork.work(TxInHandleFailureWrappingWork.java:41)
at org.bonitasoft.engine.execution.work.failurewrapping.TxInHandleFailureWrappingWork.work(TxInHandleFailureWrappingWork.java:41)
at org.bonitasoft.engine.execution.work.InSessionBonitaWork.work(InSessionBonitaWork.java:59)
at org.bonitasoft.engine.work.BonitaThreadPoolExecutor.lambda$submit$1(BonitaThreadPoolExecutor.java:132)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1763)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1694)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.find(AbstractEntityManagerImpl.java:1141)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.find(AbstractEntityManagerImpl.java:1068)
at org.bonitasoft.engine.business.data.impl.JPABusinessDataRepositoryImpl.findById(JPABusinessDataRepositoryImpl.java:196)
... 24 more
Caused by: org.hibernate.HibernateException: Logical connection is closed
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:164)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.connection(StatementPreparerImpl.java:63)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:162)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:186)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:160)
at org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.prepareQueryStatement(AbstractLoadPlanBasedLoader.java:257)
at org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.executeQueryStatement(AbstractLoadPlanBasedLoader.java:201)
at org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.executeLoad(AbstractLoadPlanBasedLoader.java:137)
at org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.executeLoad(AbstractLoadPlanBasedLoader.java:102)
at org.hibernate.loader.entity.plan.AbstractLoadPlanBasedEntityLoader.load(AbstractLoadPlanBasedEntityLoader.java:186)
at org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:4126)
at org.hibernate.event.internal.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:503)
at org.hibernate.event.internal.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:468)
at org.hibernate.event.internal.DefaultLoadEventListener.load(DefaultLoadEventListener.java:213)
at org.hibernate.event.internal.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:275)
at org.hibernate.event.internal.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:151)
at org.hibernate.internal.SessionImpl.fireLoad(SessionImpl.java:1106)
at org.hibernate.internal.SessionImpl.access$2000(SessionImpl.java:176)
at org.hibernate.internal.SessionImpl$IdentifierLoadAccessImpl.load(SessionImpl.java:2587)
at org.hibernate.internal.SessionImpl.get(SessionImpl.java:991)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.find(AbstractEntityManagerImpl.java:1110)
... 26 more
2020-09-28 00:08:00.537 -0400 FINEST: org.bonitasoft.engine.transaction.JTATransactionServiceImpl THREAD_ID=106 | HOSTNAME=quantum-srv1 | Completing transaction in thread 106 TransactionImple < ac, BasicAction: 0:ffff0afe3223:e16e:5f6b86b0:1a3e76 status: ActionStatus.ABORT_ONLY >
2020-09-28 00:08:00.537 -0400 FINEST: org.bonitasoft.engine.transaction.JTATransactionServiceImpl THREAD_ID=106 | HOSTNAME=quantum-srv1 | Rolling back transaction in thread 106 TransactionImple < ac, BasicAction: 0:ffff0afe3223:e16e:5f6b86b0:1a3e76 status: ActionStatus.ABORT_ONLY >
2020-09-28 00:08:00.538 -0400 FINE: com.bonitasoft.engine.work.DefaultExceptionRetryabilityEvaluator THREAD_ID=106 | HOSTNAME=quantum-srv1 | TENANT_ID=108 | Will retry, matching exception found: Exceptions:
↳org.bonitasoft.engine.commons.exceptions.SRetryableException: javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed ☚
↳javax.persistence.PersistenceException
↳org.hibernate.HibernateException
2020-09-28 00:08:00.538 -0400 WARNING: com.bonitasoft.engine.work.RetryingWorkExecutorService THREAD_ID=106 | HOSTNAME=quantum-srv1 | TENANT_ID=108 | Work WorkDescriptor[EXECUTE_ACTIVITY_CONNECTOR,parameters={processDefinitionId=4690108313069717636, processInstanceId=35238, connectorInstanceId=1987488, flowNodeDefinitionId=6178904343737415026, connectorDefinitionName=GetProductName, flowNodeInstanceId=3193478}] encountered an exception but it can be retried. Will retry. Attempt 1 on 10 with a delay of 1000 ms
2020-09-28 00:08:00.538 -0400 FINE: com.bonitasoft.engine.work.RetryingWorkExecutorService THREAD_ID=106 | HOSTNAME=quantum-srv1 | TENANT_ID=108 | Exception encountered during retry
org.bonitasoft.engine.commons.exceptions.SRetryableException: javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed
at org.bonitasoft.engine.business.data.impl.JPABusinessDataRepositoryImpl.findById(JPABusinessDataRepositoryImpl.java:199)
at org.bonitasoft.engine.business.data.BusinessDataRetriever.getSimpleBusinessData(BusinessDataRetriever.java:53)
at org.bonitasoft.engine.business.data.BusinessDataRetriever.getBusinessData(BusinessDataRetriever.java:95)
at org.bonitasoft.engine.expression.BusinessDataExpressionExecutorStrategy.evaluate(BusinessDataExpressionExecutorStrategy.java:68)
at org.bonitasoft.engine.expression.BusinessDataExpressionExecutorStrategy.evaluate(BusinessDataExpressionExecutorStrategy.java:84)
at org.bonitasoft.engine.expression.impl.ExpressionServiceImpl.evaluate(ExpressionServiceImpl.java:154)
at org.bonitasoft.engine.core.expression.control.api.impl.ExpressionResolverServiceImpl.evaluateExpressionsOfKind(ExpressionResolverServiceImpl.java:225)
at org.bonitasoft.engine.core.expression.control.api.impl.ExpressionResolverServiceImpl.evaluateAllExpressionsWithNoDependencies(ExpressionResolverServiceImpl.java:182)
at org.bonitasoft.engine.core.expression.control.api.impl.ExpressionResolverServiceImpl.evaluateExpressionsFlatten(ExpressionResolverServiceImpl.java:115)
at org.bonitasoft.engine.core.expression.control.api.impl.ExpressionResolverServiceImpl.evaluate(ExpressionResolverServiceImpl.java:83)
at org.bonitasoft.engine.core.connector.impl.ConnectorServiceImpl.evaluateInputParameters(ConnectorServiceImpl.java:314)
at org.bonitasoft.engine.connector.ConnectorServiceDecorator.evaluateInputParameters(ConnectorServiceDecorator.java:103)
at org.bonitasoft.engine.execution.work.ExecuteConnectorWork$EvaluateParameterAndGetConnectorInstance.call(ExecuteConnectorWork.java:257)
at org.bonitasoft.engine.execution.work.ExecuteConnectorWork$EvaluateParameterAndGetConnectorInstance.call(ExecuteConnectorWork.java:215)
at org.bonitasoft.engine.transaction.JTATransactionServiceImpl.executeInTransaction(JTATransactionServiceImpl.java:274)
at org.bonitasoft.engine.execution.work.ExecuteConnectorWork.work(ExecuteConnectorWork.java:148)
at org.bonitasoft.engine.execution.work.failurewrapping.TxInHandleFailureWrappingWork.work(TxInHandleFailureWrappingWork.java:41)
at org.bonitasoft.engine.execution.work.failurewrapping.TxInHandleFailureWrappingWork.work(TxInHandleFailureWrappingWork.java:41)
at org.bonitasoft.engine.execution.work.failurewrapping.TxInHandleFailureWrappingWork.work(TxInHandleFailureWrappingWork.java:41)
at org.bonitasoft.engine.execution.work.failurewrapping.TxInHandleFailureWrappingWork.work(TxInHandleFailureWrappingWork.java:41)
at org.bonitasoft.engine.execution.work.InSessionBonitaWork.work(InSessionBonitaWork.java:59)
at org.bonitasoft.engine.work.BonitaThreadPoolExecutor.lambda$submit$1(BonitaThreadPoolExecutor.java:132)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1763)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1694)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.find(AbstractEntityManagerImpl.java:1141)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.find(AbstractEntityManagerImpl.java:1068)
at org.bonitasoft.engine.business.data.impl.JPABusinessDataRepositoryImpl.findById(JPABusinessDataRepositoryImpl.java:196)
... 24 more
Caused by: org.hibernate.HibernateException: Logical connection is closed
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:164)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.connection(StatementPreparerImpl.java:63)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:162)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:186)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:160)
at org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.prepareQueryStatement(AbstractLoadPlanBasedLoader.java:257)
at org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.executeQueryStatement(AbstractLoadPlanBasedLoader.java:201)
at org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.executeLoad(AbstractLoadPlanBasedLoader.java:137)
at org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.executeLoad(AbstractLoadPlanBasedLoader.java:102)
at org.hibernate.loader.entity.plan.AbstractLoadPlanBasedEntityLoader.load(AbstractLoadPlanBasedEntityLoader.java:186)
at org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:4126)
at org.hibernate.event.internal.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:503)
at org.hibernate.event.internal.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:468)
at org.hibernate.event.internal.DefaultLoadEventListener.load(DefaultLoadEventListener.java:213)
at org.hibernate.event.internal.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:275)
at org.hibernate.event.internal.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:151)
at org.hibernate.internal.SessionImpl.fireLoad(SessionImpl.java:1106)
at org.hibernate.internal.SessionImpl.access$2000(SessionImpl.java:176)
at org.hibernate.internal.SessionImpl$IdentifierLoadAccessImpl.load(SessionImpl.java:2587)
at org.hibernate.internal.SessionImpl.get(SessionImpl.java:991)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.find(AbstractEntityManagerImpl.java:1110)
... 26 more
2020-10-01 16:43:58.026 -0400 WARN: com.arjuna.ats.arjuna ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0afe3223:d3cd:5f763ec9:1ffa in state RUN
2020-10-01 16:43:58.026 -0400 WARN: com.arjuna.ats.arjuna ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0afe3223:d3cd:5f763ec9:1ffe in state RUN
2020-10-01 16:43:58.026 -0400 WARN: com.arjuna.ats.arjuna ARJUNA012095: Abort of action id 0:ffff0afe3223:d3cd:5f763ec9:1ffa invoked while multiple threads active within it.
2020-10-01 16:43:58.026 -0400 WARN: com.arjuna.ats.arjuna ARJUNA012381: Action id 0:ffff0afe3223:d3cd:5f763ec9:1ffa completed with multiple threads - thread Bonita-Worker-108-04 was in progress with sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.FutureTask.awaitDone(FutureTask.java:426)
java.util.concurrent.FutureTask.get(FutureTask.java:204)
org.bonitasoft.engine.classloader.ClassLoaderUpdater.execute(ClassLoaderUpdater.java:64)
org.bonitasoft.engine.classloader.ClassLoaderUpdater.initializeClassLoader(ClassLoaderUpdater.java:47)
org.bonitasoft.engine.classloader.ClassLoaderServiceImpl.getLocalClassLoader(ClassLoaderServiceImpl.java:150)
org.bonitasoft.engine.classloader.ClassLoaderServiceImpl.getLocalClassLoader(ClassLoaderServiceImpl.java:140)
org.bonitasoft.engine.classloader.ClassLoaderServiceImpl.getLocalClassLoader(ClassLoaderServiceImpl.java:55)
org.bonitasoft.engine.execution.FlowNodeExecutorImpl.stepForward(FlowNodeExecutorImpl.java:137)
org.bonitasoft.engine.execution.FlowNodeExecutorImpl.executeFlowNode(FlowNodeExecutorImpl.java:301)
org.bonitasoft.engine.execution.work.ExecuteFlowNodeWork.work(ExecuteFlowNodeWork.java:73)
org.bonitasoft.engine.execution.work.TxBonitaWork.lambda$work$0(TxBonitaWork.java:42)
org.bonitasoft.engine.execution.work.TxBonitaWork$$Lambda$150/1691360812.call(Unknown Source)
org.bonitasoft.engine.transaction.JTATransactionServiceImpl.executeInTransaction(JTATransactionServiceImpl.java:274)
org.bonitasoft.engine.execution.work.TxBonitaWork.work(TxBonitaWork.java:42)
org.bonitasoft.engine.execution.work.LockProcessInstanceWork.work(LockProcessInstanceWork.java:63)
org.bonitasoft.engine.execution.work.failurewrapping.TxInHandleFailureWrappingWork.work(TxInHandleFailureWrappingWork.java:41)
org.bonitasoft.engine.execution.work.failurewrapping.TxInHandleFailureWrappingWork.work(TxInHandleFailureWrappingWork.java:41)
org.bonitasoft.engine.execution.work.failurewrapping.TxInHandleFailureWrappingWork.work(TxInHandleFailureWrappingWork.java:41)
org.bonitasoft.engine.execution.work.InSessionBonitaWork.work(InSessionBonitaWork.java:59)
org.bonitasoft.engine.work.BonitaThreadPoolExecutor.lambda$submit$1(BonitaThreadPoolExecutor.java:132)
org.bonitasoft.engine.work.BonitaThreadPoolExecutor$$Lambda$148/440663460.run(Unknown Source)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
-10-01 16:43:58.026 -0400 WARN: com.arjuna.ats.arjuna ARJUNA012108: CheckedAction::check - atomic action 0:ffff0afe3223:d3cd:5f763ec9:1ffa aborting with 1 threads active!
2020-10-01 16:43:58.026 -0400 WARN: com.arjuna.ats.arjuna ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff0afe3223:d3cd:5f763ec9:1ffa
2020-10-01 16:43:58.026 -0400 WARN: com.arjuna.ats.arjuna ARJUNA012095: Abort of action id 0:ffff0afe3223:d3cd:5f763ec9:1ffe invoked while multiple threads active within it.
2020-10-01 16:43:58.026 -0400 WARN: com.arjuna.ats.arjuna ARJUNA012381: Action id 0:ffff0afe3223:d3cd:5f763ec9:1ffe completed with multiple threads - thread BonitaTaskExecutor was in progress with java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
java.net.SocketInputStream.read(SocketInputStream.java:171)
java.net.SocketInputStream.read(SocketInputStream.java:141)
org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:214)
org.postgresql.core.PGStream.receive(PGStream.java:462)
org.postgresql.core.PGStream.receiveTupleV3(PGStream.java:422)
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2146)
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143)
org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:106)
sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.postgresql.ds.PGPooledConnection$StatementHandler.invoke(PGPooledConnection.java:428)
com.sun.proxy.$Proxy11.executeQuery(Unknown Source)
org.apache.tomcat.dbcp.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122)
org.apache.tomcat.dbcp.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122)
org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:82)
org.hibernate.loader.Loader.getResultSet(Loader.java:2066)
org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1863)
org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1839)
org.hibernate.loader.Loader.doQuery(Loader.java:910)
org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:355)
org.hibernate.loader.Loader.doList(Loader.java:2554)
org.hibernate.loader.Loader.doList(Loader.java:2540)
org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2370)
org.hibernate.loader.Loader.list(Loader.java:2365)
org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:497)
org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:387)
org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:236)
org.hibernate.internal.SessionImpl.list(SessionImpl.java:1300)
org.hibernate.internal.QueryImpl.list(QueryImpl.java:103)
org.hibernate.internal.AbstractQueryImpl.uniqueResult(AbstractQueryImpl.java:966)
org.bonitasoft.engine.persistence.AbstractHibernatePersistenceService.selectOne(AbstractHibernatePersistenceService.java:397)
org.bonitasoft.engine.persistence.AbstractHibernatePersistenceService.selectOne(AbstractHibernatePersistenceService.java:320)
org.bonitasoft.engine.dependency.impl.TenantDependencyService.getDependencyContentOnly(TenantDependencyService.java:232)
org.bonitasoft.engine.dependency.impl.AbstractDependencyService.lambda$getDependenciesResources$0(AbstractDependencyService.java:70)
org.bonitasoft.engine.dependency.impl.AbstractDependencyService$$Lambda$128/1896934036.apply(Unknown Source)
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
org.bonitasoft.engine.classloader.BonitaClassLoader.addResources(BonitaClassLoader.java:101)
org.bonitasoft.engine.classloader.BonitaClassLoader.<init>(BonitaClassLoader.java:70)
org.bonitasoft.engine.classloader.ClassLoaderUpdater.doInitializeClassLoader(ClassLoaderUpdater.java:56)
org.bonitasoft.engine.classloader.ClassLoaderUpdater.lambda$initializeClassLoader$1(ClassLoaderUpdater.java:48)
org.bonitasoft.engine.classloader.ClassLoaderUpdater$$Lambda$125/2010267676.call(Unknown Source)
org.bonitasoft.engine.transaction.JTATransactionServiceImpl.executeInTransaction(JTATransactionServiceImpl.java:274)
org.bonitasoft.engine.classloader.ClassLoaderUpdater.lambda$inTransaction$3(ClassLoaderUpdater.java:85)
org.bonitasoft.engine.classloader.ClassLoaderUpdater$$Lambda$126/1505813797.call(Unknown Source)
org.bonitasoft.engine.classloader.ClassLoaderUpdater.lambda$inSession$2(ClassLoaderUpdater.java:77)
org.bonitasoft.engine.classloader.ClassLoaderUpdater$$Lambda$127/1081293638.call(Unknown Source)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
2021-10-01 16:43:58.026 -0400 WARN: com.arjuna.ats.arjuna ARJUNA012108: CheckedAction::check - atomic action 0:ffff0afe3223:d3cd:5f763ec9:1ffe aborting with 1 threads active!
2020-10-01 16:43:58.526 -0400 WARN: com.arjuna.ats.arjuna ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0afe3223:d3cd:5f763ec9:1ffe in state CANCEL
2020-10-01 16:43:58.526 -0400 WARN: com.arjuna.ats.arjuna ARJUNA012378: ReaperElement appears to be wedged: org.postgresql.core.QueryExecutorBase.getTransactionState(QueryExecutorBase.java:252)
org.postgresql.jdbc.PgConnection.rollback(PgConnection.java:798)
org.postgresql.xa.PGXAConnection.rollback(PGXAConnection.java:449)
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:362)
com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:3023)
com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:3002)
com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1674)
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:124)
com.arjuna.ats.arjuna.AtomicAction.cancel(AtomicAction.java:215)
com.arjuna.ats.arjuna.coordinator.TransactionReaper.doCancellations(TransactionReaper.java:381)
com.arjuna.ats.internal.arjuna.coordinator.ReaperWorkerThread.run(ReaperWorkerThread.java:78)
2021-08-27 13:46:15.421 +0000 WARN (Transaction Reaper) com.arjuna.ats.arjuna ARJUNA012117: TransactionReaper::check timeout for TX 0:ffffac140ec7:8051:611e75cd:dddc in state RUN
2021-08-27 13:46:15.431 +0000 WARN (Transaction Reaper Worker 0) com.arjuna.ats.arjuna ARJUNA012095: Abort of action id 0:ffffac140ec7:8051:611e75cd:dddc invoked while multiple threads active within it.
2021-08-27 13:46:15.433 +0000 WARN (Transaction Reaper Worker 0) com.arjuna.ats.arjuna ARJUNA012381: Action id 0:ffffac140ec7:8051:611e75cd:dddc completed with multiple threads - thread http-nio-8080-exec-30 was in progress with java.base@11.0.11/java.net.SocketInputStream.socketRead0(Native Method)
java.base@11.0.11/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
java.base@11.0.11/java.net.SocketInputStream.read(SocketInputStream.java:168)
java.base@11.0.11/java.net.SocketInputStream.read(SocketInputStream.java:140)
org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:140)
org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:109)
org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:67)
org.postgresql.core.PGStream.receiveChar(PGStream.java:306)
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1952)
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143)
org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:106)
org.apache.tomcat.dbcp.dbcp2.PoolableConnection.validate(PoolableConnection.java:302)
org.apache.tomcat.dbcp.dbcp2.PoolableConnectionFactory.validateConnection(PoolableConnectionFactory.java:656)
org.apache.tomcat.dbcp.dbcp2.PoolableConnectionFactory.validateObject(PoolableConnectionFactory.java:674)
org.apache.tomcat.dbcp.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:476)
org.apache.tomcat.dbcp.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:353)
org.apache.tomcat.dbcp.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:134)
org.apache.tomcat.dbcp.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:733)
org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122)
org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator$ConnectionProviderJdbcConnectionAccess.obtainConnection(JdbcEnvironmentInitiator.java:180)
org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:68)
org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)
org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:101)
org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:263)
org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:237)
org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
org.hibernate.id.factory.internal.DefaultIdentifierGeneratorFactory.injectServices(DefaultIdentifierGeneratorFactory.java:152)
org.hibernate.service.internal.AbstractServiceRegistryImpl.injectDependencies(AbstractServiceRegistryImpl.java:286)
org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:243)
org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
org.hibernate.boot.internal.InFlightMetadataCollectorImpl.<init>(InFlightMetadataCollectorImpl.java:175)
org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:118)
org.hibernate.boot.model.process.spi.MetadataBuildingProcess.build(MetadataBuildingProcess.java:83)
org.hibernate.boot.internal.MetadataBuilderImpl.build(MetadataBuilderImpl.java:473)
org.hibernate.boot.internal.MetadataBuilderImpl.build(MetadataBuilderImpl.java:84)
org.hibernate.boot.MetadataSources.buildMetadata(MetadataSources.java:185)
org.bonitasoft.engine.business.data.impl.SchemaManagerUpdate.buildConfiguration(SchemaManagerUpdate.java:62)
org.bonitasoft.engine.business.data.impl.SchemaManagerUpdate.update(SchemaManagerUpdate.java:94)
org.bonitasoft.engine.business.data.impl.BusinessDataModelRepositoryImpl.update(BusinessDataModelRepositoryImpl.java:186)
org.bonitasoft.engine.business.data.impl.BusinessDataModelRepositoryImpl.createAndDeployServerBDMJar(BusinessDataModelRepositoryImpl.java:178)
org.bonitasoft.engine.business.data.impl.BusinessDataModelRepositoryImpl.install(BusinessDataModelRepositoryImpl.java:163)
org.bonitasoft.engine.api.impl.TenantAdministrationAPIImpl.installBusinessDataModel(TenantAdministrationAPIImpl.java:156)
jdk.internal.reflect.GeneratedMethodAccessor163.invoke(Unknown Source)
java.base@11.0.11/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.base@11.0.11/java.lang.reflect.Method.invoke(Method.java:566)
org.bonitasoft.engine.api.impl.ServerAPIImpl.invokeAPI(ServerAPIImpl.java:427)
org.bonitasoft.engine.api.impl.ServerAPIImpl.lambda$invokeAPIInTransaction$1(ServerAPIImpl.java:395)
org.bonitasoft.engine.api.impl.ServerAPIImpl$$Lambda$475/0x0000000840c37840.call(Unknown Source)
org.bonitasoft.engine.transaction.JTATransactionServiceImpl.executeInTransaction(JTATransactionServiceImpl.java:269)
org.bonitasoft.engine.api.impl.ServerAPIImpl.invokeAPIInTransaction(ServerAPIImpl.java:391)
org.bonitasoft.engine.api.impl.ServerAPIImpl.invokeAPI(ServerAPIImpl.java:265)
org.bonitasoft.engine.api.impl.ServerAPIImpl.invokeMethod(ServerAPIImpl.java:121)
org.bonitasoft.engine.api.impl.ClientInterceptor.invoke(ClientInterceptor.java:79)
com.sun.proxy.$Proxy143.installBusinessDataModel(Unknown Source)
org.bonitasoft.web.rest.server.api.bdm.BusinessDataModelResource.addBDM(BusinessDataModelResource.java:65)
jdk.internal.reflect.GeneratedMethodAccessor161.invoke(Unknown Source)
java.base@11.0.11/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.base@11.0.11/java.lang.reflect.Method.invoke(Method.java:566)
org.restlet.resource.ServerResource.doHandle(ServerResource.java:508)
org.restlet.resource.ServerResource.post(ServerResource.java:1341)
org.restlet.resource.ServerResource.doHandle(ServerResource.java:606)
org.bonitasoft.web.rest.server.api.resource.CommonResource.doHandle(CommonResource.java:206)
org.restlet.resource.ServerResource.doNegotiatedHandle(ServerResource.java:662)
org.restlet.resource.ServerResource.doConditionalHandle(ServerResource.java:348)
org.restlet.resource.ServerResource.handle(ServerResource.java:1020)
org.restlet.resource.Finder.handle(Finder.java:236)
org.restlet.routing.Filter.doHandle(Filter.java:150)
org.restlet.routing.Filter.handle(Filter.java:197)
org.restlet.routing.Router.doHandle(Router.java:422)
org.restlet.routing.Router.handle(Router.java:641)
org.restlet.routing.Filter.doHandle(Filter.java:150)
org.restlet.routing.Filter.handle(Filter.java:197)
org.restlet.routing.Filter.doHandle(Filter.java:150)
org.restlet.routing.Filter.handle(Filter.java:197)
org.restlet.routing.Filter.doHandle(Filter.java:150)
org.restlet.engine.application.StatusFilter.doHandle(StatusFilter.java:140)
org.restlet.routing.Filter.handle(Filter.java:197)
org.restlet.routing.Filter.doHandle(Filter.java:150)
org.restlet.routing.Filter.handle(Filter.java:197)
org.restlet.engine.CompositeHelper.handle(CompositeHelper.java:202)
org.restlet.engine.application.ApplicationHelper.handle(ApplicationHelper.java:77)
org.restlet.Application.handle(Application.java:385)
org.bonitasoft.web.rest.server.BonitaRestletApplication.handle(BonitaRestletApplication.java:191)
org.restlet.routing.Filter.doHandle(Filter.java:150)
org.restlet.routing.Filter.handle(Filter.java:197)
org.restlet.routing.Router.doHandle(Router.java:422)
org.restlet.routing.Router.handle(Router.java:641)
org.restlet.routing.Filter.doHandle(Filter.java:150)
org.restlet.routing.Filter.handle(Filter.java:197)
org.restlet.routing.Router.doHandle(Router.java:422)
org.restlet.routing.Router.handle(Router.java:641)
org.restlet.routing.Filter.doHandle(Filter.java:150)
org.restlet.routing.Filter.handle(Filter.java:197)
org.restlet.engine.CompositeHelper.handle(CompositeHelper.java:202)
org.restlet.Component.handle(Component.java:408)
org.restlet.Server.handle(Server.java:507)
org.restlet.engine.connector.ServerHelper.handle(ServerHelper.java:63)
org.restlet.engine.adapter.HttpServerHelper.handle(HttpServerHelper.java:143)
org.restlet.ext.servlet.ServerServlet.service(ServerServlet.java:1117)
javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:232)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:167)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:194)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:167)
org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:194)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:167)
org.bonitasoft.console.common.server.login.filter.AbstractAuthorizationFilter.doFilter(AbstractAuthorizationFilter.java:61)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:194)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:167)
org.bonitasoft.console.common.server.login.filter.AbstractAuthorizationFilter.doFilter(AbstractAuthorizationFilter.java:61)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:194)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:167)
org.bonitasoft.console.common.server.filter.NoCacheFilter.doFilter(NoCacheFilter.java:51)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:194)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:167)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:544)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:364)
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:624)
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1651)
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
Wildfly
These messages can be found in the server.log file:
2018-10-08 15:23:41,114 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0afe21b2:4672bcb6:5bbb73ed:6930 in state RUN
2018-10-08 15:23:41,117 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0afe21b2:4672bcb6:5bbb73ed:6930 invoked while multiple threads active within it.
2018-10-08 15:23:41,137 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012381: Action id 0:ffff0afe21b2:4672bcb6:5bbb73ed:6930 completed with multiple threads - thread default task-23 was in progress with java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.socketRead(SocketInputStream.java:116) java.net.SocketInputStream.read(SocketInputStream.java:171) java.net.SocketInputStream.read(SocketInputStream.java:141) ...
2018-10-08 15:23:41,138 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012108: CheckedAction::check - atomic action 0:ffff0afe21b2:4672bcb6:5bbb73ed:6930 aborting with 1 threads active!
2018-10-08 15:23:41,615 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0afe21b2:4672bcb6:5bbb73ed:6930 in state CANCEL
2018-10-08 15:23:41,616 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012378: ReaperElement appears to be wedged: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) ...
2018-10-08 15:23:42,116 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0afe21b2:4672bcb6:5bbb73ed:6930 in state CANCEL_INTERRUPTED
2018-10-08 15:23:42,117 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012120: TransactionReaper::check worker Thread[Transaction Reaper Worker 0,5,main] not responding to interrupt when cancelling TX 0:ffff0afe21b2:4672bcb6:5bbb73ed:6930 -- worker marked as zombie and TX scheduled for mark-as-rollback
2018-10-08 15:23:42,117 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012110: TransactionReaper::check successfuly marked TX 0:ffff0afe21b2:4672bcb6:5bbb73ed:6930 as rollback only
2018-11-29 17:18:17,459 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff7f000101:-201592e0:5c000d43:e772 in state RUN
2018-11-29 17:18:17,478 DEBUG [org.hibernate.engine.jdbc.internal.LogicalConnectionImpl] (default task-36) Obtained JDBC connection
2018-11-29 17:18:17,508 DEBUG [org.hibernate.engine.jdbc.internal.LogicalConnectionImpl] (default task-36) Releasing JDBC connection
2018-11-29 17:18:17,508 DEBUG [org.hibernate.engine.jdbc.internal.LogicalConnectionImpl] (default task-36) Released JDBC connection
2018-11-29 17:18:17,513 DEBUG [org.hibernate.engine.jdbc.internal.LogicalConnectionImpl] (default task-36) Obtaining JDBC connection
2018-11-29 17:18:17,516 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff7f000101:-201592e0:5c000d43:e772 invoked while multiple threads active within it.
2018-11-29 17:18:17,517 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012381: Action id 0:ffff7f000101:-201592e0:5c000d43:e772 completed with multiple threads - thread default task-36 was in progress with java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ...
2018-11-29 17:18:17,517 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012108: CheckedAction::check - atomic action 0:ffff7f000101:-201592e0:5c000d43:e772 aborting with 1 threads active!
2018-11-29 17:18:17,558 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff7f000101:-201592e0:5c000d43:e772
2018-11-29 17:18:17,571 WARN [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-36) SQL Error: 0, SQLState: null
2018-11-29 17:18:17,571 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-36) javax.resource.ResourceException: IJ000457: Unchecked throwable in managedConnectionReconnected() cl=org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@63fbf68b[state=NORMAL managed connection=org.jboss.jca.adapters.jdbc.xa.XAManagedConnection@78744f3f connection handles=0 lastReturned=1543508297571 lastValidated=1543508198248 lastCheckedOut=1543508297513 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool@3ee3e0ef mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool@31f31673[pool=BusinessDataDS] xaResource=XAResourceWrapperImpl@1f3212c3[xaResource=org.jboss.jca.adapters.jdbc.xa.XAManagedConnection@78744f3f pad=false overrideRmValue=null productName=Microsoft SQL Server productVersion=13.00.4001 jndiName=java:jboss/datasources/BusinessDataDS] txSync=null]
2018-11-29 17:18:17,571 DEBUG [org.hibernate.engine.jdbc.internal.LogicalConnectionImpl] (default task-36) Aggressively releasing JDBC connection
2018-11-29 17:18:17,571 DEBUG [org.hibernate.engine.jdbc.internal.LogicalConnectionImpl] (default task-36) Aggressively releasing JDBC connection
2018-11-29 17:18:17,571 WARN [com.arjuna.ats.arjuna] (default task-36) ARJUNA012077: Abort called on already aborted atomic action 0:ffff7f000101:-201592e0:5c000d43:e772
2018-11-29 17:18:17,571 DEBUG [com.bonitasoft.engine.api.impl.ServerAPIExt] (default task-36) THREAD_ID=233 | HOSTNAME=quantum-srv1 | TENANT_ID=1 | javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff7f000101:-201592e0:5c000d43:e772: org.bonitasoft.engine.transaction.STransactionCommitException: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff7f000101:-201592e0:5c000d43:e772 ... Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff7f000101:-201592e0:5c000d43:e772 ...
2018-11-29 17:18:17,571 TRACE [com.bonitasoft.engine.api.impl.ServerAPIExt] (default task-36) THREAD_ID=233 | HOSTNAME=quantum-srv1 | TENANT_ID=1 | End Server API call org.bonitasoft.engine.api.CommandAPI execute
2018-11-29 17:18:17,572 SEVERE [org.restlet.Component.BonitaSPRestletApplication] (default task-36) Exception or error caught in server resource: org.bonitasoft.engine.exception.BonitaRuntimeException: USERNAME=walter.bates | org.bonitasoft.engine.transaction.STransactionCommitException: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff7f000101:-201592e0:5c000d43:e772 ... Caused by: org.bonitasoft.engine.transaction.STransactionCommitException: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff7f000101:-201592e0:5c000d43:e772 ... Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff7f000101:-201592e0:5c000d43:e772 ...
2018-11-29 17:18:17,572 SEVERE [org.bonitasoft.web.rest.server.api.resource.CommonResource] (default task-36) USERNAME=walter.bates | org.bonitasoft.engine.transaction.STransactionCommitException: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff7f000101:-201592e0:5c000d43:e772: org.bonitasoft.engine.exception.BonitaRuntimeException: USERNAME=walter.bates | org.bonitasoft.engine.transaction.STransactionCommitException: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff7f000101:-201592e0:5c000d43:e772 ... Caused by: org.bonitasoft.engine.transaction.STransactionCommitException: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff7f000101:-201592e0:5c000d43:e772 ... Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff7f000101:-201592e0:5c000d43:e772 ...
2018-11-29 17:18:18,129 INFO [org.hibernate.event.internal.DefaultLoadEventListener] (default task-36) HHH000327: Error performing load command : org.hibernate.HibernateException: Logical connection is closed
2018-11-29 17:18:18,131 DEBUG [org.hibernate.engine.jdbc.internal.LogicalConnectionImpl] (default task-36) Aggressively releasing JDBC connection
2018-11-29 17:18:18,131 DEBUG [org.hibernate.engine.jdbc.internal.LogicalConnectionImpl] (default task-36) Aggressively releasing JDBC connection
2018-11-29 17:18:18,131 TRACE [org.hibernate.engine.jdbc.internal.LogicalConnectionImpl] (default task-36) Closing logical connection
2018-11-29 17:18:18,131 TRACE [org.hibernate.engine.jdbc.internal.LogicalConnectionImpl] (default task-36) Logical connection closed
2018-11-29 17:18:18,131 DEBUG [org.hibernate.engine.jdbc.internal.LogicalConnectionImpl] (default task-36) Aggressively releasing JDBC connection
2018-11-29 17:18:18,131 TRACE [org.hibernate.engine.jdbc.internal.LogicalConnectionImpl] (default task-36) Closing logical connection
2018-11-29 17:18:18,131 TRACE [org.hibernate.engine.jdbc.internal.LogicalConnectionImpl] (default task-36) Logical connection closed
2018-11-29 17:18:18,131 DEBUG [com.bonitasoft.engine.api.impl.ServerAPIExt] (default task-36) THREAD_ID=233 | HOSTNAME=quantum-srv1 | TENANT_ID=1 | javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed: org.bonitasoft.engine.persistence.SRetryableException: javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed at org.bonitasoft.engine.business.data.impl.JPABusinessDataRepositoryImpl.findById(JPABusinessDataRepositoryImpl.java:193) ... Caused by: javax.persistence.PersistenceException: org.hibernate.HibernateException: Logical connection is closed ...