Issue with upgrading Bonita Community Edition 6.3.2 to latest

Submitted by Michele.DuBose on Tue, 06/05/2018 - 19:16
1
0
-1

Hello,

I am in the process of upgrading my Bonita platform to the latest version. I'm following the documentation https://documentation.bonitasoft.com/bonita/7.6/migrate-from-an-earlier-... and noticed the section that talks about the constraints. I've ran the migration script for version 7.0 and it seems to be stuck at 'Migrating Arch_data_mapping'. This script has been running for 5 days and I'm not sure if it's continuing or have silently failed. According to the documentation, if you add indexes to the arch_data_mapping table, the migration will not complete. Does this mean, I will notice it failing or it will appear as if it's continuing but really isn't? Also, I have not added any indexes to tables but have noticed that there are indexes added to the arch_data_mapping table. It appears that the indexes were created by the application. Will the indexes created by the application affect the migration from completing?

bonita 6.32
Migration Tool

Comments

Submitted by Michele.DuBose on Mon, 06/11/2018 - 22:07

I would like to mention that the command 'Migrating Arch_data_mapping' is displayed in the command prompt. The command prompt has not closed which tells me the script has not failed. Also, I'm able to view the query in my database that this command is running. In addition, the database tables in the file system are displayed and the Arch_data_mapping table has a time stamp that has been updating. These all seem to be an indication that the migration is still running but is it suppose to take this long on one command?

Permalink
1 answer

Submitted by pierrick.voulet on Mon, 06/11/2018 - 22:47
1
+1
-1

Hello,

If the table only has the indexes coming by default (none added by yourself) this should run fine.

Migration time depends on the amount of data there is in the DB. The archive tables are usually the largest part and can take a while. It is not common to see a migration spending that much time though, could you elaborate on your DB/table sizes?

Comments

Submitted by Michele.DuBose on Tue, 06/12/2018 - 02:52

This is the query that's running: DELETE FROM arch_data_mapping WHERE NOT EXISTS (SELECT NULL FROM arch_data_instance i WHERE i.sourceObjectId = arch_data_mapping.dataInstanceId)

I ran I SELECT Count on both tables and for arch_data_mapping table, there are 7,509,234 records. For arch_data_instance table, there are 1,521,491 records. DB size in MB is 1736.3.

Permalink
Submitted by pierrick.voulet on Thu, 06/14/2018 - 23:49

What migration tool version do you use?
During the migration from what version to what version is it stuck? You should be able to get the information from the logs.

Permalink
Submitted by Michele.DuBose on Fri, 06/15/2018 - 18:15

I'm using version 7.0 of the migration tool. It's stuck when going from 6.3.2 to 6.3.3. When you say, get information from the logs, are you referring to the migration logs or the bonita home logs? If it's the migration logs, the logs do not tell me anything. It just displays what step the migration is on which is 'Migrating Arch_data_mapping'. This is what I see:
+---------------------------------------------+
| Migration of version 6.3.2 to version 6.3.3 |
| migration number 1 of a total of 16 |
+---------------------------------------------+
Continue migration? (yes/no):
Migration of database:

| [ Migrating 1/6 ]
| Description : Update default theme
| | executing SQL statement:[UPDATE theme
SET lastUpdateDate = 1527787977822
WHERE isDefault = true
AND type = 'PORTAL';

UPDATE theme
SET lastUpdateDate = 1527787977822 + 1
WHERE isDefault = false
AND type = 'PORTAL';
]
| [ Migration step success in 0.453 seconds. Migration started 4.109 seconds ago. ]
|
| [ Migrating 2/6 ]
| Description : Clean archive data mapping orphans.
| | Deleting archived data mapping orphans for all tenants

Permalink
Submitted by pierrick.voulet on Mon, 06/18/2018 - 19:37

Ok so 7.0 is the version of Bonita you are targeting during this migration.

The version of the migration tool that you should be using is 1.24.3 as it is the latest version that can complete such a migration (from a version 6 to version 7).

If you use this version of this migration, you may be experiencing a slowness in its execution and will need to wait until completion to avoid any data corruption. You may want to give another try executing the migration into two separated steps:
- First migration to 6.3.2
- Manually run the long SQL request (nothing structural so should not have any impact)
- Second migration to 7.0

Before doing something like this, please make sure you do backup your entire platform as discussed in the documentation here .

Permalink
Submitted by Michele.DuBose on Mon, 06/18/2018 - 19:42

Yes I am using the migration tool version 1.24.3. I don't understand your suggestion in executing the migration into two separated steps. I am coming from version 6.3.2. The migration gets stuck at 6.3.3 with the database query taking a long time to execute.

Permalink
Submitted by pierrick.voulet on Mon, 06/18/2018 - 19:53

I was suggesting that you do restart the migration entirely, this time trying to manually run the SQL request that blocks the migration execution directly on your DBMS before.

Permalink
Submitted by Michele.DuBose on Tue, 06/19/2018 - 16:27

I ran the query in MySQL workbench and it timed out. It seems the issue is not with the migration script but the database volume. The idea of purging data seems risky especially because this is an open source application. I'm not sure what to do at this point.

Permalink
Submitted by pierrick.voulet on Tue, 06/19/2018 - 23:58

What about breakdown the SQL request execution into smaller ones. Each of them could run on a small set of rows instead of entire table for example.

Permalink
Submitted by Michele.DuBose on Thu, 06/28/2018 - 20:35

Thanks pierrick.voulet. I was able to get pass the issue with the query command by setting indexes on the arch_data_mapping and the arch_data_instance tables even though the documentation said by doing this, the migration would fail. The migration from 6.3.2 to version 7.0 did not fail by doing so. Now, I am attempting to migrate from version 7.0 to 7.6.3 by running the migration tool-2.29.0. When I run this tool, it fails at version 7.4.0 with the following error message: Exception in thread "main" java.lang.IllegalStateException: some processes could not be migrated. see error log for more information. What does this mean?

Permalink
Submitted by pierrick.voulet on Thu, 06/28/2018 - 20:48

Good trick, the best would be to migrate with the indexes and then remove them for the next steps to limit the risk of board effects (you can specify what is the target version launching a migration run - so you can do a first migration run with the indexes to pass the step that was failing only, remove the indexes and launch a second migration to pass all the rest).

This is likely to be an error during an attempt to migrate an already-deployed process definition (XML located in DB) into the new format used in 7.4.0. Anything interesting in the logs?

Permalink
Submitted by Michele.DuBose on Thu, 06/28/2018 - 20:50

The error log, I assume this is speaking of is the migration log. If so, this is what is displayed.

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:80)
at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:60)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:235)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247)
at org.bonitasoft.migration.version.to7_4_0.MigrateProcessDefinitionXmlWithXSD.execute(MigrateProcessDefinitionXmlWithXSD.groovy:50)
at org.bonitasoft.migration.version.to7_4_0.MigrateProcessDefinitionXmlWithXSD$execute.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.bonitasoft.migration.version.to7_3_1.AddAvatarPermission$execute.call(Unknown Source)
at org.bonitasoft.migration.core.MigrationRunner$_run_closure1$_closure5.doCall(MigrationRunner.groovy:55)
at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1019)
at groovy.lang.Closure.call(Closure.java:426)
at groovy.lang.Closure.call(Closure.java:442)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2030)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2015)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2056)
at org.codehaus.groovy.runtime.dgm$162.invoke(Unknown Source)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.bonitasoft.migration.core.MigrationRunner$_run_closure1.doCall(MigrationRunner.groovy:51)
at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1019)
at groovy.lang.Closure.call(Closure.java:426)
at groovy.lang.Closure.call(Closure.java:442)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2030)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2015)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2056)
at org.codehaus.groovy.runtime.dgm$162.invoke(Unknown Source)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.bonitasoft.migration.core.MigrationRunner.run(MigrationRunner.groovy:42)
at org.bonitasoft.migration.core.MigrationRunner$run.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.bonitasoft.migration.core.Migration.run(Migration.groovy:56)
at org.bonitasoft.migration.core.Migration$run.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.bonitasoft.migration.core.Migration.main(Migration.groovy:41)

Permalink
Submitted by Michele.DuBose on Thu, 06/28/2018 - 20:58

Also, what do you mean by already deployed process definition?

Permalink
Submitted by pierrick.voulet on Thu, 06/28/2018 - 21:40

Process Definition: the process artifact that you build when you develop a process from the Bonita Studio (XML with specific XSD format)

Deployed Process Definition: a Process Definition that you have deployed on a Bonita Platform in order to use it. The Process Definition is then stored in the Bonita Engine DB and is used at runtime to start new cases, execute tasks, etc.

As the format of a valid Process Definition may change over time (and Bonita version), the migration tool takes care of them when necessary. It seems that there is some transformation to be done when you migrate to 7.4.0 because the migration fails trying to do it: you can see "MigrateProcessDefinitionXmlWithXSD" in the error logs.

How many deployed Process Definitions do you have on your platform before migration? If you have only few Process Definitions you might want to try to identify which of them generate errors and may be come with steps to reproduce the issue.

Regardless of how many you have, it is always better to remove the ones that are useless (disable and delete) to reduce the size of the DB, improve the migration performance and avoid useless risks of failures during migration.

You should be good with a 7.3 version after this migration which is already much better than 6.x. I suggest that you continue developing with this version as for now.

Permalink
Submitted by Michele.DuBose on Thu, 06/28/2018 - 22:25

I have 4 deployed process definitions in which all are used. Each process definition worked as expected before the migration so how would I go about identifying which generates an error? Also, are you suggesting I stop with version 7.3? If so, I assume I need to download the target version 7.3 instead of target version 7.6.3.

Permalink
Submitted by pierrick.voulet on Fri, 06/29/2018 - 00:01

You might find a hint in the logs like process name, version or ID. If not, because you have only few processes, if you did back up the entire platform before migrating you could rollback, remove one process and try the migration again. This is the only way I can think of.

You are right, you will need to download the 7.3.3 if you decide to continue with this version for now.

Permalink
Submitted by Michele.DuBose on Wed, 07/11/2018 - 22:48

Thanks pierrick.voulet for all your help. I left the migration at 7.3.3 since it failed when trying to migrate to 7.4 due to the already deployed process definition. By reviewing the migration logs, I can see that it migrated the process definition to the new format when migrating to 7.3.0. Below is the verbiage from the log. I've started tomcat and logged into the portal. Now, I'm getting a "Failed to deserialize the XML string provided" error when I go to the application log. Is this the same type of error I got when attempting to migrate my platform to v7.4? If so, I do not understand the cause of the error since from the migration log, it states that the process definition migrated successfully.

[INFO] | Execute migration step: Update process definition xml to the new format
[INFO] | --> Migration step success in 0.172 seconds. Migration started 39.501 seconds ago. --
[INFO] ---------------
[INFO] ---------------
[INFO] | Execute migration step: Add auto login settings from process to tenant portal configuration file
[INFO] | --> Migration step success in 0.093 seconds. Migration started 39.594 seconds ago. --
[INFO] ---------------
[INFO] Updating platform version in the database ...
[INFO] Platform version in database is now 7.3.0

Permalink
Submitted by pierrick.voulet on Tue, 07/17/2018 - 03:49

Sorry but I never saw this issue before, you might want to make another clean try of the migration from the beginning with 7.3.3 as a target. Actually, this could be related to some corruption due to your previous tentative to go further than the 7.3.3 version as a target.

HIH

Permalink
Submitted by Michele.DuBose on Tue, 07/17/2018 - 17:10

That's what I was thinking and was planning to migrate again from 6.3.2 stopping at 7.3.3. Thanks again.

Permalink
Submitted by Michele.DuBose on Fri, 07/27/2018 - 21:30

Hi pierrick.voulet. The migration from 6.3.2 to 7.3.3 has completed successfully. Now, when I run the startup.bat and open http://localhost:8080/bonita, everything works as expected. When running tomcat 7 as a service, http://localhost:8080 comes up but http://localhost:8080\bonita does not. I've added the configurations in the tomcat7w file such as pointing to the jdk file, and added settings in the java options section. I noticed in the tomcat7w that it also has a java 9 options section with information in it. Is this needed? In the bonita log files, I'm getting warnings such as 'Invalid bean definition with name...'. What does this mean? Could this be a cause to the reason http://localhost:8080\bonita does not work? The java options that I've added in the tomcat7w file are the same settings in the setenv.bat file. I believe when running the startup.bat, it's referencing the setenv.bat file.

Thank you for all your help.

Permalink
Submitted by pierrick.voulet on Fri, 07/27/2018 - 22:41

Hi Michele - I do not have experience in setting Tomcat as a service but you should create a new question in the forum to have more members aware.

Permalink
Notifications