Liferay 6.2 > 7 upgrade issues

[Pieter Roos, modificado hace 7 años.]

HI all,

As a Liferay enthousiast I am really disappointed about the new 7 GA3 release (personal experience is anything below GA3 is totally unstable), all kind of strange errors occur such as the one below which prevents me from upgrading, no explanation table info or anything given. Having a lot of problems with duplicate key errors
com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '113677-60865' for key 'IX_7332B44F'.
For a table that was already imported with SQL importer. And note it is from a live production database where only the database is converted to UTF-8.

When using a fresh install I am unable to add 'Basic Web Content' as it is simply unavailable.

Please Liferay make it work better than this... Personally I see Liferay 7 as not fit for production as these failures should be easilty fixable or not occur in the first place.

[David H Nebinger, modificado hace 7 años.]

So usually if you dig into the index definition and then check the source table you can see that duplicate entries actually do exist and blocks the upgrade.

The failure of the DB upgrade is a good thing - it highlights issues in your data that may lead to instability or corruption in the future. You are required to fix the data issues, also a good thing, as there is no easy way for the system to decide which record should be kept.

For the "I am unable to add basic web content", throwing out a generic statement w/o providing any details is not helpful to you nor anyone getting on this thread in the future. Lots of folks have been using LR7 and have been creating web content through all of the previous GAs as well as GA3. If there was some sort of issue, it would have been reported long before now.






Come meet me at the LSNA!

[Zak Thompson, modificado hace 7 años.]

Hi David,

Do you have any tricks for hunting down these index issues?

We are running into a similar issue upgrading our database:

com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '6796697-1098239' for key 'IX_7332B44F'

       at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:91)

       at com.liferay.portal.kernel.upgrade.util.UpgradeProcessUtil._upgradeProcess(UpgradeProcessUtil.java:175)

       at com.liferay.portal.kernel.upgrade.util.UpgradeProcessUtil.upgradeProcess(UpgradeProcessUtil.java:143)

       at com.liferay.portal.kernel.upgrade.util.UpgradeProcessUtil.upgradeProcess(UpgradeProcessUtil.java:125)

       at com.liferay.portal.events.StartupHelper.upgradeProcess(StartupHelper.java:164)

       at com.liferay.portal.events.StartupHelperUtil.upgradeProcess(StartupHelperUtil.java:81)

       at com.liferay.portal.tools.DBUpgrader.upgrade(DBUpgrader.java:157)

       at com.liferay.portal.tools.DBUpgrader.main(DBUpgrader.java:103)

Caused by: com.liferay.portal.kernel.upgrade.UpgradeException: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '6796697-1098239' for key 'IX_7332B44F'

       at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:91)

       at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:115)

       at com.liferay.portal.upgrade.UpgradeProcess_7_0_1.doUpgrade(UpgradeProcess_7_0_1.java:42)

       at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:88)

       ... 7 more

Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '6796697-1098239' for key 'IX_7332B44F'

       at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

       at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

       at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

       at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

       at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)

       at com.mysql.jdbc.Util.getInstance(Util.java:386)

       at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1040)

       at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4120)

       at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4052)

       at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2503)

       at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2664)

       at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2794)

       at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)

       at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1379)

       at com.liferay.portal.upgrade.v7_0_1.UpgradeDocumentLibrary.updateTikaRawMetadataFileEntryMetadata(UpgradeDocumentLibrary.java:128)

       at com.liferay.portal.upgrade.v7_0_1.UpgradeDocumentLibrary.doUpgrade(UpgradeDocumentLibrary.java:59)

       at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:88)

       ... 10 more

However, when we look at the keys specified by the index specified, it is an index on DDMStructureId and fileVersionId. When we use those keys to manually examine the metadata table as well as entries in the DDMStructure and fileVersion tables, there is only one entry in the tables based on the specified pair.

Is it possible that the Liferay Upgrade is attempting to re-create the file metadata and causing these duplicate errors?

EDIT: For the given key pair on the index, heres the result of the query:

select * from dlfileentrymetadata where DDMStructureId=6796697 AND fileVersionId=1098239;

Result: (1 row)

# uuid_, fileEntryMetadataId, DDMStorageId, DDMStructureId, fileEntryTypeId, fileEntryId, fileVersionId
'4119028b-e58b-44c1-a9d5-a5c9eb481de1', '7327079', '7327080', '6796697', '0', '1098238', '1098239'

[David H Nebinger, modificado hace 7 años.]

Sorry, Zak, that's the same process I use to track the issue down; usually I end up finding some cruft in the DB that can be cleaned out.

When Liferay is updating a table, it's usually not trying to duplicate records in the same table violating keys on it's own.

Can I ask what version of Liferay are you trying to upgrade to? GA3 is out now and it should have fixed known errors in previous releases. Have you tried with GA3 to see if perhaps it's a non-issue now?





Come meet me at the LSNA!

[Zak Thompson, modificado hace 7 años.]

I believe I'm on GA3, the instance I'm working with right now was created via the Liferay IDE download bundle when creating a project workspace, and I created the instance on Monday of this week. It's possible that this might be GA2 though. If it is, is there a way to upgrade the GA2 instance to a GA3 instance, or should I just re-create from scratch for GA3?

[David H Nebinger, modificado hace 7 años.]

Oh, I wouldn't go through this pain for a development or starting instance...

You only want to do this pain when you are upgrading a production environment.

For your local, I'd just throw out the bundle and start over with GA3...





Come meet me at the LSNA!

[Zak Thompson, modificado hace 7 años.]

We're testing the DB upgrade tool against a cutback from a production database. Its a proof of concept for one of our clients for migrating from 6.2 to 7, along with developing a few new features for the POC, so we're trying to mimic what would happen in the case of updating a production environment. If it ends up being an issues with the data in the DB, we'll most likely have to write scripts then to alleviate the issues with the current DB so that the upgrade can go smoothly if they decide to go through with switching to 7.

EDIT: Because all of the data was with the dlfileentrymetadata table, for the last run I did I simply copied the data into a temp table just to keep the data, and then deleted all the entries in the dlfileentrymetadata table. The upgrade process is still currently running, but it has made it past the point where the exceptions started showing up last time.

[David H Nebinger, modificado hace 7 años.]

Okay, so discard whatever you were doing w/ the GA2 upgrade and start over using GA3 against the 6.2 cut.

Worst comes to worst, you could try removing the index, allow the upgrade to complete, check the table and then re-enable the index. It's a fugly way to get there, but it might get you around the problem temporarily...





Come meet me at the LSNA!

[Peter Ros, modificado hace 7 años.]

Thanks for your answers! I was wondering how this errors could appear in the first place (to better understand its root cause), since the database functions well now. It appears the indexes are added during setup?

Your solution to remove indexes would be an option for a test environment but for my production environment its unacceptable since I can't check the impact of it on further processes.

Thanks again!