Upgrade 6.0.6 to 6.1.0 beta 3 fail referential integrity

[Linus Sphinx, geändert vor 12 Jahren.]

In ver. 6.0.6 I re-index the database, shutdown, copy the data folder over to a 6.1.0 installation, delete a few extra webapps, add liferay-ext.portal/ROOT.xml to point it at the database and ldap, reasonable setenv.sh, fire it up and fail with, well first there is a couple file not found which true enough don't exist on the document library section both which it seems to survive, then it hits the schema upgrade for the image gallery and goes all fatal and shutdown after:

17:14:05,460 ERROR [MainServlet:204] com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '3298335-0-stats pic combo' for key 3
com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '3298335-0-stats pic combo' for key 3
at com.liferay.portal.events.StartupAction.run(StartupAction.java:57)

I look at the data, seems ok, export it to text, import it into a new database with no referential integritiy issues at all. Really, seems just fine. Not much error message but I look for the data and three columns that match the three part key of a third index and come up with DLFileEntry table that matches, UNIQUE KEY IX_ED5CA615, do a quick query and come away even more puzzled:

[linus@localhost liferay-portal-6.1.0]$ echo "select groupId,folderId,title from DLFileEntry where groupId = 3298335 and folderId = 0 and title = 'stats pic combo' " | mysql -B -N -u admin -pr0rc1i$ mrswwpe2456 | grep pic
3298335 0 stats pic combo
[linus@localhost liferay-portal-6.1.0]$

Seems to answer the question how could there be duplicate entries on a key field, there isn't, but begs more questions of the upgrade process.
Full stack trace available here: catalina.out

ED: mysql-5.0.77-4 from redhat el5 rpm

[Mika Koivisto, geändert vor 12 Jahren.]

Make sure you have the correct dl.store.impl set (property and classname name changed). Also make sure you set image.hook.impl so that Image Gallery files are migrated to document library. I fixed some upgrade issues recently so you might want to try latest trunk.

[Linus Sphinx, geändert vor 12 Jahren.]

Thanks, I'll try the latest and have a look, not overriding thoses classes as far as I know.

[Alberto Chaparro, geändert vor 12 Jahren.]

Hi,

As Mika says, I recommend you to download sql upgrade files from trunk and use them.

On the other hand, your error is very common when we starts up the server to migrate the database and an error is displayed during this process, we fix the error and after we restart the server again. In these cases we need to restore the previous database (6.0.6 in your case) before restarting the server again to migrate to 6.1 (in your case).

Regards.

[Linus Sphinx, geändert vor 12 Jahren.]

Building the trunk from, `svn co svn://svn.liferay.com/repos/public/portal/trunk --username=guest`, and installing it, copy over data folder wholesale, set portal-ext.properties, ROOT.xml and setenv.sh, still gives me the referential integrity errors in;

Upgrading com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery:

16:47:19,632 ERROR [MainServlet:197] com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: com.mysql.jdbc.exceptions.MySQLIntegrityConstraintViolationException: Duplicate entry '3298335-0-stats pic combo' for key 3

com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: com.mysql.jdbc.exceptions.MySQLIntegrityConstraintViolationException: Duplicate entry '3298335-0-stats pic combo' for key 3

When there is no duplicate key. Says it twice, that may be a clue. Is there any fresher build?

[Trey Howard, geändert vor 12 Jahren.]

Linus

Did you get your problem resolved?
I'm having a separate but potentially related problem with the upgrade to 6.1 (beta 4 and trunk).
My error is on a permission record but seems similar in that it's an issue with referential integrity.

-Trey

[Linus Sphinx, geändert vor 12 Jahren.]

Thanks for asking but no, still struggling. Caught the flu, really knocked me on my ass for several weeks. The problem seems to, and this is a few weeks old recollection colored by spats of deep fever induced delirium, I'll be starting again tomorrow and send some more details, stem from merging the graphics library into the document library. The name minus extension is in a non-unique column and is copied into a column that's part of a unique index key, hence all the referential integrity grief. To make my life worse it seems they have a lot of files using the same name and different extensions. Not a problem for the old image tables but fatal in the document table. I cannot figure out how the missing files got entries to be missing but will start working on some pre-upgrade scripts I'll gladly share too.

[Trey Howard, geändert vor 12 Jahren.]

I'm still getting this error upgrading to the GA release.
Based on what I read in the User Guide this is somewhat expected since docs and images have been merged into a single table.
The User Guide also says to define an 'image.hook.impl' setting.
I did not have specified in 6.0.6 so it's not clear to me which setting I should use for 6.1.

Any help would be appreciated.

Thanks
-Trey

[Linus Sphinx, geändert vor 12 Jahren.]

Thinking if you copy the data folder from the old installation to under the new installation then the default hook value should work.

[Trey Howard, geändert vor 12 Jahren.]

I have tried all 3 settings and it's definitely defaulting to the FileSystemHook.
What's extra frustrating is that I starting working from a clean 6.0.6 install--default image gallery and only a couple of files in the document library.

I copied my portal-ext.properties and data directory from 6.0.6 to 6.1 and started server.

I have attached the full stack trace.
It would be greatly appreciated if anyone could help me debug this.
Here's my 6.1 portal-ext.props:
...jdbc stuff...
browser.launcher.url=
image.hook.impl=com.liferay.portal.image.FileSystemHook

Thanks
-Trey

[Trey Howard, geändert vor 12 Jahren.]

I wanted to follow up...
After re-installing a clean 6.0.6 and uploading some docs, migration worked.
Something must have gotten corrupted in my original tests.

The only other error I encountered was a known JVM bug: http://issues.liferay.com/browse/LPS-21520
That can be fixed by overriding the bundled JRE with a more recent version (see ticket).

-Trey

[MICHAIL MOUDATSOS, geändert vor 12 Jahren.]

Trey Howard:

After re-installing a clean 6.0.6 and uploading some docs, migration worked.

However, this is not a solution for an existing installation with existing users you don't want to loose, ctra, right?

I have stumbled upon various problems as well, such as images missing - as I have stated here but even when I found those folders and added them to the installation I got a
13:24:15,623 ERROR [MainServlet:198] com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portlet.documentlibrary.DuplicateFileException: C:\Liferay\liferay-portal-6.1.0-ce-ga1\data\document_library\10131\0\document_thumbnail\10156\181\10421\25807.png\1.0
com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portlet.documentlibrary.DuplicateFileException: C:\Liferay\liferay-portal-6.1.0-ce-ga1\data\document_library\10131\0\document_thumbnail\10156\181\10421\25807.png\1.0

which really resembles your error. They might be related (considering that you use MySQL and I use SQL Server)
I also tried the image.hook.impl=com.liferay.portal.image.FileSystemHook setting but still get the afore-mentioned error

Succeeding an upgrade over a play-toy 6.0.6 installation hasn't really any practical use, has it?

[Bin Li, geändert vor 12 Jahren.]

What is the ultimate solution?
When I upgrade from liferay-portal-6.0.6-ce to liferay-portal-6.1.0-ce-ga1, I also got similar issue with following exception:

16:10:42,736 INFO [UpgradeProcess:205] Upgrading com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery
16:10:43,984 ERROR [MainServlet:198] com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeExce
ption: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portlet.documentlibrary.DuplicateFileException: .../liferay-portal/data/document_library/11302/0/document_thumbnail/11324/253/56/145661/270653.jpg/1.0
com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgr
ade.UpgradeException: com.liferay.portlet.documentlibrary.DuplicateFileException: .../liferay-portal/data/docum
ent_library/11302/0/document_thumbnail/11324/253/56/145661/270653.jpg/1.0
at com.liferay.portal.events.StartupAction.run(StartupAction.java:58)
at com.liferay.portal.servlet.MainServlet.processStartupEvents(MainServlet.java:1267)
at com.liferay.portal.servlet.MainServlet.init(MainServlet.java:195)
at javax.servlet.GenericServlet.init(GenericServlet.java:160)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1266)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1185)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1080)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5001)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5289)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:866)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:842)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1581)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portl
et.documentlibrary.DuplicateFileException: .../liferay-portal/data/document_library/11302/0/document_thumbnail/
11324/253/56/145661/270653.jpg/1.0
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:211)
at com.liferay.portal.upgrade.UpgradeProcessUtil._upgradeProcess(UpgradeProcessUtil.java:93)
at com.liferay.portal.upgrade.UpgradeProcessUtil.upgradeProcess(UpgradeProcessUtil.java:46)
at com.liferay.portal.events.StartupHelper.upgradeProcess(StartupHelper.java:81)
at com.liferay.portal.events.StartupHelperUtil.upgradeProcess(StartupHelperUtil.java:40)
at com.liferay.portal.tools.DBUpgrader.upgrade(DBUpgrader.java:119)
at com.liferay.portal.events.StartupAction.doRun(StartupAction.java:144)
at com.liferay.portal.events.StartupAction.run(StartupAction.java:52)
... 20 more
Caused by: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portlet.documentlibrary.DuplicateFileException:
.../liferay-portal/data/document_library/11302/0/document_thumbnail/11324/253/56/145661/270653.jpg/1.0
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:211)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:225)
at com.liferay.portal.upgrade.UpgradeProcess_6_1_0.doUpgrade(UpgradeProcess_6_1_0.java:67)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:208)
... 27 more
Caused by: com.liferay.portlet.documentlibrary.DuplicateFileException: .../liferay-portal/data/document_library
/11302/0/document_thumbnail/11324/253/56/145661/270653.jpg/1.0
at com.liferay.portlet.documentlibrary.store.FileSystemStore.addFile(FileSystemStore.java:76)
at com.liferay.portlet.documentlibrary.store.BaseStore.addFile(BaseStore.java:105)
at com.liferay.portlet.documentlibrary.store.StoreProxyImpl.addFile(StoreProxyImpl.java:52)
at com.liferay.portlet.documentlibrary.store.SafeFileNameStoreWrapper.addFile(SafeFileNameStoreWrapper.java:70)
at com.liferay.portlet.documentlibrary.store.DLStoreImpl.addFile(DLStoreImpl.java:98)
at com.liferay.portlet.documentlibrary.store.DLStoreImpl.addFile(DLStoreImpl.java:171)
at com.liferay.portlet.documentlibrary.store.DLStoreUtil.addFile(DLStoreUtil.java:186)
at com.liferay.portlet.documentlibrary.util.DLPreviewableProcessor.addFileToStore(DLPreviewableProcessor.java:184)
at com.liferay.portlet.documentlibrary.util.ImageProcessorImpl._storeThumbnail(ImageProcessorImpl.java:269)
at com.liferay.portlet.documentlibrary.util.ImageProcessorImpl.storeThumbnail(ImageProcessorImpl.java:130)
at com.liferay.portlet.documentlibrary.util.ImageProcessorUtil.storeThumbnail(ImageProcessorUtil.java:87)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.migrateImage(UpgradeImageGallery.java:359)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.migrateImageFiles(UpgradeImageGallery.java:398)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.doUpgrade(UpgradeImageGallery.java:248)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:208)
... 30 more
Stopping the server due to unexpected startup errors

[Eike Salow, geändert vor 12 Jahren.]

Hi Bin Li,
is this your first exception while upgrading or did you have a fail try before?

regards

Eike

[MICHAIL MOUDATSOS, geändert vor 12 Jahren.]

First I had some file missing exceptions that abnormally ended installation (which in my opinion is unacceptable). When I finally found from older installations (back to 6.0.5!) the data/images/* files that were missing and added them to my new 6.1 installation, I re-initialized DB and then I got this error.

I have to admit that for the latter try I did not reinitialize the files of th liferay installation on disk, beacause I had already tried both ways (with and without initializing the files ON DISK) for the previous error (missing file) and it yielded the same error so I assumed that it shouldnt matter if I tried with same DISK installation files since tomcat ended so quickly just after it read the portal-ext.properties file and started initializing DB tables...

I don't know if this was a false assumptio, but it was taking so long to re-extract the files from the .zip all over again and all over again, every time I changed a setting in portal.extproperties (such as image.hook.impl)

So, now that you know, what do you think?

[Bin Li, geändert vor 12 Jahren.]

Yes, this is first exception while upgrading.
Actually, we use image.hook.impl=com.liferay.portal.image.DatabaseHook in portal-ext.properties.

We have also tried image.hook.impl=com.liferay.portal.image.FileSystemHook (restore 6.0.6 database first and then try), which will throw:
16:22:12,258 ERROR [MainServlet:198] com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeExce
ption: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portlet.documentlibrary.NoSuchFileException: .../liferay-portal/data/images/14/56/145662.jpg.

So seems using DatabaseHook should be the correct configuration. Though the image file should be stored in db by using DatabaseHook, and there is no file under folder .../liferay-portal/data/document_library/11302/0 or in tables image and igimage, don't know why it still complains:
com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeExce
ption: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portlet.documentlibrary.DuplicateFileException: .../liferay-portal/data/document_library/11302/0/document_thumbnail/11324/253/56/145661/270653.jpg/1.0

[MICHAIL MOUDATSOS, geändert vor 12 Jahren.]

I get the same message even with image.hook.impl=com.liferay.portal.image.FileSystemHook. (I also got the file missing exceptions you are stating but I found all files missing from data/image folder and I resolved this issue. Then I came across the exact issue you 're dealing with...)

Can anyone help with this? (I think however we should make a new post because the title of this one directs to beta version of 6.1 and ppl may not feel it worths reading it)

[Eike Salow, geändert vor 12 Jahren.]

Hi Bin,
did you copy also you data directory especially the images into the new liferay 6.1. data folder?

if yes, and you have still the same problem while upgading,

i think you might have follwing issue: your old liferay installation 6.06. has some dead imagegallery references inside the database.

- you can figure it out, when you start your old installation with liferay 6.06
- then you go to the document library of each community and organisation.
- if you found some pictures you wich are not displayed in the doc lib you have to delete those entries
inside the database check tables IGimage and Image:
you can get the image id by analysing the url of a non diplayed picture.

after you clean your db , you have to
copy the data lib again and start the upgrade process again.


regards
Eike

[Bin Li, geändert vor 12 Jahren.]

Sorry, finally I found that the DuplicateFileException is due to my second run upgrading (first run upgrade has a java.io.EOFException), though I restored the DB, the data folder under $LIFERAY_HOME was not restored for the second run. Sorry for the misleading.

Here is the exception when I first ran upgrade:
17:13:49,654 INFO [UpgradeProcess:205] Upgrading com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery
17:13:50,987 ERROR [Base64:250] java.io.EOFException
java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2281)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2750)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:780)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:280)
at com.liferay.portal.kernel.util.Base64._stringToObject(Base64.java:240)
at com.liferay.portal.kernel.util.Base64.stringToObject(Base64.java:192)
at com.liferay.portal.image.DatabaseHook.getImageAsBytes(DatabaseHook.java:32)
at com.liferay.portal.image.DatabaseHook.getImageAsStream(DatabaseHook.java:36)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.migrateImage(UpgradeImageGallery.java:357)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.migrateImageFiles(UpgradeImageGallery.java:398)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.doUpgrade(UpgradeImageGallery.java:248)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:208)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:225)
at com.liferay.portal.upgrade.UpgradeProcess_6_1_0.doUpgrade(UpgradeProcess_6_1_0.java:67)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:208)
at com.liferay.portal.upgrade.UpgradeProcessUtil._upgradeProcess(UpgradeProcessUtil.java:93)
at com.liferay.portal.upgrade.UpgradeProcessUtil.upgradeProcess(UpgradeProcessUtil.java:46)
at com.liferay.portal.events.StartupHelper.upgradeProcess(StartupHelper.java:81)
at com.liferay.portal.events.StartupHelperUtil.upgradeProcess(StartupHelperUtil.java:40)
at com.liferay.portal.tools.DBUpgrader.upgrade(DBUpgrader.java:119)
at com.liferay.portal.events.StartupAction.doRun(StartupAction.java:144)
at com.liferay.portal.events.StartupAction.run(StartupAction.java:52)
at com.liferay.portal.servlet.MainServlet.processStartupEvents(MainServlet.java:1267)
at com.liferay.portal.servlet.MainServlet.init(MainServlet.java:195)
at javax.servlet.GenericServlet.init(GenericServlet.java:160)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1266)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1185)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1080)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5001)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5289)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:866)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:842)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1581)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
17:13:50,994 ERROR [MainServlet:198] com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: java.lang.NullPointerException
com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: java.lang.NullPointerException
at com.liferay.portal.events.StartupAction.run(StartupAction.java:58)
at com.liferay.portal.servlet.MainServlet.processStartupEvents(MainServlet.java:1267)
at com.liferay.portal.servlet.MainServlet.init(MainServlet.java:195)
at javax.servlet.GenericServlet.init(GenericServlet.java:160)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1266)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1185)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1080)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5001)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5289)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:866)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:842)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1581)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: java.lang.NullPointerException
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:211)
at com.liferay.portal.upgrade.UpgradeProcessUtil._upgradeProcess(UpgradeProcessUtil.java:93)
at com.liferay.portal.upgrade.UpgradeProcessUtil.upgradeProcess(UpgradeProcessUtil.java:46)
at com.liferay.portal.events.StartupHelper.upgradeProcess(StartupHelper.java:81)
at com.liferay.portal.events.StartupHelperUtil.upgradeProcess(StartupHelperUtil.java:40)
at com.liferay.portal.tools.DBUpgrader.upgrade(DBUpgrader.java:119)
at com.liferay.portal.events.StartupAction.doRun(StartupAction.java:144)
at com.liferay.portal.events.StartupAction.run(StartupAction.java:52)
... 20 more
Caused by: com.liferay.portal.kernel.upgrade.UpgradeException: java.lang.NullPointerException
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:211)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:225)
at com.liferay.portal.upgrade.UpgradeProcess_6_1_0.doUpgrade(UpgradeProcess_6_1_0.java:67)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:208)
... 27 more
Caused by: java.lang.NullPointerException
at com.liferay.portal.kernel.io.unsync.UnsyncByteArrayInputStream.<init>(UnsyncByteArrayInputStream.java:31)
at com.liferay.portal.image.DatabaseHook.getImageAsStream(DatabaseHook.java:36)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.migrateImage(UpgradeImageGallery.java:357)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.migrateImageFiles(UpgradeImageGallery.java:398)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.doUpgrade(UpgradeImageGallery.java:248)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:208)
... 30 more
Stopping the server due to unexpected startup errors

[Bin Li, geändert vor 12 Jahren.]

Finally, we found the root cause:
It happens that there is a null content in table image.text_, there is an image file in file system other than in our DB. Perhap it is a legacy issue in our version upgrade previously.So find this file in image gallery and delete it, then do the upgrade. The error is gone.

Bin Li:

Sorry, finally I found that the DuplicateFileException is due to my second run upgrading (first run upgrade has a java.io.EOFException), though I restored the DB, the data folder under $LIFERAY_HOME was not restored for the second run. Sorry for the misleading.

Here is the exception when I first ran upgrade:
17:13:49,654 INFO [UpgradeProcess:205] Upgrading com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery
17:13:50,987 ERROR [Base64:250] java.io.EOFException
java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2281)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2750)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:780)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:280)
at com.liferay.portal.kernel.util.Base64._stringToObject(Base64.java:240)
at com.liferay.portal.kernel.util.Base64.stringToObject(Base64.java:192)
at com.liferay.portal.image.DatabaseHook.getImageAsBytes(DatabaseHook.java:32)
at com.liferay.portal.image.DatabaseHook.getImageAsStream(DatabaseHook.java:36)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.migrateImage(UpgradeImageGallery.java:357)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.migrateImageFiles(UpgradeImageGallery.java:398)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.doUpgrade(UpgradeImageGallery.java:248)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:208)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:225)
at com.liferay.portal.upgrade.UpgradeProcess_6_1_0.doUpgrade(UpgradeProcess_6_1_0.java:67)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:208)
at com.liferay.portal.upgrade.UpgradeProcessUtil._upgradeProcess(UpgradeProcessUtil.java:93)
at com.liferay.portal.upgrade.UpgradeProcessUtil.upgradeProcess(UpgradeProcessUtil.java:46)
at com.liferay.portal.events.StartupHelper.upgradeProcess(StartupHelper.java:81)
at com.liferay.portal.events.StartupHelperUtil.upgradeProcess(StartupHelperUtil.java:40)
at com.liferay.portal.tools.DBUpgrader.upgrade(DBUpgrader.java:119)
at com.liferay.portal.events.StartupAction.doRun(StartupAction.java:144)
at com.liferay.portal.events.StartupAction.run(StartupAction.java:52)
at com.liferay.portal.servlet.MainServlet.processStartupEvents(MainServlet.java:1267)
at com.liferay.portal.servlet.MainServlet.init(MainServlet.java:195)
at javax.servlet.GenericServlet.init(GenericServlet.java:160)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1266)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1185)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1080)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5001)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5289)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:866)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:842)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1581)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
17:13:50,994 ERROR [MainServlet:198] com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: java.lang.NullPointerException
com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: java.lang.NullPointerException
at com.liferay.portal.events.StartupAction.run(StartupAction.java:58)
at com.liferay.portal.servlet.MainServlet.processStartupEvents(MainServlet.java:1267)
at com.liferay.portal.servlet.MainServlet.init(MainServlet.java:195)
at javax.servlet.GenericServlet.init(GenericServlet.java:160)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1266)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1185)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1080)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5001)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5289)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:866)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:842)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1581)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: java.lang.NullPointerException
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:211)
at com.liferay.portal.upgrade.UpgradeProcessUtil._upgradeProcess(UpgradeProcessUtil.java:93)
at com.liferay.portal.upgrade.UpgradeProcessUtil.upgradeProcess(UpgradeProcessUtil.java:46)
at com.liferay.portal.events.StartupHelper.upgradeProcess(StartupHelper.java:81)
at com.liferay.portal.events.StartupHelperUtil.upgradeProcess(StartupHelperUtil.java:40)
at com.liferay.portal.tools.DBUpgrader.upgrade(DBUpgrader.java:119)
at com.liferay.portal.events.StartupAction.doRun(StartupAction.java:144)
at com.liferay.portal.events.StartupAction.run(StartupAction.java:52)
... 20 more
Caused by: com.liferay.portal.kernel.upgrade.UpgradeException: java.lang.NullPointerException
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:211)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:225)
at com.liferay.portal.upgrade.UpgradeProcess_6_1_0.doUpgrade(UpgradeProcess_6_1_0.java:67)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:208)
... 27 more
Caused by: java.lang.NullPointerException
at com.liferay.portal.kernel.io.unsync.UnsyncByteArrayInputStream.<init>(UnsyncByteArrayInputStream.java:31)
at com.liferay.portal.image.DatabaseHook.getImageAsStream(DatabaseHook.java:36)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.migrateImage(UpgradeImageGallery.java:357)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.migrateImageFiles(UpgradeImageGallery.java:398)
at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.doUpgrade(UpgradeImageGallery.java:248)
at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:208)
... 30 more
Stopping the server due to unexpected startup errors

[Bin Li, geändert vor 12 Jahren.]

A new issue comes:
Since we use permissions.user.check.algorithm=5 in previous version, it will show warning in the log when starts up:
06:20:38,891 WARN [MainServlet:901] Liferay is configured to use permission algorithm 5. Versions after 6.1 will only support algor
ithm 6 and above. Please sign in as an administrator, go to the Control Panel, select "Server Administration", select the "Data Migr
ation" tab, and convert from this legacy permission algorithm as soon as possible.

So we followed this operation (go to the Control Panel, select "Server Administration", select the "Data Migration" tab, and convert from this legacy permission algorithm), but it has following errors:
07:01:41,730 DEBUG [MaintenanceUtil:64] Updating ResourcePermission table
07:01:44,553 FATAL [ConvertProcessMessageListener:35] Unable to process message {destinationName=liferay/convert_process, response=n
ull, responseDestinationName=null, responseId=null, payload=com.liferay.portal.convert.ConvertPermissionAlgorithm, values={principal
Password=..., principalName=10129, companyId=10109}}
com.liferay.portal.convert.ConvertException: java.sql.BatchUpdateException: Duplicate entry '11302-100-4-11902_LAYOUT_100-11308' for
key 2
at com.liferay.portal.convert.ConvertProcess.convert(ConvertProcess.java:53)
at com.liferay.portal.convert.messaging.ConvertProcessMessageListener.doReceive(ConvertProcessMessageListener.java:47)
at com.liferay.portal.convert.messaging.ConvertProcessMessageListener.receive(ConvertProcessMessageListener.java:32)
at com.liferay.portal.kernel.messaging.InvokerMessageListener.receive(InvokerMessageListener.java:65)
at com.liferay.portal.kernel.messaging.SerialDestination$1.run(SerialDestination.java:101)
at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask._runTask(ThreadPoolExecutor.java:669)
at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask.run(ThreadPoolExecutor.java:580)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.sql.BatchUpdateException: Duplicate entry '11302-100-4-11902_LAYOUT_100-11308' for key 2
at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1257)
at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:943)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:1723)
at com.liferay.portal.upgrade.util.Table.populateTableRows(Table.java:538)
at com.liferay.portal.upgrade.util.Table.populateTable(Table.java:500)
at com.liferay.portal.upgrade.util.Table.populateTable(Table.java:443)
at com.liferay.portal.convert.ConvertPermissionAlgorithm.convertToBitwise(ConvertPermissionAlgorithm.java:176)
at com.liferay.portal.convert.ConvertPermissionAlgorithm.doConvert(ConvertPermissionAlgorithm.java:693)
at com.liferay.portal.convert.ConvertProcess.convert(ConvertProcess.java:44)
... 7 more

[Trey Howard, geändert vor 12 Jahren.]

@MICHAIL
You're right, what worked for me in local testing will not work for a real upgrade.
In fact, I have finished my local testing and am moving on to our staging installation and immediately hit this problem.
I'm about to try Eike's suggestion to prune the image gallery in 6.0 before upgrading to 6.1.
Has anyone else tried this? Any luck?

-Trey

[Trey Howard, geändert vor 12 Jahren.]

@Eike

My image gallery is empty in the admin GUI, but I saw files on the hard drive--is this the a corrupt gallery?
I deleted everything within the liferay/images directory and got the same error on startup--on the same file even!
This is really frustrating.
Any help is appreciated.

Even though this ticket is marked as closed, it matches almost exactly to the error I/we are seeing.

-Trey

Edit: The liferay/data/images directory contains 195 files files across all the sub-directories.
In the database, the 'igimage' table has 11 records and the 'image' table has 152 records.

[Trey Howard, geändert vor 12 Jahren.]

After some digging into the database I found the root of my problems.
There were 2 records in the igimage table with the same groupId, folderId, and name.
This is fine because the igimage table uses a non-unique index: KEY `IX_AAE8DF83` (`groupId`,`folderId`,`name`)

However, the dlfileentry table use a unique index: UNIQUE KEY `IX_ED5CA615` (`groupId`,`folderId`,`title`)
So when the upgrade moves the igimage records to dlfileentry it blows up with the exception:
com.mysql.jdbc.exceptions.MySQLIntegrityConstraintViolationException: Duplicate entry '10546-0-bengals' for key 'IX_ED5CA615'

I have manually de-duped in the 6.0 database by changing the name.
The UpgradeImageGallery step succeeds--currently testing that this change didn't break something else.

HTH
-Trey

[Linus Sphinx, geändert vor 12 Jahren.]

Sick cognos system has stopped me completely from getting back to this. What I was thinking last time I worked on it is that maybe you've got corrupt data ok, you're importing it and building a new set, just don't import the corruption. There is no exception handling in the upgrade for missing images like there is for documents, should not be fatal.

[Linus Sphinx, geändert vor 12 Jahren.]

Checking out and building 6.1.x from svn seems to be working for me, I see the upgrade is cleaning up for me now;

16:41:57,997 WARN [UpgradeImageGallery:357] Ignoring exception for image 852819
com.liferay.portlet.documentlibrary.NoSuchFileException: /usr/local/liferay-portal-6.1.1/data/images/85/28/852819.jpg

Bloody marvelous! Got an issue using the databases after it's upgraded, system comes up, no errors, starts listening but when I browse it I cannot reach the homepage, goes into a redirection loop returning 302's til it fails. Hostname is not the same in the DB, tried renaming my laptop to match but no cigar. Seems to be in the trunk also, I get the same behavior for 6.0.6 -> 6.1.1 and 6.0.6 -> 6.2.0

the only other warning I see in upgrading now besides the above is a single:

16:43:01,227 WARN [BaseDB:489] Specified key was too long; max key length is 767 bytes: create unique index IX_DD635956 on Lock_ (className, key_, owner);

have not looked into that, still trying to log in or see a home page and getting the below mess, browser says, "The page isn't redirecting properly Firefox has detected that the server is redirecting the request for this address in a way that will never complete. This problem can sometimes be caused by disabling or refusing to accept cookies.". Cleared cookies, cache, my calendar and my desk with no effect.

[root@ea-portal ~]# wget http://localhost
--2012-03-21 10:05:54-- http://localhost/
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://ea-portal.wate.com/;jsessionid=BAC8E4E97BF66D983A4D4962EAB60A93 [following]
--2012-03-21 10:05:54-- http://ea-portal.wate.com/;jsessionid=BAC8E4E97BF66D983A4D4962EAB60A93
Resolving ea-portal.wate.com... 13.20.20.22
Connecting to ea-portal.wate.com|13.20.20.22|:80... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://ea-portal.wate.com/;jsessionid=BAC8E4E97BF66D983A4D4962EAB60A93 [following]

... 18 more same message here

--2012-03-21 10:05:54-- http://ea-portal.wate.com/;jsessionid=BAC8E4E97BF66D983A4D4962EAB60A93
Connecting to ea-portal.wate.com|13.20.20.22|:80... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://ea-portal.wate.com/;jsessionid=BAC8E4E97BF66D983A4D4962EAB60A93 [following]
20 redirections exceeded.

I'll start another thread on this, does not look related to referential integrity.