Upload file size limit

[Ryan Avery，修改在10 年前。]

Hello,

I've searched all over Google and these forums to get this working right. I have Liferay 6.1.1 CE GA2 installed on Windows Server 2008 using PostgreSQL as the database backend, and I am encountering an issue with file size limits. We first had issues uploading Microstation drawing files (.dgn) but I found I could change allowable extensions under Control Panel -> Server Administration -> File Uploads. Adding the .dgn extension allowed us to upload small .dgn files. Here is where it gets interesting. If I try to upload a 1.2MB dgn file, I get an error saying the file is too large. Fine, so I set the Documents and Media Maximum file size to match the General Overall Maximum File Size of 104857600, or 100MB. This did not work, I still could not upload the 1.2MB dgn file. However, I have been able to upload 15MB pdf files, so clearly the file size is not the only problem.

When I reviewed the portal properties under server administration, I found that the dl.file.max.size variable was still set to 307200, even though this should have been set by the Documents and Media Maximum File Size. Assuming this part of the Server Administration interface was not working properly, I decided to set it manually by creating portal-ext.properties in my LIFERAY HOME folder and setting:

dl.file.max.size=0

I restarted the Liferay service and checked the portal properties. Indeed, now the dl.file.max.size variable was set to 0 (which I understand to mean unlimited up to the general file size setting set by com.liferay.portal.upload.UploadServletRequestImpl.max.size). This still did not allow me to upload the 1.2MB file, so I decided to explicitly match the general maximum by setting:

dl.file.max.size=104857600

I restarted Liferay once again and I *still* was unable to upload a 1.2 MB dgn file. However, smaller dgn files upload fine, so it's not a file extension problem, and larger files of other types upload fine, so it's not necessarily the file size that's the problem either.

My questions:

• Why doesn't the Documents and Media Maximum file size properly set dl.file.max.size?
• What other settings could affect my ability to upload a 1.2MB dgn file?

Please let me know if there is any other information I can provide to help diagnose this issue. In all cases, the log file has been silent, because no stack errors are occurring - it just blocks the upload and says the file is too large when it certainly is not. Thanks for your insights.

[Péter Borkuti，修改在10 年前。]

Hi Ryan,

Unfortunately I could not reproduce your issue.

I set up liferay 6.1 ce ga2 bundle
with portal-ext.properties here:

dl.file.max.size=30072000
dl.file.extensions=*
com.liferay.portal.upload.UploadServletRequestImpl.max.size=104857600
com.liferay.portal.upload.LiferayFileItem.threshold.size=262144
com.liferay.portal.upload.LiferayInputStream.threshold.size=10485760
journal.image.small.max.size=51200
journal.image.extensions=.gif,.jpeg,.jpg,.png

(I set manually only

com.liferay.portal.upload.UploadServletRequestImpl.max.size
dl.file.max.size=30072000

the other properties are from portal.properties)

I uploaded this picture on Control Panel, Document and Media
http://www.kenrockwell.com/trips/2009-10/images/L1004220.DNG (18 Megabyte).

There was no thumbnail no preview, but I could download it.
It downloaded with the name of "test.dng.tiff". The file size was OK, and IrfanView showed the picture well.

[Ryan Avery，修改在10 年前。]

Hi Peter,

You said you uploaded a .dng file, but my problems are with .dgn, note the g is before the n. These are CAD files from the Microstation computer aided design program. I am able to upload other large files, just not DGN files. Can you try again with a dgn?

Thanks,
Ryan

[Wilson Man，修改在10 年前。]

hey Ryan, that's an odd issue indeed. Were there any errors in your app-server logs?

[Ryan Avery，修改在10 年前。]

Hi Wilson,

No, no error messages at all in the Liferay or Tomcat logs about this, since something is causing Liferay to think that the file is too large when it is not. Actually, I just completed some additional troubleshooting and found the following:

I have been able to upload on DGN file to my Liferay Portal. All other uploads, regardless of file size, have failed. I opened the contents of these GDN files in a text editor (Vim) and found that all the DGN files that do not work start with the following two characters:

ÐÏ

The file I uploaded that worked did not start with these two characters. Furthermore, by editing the file and removing the first character (the 'Ð'), the file was then able to be uploaded to Liferay! Obviously, this is not a solution for me to ask everyone to remove this character, and it may damage the DGN file. However, this should give your engineers a good place to look - something about the 'Ð' character at the beginning of the file is causing some error which is not logged and results in Liferay thinking that the file is too large.

I heard that this may be a known issue in Liferay 6.1, but I haven't been able to locate a ticket noting this. If somebody clues me in to this ticket, I'm happy to attach this troubleshooting information to help find a solution more quickly since it's clearly how the first character of the file stream is handled that is causing the problem.

Thanks!
Ryan

[Wilson Man，修改在10 年前。]

I did a brief search and couldn't find a similar issue on JIRA.
I would encourage you to open a new bug ticket here: https://issues.liferay.com/browse/LPS

If you have a small sample file that we can use as well, that would be helpful!
thanks,
Wilson

[Ryan Avery，修改在10 年前。]

Thanks Wilson. I added an issue:
https://issues.liferay.com/browse/LPS-38493

Hopefully this helps in finding a solution!

Ryan

[Wilson Man，修改在10 年前。]

hey Ryan, did a bit digging and found that it's a bug with the version of Tika we use. If you upgrade your instance of Liferay to use Tika 1.4 and Apache POI 3.9, the upload issue goes away. However, I'm not sure if those newer versions would introduce other problems since we have not tested them. If you want to try it, go ahead and download those newer versions.

In your tomcat/webapps/ROOT/WEB-INF/lib,
remove all JARs starting with tika (2 of them)
remove all JARS starting with poi (4-5 of them)

put the tika 1.4 jar in back into that dir (1 file only)
put the apache POI jars (except the example one) into that dir (4-5 of them)

restart Tomcat.

[Ryan Avery，修改在10 年前。]

Hi Wilson,

This is great - I'll discuss it with the team today and see if they mind going a bit experimental. Thanks much for the support, and I'll be sure to provide updates if it works for us, as well as whether or not we encounter any other issues.

Ryan

[Ryan Avery，修改在10 年前。]

I downloaded Tika 1.4 and POI 3.9 and replaced the jar files in WEB-INF/lib and restarted our Liferay 6.1.1 CE installation. So far, everything works fine. I was able to upload the dgn file and things appear to work fine. We will continue to regression test and let you know if anything pops up here.

A few final questions - when Liferay 6.2 CE is released, does it already include these updated libraries? Just curious for our upgrade path when that version is released. Also, I'm assuming these tika and POI libraries should only affect the Documents and Media library, or do other Liferay components use these libraries?

Again, thanks so much for your help Wilson, this is why I love open source!

Ryan

[Wilson Man，修改在10 年前。]

I'm not sure which version 6.2 CE will have, but looks like it will at least have Tika 1.3 and POI 3.7 (possibly something more recent).
It shouldn't affect your upgrade. However, you can repeat the same drill if you need to.

and AFAIK, you're right that the documents and media library is the only component that uses these libs. If i recall anything else, i'll let you know.

[Ryan Avery，修改在10 年前。]

Good to know. Thanks again Wilson!

[Alexis Araya，修改在7 年前。]

Ryan Avery:

I downloaded Tika 1.4 and POI 3.9 and replaced the jar files in WEB-INF/lib and restarted our Liferay 6.1.1 CE installation. So far, everything works fine. I was able to upload the dgn file and things appear to work fine. We will continue to regression test and let you know if anything pops up here.

A few final questions - when Liferay 6.2 CE is released, does it already include these updated libraries? Just curious for our upgrade path when that version is released. Also, I'm assuming these tika and POI libraries should only affect the Documents and Media library, or do other Liferay components use these libraries?

Again, thanks so much for your help Wilson, this is why I love open source!

Ryan

Hi, try replacing the libs, but I still have the same error, utd has done something more?
I have Liferay 6.1.2 ce GA3

Remove :

poi-ooxml-schemas.jar
poi-ooxml.jar
poi-scratchpad.jar
poi.jar
tika-core.jar
tika-parsers.jar

Add :
tika-app-1.4.jar
poi-3.9-20121203.jar
poi-ooxml-3.9-20121203.jar
poi-ooxml-schemas-3.9-20121203.jar
poi-scratchpad-3.9-20121203.jar