Upload problem with large file on document library

[Alessandro Perazzetta，修改在14 年前。]

Hi all,

I'm having some troubles with document library upload process with very large files around 500MB + on LP5.2,
the Portal issues a OutOfMemoryException with regards to Heap Space.

these my settings on portal-ext.properties:

# Increase Document Library upload filesize
# 0 Unlimited
#dl.file.max.size=0
dl.file.max.size=1048576000

##
## Upload Servlet Request
##

#
# Set the maximum file size. Default is 1024 * 1024 * 100.
#
#com.liferay.portal.upload.UploadServletRequestImpl.max.size=1048576000
com.liferay.portal.upload.UploadServletRequestImpl.max.size=800000000

#
# Set the temp directory for uploaded files.
#
#com.liferay.portal.upload.UploadServletRequestImpl.temp.dir=C:/Temp
com.liferay.portal.upload.UploadServletRequestImpl.temp.dir=/tmp

#
# Set the threshold size to prevent extraneous serialization of uploaded
# data.
#
com.liferay.portal.upload.LiferayFileItem.threshold.size=262144

#
# Set the threshold size to prevent out of memory exceptions caused by
# caching excessively large uploaded data. Default is 1024 * 1024 * 10.
#
com.liferay.portal.upload.LiferayInputStream.threshold.size=10485760



#
# Use our own StringMaker to extend StringBuffer and ByteArrayMaker to
# extend ByteArrayOutputStream. This will add analytical tracking of our
# char and byte usage so we can better optimize memory by minimizing
# unnecessary garbage collecting that comes from unspecified array
# growth.
#
com.liferay.portal.kernel.util.ByteArrayMaker.initial.size=8000
com.liferay.portal.kernel.util.StringMaker.initial.size=128


and my JVM settings:
<jvm-options>-Xms2048m</jvm-options>
<jvm-options>-Xmx2650m</jvm-options>
<jvm-options>-XX:MaxPermSize=128m</jvm-options>

but none of these changes make the upload working for a file larger than 500M

referring to this issue http://issues.liferay.com/browse/LEP-4277 seems that there's a ByteArrayMaker that shadow the upload for cache and even if i setup the threshold i ran however into this problem (OutOfMemoryError):

[#|2009-06-18T12:17:59.343+0200|SEVERE|sun-appserver2.1|javax.enterprise.system.container.web|_ThreadID=19;_ThreadName=httpSSLWorkerThread-8080-1;_RequestID=14027152-c9f3-47ef-b3c5-70ab6099b80e;|ApplicationDispatcher[] PWC1231: Servlet.service() for servlet Main Servlet threw exception
java.lang.OutOfMemoryError
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:247)
at com.liferay.portal.util.FileImpl.write(FileImpl.java:753)
at com.liferay.portal.util.FileImpl.write(FileImpl.java:763)
at com.liferay.portal.kernel.util.FileUtil.write(FileUtil.java:284)
at com.liferay.documentlibrary.util.FileSystemHook.addFile(FileSystemHook.java:94)
at com.liferay.documentlibrary.service.impl.DLLocalServiceImpl.addFile(DLLocalServiceImpl.java:73)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy227.addFile(Unknown Source)
at com.liferay.portlet.documentlibrary.service.impl.DLFileEntryLocalServiceImpl.addFileEntry(DLFileEntryLocalServiceImpl.java:278)
at com.liferay.portlet.documentlibrary.service.impl.DLFileEntryLocalServiceImpl.addFileEntry(DLFileEntryLocalServiceImpl.java:173)
at com.liferay.portlet.documentlibrary.service.impl.DLFileEntryLocalServiceImpl.addFileEntry(DLFileEntryLocalServiceImpl.java:135)

[Tarun S. Kayasth，修改在14 年前。]

Hi Alessandro,

Did you get any solution for your problem?

Thanks,

Tarun Kayasth

[Lisa Simpson，修改在14 年前。]

Try raising the memory parameters on your Java options.

[Alessandro Perazzetta，修改在14 年前。]

Just tried without positive effects, also exceeding params could trap on some memory limit problems

[Gerimint Allat，修改在14 年前。]

Lisa Simpson:

Try raising the memory parameters on your Java options.

I can confirm on that. I set the maximum uploadable file size to 100MB (dl.file.max.size=104857600 and com.liferay.portal.upload.UploadServletRequestImpl.max.size=104857600, nothing else) in the properties file and uploading didn't work until I doubled all memory-related JVM parameters I am aware of:
-... (other JVM options)
-XX:MaxPermSize=1024M
and
"Initial memory pool" to 256 MB
"Maximum memory pool" to 512 MB

I have no idea which parameter did the trick and how it relates exactly to the uploading problem (if you want to upload even larger files), but it seems that twiddling with memory settings might provide a quick solution - until you find what the real problem is, at least.

[Alessandro Perazzetta，修改在14 年前。]

No solution yet

[Himanshu Bhargav，修改在14 年前。]

Hi ,

Could you please help me out to setup the Document Library plugin to set the maximum file size. Currently we can upload upto 100 Mb only. Please provide the steps how to upload upto 500 MB.

Looking for your response.


Many Thanks in advance.

Himanshu Bhargav