Document Library is temporarily unavailable

[ruchi sharma, modificado 12 Anos atrás.]

Hello
I am trying to upload a document in document library (folder abc) .
The strange thing is I am able to upload document(same document) in root folder and other existing folders as well.
If I create a new folder and try to upload the document in it I got the following error message and the exception.

-Using the classic uploader doesn't work and I get a message saying: "document library is currently unavailable".

-Using the new default uploader (the one that allows to select multiple files) I don't get any error message but the files are not uploaded.
got the following exception
09:59:15,728 ERROR [jsp:154] com.liferay.portal.kernel.exception.SystemException: java.io.FileNotFoundException: /data/liferay-portal-cdpulse/liferay60/data/document_library/1/13594470/128760/1.0 (No such file or directory)
at com.liferay.documentlibrary.util.FileSystemHook.addFile(FileSystemHook.java:109)
at com.liferay.documentlibrary.util.HookProxyImpl.addFile(HookProxyImpl.java:74)
at com.liferay.documentlibrary.util.SafeFileNameHookWrapper.addFile(SafeFileNameHookWrapper.java:100)
at com.liferay.documentlibrary.service.impl.DLLocalServiceImpl.addFile(DLLocalServiceImpl.java:71)
at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:75)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:58)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:58)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:58)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:58)
at com.liferay.portlet.documentlibrary.service.impl.DLFileEntryLocalServiceImpl.addFileEntry(DLFileEntryLocalServiceImpl.java:228)
at com.liferay.portlet.documentlibrary.service.impl.DLFileEntryLocalServiceImpl.addFileEntry(DLFileEntryLocalServiceImpl.java:119)
at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:75)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:58)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:58)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:58)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:58)
at com.liferay.portlet.documentlibrary.service.impl.DLFileEntryServiceImpl.addFileEntry(DLFileEntryServiceImpl.java:70)
at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:75)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:58)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:58)

Any help would be really appreciated.
Thanks
Ruchi

[Joaquin Cabal, modificado 12 Anos atrás.]

Did you have any error when you created the "abc" folder? Maybe yoy can delete the folder and create it again

[ruchi sharma, modificado 12 Anos atrás.]

Hello Joaquin
No I din't get any error while creating the folder abc.
Yes I have tried to delete it and creating it back again.
And I have also tried to create a new folder, still I am not able to upload any document in it. But I am able to upload doc in other folders and root folders.
Thanks very much.
Ruchi

[Hitoshi Ozawa, modificado 12 Anos atrás.]

Did you create subfolders from the Control Panel -> Document and Media Library logged in as an Admin and try uploading file from there.

[ruchi sharma, modificado 12 Anos atrás.]

Hello Hitoshi,
Yes, i am logged in as admin. And i have created subfolder from
Control panel >Document Library > abc (and then i was trying to upload document)
Just to let you know I am using Liferay 6.
Thanks
Ruchi

[Hitoshi Ozawa, modificado 12 Anos atrás.]

Are you using Liferay 6.1.0 GA1? I haven't encountered this problem on Windows, Ubuntu, CentOS with IE, Firefox, Chrome.

If this is just a test system, try deleting your Liferay database and recreating it over again.

[ruchi sharma, modificado 12 Anos atrás.]

Hello HItoshi,
I am using Liferay Portal Enterprise Edition 6.0.10 and I am working in windows with fiefox (i have tested on diff browsers as well like ie and chrome i am having the same problem).
And its the live system .. so cant delete the db.
And looking at the stack trace it seems that is not finding the path to the folder where the documents need to be uploaded to.
Thanks
Ruchi

[Hitoshi Ozawa, modificado 12 Anos atrás.]

Just to make sure I understand your situation. If you create a new folder with a different name, you're able to upload files to it?
It's just to the folder with name "abc" that's giving you a problem?

And looking at the stack trace it seems that is not finding the path to the folder where the documents need to be uploaded to.

I usually try reindexing first and erasing all the caches. If this doesn't help, it means the database got corrupted. A simple way is to start over but that won't be an option for you.
It sometimes worked when I used WebDAV to delete the folder.

Off topic, but I'm also currently working on fixing a live system with a corrupted database.

[ruchi sharma, modificado 12 Anos atrás.]

No, I am not able to upload document in any new created folder / subfolder.
And I have tried to clear the cache from control panel >server > server administration > actions (i have done following actions)
Run the garbage collector to free up memory.
Clear content cached by this VM.
Clear content cached across the cluster.
Clear the database cache.
not reindexing.
and after that again tried to upload document but this also doesn't work.
Hope its clear to you now.

Many Thanks
Ruchi

[Hitoshi Ozawa, modificado 12 Anos atrás.]

Does the following folder exist?

/data/liferay-portal-cdpulse/liferay60/data/document_library/1/13594470/128760/1.0

[ruchi sharma, modificado 12 Anos atrás.]

Hitoshi Ozawa:

Does the following folder exist?

/data/liferay-portal-cdpulse/liferay60/data/document_library/1/13594470/128760/1.0

Hello Hitoshi,
No the mentioned folder doesn't exist.
I am able to navigate till "data/liferay-portal-cdpulse/liferay60/data/document_library/1/" folder (and after that no such file or directory)

Many Thanks
Ruchi

[Hitoshi Ozawa, modificado 12 Anos atrás.]

Sorry, ruchi, but your database table containing document library information seems to be corrupt. It's not just the table containing the document library structure and file information, but also the permission table to resources. This isn't very simple to fix. I think you'll need someone to be on-site to fix it for you.

[ruchi sharma, modificado 12 Anos atrás.]

Hello Hitoshi,
Thanks very much for getting back to me.
Well I'll discuss this problem with my collegues now ... it really sounds worst.
It would be great, if you could let me know which database tables are involved in storing information for document library structure.
Regards
Ruchi

[Sandeep Nair, modificado 12 Anos atrás.]

From what i see I feel that the folder information is successfully saved in database, but the folder is not created in Linux filesystem. This can very well be permission issue in Linux file system.

Regards,
Sandeep

[Hitoshi Ozawa, modificado 12 Anos atrás.]

Yes , Sandeep, The database and file system is out of sync. Do you know of a simple way to check the integrity and get everything back to sync?

I just had one of my user with a similar condition but with user table. Unfortunately, Liferay doesn't seem to update tables using transaction.
Thus, file information may have been updated in the database, but resource permission table may not have been updated when there is an I/O error.

[Sandeep Nair, modificado 12 Anos atrás.]

Yes exactly. But for now since it is a prod issue, one thing that can be quickly done is check whether there is a write permission in folder(linux filesystem). And then create a new folder again and see if the folder is getting added in filesystem.

Regards,
Sandeep

[Hitoshi Ozawa, modificado 12 Anos atrás.]

But for now since it is a prod issue, one thing that can be quickly done is check whether there is a write permission in folder(linux filesystem).

You've given me a clue. ruchi, have you changed the way Liferay is started? Such as changing user who starts it (e.g. changed it to start as a service instead of manually)?

[ruchi sharma, modificado 12 Anos atrás.]

Hitoshi Ozawa:

But for now since it is a prod issue, one thing that can be quickly done is check whether there is a write permission in folder(linux filesystem).

You've given me a clue. ruchi, have you changed the way Liferay is started? Such as changing user who starts it (e.g. changed it to start as a service instead of manually)?

Hi Hitoshi,
I din't get you completely but as far as I know I am doing it manually (with startup.sh).
Hope this make sense.
Thanks
RUchi

[Hitoshi Ozawa, modificado 12 Anos atrás.]

I din't get you completely but as far as I know I am doing it manually (with startup.sh).

Did you by any chance change the user which you use to login to startup liferay? Something changed
and it's probably not a very good idea to "experiment" on your production server till you find the cause.

I usually back up the system and recover it on a development server to experiment with.

[ruchi sharma, modificado 12 Anos atrás.]

Hello Hitoshi,
No we do not change any thing.
Actually I found the reason why this problem occurred and it is mentioned below . Please have a look.
Many thanks
Ruchi

[Sandeep Nair, modificado 12 Anos atrás.]

Thats a good catch. So basically this is an OS file system issue. This is to be handled and taken care of by some of your System operations guys, as to whatever is necessary for them to increase the number.

http://superuser.com/questions/298420/cannot-mkdir-too-many-links

REgards,
Sandeep

[ruchi sharma, modificado 12 Anos atrás.]

Thanks very much sandeep ...

[Sandeep Nair, modificado 12 Anos atrás.]

What i would suggest you to do is try creating a folder manually using the user you now use to startup liferay at exactly the same place where liferay is trying to create a folder. You will immediately come to know if you have proper privileges to create folder and files.

regards,
Sandeep

[ruchi sharma, modificado 12 Anos atrás.]

Hello Sandeep,
I have tried to create the folder where liferay creates the folder (when new folder is created)
I got this error :
mkdir: cannot create directory `12500312': Too many links
This means the limit of subfolder/folder is exceeded with in the directory in file system.
Thats why when I am creating a new folder I am not able to upload any document, else I am able to do so.

Question
As we know liferay by default creates folder within this directory
"C:\liferay-portal-tomcat-6.0-ee\liferay-portal-6.0-ee\data\document_library\1"
when we add any new folder in document library.

Is it possible (to resolve the uploading) that liferay automatiocally creates new folder(say 2), when directoy/folder 1 exceeds the limit ??
Is this kind of thing is possible to configure??
OR
Is their any other approach to solve this uploading problem??

Many Thanks
Ruchi

[Hitoshi Ozawa, modificado 12 Anos atrás.]

Well, you've found the cause of the problem but really haven't solved the problem yet because you're Liferay database would still be no in sync with your file system. You may be able to solve this error by manually creating the directory and deleting it and creating it again.

That said, I would never have dreamt of creating 32,000 sub-directories. Learned something new today

[ruchi sharma, modificado 12 Anos atrás.]

Hitoshi Ozawa:

Well, you've found the cause of the problem but really haven't solved the problem yet because you're Liferay database would still be no in sync with your file system. You may be able to solve this error by manually creating the directory and deleting it and creating it again.

That said, I would never have dreamt of creating 32,000 sub-directories. Learned something new today

Hello Hitoshi,
Its true i got the cause for the problem, but dint get the solution.
But I tried to solve in the following way
FIRST,
Well, I change the property "dl.hook.file.system.root.dir" so that the documents uploaded will b sored at other new location and is working fine.
But I am not able to access the old documents stored at old location. (Is this possible or not)
I have mentioned this problem in the following thread
http://www.liferay.com/community/forums/-/message_boards/message/13298273
SECOND,
If first doest not work out , I was thinking of changing the file system form ext3 to ext4.

Do you/anybody have any suggestions about this ??
Thanks
Ruchi

[ruchi sharma, modificado 12 Anos atrás.]

Sandeep Nair:

From what i see I feel that the folder information is successfully saved in database, but the folder is not created in Linux filesystem. This can very well be permission issue in Linux file system.

Regards,
Sandeep

Hello Sandeep you are correct , the folder information is successfully saved in database.(in dlfolder table).
I have also tried to check the permission to the following folder
:/data/cdpulse/liferay60/data/document_library# ls -l
total 740
drwxr-xr-x 32000 root root 753664 2012-01-18 11:36 1

Many Thanks
Ruchi