Forums de discussion
Thumbnail generation workaround
Andrea Nicolini, modifié il y a 7 années.
Thumbnail generation workaround
New Member Publications: 16 Date d'inscription: 31/03/16 Publications récentes
Hi all,
I'd like to share a solution I adopted for a problem that I think is quite common among Liferay users.
Our site relies heavily on Liferay generated thumbnail and we often faced problems of misgenerated (eg: CMYK) or missing thumbnails, so I started to think about an alternative.
My first try was to write a webservice which, using liferay api, triggered thumbnail generation on demand, so that if you delete all server thumbnail you can regenerate them in one shot, with no need to open documents in Documents And Media.
It worked, but this approach was still not flexible enough and it put an heavy load on liferay instance while running; so I decided to follow another track.
I ended up writing a simple web service which is only responsible of returning the list of DL paths to source images and DL paths to thumbnail images. Then I wrote a simple bash script that uses these paths to generate thumbnails the way I want.
Here's my webservice:
And this is my bash script used for actual conversion:
With this simple solution I managed to work with thumbnails in a much more comfortable way and I hope this can be of some help to someone who is facing similar problems.
Bye.
I'd like to share a solution I adopted for a problem that I think is quite common among Liferay users.
Our site relies heavily on Liferay generated thumbnail and we often faced problems of misgenerated (eg: CMYK) or missing thumbnails, so I started to think about an alternative.
My first try was to write a webservice which, using liferay api, triggered thumbnail generation on demand, so that if you delete all server thumbnail you can regenerate them in one shot, with no need to open documents in Documents And Media.
It worked, but this approach was still not flexible enough and it put an heavy load on liferay instance while running; so I decided to follow another track.
I ended up writing a simple web service which is only responsible of returning the list of DL paths to source images and DL paths to thumbnail images. Then I wrote a simple bash script that uses these paths to generate thumbnails the way I want.
Here's my webservice:
public List<string> getThumbnailsPaths(int totDocs){
List<string> results = new ArrayList<string>();
List<string> extsToExt = new ArrayList<string>();
List<string> extsToPng = new ArrayList<string>();
List<dlfileentry> files = null;
DLFileVersion fileVer = null;
DLFolder fldr = null;
Date modDate = null;
String filePath = null;
String ext = "";
int j = 0;
//Valid extensions
extsToPng.add("eps");
extsToPng.add("pdf");
extsToPng.add("tif");
extsToExt.add("gif");
extsToExt.add("png");
extsToExt.add("jpg");
extsToExt.add("bmp");
try{
if(totDocs == 0)
totDocs = DLFileEntryLocalServiceUtil.getDLFileEntriesCount();
//fetch documents from DL
files = DLFileEntryLocalServiceUtil.getDLFileEntries(0,totDocs);
for(DLFileEntry file: files){
j++;
ext = file.getExtension();
if(!extsToExt.contains(ext) && !extsToPng.contains(ext))
continue;
StringBuffer outPath = new StringBuffer("/");
outPath.append(file.getCompanyId() + "/");
outPath.append(file.getFolderId() + "/");
outPath.append(file.getName() + "/");
outPath.append(file.getVersion() + "|");
fileVer = file.getFileVersion();
outPath.append(file.getGroupId());
outPath.append(DLUtil.getDividedPath(file.getFileEntryId()));
if (fileVer.getFileVersionId() > 0) {
outPath.append("/");
outPath.append(fileVer.getFileVersionId());
}
if(extsToPng.contains(ext)){
outPath.append(".png");
}else{
outPath.append("." + ext);
}
results.add(outPath.toString());
}
}catch(SystemException | PortalException e){
e.printStackTrace();
}
return results;
}
</dlfileentry></string></string></string></string></string></string></string>
And this is my bash script used for actual conversion:
#!/bin/sh
renice -n 10 $$
DLBASEPATH="/opt/liferay/data/document_library"
TNBASEPATH="$DLBASEPATH/10157/0/document_thumbnail"
THUMBSIZE="128x128"
THUMBSIZE1="240x310"
GREPATH="$1"
MAXDBAGE=3600
MAXCONV=100
CURLCMD="curl -s http://localhost:8080/api/jsonws/yourwshere/get-thumbnails-paths -u user:pass -d totDocs=0"
if [ -f "/tmp/thumbs.db.tmp" ]; then
CURSTAMP=$(date +'%s')
TDBSTAMP=$(stat -c %Y "/tmp/thumbs.db.tmp")
if [ $((CURSTAMP - TDBSTAMP)) -gt $MAXDBAGE ]; then
echo "Querying Liferay DB..."
$CURLCMD | jq -r '.[]' > /tmp/thumbs.db.tmp
fi
else
echo "Querying Liferay DB..."
#I'm using jq to parse json list of strins
$CURLCMD | jq -r '.[]' > /tmp/thumbs.db.tmp
fi
#Skip if source file doesn't match filter
if [ -n "$GREPATH" ]; then
echo "Active filter: $GREPATH"
grep "$GREPATH" /tmp/thumbs.db.tmp > /tmp/thumbs.db
else
cp /tmp/thumbs.db.tmp /tmp/thumbs.db
fi
while read LINE; do
if [ $MAXCONV -eq 0 ]; then
break
fi
IFS='|' read -r -a DOCPATHS <<< $LINE
SRCPATH="$DLBASEPATH${DOCPATHS[0]}"
TNPATH="$TNBASEPATH/${DOCPATHS[1]}"
TNPATH1=$(echo $TNPATH | sed 's/\(^[a-z0-9/_]\+\)\(\.[a-z]\+\)/\1-1\2/') #This is custom thumbnail path: add "-1" in dirname.
if [ -f $SRCPATH ]; then
mkdir -p $TNPATH
mkdir -p $TNPATH1
#Process only out-of-date files
if [ -f "$TNPATH/.tstamp" -a -f "$TNPATH/1.0" ]; then
SRCSTAMP=$(stat -c %Y "$SRCPATH")
TNTSTAMP=$(stat -c %Y "$TNPATH/1.0")
TSTSTAMP=$(stat -c %Y "$TNPATH/.tstamp")
if [ $TNTSTAMP -ge $TNTSTAMP -a $SRCSTAMP -le $TSTSTAMP ]; then
continue
fi
fi
echo -n "Processing: $SRCPATH "
DBLPATH=$(echo $TNPATH | sed 's/\(^[a-z0-9/_]\+\).[a-z]\+/\1\*/')
DBLCONT=$(ls -d $DBLPATH | wc -l)
if [ $DBLCONT -gt 2 ]; then
echo "Double dir error: $TNPATH"
echo $TNPATH >> /opt/liferay/data/doubledir.log
continue
fi
MIME=$(file -b --mime-type $SRCPATH)
case "$MIME" in
image/*)
echo "-> IMG:$TNPATH/1.0"
convert "$SRCPATH" -thumbnail $THUMBSIZE1 "png:$TNPATH1/1.0"
convert "$TNPATH1/1.0" -thumbnail $THUMBSIZE "$TNPATH/1.0"
touch "$TNPATH/.tstamp"
MAXCONV=$((MAXCONV-1))
;;
application/*)
echo "-> PDF:$TNPATH/1.0"
#Imagemagick hangs on some pdfs, so I generate png first using gs.
gs -q -sDEVICE=png16m -dNOPAUSE -dBATCH -dFirstPage=1 -dLastPage=1 -sOutputFile="/tmp/thumbnail.png" "$SRCPATH"
convert "/tmp/thumbnail.png" -thumbnail $THUMBSIZE1 "png:$TNPATH1/1.0"
convert "$TNPATH1/1.0" -thumbnail $THUMBSIZE "$TNPATH/1.0"
touch "$TNPATH/.tstamp"
MAXCONV=$((MAXCONV-1))
;;
*)
echo "-> unmanaged mime type: $MIME"
;;
esac
#else
# echo "Not found: $SRCPATH"
fi
done < /tmp/thumbs.db
With this simple solution I managed to work with thumbnails in a much more comfortable way and I hope this can be of some help to someone who is facing similar problems.
Bye.
David H Nebinger, modifié il y a 7 années.
RE: Thumbnail generation workaround
Liferay Legend Publications: 14914 Date d'inscription: 02/09/06 Publications récentes
Andrea, thanks for the contribution, but what version of Liferay are you having these issues with?
Have you enabled the external tools (open office, image magick, ghostscript and xuggler)? These tools, if enabled, will be used to create thumbnails in lieu of the built-in Liferay code.
I always recommend enabling these tools to offload the work out of Liferay and generate better, consistent results.
Have you enabled the external tools (open office, image magick, ghostscript and xuggler)? These tools, if enabled, will be used to create thumbnails in lieu of the built-in Liferay code.
I always recommend enabling these tools to offload the work out of Liferay and generate better, consistent results.
Andrea Nicolini, modifié il y a 7 années.
RE: Thumbnail generation workaround
New Member Publications: 16 Date d'inscription: 31/03/16 Publications récentes
David,
I'm using Liferay CE 6.2 GA6 and yes, I enabled Imagemagick but after looking at the code and at some bug report like this:
https://issues.liferay.com/browse/LRDOCS-1616, I came to the conclusion that it is not used the way one would expect.
Actually it seems to be used only for CYMK to RGB conversion, but I might be wrong.
Furthermore, using an external script I can run conversion whenever I want, and I can also generate thumbnail for files that aren't normally thumbnailed.
I know, this is a quick and dirty solution, but it meets my needs.
Thank you for your interest.
Bye.
I'm using Liferay CE 6.2 GA6 and yes, I enabled Imagemagick but after looking at the code and at some bug report like this:
https://issues.liferay.com/browse/LRDOCS-1616, I came to the conclusion that it is not used the way one would expect.
Actually it seems to be used only for CYMK to RGB conversion, but I might be wrong.
Furthermore, using an external script I can run conversion whenever I want, and I can also generate thumbnail for files that aren't normally thumbnailed.
I know, this is a quick and dirty solution, but it meets my needs.
Thank you for your interest.
Bye.