Fórum

Broken Control Panel and Performance after upgrade to Liferay 6.2

Alexander Lukashov, modificado 7 Anos atrás.

Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
Hi!

I have upgraded from Lifereay 6.1. to Liferay 6.2. and now I have got follow issues:

1. Control Panel Issue.
Control Panel looks like totally broken. It seems that system couldn't load proper CSS for it.

2. CKEditor Issue
When I open Control Panel, I keep getting error that CKEDITOR is not defined.

3. Performance Issues. Pages are loading very slow if compare to Liferay 6.1. I keep getting 503 error and had to restart Liferay several times per day. JVM and Catalina Args have been copied from the Liferay 6.1. Machine has the same memory capabilities, as those, where I have Liferay 6.1. running.

Thanks in advance, waiting for your advices.
thumbnail
Juan Gonzalez, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Liferay Legend Postagens: 3089 Data de Entrada: 28/10/08 Postagens Recentes
Can you review server logs and paste here errors, if any?

About performance, it's true that Liferay 6.2 requires a bit more of JVM heap comparing to 6.1. Which settings of -Xmx and XX:MaxPermSize do you have now?

Thanks.
Alexander Lukashov, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
Here are CATALINA_OPTS from the setenv.sh

CATALINA_OPTS="$CATALINA_OPTS -Dfile.encoding=UTF8 -Dorg.apache.catalina.loader.WebappClassLoader.ENABLE_CLEAR_REFERENCES=false -Duser.timezone=GMT"
CATALINA_OPTS="$CATALINA_OPTS -Duser.language=en -Duser.region=US"
CATALINA_OPTS="$CATALINA_OPTS -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XXemoticonarallelGCThreads=4 -XX:+CMSCompactWhenClearAllSoftRefs -XX:CMSInitiatingOccupancyFraction=85 -XX:+CMSScavengeBeforeRemark -XX:+CMSConcurrentMTEnabled -XXemoticonarallelCMSThreads=2"
CATALINA_OPTS="$CATALINA_OPTS -server -XX:NewSize=700m -XX:MaxNewSize=700m -Xms2048m -Xmx2048m -XX:MaxPermSize=784m -XX:SurvivorRatio=6 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=15"

About the system logs. Here is one of the errors, which I have mentioned:

java.lang.IllegalStateException: this writer hit an OutOfMemoryError; cannot commit
at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3399)
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3524)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3506)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3490)
at com.liferay.portal.search.lucene.IndexAccessorImpl._doCommit(IndexAccessorImpl.java:411)
at com.liferay.portal.search.lucene.IndexAccessorImpl._commit(IndexAccessorImpl.java:367)
at com.liferay.portal.search.lucene.IndexAccessorImpl._write(IndexAccessorImpl.java:560)
at com.liferay.portal.search.lucene.IndexAccessorImpl.updateDocument(IndexAccessorImpl.java:324)
at com.liferay.portal.search.lucene.LuceneHelperImpl.updateDocument(LuceneHelperImpl.java:793)
at com.liferay.portal.search.lucene.LuceneHelperUtil.updateDocument(LuceneHelperUtil.java:428)
at com.liferay.portal.search.lucene.LuceneIndexWriter.updateDocument(LuceneIndexWriter.java:118)
at com.liferay.portal.search.lucene.LuceneIndexWriter.updateDocuments(LuceneIndexWriter.java:138)
at sun.reflect.GeneratedMethodAccessor982.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.liferay.portal.kernel.messaging.proxy.ProxyRequest.execute(ProxyRequest.java:85)
at com.liferay.portal.kernel.messaging.proxy.ProxyMessageListener.receive(ProxyMessageListener.java:51)
at com.liferay.portal.kernel.messaging.InvokerMessageListener.receive(InvokerMessageListener.java:72)
at com.liferay.portal.kernel.messaging.ParallelDestination$1.run(ParallelDestination.java:71)
at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask._runTask(ThreadPoolExecutor.java:682)
at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask.run(ThreadPoolExecutor.java:593)

And in most cases it causes to Java Heap Space.
thumbnail
Olaf Kock, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Liferay Legend Postagens: 6396 Data de Entrada: 23/09/08 Postagens Recentes
In that case you'll need to raise your memory that's available to the JVM - see Juan's comment. I recommend to set the minimum and maximum to the same value (as you already did). Try raising them to "-Xms3072m -Xmx3072m" - if you still have memory problems, you might have a memory leak somewhere. Performance tuning results for 6.1 aren't necessarily valid for 6.2 as you can see.

You might also check if there are unused plugins that are deployed - if you undeploy them, you're saving some memory.

And make sure that the server has enough RAM to not run into the use of virtual memory, which would kill your performance again.
Alexander Lukashov, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
Thanks for the answer. I have set up provided values, will check further performance.

And what about broken Control Panel? Which direction should I look to find out the resolution?
thumbnail
Juan Gonzalez, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Liferay Legend Postagens: 3089 Data de Entrada: 28/10/08 Postagens Recentes
And what about broken Control Panel? Which direction should I look to find out the resolution?


This can be caused by multiple issues. For example, those OOM could have caused the css files to not have being minified properly.

Which app server are you using?
Alexander Lukashov, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
I'm using tomcat-7.0.62.
thumbnail
Juan Gonzalez, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Liferay Legend Postagens: 3089 Data de Entrada: 28/10/08 Postagens Recentes
Ok, can you try:

* Shutdown server.
* Apply those improvements Olaf said in JVM.
* Remove <TOMCAT>/temp files and directories
* Remove <TOMCAT>/work files and directories
* Startup server.

Attach here the startup server logs so we can see if there are something suspicious.

Thanks.
Alexander Lukashov, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
Here are logs attached.
thumbnail
Juan Gonzalez, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Liferay Legend Postagens: 3089 Data de Entrada: 28/10/08 Postagens Recentes
I didn't see anything "too" bad that explains the control panel problem you're having now.

Does it work right now?

I saw you have some custom themes. If you have some custom one for control panel, then that would be a good bug candidate.

Another thing you should check for sure is the S3 amazon config, as without that you can't use Documents & Media.
Alexander Lukashov, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
No, it's doesn't work after restart.

These customs themes are for site, not for Control Panel. Control Panel have default control panel theme. I have checked that in previous version.

Errors with S3 appeared for Liferay 6.1. as well, it's a staging environment, currently we are testing the possibility of upgrade from Liferay 6.1
thumbnail
Juan Gonzalez, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Liferay Legend Postagens: 3089 Data de Entrada: 28/10/08 Postagens Recentes
I guess you cleaned browser cache after upgrade to 6.2 right?

Can you open browser network console and paste here any error (404 etc) when accessing Control Panel?

Additionally, are you accessing Liferay from an external host (non-localhost), right?

Thanks.
Alexander Lukashov, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
After clearing cache I have got 500 and then 503 error with description that server temporarily is not available now. But there was no any log entry in the log describing the reason.

After all I have got follow error:

15:28:50,135 WARN [C3P0PooledConnectionPoolManager[identityToken->j8wD3ocx]-AdminTaskTimer][ThreadPoolAsynchronousRunner:275] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6b9da4ab -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!


Yes, I'm accessing liferay from the external host, it's deploying not in local host.
thumbnail
Juan Gonzalez, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Liferay Legend Postagens: 3089 Data de Entrada: 28/10/08 Postagens Recentes
Ok, more questions:

* Is you server public? I mean, just after starting it up, people can access it or you restrict the access?
* Can you attach your portal-ext.properties?
* Did you increase your DB connection pool to see if it improves?

I saw you're using Liferay EE, so probably best solution would be to open a LESA ticket in your case.
Alexander Lukashov, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
Hi!

Here is my answers:

1) It's public, but restricted with apache server, since it's staging site, so only development team could access it.
2) I have attached portal-ext file
3) I've tried, but it didn't helped. All current settings are in attached file.


Thanks in advance.
Alexander Lukashov, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
Hi!

Now performance issue has been fixed, but Control Panel and CKEditor are still boroken with Styles.

As I have mentioned Liferay 6.2 uses new version of Bootstrap. Could it be connected with it and how could I set up proper version for Control Panel?

Thanks!
thumbnail
Juan Gonzalez, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Liferay Legend Postagens: 3089 Data de Entrada: 28/10/08 Postagens Recentes
Hi Alexander.

Can you enable this filter, then clean work and temp directories, and restart server?

com.liferay.portal.servlet.filters.strip.StripFilter=true


BTW, I think you can safely remove this property

control.panel.layout.regular.theme.id=controlpanel


just in case it's related with the issue.

Then clean browser cache and try to access again.

Thanks.
Alexander Lukashov, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
Hi, Juan!

I've done requested changes, but it didn't help.

Control Panel looks the same. Attached part of screen shot.

Thanks in advance.
Alexander Lukashov, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
Hi, Juan!

Have you got a chance to take a look on the broken panel?

Thanks in advance
thumbnail
Juan Gonzalez, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Liferay Legend Postagens: 3089 Data de Entrada: 28/10/08 Postagens Recentes
Didn't have time.

Are there any errors when looking into browser network logs? I mean 404 or 500 errors.

Additionally, any errors in server logs after latest changes?
Alexander Lukashov, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
Hi!

There is not error messages in tge log file.

The only errors, which we have got are in browser.
I've attached screenshot.

Thanks in advance.
Alexander Lukashov, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
Hi!

I've done additional things:

  • restored DB and run upgrade process again;
  • downloaded and install the latest available patch for Liferay;
  • disabled minifier in portal-ext.properties

theme.css.fast.load=false
javascript.fast.load=false
minifier.enabled=false


Before starting cleared work and temp folder, cleared browser cache. But the issue with Control Panel is the same - it looks broken, part of functionality is not enabled. Also CKEditor is not reachable and keep getting errors about it in Browser Console.

Could you check this and point what could be done else? There is no error message on startup, performance is much better. This is very critical for us to resolve this issue.

Thanks in advance.
thumbnail
Arun Das, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Regular Member Postagens: 166 Data de Entrada: 23/07/12 Postagens Recentes
Hi,
Since you are using Apache to front tomcat, please add the following to your portal-ext.properties file(hope your are using port 80 for apache).
web.server.http.port=80
and restart liferay.

Arun
thumbnail
Juan Gonzalez, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Liferay Legend Postagens: 3089 Data de Entrada: 28/10/08 Postagens Recentes
My best guess at this point is about your apache frontend web server.

I see you are accessing through https, probably there are problems when passing request from Apache to Liferay.

Accessing Liferay server without Apache in plain http request could help in discard this.
thumbnail
Arun Das, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Regular Member Postagens: 166 Data de Entrada: 23/07/12 Postagens Recentes
Hi,
I didn't notice https. Thanks Juan for pointing it out. So you should add the following if you are using https
web.server.https.port=443
web.server.protocol=https



Arun
Alexander Lukashov, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2

Junior Member Postagens: 26 Data de Entrada: 08/06/15 Postagens Recentes
Hi!

Thank you for all your efforts and help. Finally I have found the reason of this issue. There was a file in the custom hook - top_head.jsp, and it has caused to this issue.

This thread could be closed, since all issues have been resolved.

Thanks!
thumbnail
Juan Gonzalez, modificado 7 Anos atrás.

RE: Broken Control Panel and Performance after upgrade to Liferay 6.2 (Resposta)

Liferay Legend Postagens: 3089 Data de Entrada: 28/10/08 Postagens Recentes
Alexander Lukashov:
Hi!

Thank you for all your efforts and help. Finally I have found the reason of this issue. There was a file in the custom hook - top_head.jsp, and it has caused to this issue.

This thread could be closed, since all issues have been resolved.

Thanks!


Ouch! It isn't usually a good idea to hook that file, and indeed it's one of the first things to look at when upgrading, just checking the hooks.

Ok, after all I think other things were improved too, glad you found the issue.