robots.txt placement

[Bill Kudrle, modificado 17 Anos atrás.]

This may be an obvious question to others, but I was wondering where to place the robots.txt file in a Liferay portal. The robots.txt is of course the file used by web bots (e.g., googlebot) to say whether they have permission or not to index parts of a site. Should the robots.txt go in the root directory (e.g., same level as WEB-INF, html, etc.) or perhaps in the html subdirectory, or perhaps someplace else?

[Ray Reid, modificado 14 Anos atrás.]

Hi, I'm wondering the same as above..

I've a 'robots.txt' file in the Glassfish Liferay root directory

'/Apps/glassfish/domains/domain1/applications/j2ee-modules/liferay-portal'

The robots.txt file contains the following :

User-agent: *
Disallow: /

This should be enough to warn off any bots... then to double check, I've got the following line in index.jsp which is in the same directory:

<meta name="*" content="noindex,nofollow">

And after all this I'm still getting 'attacked' by bots, only this MSN '65.55.106.204' tried it ...

Please Help...

[Jörg Halitschke, modificado 14 Anos atrás.]

Ray Reid:

<meta name="*" content="noindex,nofollow">

The correct Meta-Variable is <meta name="robots" content="noindex,nofollow">.

Is the robots.txt available on your servers root?
http://yourserverdomain.com/robots.txt

On a Tomcat it's e.g. /home/liferay/liferay-portal-5.2.3/tomcat/webapps/ROOT
I put that two restrictions on my system, so let's see if the bots keep away.

[Chris Tantalo, modificado 13 Anos atrás.]

where does this file go for liferay 5.2.3 running tomcat?

[Fred Wu, modificado 12 Anos atrás.]

Check this one out first: http://issues.liferay.com/browse/LPS-10275

So the work-around is to configure the portal-ext.properties and add those two robots templates into portal-impl.jar file and restart tomcat, then robots.txt under the ROOT can be found when the browser or client requests it.

[Julio Hurtado, modificado 11 Anos atrás.]

yo tengo el liferay 6.0.6 + jboss, sin embargo el robots.txt lo tengo al siguiente nivel y no me funciona D:\LiferayIBK\jboss-5.1.0\server\default\deploy\ROOT.war.. me aparece el mensaje de

Estado
No encontrado

El recurso solicitado no ha sido encontrado.

http://192.168.1.164/web/guest/robots.txt

thanks

[Julio Hurtado, modificado 11 Anos atrás.]

I have the liferay + jboss 6.0.6, but the robots.txt I have it the next level and does not work D: \ LiferayIBK \ jboss-5.1.0 \ server \ default \ deploy \ ROOT.war .. I get the message

state
not Found

The requested resource was not found.

http://192.168.1.164/web/guest/robots.txt

thanks

[Fred Wu, modificado 11 Anos atrás.]

Compruebe éste, en primer lugar: http://issues.liferay.com/browse/LPS-10275

Así que la solución consiste en configurar los portales-ext.properties y añadir las dos plantillas de los robots en el portal impl.jar archivo y reiniciar Tomcat, entonces robots.txt en la raíz se puede encontrar cuando las peticiones de tu navegador o cliente de la misma.