Indexing By Locale

[Ray Augé, geändert vor 15 Jahren.]

Hey All,

It has occurred to me that our indexing design might be a problem for
multi-locale sites.

Can anyone speak to encountering problems with how we index content, not
in general, but specifically with respect to localized content?

I have spoken to Bruno Farache (our indexing expert) and have sort of
considered how we might address it but only if anyone is seeing
problems.

[jenny heinrich, geändert vor 15 Jahren.]

Hello,

does liferay support multi locale sites for switzerland? (de_CH, it_CH, fr_CH for pages as locale - cms and jsp)

I did the forum post in the development forum:
http://www.liferay.com/web/guest/community/forums/-/message_boards/message/1971546

Thanks,

Jenny

[Ray Augé, geändert vor 15 Jahren.]

I don't believe so. But this is merely a question of providing a
translation.

Please visit the New Translations Forum category for details:


Categories » Liferay Core Developers » Translations

[ITCSrikanth ITCReddy ITCSanivarapu, geändert vor 15 Jahren.]

Ray Augé:

I don't believe so. But this is merely a question of providing a
translation.

Please visit the New Translations Forum category for details:


Categories » Liferay Core Developers » Translations

Hi Ray,

In my portal-ext.properties files i've set

locale.default.request=true.

and i have changed my system language to finnish.

Now before login it is showing content in finnish.
But after login it is changing to english.
My need is , i want to show the site still in finnish.

If u open site in japan then the entire portal should display in japanese language , even before login and after login.

Can you please tel me any suggestions about this?

Thanks & Regards,
Srikanth Reddy.S

[Ray Augé, geändert vor 15 Jahren.]

There is a default you can specify to set the default locale for new
users (see portal.properties file).

Once they are created, the locale setting is under their control. They
can set it as they please in their account settings.

[Dmitry Babain, geändert vor 15 Jahren.]

sorry if i misinterpreted your question...

We have a problem with indexing document content in Document Library.

For example, .odt document with russian words in content will be searchable (so i can find documents, containing specific russian word). But .doc and .rtf with the same content won't be searchable.

[Bruno Farache, geändert vor 15 Jahren.]

Dmitri, this usually happens when documents are not saved with UTF-8 encoding. Usually you can set the encoding when you are saving the file.

[Ezequiel Chavez, geändert vor 15 Jahren.]

Hi Ray thanks for care about this issue, for me the process of indexing and searching should be by locale, I mean that all content should be indexed based on the current locale of the user/selected locale (combo box)/availables localized content, also all search on indexed content should specify a locale (default portal locale, user session locale, selecting from available locales).

in journal articles the Indexer java class would be something like:

Document doc = new DocumentImpl();

doc.addUID(PORTLET_ID, articleId);

doc.addKeyword(Field.COMPANY_ID, companyId);
doc.addKeyword(Field.PORTLET_ID, PORTLET_ID);
doc.addKeyword(Field.GROUP_ID, groupId);

//for each localized title
Iterator it = titlesByLocale.keySet().iterator();
while (it.hasNext()) {
     String locale = (String) it.next();
     String title = (String) titlesByLocale.get(locale);
     [b]doc.addText(Field.TITLE + "_" + locale, title);[/b]//Create field "title_en_US", "title_es_ES", etc.
}
//for each localized content
Iterator it = contentByLocale.keySet().iterator();
while (it.hasNext()) {
     String locale = (String) it.next();
     String content = (String) contentsByLocale.get(locale);
     [b]doc.addText(Field.CONTENT + "_" + locale, content);[/b]//Create field "content_en_US", "content_es_ES", etc.
}
doc.addText(Field.DESCRIPTION, description);

doc.addModifiedDate();

doc.addKeyword(Field.ENTRY_CLASS_PK, articleId);
doc.addKeyword("version", version);
doc.addKeyword("type", type);
doc.addDate("displayDate", displayDate);

doc.addKeyword(Field.TAGS_ENTRIES, tagsEntries);

is only an idea

[Ray Augé, geändert vor 15 Jahren.]

This is exactly what I thought of doing as well.

Bruno mentioned to me that this might not be the way to go though.. that
it might be better to go with a new Document for each locale...

I don't exactly recall the argument right now. Perhaps, he can refresh
my memory tomorrow.

[Bruno Farache, geändert vor 15 Jahren.]

I'm still awaken . My argument was that you can't use different Lucene Analyzers for each field within a document. One language analyzer parses all document fields at once.

For example, if an article has two titles, one in Portuguese and other in Spanish, if you use the Spanish analyzer, title_pt_BR will have some Spanish "stop words" removed (irrelevant words like: el, la, un...), but these Spanish stop words may be relevant in Portuguese language...

Removing stop words is just one of the phases that Analyzers do, there are more stuff they change that are specific to certain languages.

[Bruno Farache, geändert vor 15 Jahren.]

OK, sorry, I was wrong, it is possible to have separate analyzers for different fields, and it's called, guess what... PerFieldAnalyzerWrapper.

We just need to use it while adding localized fields...

[Ray Augé, geändert vor 15 Jahren.]

Awesome... This is a far less intrusive change than many documents, 1
per locale.