Searching for JPA entities in Liferay

So you want to search for your custom JPA entities in Liferay? Quite some documentation is already available but it is somewhat scattered over several blogposts and articles. With this blogpost I would like to give you a summary of everything you need to do, or more precisely everything we did, to make custom JPA entities searchable in the standard Liferay Search Portlet.

The requirement

One of our customers wanted the search results to not only contain the default Liferay objects like articles, documents, etc... But also some of his own custom domain objects. We had created a custom lightweight Support Ticket framework with a set of limited functionalities. It allowed a user to create a ticket and post comments to it. The additional requirement was that a user had to be able to also search for these tickets based on title, description and... oh yeah the comments too. In this blog I will use a very simplified version of this domain that shows all the steps necessary to make a JPA entity searchable. The domain only has 1 JPA entity, SupportTicket, together with a repository and service. There is also a simple portlet that can be used to create and show instances of this entity. So let's go on a journey, a quest you might even call it, to enable the search functionality for your custom JPA entities. Eventually you will get to the end point where you will use the Liferay Search Portlet to search and find your own JPA entities. You can also create your own custom search (taking customization to the max!) but that is outside the scope of this blog post. In the resources section however you can find links for more information on this matter. In a vanilla Liferay 6.2 the result should look like this:

Full Result

Enter the Indexer

Liferay’s search & indexing functionality is provided by Apache Lucene, a Java search framework. It converts searchable entities into documents. Documents are custom objects that correspond to searchable entities. Usually performing full text searches against an index is much faster than searching for entities in a database. You can find more information about Lucene on its own website. To start you will need to create an indexer class. This class will hold the information required to transform your JPA entity into a Lucene Document. Next you will need to register this Indexer in the portlet so that the Liferay framework knows about it and can use it when necessary.

Creating the Indexer

I already made clear that the Indexer class is responsible for creating the Lucene documents for your JPA entity. It is important to know that this is where you can decide what fields of your entity will get indexed in order to be searchable. You can also specify if this needs to be a term or a phrase. And of course there are some other settings you can implement such as the name of the associated portlet, permissions, ... Have a look at the Sources for blog posts with more information. Liferay provides you with a default implementation named BaseIndexer as an easy start. This abstract class should be extended and the required methods implemented. Enough of the theoretical stuff, let's get started. As soon as you extend the BaseIndexer, you will need to overwrite the following methods. I'm not going to go into the details of all the implementations, I refer you to the Github project. Mostly the implementation will depend on your own requirements and domain.

  • doDelete: delete the document that corresponds to the object parameter
  • doGetDocument: specify which fields the Lucene Index will contain
  • doGetSummary: used to show a summary of the document in the search results
  • doReindex (3 times): will be called when the index is updated
  • getPortletId (2 times): to be able to narrow searches for documents created in a certain portlet instance (hence it needs to be the full portlet Id)
  • getClassNames: the FQCN of the entities this indexer can be used for

Something important to notice in this class is the difference between document.addKeyword and document.addText in the doGetDocument method.

  • addKeyword: adds a single-valued field, with only one term
  • addText: adds a multi-valued field, with multiple terms separated by a white space

The doGetDocument method is the most important piece of code in the TicketIndexer. In this method an incoming Object is transformed into an outgoing (Lucene) Document. The object is of type Ticket as the Indexer has been registered to be used for objects of that type. This registration occurs in the getClassNames method. A new Document object holding the necessary searchable fields can be created from this Ticket. Notice that all the Keyword/Text fields are already predefined in Liferay so you can use these constants. This is quite handy but you can always provide your own names.

@Override protected Document doGetDocument(Object obj) throws Exception {
   Ticket ticket = (Ticket) obj;

   List<TicketComment> comments = BeanLocator.getBean(TicketCommentService.class).getComments(ticket);

   Document document = new DocumentImpl();

   document.addUID(PORTLET_ID, ticket.getId());
   document.addKeyword(Field.COMPANY_ID, PortalUtil.getDefaultCompanyId());
   document.addKeyword(Field.ENTRY_CLASS_NAME, TICKET_CLASS_NAME);
   document.addKeyword(Field.ENTRY_CLASS_PK, ticket.getId());
   document.addKeyword(Field.PORTLET_ID, PORTLET_ID);
   document.addText(Field.TITLE, ticket.getSubject());
   document.addText(Field.CONTENT, ticket.getDescription());
   document.addText(Field.COMMENTS, Collections2.transform(comments, getComments()).toArray(new String[] {}));

   return document}

Registering the Indexer

Once the Indexer is created, you will need to register it in the Liferay platform. This is a very simple action and it is completed by adding the following line into your portlet’s liferay-portlet.xml (check the DTD for the exact location).

<indexer-class>FQCN of your custom Indexer</indexer-class>

Once your portlet is redeployed Liferay will automatically register your indexer into its framework.

Tip
Just by registering your indexer, existing entities won’t be indexed. As a Liferay admin however, you can trigger a reindex from the Control Panel. Just go to Configuration > Server Administration > Resources (tab) and locate the  Execute button next to the  Reindex all search indexes entry. Be advised that if you already have a large index (due to web content, documents, …) this may take a while.
Info
You can use some index inspection tool like Luke to inspect and search through your index. If your system contains at least one instance of your custom JPA entity you should see those pop up in Luke after doing a reindex, which means your Indexer implementation did its job.
Info
If you are eager to know whether or not your current progress has had any impact, you can already update the Liferay Search Portlet as described in Executing the Search.

Using the indexer programmatically

No doubt it is great that your existing entities are now indexed. But you don’t want to trigger this indexing yourself manually, do you? By all means you don’t, you want it to happen automagically. Or at least programmatically. That is why you are reading this blog post anyway. So how about actually using the indexer yourself in the creation of a support ticket? Actually you want to use your newly created Indexer at the right time. But what is the right time? Well, right after a new entity is created. Or updated. Or even deleted. So you will need to use your Indexer at these moments, using the code below. We used the nullSafeGetIndexer as it returns a dummy indexer when no indexer can be found for the class, contrary to the default getIndexer() which returns null.

Indexer indexer = IndexerRegistryUtil.nullSafeGetIndexer(Ticket.class.getName());
indexer.reindex(ticket);

This code instructs your indexer, which you've registered with the FQCN of your entity, to reindex any added or updated entities. It delegates to the doReindex methods you have implemented in your own Indexer. When an entity is deleted its corresponding document should not show up in any search results anymore. You will need to call the code below when you are deleting your entity.

Indexer indexer = IndexerRegistryUtil.nullSafeGetIndexer(Ticket.class.getName());
indexer.delete(ticket);

Redeploying your portlet should do the trick of having each new entity being registered as a Document in the index as well. Go and try it out (if you have a create option of course).

Executing the Search

In this blog post Liferay's Search Portlet will be used to search and find your custom entities. You can always create a custom search portlet as well, check the Sources section for articles on how to achieve that. But for our customer we decided that integrating the custom JPA entity into Liferay's Search Portlet was the best solution as other content, e.g. Web Content, should also be searchable. By doing this the client had a nice integration with the default Liferay functionality. The Liferay Search Portlet will need to be updated to also take into account your entity by adding the portlet to a page and adjusting its configuration. But first check out the portlets normal behaviour. As an admin, add the portlet to a page of your choice. Search for the value of one of the earlier defined searchable fields in the Indexer and hit Search. Notice that there are no search results found and that this is clearly communicated.

No Results

Now go to the Configuration panel of the portlet, select Advanced and add the FQCN of your entity between quotes in the asset_entries list. Keep in mind the comma separation.

{
   "displayStyle": "asset_entries",
   "fieldName": "entryClassName",
   "static": false,
   "data": {
      "frequencyThreshold": 1,
      "values": [
         "com.liferay.portal.model.User",
         "com.liferay.portlet.bookmarks.model.BookmarksEntry",
         "com.liferay.portlet.bookmarks.model.BookmarksFolder",
         "com.liferay.portlet.blogs.model.BlogsEntry",
         "com.liferay.portlet.documentlibrary.model.DLFileEntry",
         "com.liferay.portlet.documentlibrary.model.DLFolder",
         "com.liferay.portlet.journal.model.JournalArticle",
         "com.liferay.portlet.journal.model.JournalFolder",
         "com.liferay.portlet.messageboards.model.MBMessage",
         "com.liferay.portlet.wiki.model.WikiPage",
         "be.olaertskoen.blog.search.Ticket"
      ]
   },
   "weight": 1.5,
   "className": "com.liferay.portal.kernel.search.facet.AssetEntriesFacet",
   "label": "asset-type",
   "order": "OrderHitsDesc"
},

Hit Save, close the panel and perform the same search again. Notice that the search result page has changed. It looks like the portlet found something but doesn't really know how to show it. Oh me oh my...

First Results

Probably this is not exactly what you wanted. Hell, it should be nothing you ever wanted! What use was all of this? Well, it was the preparation for what is to follow. If you followed the hints or tip above, you already know the indexer has done its work. Now it is time to actually use its results.

How about some usable search results?

You now know that your entities can be found but that the search results look nothing like you ever dreamed of. Next up you are going to get that party started and you should come to an end of this search functionality quest. As you probably know, or not if you are new to the game, but Liferay uses their Asset framework extensively. Not only in the Asset Publisher or for all of their web content, but also for their search results. Yes, that is right: the results in the search portlet are rendered using Assets. Therefor you will also need to create an Asset Renderer for your JPA entities. And more or less consistently you will need to register it as well. At last you will need to use it one way or another.

Creating an Asset Renderer

Similar as to the Indexer there is a base class you can easily extend, BaseAssetRenderer, to implement your own version. This class will give you a bunch of methods you will need to overwrite. One if them is to provide a summary for your entity, another is for the title. Two other important methods are render and getUrlViewInContext. The first one, render, will return a String with the url leading to the full content of your entity. It will use an Asset Publisher on a hidden page created for the Search Portlet. In our project we have an entire page reserved for this so we need to return the url of that detail page (here the page to edit the ticket).

@Override
public String render(RenderRequest renderRequest, RenderResponse renderResponse, String template) throws Exception {
   return TicketDetailFriendlyUrl.create(ticket.getId());
}

The second method, getUrlViewInContext, will be used by the Search Portlet to render the asset in context. This is an option you can activate in the Search Portlet. The result is that the Asset Entry is shown in its own context unlike the previous where the entry is shown in an Asset Publisher. This is actually the default setting and according to me the nicest solution.

@Override
public String getURLViewInContext(LiferayPortletRequest liferayPortletRequest, LiferayPortletResponse liferayPortletResponse, String noSuchEntryRedirect) throws Exception {
   ThemeDisplay themeDisplay = (ThemeDisplay) liferayPortletRequest.getAttribute(WebKeys.THEME_DISPLAY);
   Group group = themeDisplay.getScopeGroup();
   boolean isPrivate = themeDisplay.getLayout().isPrivateLayout();
   String groupUrl = PortalUtil.getGroupFriendlyURL(group, isPrivate, themeDisplay);

   return groupUrl + TicketDetailFriendlyUrl.create(ticket.getId());
}

Fabricating the Asset Renderer

For assets there is actually a required Factory to be used for the AssetRenderers. But again this is easy to achieve as Liferay provides a  BaseAssetRendererFactory for you to extend. It contains three methods: one for the type, one for the class name (both methods for our entity class of course) and one method that returns an AssetRenderer. In that method you will need to create a new instance of your entity based on the incoming id and type. The latter was not required in our case as the renderer is only registered for Objects of type Ticket.
@Override
public AssetRenderer getAssetRenderer(long classPK, int type) throws PortalException, SystemException {
   TicketService ticketService = BeanLocator.getBean(TicketService.class);
   Ticket ticket = ticketService.getTicket(classPK);

   return new TicketAssetRenderer(ticket);
}
Registering the AssetRendererFactory

Unlike the Indexer you are not going to register the AssetRenderer but the AssetRendererFactory in your liferay-portlet.xml. Why else would you have created that class, right? It is easily performed by adding an asset-renderer-factory tag, containing the factory's FQCN, just before the instanceable tag.

<asset-renderer-factory>be.olaertskoen.blog.search.tickets.renderer.TicketAssetRendererFactory</asset-renderer-factory>
Warning
To actually get the portlet started, you will need to add a non-empty  portlet.properties into the resources folder. We just added a property  plugin.package.name.

If you redeploy your portlet, you will again see some changes in the search results.

Fault Asset Results

Wait, where is the title? Where is the description? You coded that in the AssetRenderer, didn't you! Why are you not seeing any actual result? Why is the portal footer (that Powered By Liferay statement) suddenly shown in the search results? Well, Liferay would like to render an asset, but actually there is no asset. So it just can't render an Asset. If you would open your servers log, you will see an error message like this:

NoSuchEntryException: No AssetEntry exists with the key {classNameId=10458, classPK=1}

While you already had existing entities there were never any AssetEntries created for them unless you already implemented the Asset framework for some other reason. Hence you will need to use that framework into your entity creation process as well.

Creating Assets

You will need to adjust the add, update and delete code of your custom entities. Whenever a new entity is created, an AssetEntry needs to be created as well. When the entity is updated, the corresponding AssetEntry should be updated as well. And when an entity is deleted the corresponding AssetEntry should be deleted as well. No need for orphaned database entries! As you know, Liferay provides you with lots of LocalServices to use their framework. This is also the case for the Asset framework. When you create an instance of your entity you can use the AssetEntryLocalServiceUtil to create the corresponding AssetEntry using the following code:

String ticketFriendlyUrl = TicketDetailFriendlyUrl.create(ticket.getId());

Long userId = Long.valueOf(getExternalContext().getUserPrincipal().getName());
ThemeDisplay themeDisplay = getThemeDisplay();
Long groupId = themeDisplay.getScopeGroupId();


AssetEntryLocalServiceUtil.updateEntry(userId, groupId, ticket.getCreateDate(),
                                       ticket.getCreateDate(), Ticket.class.getName(), ticket.getId(), String.valueOf(ticket.getId()), 0,
                                       new long[] {}, new String[] {}, true, null, null, null, "text/html", ticket.getSubject(),
                                       ticket.getDescription(), ticket.getDescription(), ticketFriendlyUrl, null, 0, 0, null, false);

It is usually a good idea to do this right before the code you added to create the index entry for your entity. Now after you redeploy, you will need to create a new instance of your custom entity. For the existing ones there still are no AssetEntries present in Liferay. But when you create a new instance an AssetEntry will be created automagically (well at least we magicians now know the programmatics behind it). So go ahead and create a new instance, you will probably already have a portlet for that.

Alert
As written earlier you will need to include similar code in your update and delete code for your entities. In this blog post and example portlet these have been left out.

Next try and search for it using one of the properties you declared in the Indexer class and… Tadaa!

Asset Result

Congratulations! You now have a good looking search result. Hm... Wait, it is not really 100%... Why is the FQCN shown as type of the entity? That does not look so nice! And it is even shown twice! Merde, will this journey never end?

Naming your entities

Be sure, our quest will soon come to an end. To provide your entities with a nice name you can again use Liferay's framework (how many rabbits are there in that hat?). All you need to do is add a translation for it, simple as that. If you look closely it is not really the FQCN of your custom entity that is shown in the Search Portlet, it is prepended with model.resource. You can add that entire output into a language hook as the key and provide it with a value (Superentity, Batentity, Spiderentity or whatever you want to name it).

model.resource.be.olaertskoen.blog.search.Ticket=Support Ticket

Deploy this language hook and reload your previous search.

Full Result

Finally, it’s over

Isn’t that nice! You are using Liferay's Search Portlet to search for your own custom entities! What a journey that has been. But our customer was very satisfied with this functionality and I hope yours will be too.

Done

With this you now know how to enable a full fledged search option for your custom JPA entities in Liferay. Just as a reminder, here is a short overview of the steps you took.

  • You started out with creating, registering and using an Indexer class that transforms your custom entities into Lucene Documents.
  • Next you configured the default Liferay Search Portlet to also take your custom JPA entities into account in the search queries. This was an easy configuration in the Portlet itself.
  • Last but not least you created, registered and used an AssetRenderer to nicely render the AssetEntry for your custom JPA entity in the Search Portlet. In this step you also added code to create and maintain that AssetEntry according to the lifecycle of your own entities.
Those three steps are all it takes to enable the search functionality for your custom JPA entities created in Liferay. Enjoy! You can find all the example code for this post in my Github project.

Sources

As mentioned at the start of this blog post, there are several articles and blog posts concerning search and what is required. But we needed to combine several of them to full enable all the features as described in this blog post. Below is a list of the Liferay sources we used. They are still worth reading if you want to know more on what lies behind the scenes or if you need to implement some other specific features.
 
ブログ
I am so happy to finally see a documentation on this topic. I digged over the internet to make searching for custom JPA entity, not generated by ServiceBuilder. thanks, Im gonna try this.
Im having troubles adding FQCN of my class to asset_entries list. first of all I cant find such configuration in my portlet where there is this json based config in liferay 7. but it seems I can only choose between available portlets for search content and mine is not available..