Best way to build Liferay DXP from source?

[Zak Thompson, geändert vor 6 Jahren.]

I'm currently in the process of trying to track down an issue where some assets that should be appearing in a dynamic asset publisher are not. I believe it is due to a data integrity issue in my DB, but I'm not able to nail it down the cause 100% yet. I've tracked the SQL generation for the asset query to the "buildAssetQuerySQL" method in the AssetEntryFinderImpl class. I'd like to build the portal from source so that I can add some logging to this method as well as a few others, as the base AssetEntryFinder and AssetEntryQuery code have no logging statements.

However, when I download the source for the fixpack we are on, it does not have any of the build files necessary to build the portal. The same is true of the service pack source code bundles, they don't have the build files. What is the proper way to get the necessary environment to build the full portal from source?

The best I could come up with is cloning portal master and copy-pasting the fix pack code over it, but that doesn't feel right to me.

[David H Nebinger, geändert vor 6 Jahren.]

Zak Thompson:

I'd like to build the portal from source so that I can add some logging to this method as well as a few others, as the base AssetEntryFinder and AssetEntryQuery code have no logging statements.

You can't. The source is provided only to support debugging, it is not provided for you to build on your own.

The best I could come up with is cloning portal master and copy-pasting the fix pack code over it, but that doesn't feel right to me.

Because it is not right. Portal-master is for CE, first of all, not DXP, plus portal-master has stuff layered in for 7.1 already, it is definitely not going to be what you want to do.

[Zak Thompson, geändert vor 6 Jahren.]

Am I looking at having to do an EXT to override the finderimpl to add the desired logging then? Since the finderimpl is not an OSGIcomponent, I don't think I'd be able to use service ranking stuff to override the class.

[David H Nebinger, geändert vor 6 Jahren.]

I think your original premise is off.

If there were data integrity issues related to assets and asset publisher, there would be lots of folks raising all kinds of hell.

Instead of jumping to a conclusion, can we step all the way back to the original problem? I mean, doing an EXT plugin to override a class is a rough path to take, especially if it is just to add some logging...

[Zak Thompson, geändert vor 6 Jahren.]

The issue we're running into is that the asset publisher is not displaying all of the content it should be. This is happening in two cases with two different tags. We have remote-staging set up for the production environment. We also are using DB driven asset publishers currently instead of elasticsearch driven.

One tag has 21 entries assigned to it, the other has 16 entries. Each tag has an asset publisher, where the only restriction on what assets are shown is that they have the particular tag. On the staging site, the first tag only shows 20 entries in the asset publisher instead of 21. On production, the asset that is missing on staging is shown in the asset publisher, but two other entries are missing. For the tag with 16 entries, it shows 13 on the staging site. The three that are missing on staging are also present on production, but there are another two that are missing that were present on staging.

I ran the following query to verify that the tag links properly exist, and for each tag, it lists 21 and 16 assets respectively, so it seems like the tag links are not the issue. When run on the prod and staging DBs, they return identical title lists, so it seems like at least the asset entries are properly setup:

SELECT title
FROM assetentry e
	JOIN assetentries_assettags m ON e.entryid = m.entryid
	JOIN assettag t ON t.tagid = m.tagid
WHERE t.name ILIKE '%${TAG_NAME}%'
ORDER BY title ASC;

Since it seems like the tag links are set up properly, I wasn't sure if there was some other data in the database that is causing issues. Thats why I wanted to look at the AssetEntryQuery objects and the generated SQL query from the finder, so that I could see specifically what is being looked for, and what could be causing the entries to not be found.

[David H Nebinger, geändert vor 6 Jahren.]

Zak Thompson:

One tag has 21 entries assigned to it, the other has 16 entries. Each tag has an asset publisher, where the only restriction on what assets are shown is that they have the particular tag. On the staging site, the first tag only shows 20 entries in the asset publisher instead of 21. On production, the asset that is missing on staging is shown in the asset publisher, but two other entries are missing. For the tag with 16 entries, it shows 13 on the staging site. The three that are missing on staging are also present on production, but there are another two that are missing that were present on staging.

Since I don't know, I have to ask, are the missing assets actually in the target systems? They'd have to have the right status, the right tag, etc.

Also what version are you running here? I know in DXP there were lots of updates for remote staging dealing w/ issues of things not getting published correctly and, IIRC there was at least one issue w/ categories and tags.

I ran the following query to verify that the tag links properly exist, and for each tag, it lists 21 and 16 assets respectively, so it seems like the tag links are not the issue. When run on the prod and staging DBs, they return identical title lists, so it seems like at least the asset entries are properly setup:

Well, the title might be the same but the status might not. Or the expired date might not. Or the site, group, whatever might not, ...

Since it seems like the tag links are set up properly, I wasn't sure if there was some other data in the database that is causing issues. Thats why I wanted to look at the AssetEntryQuery objects and the generated SQL query from the finder, so that I could see specifically what is being looked for, and what could be causing the entries to not be found.

Okay, so for your purposes there's a handful of service methods you need additional logging on; I would do a service wrapper combined w/ enabling logging of Hibernate SQL. In the service wrapper you wrap the method with the AEQ so you can log it and also "mark" in the log that you're about to do the query. That way when you open the logs you can search for your "mark" and weed out the fluff.

To enable the SQL logging, add hibernate.show_sql=true to your portal-ext.properties.

You may actually be looking at some sort of remote staging issue. Knowing what version you're running will help determine that...

[Zak Thompson, geändert vor 6 Jahren.]

Thanks for the service wrapper tip David, I completely forgot those existed. I'll work on getting one of those set up with the logging.

I believe all the target assets have been replicated properly, but I was only looking at titles when I initially looked. I'll double check all things like tags, asset links, status, etc.

We are currently on fix pack DE-29.

[David H Nebinger, geändert vor 6 Jahren.]

Zak Thompson:

We are currently on fix pack DE-29.

Okay, so we're currently at fix pack 32, so it's always recommended to try against the later version to see if the issue has been fixed.

Also, if you haven't done so already, get your LESA ticket open so support can start evaluating...

[Zak Thompson, geändert vor 6 Jahren.]

Hi David,

After a bunch of analysis, I discovered that the issue was in a custom display template and not in the assets being returned from Liferay.
The template author had used an indexed based for loop as they needed to access the current iteration index in the loop, but they had a logic bug where the index would also get incremented inside an if statement the loop in some cases, resulting in elements being skipped.

Once I got the hibernate logging turned on with your help, I was able to quickly verify that the issue wasn't on the back end, and was able to direct my attention to the display template. The front end code threw zero errors, which is why I initially thought it was us having some incorrect data.

Thanks for all the help!