Why we need a new liferay-npm-bundler (3 of 3)

Technical Blogs June 8, 2018 By Ivan Zaera Staff

A real life example of the use of bundler 2.x

This is the last of a three articles series motivating and explaining the enhancements we have done to Liferay's npm bundler. You can read previous article here.

To analyze how the bundler works we are going to examine a real life example comprising a portlet and an OSGi bundle providing Angular so that the portlet can import it. The project looks like this:


 npm-angular5-portlet-say-hello
     package.json
        {
            "name": "npm-angular5-portlet-say-hello",
            "version": "1.0.0",
            "main": "js/angular.pre.loader.js",
            "scripts": {
                "build": "tsc && liferay-npm-bundler"
            }
            …
        }
     tsconfig.json
        {
            "target": "es5",
            "moduleResolution": "node",
            …
        }
     .npmbundlerrc
        {
            …  
            "exclude": {
                "*": true
            },
            "config": {
                "imports": {
                    "npm-angular5-provider": {
                        "@angular/animations": "^5.0.0",
                        "@angular/cdk": "^5.0.0",
                        "@angular/common": "^5.0.0",
                        "@angular/compiler": "^5.0.0",
                        …
                    },
                    "": {
                        "npm-angular5-provider": "^1.0.0"
                    }
                }
            }
        }
     src/main/resources/META-INF/resources/css
         indigo-pink.css
            …  
     src/main/resources/META-INF/resources/js
         angular.pre.loader.ts
            // Bootstrap shims and providers
            import 'npm-angular5-provider';
            …  
        …
     npm-angular5-provider
         package.json
            {
                "name": "npm-angular5-provider",
                "version": "1.0.0",
                "main": "bootstrap.js",
                "scripts": {
                    "build": "liferay-npm-bundler"
                },
                "dependencies": {
                    "@angular/animations": "^5.0.0",
                    "@angular/cdk": "^5.0.0",
                    "@angular/common": "^5.0.0",
                    "@angular/compiler": "^5.0.0",
                    …
                }
                …
            }
     src/main/resources/META-INF/resources
         bootstrap.js
            /**
              * This file includes polyfills needed by Angular and must be loaded before the app.
            …  
            require('core-js/es6/reflect');
            require('core-js/es7/reflect');
            …
            require('zone.js/dist/zone');
            …
    …


You can find the whole project available for download here. Also, keep in mind that it is supposed to be run in Liferay 7.1.0 B2 at least (download it from here). It will not work in Liferay 7.0.0 unless you do some modifications!

As you can see, the portlet's build process includes calling the Typescript compiler (tsc) and then the bundler. We need to invoke tsc because Angular is based on the Typescript language and tsc is responsible for transpiling it to ES5. The Typescript compiler is configured in the tsconfig.json file and it is important that we set its output to es5 and its module resolution to node. That is because the bundler always expects that the input JS files are in those language and module formats.

Next, have a look at .npmbundlerrc where the imports for Angular are configured. Please note that we also import npm-angular5-provider with no namespace because we are going to invoke one of its modules to bootstrap Angular shims: see the angular.pre.loader.ts file, where npm-angular5-provider is imported. That import, in turn, loads npm-angular5-provider's main file (bootstrap.js).

Also, pay attention to the exclude section where every dependency of npm-angular5-portlet-say-hello is excluded to prevent Angular from appearing inside its JAR. This makes the build process faster and optimizes deployment but don't worry if you forget to exclude any unneeded dependency because nothing will fail: it just won't be used and will use a bit more space than needed.

The setup for npm-angular5-provider is very simple. It just declares Angular dependencies and invokes liferay-npm-bundler to bundle them. No need to do anything in this project. However, note how it also includes the bootstrap.js that is responsible for loading some shims needed by Angular. This file must always be invoked (by importing npm-angular5-provider from any portlet using it) before any portlet is run so that Angular doesn't fail because of missing APIs.

To finish with, check out the indigo-pink.css file of npm-angular5-portlet-say-hello. To keep this example simple, we have copied this file from the @angular/material npm package. It contains a prebuilt theme suitable for the Angular's Material Design widgets framework. In a real setup, that file's styles should be provided by a Liferay theme instead of being directly bundled inside each portlet needing it.

Now, suppose we run both builds. Let's see how the output would look like:


 npm-angular5-portlet-say-hello
     build/resources/main/META-INF/resources
         package.json
            {
                "dependencies": {
                    "@npm-angular5-provider$angular/animations": "^5.0.0",
                    "@npm-angular5-provider$angular/cdk": "^5.0.0",
                    "@npm-angular5-provider$angular/common": "^5.0.0",
                    "@npm-angular5-provider$angular/compiler": "^5.0.0",
            …
         js
             angular.loader.js
                "use strict";

                Liferay.Loader.define(
 ➥ "npm-angular5-portlet-say-hello@1.0.0/js/angular.loader",
 ➥ ['module', 'exports', 'require',
 ➥ '@npm-angular5-provider$angular/platform-browser-dynamic',
 ➥ './app.component',
 ➥ ...
 ➥ function (module, exports, require) {
                    var define = undefined;
                …
 npm-angular5-provider
     build/resources/main/META-INF/resources
         package.json
            {
                "name": "npm-angular5-provider",
                "version": "1.0.0",
                "main": "bootstrap.js",
                "dependencies": {
                    "@npm-angular5-provider$angular/animations": "^5.0.0",
                    "@npm-angular5-provider$angular/cdk": "^5.0.0",
                    "@npm-angular5-provider$angular/common": "^5.0.0",
                    "@npm-angular5-provider$angular/compiler": "^5.0.0",
                    …
                }
                …
            }
         bootstrap.js
            Liferay.Loader.define(
➥ 'npm-angular5-provider@1.0.0/bootstrap',
➥ ['module', 'exports', 'require',
➥ 'npm-angular5-provider$core-js/es6/reflect',
➥ ...
➥ function (module, exports, require) {
                var define = undefined;
                /**
                * This file includes polyfills needed by Angular and must be loaded before the app.
                …  
                require('npm-angular5-provider$core-js/es6/reflect');
                require('npm-angular5-provider$core-js/es7/reflect');
                …
                require('npm-angular5-provider$zone.js/dist/zone');
                …
            }
        …
         node_modules/npm-angular5-provider$core-js@2.5.7
             index.js
                Liferay.Loader.define(
➥ 'npm-angular5-provider$core-js@2.5.7/index',
➥ ['module', 'exports', 'require',
➥ './shim',
➥ ...
➥ function (module, exports, require) {
                    var define = undefined;
                    require('./shim');
            …
        …


Take a look at the output of npm-angular5-provider. As you can see, the bundler has copied the project and node_modules' JS files to the output and has wrapped them inside a Liferay.Loader.define() call so that the Liferay AMD Loader know how to handle them. Also, the module names in require() calls and inside the Liferay.Loader.define() dependencies array have been namespaced with the npm-angular5-provider$ prefix to achieve dependency isolation.

You may also have noted the var define = undefined; addition to the top of the file. This is introduced by liferay-npm-bundler to make the module think that it is inside a CommonJS environment (instead of an AMD one). This is because some npm packages are written in UMD format and, because we are wrapping it inside our AMD define() call, we don't want them to execute their own define() but prefer them to take the CommonJS path, where the exports are done through the module.exports global.

We have said that liferay-npm-bundler added these modifications but, to be fair, the real responsible is babel-plugin-wrap-modules-amd, a Babel plugin that is executed by Babel when it is invoked from the liferay-npm-bundler in one of its build phases.

If you are curious on how that plugin is configured, take a look at the default preset used by liferay-npm-bundler where the liferay-standard Babel preset is referenced which, in turn, configures the babel-plugin-wrap-modules-amd plugin.

Now, let's look at the package.json file and notice how the dependencies have been namespaced too. This is necessary to make the namespaced define() and require() calls work inside the JS modules, and it is done by the liferay-npm-bundler-plugin-namespace-packages plugin, configured here.

There are more plugins involved in the build that serve miscellaneous purposes. You can check their descriptions and use in the Liferay Docs.

Now let's see how the bundler has modified npm-angular5-portlet-say-hello. In this case we will only pay attention to the changes made in two files, because the rest is more or less the same as with the npm-angular5-provider.

First of all, the angular-loader.ts file has been converted to angular.loader.js. This has happened in two steps:

  1. The Typescript compiler transpiled angular-loader.ts to angular.loader.js generating a CommonJS module written in ECMAscript 5.
  2. The bundler then wrapped that code inside a Liferay.Loader.define() call to make it executable inside Liferay AMD Loader.

But more important: the module is importing Angular modules like @angular/platform-browser-dynamic which the bundler usually namespaces with the bundle's name (in this case npm-angular5-portlet-say-hello) but, because we are importing them from npm-angular5-provider, they have been namespaced with npm-angular5-provider$ instead so that they are loaded from that bundle at runtime (by the Liferay AMD Loader).

Finally, if you look at the dependencies inside the package.json file you will notice that the bundler has injected the ones pertaining to npm-angular5-provider to make them available at runtime.

And that's it. Shall this huge and boring series of articles help you understand how the new bundler works and how to leverage it to deploy your most exciting portlets.

Have fun coding!

Why we need a new liferay-npm-bundler (2 of 3)

Company Blogs June 7, 2018 By Ivan Zaera Staff

How we are fixing the problems learned in bundler 2.x

This is the second of a three articles series motivating and explaining the enhancements we have done to Liferay's npm bundler. You can read the first article to find out the motivation for taking the actions explained in this one.

To solve the problems we saw in the previous article we followed this process:

  1. Go one step back in deduplication and behave like standard bundlers (webpack, Browserify, etc.).
  2. Fix peer dependencies support.
  3. Provide a way to deduplicate modules.

Step 1 gets us to a point where we have repeatable builds that mimick what the developer has in his node_modules folder when the projects are deployed to the server, and run in the browser. We implement it by isolating project dependencies (namespacing the packages) so that the dependencies of each project are isolated between them.

With that done, we get 2 nearly for free, because we just need to inject virtual dependencies in package.json files when peer dependencies are used. But this time we know which version constraints to use, because the project's whole dependency tree is isolated from other projects. That is, we have something similar to a node_modules folder available in the browser for each project.

Finally, because we have lost deduplication with steps 1 and 2 and that leads to a solution which is equivalent to standard bundlers, we define a way to deduplicate packages manually. This new way of deduplication is not automatic but leads to full control (during build time) of how each package is resolved.

Let's see the steps in more detail...

Isolation of dependencies

To achieve the isolation the new bundler just prefixes each package name with the name of the project and rewrites all the references in the JS files. For example, say you have our old beloved my-portlet project:


 package.json
    {
        "name": "my-portlet",
        "version": "1.0.0",
        "dependencies": {
            "isarray": "^1.0.0"
        }
    }
 node_modules/isarray
     package.json
        {
            "name": "isarray",
            "version": "1.0.1",
            "main": "index.js"
        }
 META-INF/resources
     view.jsp
        <aui:script require="my-portlet@1.0.0/js/main"/>
     js
         main.js
            require('isarray', function(isarray) {
                console.log(isarray([]));
            });


When we bundle it with bundler 1.x we get something like:


 META-INF/resources
     package.json
        {
            "name": "my-portlet",
            "version": "1.0.0",
            "dependencies": {
                "isarray": "^1.0.0"
            }
        }
     view.jsp
        <aui:script require="my-portlet@1.0.0/js/main"/>
     js
         main.js
            require('isarray', function(isarray) {
                console.log(isarray([]));
            });
     node_modules/isarray
         package.json
            {
                "name": "isarray",
                "version": "1.0.1",
                "main": "index.js"
            }


But if we use bundler 2.x it changes to:


 META-INF/resources
     package.json
        {
            "name": "my-portlet",
            "version": "1.0.0",
            "dependencies": {
                "my-portlet$isarray": "^1.0.0"
            }
        }
     view.jsp
        <aui:script require="my-portlet@1.0.0/js/main"/>
     js
         main.js
            require('my-portlet$isarray', function(isarray) {
                console.log(isarray([]));
            });
     node_modules/my-portlet$isarray
         package.json
            {
                "name": "my-portlet$isarray",
                "version": "1.0.1",
                "main": "index.js"
            }


As you see, we just needed to prepend my-portlet$ to each dependency package name and that way, each project will load its own dependencies and won't collide with any other project. Easy.

If we did now the same test of deploying my-portlet and his-portlet, each one would get its own versions simply because we have two different isarray packages: one called my-portlet$isarray and another called his-portlet$isarray.

Peer dependency support

Because we have isolated dependencies per portlet, we can now honor peer dependencies perfectly. For example, remember the Diverted peer dependencies section in the previous article: with bundler 1.x, there was only one a-library package available for everybody. But with the new namespacing technique, we have two a-librarys: my-portlet$a-library and his-portlet$a-library.

Thus, we can resolve peer dependencies exactly as stated in both projects because their names are prefixed with the project's name:

my-portlet@1.0.0
    ➥ my-portlet$a-library 1.0.0
    ➥ my-portlet$a-helper 1.0.0

his-portlet@1.0.0
    ➥ his-portlet$a-library 1.0.0
    ➥ his-portlet$a-helper 1.2.0

And in this case, my-portlet$a-library will depend on a-helper at version 1.0.0 (which is namespaced as my-portlet$a-helper) and his-portlet$a-library will depend on a-helper at version 1.2.0 (which is namespaced ashis-portlet$a-helper).

How does all this magic happen? Easy: we have just created a new bundler plugin named liferay-npm-bundler-plugin-inject-peer-dependencies that scans all JS modules for require calls and injects a virtual dependency in the package.json file when a module from an undeclared package is required.

So, for example, let's say you have the following project:


 META-INF/resources
     package.json
        {
            "name": "my-portlet",
            "version": "1.0.0",
            "dependencies": {
                "isarray": "^1.0.0"
            }
        }
     js
         main.js
            require(['isarray', 'isobject', function(isarray, isobject) {
                console.log(isarray([]));
                console.log(isobject([]));
            });
     node_modules
         isarray
             package.json
                {
                    "name": "isarray",
                    "version": "1.0.1",
                    "main": "index.js"
                }
         isobject
             package.json
                {
                    "name": "isobject",
                    "version": "1.1.0",
                    "main": "index.js"
                }


As you can see, there's no dependency to isobject in the package.json file.

However, if we run the project through the bundler configured with the inject-peer-dependencies plugin, it will find out that main.js is requiring isobject and will resolve it to version 1.1.0 which can be found in the node_modules folder.

After that, the plugin will inject a new dependency in the output package.json so that it looks like this:

{
    "name": "my-portlet",
    "version": "1.0.0",
    "dependencies": {
        "isarray": "^1.0.0",
        "isobject": "1.1.0"
    }
}

Note how, being an injected dependency, isobject's version constraints are its specific version number, without caret or any other semantic version operator. This makes sense, as we want to honor the exact peer dependency found in the project and thus we cannot inject a more relaxed semantic version expression because it could lead to unstable results.

Also keep in mind that these transformations are made in the output files (the ones in your build directory), not on your original source files.

Deduplication of packages (imports)

As we said before, the problem with namespacing is that each portlet is getting its own dependencies and we don't deduplicate any more. If we always used the bundler this way, it won't make too much sense, because we could obtain the same functionality with standard bundlers like webpack or Browserify and wouldn't need to rely on a specific tool like liferay-npm-bundler.

But, being Liferay a portlet based architecture, it would be quite useful if we could share dependencies among different portlets. That way, if a page is composed of five portlets using jQuery, only one copy of jQuery would be loaded by the JS interpreter to be used by the five different portlets.

With bundler 1.x that deduplication was made automagically, but we had no control over it. However, with version 2.x, we may now import packages from an external OSGi bundle, instead of using our own. That way, we can put shared dependencies in one project, and reference them from the rest.

Let's see an example: imagine that you have three portlets that use our favorite Non Existing Wonderful UI Components framework (WUI). Suppose this quite limited framework is composed of 3 packages:

  1. component-core
  2. button
  3. textfield

Now, say that we have these three portlets:

  1. my-toolbar
  2. my-menu
  3. my-content

Which we use to compose the home page of our site. And the three depend on the WUI framework.

If we just use the bundler to create three OSGi bundles, each one will package a copy of WUI inside it and use it when the page is rendered, thus leading to a page where your browser loads three different copies of WUI in the JS interpreter.

To avoid that, we will create a fourth bundle where WUI is packaged and import the WUI packages from the previous three bundles. This will lead to an structure like the following:


 my-toolbar
     .npmbundlerrc
        {
            "config": {
                "imports": {
                    "wui-provider": {
                        "component-core": "^1.0.0",
                        "button": "^1.0.0",
                        "textfield": "^1.0.0"
                    }
                }
            }
        }
 my-menu
     .npmbundlerrc
        {
            "config": {
                "imports": {
                    "wui-provider": {
                        "component-core": "^1.0.0",
                        "button": "^1.0.0",
                        "textfield": "^1.0.0"
                    }
                }
            }
        }
 my-content
     .npmbundlerrc
        {
            "config": {
                "imports": {
                    "wui-provider": {
                        "component-core": "^1.0.0",
                        "button": "^1.0.0",
                        "textfield": "^1.0.0"
                    }
                }
            }
        }
 wui-provider
     package.json
        {
            "name": "wui-provider",
            "dependencies": {
                "component-core": "^1.0.0",
                "button": "^1.0.0",
                "textfield": "^1.0.0"
            }
        }


As you can see, the three portlets declare the WUI imports in the .npmbundlerrc file. They would probably also be declared in the bundles' package.json files though they can be omitted too and it will still work in runtime because they are being imported.

So, how does this work? The key concept behind imports is that we switch the namespace of certain packages thus pointing them to an external bundle.

So, say that you have the following code in my-toolbar portlet:

var Button = require('button');

This would be transformed to the following when run through the bundler unconfigured:

var Button = require('my-toolbar$button');

But, because we are saying that button is imported from wui-provider, it will be changed to:

var Button = require('wui-provider$button');

And also, a dependency on wui-provider$button at version ^1.0.0 will be introduced in my-toolbar's package.json file so that the loader may look for the correct version.

And that's enough, because once we require wui-provider$button at runtime, we will jump to wui-provider's context and load the subdependencies from there on, even if we are executing code from my-toolbar.

If you give it a thought, that's logical because wui-provider's modules are namespaced too and once you load a module from it, it will keep requiring wui-provider$ prefixed modules all they way down.

So, that's pretty much the motivation and implementation of bundler 2.x. Hope it will shed some light on why we needed these changes and how we are now founding the npm SDK on much more stable roots.

You can now read the last article of the series analyzing a real life example of using bundler 2.0 within an Angular portlet.

Why we need a new liferay-npm-bundler (1 of 3)

Technical Blogs June 5, 2018 By Ivan Zaera Staff

What is the problem with bundler 1.x

This is the first of a three articles series motivating and explaining the enhancements we have done to Liferay's npm bundler. You can learn more about it in its first release blog post.

How bundler 1.x works

As you all know, the bundler lets you package your JS files and npm packages inside Liferay OSGi bundles so that they can be used from portlets. The key feature is that you may use a standard npm development workflow and it will work out of the box without any need for complex deployments or setups.

To make its magic, the bundler grabs all your npm dependencies, puts them inside your OSGi bundle and transforms them as needed to be run inside portlets. Among these transformations, one of the most important is converting from CommonJS to AMD, because Liferay uses an AMD compliant loader to manage JS modules.

Once your OSGi bundle is deployed, every time a user visits one of your pages, the loader gets information about all deployed npm packages, resolves your dependency tree and loads all needed JS modules.

For example: say you have a portlet named my-portlet that once started loads a JS module called my-portlet/js/main and ultimately, that module depends on isarray npm package to do its job. That would lead to a project containing these files (among others):


package.json
    {
        "name": "my-portlet",
        "version": "1.0.0",
        "dependencies": {
            "isarray": "^1.0.0"
        }
    }
node_modules/isarray
   
package.json
        {
           "name": "isarray",
           "version": "1.0.1",
           "main": "index.js"
        }
META-INF/resources
   
view.jsp
        <aui:script require="my-portlet@1.0.0/js/main"/>
   
js
        main.js
            require('isarray', function(isarray) {
                console.log(isarray([]));
            });


Whenever you hit my-portlet's view page the loader looks for the my-portlet@1.0.0/js/main JS module and loads it. That causes main.js to be executed and when the require call is executed (note that it is the AMD require, not the CommonJS one) the loader gets information from the server, which has scanned package.json, to determine the version number of isarray package and find a suitable version among all those deployed. In this case, if only your bundle is deployed to the portal, main.js will get isarray@1.0.1, which is the version bundled inside your OSGi JAR file.

What if we deploy two portlets with shared dependencies

Now imagine that one of your colleagues creates a portlet named his-portlet which is identical to my-portlet, but because he developed it later, it bundles isarray at version 1.2.0 instead of 1.0.1. That would lead to a project containing these files (among others):


package.json
    {
        "name": "his-portlet",
        "version": "1.0.0",
        "dependencies": {
            "isarray": "^1.0.0"
        }
    }
node_modules/isarray
    package.json
        {
            "name": "isarray",
            "version": "1.2.0",
            "main": "index.js"
        }
META-INF/resources
   
view.jsp
        <aui:script require="his-portlet@1.0.0/js/main"/>
    js
       
main.js
            require('isarray', function(isarray) {
                console.log(isarray([]));
            });


In this case, whenever you hit his-portlet's view page the loader looks as before for the his-portlet@1.0.0/js/main JS module and loads it. Then the require call is executed and the loader finds a suitable version. But now something has changed because we have two versions of isarray deployed in the server:

  • 1.0.1 bundled inside my-portlet
  • 1.2.0 bundled inside his-portlet

So, which one is the loader giving to his-portlet@1.0.0/js/main? As we said, it gives the best suitable version among all deployed. That means the newest version satisfying the semantic version constraints declared in package.json. And, for the case of his-porlet that is version 1.2.0 because it satisfies semantic version constraint ^1.0.0.

Looks like everything is working like with my-porlet, doesn't it? Well, not really. Let's look at my-portlet again, now that we have two versions of isarray. In my-portlet's package.json file the semantic version constraint for isarray is ^1.0.0 too, so, what will it get?

Of course: version 1.2.0 of isarray. That is because 1.2.0 better satisfies ^1.0.0 than 1.0.1 and, in fact, it's similar to what npm would do if you rerun npm install in my-portlet as it will find a newer version in http://npmjs.com and will update it.

Also, this strategy will lead to deduplication of the isarray package and if both my-portlet and his-portlet are placed in the same page, only one copy of isarray will be loaded in the JS interpreter.

But that's perfect! What's the problem, then?

Although this looks quite nice, it has some drawbacks. One is already seen in the example: the developer of my-portlet was using isarray@1.0.1 in his local development copy when he bundled it. That means that all tests were done with that version. But then, because a colleague deployed another portlet with an updated isarray his bundle changed and decided to use a different version which, even if it is declared semantically compatible, may lead to unexpected behaviours.

Not only that, but the fact that version 1.0.1 or 1.2.0 is loaded for my-portlet is not decided in any way by the developer of my-portlet and changes depending on what is being deployed in the server.

Those drawbacks are very easy to spot, but if we look in depth, we can find two subtler problems that may lead to unstabilities and hard to diagnose bugs.

Transitive dependencies shake

Because the bundler 1.x solution decides to perform aggressive semantic version resolution, the dependencies graph of any project may lead to unexpected results depending on how semantic version constraints are declared. This is specially important for what I call framework packages, as opposed to library packages. This is not a formal definition, but I refer to framework packages when using npm packages that are supposed to call the project's code, while library packages are supposed to be called from the project's code.

When using library packages, a switch of version is not so bad, because it usually leads to using a newer (and thus more stable) version. That's the case of the isarray example above.

But when using frameworks, you usually have a bunch of packages that are supposed to cooperate together and be in specific versions. That, of course, depends on how the framework is structured and may not hold true for all of them, but it is definitely easier to have problems in a dependency graph where some subset of packages are tightly coupled than in one where every package is isolated and doesn't worry too much about the others.

Let's see an example of what I'm referring to: imagine you have a project using the Non Existing Wonderful UI Components framework (let's call it WUI). That framework is composed of 3 packages:

  1. component-core
  2. button
  3. textfield

Packages 2 and 3 depend on 1. And suppose that package 1 has a class called Widget from which Button (in package 2) and TextField (in package 3) extend. This is a usual widget based UI pattern, you get the idea. Now, let's suppose that Widget has this check somewhere in its code:

Widget.sayHelloIfYouAreAWidget = function(widget) {
    if (widget instanceof Widget) {
        console.log('Hey, I am a widget, that is wonderful!');
    }
};

The function tests if some object is extending from Widget by looking at its prototype and says something if it holds true.

Now, say that we have two portlet projects again: my-portlet and his-portlet (not the ones we were using above, but two new portlet projects that use WUI) and their dependencies are set like this:

my-portlet@1.0.0
    ➥ button 1.0.0
    ➥ textfield 1.2.0
    ➥ component-core 1.2.0

his-portlet@1.0.0
    ➥ button 1.5.0
    ➥ textfield 1.5.0
    ➥ component-core 1.5.0

In addition, the dependencies of button and textfield are set like this:

button@1.0.0
    ➥ component-core ^1.0.0

button@1.5.0
    ➥ component-core ^1.0.0

textfield@1.2.0
    ➥ component-core ^1.0.0

textfield@1.5.0
    ➥ component-core ^1.0.0

If the two portlets are created at different times, depending on what is available at http://npmjs.com, you may get the following versions after npm install:

my-portlet@1.0.0
    ➥ button 1.0.0
        ➥ component-core 1.2.0
    ➥ textfield 1.2.0
        ➥ component-core 1.2.0
    ➥ component-core 1.2.0

his-portlet@1.0.0
    ➥ button 1.5.0
        ➥ component-core 1.5.0
    ➥ textfield 1.5.0
        ➥ component-core 1.5.0
    ➥ component-core 1.5.0

This assumes that the latest version of component-core available when npm install was run in my-portlet was 1.2.0, but then it was updated and by the time that his-portlet ran npm install the latest version was 1.5.0.

What happens when we deploy my-portlet and his-portlet?

Because the platform will do aggressive deduplication you will get the following dependency graphs:

my-portlet@1.0.0
    ➥ button 1.0.0
        ➥ component-core 1.5.0 (✨ note that it gets 1.5.0 because `his-portlet` is providing it)
    ➥ textfield 1.2.0
        ➥ component-core 1.5.0 (✨ note that it gets 1.5.0 because `his-portlet` is providing it)
    ➥ component-core 1.2.0 (✨ note that the project gets 1.2.0 because it explicitly asked for it)

his-portlet@1.0.0
    ➥ button 1.5.0
        ➥ component-core 1.5.0
    ➥ textfield 1.5.0
        ➥ component-core 1.5.0
    ➥ component-core 1.5.0

We are almost there. Now imagine that both my-portlet and his-portlet do this:

var btn = new Button();
Widget.sayHelloIfYouAreAWidget(btn);

Will it work as expected in both portlets? As you may have guessed, the answer is no. It will definitely work in his-portlet but in the case of my-portlet the call to Widget.sayHelloIfYouAreAWidget won't print anything because the instanceof check will be testing a Button that extends from Widget at component-core@1.5.0 against Widget at component-core@1.2.0 (because the project code is using that version, not 1.5.0) and thus will fail.

I know this is a fairly complicated (and maybe unstable) setup that can ultimately be fixed by tweaking the framework dependencies or changing the code, but it is definitely a possible one. Not only that, but there's no way for a developer to know what is happening until he deploys the portlets and, even if a certain combination of portlets works now, it could fail after if a new portlet is deployed.

On the contrary, in a scenario where the developer was using a standard bundler like webpack or Browserify the final build would be predictable for both portlets and would work as expected, each one loading its own dependencies. The drawback would be that with standard bundlers there's no way to deduplicate and share dependencies between them.

Diverted peer dependencies

Let's see another case where the bundler 1.x cannot satisfy the build expectations. This time it's with peer dependencies. We will again use two projects named my-portlet and his-portlet with the following dependencies:

my-portlet@1.0.0
    ➥ a-library 1.0.0
    ➥ a-helper 1.0.0

his-portlet@1.0.0
    ➥ a-library 1.0.0
    ➥ a-helper 1.2.0

At the same time, we know that a-library has a peer dependency on helper ^1.0.0. That is:

a-library@1.0.0
    ➥ [peer] a-helper ^1.0.0

So, in both projects, the peer dependency is satisfied, as both a-helper 1.0.0 (in my-portlet) and 1.2.0 (in his-portlet) satisfy a-library's semantic version constraint ^1.0.0.

But now, what happens when we deploy both portlets to the server? Because the platform aggressively deduplicates, there will only be one a-library package in the system making it impossible that it depends on a-helper 1.0.0 and 1.2.0 at the same time. So the most rational decision -probably- is to make it depend on a-helper 1.2.0.

That looks OK for this case as we are satisfying semantic version constraints correctly, but we are again changing the build at runtime without any control on the developer side and that can lead to unexpected results.

However, there's a subtler scenario where the bundler doesn't know how to satisfy peer dependencies and it's when peer dependencies appear in a transitive path.

So, for example, say that we have these dependencies:

my-portlet@1.0.0
    ➥ a-library 1.0.0
    ➥ a-sub-helper 2.0.0

a-library@1.0.0
    ➥ [peer] a-helper >=1.0.0

a-helper@1.0.0
    ➥ a-sub-helper 1.0.0

Now suppose that a-library requires a-sub-helper in one of its modules. In this case, when run in Node.js, a-library receives a-sub-helper at version 2.0.0, not 1.0.0. That's because it doesn't matter that a-library peerdepends on a-helper to resolve a-sub-helper, but a-sub-helper is simply resolved from the root of the project because a-library is not declaring it as a dependency, but just relying on a-helper to provide it.

But this cannot be reproduced in Liferay, because it needs to know the semantic version constraints of each dependency package as it doesn't have any node_modules where to look up for packages. We could fix it by injecting an extra dependency in a-library's package.json to a-sub-helper 2.0.0 but that would work for this project, not for all projects deployed in the server. That is because, as we saw in the previous deployment in this same section, there's only one a-library package for everybody, but at the same time we can have several projects where a-sub-helper resolves to a different version when required from a-library.

In fact, we used this technique for Angular peer dependencies by means of the liferay-npm-bundler-plugin-inject-angular-dependencies plugin and it used to work if you only deployed one version of Angular. But things became muddier if you deployed several ones.

For these reasons, we needed a better model where the whole solution could be grown in the future. And that need led to bundler 2.x where we have -hopefully- created a solid foundation for future development of the bundler.

If you liked this, head on to the next article where we explain how bundler 2.x addresses these issues.

Modern frontend workflows in Liferay Portal

Company Blogs October 2, 2017 By Ivan Zaera Staff

Dear Liferay Developers:

We would like to share with you a new feature we have been working in very hard during the last months.

As we all know, Liferay Portal’s current support for frontend development does not fully meet the expectations of the standard frontend developer. These expectations include leveraging npm, the most used package manager, to manage dependencies in projects. This cannot be done with the current tooling unless you apply some effort and tweaks in the build process. Even after doing that, the current AMD Loader does not implement semantic versioning which may lead to problems when trying to make portlets cooperate between them.

To fix that situation we have extended the Portal, the AMD Loader, and created a set of tooling to let you use npm in your projects so that you can then deploy them to the Portal and see everything working seamlessly with little effort. 

Also, we have taken into account the modularity of the Portal from the beginning so that even portlets that don’t know each other may share npm packages with zero configuration, optimizing the performance and not overloading the browser with multiple instances of the same package.

This magic is mainly done by the new liferay-npm-bundler tool, a bundler inspired by others like Browserify or Webpack that targets Liferay Portal as the platform for deployment. 

You can find more information about the use of this new feature in the Liferay Developer Network, under this section.

Also, we have prepared some examples demonstrating the use of the most popular frameworks out there (like Angular, React, Vue.js, …) inside portlets. You can check them here.

And of course, you can hear us talking about this stuff in the upcoming events:

See you there or in the cyberspace!

Showing 4 results.