YUI, Liferay, and the future

As many of you may have read, Yahoo is immediately stopping development on the YUI library.

This decision, while it will have a significant impact, is not news to us. Given our close relationship with the YUI team, we knew this was coming, and have been discussing for a couple of months different plans of action.

In the spirit of transparency, I do want to say that we don't yet have an official direction decided, but we are looking at all of our options, as well as discussing with our partners, and other companies/organizations with a vested interest in keeping YUI alive.

I'd like to address two things in this blog post: what this means for existing EE customers, and what options are we looking towards.

 

What does this mean for existing EE customers?

Nothing will change for our existing customers. We have long had a fork of YUI that we apply our patches and changes to, and we will continue to deliver bug fixes and needed changes for as long as that version of EE is supported.

 

What options are we looking towards?

I'd like to list off a few options that we're discussing, partially to reinforce that we are seriously thinking about it, but also to solicit feedback and ideas regarding your needs or concerns.

Of course, the most obvious possibility is to take over stewardship of YUI, whether that is us on our own, or with any number of the other large companies that leverage YUI heavily.

Another possibility is to of course migrate off of YUI, and while still keeping AlloyUI as the wrapping library, but move to some other library internally, and still keep the functionality as close as possible.

One more idea we've discussed is to possibly take our fork of YUI, and branch off a next generation version, like a YUI4, that cleans out the legacy code, streamlining and simplifying the library.

Ultimately, we want to take the path that best serves our community and clients, and helps deliver amazing experiences as quickly and easily as possible.

As we get more info, and have more info to provide, we'll make sure to keep you in the loop on the path we're going to go with.

If you have ideas, thoughts, concerns or questions, don't hesitate to let us know.

Blogs
I think they should move to a cooler framework. As Angular for example. YUI is great framework, but the problem of YUI always has been the learning curve. I am working with YUI since 5 years ago, and when I tested Angular or Knockout, for example it's another level. You only need to see the github stars and you can find the answer. Best Regards, Dariel, another YUI fan.
I think there might be quite a lot of community support for a YUI4 path. In my view, offering a consistent philosophy and API is an enormous benefit, which is lost with the 'lots of little packages strung together'. To me, YUI has also always developed and embodied various good practices that are not always present in other libs. (And evaluating this adds overhead).

I hope you're separately reaching out to those fairly well known other big users, but I can definitely see a place (and would want to contribute to personally) a YUI4 that
- removed as much of the old browser workarounds as possible
- maintained the commitment to quality, testing, documentation
- maintained a fairly consistent API across the library

And thanks, btw!
I personally think AngularJS will eventually lose out to frameworks such as React.js. It's popular now, but imho the writing is already on the wall. I think YUI is still a great framework but there are a lot of smaller frameworks that do stuff that YUI does also, and sometimes better, e.g. hammer.js, require.js. However, YUI still offers a solid integrated concept. So, I think a manageable direction would be to swap out the parts of YUI that are maintained by popular alternatives. I think there's also a lot of good work done with Promises by Marco Asbreuk.

I think a worthwhile discussion might be to talk about the parts of YUI that are available in the wild. YUI4 could act as a wrapper around these functionalities, easing the amount of maintenance required.

I think the App framework in YUI never really took off. So, it would make sense to replace that with something else, again looking at the best of breed in the wild. Ideally, this would be pluggable, so one could use stuff like React or Knockout as one desires.

The big complaint about YUI was that it was monolithic. It certainly is a big library. The only way to keep it alive is to keep the core (node, event loader, io, widget, plugin etc), but wrap and replace as much as possible with community material.

Also, documenting the way one can extend YUI with certain functionalities that are not native would make sense. I'm disappointed that Yahoo has just completely dropped everything and not taken steps in this direction.
Hi Marc,
Thanks for writing. There's a lot I agree with in your post (I too see a stronger future for React than Angular, mainly in that React tries to solve a smaller problem and integrates fairly easily with other libraries, among other things).

One area I would probably disagree would be that require.js did it's job better than YUI's loader did.

I would say it's benefit, of course, is supporting the AMD module definition (or even if they supported CommonJS), whereas YUI's only really ever supported their own, and more confusing, module format.

But one thing I haven't been able to find anywhere, and probably one thing I would want to make sure we maintain going forward, is the combo-handling aspect of the loader.

Every other system I've seen has some sort of build-time optimization script that will combine everything into a single file. But for apps that live in a portal, this doesn't really work because the app doesn't know at build time what the environment will look like at runtime, so there's a pretty high likelihood that there will be a LOT of duplicate data being transferred for those apps.

I could have missed something that handles this currently, or maybe going forward, writing this sort of dynamic combo-handling for require.js or equivalent is one thing we can contribute, but that is something I think we definitely should keep going forward.

Thanks for your thoughts, it's great to get this kind of feedback emoticon
From a portlet development perspective, I am looking for something closer to Angular than React or than the standard YUI. Usually, I use taglibs AUI, LIFERAY-UI and others to construct my views. After that I create controllers for my application logics which are bind to my HTML components.
I create a little framework on top of AlloyUI that monitors my components for event handling, transforming a portlet to full-ajax without change in the JSP code and support Liferay form validation, remove all JS code in JSPs. I use a similar concept like in Angular so I can have a controller per portlet or portion of the portlet. I also have directives like in Angular.
1

2
<a href="#"
3
      data-load-new-popup="1"
4
      data-popup="clientSelectorPop"
5
      data-ajax-load-disabled="1"
6
      data-url="#"
7
      data-url-preprocess="triggerEntrepriseChange">
8
   <span class="taglib-text " >
9
       Select
10
   </span>
11
</a>
I think trying to continue YUI is a mistake for all the reasons in the yahoo announcement. I understand the pain for people that have a lot of code in YUI. But, as Yahoo points out, the world is changing and frameworks like YUI (and to some extent jQuery) are not the current trajectory. Hanging on to older technology just means the pain will be that much greater when you finally have to move. My vote is for angular. It's the first js library since jQuery that I've felt like has legs. I also think the concept of controllers and views maps well into the portlet architecture. Just my two cents
Thanks everyone for all of the replies and ideas.

I'm actually curious for those who have said they think AngularJS is the direction we should go, how extensive is your experience with it? Has anyone created anything beyond a fairly simple app or "Hello world" demo?

The reason I ask is because for me, Angular, while popular, has always seemed like a ton of abstraction and complexity, with some benefits and some drawbacks.

I won't lie, I really want to like Angular, mainly because of the data binding portion, but the fragile dependency injection, the heavy client side rendering portion, and the added complexity of concepts like Proxies, Services, Factories, etc all seem like downsides for a lot of applications.

So I'm curious how people are using it, and how much it's helping people solve their problems, beyond just how popular it may be currently.

That's not to completely rule it out, but Angular is a VERY opinionated framework, and I would want to make sure that it solves our community's needs first and foremost.
Hi Nate,
Keep in mind that Ember is also extremely opinionated (based on everything I've read/heard about it). Since Liferay (and alloy) tended to be somewhat opinionated, I was wondering if you've considered giving Backbone (and underscore) a thought. Backbone is one of the least opinionated, in addition to having a great community. Also, it's a bit more established than Angular. IDK, I'd say Angular would be a be risk, and the reward would be, sexyness?? Backbone has databinding support, just not OOTB.
Keeping/forking YUI and evolving it would be OK for me. Choosing an new existing framework that is mature, popular, easy to use/understand and thoroughly documented would also do, but for the love of FSM do not create another framework of your own.
I've done my fair share of different projects on Liferay and from my experience I would like to point out that:

- Every time I've tried my best to build my portlets and applications fully on AUI() alone I end up spending 2 - 4x more time trying to get things working partly because perhaps I don't know the API by head and partly because there's just not enough working examples for everything one could copy-paste easily and fiddle with. This applies mostly for AUI() scripting part as taglib is working fine but sometimes lacks some built-in features forcing us to apply extra code. A simple auto-complete that uses service-builder /api/jsonws/ as datasource for example requires rather extensive amount of javascript for example.

- Almost every time I end up adding jQuery to the mix, mostly not because of jQuery's core features but rather to use some plugin written on jQuery / jQueryUI.

- For the future I really hope you either migrate off of YUI in favor of jQuery or AngularJS or you take over maintaining YUI completely and vastly improve it.

- Either way I personally think the amount of documentation and examples for AUI taglib and specifically AUI() scripting inside Liferay is not enough - There should be more examples. And perhaps more tags in the AUI taglib emoticon

PS: Regarding AngularJS - I love it, it's awesome and one day I think it will overtake jQuery BUT it's a whole framework not just a library and integrating that into the Portlet world is a little bit trickier.
Sorry you can delete the previous comment. I work for a company where we're at a cross roads as well with YUI. We're asking much of the same questions that have been asked here. However, our answer is somewhat dependant on whether or not Liferay will continue to build on YUI 3. With that said, are there any timelines where we could expect to see an answer in either direction? Much appreciated!
First, let me make clear my comments are solely my own, and do not represent the view of anyone else.

I would strongly urge you to avoid adopting YUI, unless you're able to get Liferay or a consortium of YUI users to commit the necessary resources. Committing resources not only means maintaining AUI on top of YUI, but also maintaining the YUI core across new and ever changing browsers, fixing the rough edges, answering forum questions in a timely manner, and providing meaningful and useful documentation, not just sparse examples. This even means fixing the alloyui.com site so it's usable for prior versions. (Try clicking on the version on the header - it reverts to 2.5 as soon as you go to another page, and the API pages don't give you any option except 2.5).

Before posting this comment, I wanted to make sure I was being fair and not basing my comments on prior experiences. While I do have experience with various web frontends (Struts, JSP, JSF, IcesFaces, PrimeFaces, JQuery, AngularJS), I don't spend hours a day (or even a month) in this sort of work. So I created a fresh project under Liferay 6.2GA2 CE, and tried creating a portlet using IO, Node, Data Tables, and Modal from AlloyUI.

The resulting prototype required more time and pain than I expected. While I was able to bring up the basics in a few days, I estimate it would take about the same amount of time to add the functionality I wanted (like a checkbox in the Data Table) and smoothing off the rough edges. I ran into conflicting examples and examples that didn't work (even when matched with AlloyUI 2.0 under 6.2GA2). And when I searched the CE source for Modal examples, I found only two examples, including one that was extending the base class to provide a utility used by the rest of Liferay. Overall, the prototype took longer than I expected from modern tools, and had both problems and unanswered questions.

So please, either fully commit to supporting something that the unwashed masses like me can use effectively, or adopt something else that you will not have to support and maintain. Adding YUI support to what you're already trying to support will not improve the situation.
DO NOT MAINTAIN/FORK YUI!

I am totally with Bob on this one...

Maintaining a complete framework i.e YUI3 or a YUI4 fork takes a huge amount of resources, resources that I believe those of us on the outside would prefer were allocated towards portal expansion, not JS framework maintenance.

Each new browser patch release, most of whom are typically self-installed these days, would bring a new round of compatibility testing, bugfixes, etc., and we would have no place to go for fixes except Liferay. There's already a backlog of YUI bugs, are you going to take them over or just ignore them because they haven't been hit by the AUI team? How are you going to add new functionality if you're busy fixing and testing the old?

And how about browser support in general? YUI had a larger team and a bigger desire to be compatible with a wider range of browsers than what Liferay may like. Would you then be publishing a list of supported browsers and the rest be darned?

The biggest message that you should take from Yahoo's announcement isn't that their project was a failure, or didn't do what it was supposed to, etc. The biggest message that you should hear is that, for as large as Yahoo still is, they didn't feel it was economical to continue development of this framework. I would argue that if it is not economical for them, neither is it economical for Liferay.

Honestly, Nate, you guys doing the AUI javascript stuff are really bright, and I give you all kinds of props for the work that you've done, and I wish I knew half as much JS that you and your team. Unfortunately, from the outside there are huge complaints with lack of doco, lack of support, tough learning curve, lack of experienced resources to hire from (how many developers outside of yahoo have YUI experience, I certainly don't know of any), lack of suitable release cycle (Liferay can do a new CE release every 6 mo or so, but I can't see that a javascript framework can survive on a similar release plan), lack of a source for JS updates outside of Liferay (i.e. in jquery we could just download and use a newer jquery script) ... If Liferay did take over stewardship of YUI or some YUI fork, we have no reason to believe any of these things would improve.

Outside of Liferay, we're still using jquery. We never gave up on it. It's got a solid history, lots of experienced resources to pick from, tons of documentation and blogs and examples across the net, and honestly most of us have never run into the issues that prevented jquery from working well within a portlet.

I've always maintained to folks on the outside that for most of their complaints, they must understand that the concerns of the portal are often times very different than theirs, but that doesn't make them wrong. But the flip side of this is that sometimes it seems that the concerns from the outside don't seem to matter (not saying this is true because I don't believe it is, but this is sometimes the impression).

Personally, I would advocate for the following:

1. A thin layer for Liferay on top of an externally-maintained JS lib (perhaps more willing to accept downstream changes than jquery was). Should be an industry standard that either already has been established or is gaining acceptance and momentum (i.e. Angular seems to have much more momentum than Ember, for example).
2. Whatever is created is fully documented and examples (outside of LR source) given.
3. Supports a faster release cycle so we can get JS fixes sooner than the next CE release.

Whether jQuery or Angular or any of the others mentioned in the comments satisfies this list I'll leave to others to answer. Ultimately my concern is being able to create portlets quickly using standard tools, languages, and processes. The more non-standard the environment becomes, the more time it will take to create portlets, train and retain resources, etc.
This is solid advice.

If you stay with YUI/AlloyUI you need to take over all the necessary responsibilities and step up your game because you won't be solely operating in a small niche that you largely control anymore. So it might be a valid option, but not one I see Liferay executing successfully with the history it has and I've seen through the years.

What I'd rather see is what David tackles in the last part of his comment: choose an existing, well known, widely used and established framework and commit code to it that adapts it for use inside a portal. This way Liferay's involvement remains smaller and more sustainable. It's a bit like JSF and the JSF portal bridge where Liferay focuses on the adapting/bridging an established framework like JSF to work within Liferay, but didn't invent the JSF part itself (but now does help evolve the standard via the JSR).
Hi David,
Thanks for your thoughtful reply and ideas. I don't necessarily disagree with you, and I would not want us to take over the project if we couldn't execute on it properly.

Moving back to jQuery is definitely a contender for the options, as in the time since 2009 when we left, they've definitely addressed many of our previous concerns, and the modularity that YUI helped influence has now become much more widespread (though the jQuery plugin ecosystem still has most of the same issues, but our team has grown enough to where we can avoid most of those issues ourselves).

There are a couple of areas that you mentioned that I would like to address, more to communicate some of the issues we've found than to argue them, as perhaps you have some thoughts or ideas as well. But if not, they at least might as well give some context into how we're approaching it.

1. "A thin layer for Liferay on top of an externally-maintained JS lib"
This, of course, is the ideal. I've found that no matter what, there are always some portal concerns that upstream library maintainers will never accept. Hopefully the library is extensible enough to allow customizations or extensions to work around it, or add the functionality needed. Unfortunately, this isn't always possible, or if it is, the cost is to keep increasing the size of the JS files by adding code to work around it.

2. "Whatever is created is fully documented and examples"
I agree 100%, and this is something that while we've been improving, we can do a much better job.

3. "Supports a faster release cycle so we can get JS fixes sooner than the next CE release."
This is part of the reason we encourage people to use EE, which is that we will provide fixes back before the CE version.
However, this one I would *love* to find a way to allow this, but none of us have been able to find the perfect solution that everyone seems to want.
Here's what I mean:
What I've found with *any* JS library (and actually this is far more often with the type of library you described in point 1, with it being considered an industry standard), is that you can include version X of said library in the product, and either in-between CE releases, or over the course of an EE release, people will want to upgrade said library.
The issue with doing that though is that there may be breaking changes in those versions that other applications (Liferay apps or third-party apps) rely on.
For instance, you can look at the changes that were added to jQuery 1.9: http://jquery.com/upgrade-guide/1.9/#changes-of-note-in-jquery-1-9
(I'm using jQuery as an example, but this equally applies to any library).
You'll notice that certain methods were removed, changed, etc.
The jQuery Migrate plugin would help alleviate some of that, but what happens when you have 1 application or component relying on the older API, and perhaps including a new plugin relying on the new API?
Whats worse is when they exist on the same page, which just ends up throwing an error somewhere.

The only other alternative, and it's not any better, is to allow some way for multiple versions of that library to be loaded (usually via namespacing), but this is not great either because you can easily start doubling or tripling the size of the JS needed.

Overall, there seems to be an iron triangle of concerns in the sense of you can pick any two of the following:
1. Having a standard library included
2. Have it download fast
3. Being able to upgrade/use any library you like

But I have yet to find a way that you can have all 3.

I'm not saying there doesn't exist a way, but I haven't been able to find it, but I would love to hear any ideas emoticon

Again, I don't say all of that to be stand-offish or defensive, but more or less to say that whatever we choose will probably involve some tradeoffs that some people may not be thrilled about.
But, of course, we're definitely going to do everything we can to limit those negative tradeoffs.

Thanks again David for your reply and thoughts emoticon

PS (and thanks everyone else who has commented but I haven't had a chance to respond individually to).
Thanks for the update, Nate. I don't want to sound like a jquery zealot (I'm not). Whether jquery, angular, or whatever, I'll just be happy as long as it is an industry 'standard' and not a Liferay-only solution.

If you take over stewardship of YUI or just a fork, invariably it's going to become a Liferay-only solution. I know that originally AUI was supposed to be used even for non-portal projects; even though that field of dreams was built, I'm not aware of anyone who's come (who is not using Liferay). So this would become a liferay-only path; I wouldn't be able to hire and use a web guy w/o first getting them trained on the LR-only javascript stuff. LR would need to build a big YUI/AUI support team for cross-browser testing, releases, etc, and even if it was 'open source', it would have the same eyes on it that LR gets, so general JS folks would not necessarily get involved.

The great part about jquery was the wide array of non-LR doco, developers, and examples. Some examples would or wouldn't work in the portal, but we had so many different examples there were plenty of things that we, as developers, could try until we got things right. And there were non-LR plugins that we could easily adapt and use and pick from, even with varying degrees of success. Don't like tooltip plugin X, well then give Y or Z a try. We could do google searches and find hits on jquery.org, stack exchange, etc. that may or may not be LR-specific, but may offer a glimmer of hope for the portal-based issue that we were facing.

I don't know that YUI ever had the kind of adoption jquery had (there are some YUI books, but never the coverage jQuery got), but the available pool of examples, resources, and doco never seemed on par.

So, as I said I am not a jQuery zealot. Comparatively, though, it seems to me that I'm better off if Liferay uses jQuery than AUI, at least from a developer perspective. Even if it was a tweaked jquery-min.js that had some portal-specific overrides, I'm still better off than if Liferay continues with AUI. Similar evaluations of Angular or React or <insert framework here> vs AUI tend to go against AUI for all the same reasons.
Hi Nate!
My 2cents are about older code we had already written in Alloy and products that we had released to our customers in the past months.. How we do with them if LR discontinued the use of YUI / AlloyUI framework?

And what about the learning curve that we (and our customers, that are software houses too) had already payed to play with the new JS framework of Liferay?

Surely, when you guys switched from the old JQuery to the new Alloy paradigm for us was a great pain.. But now we had already known how to manage the "complexity" of the framework and with few lines of code we can create UI plugins more faster and stable than the JQuery versions were..

Generally speaking, changin' the JS framework is always a pain for all the developers: new code paradigms, new API, new cross browser compatibility pains.. As do as on the server side code if you switch from a technology to another.

First write this post my ideal situation was that LR takes the ownership of YUI framework and manage to release a new version of it integrating the Alloy widgets inside the YUI core..

After I've read David & Bob posts, I'm really confusing about what the right direction could be..

I've at least one more point to add to discussion: Angular (or another JS framework we choose) implies for us (developer, mantainers of software solutions based on LR stack -you always rock guys! ;)-) a new learning curve, a new paradigm shift and a new challange.. That we had already done one year ago..

We can choose a new UI framework but please ;), let be the last one for 3 years in the future! emoticon

We cannot reinvent the wheel every time a new LR release is launched on the market; is too expensive for us and is too expensive for our customers..

JQuery had a lot of troubles itself, but it still used, mantained, documented and adopted on the web since a lot of time.. Are we sure that the KISS paradigm is not good enought for us too?? emoticon

Thanks for your update and for discuss with us the future of LR; again: you rock guys! emoticon

Jed
I'd prefer a switch to JSF based GUIs ;). The code would be much more cleaner and the separation of the model, the view and the controller too. Now in the jsps you find JS, HTML and java code. Modifications would be easier because if you want to change the display you would change the xhtml and if you want to change what is getting displayed you would change the controller. This would probably be a huge change but the benefits are worth of it - at least in my eyes emoticon.
Nate, one of the constraints you mentioned earlier ("Being able to upgrade/use any library you like") may not really be that significant. Sure, JavaScript libraries may be updated several times a year to fix bugs and add features. But that doesn't mean they'll automatically work with Liferay or AlloyUI. Liferay, like any other large software system, is built and tested with a particular set of libraries and components. You can't just randomly drop in a new component like Hibernate, Lucene, or AlloyUI and expect things to work. There are certain assumptions built into the code based on the components used, and if you update a component you have to update the code based on the interface presented by that component (OSGI may change some of this, but it's probably not important for this discussion).

So what do you do for the pioneers that ignore your warnings that there is danger on the other side of the hill? First off, you probably make clear that there is no official support for changing the "official" JavaScript library, other than the good will of those in the forums. There is already precedence for that approach, even when using EE. Secondly, you might consider making it a little easier to replace the JavaScript library in the two common scenarios.

The two scenarios where people change JavaScript libraries are local ("just this portlet") and global. In the local scenario, changing the library is a lot easier because of closures, noConflict, and the fact there is a lot less code to deal with. For the global scenario, you could introduce a mechanism that makes it easier to change the library used by Liferay; say, by putting the library info in a property file. The pioneers can take this and will either come back with something magnificent or will never be heard from again. It's not your problem in either case. I'd like to see you address other issues instead.