Content Creation is Not a Development Activity!

Let me say that again:

Content Creation is Not a Development Activity!

Pretty strong statement, but for for those considering Liferay as a platform it's really an important concept, one that lives in the core of Liferay.

Note: This is really a different kind of blog post for me. I prefer to keep things technical and discuss practical solutions to common problems. But I've actually seen this problem come up in many projects and I'm sure many of you have seen it too. So forgive me while I climb up on my soap box and preach for a while...

The Authors Dilemma

Since the first cave drawings, authors have been able to express themselves using basic tools to put thought to paper (or some other transferrable media). The only barriers to creating content was the availability of the basic instruments.

In the modern era, publishing became a structured process that applied to most physical print media. Whether a book, a magazine or a newspaper, the following roles are involved:

• Writer - Responsible for authoring the content.
• Editor - Approves content and edits for length, content, format, syntax and spelling.
• Illustrator - Provides supporting images if it's illustrated.
• Production Designer - Handles page layout and general design.
• Printer - Generates the final physical media.

With the invention of the world wide web, publishing on the web still needed to follow this standard process (in lieu of the printer, instead you have the web browser). After all, it is still publishing even though it's on a digital platform.

The dilemma for these roles for web publishing is that the simple, basic tools that were used in the past were replaced with complex computer languages and required software developers (and often development teams). Even the initial HTML-only pages still required computer knowledge, tools and skills, things that the average content authors don't have.

Unfortunately, adding software developers into the content authoring process brought with it the software development process.

Publishing and the Software Development Process

After a lot of pain and suffering, software development lifecycles and processes were defined and implemented to reduce or eliminate bugs. An experienced developer knows and appreciates the separate environments for development, testing, UAT and production. A development team's goal is to push the promotion of the software artifacts through each successive enviroment and eventually make it to production.

The publishing process actually overlays onto this development process without too much contention, especially given the original nature of web publishing.

Remember way back in 1995 when we first started creating web pages? I do so I guess that dates me a little. At that point in time we were creating complete HTML pages with content wrapped in <b>, <i>, <u> and (god forbid) <blink> tags. We'd connect all of the pages with simple <a> tags and had some <img> tags too. Pages were extremely simple, but more important, an HTML page contained everything.

We quickly moved away from simple HTML and started doing some JSP pages, some PHP pages, some perl, some ruby, some <insert favorite web language here>... We added CSS to style the content and JavaScript to liven it up.

Even with this shift, however, the files contained everything. Even with web applications, REST, javascript SPAs, etc. we still have the same basic concept that the files contained everything.

So with web publishing, the artifacts created by the developer could be promoted through the environments just as non-web projects. The classic publishing roles would overlay onto the environments, perhaps in a different order, but in each of the environments the different roles could effect change on the content and, on final approval, can be published on the production environment.

It's been like that for 20 years now, so long that it's practically ingrained in our enterprises that for web content creation/publishing we need to get all those environments lined up and ready for content promotion.

Enter The Liferay

So we come to Liferay with this expectation on how web content publishing works. I mean, it's been this way for 20 years, so this is the only way it can and should work. Right?

Well, wrong. Liferay actually allows the restoration of the classic publishing roles. Through existing facilities already baked into the Liferay core, a writer can author content. Through the use of workflow, editors can approve and edit content. Illustrators can upload images. Using local or remote staging, production designers can deal with page layouts and design. The printer doesn't generate physical media, but they can be responsible for publishing the staging content to the live site.

Great! We've restored the classic publishing process, so everything is puppies and flowers!

Well, not really. Liferay really sticks it to the old time web content developers and project managers because it practically eliminates the ability to promote content between environments. Liferay really doesn't allow you to create content in development and promote it to test, UAT and eventually production.

This seems to blow everyone's minds. How is a PM supposed to say "yes, the developer did the job correctly and the content is ready for promotion'? How does a developer provide a software artifact to support marking the end of a phase of the SDLC? We can't possibly do a real web project on Liferay without support for web content promotion...

The Trouble with LARs

It's usually this point where someone will say "Hey, I can export pages/content as a LAR file from one environment and then import it into another! I can force Liferay to support web content promotion..."

That's really a problem. Often it will work in a test case or for an initial short run, but LARs are really the round peg for a "web content promotion" square hole. In order to get LARs to work in this process, you really end up shoving and pushing and hammering to try to get it to work.

Here's some finer points about LARs that often need to be mentioned when discussing using them in this fashion:

• They are extremely fragile. LAR imports can break if you even look at them funny.
• Failures are reported like idiot lights on a car dashboard. The errors logged during failures are quite obtuse. They don't tell you what part of the LAR failed, they don't report what conflicts actually are, and they certainly don't suggest how you fix the problem.
• LAR imports terminate on first exception. If your LAR actually has five potential import exceptions, you're not going to know it until you complete a few rounds of export/import/analyze to work through all the issues individually.
• LARs are tightly coupled to the version of Liferay. LARs are stamped with the version of Liferay the data was exported from and will not import into a different Liferay version. And this is not a major version restriction (i.e. from 6.1 to 6.2), this is an exact version match on specific GA or SP release.
• LARs can be tightly coupled to installed plugins. Depending upon options selected during LAR export, the LAR file can actually contain data related to deployed plugins; if the plugins are not available on the target system, the import will fail. For example, exporting a page that has a calendar instance, the LAR will contain info about this; if the target Liferay does not have the calendar plugin installed, the LAR import will fail. One might expect the page to load but exclude the calendar, but it is treated as an entire failure.
• LARs do not maintain asset IDs during import. LARs actually do their work on asset names. This allows the LAR import to work by reassigning a valid surrogate ID in the system where the assets are imported. However, if any of your code refers to an asset by ID that code will likely fail in other environments as surrogate ID values will not be guaranteed to match. So if you use a simple URL to embed an image in web content using the ID, it will need to be adjusted in the target system to get it to work. These kinds of things can be tricky because there is no failure at LAR import time, it really will only be noticed by someone viewing the imported LAR content in the site.
• It is super easy to export a LAR that cannot be imported. This is really frustrating and happens a lot with structures. When you're exporting a journal article that uses a structure it is normal to include the structure in the LAR. But after you do this once, the structure would have already been loaded into the target environment (maybe). So you should stop including the structure in the LAR as this allows the import to work until, of course, you try to import into an environment that doesn't have the structure loaded...
• LAR created/modified users and times. This is valuable meta information, but if I give you a LAR that has web content that I created, it won't keep that information in your system unless I'm a user in your environment.
• Big LAR files fail to load for mysterious reasons. Often they are due to some of the previously raised points, but other times they just outright fail and don't give you a reason.

And seriously, the list of issues goes on, but I think you get the point.

Long story short, the idea that LARs can be used to support a long term process of promoting web content between environments is really doomed to failure. Those of you who are using LARs to support web content promotion, well I'm sure you've already seen some of these issues and, if you haven't yet, well then I'd say so far you have been lucky but predict that luck will run out.

LARs were designed for a specific use case, but web content promotion between environments was not one of them.

Restoring Sanity

No, LARs are not broken, so don't go opening bugs on Liferay.com to get web content promotion working.

What's broken is the view that web content creation must be treated as a development activity.

It's not, and everyone really should be happy about that. Developers are not web content creators and project managers are not editors nor designers and web content creation is not a project.

So restore sanity in your Liferay installation. Get the PMs and developers out of the web content creation business.

Set up a workflow process to allow knowledgable folks to review and approve content. Set up staging (either local or remote) to give folks time to lay things out and have them reviewed and approved. Manage all of the normal activities that a publishing process requires.

But do all of these things in production. Don't try to push down to the lower levels of a SDLC setup, and don't try to force web content creation through the promotion process as though it's a software development artifact.

Liferay spent a lot of time and effort bringing these web content creation and publication tools into Liferay so the content creation process could be freed from the grips of the IT department.

Take advantage of this freedom. It is yours...

Update 07/2023: What is Content?

This blog post has been up a long time. It still very much applies even in the latest 7.4 version.

However, there is this basic question that plagues most of us when using Liferay - what is content?

There are obvious answers to this of course. Web content articles are certainly content, blog posts are content, forum posts are content, ...

But there are many other things that Liferay considers content that most developers (including myself), would not.

A web content template, for example, is a Freemarker script. That's like all code, so certainly it should be treated like code and live in the developer's source repository, should be developed, tested, approved and promoted like any other code artifact.

This same thing can also apply to:

• Web content structures
• Widget templates
• ADTs
• Workflow definitions
• Custom fragments
• Object definitions
• Site/Page templates
• Forms
• Blueprint Recipes
• Etc

There are Liferay things which typically require some development experience/background/skills to build, and they often require testing and approval of some kind.

And yet, we have Liferay's definition for content:

Anything that is created inside of the Liferay UI and lives in the Liferay database is content.

That's right. All of the things listed above, which as a developer I would want to classify and treat as a development artifact, Liferay considers to be content. And because they are content, there is little if any support for storing in a developer source repository or promotion between environments.

The argument goes like this... Since we know that Web Content is classified as content, and we are only creating the Web Content in production, if that Web Content is dependent upon a structure and template, well those too only should be built in production. There's no benefit to creating them in our test environment because we are not creating Web Contents and promoting them from there.

Probably the one item I have the most issue with is the custom fragments. Yes you can build them in the UI and yes they get stored in the database, so per Liferay's definition this is content, but they're really all code, yeah? HTML, CSS and JS together to create the right presentation, this doesn't seem like content to me at all. Fortunately for this one, though, I can use the fragment toolkit to build the custom fragment outside of Liferay and treat it like the development activity that it is, and that's the only way I do custom fragments... But still, it is too tempting to just build it in the UI so you can make quick changes, test, more changes, ... The development cycle is much faster, but I know I'm going to struggle moving those things later on.

There are some aspects that do have some export/import support. Objects, for example, can be exported as JSON and imported into a new environment, so you can do a form of promotion, and through a Batch Client Extension, you could turn your export into a deployable artifact to automate the deployment into other environments, but there can be shortcomings to this. If, for example, you create a Form that is bound to the object definition and is used for adding new objects, you can promote your objects but you're going to struggle trying to get that form to promote and would probably end up re-creating it anyway, so why go through this headache?

The best advice that I can give you on these things is that you should just follow Liferay's lead and treat them like content . Build your templates in prod. Define your objects in prod. If you're not going to use the fragment toolkit to build custom fragments, then only build custom fragments in prod.

Of course, enable staging so that you have a separation between your live data and your staging data...

But continue to treat these things as content. Even though, as a developer, you will disagree that these items are content, to Liferay they absolutely still are.

Trust me, you'll have a much easier time if you embrace this rule than if you try and go against the grain...

The Exception

There is one exception though that I want to share - the Site Initializers...

• https://liferay.dev/blogs/-/blogs/introducing-site-initializers
• https://liferay.dev/blogs/-/blogs/site-initializers-101

As a quick review, a Site Initializer is a developer artifact that can contain basically everything from the list above, but those files can be stored in your source repository, and they can be built and deployed and promoted between environments.

So it solves all of our "this is really code" concerns.

But there are two things to keep in mind:

First, you have to plan for this. You can't get up one day and say "Hmm, I'll take my site and create an initializer out of it and then start promoting it" (although I did find an interesting project that I haven't tested, https://npm.io/package/generator-liferay-site-initializer). It can take a lot of work to extract your assets, transform as necessary, add to a site initializer project, build, deploy and test it. If you plan for this from the start and update it as you build your site out and continually test it, it will be a much easier path.

Second, for most Liferay versions a Site Initializer is only good for creating a new site. Doesn't work so well when you have a site and just want to update it. However, later versions of 7.4 support updating an existing site from a Site Initializer. Currently it is hidden behind a feature flag to use right now, but expect it to be fully available in a future bundle. Read more about it here: https://liferay.dev/blogs/-/blogs/site-initializers-update-support.

[Bob Ward]

We have been very interested in this topic as it directly relates to IT trying to accommodate our Marketing dept requirements to: 1. QA our www site with all the content in place (execs can't review with ipsum lorem content) 2. not recreate content on more than one environment

On the IT side we don't want to keep content in sync between environments as this is more work.

We would completely buy into the concept of using our PROD staging site as the QA location but we have a challenge and that is our design approach frequently requires a theme deployment in order to accommodate new pages and/or template updates. This is prohibiting the flexibility we want to have in using the staging site.

I have been told we included most of the CSS and javascript in the theme in order to consolidate code and improve performance. We have about 15 templates on the main site ... reengineering them to gain the flexibility we want may be a lot.

We would be interested in any thoughts or suggestions .. IT would certainly like to give Marketing the flexibility and QA ability but we can't risk frequent theme deployments to PROD. Those deployments need to be controlled and monitored by IT.

[David H Nebinger]

That is a tough one, Bob...

Normally you'd see the theme get created and deployed, but afterwards the content creation would be based upon the theme.

Sure the theme would need to be tweaked from time to time but one would expect some stability of the theme over time.

That of course is probably the 80% of the 80/20 rule, and it sounds like you're in the 20% category.

Would it be possible to take advantage of some of the other styling support Liferay offers? Look and feel/advanced tab for the web content display or embedded styles within the content article itself?

[Traolly Xiong]

Hello David,
Having the CSS / JS in the theme improves the performance on our site in many ways (i.e. merging CSS, combining JS, minification, compression, cacheable of external resources on CDNs and locally, lighter page sizes, fewer HTTP requests in which will allow more browser parallel connections and mitigate blocking other requests, etc.). The templates will mainly just have HTML markup and Freemarker code with little embedded CSS / JS for unique use cases. Doing this would also almost eliminate duplicated CSS / JS code in templates. For a global company with a rich and robust site that is also responsive, users would find this design more beneficial especially for users with a slower internet / mobile network connection.

With that said, I had a few questions if you don't mind.
1) If we do move the CSS / JS from theme into the web content display portlet config "look/feel", it would seem that we would have to add it each and every portlet as they are unique even if when they rendering the same content. That would be tedious work, correct? Would you agree if keeping them in sync would also be harder to manage? I would think the same would apply to the web content itself approach as well.
2) If we do move the CSS / JS from the theme to the portal level (templates, portlet config, web content itself, portal pages overrides, etc.), how do we avoid duplicate / redundant CSS / JS as that wouldn't allow us to thin out the page size or cache the CSS / JS, and potentially cause js conflicts?
3) Part of the 80/20 you mentioned, how would you best describe their (80%) development process with the minimal theme updates / deployments
and also have a good front-end performance experience?

Let us know your wise thoughts of if you need any more info.

Thanks in advance.

Traolly

[David H Nebinger]

Hey, Traolly.

Certainly you want to do your styling in the theme because it has those advantages and also promotes uniformity within your content.

I was proposing it for Bob only because his group would be stuck waiting for IT to deliver theme changes and have to do a coordinated release. In his situation it may make more sense to look at an alternate approach than straight theming. Even if they go about putting styling in the content or page, I would still suggest sending it to IT to integrate in the theme. When (if) they do the next theme deployment, the styling can be shared with content delivered later.

I am certainly not recommending that we all stop using a theme for the styling, putting the styling into the page or the content for most of us is definitely the wrong thing to do.

All of your criticism of this is spot on. I'm sure if we put our heads together we could knock out a few more ways in which moving styling out of the theme would prove problematic.

This was a suggestion really to help Bob separate the content creation from the development process. Sounds like he's got a fast-moving marketing group that needs to set up and maintain content that would change often, far more often than an IT project cycle for theme deployment. In his situation it may be better for him to allow for duplicate or redundant styling, etc. If that's the price to pay for being able to keep up with their necessary site changes, well then it is what it is.

For the 80% of us (well, it's probably a lot more than 80% but that doesn't have a nice rule like the 80/20 one), we're going to engage our theme folks early in the site process. They'll lay out the design, create all of the styling and hand a list of classes to the content folks so they can decorate the content appropriately.

Most of the time the theme gets done and it will have minor changes afterwards, oddball things to make a piece of content work or a tweak for a browser issue, etc. But in general the theme should work until your next big redesign (in some cases that redesign will never come, so don't hold your breath waiting for it).

So in this slot, the folks doing the various roles of web content publication can leverage the inline editor, the workflow approval process, and staging (local or remote) to happily complete the publication process.

[Traolly Xiong]

As always thanks for the response.

We may just have to prioritize what we really want, and adjust our design and process that will make all parties content. There's going to be trade offs regardless of the approach and we just have to find the middle ground.

Thanks.

[Adnan Yaqoob]

David,
To my knowledge it is a good practice to have staging server for content authoring. On staging server your content editor create/modify the contents. The whole site can be simulated with content and any custom development including themes.

If everything looks good the publisher cane push the content to production site ( in Liferay using Publish to Live).

Are you suggesting to use Production server directly for content editing? if yes wouldn't it effect performance on production?

Generally Production servers are not use for content authoring, please suggest.

[David H Nebinger]

Certainly not, Staging is definitely appropriate for that activity.

This was a post arguing in favor of using staging for your content creation bound to your production instance.

It was in response to a post in the forums that was about, IIRC, using LARs to export content created in a dev environment and pushing through to prod and how they were having issues getting the LARs to import correctly in each environment.

If you're using staging, either local or remote, for your content creation and then publishing live to your production site, you are spot on with your process, doing things the Liferay way.

[Adnan Yaqoob]

Thanks for clarification.

In fact I asked the question because I'm in favor doing what I mentioned above. But few fellows who don't know much about content authoring and the standard practices want to permit content authoring directly on production instance. They believe the CMS servers should be able to handle or what's the purpose using CMS.

Please advice, if my approach is wrong or ? what will be the performance impact. If Liferay had any document/link suggesting the best practices to use Staging for content authoring and publishing that can help

[David H Nebinger]

Staging allows you to see a whole page as it will be rendered, prior to publication. Workflow on a web content article allows you to approve individual content, but won't allow you to get a full view until you publish.

Staging also allows you to deal with publishing a full page instead of individual approvals.

As far as performance goes, well there really shouldn't be a performance impact, at least not a noticeable one. Users, when they access the site, will see the published content and your approvers will see the staged content.

Whether to use staging or not, well that should depend upon your organizational requirements. If you have little if any web content or you're not worried about approving and previewing, then staging and workflow are unnecessary overkill. Sometimes workflow is all you need, sometimes just staging, And sometimes neither.

[Fernando Fernandez]

"LARs are not broken".

I beg to differ. :-) Exporting and importing websites is, de facto, a missing feature from Liferay, since ever. Not because it doesn't exist, but because it's so broken it's barely usable.

As a Liferay user, it's unacceptable that I cannot export a working website to a different instance of the same Liferay version.

As you stated, LAR export/import is full of problems that will make it unusable in many situations. 

For starters, I think this should be worked on:

- Invalid data errors should be handled at export-time, not import-time; tools should be available to detect and solve data errors preemptively

- Errors should be made readable by a business user trained in content management; no developer should be needed to decode messages or analyse the DB

- Dependencies from stuff outside the site should be clearly listed on an export report: content from other websites, users, roles, global site components, etc.; tools should be made available to export and import those too

- Import should not give up on first error: it should try to import as much as possible; an half-imported website might be better than nothing, since there are not many alternatives

[David H Nebinger]

They're not broken, though, they are just extremely fragile.

If you take care when building your site _not_ to depend on resources from the global site or from any other site, and if you take care when exporting your LAR file to include everything that will be needed, you _can_ be successful importing into another instance that is the _exact same version_ and _exact same configuration_ and _exact same customizations_.

The problem is that most of us do not take that care. There are benefits of using a global scope. There are benefits to leveraging things from other sites. And it's hard to match version (down to the specifically deployed hotfix) or configuration (i.e. the same languages set up, etc).

Broken implies that it would _never_ work. But it does work under a specific set of conditions. Your car isn't broken if it can't drive through a river to get to a town on the other side, it's just not designed to support that.

And that's really the crux of the problem. LARs work for the use cases they were designed to support, but they weren't designed to handle many of the use cases we might want to use them for.

For some of your suggestions, though, they're not really possible. Such as invalid data, when the export happens it isn't clear that the data is invalid at all. A reference can be part of the export, but the system cannot know that the reference is satisfiable on the target site or not. That's why those things happen at import - only then can you determine if data is invalid or not.

The error messages, yeah I'm with you on that one. They're the same kind of messages you get when staging fails and they aren't all that helpful either, but at the same time, useful messages can be hard to render during import. Take a reference to a file that might be on "/folder/folder/folder/Fancy Name" from the source side; the export doesn't include all of those details, just the reference (basically the UUID to match to, maybe the file name). On import, the system will determine that the file is missing, but it can't give you much useful detail. It doesn't know that the filename that it may have is under some specific path on the source side, so often times it is missing enough context to make for a useful error message anyway.

External dependencies? Yeah, those are tough. Say you have a global structure X. Sure you could export site Y and maybe include structure X, but problems come if you try to load into an instance where there is already a structure X in the global site. How do you reconcile that? Is it a new version? What happens to content in that instance that is using the current X, how could it handle a new X that might be completely different or even exactly the same?

These are the kinds of problems that are at the core of the export/import problem, and many of them are issues with missing/incomplete context or conflict resolution.

Both of these are really, really tough problems to solve and you'll likely never get one solution that works for everyone.

But that brings us back to the content of this blog post - Liferay isn't designed to support content/site promotion, copying, export/import, etc. And rather than trying to force it and fight it, instead you should ditch that idea and embrace the tools that Liferay does support, including staging and now publications. Build your content in the prod environment in staging or publications, publish it to live when it is ready. Liferay _is_ designed to handle this process, so if you embrace it you'll have a much better time with Liferay than if you try and fight it with export/import attempts.

[Fernando Fernandez]

I understand your arguments and also understand the product strategy, but I cannot agree. Still, for all the difficulties in my proposals, there's at least a couple I think should be easy to solve: 1. import should not give up upon first error, and should try to recover as much as possible from the LAR file

2. errors should be made easier to read and the import process should try to help the common user understand what's wrong These, by themselves, would make a big difference in the usefulness of LARs.