Liferay Documentation Survey Results

This year, the Knowledge Management team at Liferay set out not only to document Liferay 7/DXP, but to document it well. To do this, we identified several improvements we wanted to make:

  • We wanted to connect powerfully with users and customers. To do this, we made a significant effort toward documenting the right things, the things people needed the most to learn about. We also did what we could to promote the documentation, through our @Liferaydocs twitter account, being active on the forums, and responding to feedback.
  • We worked to improve the quality of the docs. We divided ourselves up according to Engineering’s functional teams. We worked closely with developers to produce docs not from the perspective of outsiders trying to figure out the product, but from the inside, describing how the product is designed to work, putting the most important features front and center.
  • We collaborated cross-functionally with other groups like Liferay’s Support team. This helped us improve the docs by identifying and answering the questions users and customers asked us about.
  • Finally, we looked for ways to broaden our communication services and skills. Realizing we have someone trained in CGI, a musician, and a voice-over artist on the team, we started enhancing several tutorials with videos. Look for more of these in the future.

But it wasn’t enough to try all these things, pat ourselves on the back, and say we took Liferay’s documentation to the next level. We care deeply about Liferay developers and their experience with the platform. The only way to really know if we were succeeding was to ask. So we took to Liferay’s Twitter, community chat, and internal systems and asked a whole bunch of questions to help us determine the state of the docs.

Here’s what we learned.

Survey Population

Part of the survey asked about the people filling it out: how long have you worked with Liferay; how often do you code on Liferay’s platform, etc. We wanted to know who we were reaching and what their needs are.

First, the experience level: how long have responders been working with Liferay’s development platform?

The majority of responders are experienced developers. More than half have worked with Liferay’s platform for 5 or more years. Interestingly, though, we have almost as many developers new to the platform as have been working with it for a long time.

Next question: how often have you used Liferay’s documentation?

About two thirds say they use the documentation regularly. This means we have well-informed responders who have regularly searched for and used Liferay’s documentation. Only six said they rarely used the documentation.

Next question: how often do you code on Liferay’s platform?

Again, the majority of responders regularly write code for Liferay’s development platform. This is good: it means most responders refer to the documentation on a regular basis.

Next, we wanted to find out how many people from Liferay responded versus customers and those from the open source community.

We were hoping for more of an equal distribution between developers employed by Liferay and those from the community, so this gives us our first take-away from the survey: how can we better get feedback from the community and our customers? Clearly, the way we went about it didn’t work well.

Reference Docs

There were several questions that examined Liferay’s reference documentation. Let’s dive into those responses.

First, we wanted to ascertain if people were using the reference documentation.

Developers are equally divided between Hourly/Daily/Weekly and Occasionally/Never. We’ll get to why next.

We then asked if developers could find what they wanted in the reference documentation.

Overwhelmingly, developers were either neutral or disagreed with this statement. The plain fact is that Liferay has struggled to document our APIs satisfactorily. While we have made great strides, more needs to be done.

To fix this, we’ve initiated a new process that should streamline Javadoc creation for developers. We’ll follow this closely to make the best impact on our API docs as possible.

Related to this is the experience of finding the API docs, so we asked about that too.

Here, we are doing slightly better. Over the past year, we made an effort to style our API docs to make them easier to read than what’s generated by default. We still have a lot of work to do, however: our presentation on docs.liferay.com is still a default Apache file system browser. We plan to launch a project in 2018 to fix this.

We do invite the community to help with the docs if you’re so inclined. Does anybody know about this? Let’s find out.

This clearly indicates that people are not familiar with the process for submitting Javadocs. We’ve modified the guidelines in our GitHub repository accordingly; please check our repo any time you have a question about the Liferay documentation process. Please also feel free to submit feedback on LDN or on Liferay’s community forums.

We asked other questions about the rules for submitting, but since most people weren’t familiar with how to submit Javadocs, those results aren’t surprising.

The most reasonable interpretation of this is that since most weren’t familiar with how to submit Javadocs, most were neutral on whether there are a lot of rules.

Let’s move on to another section of the survey: how well people can navigate the docs.

Some of the most interesting feedback came from the question we asked about navigating the docs.

Less than a quarter of developers agreed with this statement. The number one takeaway from the survey seems to be this: when developers look for docs on a particular subject, it’s hard to find them.

We have big plans for LDN in 2018: integrating it with community.liferay.com and upgrading it to 7.1, along with an entirely new user experience. This includes taking steps to make the docs easier to find and easier to navigate.

Next, we examined whether we had the right format for the docs.

Format

When we first launched LDN, we knew we had two different audiences for docs: newcomers who need a longer, step-by-step series of tutorials to help them get off the ground, and veterans who understand the framework and need short, focused tutorials that cover what they’re working on at the moment. We dubbed the first type of documentation Learning Paths to evoke their longer experience as a journey down a path to Liferay enlightenment. We dubbed the second type as Tutorials to highlight their standalone nature.

We wanted to know if this was working. Do people like the longer Learning Paths, or would they prefer shorter, more focused Tutorials?

Far and away, developers like both. This makes sense: all of us were beginners once, and those longer “from zero to application” Learning Paths are extremely helpful when you don’t even know what you don’t know. Similarly, once developers have their feet under them, they want something quick to refer to, and Tutorials are good for that.

Next, we asked about the format. We’ve gone to web-only documentation for 7.0; was that the right decision, or did developers like the physical books, ebooks, and PDFs we created in the past?

We asked responders to rank the three formats in order of priority. Clearly, the web is the winner here as the number one priority, with ebook/PDF coming in second, and physical book coming in third.

Our final set of questions focused on the docs’ content. This was the core of the survey and the data we wanted the most.

Content

Our questions about the content focused on the quality of the docs: are they relevant; are they detailed enough; do they help people learn? Where they fail, are we headed in the right direction on improving them?

We really wanted to know these things, because we want the docs to serve users and developers well.

The first question asked whether the docs help users learn new concepts.

We were very glad to hear that the vast majority of developers learned new concepts from the docs always/regularly/sometimes. Only 8 developers said rarely or never.

It always helps, however, to ask for the same information in a slightly different way, so we also asked if readers found the answers to questions they had about Liferay from the docs.

We had a similar, but slightly less rosy result. Clearly the always/regularly/sometimes has the majority, but the always and regularly people are fewer than the sometimes people, and we have a larger group who says rarely/never (almost a third). It’s clear that we must do more to learn what developers are struggling with and get those questions answered.

One of the things we want to do more in 2018 is to be more active on the forums, learning what people are struggling with, and then documenting those things. We are also working more closely with our Support team and filling out those areas in the docs where we find customers struggle.

We weren’t done asking about this, though. Next, we had some questions on the level of detail in the docs.

Here, the always/regularly and the rarely/never groups take a little more than half, while the sometimes group is the largest. This was one of the items we tried to address this year by working more closely with Liferay’s developers. Using their knowledge, we dived more deeply into the various development topics. These results show us we made some headway, but we must do more.

We next asked about whether what is documented is relevant to readers’ needs.

Here, the agree/strongly agrees win out, with only a small fraction disagreeing. For the most part, we’re documenting the right things; the things people need to know to get their work done.

Beyond the content, we also wanted to know if we were communicating well. To find out, we asked if the documentation’s writing style is easy to understand and follow.

Almost three-quarters of responders said they agreed or strongly agreed that the docs have a writing style that’s easy to understand and follow. We work very hard on this. I will have a blog entry later on our process, but it’s a multi-step process that’s designed to produce clean and clear prose that’s easy to understand, for both native English speakers and non-native English speakers.

Finally, we wanted to find out about how the documentation trends: is it getting better over time?

Overwhelmingly, the response is yes, the docs are improving. That’s really great to hear, but we know we still have many areas of improvement. We will endeavor to make this improvement trend continue.

Well, that’s the survey. Let me sum up the takeaways here at the end, because they will inform our goals for 2018:

  • We must find better ways of garnering feedback from outside Liferay.
  • We must increase coverage of Liferay’s API documentation (Javadoc, Taglib doc, JSdoc).
  • We must make the presentation of the API docs better. Our redesign of LDN must focus on making the docs easier to navigate.
  • We must be more active on the forums, community chat, and anywhere else to learn about what developers struggle with, so we can document the right things.
  • We want to document the right things with a deeper level of detail.

Thank you to everyone who filled out the survey this year! We will check back next year with another survey to see if we’re able to improve on all these items in 2018.