Beta test period for 4.4

[Joel Kozikowski, modificado hace 16 años.]

I just received my email about joining the new Advisory Board, so I filled out the information on the form. First, the "Testing of ServiceBuilder" was indicated as a "required field." I had to check them so the system would accept my submission. I'll TRY to test that, but can't make any guarantees!

My initial comment is about the beta period: 2 weeks is a bit ambitious! It may also be overly optimistic. Just FYI, the beta period of some other software that I'm working on (unrelated to Liferay) has lasted about 8 months!

I can understand the desire to insure that testers get the code and start working with it, but this process doesn't always move as fast as one would like. First, you've got to get people to actually install the system. With other deadlines looming and the holidays coming up, I was actually planning on STARTING the testing process Jan 1.

Next, it will take some time for me to test locally. I'm sure there will be issues in the conversion (there always are). It could take some time to identify the issues, and fix them (and/or correct them). After that, I'll need to roll the system out to MY beta site, which will allow my end users to play with it.

Since this is the first time this program is being tried, we'll just have to "see" how it goes. But, in the name of "managing expectations" for all stake-holders, I think you'll need to start thinking of a beta period lasting several months. Of course, it all depends on the quality of the initial code, the number of "new" features, etc.

My recommendation is you try to set deadlines for testers to START using the system. The "end" of the beta period will be self deterministic: when everyone has installed the system and excersied it, only then will you get a feel for the issues that exist (or that don't). Only once the bug reports have died down should you think about naming something a release candidate. I'd caution about using the release designation FINAL until the community at large has had a chance to work with the code for an extended period of time.

I mentioned in a previous post that it is much easier for people to forgive problems IF they know something is not ready for prime time. Your credibility is only on the line once you call something "final." Prior to that, you're off the hook. So, I think the process should be: freeze the feature set and release to beta. Once you have confirmed that a "suffucient" number of beta testers have used the system and feel the code is fairly solid (a determintation that should involve the opinions of the board), you change the name to "release candidate" and let the community at large work with the code. I wouldn't call something "final" until its been out there a while.

Note that any one community member can pick up the "latest and greatest" at any point from SVN, so there is no real "hurry" to release something. If someone wants or needs some new feature, they are free to get the code at any point. They are also free to grab a distribution that is a "beta" or a "release candidate." But, by naming them appropriately, they should be aware of the risks.

A new to the community user should be able to come in and grab a "final" release and have a great amount of confidence that things will work and will be solid. They don't care as much about the whiz-bang features as much as seeing if the thing works.

[Alex Owen Wallace, modificado hace 16 años.]

I agree with your statements related final releases...

I need some features from 4.3.4, and now 4.3.5 is out... However, i've read comments that using 4.3.5 on existing content (an already populated database) is broken...

I need to upgrade our customizations to 4.3.5, but if it won't work with an existing db, then i can't really use it...

I loved your idea that instead of comming up with new stable versions so quickly, an patch approach should be used to fix these bugs, and not add any new features... (from another post)

[Alex Owen Wallace, modificado hace 16 años.]

Should I assume that the 4.4 codebase is only available to the selected beta testers?

Something you may have received as a beta tester, but just in case you didn't, Is a list of all the stuff you should test for.

[Joel Kozikowski, modificado hace 16 años.]

Alex Owen Wallace:

Should I assume that the 4.4 codebase is only available to the selected beta testers?

No - you are free to check out whatever code from whichever branch you want.

This email relates to a formal beta program that Liferay is starting. Certain community members have been invited to join the Liferay Advisory Board, and this forum category is for board members to discuss all things relating to it.

Funny - I was under the impression that this category was "read only" by all community members, and posting was only allowed by people on the Advisory Board. Alex - were you invited to participate? If not, then my first bug report is that the security system on the forums don't work

[Brian Chan, modificado hace 16 años.]

It's working as designed.

Only core developers can ADD new threads.

Anyone in the community can reply though. The goal is to encourage as much participation as possible from everyone, while limiting the amount of topics so we're not talking about every possible thing.

[Joel Kozikowski, modificado hace 16 años.]

Brian Chan:

Only core developers can ADD new threads.

Anyone in the community can reply though. The goal is to encourage as much participation as possible from everyone, while limiting the amount of topics so we're not talking about every possible thing.

Great! That seems like a good compromise between trying to control the number of topics, and being "exclusive."

BTW Alex - I didn't mean to sound snotty and exclusive. I was just trying to figure out how the system was supposed to work (and whether or not it was working in the first place)

[Alex Owen Wallace, modificado hace 16 años.]

Joel Kozikowski:

BTW Alex - I didn't mean to sound snotty and exclusive. I was just trying to figure out how the system was supposed to work (and whether or not it was working in the first place)

No bigge! I'd love to be part of that board though! Maybe when I get my Jedi Knight rank

[Brian Chan, modificado hace 16 años.]

Hi Alex,

We fixed the database migration issues in 4.3.6 now. Trying to keep the minor dot releases out every 2 weeks.

[Alex Owen Wallace, modificado hace 16 años.]

Brian Chan:

We fixed the database migration issues in 4.3.6 now. Trying to keep the minor dot releases out every 2 weeks.

Thank you Brian!

Since I'm about to try upgrading to 4.3.5, but know in advance that the db migration issue will be there... Is there anything I can use, that will safely get me beyond that bug?

Should i check out the https://lportal.svn.sourceforge.net/svnroot/lportal/portal/branches/4.3.x/ branch? Is that kind of stable? will that be better than 4.3.5?

It sounds like it will take another week at least for 4.3.6, right?

Thanks a bunch!

[Michael Young, modificado hace 16 años.]

Hey Joel,

I've fixed the issue in the inscription form, so it should be good now.

As for the beta testing period, it's probably going to be a bit expedited this time around. Our goal with this initial testing period are really to get the most glaring issues out of the way so that we've taken a "step up" on our final releases. We realize that it's not going to be perfect, especially with the 4.4 release, but we wanted to at least give this pilot run a chance to improve the quality.

As stated by Jorge in this blog entry, http://www.liferay.com/web/jferrer/1/blogs/we_want_the_best_release_process_possible, we've decided to embark on a release process that is very similar to Ubuntu's. That is, very frequent releases (5.0. 5.1, etc), every two months. We will then designate one of these releases as LTS (Long Term Support), and continue to maintain that release for a fixed time, say 12-18 months, until we sunset that and designate another release for LTS.

Given a 2 month release cycle, we came up with a 2 week long beta testing period. Our goal as stated in the blog entry is to be able to surface new features quickly and at the same time maintaining stability. This is definitely a different style for doing releases, but we feel it aligns with our business climate more closely than with other more traditional release cycles.

For our next beta testing cycle, we'll give more advanced notice for the beginning of the testing cycle instead of just throwing everyone into the fire.

[Joel Kozikowski, modificado hace 16 años.]

Michael Young:

we've decided to embark on a release process that is very similar to Ubuntu's. That is, very frequent releases (5.0. 5.1, etc), every two months. We will then designate one of these releases as LTS (Long Term Support), and continue to maintain that release for a fixed time, say 12-18 months, until we sunset that and designate another release for LTS.

If I understand correctly, the LTS release will be "feature frozen", but will continue to have bug fixes applied to it? It sounds like it will work something like this: 4.4.0 will be released, then 4.4.1, then 4.5.0, 4.5.1, 4.5.2, then 5.0, 5.1, etc. Depending on the stability of things, we may find, for example, that 4.5.6 is the most "stable" of the string of releases, so that becomes the LTS release until another, more advanced version proves its worthiness?

Do you anticpate there being one and only one LTS release at a time (sounds like it)? Or, will there be a LTS release for the 4.x series, one for the 5.x series, etc.?

Maybe I'm getting too hung up on the term "final release." What I am really looking for is "production stable," so perhaps the LTS release will be the one that fits the bill. My recommendation is that your release strategy be clearly spelled out on the download page, so that end users can pick a release that is appropriate for their particular risk tolerance.

[Michael Young, modificado hace 16 años.]

There will be one long term release at a time. The first LTS release will be 4.4.x. This means that as we release 5.0, 5.1, 5.x, we will be maintaining 4.4.x for an extended specified period of time, so you might see releases up to 4.4.20, for example. All releases will be feature frozen and micro-releases (eg 4.4.1, 4.4.2) will have nothing but bug fixes.

As we develop new feature releases like 5.0, 5.1, we will still maintain micro releases, but only one release behind. For example, if we are in the midst of 5.3 development, we will still be making 5.2.x releases until 5.3 is released. At the point 5.4 development starts and we will be doing 5.3.1, etc. At the same time the LTS release will continued to be maintained.

So to summarize, we will maintain 3 simultaneous branches at one time:

LTS: eg 4.4
Current Release: eg 5.2.x
Branch: eg the upcoming 5.3.0

[Jorge Ferrer, modificado hace 16 años.]

Thanks for your comments Joel. They are always very thoughtful.

I agree that we may have to adjust the timings of both the releases and the time for beta testing. But we also think that it should be possible to reduce the traditional period of time within releases by doing continued testing and review of code. We'll try hard to make the 2 month release work and will try hard to not increase the release period.

And thanks again for joining the Advisory Board.

[Joel Kozikowski, modificado hace 16 años.]

Jorge Ferrer:

I agree that we may have to adjust the timings of both the releases and the time for beta testing. But we also think that it should be possible to reduce the traditional period of time within releases by doing continued testing and review of code. We'll try hard to make the 2 month release work and will try hard to not increase the release period.

A release cycle this rapid can work great in some situations, but not in others. I think it is important to consider the types of changes going into each release cycle. The best and most significant example I can come up with (as it was the cause of most of my pain) was the change to the DB's primary keys. That took a considerable amount of refactoring on my end to "catch up."

With the new system being proposed, it would seem to me that a major change like that should be the ONLY thing that happens in one of the "two month release cycles." Or, perhaps only the "user id" key is changed in one release cycle plus all the other features, then the "organization id" is chagned in the next release cycle plus all the other features, etc.

What is your current policy on version numbering in general? What made the version number of the above mentioned refactoring 4.3 and not 5.0?

Ideally, there would be a number in the version number that represents the data structures and/or the service APIs. Everything from the 5.x series, for example, would use the same data structures and APIs. You could add NEW tables and services to 5.1, but not change old structures. If you need to change the data structures, rename the version to 6.0, etc.

Or, is that what you use the second number for? Should I assume that 4.3.x is data compatible, but 4.4.x may require code refactoring on my part?

This all points to a need to carefully consider the significance of changes that require refactoring of existing code. NEW features are one thing. Changing behaviors and structures of EXISTING features are another. We need to minimize the number of changes requiring the later in any one release cycle.