Pootle sync for Liferay release

[Daniel Sanz, modificado 8 Anos atrás.]

Hi guys,

I just wanted to inform you that Pootle server is undergoing a deep change in the way translation projects are being mapped to the source code. This is in preparation for the new Liferay release which as you know is vastly modular.

Before the change: 1 translation project -> 1 software module in the code.
Examples: we had 1 project in pootle for each portlet, osgi module, etc. Basically, any directory containing Language.properties was considered as a pootle project.

After the change: 1 translation project -> 1 github repository.
Examples: we have 1 project for the plugins repository, another for the portal and another for audience targeting.

There are strong reasons for this change. Basically, the number of modules and its translation keys changed so frequently that the contents in Pootle were always far from the freshest ones. In addition, the sync algorithm was designed in 2011 when the portal software layout was quite different in nature, so the current source code dynamism had to be dealt with via manual pre-sync operations.

The sync and provisioning processes have been redesigned and rewritten . Now I'm finishing the conversions required in order to make this happen.

I'm working with a pootle DB copy taken yesterday. This means that contributions made after that date might not be updated in the portal.

I'll do my best to keep them but it's better if you hold off contributing translations via translate.liferay.com until I update its database with the new project layout. This operation may take up some days.

I'll let you know when everything is ready.

Thanks for understanding.
Best!

[Brendan Johan Lee, modificado 8 Anos atrás.]

This is awesome Daniel!

If I'm reading this correctly, once this is done, we can simply submit pull requests to a single git-repo to do translations? If so, this is super cool, and will make life so much easier.

[Daniel Sanz, modificado 8 Anos atrás.]

Hi Brendan,

indeed that option has always been there: sync tool not only updates the set of translatable keys in pootle. It also updates translations coming from source code into pootle if following conditions are met:
1. key in source code language file is translated (i.e. does not end in Auto Translation or stuff like that)
2. key in pootle is not translated

Basically, the difference is that now I'm joining a bunch of small resource bundles in source code into a single translation project in pootle. This will allow developers to move bundles between modules, and any sort of refactoring without compromising the sync process.

As a result, Pootle will be able to contain the latest keys coming from master more often.

Nowadays we have 3 repositories from where pootle syncs, which means that we'll have 3 translation projects (instead of ~300 we have now). In the near future we'll have more repositories.

Hope this helps to clarify.
If time allows I'll write a blog entry containing more details. I think community deserves it.

Best

[Brendan Johan Lee, modificado 8 Anos atrás.]

Indeed. But the main issue with how it previously worked (that made me just continue using pootle) was the fact that we needed to know who to send the pull request to (ie we needed to know who was responsible for each individual project before sending requests). That is at least how I understood the issue after talking with Olaf and James at the last unconference in Germany. I'm guessing that this will no longer be an issue, once everything is collected in a single project (or did I completely misunderstand what you said about a single repository?)

[Daniel Sanz, modificado 8 Anos atrás.]

I'm not sure I understood. Let me provide some more info to make sure we're on the same page.

Traditionally, there are 2 ways to translate:
1. Send PR to someone. This can be sent to component dev lead or directly to repository gatekeeper (the guy who can do a git push on it)
2. Use pootle. Pootle works with translation projects which are mapped to source code resource bundles.

Each repository has a gatekeeper and PR senders must know who the gatekeeper is. Currently, in the same repo we have a bunch of components so that different people is responsible for them. That introduces an additional review layer.

sync-tool knows that information and prepares the PR automatically against the gatekeeper. The reason is that it makes no sense that the dev lead reviews translations for languages she knows nothing about. As sync-tool prepares the PR by syncing and running build-lang, there is no source formatting issues so there is no need for the component dev lead to review. That's why the PR is sent against the repository gatekeeper.

None of this has changed. In particular, PR senders still need to know who send the PR to.

Note that sync-tool knows:
* The set of github repositories to sync from
* The set of resource bundles found in each repository
* The set of pootle translation projects
* A mapping between pootle translation projects and resourve bundles

What has changed (will change) is the Pootle project mapping to resource bundles. We are going to switch from 300 small translation projects to 3 big ones.

Before the change, each resource bundle in source code had its own translation project in pootle.
After the change, sync tool will take all resource bundles in a repo, join them, and use that data to sync from/to a single translation project associated to that repo.

The implications of this are:
* For developers they can refactor resource bundles as they wish, as long as they don't change the repo where the resource bundle lives. sync-tool guarantees that translations will be synced in presence of such refactorings without loosing data coming from pootle.
* For translators: they won't need to browse a huge list of small translation projects in pootle. Therefore the "translation project" concept will embody a source code repository where different modules can define resource bundles but translators don't need to care which resource bundle are translating.

This model fits much better with current needs from both sides. Moreover, the increasing number of repositories demands this new mapping.

Please let me know if this helps.

Best!

[Brendan Johan Lee, modificado 8 Anos atrás.]

Okey. I think I get the deal. Basically what happened was when discussing this previously it was mentioned that you wold need to know who the dev lead for each individual module to be able to send pull requests for translations, but I take it that is wrong, as long as one can send a pull requests to a single "gatekeeper". (Any info on who this is for liferay-portal and liferay-plugins?)

[Daniel Sanz, modificado 8 Anos atrás.]

Hi,

the fact that sync-tool sends PR to the gatekeeper does not imply that manual PR created by translators (either from Liferay staff or from community) have to be sent directly to him.

Manual PR can be wrong due to a variety of reasons:
* source formatting
* file encoding
* key-value pair syntax

They should be sent to the dev lead for each component.

That's why I've said that none of this really changed. It's all about how translators will perceive Pootle interaction.

Best!