Data Migration in Liferay DXP

[6年前 に Chandrashekar NM によって更新されました。]

Hello.

We are working in a project where we have documents (and its metadata) around 2TB and these data/ documents have to be migrated to Liferay DXP from legacy technologies of Java.

These data were created using some of the legacy technologies of Java. The documents contain Word, Excel, PPT and pdf etc.,

I would like to know whether these data can be migrated to Liferay DXP within one week.

In that case what would be the configurations/ infrastructure is required in Liferay(for eg: number of servers required).

Also, is it feasible/ possible to have multiple Liferay servers(in non-clustered mode) connected to single MySQL database in order to reduce the load on the application server?

Are there any challenges that are fore seen with my above scenario?


Thanks & Regards,
Chandra

[6年前 に Olaf Kock によって更新されました。]

Chandrashekar NM:

We are working in a project where we have documents (and its metadata) around 2TB and these data/ documents have to be migrated to Liferay DXP from legacy technologies of Java.
...
I would like to know whether these data can be migrated to Liferay DXP within one week.

It depends (TM). 2TB doesn't tell me much about what has to be done. Based on your description, I'm assuming that these are largely documents that should be imported into Liferay document repository - but this is probably your least worry: When they're in there, they most likely have a changed URL, and in order to find them again, you'll need to update pointers to them (unless you just are happy with them stored in folders).

Documents in Liferay not only get stored, but also indexed. If these 2TB are largely text data, it will take quite a bit. If they're largely image data, the index will be a lot smaller. Liferay DXP uses Elasticsearch out of the box, and you can scale out the searching and indexing capabilities. You'll also need to pay attention to the document storage

Chandrashekar NM:

Also, is it feasible/ possible to have multiple Liferay servers(in non-clustered mode) connected to single MySQL database in order to reduce the load on the application server?

It's possible to have multiple Liferay servers in clustered mode. I don't know why you wouldn't want to cluster them, when that's the well established way to operate multiple servers on the same database. The communication load on the appserver most likely isn't the main point of concern during such an import task.

Chandrashekar NM:

Are there any challenges that are fore seen with my above scenario?

Yes: "2TB" by far is not enough of a description of your existing data. And the number of required servers for operation after this import will largely depend on the number of concurrent users and usage patterns, rather than on the amount of data stored. In other words: You'll find a few pointers and reports in a forum post like this, but shouldn't expect a definitive answer.

As you explicitly mention Liferay DXP, you might want to get in contact with Liferay and the Global Services department, which will be able to ask a lot more detailed questions before giving more detailed answers, and also help you with the proper setup.

[6年前 に Chandrashekar NM によって更新されました。]

Hello Olaf Kock,
Thanks for quick reply.
I would like to answer some of your questions, before that let me explain my use case clearly.

• We have document related meta data information stored in excel file. The path to the actual document is also provided in the excel file.
• We create a Liferay portlet(background process) to read the excel file row by row and put the metadata in MySQL database and store the document in S3 bucket of AWS
• Documents can be PDF, WORD, EXCEL, PPT etc.,. (size of each file can be from 1MB to 100M
• Documents can also be from Sharepoint and SAP ILM tool which needs to be migrated to Liferay.
• Our idea is to have these documents in S3/ EC2 instance of AWS and read the data from S3/EC2 and store it in Liferay supported architecture.

All these data and documents sum up to ~2TB.

Our approach is to have 2 Liferay servers in clustered mode connecting to single database in AWS cloud. My concern is that migrating these data into the Liferay and database may cause too much load on the servers for migration activity. We are planning to buy Enterprise license from Liferay with 2 Production servers in clustered mode. We presume that we may need more servers during migration to avoid heavy load on the application servers. Also as I mentioned earlier we have a strict deadline of 1 week for completing the migration.

So we are thinking of having multiple Liferay servers(in non-clustered mode) connecting to the single database only for the migration activity.

Please let me know your views.

Thanks & Regards,
Chandra