ElasticSearch Protocol Implementation

[Juan Miguel Bonilla López，修改在7 年前。]

Hi there,

I need to know if the ElasticSearch implementation in Liferay 7.0 uses TCP or HTTP protocol to communicate, since ElasticSearch Service in AWS does not allow TCP communication. I'm wondering if someone has an idea.

Thanks,

J.M.Bonilla.

[Jorge Díaz，修改在7 年前。]

Hi Juan Miguel,

Liferay 7.0 will only connect to elasticsearch using TCP transport protocol:

• Liferay 7.0 uses ElasticSearch Java native client. That java client only connects to TCP transport protocol, it cannot connect to HTTP one.
• There are other Java clients (for example Jest) that allows calling from Java to HTTP REST protocol but that clients are not created by ElasticSearch, they are not official.
• More info , see: https://www.elastic.co/blog/found-java-clients-for-elasticsearch

About Amazon Elasticsearch Service, it has some problems:

• It only supports Elasticsearch 1.5.2. Liferay 7.0 requires Elasticsearch 2.2.
• More info: https://forums.aws.amazon.com/thread.jspa?threadID=219245 and https://aws.amazon.com/elasticsearch-service/faqs/
• It only supports HTTP REST protocol (on port 80). It does not support TCP transport
• More info: http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-limits.html

So you cannot use Amazon Elasticsearch Service with Liferay 7.0 as you cannot connect to TCP transport port. In case you will able to connect, its elasticsearch version is outdated (they are offering 1.5.2 and it will become EOL by ElastciSearch in September: https://www.elastic.co/support/eol)

My advice is to create a standard virtual machine in amazon and install on it ElasticSearch 2.2 by hand. You will be able to connect to TCP transport port and you will not have problems with ElasticSearch versions

Regards,
Jorge Díaz

[Juan Miguel Bonilla López，修改在7 年前。]

Hi Jorge,

That's exactly what I needed. Anyway, I think the best option would be to implement HTTP client to integrate ElasticSearch Service in AWS in order to avoid or minimize the system administration and improve the scalability (amazon will do this better!). I would only choose a custom server with elastic if I need some feature not available in Amazon, since I'm sure they will upgrade the versions in the future as they already do with databases, and other services.

On the other hand, about the implementation I think it is not a good decision from Liferay. HTTP protocol is more standard and used in other communications between servers and as Amazon documentation says, TCP is used internally only.

Regards.

[Jorge Díaz，修改在7 年前。]

Juan Miguel Bonilla López:

On the other hand, about the implementation I think it is not a good decision from Liferay. HTTP protocol is more standard and used in other communications between servers and as Amazon documentation says, TCP is used internally only..

As I explained in my previous message, Liferay 7 is using native Java client as it is the supported library from elasticsearch guys, see following links:

• https://www.elastic.co/guide/en/elasticsearch/guide/2.x/_talking_to_elasticsearch.html
• https://www.elastic.co/guide/en/elasticsearch/client/index.html

Java API
If you are using Java, Elasticsearch comes with two built-in clients that you can use in your code: Node client, Transport client
Both Java clients talk to the cluster over port 9300, using the native Elasticsearch transport protocol.
RESTful API with JSON over HTTP
All other languages can communicate with Elasticsearch over port 9200 using a RESTful API, accessible with your favorite web client. In fact, as you have seen, you can even talk to Elasticsearch from the command line by using the curl command.

I will ask internally to Liferay developers if there is any plan about using HTTP RESTful API, but as there is no official support to Java clients for HTTP protocol, in my opinion it will difficult to implement and support that.
(the only Java client that uses HTTP RESTful is https://github.com/searchbox-io/Jest, that is not created by elastic search)

[Raphael Crivelli，修改在7 年前。]

Hi Guys

I would like to take up this topic again.
I have an Liferay 7 in Elastic Beanstalk with RDS, S3 and Route 53.
In my opinion it would be great if we can use AWS ES instead of an additional EC2 instance.

Obviously AWS has changed a lot since May. Now im able to use Elasticsearch v2.3.
Is it now possible to connect ES to Liferay, or does the TCP restriction still exist?

Does anyone has an workaround?

Thank you.

[Jorge Díaz，修改在7 年前。]

Raphael Crivelli:

Obviously AWS has changed a lot since May. Now im able to use Elasticsearch v2.3.
Is it now possible to connect ES to Liferay, or does the TCP restriction still exist?

Liferay uses java native client of Elasticsearch (see my explanation in previous comments).
The java native client only uses transport protocol, so you cannot connect to AWS in case amazon doesn't open the transport tcp port.

About all the issues of elasticsearch java native client, it seems there is on elasticsearch roadmap a new oficial java client, more info see:
- https://www.elastic.co/blog/state-of-the-official-elasticsearch-java-clients

[Diego Alejandro Palmeira，修改在7 年前。]

Jorge Díaz:

Raphael Crivelli:

Obviously AWS has changed a lot since May. Now im able to use Elasticsearch v2.3.
Is it now possible to connect ES to Liferay, or does the TCP restriction still exist?

Liferay uses java native client of Elasticsearch (see my explanation in previous comments).
The java native client only uses transport protocol, so you cannot connect to AWS in case amazon doesn't open the transport tcp port.

About all the issues of elasticsearch java native client, it seems there is on elasticsearch roadmap a new oficial java client, more info see:
- https://www.elastic.co/blog/state-of-the-official-elasticsearch-java-clients

Hi all, I cannot believe Liferay does not support AWS Elasticsearch. It implies to install at least 3 extra servers to HA support, with all costs related to operation and maintenance. It's not a a very nice news for sure.

Don't you think AWS Elasticsearch should be included out-of-the-box with the EE product? It's important for cloud customers.

Shouldn't be part of Liferay DXP strategy be prepared for Cloud requirements?

[Jorge Diaz，修改在3 年前。]

Hi all,
Starting with 7.3 GA4 version, Liferay has moved to Elasticsearch 7.x using the java REST client. So now it could be possible to connect using the HTTP protocol.
For more information see section "ElasticSearch 7.3" in this announcement: https://liferay.dev/blogs/-/blogs/liferay-portal-7-3-ce-ga4-release-1
Future Liferay  DXP 7.3 that will released at the end of the year will also use this java REST client, so you will be also able to connect to Elasticsearch 7.x using the HTTP protocol.
Note: Liferay DXP 7.2 also supports Elasticsearch 7.x but it uses the old Transport client, so that version is not able to connect using the HTTP protocol.