Deploying an ElasticSearch Cluster in a Linux Environment

  • Introduction
  • Environment
  • Cluster Installation and Deployment
    • First Machine
    • Second Machine
    • Third Machine
    • Change the owner and group of the elasticsearch directory
    • Switch to the es user and start the es on all three machines
  • Cluster Startup Verification
    • Restart the three nodes in sequence
    • Verification
  • es Cluster Authentication
    • x-pack Authentication
  • Reference Documents

Introduction

❝

This article introduces the installation and deployment of an es cluster under Linux, with a cluster consisting of three nodes.

Environment

  • elasticsearch-7.9.3-linux-x86_64.tar.gz
  • jdk-8u321-linux-x64.tar.gz
  • CentOS-7-x86_64-Minimal-1908.iso

Cluster Installation and Deployment

First Machine

Create Directory

// Extract elasticsearch-7.9.3 to this directory
mkdir -p /usr/local/elasticsearch
// Data directory
mkdir -p /usr/local/elasticsearch/data
// Log directory
mkdir -p /usr/local/elasticsearch/log

Configure elasticsearch.yml File

# Cluster Name
cluster.name: keweizhou
# Node Name
node.name: node-nj-es-128
# Node Roles
node.master: true
node.data: true
# Maximum Number of Nodes
node.max_local_storage_nodes: 3
# Bound IP Address
network.host: 192.168.30.128
# External Port
http.port: 9200
# Inter-node Communication Port
transport.tcp.port: 9800
# Zen Discovery Config
discovery.seed_hosts: [ "192.168.30.128:9800","192.168.30.129:9800","192.168.30.130:9800" ]
discovery.zen.ping_timeout: 60s
discovery.zen.join_timeout: 60s
discovery.zen.master_election.ignore_non_master_pings: true
# Cluster Election
cluster.initial_master_nodes: [ "node-nj-es-128","node-nj-es-129","node-nj-es-130" ]
# Data and Log Directories
path.data: /usr/local/elasticsearch/data
path.logs: /usr/local/elasticsearch/log

# After configuration, monitoring data can be viewed in Kibana
xpack.monitoring.enabled: true
xpack.monitoring.collection.enabled: true

# Enable or disable Cross-Origin Resource Sharing
http.cors.enabled: true
### Allowed Origins
http.cors.allow-origin: "*"

Modify jvm.options File

cd /usr/local/elasticsearch/config
vim

Currently, the cluster is set to 2g

-Xms1g
-Xmx1g

Second Machine

Configure elasticsearch.yml File

  1. Copy the /usr/local/elasticsearch directory from the first machine to the second machine
scp -r elasticsearch/ [email protected]:/usr/local

2. Modify the configuration file. Change some configurations while keeping others consistent with machine 128.

node.name: node-nj-es-129
network.host: 192.168.30.129

Third Machine

  1. Copy the /usr/local/elasticsearch directory from the first machine to the third machine
scp -r elasticsearch/ [email protected]:/usr/local

2. Modify the configuration file. Change some configurations while keeping others consistent with machine 128.

node.name: node-nj-es-130
network.host: 192.168.30.130

Change the Owner and Group of the elasticsearch Directory

  1. Add a new es user and group
groupadd es
//useradd [user name] -g [group name] -p [password]
useradd es -g es -p xxx

Change the owner and group of the /usr/local/elasticsearch directory and its contents on all three machines to the es user

  1. Change
//chown -R [user name:group name]
chown -R es:es /usr/local/elasticsearch

Switch to the es User and Start es on All Three Machines

Before starting, switch to another user and create a new es user and group. This is necessary because Elasticsearch cannot run as the root user.

su es
./elasticsearch

Possible Issues

  • If you encounter a prompt regarding jdk11, modify the elasticsearch-env (in the bin directory) to use the built-in jdk of es.
# use elasticsearch jdk
ES_JAVA_HOME="/usr/local/elasticsearch/jdk"

# now set the path to java
if [ ! -z "$ES_JAVA_HOME" ]; then
  JAVA="$ES_JAVA_HOME/bin/java"
  JAVA_TYPE="ES_JAVA_HOME"
elif [ ! -z "$JAVA_HOME" ]; then
  JAVA="$JAVA_HOME/bin/java"
  JAVA_TYPE="JAVA_HOME"
...
  • If you encounter an error similar to: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]. The modification method is:
vim /etc/security/limits.conf
es  -  nofile  65535
  • If you encounter an error similar to: max number of threads [3795] for user [es] is too low, increase to at least [4096]. The modification method is:
vim /etc/security/limits.conf
es  -  nproc  4096

Reference: https://blog.csdn.net/qq_35787138/article/details/117756903

  • If you encounter an error similar to: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]. The modification method is:
vim /etc/sysctl.conf
vm.max_map_count=262144

If it does not take effect, refresh it:

sysctl -p

Reference: https://blog.csdn.net/weixin_39643007/article/details/108435139

  • If you encounter a failure when sending a validation request to the node, it may be because the node was previously started, has historical data, and that data is inconsistent with the current cluster data. From the logs, it can be seen that the cluster UUIDs of both are different. Therefore, you need to clear the old data and restart to allow the node to join the cluster. Find the data storage directory corresponding to path.data in elasticsearch.yml, clear that directory, and then restart.

Cluster Startup Verification

Restart the Three Nodes in Sequence

./elasticsearch -d

Verification

Cluster Health Status Verification

curl http://192.168.30.128:9200/_cat/health?v
curl http://192.168.30.129:9200/_cat/health?v
curl http://192.168.30.130:9200/_cat/health?v

Verification Result: Deploying an ElasticSearch Cluster in a Linux Environment

Cluster Node Verification

curl http://192.168.30.128:9200/_cat/nodes?v

Verification Result:

Deploying an ElasticSearch Cluster in a Linux Environment
es Cluster Node Status

es Cluster Authentication

x-pack Authentication

To be supplemented

Reference Documents

  1. Errors encountered when installing ElasticSearch on Linux
  2. Future versions of Elasticsearch will require Java 11; your Java version from
  3. https://www.devopsschool.com/blog/elastic-search-error-max-file-descriptors-4096-for-elasticsearch-process-is-too-low-increase-to-at-least-65535/
  4. https://blog.csdn.net/qq_35787138/article/details/117756903
  5. https://discuss.elastic.co/t/elasticsearch-start-failed-to-resolve-host-host1/136399
  6. https://blog.csdn.net/c15158032319/article/details/125424331
  7. https://blog.csdn.net/weixin_30263277/article/details/99062372
  8. java.lang.IllegalStateException: failure when sending a validation request to node #88
  9. CentOS7 installation of elasticsearch-7.9.3 cluster

Related posts

  1. Cross-Script Interface Association and Parameter Passing in HttpRunner3
  2. Linux | I.MX6ULL Peripheral Function Verification (Day 10)
  3. Smart Lock and APP Bluetooth BLE Communication Solution: Service Design, Security Mechanisms, and Development Practical Guide
  4. Linux Philosophy: Everything is a File
  5. Mastering Linux DNS: From Beginner to Expert
  6. Key Technologies and Design Points of 6G Integrated Sensing and Computing Networks
  7. China’s Chips! The Hidden Gold Mine of Chip Manufacturers in the Mid-Range Market
  8. Heat Management and Safety for Home Servers and Network Devices: Special Protection for 24/7 Operating Equipment
  9. How to Charge for RK3588 Baseboard Design
  10. What Is Linux Mobile
  11. Installation and Configuration of PostgreSQL Debugger Plugin on Debian Linux
  12. Exploring Linux on MCU Series Part Nine: Introduction to the SDRAM Controller of STM32F429
  13. 10 Commands That Can Destroy Your Linux System
  14. Comic Analysis: What Does the Linux Kernel Look Like?
  15. How to Analyze and Resolve Intermittent Packet Loss on Linux Servers?
  16. The True Advantages of Linux and Learning Methods
  17. Step-by-Step Guide to Customizing Linux Boot Logo Based on IMX93-EVK with MIPI-DSI Screen
  18. 5 Security Points to Note in Linux Account Management: Don’t Wait Until It’s Too Late!
  19. Linux Toolchain: Secrets to Efficient Development and Management

Leave a Comment