ElasticSearch的安装与配置

ElasticSearch的安装与配置

Administrator 1362 2019-06-12
  1. 安装ElasticSearch
  2. 配置ElasticSearch
  3. curl查看ElasticSearch

安装ElasticSearch

  • 安装ElasticSearch可以参考官方文档;

  • 实际安装ElasticSearch可以使用yum安装，首先需要导入rpm的key：

    rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

  • 然后在/etc/yum.repos.d/目录下创建elasticsearch的yum仓库文件elastic.repo，写入以下内容：

    [elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

  • 完成后就可以使用命令yum list | grep elastic查看仓库是否生效：

    [root@centos_1 ~]# yum list |grep elastic
apm-server.i686                             6.8.0-1                    elasticsearch-6.x
apm-server.x86_64                           6.8.0-1                    elasticsearch-6.x
auditbeat.i686                              6.8.0-1                    elasticsearch-6.x
auditbeat.x86_64                            6.8.0-1                    elasticsearch-6.x
elasticsearch.noarch                        6.8.0-1                    elasticsearch-6.x
filebeat.i686                               6.8.0-1                    elasticsearch-6.x
filebeat.x86_64                             6.8.0-1                    elasticsearch-6.x
heartbeat-elastic.i686                      6.8.0-1                    elasticsearch-6.x
heartbeat-elastic.x86_64                    6.8.0-1                    elasticsearch-6.x
journalbeat.i686                            6.8.0-1                    elasticsearch-6.x
journalbeat.x86_64                          6.8.0-1                    elasticsearch-6.x
kibana.x86_64                               6.8.0-1                    elasticsearch-6.x
kibana-oss.x86_64                           6.3.0-1                    elasticsearch-6.x
logstash.noarch                             1:6.8.0-1                  elasticsearch-6.x
metricbeat.i686                             6.8.0-1                    elasticsearch-6.x
metricbeat.x86_64                           6.8.0-1                    elasticsearch-6.x
packetbeat.i686                             6.8.0-1                    elasticsearch-6.x
packetbeat.x86_64                           6.8.0-1                    elasticsearch-6.x
pcp-pmda-elasticsearch.x86_64               4.1.0-5.el7_6              updates
rsyslog-elasticsearch.x86_64                8.24.0-34.el7              base

  • 仓库生效后，使用yum install -y elasticsearch安装ElasticSearch，安装速度会比较慢，可以到官方地址直接下载rpm包进行安装，这里提供的rpm包地址为6.0.0的版本。

  • 对于下载的rpm包，使用rpm -ivh elasticsearch-6.0.0.rpm命令进行安装：

    [root@centos_1 ~]# rpm -ivh elasticsearch-6.0.0.rpm
准备中...                          ################################# [100%]
Creating elasticsearch group... OK
Creating elasticsearch user... OK
正在升级/安装...
   1:elasticsearch-0:6.0.0-1          ################################# [100%]
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service

  • 在三台机器上都安装elasticsearch软件包。

配置ElasticSearch

  • 安装完elasticsearch后，可以使用rpm -ql elasticsearch查看es软件包安装的文件：

    [root@centos_1 ~]# rpm -ql elasticsearch
/etc/elasticsearch/elasticsearch.yml
/etc/elasticsearch/jvm.options
/etc/elasticsearch/log4j2.properties
/etc/init.d/elasticsearch
/etc/sysconfig/elasticsearch
/usr/lib/sysctl.d/elasticsearch.conf
/usr/lib/systemd/system/elasticsearch.service
/usr/lib/tmpfiles.d/elasticsearch.conf
...

  • 其中es有两个配置文件，分别为/etc/elasticsearch/elasticsearch.yml和/etc/sysconfig/elasticsearch，elasticsearch配置文件是es服务本身的配置文件，如配置文件PATH，PID，JAVA相关选项等等，而elasticsearch.yml则是集群、节点、数据、日志路径、监听IP和端口等的配置文件；

  • 首先在主节点机器128上编辑/etc/elasticsearch/elasticsearch.yml文件，增加或更改以下配置：

    cluster.name: evobot_es_test
node.name: vm1
node.master: true //表示该机器为主节点
node.data: false  //表示该机器非数据节点
#network.host: 0.0.0.0 //监听全部IP，但不够安全，在机器有公网IP的情况下，建议指定监听内网IP
network.host: 192.168.139.128
http.port: 9200
discovery.zen.ping.unicast.hosts: ["centos_1", "centos_2", "centos_3"]  //定义集群内的角色，可以使用hosts定义过的主机名或者IP地址

  • 编辑完之后保存，然后在另外两台数据节点同样编辑配置文件，并根据实际角色修改相应的配置，如下：

    cluster.name: evobot_es_test
node.name: vm2
node.master: false
node.data: true
network.host: 192.168.139.130
http.port: 9200
discovery.zen.ping.unicast.hosts: ["centos_1", "centos_2", "centos_3"]

  • 三台机器都配置完成后，就可以启动三台机器的es服务，使用systemctl start elasticsearch命令启动服务，启动后查看es进程是否正常运行：

    [root@centos_1 ~]# systemctl start elasticsearch
[root@centos_1 ~]# ps aux |grep elasticsearch
elastic+  10924 59.5 44.1 3510316 822924 ?      Ssl  23:58   0:04 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/elasticsearch -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet
root      10967  0.0  0.0 112724   984 pts/0    S+   23:58   0:00 grep --color=auto elasticsearch

  • 如果es进程没有正常运行，首先查看/var/log/elasticsearch/目录下是否有日志，如果该目录下没有日志，说明不是es的配置问题导致的无法启动，再去查看/var/log/message中的日志信息：

    Jun 18 23:59:28 vm2 elasticsearch: which: no java in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)
Jun 18 23:59:28 vm2 systemd: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Jun 18 23:59:28 vm2 elasticsearch: could not find java; set JAVA_HOME or ensure java is in PATH

    根据以上日志可以判断，系统的JDK环境存在问题导致es无法启动，需要查看JAVA_HOME的配置是否正确，首先查看java是否安装，然后/etc/profile内的java环境变量是否添加了export，再使用命令ln -s /usr/local/jdk1.8/bin/java /usr/bin/将java命令软连接到/usr/bin下。

  • 服务启动后，查看/var/log/elasticsearch/[cluster_name].log是否存在报错，例如：

    [2019-06-19T00:10:14,193][INFO ][o.e.d.z.ZenDiscovery] [vm2] failed to send join request to master [{vm1}{Gi8jOhmWQzC5c1irLNrMTw}{dPNVNuNsSA26h0Fb4khxbw}{192.168.139.128}{192.168.139.128:9300}], reason [RemoteTransportException[[vm1][192.168.139.128:9300][internal:discovery/zen/join]]; nested: ConnectTransportException[[vm2][192.168.139.130:9300] connect_timeout[30s]]; nested: IOException[没有到主机的路由: 192.168.139.130/192.168.139.130:9300]; nested: IOException[没有到主机的路由]; ]

    上面的报错是由于selinux没有关闭或者防火墙规则导致，关闭selinux并在防火墙添加允许es的9200和9300端口后，重启es服务，其中9200端口是集群本身使用的端口，9300则是节点数据传输使用的端口。

curl查看ElasticSearch

  • 检测es服务是否健康，可以使用curl '192.168.139.128:9200/_cluster/health?pretty'命令对主节点的es服务健康状况进行检测：

    [root@centos_1 ~]# curl '192.168.139.128:9200/_cluster/health?pretty'
{
  "cluster_name" : "evobot_es_test",
  "status" : "green",	#健康
  "timed_out" : false,  #无超时
  "number_of_nodes" : 3, #3个节点
  "number_of_data_nodes" : 2, #两个数据节点
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

  • 除了查看es的健康信息，还可以查看es的集群信息，使用curl '192.168.139.128:9200/_cluster/state?pretty'命令进行查看：

    [root@centos_1 ~]# curl '192.168.139.128:9200/_cluster/state?pretty'
{
  "cluster_name" : "evobot_es_test",
  "compressed_size_in_bytes" : 348,
  "version" : 6,
  "state_uuid" : "G2qRbuQ3Tge5n2SDghXCeg",
  "master_node" : "Gi8jOhmWQzC5c1irLNrMTw",
  "blocks" : { },
  "nodes" : {
    "SQv2ouFRRMWKcWD-gZ6uIw" : {
      "name" : "vm3",
      "ephemeral_id" : "K3jxOnTWTBSXnm92d-Hrhg",
      "transport_address" : "192.168.139.131:9300",
      "attributes" : { }
    },
    "4jK1ltlJSM-gwC0RWY3rDg" : {
      "name" : "vm2",
      "ephemeral_id" : "l61iVh8tQlKNitSg6lyYCw",
      "transport_address" : "192.168.139.130:9300",
      "attributes" : { }
    },
    "Gi8jOhmWQzC5c1irLNrMTw" : {
      "name" : "vm1",
      "ephemeral_id" : "PIo1OF9HQSKNFxG70Puz-Q",
      "transport_address" : "192.168.139.128:9300",
      "attributes" : { }
    }
  },
...
ElasticSearch