docker搭建es集群實(shí)現(xiàn)過(guò)程詳解
前言
該系列默認(rèn)開啟Nacos 服務(wù),還不會(huì)搭建的小伙伴可以參考往期文章~
本節(jié)重點(diǎn)是給大家介紹利用docker來(lái)搭建Es集群,廢話不多說(shuō)直接開整吧~
什么是es
同樣的，在學(xué)習(xí)之前，先了解一下這玩意到底是個(gè)啥?
es這個(gè)名詞或許大家都聽過(guò)，它的全稱是Elasticsearch,它是一個(gè)分布式文檔儲(chǔ)存中間件，它不會(huì)將信息儲(chǔ)存為列數(shù)據(jù)行，而是儲(chǔ)存已序列化為 JSON 文檔的復(fù)雜數(shù)據(jù)結(jié)構(gòu)。當(dāng)你在一個(gè)集群中有多個(gè)節(jié)點(diǎn)時(shí)，儲(chǔ)存的文檔分布在整個(gè)集群里面，并且立刻可以從任意節(jié)點(diǎn)去訪問(wèn)。
當(dāng)文檔被儲(chǔ)存時(shí)，它將建立索引并且近實(shí)時(shí)（1s）被搜索。 Elasticsearch 使用一種被稱為倒排索引的數(shù)據(jù)結(jié)構(gòu)，該結(jié)構(gòu)支持快速全文搜索。在倒排索引里列出了所有文檔中出現(xiàn)的每一個(gè)唯一單詞并分別標(biāo)識(shí)了每個(gè)單詞在哪一個(gè)文檔中。有時(shí)候面試官會(huì)問(wèn)，es為什么這么快?這也是一個(gè)小的知識(shí)點(diǎn)。
通過(guò)上面簡(jiǎn)單的介紹，我們大體可以知道，它是用來(lái)做數(shù)據(jù)檢索的，而且速度特別快。
不知道小伙伴們有沒有遇到過(guò)這樣一個(gè)問(wèn)題，比方說(shuō)我們?cè)谟胹ql查商品庫(kù)表的時(shí)候,想要通過(guò)某個(gè)關(guān)鍵詞來(lái)匹配相應(yīng)的商品，當(dāng)數(shù)據(jù)量很小的時(shí)候ok,但是隨著商品數(shù)據(jù)的不斷導(dǎo)入，后期的數(shù)據(jù)量越來(lái)越大，而且都是關(guān)聯(lián)著好幾張表，這時(shí)候我們用sql去查詢我們想要的數(shù)據(jù)的時(shí)候，會(huì)顯得特別吃力，這種是相當(dāng)危險(xiǎn)的操作，因?yàn)榭赡軙?huì)把整張表鎖死，導(dǎo)致我們的系統(tǒng)出現(xiàn)故障，如果其它系統(tǒng)也使用這個(gè)庫(kù)，那么也會(huì)受到影響。所以這時(shí)候，我們就需要借助es這種中間件來(lái)幫我們處理這種需求，系統(tǒng)的性能也會(huì)有顯著的提升，當(dāng)然，維護(hù)上也會(huì)增加一些難度，當(dāng)然也不是啥都上es的。其實(shí)我們也可以使用其它的比如mongo，如何選取，取決于系統(tǒng)架構(gòu)和實(shí)際的業(yè)務(wù)場(chǎng)景。
使用docker搭建es集群
為了大家快速的體驗(yàn)到es，這里推薦大家使用docker來(lái)搭建，因?yàn)樗容^方便。但是生產(chǎn)中，如果你對(duì)docker不是很熟悉，維護(hù)會(huì)稍微有點(diǎn)麻煩，那么建議你還是到官網(wǎng)去下載具體的安裝包，本節(jié)默認(rèn)大家都已經(jīng)安裝好了docker。如果你還不知道docker是啥也沒關(guān)系，這個(gè)后邊我會(huì)專門給大家講講，本節(jié)跟著我敲就可以了。
docker的安裝非常簡(jiǎn)單，官網(wǎng)都有具體的平臺(tái)的安裝包,win和mac都有，無(wú)腦安裝就好了。win11安裝可能會(huì)遇到wsl的問(wèn)題，需要開啟linux子系統(tǒng)，如果啟動(dòng)錯(cuò)誤，直接百度錯(cuò)誤就好了，已經(jīng)有人踩過(guò)坑了。
下面，我們進(jìn)入正題,首先啟動(dòng)好docker，本節(jié)帶大家安裝的是7.6.2的版本，這個(gè)版本相對(duì)好一些，控制臺(tái)的功能也都很完善。
執(zhí)行已下命令獲取官方鏡像， 打開cmd/mac終端：
# es鏡像 docker pull docker.elastic.co/elasticsearch/elasticsearch:7.6.2 # kibana鏡像 docker pull docker.elastic.co/kibana/kibana:7.6.2
kibana它是一個(gè)可視化的平臺(tái)，我們查看數(shù)據(jù)就是通過(guò)它，es只是用作數(shù)據(jù)引擎，市面上也有一些第三方的工具，但是官方的這個(gè)已經(jīng)非常完善了，界面也很美觀。
緊接著，進(jìn)入指定安裝目錄,比方說(shuō)當(dāng)前目錄叫es,終端進(jìn)入這個(gè)目錄后執(zhí)行一下命令:
# kibana數(shù)據(jù)掛載的目錄 mkdir data/kibana # 三個(gè)節(jié)點(diǎn)數(shù)據(jù)掛載的目錄 mkdir data/node1 mkdir data/node2 mkdir data/node3
這一步主要是創(chuàng)建相關(guān)的目錄，因?yàn)楹筮?code>docker的數(shù)據(jù)卷會(huì)映射到該目錄，這樣做的目的是防止容器意外銷毀后的數(shù)據(jù)丟失。這里為什么是三個(gè)節(jié)點(diǎn),因?yàn)?code>es集群至少需要三個(gè)節(jié)點(diǎn)，這是跟它的內(nèi)部機(jī)制有關(guān)，為了防止腦裂現(xiàn)象，這里就不給大家過(guò)多展開了
接下來(lái)進(jìn)入data/kibana目錄,新建kibana.yml,這個(gè)文件是它的配置文件，后邊我們會(huì)把它映射到docker容器內(nèi)部
# ## ** THIS IS AN AUTO-GENERATED FILE ** ## # # # Default Kibana configuration for docker target server.name: kibana server.host: "0" elasticsearch.hosts: [ "http://es01:9200","http://es02:9200","http://es03:9200" ] xpack.monitoring.ui.container.elasticsearch.enabled: true i18n.locale: zh-CN
elasticsearch.hosts指的是三個(gè)es節(jié)點(diǎn),會(huì)和這些節(jié)點(diǎn)進(jìn)行通信
進(jìn)入node1，同樣新建配置文件elasticsearch.yml
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: es-cluster # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: es01 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 0.0.0.0 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["es01","es02","es03"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["es01","es02","es03"] # bootstrap.memory_lock: true # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true http.cors.enabled: true http.cors.allow-origin: '*' http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type node.master: true
我們把node1作為主節(jié)點(diǎn),也就是老大，node.master: true可以配置。為了使它支持中文分詞,我們給它安裝一下插件, 到倉(cāng)庫(kù)下載指定版本的插件https://github.com/medcl/elasticsearch-analysis-ik/releases,然后我們解壓到node1根目錄，然后重新命名為ik目錄,然后再新建一個(gè)Dockerfile用來(lái)重構(gòu)```es````鏡像，沒錯(cuò)，后邊我們就使用我們重構(gòu)好的鏡像，這樣就自動(dòng)安裝好了插件
Dockerfile文件內(nèi)容
FROM docker.elastic.co/elasticsearch/elasticsearch:7.6.2 COPY --chown=elasticsearch:elasticsearch elasticsearch.yml /usr/share/elasticsearch/config/ ADD ik /usr/share/elasticsearch/plugins/ik ADD ik/config /data/erms/es/node1/ik/config
下面我們進(jìn)入node2目錄，這個(gè)目錄只需要放配置文件就好了
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: es-cluster # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: es02 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 0.0.0.0 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["es01","es02","es03"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["es01","es02","es03"] # bootstrap.memory_lock: true # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true http.cors.enabled: true http.cors.allow-origin: '*' http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type node.data: true
這里我們指定為數(shù)據(jù)節(jié)點(diǎn)node.data: true用來(lái)做副本
同樣的node3
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: es-cluster # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: es03 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 0.0.0.0 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["es01","es02","es03"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["es01","es02","es03"] # bootstrap.memory_lock: true # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true http.cors.enabled: true http.cors.allow-origin: '*' http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type node.data: true
然后我們回到根目錄(es),新建一個(gè)docker-compose.yaml,我們使用docker-compose來(lái)編排我們的容器，默認(rèn)安裝好docker desktop就自動(dòng)給我們安裝好了docker-compose
version: '3'
services:
es01:
image: ${image}
container_name: es01
environment:
- discovery.seed_hosts=es02,es03
- cluster.initial_master_nodes=es01,es02,es03
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/node1/data:/usr/share/elasticsearch/data
- ./data/node1/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./data/node1/plugins:/usr/share/elasticsearch/plugins
ports:
- 9200:9200
networks:
- elastic
es02:
image: ${image}
container_name: es02
environment:
- discovery.seed_hosts=es01,es03
- cluster.initial_master_nodes=es01,es02,es03
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/node2/data:/usr/share/elasticsearch/data
- ./data/node2/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./data/node2/plugins:/usr/share/elasticsearch/plugins
ports:
- 9201:9201
networks:
- elastic
es03:
image: ${image}
container_name: es03
environment:
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02,es03
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/node3/data:/usr/share/elasticsearch/data
- ./data/node3/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./data/node3/plugins:/usr/share/elasticsearch/plugins
ports:
- 9202:9202
networks:
- elastic
kibana:
image: ${image_kibana}
container_name: kibana
depends_on:
- es01
environment:
ELASTICSEARCH_URL: http://es01:9200
ELASTICSEARCH_HOSTS: http://es01:9200
volumes:
- ./data/kibana/kibana.yml:/usr/share/kibana/config/kibana.yml
networks:
- elastic
ports:
- 5601:5601
networks:
elastic:
driver: bridge
這個(gè)文件有點(diǎn)長(zhǎng)，不懂沒關(guān)系，跟著配就完了。${image}是一個(gè)占位符，所以我們還需要指定環(huán)境變量,然后新建一個(gè).env
image=m/es image_kibana=docker.elastic.co/kibana/kibana:7.6.2
m/es這個(gè)是我們重構(gòu)后的鏡像名稱，下面我們就來(lái)重構(gòu)鏡像
進(jìn)入data/node1執(zhí)行
docker build -t m/es .
執(zhí)行完成后，到根目錄執(zhí)行啟動(dòng)命令:
docker-compose up -d
如果你想看實(shí)時(shí)日志，把-d去掉，這個(gè)是后臺(tái)運(yùn)行，初次啟動(dòng)，可能要花費(fèi)一些時(shí)間。
啟動(dòng)成功后，我們可以訪問(wèn)一些es1的節(jié)點(diǎn)localhost:9200,可以查看節(jié)點(diǎn)的信息,如果顯示正常，說(shuō)明已經(jīng)搭建成功了，下面我們直接進(jìn)入kibana控制臺(tái)
http://localhost:5601/,初次進(jìn)入會(huì)讓你設(shè)置控制臺(tái)的密碼
我們進(jìn)入控制臺(tái),執(zhí)行一下，有如下輸出，至此我們就搭建成功了
如果你想卸載它們，執(zhí)行docker-compose down就可以了，畢竟這幾個(gè)家伙特別的吃資源。這里提醒一下大家，如果想嘗試到服務(wù)器安裝，建議新開一個(gè)機(jī)器，不要直接在生產(chǎn)環(huán)境里安裝，因?yàn)橥Τ杂布Y源的，會(huì)容易出問(wèn)題
以上就是docker搭建es集群實(shí)現(xiàn)過(guò)程詳解的詳細(xì)內(nèi)容，更多關(guān)于docker搭建es集群的資料請(qǐng)關(guān)注腳本之家其它相關(guān)文章！
相關(guān)文章
Windows與Linux下關(guān)于docker的安裝使用方法有很多，今天小編這里給大家分享的是MacOS下docker的安裝與簡(jiǎn)單使用案例，非常簡(jiǎn)單，給有需要的小伙伴參考下2017-03-03
Docker搭建LibreSpeed的實(shí)現(xiàn)步驟
LibreSpeed 是一個(gè)輕量級(jí)的網(wǎng)絡(luò)速度測(cè)試工具,本文主要介紹了Docker搭建LibreSpeed的實(shí)現(xiàn)步驟,具有一定的參考價(jià)值,感興趣的可以了解一下2024-04-04
Docker下安裝Mongo4.2及客戶端工具連接Mongo
這篇文章主要介紹了Docker下安裝Mongo4.2和客戶端工具連接Mongo數(shù)據(jù)庫(kù)的方法，本文給大家介紹的非常詳細(xì)，對(duì)大家的學(xué)習(xí)或工作具有一定的參考借鑒價(jià)值,需要的朋友可以參考下2022-01-01
Docker容器與宿主機(jī)相互訪問(wèn)更方便的方法
Docker是當(dāng)今使用范圍最廣的開源容器技術(shù)之一,具有高效易用的優(yōu)點(diǎn),然而如果使用Docker時(shí)采取不當(dāng)安全策略,則可能導(dǎo)致系統(tǒng)面臨安全威脅,這篇文章主要給大家介紹了關(guān)于Docker容器與宿主機(jī)相互訪問(wèn)更方便的方法,需要的朋友可以參考下2023-05-05
Docker?Kill/Pause/Unpause命令使用與區(qū)別小結(jié)
本文詳細(xì)介紹了Docker中的三個(gè)重要命令,kill、pause和unpause,這些命令在管理和操作運(yùn)行中的容器時(shí)非常有用,具有一定的參考價(jià)值,感興趣的可以了解一下2024-03-03
Spring Boot 2.4 新特性之一鍵構(gòu)建Docker鏡像的過(guò)程詳解
這篇文章主要介紹了Spring Boot 2.4 新特性之一鍵構(gòu)建Docker鏡像的過(guò)程詳解,本文給大家介紹的非常詳細(xì)，對(duì)大家的學(xué)習(xí)或工作具有一定的參考借鑒價(jià)值，需要的朋友可以參考下2020-12-12
Docker容器端口映射后突然無(wú)法連接的排查過(guò)程
這篇文章主要給大家介紹了關(guān)于Docker容器端口映射后突然無(wú)法連接的排查過(guò)程，文中通過(guò)示例代碼介紹的非常詳細(xì)，對(duì)大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價(jià)值，需要的朋友們下面來(lái)一起學(xué)習(xí)學(xué)習(xí)吧2019-03-03
docker啟動(dòng)報(bào)錯(cuò)問(wèn)題OCI runtime create failed: c
這篇文章主要介紹了docker啟動(dòng)報(bào)錯(cuò)問(wèn)題OCI runtime create failed: container_linux.go:380: starting container process,具有很好的參考價(jià)值,希望對(duì)大家有所幫助,如有錯(cuò)誤或未考慮完全的地方,望不吝賜教2024-08-08
用docker部署RabbitMQ環(huán)境的詳細(xì)介紹
這篇文章主要介紹了用docker部署RabbitMQ環(huán)境,本文給大家介紹的非常詳細(xì)，對(duì)大家的學(xué)習(xí)或工作具有一定的參考借鑒價(jià)值，需要的朋友可以參考下2021-07-07
docker通過(guò)Dockerfile修改鏡像中tomcat的端口
8080端口會(huì)經(jīng)常出現(xiàn)被占用的情況，本文主要介紹了docker通過(guò)Dockerfile修改鏡像中tomcat的端口,具有一定的參考價(jià)值，感興趣的可以了解一下2023-10-10