Docker實現(xiàn)安裝ELK(單節(jié)點)
更新時間：2024年08月14日 16:37:56 作者：wang18057
這篇文章主要介紹了Docker實現(xiàn)安裝ELK(單節(jié)點),具有很好的參考價值,希望對大家有所幫助,如有錯誤或未考慮完全的地方,望不吝賜教
Docker 安裝ELK(單節(jié)點)
創(chuàng)建docker網(wǎng)絡(luò)
docker network create -d bridge elastic
拉取elasticsearch 8.4.3版本
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.4.3 也可能是這個 docker pull elasticsearch:8.4.3
第一次執(zhí)行docker腳本
docker run -it \ -p 9200:9200 \ -p 9300:9300 \ --name elasticsearch \ --net elastic \ -e ES_JAVA_OPTS="-Xms1g -Xmx1g" \ -e "discovery.type=single-node" \ -e LANG=C.UTF-8 \ -e LC_ALL=C.UTF-8 \ elasticsearch:8.4.3
注意第一次執(zhí)行腳本不要加-d這個參數(shù)，否則看不到服務(wù)首次運行時生成的隨機密碼和隨機 enrollment token
拷貝日志中的以下內(nèi)容，備用
? Elasticsearch security features have been automatically configured! ? Authentication is enabled and cluster connections are encrypted. ?? Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`): =HjjCu=tj1orDTLJbWPv ?? HTTP CA certificate SHA-256 fingerprint: 9204867e59a004b04c44a98d93c4609937ce3f14175a3eed7afa98ee31bbd4c2 ?? Configure Kibana to use this cluster: ? Run Kibana and click the configuration link in the terminal when Kibana starts. ? Copy the following enrollment token and paste it into Kibana in your browser (valid for the next 30 minutes): eyJ2ZXIiOiI4LjQuMyIsImFkciI6WyIxNzIuMjIuMC4yOjkyMDAiXSwiZmdyIjoiOTIwNDg2N2U1OWEwMDRiMDRjNDRhOThkOTNjNDYwOTkzN2NlM2YxNDE3NWEzZWVkN2FmYTk4ZWUzMWJiZDRjMiIsImtleSI6Img0bGNvSkFCYkJnR1BQQXRtb3VZOnpCcjZQMUtZVFhHb1VDS2paazRHRHcifQ== ?? Configure other nodes to join this cluster: ? Copy the following enrollment token and start new Elasticsearch nodes with `bin/elasticsearch --enrollment-token <token>` (valid for the next 30 minutes): eyJ2ZXIiOiI4LjQuMyIsImFkciI6WyIxNzIuMjIuMC4yOjkyMDAiXSwiZmdyIjoiOTIwNDg2N2U1OWEwMDRiMDRjNDRhOThkOTNjNDYwOTkzN2NlM2YxNDE3NWEzZWVkN2FmYTk4ZWUzMWJiZDRjMiIsImtleSI6ImhZbGNvSkFCYkJnR1BQQXRtb3VLOjRZWlFkN1JIUk5PcVJqZTlsX2p6LXcifQ== If you're running in Docker, copy the enrollment token and run: `docker run -e "ENROLLMENT_TOKEN=<token>" docker.elastic.co/elasticsearch/elasticsearch:8.4.3`
創(chuàng)建相應(yīng)目錄并復(fù)制配置文件到主機
mkdir -p apps/elk8.4.3/elasticsearch # 這個cp命令是在 /home/ubuntu目錄下執(zhí)行的 docker cp elasticsearch:/usr/share/elasticsearch/config apps/elk8.4.3/elasticsearch/ docker cp elasticsearch:/usr/share/elasticsearch/data apps/elk8.4.3/elasticsearch/ docker cp elasticsearch:/usr/share/elasticsearch/plugins apps/elk8.4.3/elasticsearch/ docker cp elasticsearch:/usr/share/elasticsearch/logs apps/elk8.4.3/elasticsearch/
刪除容器
docker rm -f elasticsearch
修改apps/elk8.4.3/elasticsearch/config/elasticsearch.yml
vim apps/elk8.4.3/elasticsearch/config/elasticsearch.yml
添加
- 增加：xpack.monitoring.collection.enabled: true
- 說明：添加這個配置以后在kibana中才會顯示聯(lián)機狀態(tài)，否則會顯示脫機狀態(tài)
啟動elasticsearch
docker run -it \ -d \ -p 9200:9200 \ -p 9300:9300 \ --name elasticsearch \ --net elastic \ -e ES_JAVA_OPTS="-Xms1g -Xmx1g" \ -e "discovery.type=single-node" \ -e LANG=C.UTF-8 \ -e LC_ALL=C.UTF-8 \ -v /home/ubuntu/apps/elk8.4.3/elasticsearch/config:/usr/share/elasticsearch/config \ -v /home/ubuntu/apps/elk8.4.3/elasticsearch/data:/usr/share/elasticsearch/data \ -v /home/ubuntu/apps/elk8.4.3/elasticsearch/plugins:/usr/share/elasticsearch/plugins \ -v /home/ubuntu/apps/elk8.4.3/elasticsearch/logs:/usr/share/elasticsearch/logs \ elasticsearch:8.4.3
啟動驗證
https://xxxxx:9200/ 用戶名:elastic 密碼在第一次啟動時保存下來的信息中查找
Kibana
安裝Kibana
docker pull kibana:8.4.3
啟動Kibana
docker run -it \ --restart=always \ --log-driver json-file \ --log-opt max-size=100m \ --log-opt max-file=2 \ --name kibana \ -p 5601:5601 \ --net elastic \ kibana:8.4.3
初始化Kibana鑒權(quán)憑證
http://xxxx:5601/?code=878708
注意：
在textarea中填入之前elasticsearch生成的相關(guān)信息,注意這個token只有30分鐘的有效期，如果過期了只能進(jìn)入容器重置token,進(jìn)入容器執(zhí)行
/bin/elasticsearch-create-enrollment-token -s kibana --url "https://127.0.0.1:9200"
kibana驗證
將服務(wù)端的log中輸出的驗證碼，輸入到瀏覽器中，我這里是628503
創(chuàng)建kibana目錄并copy相關(guān)配置信息
mkdir apps/elk8.4.3/kibana # 這個cp命令是在 /home/ubuntu目錄下執(zhí)行的 docker cp kibana:/usr/share/kibana/config apps/elk8.4.3/kibana/ docker cp kibana:/usr/share/kibana/data apps/elk8.4.3/kibana/ docker cp kibana:/usr/share/kibana/plugins apps/elk8.4.3/kibana/ docker cp kibana:/usr/share/kibana/logs apps/elk8.4.3/kibana/ sudo chown -R 1000:1000 apps/elk8.4.3/kibana
修改apps/elk8.4.3/kibana/config/kibana.yml
### >>>>>>> BACKUP START: Kibana interactive setup (2024-03-25T07:30:11.689Z)
#
# ** THIS IS AN AUTO-GENERATED FILE **
#
# Default Kibana configuration for docker target
#server.host: "0.0.0.0"
#server.shutdownTimeout: "5s"
#elasticsearch.hosts: [ "http://elasticsearch:9200" ]
#monitoring.ui.container.elasticsearch.enabled: true
### >>>>>>> BACKUP END: Kibana interactive setup (2024-03-25T07:30:11.689Z)
# This section was automatically generated during setup.
i18n.locale: "zh-CN"
server.host: 0.0.0.0
server.shutdownTimeout: 5s
# #這個ip一定是elasticsearch的容器ip，可使用docker inspect | grep -i ipaddress
elasticsearch.hosts: ['https://your ip:9200']
monitoring.ui.container.elasticsearch.enabled: true
elasticsearch.serviceAccountToken: AAEAAWVsYXN0aWMva2liYW5hL2Vucm9sbC1wcm9jZXNzLXRva2VuLTE3MTEzNTE4MTA5NDM6ZHZ1R3M5cV9RRlc2NmQ3dE9WaWM0QQ
elasticsearch.ssl.certificateAuthorities: [/usr/share/kibana/data/ca_1711351811685.crt]
xpack.fleet.outputs: [{id: fleet-default-output, name: default, is_default: true, is_default_monitoring: true, type: elasticsearch, hosts: ['https://your ip:9200'], ca_trusted_fingerprint: 5e7d9fe48c485c2761f9e7a99b9d5737e4e34dc55b9bf6929d929fb34d61a11a}]刪除容器并重啟
docker rm -f kibana docker run -it \ -d \ --restart=always \ --log-driver json-file \ --log-opt max-size=100m \ --log-opt max-file=2 \ --name kibana \ -p 5601:5601 \ --net elastic \ -v /home/ubuntu/apps/elk8.4.3/kibana/config:/usr/share/kibana/config \ -v /home/ubuntu/apps/elk8.4.3/kibana/data:/usr/share/kibana/data \ -v /home/ubuntu/apps/elk8.4.3/kibana/plugins:/usr/share/kibana/plugins \ -v /home/ubuntu/apps/elk8.4.3/kibana/logs:/usr/share/kibana/logs \ kibana:8.4.3
Logstash
Logstash拉取鏡像
docker pull logstash:8.4.3
啟動
docker run -it \ -d \ --name logstash \ -p 9600:9600 \ -p 5044:5044 \ --net elastic \ logstash:8.4.3
創(chuàng)建目錄并同步配置文件
mkdir apps/elk8.4.3/logstash # 這個cp命令是在 /home/ubuntu目錄下執(zhí)行的 docker cp logstash:/usr/share/logstash/config apps/elk8.4.3/logstash/ docker cp logstash:/usr/share/logstash/pipeline apps/elk8.4.3/logstash/ sudo cp -rf apps/elk8.4.3/elasticsearch/config/certs apps/elk8.4.3/logstash/config/certs sudo chown -R 1000:1000 apps/elk8.4.3/logstash
修改配置apps/elk8.4.3/logstash/config/logstash.yml
http.host: "0.0.0.0" xpack.monitoring.enabled: true xpack.monitoring.elasticsearch.hosts: [ "http://your ip:9200" ] xpack.monitoring.elasticsearch.username: "elastic" # 第一次啟動elasticsearch是保存的信息中查找 L3WKr6ROTiK_DbqzBr8c xpack.monitoring.elasticsearch.password: "L3WKr6ROTiK_DbqzBr8c" xpack.monitoring.elasticsearch.ssl.certificate_authority: "/usr/share/logstash/config/certs/http_ca.crt" # 第一次啟動elasticsearch是保存的信息中查找 5e7d9fe48c485c2761f9e7a99b9d5737e4e34dc55b9bf6929d929fb34d61a11a xpack.monitoring.elasticsearch.ssl.ca_trusted_fingerprint: "5e7d9fe48c485c2761f9e7a99b9d5737e4e34dc55b9bf6929d929fb34d61a11a"
修改配置apps/elk8.4.3/logstash/pipeline/logstash.conf
input {
beats {
port => 5044
}
}
filter {
date {
# 因為我的日志里，我的time字段格式是2024-03-14T15:34:03+08:00 ，所以要使用以下兩行配置
match => [ "time", "ISO8601" ]
target => "@timestamp"
}
json {
source => "message"
}
mutate {
remove_field => ["message", "path", "version", "@version", "agent", "cloud", "host", "input", "log", "tags", "_index", "_source", "ecs", "event"]
}
}
output {
elasticsearch {
hosts => ["https://your ip:9200"]
index => "douyin-%{+YYYY.MM.dd}"
ssl => true
ssl_certificate_verification => false
cacert => "/usr/share/logstash/config/certs/http_ca.crt"
ca_trusted_fingerprint => "第一次啟動elasticsearch是保存的信息中查找e924551c1453c893114a05656882eea81cb11dd87c1258f83e6f676d2428f8f2"
user => "elastic"
password => "第一次啟動elasticsearch是保存的信息中查找UkNx8px1yrMYIht30QUc"
}
}刪除容器并重新啟動
docker rm -f logstash docker run -it \ -d \ --name logstash \ -p 9600:9600 \ -p 5044:5044 \ --net elastic \ -v /home/ubuntu/apps/elk8.4.3/logstash/config:/usr/share/logstash/config \ -v /home/ubuntu/apps/elk8.4.3/logstash/pipeline:/usr/share/logstash/pipeline \ logstash:8.4.3
Filebeat
Filebeat拉取鏡像
sudo docker pull elastic/filebeat:8.4.3
FileBeat啟動
docker run -it \ -d \ --name filebeat \ --network elastic \ -e TZ=Asia/Shanghai \ elastic/filebeat:8.4.3 \ filebeat -e -c /usr/share/filebeat/filebeat.yml docker run -d --name filebeat \ -v /home/linyanbo/docker_data/filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml \ -v /home/linyanbo/docker_data/filebeat/data:/usr/share/filebeat/data \ -v /var/logs/:/var/log \ --link elasticsearch:elasticsearch \ --network elastic \ --user root \ elastic/filebeat:8.4.3
設(shè)置開機啟動
docker update elasticsearch --restart=always
配置文件
filebeat.yml
filebeat.inputs:
- type: log
enabled: true
paths:
- /var/logs/duty-admin/spring.log/crmduty-admin-2024-07-12.log
fields:
log_source: oh-promotion
fields_under_root: true
multiline.pattern: ^\d{4}-\d{1,2}-\d{1,2}
multiline.negate: true
multiline.match: after
scan_frequency: 5s
close_inactive: 1h
ignore_older: 24h
output.logstash:
hosts: ["your ip:5044"]logstash.conf
input {
beats {
port => 5044
}
}
filter {
# mutate {
# split => {"message"=>" "}
# }
mutate {
add_field => {
"mm" => "%{message}"
}
}
}
output {
elasticsearch {
hosts => ["https://your ip:9200"]
#index => "duty-admin%{+YYYY.MM.dd}"
index => "duty-admin%{+YYYY}"
ssl => true
ssl_certificate_verification => false
cacert => "/usr/share/logstash/config/certs/http_ca.crt"
ca_trusted_fingerprint => "9204867e59a004b04c44a98d93c4609937ce3f14175a3eed7afa98ee31bbd4c2"
user => "elastic"
password => "=HjjCu=tj1orDTLJbWPv"
}
}
output {
stdout {
codec => rubydebug
}
}elasticsearch.yml
cluster.name: "docker-cluster" network.host: 0.0.0.0 #----------------------- BEGIN SECURITY AUTO CONFIGURATION ----------------------- # # The following settings, TLS certificates, and keys have been automatically # generated to configure Elasticsearch security features on 11-07-2024 05:54:41 # # -------------------------------------------------------------------------------- # Enable security features xpack.security.enabled: true # 說明：添加這個配置以后在kibana中才會顯示聯(lián)機狀態(tài)，否則會顯示脫機狀態(tài) xpack.monitoring.collection.enabled: true xpack.security.enrollment.enabled: true # Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents xpack.security.http.ssl: enabled: true keystore.path: certs/http.p12 # Enable encryption and mutual authentication between cluster nodes xpack.security.transport.ssl: enabled: true verification_mode: certificate keystore.path: certs/transport.p12 truststore.path: certs/transport.p12 #----------------------- END SECURITY AUTO CONFIGURATION -------------------------
kibana.yml
### >>>>>>> BACKUP START: Kibana interactive setup (2024-07-11T06:09:35.897Z)
#
# ** THIS IS AN AUTO-GENERATED FILE **
#
# Default Kibana configuration for docker target
#server.host: "0.0.0.0"
#server.shutdownTimeout: "5s"
#elasticsearch.hosts: [ "http://elasticsearch:9200" ]
#monitoring.ui.container.elasticsearch.enabled: true
### >>>>>>> BACKUP END: Kibana interactive setup (2024-07-11T06:09:35.897Z)
# This section was automatically generated during setup.
server.host: 0.0.0.0
server.shutdownTimeout: 5s
elasticsearch.hosts: ['https://your ip:9200']
monitoring.ui.container.elasticsearch.enabled: true
elasticsearch.serviceAccountToken: AAEAAWVsYXN0aWMva2liYW5hL2Vucm9sbC1wcm9jZXNzLXRva2VuLTE3MjA2NzgxNzU2MzU6bU5RR25uQUVSaWExbUdHQ2tsODRmZw
elasticsearch.ssl.certificateAuthorities: [/usr/share/kibana/data/ca_1720678175894.crt]
xpack.fleet.outputs: [{id: fleet-default-output, name: default, is_default: true, is_default_monitoring: true, type: elasticsearch, hosts: ['https://your ip:9200'], ca_trusted_fingerprint: 9204867e59a004b04c44a98d93c4609937ce3f14175a3eed7afa98ee31bbd4c2}]
總結(jié)
以上為個人經(jīng)驗，希望能給大家一個參考，也希望大家多多支持腳本之家。
相關(guān)文章
在啟動容器后,狀態(tài)顯示為“Exited”,日志顯示連接ZooKeeper超時,原因是端口不通,通過配置安全組相關(guān)端口后重新啟動容器,查看日志顯示“started”,說明容器啟動成功2024-09-09
解決Docker Desktop運行一直轉(zhuǎn)問題
遇到Docker啟動報錯、Docker Desktop運行不正常等問題,常見解決方案包括以管理員身份運行CMD,重啟Docker服務(wù),恢復(fù)Docker Desktop出廠設(shè)置,啟用Windows功能等,最終通過運行cmd命令"netsh winsock reset"解決問題,但需注意此操作可能導(dǎo)致Docker鏡像丟失2024-09-09
這篇文章主要介紹了docker之安裝nginx的教程,具有很好的參考價值,希望對大家有所幫助,如有錯誤或未考慮完全的地方,望不吝賜教2025-03-03
本文主要介紹了docker容器重啟錯誤的解決方案,詳細(xì)的介紹了docker 重啟錯誤的原因及其解決方法,具有一定的參考價值,感興趣的可以了解一下2024-07-07
docker容器中登陸并操作postgresql的實現(xiàn)
本文主要介紹了docker容器中登陸并操作postgresql的實現(xiàn),文中通過示例代碼介紹的非常詳細(xì),對大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價值,需要的朋友們下面隨著小編來一起學(xué)習(xí)學(xué)習(xí)吧2024-02-02
docker的pdflatex環(huán)境配置的方法步驟
這篇文章主要介紹了docker的pdflatex環(huán)境配置的方法步驟，文中通過示例代碼介紹的非常詳細(xì)，對大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價值，需要的朋友們下面隨著小編來一起學(xué)習(xí)學(xué)習(xí)吧2021-03-03
Docker從1.13版本之后采用時間線的方式作為版本號，分為社區(qū)版CE和企業(yè)版EE。這篇文章給大家介紹了Centos7上安裝docker的方法，感興趣的朋友一起看看吧2018-07-07
Harbor是一個用于存儲和分發(fā)Docker鏡像的企業(yè)級Registry服務(wù)器,本文主要介紹了docker自建GitLab倉庫的實現(xiàn),具有一定的參考價值,感興趣的可以了解一下2024-07-07
關(guān)于為Windows?Service?2019?使用?Docker的問題
最近收到領(lǐng)導(dǎo)通知，甲方需要將原來的服務(wù)器遷移到新的服務(wù)器，原來的服務(wù)器上安裝了很多的服務(wù)，每次重啟之后總是有很多的問題需要人工大量的進(jìn)行干預(yù)，今天小編通過本文給大家介紹下Windows?Service?2019?使用?Docker的問題，需要的朋友參考下吧2022-05-05
docker上安裝和卸載MySQL數(shù)據(jù)庫詳細(xì)圖文教程
Docker是一種容器化技術(shù),可以快速方便地部署和管理應(yīng)用程序,Mysql 是一個開源的關(guān)系型數(shù)據(jù)庫管理,系統(tǒng)這篇文章主要給大家介紹了關(guān)于docker上安裝和卸載MySQL數(shù)據(jù)庫的相關(guān)資料,需要的朋友可以參考下2024-05-05