Docker實現(xiàn)安裝ELK(單節(jié)點)
Docker 安裝ELK(單節(jié)點)
創(chuàng)建docker網(wǎng)絡(luò)
docker network create -d bridge elastic
拉取elasticsearch 8.4.3版本
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.4.3 也可能是這個 docker pull elasticsearch:8.4.3
第一次執(zhí)行docker腳本
docker run -it \ -p 9200:9200 \ -p 9300:9300 \ --name elasticsearch \ --net elastic \ -e ES_JAVA_OPTS="-Xms1g -Xmx1g" \ -e "discovery.type=single-node" \ -e LANG=C.UTF-8 \ -e LC_ALL=C.UTF-8 \ elasticsearch:8.4.3
注意第一次執(zhí)行腳本不要加-d這個參數(shù),否則看不到服務(wù)首次運行時生成的隨機密碼和隨機 enrollment token
拷貝日志中的以下內(nèi)容,備用
? Elasticsearch security features have been automatically configured! ? Authentication is enabled and cluster connections are encrypted. ?? Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`): =HjjCu=tj1orDTLJbWPv ?? HTTP CA certificate SHA-256 fingerprint: 9204867e59a004b04c44a98d93c4609937ce3f14175a3eed7afa98ee31bbd4c2 ?? Configure Kibana to use this cluster: ? Run Kibana and click the configuration link in the terminal when Kibana starts. ? Copy the following enrollment token and paste it into Kibana in your browser (valid for the next 30 minutes): eyJ2ZXIiOiI4LjQuMyIsImFkciI6WyIxNzIuMjIuMC4yOjkyMDAiXSwiZmdyIjoiOTIwNDg2N2U1OWEwMDRiMDRjNDRhOThkOTNjNDYwOTkzN2NlM2YxNDE3NWEzZWVkN2FmYTk4ZWUzMWJiZDRjMiIsImtleSI6Img0bGNvSkFCYkJnR1BQQXRtb3VZOnpCcjZQMUtZVFhHb1VDS2paazRHRHcifQ== ?? Configure other nodes to join this cluster: ? Copy the following enrollment token and start new Elasticsearch nodes with `bin/elasticsearch --enrollment-token <token>` (valid for the next 30 minutes): eyJ2ZXIiOiI4LjQuMyIsImFkciI6WyIxNzIuMjIuMC4yOjkyMDAiXSwiZmdyIjoiOTIwNDg2N2U1OWEwMDRiMDRjNDRhOThkOTNjNDYwOTkzN2NlM2YxNDE3NWEzZWVkN2FmYTk4ZWUzMWJiZDRjMiIsImtleSI6ImhZbGNvSkFCYkJnR1BQQXRtb3VLOjRZWlFkN1JIUk5PcVJqZTlsX2p6LXcifQ== If you're running in Docker, copy the enrollment token and run: `docker run -e "ENROLLMENT_TOKEN=<token>" docker.elastic.co/elasticsearch/elasticsearch:8.4.3`
創(chuàng)建相應(yīng)目錄并復(fù)制配置文件到主機
mkdir -p apps/elk8.4.3/elasticsearch # 這個cp命令是在 /home/ubuntu目錄下執(zhí)行的 docker cp elasticsearch:/usr/share/elasticsearch/config apps/elk8.4.3/elasticsearch/ docker cp elasticsearch:/usr/share/elasticsearch/data apps/elk8.4.3/elasticsearch/ docker cp elasticsearch:/usr/share/elasticsearch/plugins apps/elk8.4.3/elasticsearch/ docker cp elasticsearch:/usr/share/elasticsearch/logs apps/elk8.4.3/elasticsearch/
刪除容器
docker rm -f elasticsearch
修改apps/elk8.4.3/elasticsearch/config/elasticsearch.yml
vim apps/elk8.4.3/elasticsearch/config/elasticsearch.yml
添加
- 增加:xpack.monitoring.collection.enabled: true
- 說明:添加這個配置以后在kibana中才會顯示聯(lián)機狀態(tài),否則會顯示脫機狀態(tài)
啟動elasticsearch
docker run -it \ -d \ -p 9200:9200 \ -p 9300:9300 \ --name elasticsearch \ --net elastic \ -e ES_JAVA_OPTS="-Xms1g -Xmx1g" \ -e "discovery.type=single-node" \ -e LANG=C.UTF-8 \ -e LC_ALL=C.UTF-8 \ -v /home/ubuntu/apps/elk8.4.3/elasticsearch/config:/usr/share/elasticsearch/config \ -v /home/ubuntu/apps/elk8.4.3/elasticsearch/data:/usr/share/elasticsearch/data \ -v /home/ubuntu/apps/elk8.4.3/elasticsearch/plugins:/usr/share/elasticsearch/plugins \ -v /home/ubuntu/apps/elk8.4.3/elasticsearch/logs:/usr/share/elasticsearch/logs \ elasticsearch:8.4.3
啟動驗證
https://xxxxx:9200/ 用戶名:elastic 密碼在第一次啟動時保存下來的信息中查找
Kibana
安裝Kibana
docker pull kibana:8.4.3
啟動Kibana
docker run -it \ --restart=always \ --log-driver json-file \ --log-opt max-size=100m \ --log-opt max-file=2 \ --name kibana \ -p 5601:5601 \ --net elastic \ kibana:8.4.3
初始化Kibana鑒權(quán)憑證
http://xxxx:5601/?code=878708
注意:
在textarea中填入之前elasticsearch生成的相關(guān)信息,注意這個token只有30分鐘的有效期,如果過期了只能進(jìn)入容器重置token,進(jìn)入容器執(zhí)行
/bin/elasticsearch-create-enrollment-token -s kibana --url "https://127.0.0.1:9200"
kibana驗證
將服務(wù)端的log中輸出的驗證碼,輸入到瀏覽器中,我這里是628503
創(chuàng)建kibana目錄并copy相關(guān)配置信息
mkdir apps/elk8.4.3/kibana # 這個cp命令是在 /home/ubuntu目錄下執(zhí)行的 docker cp kibana:/usr/share/kibana/config apps/elk8.4.3/kibana/ docker cp kibana:/usr/share/kibana/data apps/elk8.4.3/kibana/ docker cp kibana:/usr/share/kibana/plugins apps/elk8.4.3/kibana/ docker cp kibana:/usr/share/kibana/logs apps/elk8.4.3/kibana/ sudo chown -R 1000:1000 apps/elk8.4.3/kibana
修改apps/elk8.4.3/kibana/config/kibana.yml
### >>>>>>> BACKUP START: Kibana interactive setup (2024-03-25T07:30:11.689Z) # # ** THIS IS AN AUTO-GENERATED FILE ** # # Default Kibana configuration for docker target #server.host: "0.0.0.0" #server.shutdownTimeout: "5s" #elasticsearch.hosts: [ "http://elasticsearch:9200" ] #monitoring.ui.container.elasticsearch.enabled: true ### >>>>>>> BACKUP END: Kibana interactive setup (2024-03-25T07:30:11.689Z) # This section was automatically generated during setup. i18n.locale: "zh-CN" server.host: 0.0.0.0 server.shutdownTimeout: 5s # #這個ip一定是elasticsearch的容器ip,可使用docker inspect | grep -i ipaddress elasticsearch.hosts: ['https://your ip:9200'] monitoring.ui.container.elasticsearch.enabled: true elasticsearch.serviceAccountToken: AAEAAWVsYXN0aWMva2liYW5hL2Vucm9sbC1wcm9jZXNzLXRva2VuLTE3MTEzNTE4MTA5NDM6ZHZ1R3M5cV9RRlc2NmQ3dE9WaWM0QQ elasticsearch.ssl.certificateAuthorities: [/usr/share/kibana/data/ca_1711351811685.crt] xpack.fleet.outputs: [{id: fleet-default-output, name: default, is_default: true, is_default_monitoring: true, type: elasticsearch, hosts: ['https://your ip:9200'], ca_trusted_fingerprint: 5e7d9fe48c485c2761f9e7a99b9d5737e4e34dc55b9bf6929d929fb34d61a11a}]
刪除容器并重啟
docker rm -f kibana docker run -it \ -d \ --restart=always \ --log-driver json-file \ --log-opt max-size=100m \ --log-opt max-file=2 \ --name kibana \ -p 5601:5601 \ --net elastic \ -v /home/ubuntu/apps/elk8.4.3/kibana/config:/usr/share/kibana/config \ -v /home/ubuntu/apps/elk8.4.3/kibana/data:/usr/share/kibana/data \ -v /home/ubuntu/apps/elk8.4.3/kibana/plugins:/usr/share/kibana/plugins \ -v /home/ubuntu/apps/elk8.4.3/kibana/logs:/usr/share/kibana/logs \ kibana:8.4.3
Logstash
Logstash拉取鏡像
docker pull logstash:8.4.3
啟動
docker run -it \ -d \ --name logstash \ -p 9600:9600 \ -p 5044:5044 \ --net elastic \ logstash:8.4.3
創(chuàng)建目錄并同步配置文件
mkdir apps/elk8.4.3/logstash # 這個cp命令是在 /home/ubuntu目錄下執(zhí)行的 docker cp logstash:/usr/share/logstash/config apps/elk8.4.3/logstash/ docker cp logstash:/usr/share/logstash/pipeline apps/elk8.4.3/logstash/ sudo cp -rf apps/elk8.4.3/elasticsearch/config/certs apps/elk8.4.3/logstash/config/certs sudo chown -R 1000:1000 apps/elk8.4.3/logstash
修改配置apps/elk8.4.3/logstash/config/logstash.yml
http.host: "0.0.0.0" xpack.monitoring.enabled: true xpack.monitoring.elasticsearch.hosts: [ "http://your ip:9200" ] xpack.monitoring.elasticsearch.username: "elastic" # 第一次啟動elasticsearch是保存的信息中查找 L3WKr6ROTiK_DbqzBr8c xpack.monitoring.elasticsearch.password: "L3WKr6ROTiK_DbqzBr8c" xpack.monitoring.elasticsearch.ssl.certificate_authority: "/usr/share/logstash/config/certs/http_ca.crt" # 第一次啟動elasticsearch是保存的信息中查找 5e7d9fe48c485c2761f9e7a99b9d5737e4e34dc55b9bf6929d929fb34d61a11a xpack.monitoring.elasticsearch.ssl.ca_trusted_fingerprint: "5e7d9fe48c485c2761f9e7a99b9d5737e4e34dc55b9bf6929d929fb34d61a11a"
修改配置apps/elk8.4.3/logstash/pipeline/logstash.conf
input { beats { port => 5044 } } filter { date { # 因為我的日志里,我的time字段格式是2024-03-14T15:34:03+08:00 ,所以要使用以下兩行配置 match => [ "time", "ISO8601" ] target => "@timestamp" } json { source => "message" } mutate { remove_field => ["message", "path", "version", "@version", "agent", "cloud", "host", "input", "log", "tags", "_index", "_source", "ecs", "event"] } } output { elasticsearch { hosts => ["https://your ip:9200"] index => "douyin-%{+YYYY.MM.dd}" ssl => true ssl_certificate_verification => false cacert => "/usr/share/logstash/config/certs/http_ca.crt" ca_trusted_fingerprint => "第一次啟動elasticsearch是保存的信息中查找e924551c1453c893114a05656882eea81cb11dd87c1258f83e6f676d2428f8f2" user => "elastic" password => "第一次啟動elasticsearch是保存的信息中查找UkNx8px1yrMYIht30QUc" } }
刪除容器并重新啟動
docker rm -f logstash docker run -it \ -d \ --name logstash \ -p 9600:9600 \ -p 5044:5044 \ --net elastic \ -v /home/ubuntu/apps/elk8.4.3/logstash/config:/usr/share/logstash/config \ -v /home/ubuntu/apps/elk8.4.3/logstash/pipeline:/usr/share/logstash/pipeline \ logstash:8.4.3
Filebeat
Filebeat拉取鏡像
sudo docker pull elastic/filebeat:8.4.3
FileBeat啟動
docker run -it \ -d \ --name filebeat \ --network elastic \ -e TZ=Asia/Shanghai \ elastic/filebeat:8.4.3 \ filebeat -e -c /usr/share/filebeat/filebeat.yml docker run -d --name filebeat \ -v /home/linyanbo/docker_data/filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml \ -v /home/linyanbo/docker_data/filebeat/data:/usr/share/filebeat/data \ -v /var/logs/:/var/log \ --link elasticsearch:elasticsearch \ --network elastic \ --user root \ elastic/filebeat:8.4.3
設(shè)置開機啟動
docker update elasticsearch --restart=always
配置文件
filebeat.yml
filebeat.inputs: - type: log enabled: true paths: - /var/logs/duty-admin/spring.log/crmduty-admin-2024-07-12.log fields: log_source: oh-promotion fields_under_root: true multiline.pattern: ^\d{4}-\d{1,2}-\d{1,2} multiline.negate: true multiline.match: after scan_frequency: 5s close_inactive: 1h ignore_older: 24h output.logstash: hosts: ["your ip:5044"]
logstash.conf
input { beats { port => 5044 } } filter { # mutate { # split => {"message"=>" "} # } mutate { add_field => { "mm" => "%{message}" } } } output { elasticsearch { hosts => ["https://your ip:9200"] #index => "duty-admin%{+YYYY.MM.dd}" index => "duty-admin%{+YYYY}" ssl => true ssl_certificate_verification => false cacert => "/usr/share/logstash/config/certs/http_ca.crt" ca_trusted_fingerprint => "9204867e59a004b04c44a98d93c4609937ce3f14175a3eed7afa98ee31bbd4c2" user => "elastic" password => "=HjjCu=tj1orDTLJbWPv" } } output { stdout { codec => rubydebug } }
elasticsearch.yml
cluster.name: "docker-cluster" network.host: 0.0.0.0 #----------------------- BEGIN SECURITY AUTO CONFIGURATION ----------------------- # # The following settings, TLS certificates, and keys have been automatically # generated to configure Elasticsearch security features on 11-07-2024 05:54:41 # # -------------------------------------------------------------------------------- # Enable security features xpack.security.enabled: true # 說明:添加這個配置以后在kibana中才會顯示聯(lián)機狀態(tài),否則會顯示脫機狀態(tài) xpack.monitoring.collection.enabled: true xpack.security.enrollment.enabled: true # Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents xpack.security.http.ssl: enabled: true keystore.path: certs/http.p12 # Enable encryption and mutual authentication between cluster nodes xpack.security.transport.ssl: enabled: true verification_mode: certificate keystore.path: certs/transport.p12 truststore.path: certs/transport.p12 #----------------------- END SECURITY AUTO CONFIGURATION -------------------------
kibana.yml
### >>>>>>> BACKUP START: Kibana interactive setup (2024-07-11T06:09:35.897Z) # # ** THIS IS AN AUTO-GENERATED FILE ** # # Default Kibana configuration for docker target #server.host: "0.0.0.0" #server.shutdownTimeout: "5s" #elasticsearch.hosts: [ "http://elasticsearch:9200" ] #monitoring.ui.container.elasticsearch.enabled: true ### >>>>>>> BACKUP END: Kibana interactive setup (2024-07-11T06:09:35.897Z) # This section was automatically generated during setup. server.host: 0.0.0.0 server.shutdownTimeout: 5s elasticsearch.hosts: ['https://your ip:9200'] monitoring.ui.container.elasticsearch.enabled: true elasticsearch.serviceAccountToken: AAEAAWVsYXN0aWMva2liYW5hL2Vucm9sbC1wcm9jZXNzLXRva2VuLTE3MjA2NzgxNzU2MzU6bU5RR25uQUVSaWExbUdHQ2tsODRmZw elasticsearch.ssl.certificateAuthorities: [/usr/share/kibana/data/ca_1720678175894.crt] xpack.fleet.outputs: [{id: fleet-default-output, name: default, is_default: true, is_default_monitoring: true, type: elasticsearch, hosts: ['https://your ip:9200'], ca_trusted_fingerprint: 9204867e59a004b04c44a98d93c4609937ce3f14175a3eed7afa98ee31bbd4c2}]
總結(jié)
以上為個人經(jīng)驗,希望能給大家一個參考,也希望大家多多支持腳本之家。
相關(guān)文章
解決Docker Desktop運行一直轉(zhuǎn)問題
遇到Docker啟動報錯、Docker Desktop運行不正常等問題,常見解決方案包括以管理員身份運行CMD,重啟Docker服務(wù),恢復(fù)Docker Desktop出廠設(shè)置,啟用Windows功能等,最終通過運行cmd命令"netsh winsock reset"解決問題,但需注意此操作可能導(dǎo)致Docker鏡像丟失2024-09-09docker容器中登陸并操作postgresql的實現(xiàn)
本文主要介紹了docker容器中登陸并操作postgresql的實現(xiàn),文中通過示例代碼介紹的非常詳細(xì),對大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價值,需要的朋友們下面隨著小編來一起學(xué)習(xí)學(xué)習(xí)吧2024-02-02docker的pdflatex環(huán)境配置的方法步驟
這篇文章主要介紹了docker的pdflatex環(huán)境配置的方法步驟,文中通過示例代碼介紹的非常詳細(xì),對大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價值,需要的朋友們下面隨著小編來一起學(xué)習(xí)學(xué)習(xí)吧2021-03-03關(guān)于為Windows?Service?2019?使用?Docker的問題
最近收到領(lǐng)導(dǎo)通知,甲方需要將原來的服務(wù)器遷移到新的服務(wù)器,原來的服務(wù)器上安裝了很多的服務(wù),每次重啟之后總是有很多的問題需要人工大量的進(jìn)行干預(yù),今天小編通過本文給大家介紹下Windows?Service?2019?使用?Docker的問題,需要的朋友參考下吧2022-05-05docker上安裝和卸載MySQL數(shù)據(jù)庫詳細(xì)圖文教程
Docker是一種容器化技術(shù),可以快速方便地部署和管理應(yīng)用程序,Mysql 是一個開源的關(guān)系型數(shù)據(jù)庫管理,系統(tǒng)這篇文章主要給大家介紹了關(guān)于docker上安裝和卸載MySQL數(shù)據(jù)庫的相關(guān)資料,需要的朋友可以參考下2024-05-05