docker安裝RocketMQ的實現(xiàn)(附填坑經(jīng)驗connect to failed)

更新時間：2024年06月27日 09:39:32 作者：郭咖啡

本文主要介紹了docker安裝RocketMQ(附填坑經(jīng)驗connect to failed)

一、docker部署RocketMQ

1、簡易說明

在RocketMQ中，有三個關(guān)鍵組件：Namesrv（Name Server）、Broker和Console-ng（管理控制臺）。

Namesrv（Name Server）：Namesrv是RocketMQ的命名服務(wù)，負責管理整個RocketMQ集群的路由信息。每個RocketMQ集群中都至少需要一個Namesrv實例。它維護了Broker的網(wǎng)絡(luò)信息、Topic的路由規(guī)則以及Consumer的消費進度等元數(shù)據(jù)，并提供給Producer和Consumer使用。
Broker：Broker是RocketMQ的消息存儲和處理節(jié)點，負責存儲消息、處理消息的讀寫請求和轉(zhuǎn)發(fā)消息等功能。在RocketMQ集群中，可以有多個Broker實例，各個Broker通過與Namesrv交互來維護消息的元數(shù)據(jù)和路由信息，以實現(xiàn)高可用、負載均衡的消息傳輸。
Console-ng（管理控制臺）：Console-ng是RocketMQ官方提供的管理控制臺，用于管理和監(jiān)控RocketMQ集群。它提供了圖形化界面，可以進行Topic、Consumer等的配置管理、消息查詢與追蹤、監(jiān)控指標展示等操作。Console-ng對于集群的監(jiān)控和運維非常有用。

這三個組件共同構(gòu)成了RocketMQ的核心架構(gòu)，并協(xié)同工作以實現(xiàn)高可用、高性能的消息傳輸和數(shù)據(jù)管理。您可以通過啟動Namesrv、Broker來搭建一個RocketMQ集群，并使用Console-ng進行集群的管理與監(jiān)控。

2、docker拉取RocketMQ鏡像\RocketMQ控制臺

拉取最新的RocketMQ，如果下載指定版本可以去docker官網(wǎng)查看
注：Namesrv、Broker均采用rocketmqinc/rocketmq同一個鏡像

# 最新(rocketmq)
docker pull rocketmqinc/rocketmq
# 指定版本號(rocketmq)
docker pull rocketmqinc/rocketmq:<版本號>

# 最新(RocketMQ控制臺)
docker pull  pangliang/rocketmq-console-ng

3、獲取RocketMQ配置文件

可以啟動RocketMQ，然后從docker容器中拷貝出配置文件，拷出配置文件后啟動的容器就可以刪除了

docker run -d --name rmqnamesrv -p 9876:9876 rocketmqinc/rocketmq:latest sh mqnamesrv
# 進入容器(用于進入容器找到broker.conf 的位置)
docker exec -it 容器ID /bin/bash
# 從容器中下載文件到虛擬機
docker cp 容器ID:/opt/rocketmq-4.4.0/conf/broker.conf 虛擬機路徑

4、RocketMQ配置文件描述

ongPollingEnable=true
offsetCheckInSlave=false
# nameServer地址，分號分割
namesrvAddr=172.16.234.150:9876
fetchNamesrvAddrByAddressServer=false
#是否允許 Broker 自動創(chuàng)建訂閱組，建議線下開啟，線上關(guān)閉
autoCreateSubscriptionGroup=true
#是否允許 Broker 自動創(chuàng)建Topic，建議線下開啟，線上關(guān)閉
autoCreateTopicEnable=true
sendThreadPoolQueueCapacity=100000
clusterTopicEnable=true
filterServerNums=1
pullMessageThreadPoolNums=20
# broker名字，名字可重復,為了管理,每個master起一個名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a
brokerName=knBroker
#rocketmqHome=/usr/local/alibaba-rocketmq/
sendMessageThreadPoolNums=24
# 0 表示 Master，>0 表示 Slave
brokerId=0
brokerIP1=172.16.234.150
brokerTopicEnable=true
brokerPermission=6
shortPollingTimeMills=1000
clientManageThreadPoolNums=16
adminBrokerThreadPoolNums=16
flushConsumerOffsetInterval=5000
flushConsumerOffsetHistoryInterval=60000
# 在發(fā)送消息時，自動創(chuàng)建服務(wù)器不存在的topic，默認創(chuàng)建的隊列數(shù)
defaultTopicQueueNums=8
rejectTransactionMessage=false
notifyConsumerIdsChangedEnable=true
pullThreadPoolQueueCapacity=100000
# # 所屬集群名字
brokerClusterName=DefaultCluster
putMsgIndexHightWater=600000
maxTransferBytesOnMessageInDisk=65536
#檢測物理文件磁盤空間
diskMaxUsedSpaceRatio=75
checkCRCOnRecover=true
haSlaveFallbehindMax=268435
deleteConsumeQueueFilesInterval=100
cleanResourceInterval=10000
maxMsgsNumBatch=64
flushConsumeQueueLeastPages=2
syncFlushTimeout=5000
#刪除文件時間點，默認凌晨 4點
deleteWhen=04
#Broker 的角色
brokerRole=ASYNC_MASTER
destroyMapedFileIntervalForcibly=120000
#commitLog每個文件的大小默認1G
mapedFileSizeCommitLog=1073741824
haSendHeartbeatInterval=5000
#刷盤方式
flushDiskType=ASYNC_FLUSH
cleanFileForciblyEnable=true
haHousekeepingInterval=20000
redeleteHangedFileInterval=120000
#限制的消息大小
maxMessageSize=524288
flushCommitLogTimed=false
haMasterAddress=
maxTransferCountOnMessageInDisk=4
flushIntervalCommitLog=1000
#文件保留時間，默認 48 小時
fileReservedTime=72
flushCommitLogThoroughInterval=10000
maxHashSlotNum=5000
maxIndexNum=20000
messageIndexEnable=true
#存儲路徑
storePathRootDir=/root/store
#commitLog 存儲路徑
storePathCommitLog=/root/store/commitlog
#消費隊列存儲路徑存儲路徑
storePathConsumeQueue=/root/store/consumequeue
#消息索引存儲路徑
storePathIndex=/root/store/index
haListenPort=10912
flushDelayOffsetInterval=10000
haTransferBatchSize=32768
deleteCommitLogFilesInterval=100
maxTransferBytesOnMessageInMemory=262144
accessMessageInMemoryMaxRatio=40
flushConsumeQueueThoroughInterval=60000
flushIntervalConsumeQueue=1000
maxTransferCountOnMessageInMemory=32
messageIndexSafe=false
#ConsumeQueue每個文件默認存30W條，根據(jù)業(yè)務(wù)情況調(diào)整
mapedFileSizeConsumeQueue=6000000
messageDelayLevel=1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h
flushCommitLogLeastPages=4
serverChannelMaxIdleTimeSeconds=120
#Broker 對外服務(wù)的監(jiān)聽端口
listenPort=10911
serverCallbackExecutorThreads=0
serverAsyncSemaphoreValue=64
serverSocketSndBufSize=131072
serverSelectorThreads=3
serverPooledByteBufAllocatorEnable=false
serverWorkerThreads=8
serverSocketRcvBufSize=131072
serverOnewaySemaphoreValue=256
clientWorkerThreads=4
connectTimeoutMillis=3000
clientSocketRcvBufSize=131072
clientOnewaySemaphoreValue=2048
clientChannelMaxIdleTimeSeconds=120
clientPooledByteBufAllocatorEnable=false
clientAsyncSemaphoreValue=2048
channelNotActiveInterval=60000
clientCallbackExecutorThreads=2
clientSocketSndBufSize=131072

5、docker啟動RocketMQ

以下將配置文件、日志、存儲均掛載在本地

# 1、namesrv
docker run -d -p 9876:9876 \
-v /mydata/rocketmq/namesrv/logs:/root/logs \
-v /mydata/rocketmq/namesrv/store:/root/store \
-v /mydata/rocketmq/conf/broker.conf:/opt/rocketmq-4.4.0/conf/broker.conf \
--name rmqnamesrv \
rocketmqinc/rocketmq:latest sh mqnamesrv


# 2、broker
docker run -d  -p 10911:10911 -p 10909:10909 \
-v /mydata/rocketmq/broker/logs:/root/logs \
-v /mydata/rocketmq/broker/store:/root/store \
-v /mydata/rocketmq/conf/broker.conf:/opt/rocketmq-4.4.0/conf/broker.conf \
--name rmqbroker \
--add-host namesrv:172.16.234.150 \
-e "NAMESRV_ADDR=namesrv:9876" \
rocketmqinc/rocketmq:latest \
sh mqbroker -n namesrv:9876 \
-c /opt/rocketmq-4.4.0/conf/broker.conf autoCreateTopicEnable=true

# 3、Console-ng
docker run --name rocketmq-console \
-e "JAVA_OPTS=-Drocketmq.namesrv.addr=172.16.234.150:9876 \
-Dcom.rocketmq.sendMessageWithVIPChannel=false" \
-p 8080:8080 -t styletang/rocketmq-console-ng

6、進入RocketMQ控制臺

地址(IP+rocketmq-console端口)：http://172.16.234.150:8080/#/

在這里插入圖片描述

二、填坑經(jīng)驗

錯誤一: connect to <172.17.0.3:10909> failed

1、在啟動Java項目后，發(fā)送MQ消息是報以下錯誤
2、RocketMQ控制臺，集群地址顯示為docker分配的IP

進入RocketMQ控制臺，控制臺顯示的集群地址為172.17.0.3:10909，并非172.16.234.150:10909，172.17.0.3實際為docker內(nèi)部分配的ID，需要此IP修改為虛擬機的IP

為了解決以上問題使用docker創(chuàng)建了一個網(wǎng)格【docker network create rocketmq-net + docker run -d --network rocketmq-net …】，但是沒有解決此問題，實際導致此問題的是RocketMQ配置文件，請核查RocketMQ配置文件，或者采用上方提供的配置文件

com.himyidea.framework.mq.MQRuntimeException: EC = 900101: MSG = 900101 | msg=MQ Client Failure
	at com.himyidea.framework.mq.producer.impl.GeneralMQProducer.doSend(GeneralMQProducer.java:130)
	at com.himyidea.framework.mq.producer.impl.GeneralMQProducer.sendMessage(GeneralMQProducer.java:105)
	at java.lang.Thread.run(Thread.java:748)
Caused by: com.alibaba.rocketmq.client.exception.MQClientException: Send [3] times, still failed, cost [9073]ms, Topic: report_data_topic, BrokersSent: [broker-a, broker-a, broker-a]
See http://docs.aliyun.com/cn#/pub/ons/faq/exceptions&send_msg_failed for further details.
	at com.alibaba.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:522)
	at com.alibaba.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1030)
	at com.alibaba.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:989)
	at com.alibaba.rocketmq.client.producer.DefaultMQProducer.send(DefaultMQProducer.java:90)
	at com.himyidea.framework.mq.producer.impl.GeneralMQProducer.doSend(GeneralMQProducer.java:126)
	... 85 more
Caused by: com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <172.17.0.3:10909> failed
	at com.alibaba.rocketmq.remoting.netty.NettyRemotingClient.invokeSync(NettyRemotingClient.java:360)
	at com.alibaba.rocketmq.client.impl.MQClientAPIImpl.sendMessageSync(MQClientAPIImpl.java:267)
	at com.alibaba.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:251)
	at com.alibaba.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:214)
	at com.alibaba.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendKernelImpl(DefaultMQProducerImpl.java:671)
	at com.alibaba.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:440)

錯誤二: maybe your broker machine memory too small

內(nèi)存不足，以為是docker啟動命令中未傳內(nèi)存信息，實際是虛擬機可用內(nèi)存空間不足

Caused by: com.alibaba.rocketmq.client.exception.MQBrokerException: CODE: 14  DESC: service not available now, maybe disk full, CL:  0.99 CQ: -1.00 INDEX: -1.00, maybe your broker machine memory too small.

到此這篇關(guān)于docker安裝RocketMQ的實現(xiàn)(附填坑經(jīng)驗connect to failed)的文章就介紹到這了,更多相關(guān)docker安裝RocketMQ內(nèi)容請搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！

您可能感興趣的文章: