Java動態(tài)線程池插件dynamic-tp集成過程淺析
前言
dynamic-tp
是一個輕量級的動態(tài)線程池插件,它是一個基于配置中心的動態(tài)線程池,線程池的參數(shù)可以通過配置中心配置進行動態(tài)的修改,目前支持的配置中心有Apollo
,Nacos
和Zookeeper
,同時dynamic-tp
支持線程池的監(jiān)控和報警,具體特性如下:
- 基于
Spring
框架,現(xiàn)只支持SpringBoot
項目使用,輕量級,引入 starter 即可使用 - 基于配置中心實現(xiàn)線程池參數(shù)動態(tài)調(diào)整,實時生效;集成主流配置中心,已支持 Nacos、Apollo,Zookeeper, 同時也提供 SPI 接口可自定義擴展實現(xiàn)
- 內(nèi)置通知報警功能,提供多種報警維度(配置變更通知、活性報警、容量閾值報警、拒絕策略觸發(fā)報警), 默認支持企業(yè)微信、釘釘報警,同時提供 SPI 接口可自定義擴展實現(xiàn)
- 內(nèi)置線程池指標采集功能,支持通過 MicroMeter、JsonLog 日志輸出、Endpoint 三種方式,可通過 SPI 接口自定義擴展實現(xiàn)
- 集成管理常用第三方組件的線程池,已集成 SpringBoot 內(nèi)置 WebServer(tomcat、undertow、jetty)的線程池管理
- Builder 提供
TTL
包裝功能,生成的線程池支持上下文信息傳遞
為了方便用戶快速接入,筆者這里對Zookeeper
配置中心接入做一個詳細的介紹,其他配置中心的demo示例可以參考項目的example
,該模塊也是筆者貢獻的。這里我會詳細的介紹Zookeeper
配置中心的接入和對接監(jiān)控平臺Prometheus
和Grafana
。
快速開始
pom依賴
dynamic-tp-spring-boot-starter-zookeeper
是集成dynamic-tp
的starter,這里引入了micrometer-registry-prometheus
和spring-boot-starter-actuator
用于對接Prometheus
。
<dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-logging</artifactId> </dependency> <dependency> <groupId>io.github.lyh200</groupId> <artifactId>dynamic-tp-spring-boot-starter-zookeeper</artifactId> <version>1.0.2</version> </dependency> <dependency> <groupId>io.micrometer</groupId> <artifactId>micrometer-registry-prometheus</artifactId> <version>1.6.4</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> </dependencies>
YML配置
server:
port: 8888spring:
application:
name: dynamic-tp-zookeeper-demo
# 下面是接入zk配置中心的配置
dynamic:
tp:
config-type: properties # zookeeper只支持properties配置
zookeeper:
config-version: 1.0.0 # 配置版本號
zk-connect-str: 127.0.0.1:2181 # zk配置中心,如果是集群用逗號分開
root-node: /configserver/userproject # 項目節(jié)點
node: dynamic-tp-zookeeper-demo # 配置文件節(jié)點
# 對接prometheus
management:
metrics:
tags:
application: ${spring.application.name}
endpoints:
web:
exposure:
include: '*'
配置中心dynamic-tp-zookeeper-demo配置文件配置
注:Zookeeper配置中心只支持properties
類型配置,配置示例如下:
spring.dynamic.tp.executors
部分是對線程池的配置,該配置是數(shù)組類型,可以定義多個線程池
spring.dynamic.tp.executors[1].notifyItems
部分配置是對線程池報警平臺的配置,可以配置多個報警平臺
# 開啟動態(tài)線程池
spring.dynamic.tp.enabled=true
# 打印動態(tài)線程池banner
spring.dynamic.tp.enabledBanner=true
# 開啟線程池監(jiān)控指標收集
spring.dynamic.tp.enabledCollect=true
# 線程池監(jiān)控指標收集類型 logging-日志文件 micrometer-采集到第三方平臺
spring.dynamic.tp.collectorType=logging
# 采集監(jiān)控數(shù)據(jù)時間間隔 5s
spring.dynamic.tp.monitorInterval=5
spring.dynamic.tp.executors[0].threadPoolName=dynamic-tp-test-1
spring.dynamic.tp.executors[0].corePoolSize=50
spring.dynamic.tp.executors[0].maximumPoolSize=50
spring.dynamic.tp.executors[0].queueCapacity=3000
spring.dynamic.tp.executors[0].queueType=VariableLinkedBlockingQueue
spring.dynamic.tp.executors[0].rejectedHandlerType=CallerRunsPolicy
spring.dynamic.tp.executors[0].keepAliveTime=50
spring.dynamic.tp.executors[0].allowCoreThreadTimeOut=false
spring.dynamic.tp.executors[0].threadNamePrefix=test1
spring.dynamic.tp.executors[0].notifyItems[0].type=capacity
spring.dynamic.tp.executors[0].notifyItems[0].enabled=false
spring.dynamic.tp.executors[0].notifyItems[0].threshold=80
spring.dynamic.tp.executors[0].notifyItems[0].platforms[0]=ding
spring.dynamic.tp.executors[0].notifyItems[0].platforms[1]=wechat
spring.dynamic.tp.executors[0].notifyItems[0].interval=120
spring.dynamic.tp.executors[0].notifyItems[1].type=change
spring.dynamic.tp.executors[0].notifyItems[1].enabled=false
spring.dynamic.tp.executors[0].notifyItems[2].type=liveness
spring.dynamic.tp.executors[0].notifyItems[2].enabled=false
spring.dynamic.tp.executors[0].notifyItems[2].threshold=80
spring.dynamic.tp.executors[0].notifyItems[3].type=reject
spring.dynamic.tp.executors[0].notifyItems[3].enabled=false
spring.dynamic.tp.executors[0].notifyItems[3].threshold=1
spring.dynamic.tp.executors[1].threadPoolName=dynamic-tp-test-2
spring.dynamic.tp.executors[1].corePoolSize=20
spring.dynamic.tp.executors[1].maximumPoolSize=30
spring.dynamic.tp.executors[1].queueCapacity=1000
spring.dynamic.tp.executors[1].queueType=VariableLinkedBlockingQueue
spring.dynamic.tp.executors[1].rejectedHandlerType=CallerRunsPolicy
spring.dynamic.tp.executors[1].keepAliveTime=50
spring.dynamic.tp.executors[1].allowCoreThreadTimeOut=false
spring.dynamic.tp.executors[1].threadNamePrefix=test2
spring.dynamic.tp.executors[1].notifyItems[0].type=capacity
spring.dynamic.tp.executors[1].notifyItems[0].enabled=false
spring.dynamic.tp.executors[1].notifyItems[0].threshold=80
spring.dynamic.tp.executors[1].notifyItems[0].platforms[0]=ding
spring.dynamic.tp.executors[1].notifyItems[0].platforms[1]=wechat
spring.dynamic.tp.executors[1].notifyItems[0].interval=120
spring.dynamic.tp.executors[1].notifyItems[1].type=change
spring.dynamic.tp.executors[1].notifyItems[1].enabled=false
spring.dynamic.tp.executors[1].notifyItems[2].type=liveness
spring.dynamic.tp.executors[1].notifyItems[2].enabled=false
spring.dynamic.tp.executors[1].notifyItems[2].threshold=80
spring.dynamic.tp.executors[1].notifyItems[3].type=reject
spring.dynamic.tp.executors[1].notifyItems[3].enabled=false
spring.dynamic.tp.executors[1].notifyItems[3].threshold=1
創(chuàng)建線程池
創(chuàng)建線程池,會注冊到dynamic-tp
@Configuration public class ThreadPoolConfiguration { @Bean public DtpExecutor dtpExecutor() { return ThreadPoolCreator.createDynamicFast("dynamic-tp-test-1"); } @Bean public ThreadPoolExecutor threadPoolExecutor() { return ThreadPoolBuilder.newBuilder() .threadPoolName("dynamic-tp-test-2") .corePoolSize(10) .maximumPoolSize(15) .keepAliveTime(15000) .timeUnit(TimeUnit.MILLISECONDS) .workQueue(QueueTypeEnum.SYNCHRONOUS_QUEUE.getName(), null, false) .buildDynamic(); } }
使用
初次使用不注意官網(wǎng)文檔的話可能會通過@Autowired
的方式使用,目前版本這么使用是不正確的,因為當讀取配置文件后,配置文件生成的線程池對象就會把@Bean
創(chuàng)建的線程池覆蓋了,所以目前版本只能根據(jù)名字從DtpRegistry
獲取線程池對象了,筆者也和項目作者溝通過,項目作者也覺得通過@Autowired
方式更好一些,后面版本也會優(yōu)化。
@Slf4j @RestController @SuppressWarnings("all") public class TestController { @GetMapping("/dtp-zookeeper-example/test") public String test() { new Thread(() -> { try { task(); } catch (InterruptedException e) { e.printStackTrace(); } }).start(); return "success"; } public void task() throws InterruptedException { DtpExecutor dtpExecutor1 = DtpRegistry.getExecutor("dynamic-tp-test-1"); DtpExecutor dtpExecutor2 = DtpRegistry.getExecutor("dynamic-tp-test-2"); for (int i = 0; i < 100; i++) { Thread.sleep(100); dtpExecutor1.execute(() -> { log.info("i am dynamic-tp-test-1 task"); }); dtpExecutor2.execute(() -> { log.info("i am dynamic-tp-test-2 task"); }); } } }
監(jiān)控數(shù)據(jù)和第三方平臺對接
logging方式
spring.dynamic.tp.collectorType=logging
,logging方式的監(jiān)控數(shù)據(jù),采集日志文件中,文件的位置可以通過spring.dynamic.tp.logPath
配置,默認 ${user.home}/logs,日志內(nèi)容如下:
2022-03-07 13:48:31.585 INFO [dtp-monitor1:D.M.LOG] {"activeCount":0,"queueSize":0,"largestPoolSize":0,"poolSize":0,"rejectHandlerName":"CallerRunsPolicy","queueCapacity":0,"fair":false,"rejectCount":0,"waitTaskCount":0,"taskCount":0,"queueRemainingCapacity":0,"corePoolSize":40,"queueType":"SynchronousQueue","completedTaskCount":0,"dtpName":"dynamic-tp-test-2","maximumPoolSize":55}
2022-03-07 13:48:31.585 INFO [dtp-monitor1:D.M.LOG] {"activeCount":0,"queueSize":0,"largestPoolSize":0,"poolSize":0,"queueCapacity":2147483647,"fair":false,"rejectCount":0,"waitTaskCount":0,"taskCount":0,"queueRemainingCapacity":2147483647,"corePoolSize":10,"queueType":"TaskQueue","completedTaskCount":0,"dtpName":"tomcatWebServerTp","maximumPoolSize":200}
2022-03-07 13:48:36.590 INFO [dtp-monitor1:D.M.LOG] {"activeCount":0,"queueSize":0,"largestPoolSize":0,"poolSize":0,"rejectHandlerName":"CallerRunsPolicy","queueCapacity":1024,"fair":false,"rejectCount":0,"waitTaskCount":0,"taskCount":0,"queueRemainingCapacity":1024,"corePoolSize":50,"queueType":"VariableLinkedBlockingQueue","completedTaskCount":0,"dtpName":"dynamic-tp-test-1","maximumPoolSize":50}
2022-03-07 13:48:36.590 INFO [dtp-monitor1:D.M.LOG] {"activeCount":0,"queueSize":0,"largestPoolSize":0,"poolSize":0,"rejectHandlerName":"CallerRunsPolicy","queueCapacity":0,"fair":false,"rejectCount":0,"waitTaskCount":0,"taskCount":0,"queueRemainingCapacity":0,"corePoolSize":40,"queueType":"SynchronousQueue","completedTaskCount":0,"dtpName":"dynamic-tp-test-2","maximumPoolSize":55}
2022-03-07 13:48:36.590 INFO [dtp-monitor1:D.M.LOG] {"activeCount":0,"queueSize":0,"largestPoolSize":0,"poolSize":0,"queueCapacity":2147483647,"fair":false,"rejectCount":0,"waitTaskCount":0,"taskCount":0,"queueRemainingCapacity":2147483647,"corePoolSize":10,"queueType":"TaskQueue","completedTaskCount":0,"dtpName":"tomcatWebServerTp","maximumPoolSize":200}
MicroMeter方式
spring.dynamic.tp.collectorType=micrometer
采集數(shù)據(jù)到第三方平臺,這里使用的是Prometheus
??梢暬故拘枰?code>Grafana平臺配置,數(shù)據(jù)源選擇Prometheus
,然后創(chuàng)建Dashboard
,先要在Dashboard
配置動態(tài)的參數(shù)Variables
,如下圖:
配置完參數(shù)后,我們創(chuàng)建Panel
,可以創(chuàng)建多個Panel
,然后配置Panel
的數(shù)據(jù)查詢,報警顯示等等,如下圖:
最終的Dashboard如下圖:
總結(jié)
促使我使用dynamic-tp
的一個最主要的點就是它輕量級,提供了線程池監(jiān)控、報警、基于配置中心修改線程池參數(shù)功能,完全滿足了我在業(yè)務(wù)中的需要,同時監(jiān)控數(shù)據(jù)能夠?qū)庸粳F(xiàn)有的監(jiān)控平臺(Prometheus+Grafana),并且在使用上也十分方便,我在剛接觸這個插件的時候它還不支持Zookeeper
,恰好我司使用的配置中心就是Zookeeper
,因為我在此基礎(chǔ)上把Zookeeper
配置中心集成到了dynamic-tp
并貢獻給社區(qū),希望可以幫到更多的人,下一篇簡單介紹下dynamic-tp
支持Zookeeper
的實現(xiàn)細節(jié)。
到此這篇關(guān)于Java動態(tài)線程池插件dynamic-tp集成過程淺析的文章就介紹到這了,更多相關(guān)Java dynamic-tp內(nèi)容請搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家!
相關(guān)文章
Spring Boot使用GridFS實現(xiàn)文件的上傳和下載方式
這篇文章主要介紹了Spring Boot使用GridFS實現(xiàn)文件的上傳和下載方式,具有很好的參考價值,希望對大家有所幫助。如有錯誤或未考慮完全的地方,望不吝賜教2021-10-10spring使用Filter過濾器對Response返回值進行修改的方法
這篇文章主要介紹了spring使用Filter過濾器對Response返回值進行修改,本文通過實例代碼給大家介紹的非常詳細,對大家的學習或工作具有一定的參考借鑒價值,需要的朋友可以參考下2023-09-09使用Apache Spark進行Java數(shù)據(jù)分析的步驟詳解
今天我們將探討如何使用Apache Spark進行Java數(shù)據(jù)分析,Apache Spark是一個強大的大數(shù)據(jù)處理引擎,它支持批處理和流處理,特別適合處理大規(guī)模數(shù)據(jù)集,在Java中使用Spark,我們可以利用其強大的數(shù)據(jù)處理能力來進行各種數(shù)據(jù)分析任務(wù),需要的朋友可以參考下2024-07-07Spring中基于Java的配置@Configuration和@Bean用法詳解
這篇文章主要介紹了Spring中基于Java的配置@Configuration和@Bean用法詳解,Spring中為了減少xml中配置,可以聲明一個配置類(例如SpringConfig)來對bean進行配置。,需要的朋友可以參考下2019-06-06