前言

dynamic-tp是一個(gè)輕量級(jí)的動(dòng)態(tài)線程池插件，它是一個(gè)基于配置中心的動(dòng)態(tài)線程池，線程池的參數(shù)可以通過配置中心配置進(jìn)行動(dòng)態(tài)的修改，目前支持的配置中心有Apollo,Nacos和Zookeeper，同時(shí)dynamic-tp支持線程池的監(jiān)控和報(bào)警，具體特性如下：

• 基于Spring框架，現(xiàn)只支持SpringBoot項(xiàng)目使用，輕量級(jí)，引入 starter 即可使用
• 基于配置中心實(shí)現(xiàn)線程池參數(shù)動(dòng)態(tài)調(diào)整，實(shí)時(shí)生效；集成主流配置中心，已支持 Nacos、Apollo，Zookeeper， 同時(shí)也提供 SPI 接口可自定義擴(kuò)展實(shí)現(xiàn)
• 內(nèi)置通知報(bào)警功能，提供多種報(bào)警維度（配置變更通知、活性報(bào)警、容量閾值報(bào)警、拒絕策略觸發(fā)報(bào)警）， 默認(rèn)支持企業(yè)微信、釘釘報(bào)警，同時(shí)提供 SPI 接口可自定義擴(kuò)展實(shí)現(xiàn)
• 內(nèi)置線程池指標(biāo)采集功能，支持通過 MicroMeter、JsonLog 日志輸出、Endpoint 三種方式，可通過 SPI 接口自定義擴(kuò)展實(shí)現(xiàn)
• 集成管理常用第三方組件的線程池，已集成 SpringBoot 內(nèi)置 WebServer（tomcat、undertow、jetty）的線程池管理
• Builder 提供TTL包裝功能，生成的線程池支持上下文信息傳遞

具體介紹及使用方式

為了方便用戶快速接入，筆者這里對(duì)Zookeeper配置中心接入做一個(gè)詳細(xì)的介紹，其他配置中心的demo示例可以參考項(xiàng)目的example，該模塊也是筆者貢獻(xiàn)的。這里我會(huì)詳細(xì)的介紹Zookeeper配置中心的接入和對(duì)接監(jiān)控平臺(tái)Prometheus和Grafana。

快速開始

pom依賴

dynamic-tp-spring-boot-starter-zookeeper是集成dynamic-tp的starter，這里引入了micrometer-registry-prometheus和spring-boot-starter-actuator用于對(duì)接Prometheus。

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-logging</artifactId>
        </dependency>
        <dependency>
            <groupId>io.github.lyh200</groupId>
            <artifactId>dynamic-tp-spring-boot-starter-zookeeper</artifactId>
            <version>1.0.2</version>
        </dependency>
        <dependency>
            <groupId>io.micrometer</groupId>
            <artifactId>micrometer-registry-prometheus</artifactId>
            <version>1.6.4</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
    </dependencies>

YML配置

server:
  port: 8888

spring:
  application:
    name: dynamic-tp-zookeeper-demo
  # 下面是接入zk配置中心的配置  
  dynamic:
    tp:
      config-type: properties # zookeeper只支持properties配置
      zookeeper:
        config-version: 1.0.0 # 配置版本號(hào)
        zk-connect-str: 127.0.0.1:2181 # zk配置中心，如果是集群用逗號(hào)分開
        root-node: /configserver/userproject # 項(xiàng)目節(jié)點(diǎn)
        node: dynamic-tp-zookeeper-demo # 配置文件節(jié)點(diǎn)
# 對(duì)接prometheus
management:
  metrics:
    tags:
      application: ${spring.application.name}
  endpoints:
    web:
      exposure:
        include: '*'

配置中心dynamic-tp-zookeeper-demo配置文件配置

注：Zookeeper配置中心只支持properties類型配置，配置示例如下：

spring.dynamic.tp.executors部分是對(duì)線程池的配置，該配置是數(shù)組類型，可以定義多個(gè)線程池

spring.dynamic.tp.executors[1].notifyItems部分配置是對(duì)線程池報(bào)警平臺(tái)的配置，可以配置多個(gè)報(bào)警平臺(tái)

# 開啟動(dòng)態(tài)線程池
spring.dynamic.tp.enabled=true
# 打印動(dòng)態(tài)線程池banner
spring.dynamic.tp.enabledBanner=true
# 開啟線程池監(jiān)控指標(biāo)收集
spring.dynamic.tp.enabledCollect=true
# 線程池監(jiān)控指標(biāo)收集類型 logging-日志文件 micrometer-采集到第三方平臺(tái) 
spring.dynamic.tp.collectorType=logging
# 采集監(jiān)控?cái)?shù)據(jù)時(shí)間間隔 5s
spring.dynamic.tp.monitorInterval=5
spring.dynamic.tp.executors[0].threadPoolName=dynamic-tp-test-1
spring.dynamic.tp.executors[0].corePoolSize=50
spring.dynamic.tp.executors[0].maximumPoolSize=50
spring.dynamic.tp.executors[0].queueCapacity=3000
spring.dynamic.tp.executors[0].queueType=VariableLinkedBlockingQueue
spring.dynamic.tp.executors[0].rejectedHandlerType=CallerRunsPolicy
spring.dynamic.tp.executors[0].keepAliveTime=50
spring.dynamic.tp.executors[0].allowCoreThreadTimeOut=false
spring.dynamic.tp.executors[0].threadNamePrefix=test1
spring.dynamic.tp.executors[0].notifyItems[0].type=capacity
spring.dynamic.tp.executors[0].notifyItems[0].enabled=false
spring.dynamic.tp.executors[0].notifyItems[0].threshold=80
spring.dynamic.tp.executors[0].notifyItems[0].platforms[0]=ding
spring.dynamic.tp.executors[0].notifyItems[0].platforms[1]=wechat
spring.dynamic.tp.executors[0].notifyItems[0].interval=120
spring.dynamic.tp.executors[0].notifyItems[1].type=change
spring.dynamic.tp.executors[0].notifyItems[1].enabled=false
spring.dynamic.tp.executors[0].notifyItems[2].type=liveness
spring.dynamic.tp.executors[0].notifyItems[2].enabled=false
spring.dynamic.tp.executors[0].notifyItems[2].threshold=80
spring.dynamic.tp.executors[0].notifyItems[3].type=reject
spring.dynamic.tp.executors[0].notifyItems[3].enabled=false
spring.dynamic.tp.executors[0].notifyItems[3].threshold=1
spring.dynamic.tp.executors[1].threadPoolName=dynamic-tp-test-2
spring.dynamic.tp.executors[1].corePoolSize=20
spring.dynamic.tp.executors[1].maximumPoolSize=30
spring.dynamic.tp.executors[1].queueCapacity=1000
spring.dynamic.tp.executors[1].queueType=VariableLinkedBlockingQueue
spring.dynamic.tp.executors[1].rejectedHandlerType=CallerRunsPolicy
spring.dynamic.tp.executors[1].keepAliveTime=50
spring.dynamic.tp.executors[1].allowCoreThreadTimeOut=false
spring.dynamic.tp.executors[1].threadNamePrefix=test2
spring.dynamic.tp.executors[1].notifyItems[0].type=capacity
spring.dynamic.tp.executors[1].notifyItems[0].enabled=false
spring.dynamic.tp.executors[1].notifyItems[0].threshold=80
spring.dynamic.tp.executors[1].notifyItems[0].platforms[0]=ding
spring.dynamic.tp.executors[1].notifyItems[0].platforms[1]=wechat
spring.dynamic.tp.executors[1].notifyItems[0].interval=120
spring.dynamic.tp.executors[1].notifyItems[1].type=change
spring.dynamic.tp.executors[1].notifyItems[1].enabled=false
spring.dynamic.tp.executors[1].notifyItems[2].type=liveness
spring.dynamic.tp.executors[1].notifyItems[2].enabled=false
spring.dynamic.tp.executors[1].notifyItems[2].threshold=80
spring.dynamic.tp.executors[1].notifyItems[3].type=reject
spring.dynamic.tp.executors[1].notifyItems[3].enabled=false
spring.dynamic.tp.executors[1].notifyItems[3].threshold=1

創(chuàng)建線程池

創(chuàng)建線程池，會(huì)注冊(cè)到dynamic-tp

@Configuration
public class ThreadPoolConfiguration {
    @Bean
    public DtpExecutor dtpExecutor() {

        return ThreadPoolCreator.createDynamicFast("dynamic-tp-test-1");
    }
    @Bean
    public ThreadPoolExecutor threadPoolExecutor() {
        return ThreadPoolBuilder.newBuilder()
                .threadPoolName("dynamic-tp-test-2")
                .corePoolSize(10)
                .maximumPoolSize(15)
                .keepAliveTime(15000)
                .timeUnit(TimeUnit.MILLISECONDS)
                .workQueue(QueueTypeEnum.SYNCHRONOUS_QUEUE.getName(), null, false)
                .buildDynamic();
    }
}

使用

初次使用不注意官網(wǎng)文檔的話可能會(huì)通過@Autowired的方式使用，目前版本這么使用是不正確的，因?yàn)楫?dāng)讀取配置文件后，配置文件生成的線程池對(duì)象就會(huì)把@Bean創(chuàng)建的線程池覆蓋了，所以目前版本只能根據(jù)名字從DtpRegistry獲取線程池對(duì)象了，筆者也和項(xiàng)目作者溝通過，項(xiàng)目作者也覺得通過@Autowired方式更好一些，后面版本也會(huì)優(yōu)化。

@Slf4j
@RestController
@SuppressWarnings("all")
public class TestController {
    @GetMapping("/dtp-zookeeper-example/test")
    public String test() {
        new Thread(() -> {
            try {
                task();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }).start();
        return "success";
    }
    public void task() throws InterruptedException {
        DtpExecutor dtpExecutor1 = DtpRegistry.getExecutor("dynamic-tp-test-1");
        DtpExecutor dtpExecutor2 = DtpRegistry.getExecutor("dynamic-tp-test-2");
        for (int i = 0; i < 100; i++) {
            Thread.sleep(100);
            dtpExecutor1.execute(() -> {
                log.info("i am dynamic-tp-test-1 task");
            });
            dtpExecutor2.execute(() -> {
                log.info("i am dynamic-tp-test-2 task");
            });
        }
    }
}

監(jiān)控?cái)?shù)據(jù)和第三方平臺(tái)對(duì)接

logging方式

spring.dynamic.tp.collectorType=logging,logging方式的監(jiān)控?cái)?shù)據(jù)，采集日志文件中，文件的位置可以通過spring.dynamic.tp.logPath配置，默認(rèn) ${user.home}/logs，日志內(nèi)容如下：

2022-03-07 13:48:31.585 INFO [dtp-monitor1:D.M.LOG] {"activeCount":0,"queueSize":0,"largestPoolSize":0,"poolSize":0,"rejectHandlerName":"CallerRunsPolicy","queueCapacity":0,"fair":false,"rejectCount":0,"waitTaskCount":0,"taskCount":0,"queueRemainingCapacity":0,"corePoolSize":40,"queueType":"SynchronousQueue","completedTaskCount":0,"dtpName":"dynamic-tp-test-2","maximumPoolSize":55}
2022-03-07 13:48:31.585 INFO [dtp-monitor1:D.M.LOG] {"activeCount":0,"queueSize":0,"largestPoolSize":0,"poolSize":0,"queueCapacity":2147483647,"fair":false,"rejectCount":0,"waitTaskCount":0,"taskCount":0,"queueRemainingCapacity":2147483647,"corePoolSize":10,"queueType":"TaskQueue","completedTaskCount":0,"dtpName":"tomcatWebServerTp","maximumPoolSize":200}
2022-03-07 13:48:36.590 INFO [dtp-monitor1:D.M.LOG] {"activeCount":0,"queueSize":0,"largestPoolSize":0,"poolSize":0,"rejectHandlerName":"CallerRunsPolicy","queueCapacity":1024,"fair":false,"rejectCount":0,"waitTaskCount":0,"taskCount":0,"queueRemainingCapacity":1024,"corePoolSize":50,"queueType":"VariableLinkedBlockingQueue","completedTaskCount":0,"dtpName":"dynamic-tp-test-1","maximumPoolSize":50}
2022-03-07 13:48:36.590 INFO [dtp-monitor1:D.M.LOG] {"activeCount":0,"queueSize":0,"largestPoolSize":0,"poolSize":0,"rejectHandlerName":"CallerRunsPolicy","queueCapacity":0,"fair":false,"rejectCount":0,"waitTaskCount":0,"taskCount":0,"queueRemainingCapacity":0,"corePoolSize":40,"queueType":"SynchronousQueue","completedTaskCount":0,"dtpName":"dynamic-tp-test-2","maximumPoolSize":55}
2022-03-07 13:48:36.590 INFO [dtp-monitor1:D.M.LOG] {"activeCount":0,"queueSize":0,"largestPoolSize":0,"poolSize":0,"queueCapacity":2147483647,"fair":false,"rejectCount":0,"waitTaskCount":0,"taskCount":0,"queueRemainingCapacity":2147483647,"corePoolSize":10,"queueType":"TaskQueue","completedTaskCount":0,"dtpName":"tomcatWebServerTp","maximumPoolSize":200}

MicroMeter方式

spring.dynamic.tp.collectorType=micrometer 采集數(shù)據(jù)到第三方平臺(tái)，這里使用的是Prometheus?？梢暬故拘枰?code>Grafana平臺(tái)配置，數(shù)據(jù)源選擇Prometheus，然后創(chuàng)建Dashboard，先要在Dashboard配置動(dòng)態(tài)的參數(shù)Variables，如下圖：

配置完參數(shù)后，我們創(chuàng)建Panel，可以創(chuàng)建多個(gè)Panel，然后配置Panel的數(shù)據(jù)查詢，報(bào)警顯示等等，如下圖：

最終的Dashboard如下圖：

總結(jié)

促使我使用dynamic-tp的一個(gè)最主要的點(diǎn)就是它輕量級(jí)，提供了線程池監(jiān)控、報(bào)警、基于配置中心修改線程池參數(shù)功能，完全滿足了我在業(yè)務(wù)中的需要，同時(shí)監(jiān)控?cái)?shù)據(jù)能夠?qū)庸粳F(xiàn)有的監(jiān)控平臺(tái)（Prometheus+Grafana），并且在使用上也十分方便，我在剛接觸這個(gè)插件的時(shí)候它還不支持Zookeeper，恰好我司使用的配置中心就是Zookeeper，因?yàn)槲以诖嘶A(chǔ)上把Zookeeper配置中心集成到了dynamic-tp并貢獻(xiàn)給社區(qū)，希望可以幫到更多的人，下一篇簡(jiǎn)單介紹下dynamic-tp支持Zookeeper的實(shí)現(xiàn)細(xì)節(jié)。

到此這篇關(guān)于Java動(dòng)態(tài)線程池插件dynamic-tp集成過程淺析的文章就介紹到這了,更多相關(guān)Java dynamic-tp內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！

最新評(píng)論