Springboot線程池并發(fā)處理數(shù)據(jù)優(yōu)化方式
第一步：首先配置線程基本參數(shù)
可以放在application.propertes文件種也可以放在自己新建的config/文件目錄下，注意：但是需要使用@PropertySource把配置文件進(jìn)行加載。
# 異步線程配置 # 配置核心線程數(shù) async.executor.thread.core_pool_size = 8 # 配置最大線程數(shù) async.executor.thread.max_pool_size = 20 # 配置隊(duì)列大小 async.executor.thread.queue_capacity = 99999 # 配置線程池中的線程的名稱前綴 async.executor.thread.name.prefix = async-service-
第二步：讓Spring Boot加載
用來定義如何創(chuàng)建一個(gè)ThreadPoolTaskExecutor，要使用@Configuration和@EnableAsync這兩個(gè)注解，表示這是個(gè)配置類，并且是線程池的配置類
@Slf4j
@EnableAsync
@Configuration
public class RCExecutorConfig {
@Value("${async.executor.thread.core_pool_size}")
private int corePoolSize;
@Value("${async.executor.thread.max_pool_size}")
private int maxPoolSize;
@Value("${async.executor.thread.queue_capacity}")
private int queueCapacity;
@Value("${async.executor.thread.name.prefix}")
private String namePrefix;
@Bean(name = "asyncServiceExecutor")
public Executor asyncServiceExecutor() {
log.info("start asyncServiceExecutor");
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
//ThreadPoolTaskExecutor executor = new VisiableThreadPoolTaskExecutor();
//配置核心線程數(shù)
executor.setCorePoolSize(corePoolSize);
//配置最大線程數(shù)
executor.setMaxPoolSize(maxPoolSize);
//配置隊(duì)列大小
executor.setQueueCapacity(queueCapacity);
//配置線程池中的線程的名稱前綴
executor.setThreadNamePrefix(namePrefix);
// rejection-policy：當(dāng)pool已經(jīng)達(dá)到max size的時(shí)候，如何處理新任務(wù)
// CALLER_RUNS：不在新線程中執(zhí)行任務(wù)，而是有調(diào)用者所在的線程來執(zhí)行
executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
//執(zhí)行初始化
executor.initialize();
return executor;
}
}
第三步：創(chuàng)建一個(gè)service接口
是異步線程的接口，便于測試
public interface AsyncService {
/**
* 執(zhí)行異步任務(wù)
* 可以根據(jù)需求，自己加參數(shù)擬定
*/
void executeAsync();
}
第四步：編寫現(xiàn)實(shí)類
將Service層的服務(wù)異步化，在executeAsync()方法上增加注解@Async("asyncServiceExecutor")，asyncServiceExecutor方法是前面RCExecutorConfig.java中的方法名，表明executeAsync方法進(jìn)入的線程池是asyncServiceExecutor方法創(chuàng)建的。
測試方面這里我加入了一個(gè)定時(shí)任務(wù)，使用的是corn表達(dá)式。（不懂得同學(xué)可以網(wǎng)上了解一下）
@Slf4j
@Service
public class AsyncServiceImpl implements AsyncService {
@Override
@Scheduled(cron = " */1 * * * * ? ")
@Async("asyncServiceExecutor")
public void executeAsync() {
log.info("start executeAsync");
System.out.println("異步線程執(zhí)行批量插入等耗時(shí)任務(wù)");
log.info("end executeAsync");
}
}
第五步：測試結(jié)果如下
10:32:15.004 [async-service-1] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:15.004 [async-service-1] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:16.003 [async-service-2] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:16.004 [async-service-2] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:17.001 [async-service-3] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:17.001 [async-service-3] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:18.002 [async-service-4] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:18.003 [async-service-4] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:19.002 [async-service-5] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:19.003 [async-service-5] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:20.001 [async-service-6] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:20.002 [async-service-6] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:21.002 [async-service-7] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:21.002 [async-service-7] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:22.004 [async-service-8] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:22.005 [async-service-8] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:23.001 [async-service-1] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:23.003 [async-service-1] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:24.003 [async-service-2] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:24.003 [async-service-2] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:25.001 [async-service-3] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:25.001 [async-service-3] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:26.002 [async-service-4] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:26.002 [async-service-4] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:27.002 [async-service-5] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:27.003 [async-service-5] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:28.001 [async-service-6] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:28.001 [async-service-6] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:29.001 [async-service-7] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:29.002 [async-service-7] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:30.001 [async-service-8] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:30.001 [async-service-8] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:32:31.001 [async-service-1] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:32:31.001 [async-service-1] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
還沒完：
通過以上日志可以發(fā)現(xiàn)，[async-service-]是有多個(gè)線程的，顯然已經(jīng)在我們配置的線程池中執(zhí)行了，表明每次請求都快速響應(yīng)了，而耗時(shí)的操作都留給線程池中的線程去異步執(zhí)行；
還有另一個(gè)問提就是，雖然已經(jīng)用上了線程池，但是依然不清楚線程池當(dāng)時(shí)的情況，有多少線程在執(zhí)行，多少在隊(duì)列中等待呢？于是這里我創(chuàng)建了一個(gè)ThreadPoolTaskExecutor的子類，可以把每次提交線程的時(shí)候都會(huì)將當(dāng)前線程池的運(yùn)行狀況打印出來
import lombok.extern.slf4j.Slf4j;
@Slf4j
public class VisiableThreadPoolTaskExecutor extends ThreadPoolTaskExecutor {
/**
*
*/
private static final long serialVersionUID = -3518460523928455463L;
private void showThreadPoolInfo(String prefix) {
ThreadPoolExecutor threadPoolExecutor = getThreadPoolExecutor();
if (null == threadPoolExecutor) {
return;
}
log.info("{}, {},taskCount [{}], completedTaskCount [{}], activeCount [{}], queueSize [{}]",
this.getThreadNamePrefix(),
prefix,
threadPoolExecutor.getTaskCount(),
threadPoolExecutor.getCompletedTaskCount(),
threadPoolExecutor.getActiveCount(),
threadPoolExecutor.getQueue().size());
}
@Override
public void execute(Runnable task) {
showThreadPoolInfo("1. do execute");
super.execute(task);
}
@Override
public void execute(Runnable task, long startTimeout) {
showThreadPoolInfo("2. do execute");
super.execute(task, startTimeout);
}
@Override
public Future<?> submit(Runnable task) {
showThreadPoolInfo("1. do submit");
return super.submit(task);
}
@Override
public <T> Future<T> submit(Callable<T> task) {
showThreadPoolInfo("2. do submit");
return super.submit(task);
}
@Override
public ListenableFuture<?> submitListenable(Runnable task) {
showThreadPoolInfo("1. do submitListenable");
return super.submitListenable(task);
}
@Override
public <T> ListenableFuture<T> submitListenable(Callable<T> task) {
showThreadPoolInfo("2. do submitListenable");
return super.submitListenable(task);
}
}
其次：修改RCExecutorConfig.java的asyncServiceExecutor方法，
將ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor()改為ThreadPoolTaskExecutor executor = new VisiableThreadPoolTaskExecutor()
@Bean(name = "asyncServiceExecutor")
public Executor asyncServiceExecutor() {
log.info("start asyncServiceExecutor");
//ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
ThreadPoolTaskExecutor executor = new VisiableThreadPoolTaskExecutor();
//配置核心線程數(shù)
executor.setCorePoolSize(corePoolSize);
//配置最大線程數(shù)
executor.setMaxPoolSize(maxPoolSize);
//配置隊(duì)列大小
executor.setQueueCapacity(queueCapacity);
//配置線程池中的線程的名稱前綴
executor.setThreadNamePrefix(namePrefix);
// rejection-policy：當(dāng)pool已經(jīng)達(dá)到max size的時(shí)候，如何處理新任務(wù)
// CALLER_RUNS：不在新線程中執(zhí)行任務(wù)，而是有調(diào)用者所在的線程來執(zhí)行
executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
//執(zhí)行初始化
executor.initialize();
return executor;
}
測試結(jié)果如下：
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:41:35.003 [async-service-5] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:41:36.001 [sheduled-pool-1-thread-1] INFO c.a.a.e.t.VisiableThreadPoolTaskExecutor - async-service-, 2. do submit,taskCount [5], completedTaskCount [5], activeCount [0], queueSize [0]
10:41:36.001 [async-service-6] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:41:36.002 [async-service-6] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:41:37.001 [sheduled-pool-1-thread-7] INFO c.a.a.e.t.VisiableThreadPoolTaskExecutor - async-service-, 2. do submit,taskCount [6], completedTaskCount [6], activeCount [0], queueSize [0]
10:41:37.001 [async-service-7] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:41:37.002 [async-service-7] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:41:38.002 [sheduled-pool-1-thread-7] INFO c.a.a.e.t.VisiableThreadPoolTaskExecutor - async-service-, 2. do submit,taskCount [7], completedTaskCount [7], activeCount [0], queueSize [0]
10:41:38.002 [async-service-8] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:41:38.002 [async-service-8] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:41:39.001 [sheduled-pool-1-thread-7] INFO c.a.a.e.t.VisiableThreadPoolTaskExecutor - async-service-, 2. do submit,taskCount [8], completedTaskCount [8], activeCount [0], queueSize [0]
10:41:39.001 [async-service-1] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:41:39.002 [async-service-1] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:41:40.003 [sheduled-pool-1-thread-7] INFO c.a.a.e.t.VisiableThreadPoolTaskExecutor - async-service-, 2. do submit,taskCount [9], completedTaskCount [9], activeCount [0], queueSize [0]
10:41:40.003 [async-service-2] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:41:40.003 [async-service-2] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:41:41.001 [sheduled-pool-1-thread-7] INFO c.a.a.e.t.VisiableThreadPoolTaskExecutor - async-service-, 2. do submit,taskCount [10], completedTaskCount [10], activeCount [0], queueSize [0]
10:41:41.001 [async-service-3] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:41:41.001 [async-service-3] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:41:42.000 [sheduled-pool-1-thread-7] INFO c.a.a.e.t.VisiableThreadPoolTaskExecutor - async-service-, 2. do submit,taskCount [11], completedTaskCount [11], activeCount [0], queueSize [0]
10:41:42.000 [async-service-4] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:41:42.000 [async-service-4] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:41:43.001 [sheduled-pool-1-thread-7] INFO c.a.a.e.t.VisiableThreadPoolTaskExecutor - async-service-, 2. do submit,taskCount [12], completedTaskCount [12], activeCount [0], queueSize [0]
10:41:43.002 [async-service-5] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:41:43.003 [async-service-5] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:41:44.001 [sheduled-pool-1-thread-7] INFO c.a.a.e.t.VisiableThreadPoolTaskExecutor - async-service-, 2. do submit,taskCount [13], completedTaskCount [13], activeCount [0], queueSize [0]
10:41:44.001 [async-service-6] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
異步線程執(zhí)行批量插入等耗時(shí)任務(wù)
10:41:44.001 [async-service-6] INFO c.a.a.service.impl.AsyncServiceImpl - end executeAsync
10:41:45.000 [sheduled-pool-1-thread-7] INFO c.a.a.e.t.VisiableThreadPoolTaskExecutor - async-service-, 2. do submit,taskCount [14], completedTaskCount [14], activeCount [0], queueSize [0]
10:41:45.001 [async-service-7] INFO c.a.a.service.impl.AsyncServiceImpl - start executeAsync
解釋：這里意思提交了14個(gè)任務(wù)，處理了14個(gè)任務(wù)，對列中還剩0個(gè)任務(wù)
10:41:45.000 [sheduled-pool-1-thread-7] INFO c.a.a.e.t.VisiableThreadPoolTaskExecutor - async-service-, 2. do submit,taskCount [14], completedTaskCount [14], activeCount [0], queueSize [0]
到此為止就OK了！
以上為個(gè)人經(jīng)驗(yàn)，希望能給大家一個(gè)參考，也希望大家多多支持腳本之家。
相關(guān)文章
SpringBoot停止啟動(dòng)時(shí)測試檢查rabbitmq操作
這篇文章主要介紹了SpringBoot停止啟動(dòng)時(shí)測試檢查rabbitmq操作，具有很好的參考價(jià)值，希望對大家有所幫助。如有錯(cuò)誤或未考慮完全的地方，望不吝賜教2021-09-09
Java concurrency之鎖_動(dòng)力節(jié)點(diǎn)Java學(xué)院整理
這篇文章主要為大家詳細(xì)介紹了Java concurrency之鎖的相關(guān)資料，具有一定的參考價(jià)值，感興趣的小伙伴們可以參考一下2017-06-06
這篇文章主要為大家詳細(xì)介紹了Java實(shí)現(xiàn)Ip地址獲取的兩種方式,文中的示例代碼講解詳細(xì),具有一定的借鑒價(jià)值,有需要的小伙伴可以參考一下2023-09-09
Java實(shí)現(xiàn)斷點(diǎn)續(xù)傳功能的示例代碼
這篇文章主要為大家詳細(xì)介紹了如何利用Java語言實(shí)現(xiàn)網(wǎng)絡(luò)資源的斷點(diǎn)續(xù)傳功能，文中的示例代碼講解詳細(xì)，具有一定的學(xué)習(xí)價(jià)值，感興趣的可以了解一下2022-10-10
在現(xiàn)今互聯(lián)網(wǎng)時(shí)代，數(shù)據(jù)安全已經(jīng)成為了我們必須要面對的重要課題，在本文中，我們將會(huì)介紹Java中常用的幾種數(shù)據(jù)庫加密方式并分析一下它們的優(yōu)缺點(diǎn)，希望對大家有所幫助2023-05-05
Java 回調(diào)機(jī)制(CallBack) 詳解及實(shí)例代碼
這篇文章主要介紹了 Java 回調(diào)機(jī)制(CallBack) 詳解及實(shí)例代碼的相關(guān)資料,需要的朋友可以參考下2017-02-02
正則表達(dá)式中,\代表轉(zhuǎn)義字符,通常是轉(zhuǎn)義一些特殊字符,下面這篇文章主要給大家介紹了關(guān)于對Java正則表達(dá)式"\\"的相關(guān)理解,需要的朋友可以參考下2022-09-09
java實(shí)現(xiàn)發(fā)送短信驗(yàn)證碼
這篇文章主要為大家詳細(xì)介紹了java實(shí)現(xiàn)發(fā)送短信驗(yàn)證碼，具有一定的參考價(jià)值，感興趣的小伙伴們可以參考一下2019-07-07
在Struts2中如何將父類屬性序列化為JSON格式的解決方法
本篇文章，小編將為大家介紹關(guān)于在Struts2中如何將父類屬性序列化為JSON格式的解決方法，有需要的朋友可以參考一下2013-04-04
SpringBoot詳解如何整合Redis緩存驗(yàn)證碼
本文主要介紹了SpringBoot集成Redis實(shí)現(xiàn)驗(yàn)證碼的緩存簡單案例，文中通過示例代碼介紹的非常詳細(xì)，具有一定的參考價(jià)值，感興趣的小伙伴們可以參考一下2022-07-07