Springboot集成ClickHouse及應(yīng)用場(chǎng)景分析
ClickHouse應(yīng)用場(chǎng)景:
1.絕大多數(shù)請(qǐng)求都是用于讀訪問的
2.數(shù)據(jù)需要以大批次(大于1000行)進(jìn)行更新,而不是單行更新;或者根本沒有更新操作
3.數(shù)據(jù)只是添加到數(shù)據(jù)庫(kù),沒有必要修改
4.讀取數(shù)據(jù)時(shí),會(huì)從數(shù)據(jù)庫(kù)中提取出大量的行,但只用到一小部分列
5.表很“寬”,即表中包含大量的列
6.查詢頻率相對(duì)較低(通常每臺(tái)服務(wù)器每秒查詢數(shù)百次或更少)
7.對(duì)于簡(jiǎn)單查詢,允許大約50毫秒的延遲
8.列的值是比較小的數(shù)值和短字符串(例如,每個(gè)URL只有60個(gè)字節(jié))
9.在處理單個(gè)查詢時(shí)需要高吞吐量(每臺(tái)服務(wù)器每秒高達(dá)數(shù)十億行)
10.不需要事務(wù)
11.數(shù)據(jù)一致性要求較低
12.每次查詢中只會(huì)查詢一個(gè)大表。除了一個(gè)大表,其余都是小表
13.查詢結(jié)果顯著小于數(shù)據(jù)源。即數(shù)據(jù)有過濾或聚合。返回結(jié)果不超過單個(gè)服務(wù)器內(nèi)存大小
行式存儲(chǔ)對(duì)比列式存儲(chǔ):
(1)、行式數(shù)據(jù)
(2)、列式數(shù)據(jù)
(3)、對(duì)比分析
分析類查詢,通常只需要讀取表的一小部分列。在列式數(shù)據(jù)庫(kù)中可以只讀取需要的數(shù)據(jù)。數(shù)據(jù)總是打包成批量讀取的,所以壓縮是非常容易的。同時(shí)數(shù)據(jù)按列分別存儲(chǔ)這也更容易壓縮。這進(jìn)一步降低了I/O的體積。由于I/O的降低,這將幫助更多的數(shù)據(jù)被系統(tǒng)緩存。
整合Springboot:
核心依賴(mybatis plus做持久層,druid做數(shù)據(jù)源):
<dependencies> <!--clickhouse--> <dependency> <groupId>ru.yandex.clickhouse</groupId> <artifactId>clickhouse-jdbc</artifactId> <version>0.3.1-patch</version> </dependency> <!-- https://mvnrepository.com/artifact/com.alibaba/druid --> <dependency> <groupId>com.alibaba</groupId> <artifactId>druid</artifactId> <version>1.2.6</version> </dependency> <!-- https://mvnrepository.com/artifact/com.baomidou/mybatis-plus-boot-starter --> <dependency> <groupId>com.baomidou</groupId> <artifactId>mybatis-plus-boot-starter</artifactId> <version>3.4.3.1</version> </dependency> </dependencies>
配置yml文件:
spring: datasource: type: com.alibaba.druid.pool.DruidDataSource click: driverClassName: ru.yandex.clickhouse.ClickHouseDriver url: jdbc:clickhouse://127.0.0.1:8123/dbname username: username password: 123456 initialSize: 10 maxActive: 100 minIdle: 10 maxWait: 6000 mybatis-plus: mapper-locations: classpath*:mapper/*.xml configuration: log-impl: org.apache.ibatis.logging.stdout.StdOutImpl map-underscore-to-camel-case: true cache-enabled: true lazy-loading-enabled: true multiple-result-sets-enabled: true use-generated-keys: true default-statement-timeout: 60 default-fetch-size: 100 type-aliases-package: com.example.tonghp.entity
ClickHouse與Druid連接池配置類:
參數(shù)配置:
package com.example.tonghp.config; import lombok.Data; import org.springframework.boot.context.properties.ConfigurationProperties; import org.springframework.stereotype.Component; /** * @author: tonghp * @create: 2021/07/26 16:23 */ @Data @Component @ConfigurationProperties(prefix = "spring.datasource.click") public class JdbcParamConfig { private String driverClassName ; private String url ; private Integer initialSize ; private Integer maxActive ; private Integer minIdle ; private Integer maxWait ; private String username; private String password; // 省略 GET 和 SET }
Druid連接池配置
package com.example.tonghp.config; import com.alibaba.druid.pool.DruidDataSource; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import javax.annotation.Resource; import javax.sql.DataSource; import javax.swing.*; /** * @author: tonghp * @create: 2021/07/26 16:22 */ @Configuration public class DruidConfig { @Resource private JdbcParamConfig jdbcParamConfig ; @Bean public DataSource dataSource() { DruidDataSource datasource = new DruidDataSource(); datasource.setUrl(jdbcParamConfig.getUrl()); datasource.setDriverClassName(jdbcParamConfig.getDriverClassName()); datasource.setInitialSize(jdbcParamConfig.getInitialSize()); datasource.setMinIdle(jdbcParamConfig.getMinIdle()); datasource.setMaxActive(jdbcParamConfig.getMaxActive()); datasource.setMaxWait(jdbcParamConfig.getMaxWait()); datasource.setUsername(jdbcParamConfig.getUsername()); datasource.setPassword(jdbcParamConfig.getPassword()); return datasource; } }
接下來配置實(shí)體類,mapper,service,controlle以及mapper.xml。與mybatisplus操作mysql一樣的思路。
package com.example.tonghp.entity; import lombok.Data; import java.io.Serializable; /** * @author: tonghp * @create: 2021/07/26 16:31 */ @Data public class UserInfo implements Serializable { private static final long serialVersionUID = 1L; private int id; private String userName; private String passWord; private String phone; private String email; private String createDay; }
package com.example.tonghp.mapper; import com.baomidou.mybatisplus.core.mapper.BaseMapper; import com.example.tonghp.entity.UserInfo; import org.apache.ibatis.annotations.Mapper; import org.apache.ibatis.annotations.Param; import org.springframework.stereotype.Repository; import java.util.List; /** * @author: tonghp * @create: 2021/07/26 16:32 */ @Repository public interface UserInfoMapper extends BaseMapper<UserInfo> { // 寫入數(shù)據(jù) void saveData (UserInfo userInfo) ; // ID 查詢 UserInfo selectById (@Param("id") Integer id) ; // 查詢?nèi)? List<UserInfo> selectList () ; }
UserInfoMapper.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd"> <mapper namespace="com.example.tonghp.mapper.UserInfoMapper"> <resultMap id="BaseResultMap" type="com.example.tonghp.entity.UserInfo"> <id column="id" jdbcType="INTEGER" property="id" /> <result column="user_name" jdbcType="VARCHAR" property="userName" /> <result column="pass_word" jdbcType="VARCHAR" property="passWord" /> <result column="phone" jdbcType="VARCHAR" property="phone" /> <result column="email" jdbcType="VARCHAR" property="email" /> <result column="create_day" jdbcType="VARCHAR" property="createDay" /> </resultMap> <sql id="Base_Column_List"> id,user_name,pass_word,phone,email,create_day </sql> <insert id="saveData" parameterType="com.example.tonghp.entity.UserInfo" > INSERT INTO cs_user_info (id,user_name,pass_word,phone,email,create_day) VALUES (#{id,jdbcType=INTEGER},#{userName,jdbcType=VARCHAR},#{passWord,jdbcType=VARCHAR}, #{phone,jdbcType=VARCHAR},#{email,jdbcType=VARCHAR},#{createDay,jdbcType=VARCHAR}) </insert> <select id="selectById" parameterType="java.lang.Integer" resultMap="BaseResultMap"> select <include refid="Base_Column_List" /> from cs_user_info where id = #{id,jdbcType=INTEGER} </select> <select id="selectList" resultMap="BaseResultMap" > select <include refid="Base_Column_List" /> from cs_user_info </select> </mapper>
Service
package com.example.tonghp.service; import com.baomidou.mybatisplus.extension.service.IService; import com.example.tonghp.entity.UserInfo; import org.apache.ibatis.annotations.Param; import org.springframework.stereotype.Service; import java.util.List; /** * @author: tonghp * @create: 2021/07/26 16:46 */ public interface UserInfoService extends IService<UserInfo> { // 寫入數(shù)據(jù) void saveData (UserInfo userInfo) ; // ID 查詢 UserInfo selectById (@Param("id") Integer id) ; // 查詢?nèi)? List<UserInfo> selectList () ; }
ServiceImpl
package com.example.tonghp.service.impl; import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl; import com.example.tonghp.entity.UserInfo; import com.example.tonghp.mapper.UserInfoMapper; import com.example.tonghp.service.UserInfoService; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; import java.util.List; /** * @author: tonghp * @create: 2021/07/26 16:48 */ @Service public class UserInfoServiceImpl extends ServiceImpl<UserInfoMapper, UserInfo> implements UserInfoService { @Autowired UserInfoMapper userInfoMapper; @Override public void saveData(UserInfo userInfo) { userInfoMapper.saveData(userInfo); } @Override public UserInfo selectById(Integer id) { return userInfoMapper.selectById(id); } @Override public List<UserInfo> selectList() { return userInfoMapper.selectList(); } }
Controller
package com.example.tonghp.controller; import com.example.tonghp.entity.UserInfo; import com.example.tonghp.service.UserInfoService; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; import javax.annotation.Resource; import java.util.List; /** * @author: tonghp * @create: 2021/07/26 16:45 */ @RestController @RequestMapping("user") public class UserInfoController { @Autowired private UserInfoService userInfoService ; @RequestMapping("saveData") public String saveData (){ UserInfo userInfo = new UserInfo () ; userInfo.setId(4); userInfo.setUserName("winter"); userInfo.setPassWord("567"); userInfo.setPhone("13977776789"); userInfo.setEmail("winter"); userInfo.setCreateDay("2020-02-20"); userInfoService.saveData(userInfo); return "sus"; } @RequestMapping("selectById") public UserInfo selectById () { return userInfoService.selectById(1) ; } @RequestMapping("selectList") public List<UserInfo> selectList () { return userInfoService.selectList() ; } }
main()
package com.example.tonghp; import org.mybatis.spring.annotation.MapperScan; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; @SpringBootApplication @MapperScan("com.example.tonghp.mapper") public class TonghpApplication { public static void main(String[] args) { SpringApplication.run(TonghpApplication.class, args); } }
參考鏈接:
https://blog.csdn.net/weixin_46792649/article/details/115306384
https://www.cnblogs.com/ywjfx/p/14333974.html
https://www.cnblogs.com/cicada-smile/p/11632251.html
https://github.com/cicadasmile/middle-ware-parent
另外還有一種直接操作JDBC的:
https://blog.csdn.net/Alice_qixin/article/details/84957380
到此這篇關(guān)于Springboot集成ClickHouse的文章就介紹到這了,更多相關(guān)Springboot集成ClickHouse內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家!
相關(guān)文章
java評(píng)論、回復(fù)功能設(shè)計(jì)與實(shí)現(xiàn)方法
很多項(xiàng)目或者系統(tǒng)都有評(píng)論或者回復(fù)的需求,但評(píng)論回復(fù)的實(shí)現(xiàn)往往都比較復(fù)雜,也不好實(shí)現(xiàn),下面這篇文章主要給大家介紹了關(guān)于java評(píng)論、回復(fù)功能設(shè)計(jì)與實(shí)現(xiàn)的相關(guān)資料,需要的朋友可以參考下2022-06-06Spring?Boot?教程之創(chuàng)建項(xiàng)目的三種方式
這篇文章主要分享了Spring?Boot?教程之創(chuàng)建項(xiàng)目的三種方式,文章圍繞主題展開詳細(xì)的內(nèi)容介紹,具有一定的參考價(jià)值,需要的小伙伴可以參考一下2022-05-05Java并發(fā)編程Semaphore計(jì)數(shù)信號(hào)量詳解
這篇文章主要介紹了Java并發(fā)編程Semaphore計(jì)數(shù)信號(hào)量詳解,具有一定參考價(jià)值,需要的朋友可以了解下。2017-10-10