Apache ShardingSphere簡介及使用詳解
一 什么是 ShardingSphere?
1.1 背景:為什么需要分庫分表?
隨著系統(tǒng)數(shù)據(jù)量和用戶量不斷增長,傳統(tǒng)單體數(shù)據(jù)庫容易遇到瓶頸:
寫入/查詢壓力大:單表千萬級數(shù)據(jù),索引效率下降。
存儲超限:單機磁盤空間和 IOPS 不堪重負。
水平擴展困難:業(yè)務難以橫向拓展。
為了解決這些問題,開發(fā)者往往采用 分庫分表 技術,將一張大表分成多個小表,分布在不同數(shù)據(jù)庫中,實現(xiàn)水平擴展和負載均衡。
1.2 什么是 Apache ShardingSphere?
Apache ShardingSphere 是一款開源的 分布式數(shù)據(jù)庫中間件框架,可以幫助開發(fā)者輕松實現(xiàn):
- 分庫分表
- 讀寫分離
- 分布式事務
- 數(shù)據(jù)脫敏
- 可觀測性 & 擴展治理能力
ShardingSphere 核心組件包括:
組件 | 說明 |
---|---|
ShardingSphere-JDBC | JDBC 封裝層,嵌入應用中,適用于微服務架構 |
ShardingSphere-Proxy | 獨立部署的數(shù)據(jù)庫代理,支持多語言訪問 |
ShardingSphere-Sidecar(規(guī)劃中) | 面向云原生場景,Service Mesh 結構 |
1.3 ShardingSphere-JDBC 的核心功能
功能 | 簡要說明 |
---|---|
分庫分表 | 按字段配置路由策略,自動將 SQL 分發(fā)到對應的數(shù)據(jù)庫/表 |
讀寫分離 | 自動區(qū)分讀寫請求,路由到主/從數(shù)據(jù)庫 |
靈活分片策略 | 支持取模、范圍、Hint、自定義算法等多種策略 |
事務支持 | 支持本地事務和分布式事務(Seata/XA) |
透明化使用 | 對開發(fā)者來說使用方式與普通 JDBC 幾乎一致,只需配置邏輯表名 |
二、實踐部分:Java + H2 實現(xiàn)分庫分表
示例:使用 Java 原生程序 + ShardingSphere-JDBC + H2 內(nèi)存數(shù)據(jù)庫,模擬一個電商訂單系統(tǒng)按 user_id
分庫,order_id
分表。
2.1 項目結構
- 使用 H2 數(shù)據(jù)庫模擬兩個庫
ds0
,ds1
- 每個庫中創(chuàng)建兩個分表
t_order_0
,t_order_1
- 邏輯表名:
t_order
2.2 Maven 依賴配置
<dependencies> <dependency> <groupId>org.apache.shardingsphere</groupId> <artifactId>shardingsphere-jdbc-core</artifactId> <version>5.4.1</version> </dependency> <dependency> <groupId>com.h2database</groupId> <artifactId>h2</artifactId> <version>2.2.220</version> </dependency> <dependency> <groupId>com.zaxxer</groupId> <artifactId>HikariCP</artifactId> <version>5.0.1</version> </dependency> </dependencies>
2.3 核心代碼結構
1. 創(chuàng)建真實表(物理分表)
public static void createTables() throws SQLException { for (int i = 0; i < 2; i++) { String db = "ds" + i; try (Connection conn = DriverManager.getConnection("jdbc:h2:mem:" + db + ";DB_CLOSE_DELAY=-1;MODE=MySQL", "sa", "")) { Statement stmt = conn.createStatement(); for (int j = 0; j < 2; j++) { stmt.execute(String.format(""" CREATE TABLE IF NOT EXISTS t_order_%d ( order_id BIGINT PRIMARY KEY, user_id INT, status VARCHAR(50) ) """, j)); } } } }
2. 配置分庫分表數(shù)據(jù)源
public static DataSource createDataSource() throws SQLException { Map<String, DataSource> dataSourceMap = new HashMap<>(); for (int i = 0; i < 2; i++) { HikariDataSource ds = new HikariDataSource(); ds.setJdbcUrl("jdbc:h2:mem:ds" + i + ";DB_CLOSE_DELAY=-1;MODE=MySQL"); ds.setUsername("sa"); ds.setPassword(""); dataSourceMap.put("ds" + i, ds); } ShardingTableRuleConfiguration orderTableRule = new ShardingTableRuleConfiguration( "t_order", "ds${0..1}.t_order_${0..1}"); orderTableRule.setDatabaseShardingStrategy( new StandardShardingStrategyConfiguration("user_id", "dbSharding")); orderTableRule.setTableShardingStrategy( new StandardShardingStrategyConfiguration("order_id", "tableSharding")); ShardingRuleConfiguration config = new ShardingRuleConfiguration(); config.getTables().add(orderTableRule); config.getShardingAlgorithms().put("dbSharding", new AlgorithmConfiguration("INLINE", props("algorithm-expression", "ds${user_id % 2}"))); config.getShardingAlgorithms().put("tableSharding", new AlgorithmConfiguration("INLINE", props("algorithm-expression", "t_order_${order_id % 2}"))); return ShardingSphereDataSourceFactory.createDataSource(dataSourceMap, List.of(config), new Properties()); }
3. 插入與查詢數(shù)據(jù)
public static void insertOrder(DataSource ds, long orderId, int userId, String status) throws SQLException { try (Connection conn = ds.getConnection()) { PreparedStatement ps = conn.prepareStatement("INSERT INTO t_order (order_id, user_id, status) VALUES (?, ?, ?)"); ps.setLong(1, orderId); ps.setInt(2, userId); ps.setString(3, status); ps.executeUpdate(); } } public static void queryOrders(DataSource ds) throws SQLException { try (Connection conn = ds.getConnection()) { ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM t_order"); while (rs.next()) { System.out.printf("order_id=%d, user_id=%d, status=%s%n", rs.getLong("order_id"), rs.getInt("user_id"), rs.getString("status")); } } }
2.4 程序入口
public static void main(String[] args) throws Exception { createTables(); // 創(chuàng)建實際分表 DataSource shardingDataSource = createDataSource(); insertOrder(shardingDataSource, 1001, 1, "INIT"); insertOrder(shardingDataSource, 1002, 2, "PAID"); insertOrder(shardingDataSource, 1003, 3, "SHIPPED"); queryOrders(shardingDataSource); }
輸出示例
order_id=1001, user_id=1, status=INIT
order_id=1002, user_id=2, status=PAID
order_id=1003, user_id=3, status=SHIPPED
ShardingSphere 已自動根據(jù)你配置的策略將數(shù)據(jù)路由到對應的庫和表!
總結
Apache ShardingSphere 提供了強大、靈活的分庫分表能力,通過合理配置可以極大提升系統(tǒng)的性能與擴展性:
- 分片策略靈活,支持多種規(guī)則或自定義算法
- 配置簡單,無需改動業(yè)務 SQL
- 兼容性強,支持 JDBC、Spring Boot、YAML 等多種使用方式
無論你是中小項目的快速原型,還是大規(guī)模高并發(fā)系統(tǒng),ShardingSphere 都是一個值得一試的解決方案。
完整代碼
package org.example; import com.zaxxer.hikari.HikariDataSource; import org.apache.shardingsphere.driver.api.ShardingSphereDataSourceFactory; import org.apache.shardingsphere.infra.config.algorithm.AlgorithmConfiguration; import org.apache.shardingsphere.sharding.api.config.ShardingRuleConfiguration; import org.apache.shardingsphere.sharding.api.config.rule.ShardingTableRuleConfiguration; import org.apache.shardingsphere.sharding.api.config.strategy.sharding.StandardShardingStrategyConfiguration; import javax.sql.DataSource; import java.sql.*; import java.util.*; public class ShardingH2Example { public static void main(String[] args) throws Exception { createTables(); // 注意:用 DriverManager 直連底層 db 建表 DataSource shardingDataSource = createDataSource(); // 插入測試數(shù)據(jù) insertOrder(shardingDataSource, 1001, 1, "INIT"); insertOrder(shardingDataSource, 1002, 2, "PAID"); insertOrder(shardingDataSource, 1003, 3, "SHIPPED"); // 查詢測試數(shù)據(jù) queryOrders(shardingDataSource); } public static DataSource createDataSource() throws Exception { Map<String, DataSource> dataSourceMap = new HashMap<>(); // 模擬兩個庫:ds0 和 ds1(分別用兩個內(nèi)存 H2 實例模擬) for (int i = 0; i < 2; i++) { HikariDataSource ds = new HikariDataSource(); ds.setJdbcUrl("jdbc:h2:mem:ds" + i + ";DB_CLOSE_DELAY=-1;MODE=MySQL"); ds.setUsername("sa"); ds.setPassword(""); dataSourceMap.put("ds" + i, ds); } // 分表規(guī)則配置 ShardingTableRuleConfiguration orderTableRule = new ShardingTableRuleConfiguration( "t_order", "ds${0..1}.t_order_${0..1}"); // 分庫策略(user_id) orderTableRule.setDatabaseShardingStrategy(new StandardShardingStrategyConfiguration( "user_id", "dbSharding")); // 分表策略(order_id) orderTableRule.setTableShardingStrategy(new StandardShardingStrategyConfiguration( "order_id", "tableSharding")); ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration(); shardingRuleConfig.getTables().add(orderTableRule); Properties dbProps = new Properties(); dbProps.setProperty("algorithm-expression", "ds${user_id % 2}"); shardingRuleConfig.getShardingAlgorithms().put("dbSharding", new AlgorithmConfiguration("INLINE", dbProps)); Properties tableProps = new Properties(); tableProps.setProperty("algorithm-expression", "t_order_${order_id % 2}"); shardingRuleConfig.getShardingAlgorithms().put("tableSharding", new AlgorithmConfiguration("INLINE", tableProps)); return ShardingSphereDataSourceFactory.createDataSource(dataSourceMap, List.of(shardingRuleConfig), new Properties()); } public static void createTables() throws SQLException { for (int i = 0; i < 2; i++) { String dbName = "ds" + i; try (Connection conn = DriverManager.getConnection("jdbc:h2:mem:" + dbName + ";DB_CLOSE_DELAY=-1;MODE=MySQL", "sa", "")) { Statement stmt = conn.createStatement(); for (int j = 0; j < 2; j++) { String sql = String.format(""" CREATE TABLE IF NOT EXISTS t_order_%d ( order_id BIGINT PRIMARY KEY, user_id INT, status VARCHAR(50) ) """, j); stmt.execute(sql); } } } } // 輔助方法用于手動連接底層 H2 數(shù)據(jù)源 private static HikariDataSource getH2DataSource(String name) { HikariDataSource ds = new HikariDataSource(); ds.setJdbcUrl("jdbc:h2:mem:" + name + ";DB_CLOSE_DELAY=-1;MODE=MySQL"); ds.setUsername("sa"); ds.setPassword(""); return ds; } public static void insertOrder(DataSource ds, long orderId, int userId, String status) throws SQLException { try (Connection conn = ds.getConnection()) { PreparedStatement ps = conn.prepareStatement("INSERT INTO t_order (order_id, user_id, status) VALUES (?, ?, ?)"); ps.setLong(1, orderId); ps.setInt(2, userId); ps.setString(3, status); ps.executeUpdate(); } } public static void queryOrders(DataSource ds) throws SQLException { try (Connection conn = ds.getConnection()) { ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM t_order"); System.out.println("Query Results:"); while (rs.next()) { System.out.printf("order_id: %d, user_id: %d, status: %s%n", rs.getLong("order_id"), rs.getInt("user_id"), rs.getString("status")); } } } }
到此這篇關于Apache ShardingSphere 初識使用的文章就介紹到這了,更多相關Apache ShardingSphere使用內(nèi)容請搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關文章希望大家以后多多支持腳本之家!
- SpringBoot?整合?ShardingSphere4.1.1實現(xiàn)分庫分表功能
- springboot如何使用yml文件方式配置shardingsphere
- ShardingSphere如何進行sql重寫示例詳解
- SpringBoot3和ShardingSphere5框架實現(xiàn)數(shù)據(jù)分庫分表
- springboot整合shardingsphere和seata實現(xiàn)分布式事務的實踐
- Java基于ShardingSphere實現(xiàn)分庫分表的實例詳解
- ShardingSphere jdbc集成多數(shù)據(jù)源的實現(xiàn)步驟
- SpringBoot整合ShardingSphere的示例代碼
- Java中ShardingSphere分庫分表實戰(zhàn)
相關文章
Linux 7.4上安裝配置Oracle 11.2.0.4圖文教程
本文通過圖文并茂的形式給大家介紹了Linux 7.4上安裝配置Oracle 11.2.0.4的方法,非常不錯,具有參考借鑒價值,需要的朋友參考下吧2017-12-12Linux下雙網(wǎng)卡Firewalld的配置流程(推薦)
firewalld提供了一個 動態(tài)管理的防火墻,用以支持不同網(wǎng)絡區(qū)域的規(guī)則,分配對一個網(wǎng)絡及其相關鏈接和界面一定程度的信任。這篇文章給大家介紹了Linux下雙網(wǎng)卡Firewalld的配置流程,需要的朋友參考下吧2018-04-04