.Net極限生產(chǎn)力之分表分庫(kù)全自動(dòng)化Migrations?Code-First
更新時(shí)間：2022年07月19日 14:29:16 作者：薛家明
這篇文章主要介紹了.Net極限生產(chǎn)力之分表分庫(kù)全自動(dòng)化Migrations?Code-First，輕量級(jí)針對(duì)分表分庫(kù)讀寫(xiě)分離的解決方案，具有零依賴、零學(xué)習(xí)成本、零業(yè)務(wù)代碼入侵適配
開(kāi)始
本次我們的主題就是極限生產(chǎn)力,其他語(yǔ)言望塵莫及的分表分庫(kù)全自動(dòng)化Migrations Code-First 加 efcore 分表分庫(kù)無(wú)感開(kāi)發(fā)
,經(jīng)過(guò)這么多框架的兼容我自己也認(rèn)識(shí)到了一些問(wèn)題,譬如在ShardingCore初始化前使用(畢竟efcore)的初始化是在依賴注入的時(shí)候不需要手動(dòng)調(diào)用初始化，比如efcore.tool的遷移的問(wèn)題,本項(xiàng)目不能遷移,因?yàn)?code>efcore.tool在使用命令的時(shí)候不會(huì)調(diào)用Configure導(dǎo)致無(wú)法初始化的bug,導(dǎo)致遷移必須要通過(guò)新建控制臺(tái)程序,而不能在本項(xiàng)目?jī)?nèi)遷移,再或者code-first和ShardingCore的啟動(dòng)參數(shù)沖突導(dǎo)致需要平凡修改，并且不支持分庫(kù),之前有小伙伴分了300個(gè)庫(kù)如果自動(dòng)遷移不能用確實(shí)是一件很頭疼的事情,雖然這些問(wèn)題對(duì)于分庫(kù)分表而言其實(shí)是小事情，但是如果一旦分表分庫(kù)到達(dá)一定的量級(jí)就會(huì)難以維護(hù)。所以ShardingCore在最近三周內(nèi)開(kāi)啟了新的版本,新版本主要是解決上述痛點(diǎn)并且將代碼更加標(biāo)準(zhǔn)的使用
開(kāi)發(fā)軟件一般是先能用，然后好用，最后標(biāo)準(zhǔn)化，ShardingCore也是如此，因?yàn)樾枰獢U(kuò)展efcore所以有時(shí)候在不熟悉efcore的擴(kuò)展方式的時(shí)候只能靠靜態(tài)類來(lái)進(jìn)行注入訪問(wèn),而靜態(tài)類其實(shí)是一個(gè)非常不標(biāo)準(zhǔn)的用法,除非萬(wàn)不得已。那么新版本x.6.x.x ShardingCore帶來(lái)了什么請(qǐng)往下看
移除靜態(tài)容器
靜態(tài)容器的使用導(dǎo)致ShardingCore在整個(gè)應(yīng)用程序聲明周期只有一份數(shù)據(jù),那么數(shù)據(jù)都是共享的這個(gè)對(duì)于后續(xù)的測(cè)試維護(hù)擴(kuò)展是相當(dāng)?shù)牟焕模瑳](méi)有單例那種隔離性來(lái)的好,所以移除了ShardingContainer,通過(guò)提供IShardingRuntimeContext來(lái)保證和之前的參數(shù)結(jié)構(gòu)的訪問(wèn),同一個(gè)DbContext類型在使用不同的IShardingRuntimeContext后可以表現(xiàn)出不同的分表分庫(kù)特性。
原生efcore
首先我們針對(duì)原生efcore進(jìn)行擴(kuò)展來(lái)達(dá)到分庫(kù)分表+code-first自動(dòng)遷移開(kāi)發(fā)
添加依賴 ShardingCore 6.6.0.3 MySql
//請(qǐng)安裝最新版本目前x.6.0.3+,第一個(gè)版本號(hào)6代表efcore的版本號(hào) Install-Package ShardingCore -Version 6.6.0.3 Install-Package Pomelo.EntityFrameworkCore.MySql -Version 6.0.1 Install-Package Microsoft.EntityFrameworkCore.Tools -Version 6.0.6
創(chuàng)建一個(gè)todo實(shí)體
public class TodoItem
{
public string Id { get; set; }
public string Text { get; set; }
}創(chuàng)建dbcontext
簡(jiǎn)單的將對(duì)象和數(shù)據(jù)庫(kù)做了一下映射當(dāng)然DbSet+Attribute也是可以的
public class MyDbContext:AbstractShardingDbContext,IShardingTableDbContext
{
public MyDbContext(DbContextOptions<MyDbContext> options) : base(options)
{
}
public IRouteTail RouteTail { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Entity<TodoItem>(mb =>
{
mb.HasKey(o => o.Id);
mb.Property(o => o.Id).IsRequired().HasMaxLength(50).HasComment("id");
mb.Property(o => o.Text).IsRequired().HasMaxLength(256).HasComment("事情");
mb.ToTable(nameof(TodoItem));
});
}
}新建分庫(kù)分表路由
分庫(kù)路由
public class TodoItemDataSourceRoute:AbstractShardingOperatorVirtualDataSourceRoute<TodoItem,string>
{
/// <summary>
/// id的hashcode取模余3分庫(kù)
/// </summary>
/// <param name="shardingKey"></param>
/// <returns></returns>
/// <exception cref="InvalidOperationException"></exception>
public override string ShardingKeyToDataSourceName(object shardingKey)
{
if (shardingKey == null) throw new InvalidOperationException("sharding key cant null");
var stringHashCode = ShardingCoreHelper.GetStringHashCode(shardingKey.ToString());
return $"ds{(Math.Abs(stringHashCode) % 3)}";//ds0,ds1,ds2
}
private readonly List<string> _dataSources = new List<string>() { "ds0", "ds1", "ds2" };
public override List<string> GetAllDataSourceNames()
{
return _dataSources;
}
public override bool AddDataSourceName(string dataSourceName)
{
throw new NotImplementedException();
}
/// <summary>
/// id分庫(kù)
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataDataSourceBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Id);
}
public override Func<string, bool> GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator)
{
var t = ShardingKeyToDataSourceName(shardingKey);
switch (shardingOperator)
{
case ShardingOperatorEnum.Equal: return tail => tail == t;
default:
{
return tail => true;
}
}
}
}分表路由：
public class TodoItemTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<TodoItem>
{
public TodoItemTableRoute() : base(2, 3)
{
}
/// <summary>
/// 正常情況下不會(huì)用內(nèi)容來(lái)做分片鍵因?yàn)樽鳛榉制I有個(gè)前提就是不會(huì)被修改
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataTableBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Text);
}
}新建遷移數(shù)據(jù)庫(kù)腳本生成
public class ShardingMySqlMigrationsSqlGenerator:MySqlMigrationsSqlGenerator
{
private readonly IShardingRuntimeContext _shardingRuntimeContext;
public ShardingMySqlMigrationsSqlGenerator(MigrationsSqlGeneratorDependencies dependencies, IRelationalAnnotationProvider annotationProvider, IMySqlOptions options,IShardingRuntimeContext shardingRuntimeContext) : base(dependencies, annotationProvider, options)
{
_shardingRuntimeContext = shardingRuntimeContext;
}
protected override void Generate(
MigrationOperation operation,
IModel model,
MigrationCommandListBuilder builder)
{
var oldCmds = builder.GetCommandList().ToList();
base.Generate(operation, model, builder);
var newCmds = builder.GetCommandList().ToList();
var addCmds = newCmds.Where(x => !oldCmds.Contains(x)).ToList();
MigrationHelper.Generate(_shardingRuntimeContext,operation, builder, Dependencies.SqlGenerationHelper, addCmds);
}
}配置依賴注入
ILoggerFactory efLogger = LoggerFactory.Create(builder =>
{
builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole();
});
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddControllers();
builder.Services.AddShardingDbContext<MyDbContext>()
.UseRouteConfig(op =>
{
op.AddShardingTableRoute<TodoItemTableRoute>();
op.AddShardingDataSourceRoute<TodoItemDataSourceRoute>();
})
.UseConfig((sp,op) =>
{
op.UseShardingQuery((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.UseShardingTransaction((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.AddDefaultDataSource("ds0", "server=127.0.0.1;port=3306;database=mydb0;userid=root;password=root;");
op.AddExtraDataSource(sp=>new Dictionary<string, string>()
{
{"ds1", "server=127.0.0.1;port=3306;database=mydb1;userid=root;password=root;"},
{"ds2", "server=127.0.0.1;port=3306;database=mydb2;userid=root;password=root;"}
});
op.UseShardingMigrationConfigure(b =>
{
b.ReplaceService<IMigrationsSqlGenerator, ShardingMySqlMigrationsSqlGenerator>();
});
}).AddShardingCore();
var app = builder.Build();
// Configure the HTTP request pipeline.
//如果有按時(shí)間分片的需要加定時(shí)任務(wù)否則可以不加
app.Services.UseAutoShardingCreate();
using (var scope = app.Services.CreateScope())
{
var defaultShardingDbContext = scope.ServiceProvider.GetRequiredService<MyDbContext>();
if (defaultShardingDbContext.Database.GetPendingMigrations().Any())
{
defaultShardingDbContext.Database.Migrate();
}
}
//如果需要在啟動(dòng)后掃描是否有表卻掃了可以添加這個(gè)
//app.Services.UseAutoTryCompensateTable();
//......
app.Run();添加遷移文件
Add-Migration Init
啟動(dòng)程序
分表分庫(kù)自動(dòng)遷移
crud
添加todo字段并遷移
接下來(lái)我們將針對(duì)TodoItem添加一個(gè)name字段并且新增一張既不分庫(kù)也不分表的表然后進(jìn)行遷移
public class TodoItem
{
public string Id { get; set; }
public string Text { get; set; }
public string Name { get; set; }
}
public class TodoTest
{
public string Id { get; set; }
public string Test { get; set; }
}
//docontext
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Entity<TodoItem>(mb =>
{
mb.HasKey(o => o.Id);
mb.Property(o => o.Id).IsRequired().HasMaxLength(50).HasComment("id");
mb.Property(o => o.Text).IsRequired().HasMaxLength(256).HasComment("事情");
mb.Property(o => o.Name).HasMaxLength(256).HasComment("姓名");
mb.ToTable(nameof(TodoItem));
});
modelBuilder.Entity<TodoTest>(mb =>
{
mb.HasKey(o => o.Id);
mb.Property(o => o.Id).IsRequired().HasMaxLength(50).HasComment("id");
mb.Property(o => o.Test).IsRequired().HasMaxLength(256).HasComment("測(cè)試");
mb.ToTable(nameof(TodoTest));
});
}
不出意外我們成功了然后再次啟動(dòng)
啟動(dòng)程序后我們驚奇的發(fā)現(xiàn)不單原先的表新增了一個(gè)name字段，并且為分片未分開(kāi)的表也被添加進(jìn)來(lái)了
到此為止efcore的原生分庫(kù)分表+全自動(dòng)化遷移Code-First已經(jīng)全部完成,這不僅大大的提高了程序的性能并且大大的方便了開(kāi)發(fā)人員的維護(hù)。
集成AbpVNext
完成了efcore原生的分表分庫(kù)遷移我們將進(jìn)行abp下的操作
首先我們?nèi)ithub下的abp-samples里面下載對(duì)應(yīng)的demo測(cè)試，這邊選擇todo-mvc
接著我們本地打開(kāi)安裝依賴,只需要安裝·ShardingCore· 6.6.0.3。
新建兩個(gè)接口用于賦值創(chuàng)建時(shí)間和guid
因?yàn)镾hardingCore需要add,update,remove的時(shí)候shardingkey不可以為空,你可以自己賦值，但是這樣abp的部分特性就不能用了,所以我們做一下兼容
//在TodoApp.Domain.Shared新增兩個(gè)接口（非必須）
public interface IShardingKeyIsCreationTime
{
}
public interface IShardingKeyIsGuId
{
}
public class TodoItem : BasicAggregateRoot<Guid>,IShardingKeyIsGuId//,IShardingKeyIsCreationTime
{
public string Text { get; set; }
}
//不做時(shí)間分片所以不需要提前賦值
public class TodoItem : BasicAggregateRoot<Guid>,IShardingKeyIsGuId//,IShardingKeyIsCreationTime
{
public string Text { get; set; }
}AbpDbContext抽象類
因?yàn)锳bp需要繼承AbpDbContext所以這邊進(jìn)行一個(gè)修改因?yàn)镾hardingCore只需要接口所以可以滿足任何情況
//為了篇幅移除了大部分代碼剩下的可以在文末demo處查看
public abstract class AbstractShardingAbpDbContext<TDbContext> : AbpDbContext<TDbContext>, IShardingDbContext, ISupportShardingReadWrite
where TDbContext : DbContext
{
private readonly IShardingDbContextExecutor _shardingDbContextExecutor;
protected AbstractShardingAbpDbContext(DbContextOptions<TDbContext> options) : base(options)
{
var wrapOptionsExtension = options.FindExtension<ShardingWrapOptionsExtension>();
if (wrapOptionsExtension != null)
{
_shardingDbContextExecutor = new ShardingDbContextExecutor(this);
}
}
public DbContext GetDbContext(string dataSourceName, CreateDbContextStrategyEnum strategy, IRouteTail routeTail)
{
var dbContext = _shardingDbContextExecutor.CreateDbContext(strategy, dataSourceName, routeTail);
if (dbContext is AbpDbContext<TDbContext> abpDbContext && abpDbContext.LazyServiceProvider == null)
{
abpDbContext.LazyServiceProvider = this.LazyServiceProvider;
}
return dbContext;
}
}新增分庫(kù)分表路由
todoitem id取模分庫(kù)
public class TodoDataSourceRoute:AbstractShardingOperatorVirtualDataSourceRoute<TodoItem,string>
{
public override string ShardingKeyToDataSourceName(object shardingKey)
{
if (shardingKey == null) throw new InvalidOperationException("sharding key cant null");
var stringHashCode = ShardingCoreHelper.GetStringHashCode(shardingKey.ToString());
return $"ds{(Math.Abs(stringHashCode) % 3)}";//ds0,ds1,ds2
}
public override List<string> GetAllDataSourceNames()
{
return new List<string>()
{
"ds0", "ds1", "ds2"
};
}
public override bool AddDataSourceName(string dataSourceName)
{
throw new NotImplementedException();
}
public override void Configure(EntityMetadataDataSourceBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Id);
}
public override Func<string, bool> GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator)
{
var t = ShardingKeyToDataSourceName(shardingKey);
switch (shardingOperator)
{
case ShardingOperatorEnum.Equal: return tail => tail == t;
default:
{
return tail => true;
}
}
}
}todoitem text 取模分表：
public class TodoTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<TodoItem>
{
public TodoTableRoute() : base(2, 5)
{
}
public override void Configure(EntityMetadataTableBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Text);
}
}編寫(xiě)sqlserver分片遷移腳本生成
public class ShardingSqlServerMigrationsSqlGenerator: SqlServerMigrationsSqlGenerator
{
private readonly IShardingRuntimeContext _shardingRuntimeContext;
public ShardingSqlServerMigrationsSqlGenerator(IShardingRuntimeContext shardingRuntimeContext,[NotNull] MigrationsSqlGeneratorDependencies dependencies, [NotNull] IRelationalAnnotationProvider migrationsAnnotations) : base(dependencies, migrationsAnnotations)
{
_shardingRuntimeContext = shardingRuntimeContext;
}
protected override void Generate(
MigrationOperation operation,
IModel model,
MigrationCommandListBuilder builder)
{
var oldCmds = builder.GetCommandList().ToList();
base.Generate(operation, model, builder);
var newCmds = builder.GetCommandList().ToList();
var addCmds = newCmds.Where(x => !oldCmds.Contains(x)).ToList();
MigrationHelper.Generate(_shardingRuntimeContext,operation, builder, Dependencies.SqlGenerationHelper, addCmds);
}
}abp的efcore模塊注入
TodoAppEntityFrameworkCoreModule編寫(xiě)注入
public class TodoAppEntityFrameworkCoreModule : AbpModule
{
public static readonly ILoggerFactory efLogger = LoggerFactory.Create(builder =>
{
builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole();
});
public override void PreConfigureServices(ServiceConfigurationContext context)
{
TodoAppEfCoreEntityExtensionMappings.Configure();
}
public override void ConfigureServices(ServiceConfigurationContext context)
{
context.Services.AddAbpDbContext<TodoAppDbContext>(options =>
{
/* Remove "includeAllEntities: true" to create
* default repositories only for aggregate roots */
options.AddDefaultRepositories(includeAllEntities: true);
});
Configure<AbpDbContextOptions>(options =>
{
/* The main point to change your DBMS.
* See also TodoAppDbContextFactory for EF Core tooling. */
options.UseSqlServer();
options.Configure<TodoAppDbContext>(innerContext =>
{
ShardingCoreExtension.UseDefaultSharding<TodoAppDbContext>(innerContext.ServiceProvider, innerContext.DbContextOptions);
});
});
context.Services.AddShardingConfigure<TodoAppDbContext>()
.UseRouteConfig(op =>
{
op.AddShardingDataSourceRoute<TodoDataSourceRoute>();
op.AddShardingTableRoute<TodoTableRoute>();
})
.UseConfig((sp, op) =>
{
//var loggerFactory = sp.GetRequiredService<ILoggerFactory>();
op.UseShardingQuery((conStr, builder) =>
{
builder.UseSqlServer(conStr).UseLoggerFactory(efLogger);
});
op.UseShardingTransaction((connection, builder) =>
{
builder.UseSqlServer(connection).UseLoggerFactory(efLogger);
});
op.UseShardingMigrationConfigure(builder =>
{
builder.ReplaceService<IMigrationsSqlGenerator, ShardingSqlServerMigrationsSqlGenerator>();
});
op.AddDefaultDataSource("ds0", "Server=.;Database=TodoApp;Trusted_Connection=True");
op.AddExtraDataSource(sp =>
{
return new Dictionary<string, string>()
{
{ "ds1", "Server=.;Database=TodoApp1;Trusted_Connection=True" },
{ "ds2", "Server=.;Database=TodoApp2;Trusted_Connection=True" }
};
});
})
.AddShardingCore();
}
public override void OnPostApplicationInitialization(ApplicationInitializationContext context)
{
base.OnPostApplicationInitialization(context);
//創(chuàng)建表的定時(shí)任務(wù)如果有按年月日系統(tǒng)默認(rèn)路由的需要系統(tǒng)創(chuàng)建的記得開(kāi)起來(lái)
context.ServiceProvider.UseAutoShardingCreate();
//補(bǔ)償表 //自動(dòng)遷移的話不需要
//context.ServiceProvider.UseAutoTryCompensateTable();
}
}啟動(dòng)abp遷移項(xiàng)目
啟動(dòng)：
等待輸出：
插入todoitem
查詢
驗(yàn)證
到此為止我們這邊完成了針對(duì)abpvnext的分表分庫(kù)+自動(dòng)化遷移的操作
集成Furion
接下來(lái)我們開(kāi)始集成Furion的操作
首先依舊安裝依賴
3.7.5+版本直接參考demo相對(duì)簡(jiǎn)單很多
添加依賴 ShardingCore 6.6.0.3 MySql
Install-Package Furion -Version 3.7.5 //請(qǐng)安裝最新版本目前x.6.0.5+,第一個(gè)版本號(hào)6代表efcore的版本號(hào) Install-Package ShardingCore -Version 6.6.0.5 Install-Package Pomelo.EntityFrameworkCore.MySql -Version 6.0.1 Install-Package Microsoft.EntityFrameworkCore.Tools -Version 6.0.6
新增todoitem
public class TodoItem:IEntity, IEntityTypeBuilder<TodoItem>
{
public string Id { get; set; }
public string Text { get; set; }
public void Configure(EntityTypeBuilder<TodoItem> entityBuilder, DbContext dbContext, Type dbContextLocator)
{
entityBuilder.HasKey(o => o.Id);
entityBuilder.Property(o => o.Id).IsRequired().HasMaxLength(50).HasComment("id");
entityBuilder.Property(o => o.Text).IsRequired().HasMaxLength(256).HasComment("事情");
entityBuilder.ToTable(nameof(TodoItem));
}
}
新增帶分片的DbContext和Abp一樣
抽象對(duì)象直接看遠(yuǎn)嗎,這邊直接新增一個(gè)dbcontext
public class MyDbContext : AppShardingDbContext<MyDbContext>,IShardingTableDbContext
{
public MyDbContext(DbContextOptions<MyDbContext> options) : base(options)
{
}
public IRouteTail RouteTail { get; set; }
}新增分表分庫(kù)路由
新增分庫(kù)路由：
public class TodoItemDataSourceRoute:AbstractShardingOperatorVirtualDataSourceRoute<TodoItem,string>
{
/// <summary>
/// id的hashcode取模余3分庫(kù)
/// </summary>
/// <param name="shardingKey"></param>
/// <returns></returns>
/// <exception cref="InvalidOperationException"></exception>
public override string ShardingKeyToDataSourceName(object shardingKey)
{
if (shardingKey == null) throw new InvalidOperationException("sharding key cant null");
var stringHashCode = ShardingCoreHelper.GetStringHashCode(shardingKey.ToString());
return $"ds{(Math.Abs(stringHashCode) % 3)}";//ds0,ds1,ds2
}
private readonly List<string> _dataSources = new List<string>() { "ds0", "ds1", "ds2" };
public override List<string> GetAllDataSourceNames()
{
return _dataSources;
}
public override bool AddDataSourceName(string dataSourceName)
{
throw new NotImplementedException();
}
/// <summary>
/// id分庫(kù)
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataDataSourceBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Id);
}
public override Func<string, bool> GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator)
{
var t = ShardingKeyToDataSourceName(shardingKey);
switch (shardingOperator)
{
case ShardingOperatorEnum.Equal: return tail => tail == t;
default:
{
return tail => true;
}
}
}
}新增分表路由
public class TodoItemTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<TodoItem>
{
public TodoItemTableRoute() : base(2, 3)
{
}
/// <summary>
/// 正常情況下不會(huì)用內(nèi)容來(lái)做分片鍵因?yàn)樽鳛榉制I有個(gè)前提就是不會(huì)被修改
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataTableBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Text);
}
}編寫(xiě)遷移文件
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Migrations.Operations;
using Pomelo.EntityFrameworkCore.MySql.Infrastructure.Internal;
using Pomelo.EntityFrameworkCore.MySql.Migrations;
using ShardingCore.Core.RuntimeContexts;
using ShardingCore.Helpers;
namespace TodoApp;
public class ShardingMySqlMigrationsSqlGenerator:MySqlMigrationsSqlGenerator
{
private readonly IShardingRuntimeContext _shardingRuntimeContext;
public ShardingMySqlMigrationsSqlGenerator(MigrationsSqlGeneratorDependencies dependencies, IRelationalAnnotationProvider annotationProvider, IMySqlOptions options,IShardingRuntimeContext shardingRuntimeContext) : base(dependencies, annotationProvider, options)
{
_shardingRuntimeContext = shardingRuntimeContext;
}
protected override void Generate(
MigrationOperation operation,
IModel model,
MigrationCommandListBuilder builder)
{
var oldCmds = builder.GetCommandList().ToList();
base.Generate(operation, model, builder);
var newCmds = builder.GetCommandList().ToList();
var addCmds = newCmds.Where(x => !oldCmds.Contains(x)).ToList();
MigrationHelper.Generate(_shardingRuntimeContext,operation, builder, Dependencies.SqlGenerationHelper, addCmds);
}
}啟動(dòng)注入
這邊簡(jiǎn)單看了一下furion貌似沒(méi)有提供Func<IServiceProvider,DbContextOptionBuilder>的efcore注入方式所以這邊不得已采用靜態(tài)方式,
如果采用靜態(tài)的方式需要實(shí)現(xiàn)一個(gè)接口IDbContextCreator
//靜態(tài)創(chuàng)建IShardingRuntimeContext
public class ShardingCoreProvider
{
private static ILoggerFactory efLogger = LoggerFactory.Create(builder =>
{
builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole();
});
private static readonly IShardingRuntimeContext instance;
public static IShardingRuntimeContext ShardingRuntimeContext => instance;
static ShardingCoreProvider()
{
instance=new ShardingRuntimeBuilder<MyDbContext>().UseRouteConfig(op =>
{
op.AddShardingTableRoute<TodoItemTableRoute>();
op.AddShardingDataSourceRoute<TodoItemDataSourceRoute>();
})
.UseConfig((sp,op) =>
{
op.UseShardingQuery((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.UseShardingTransaction((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.AddDefaultDataSource("ds0", "server=127.0.0.1;port=3306;database=furion0;userid=root;password=root;");
op.AddExtraDataSource(sp=>new Dictionary<string, string>()
{
{"ds1", "server=127.0.0.1;port=3306;database=furion1;userid=root;password=root;"},
{"ds2", "server=127.0.0.1;port=3306;database=furion2;userid=root;password=root;"}
});
op.UseShardingMigrationConfigure(b =>
{
b.ReplaceService<IMigrationsSqlGenerator, ShardingMySqlMigrationsSqlGenerator>();
});
}).ReplaceService<IDbContextCreator, CustomerDbContextCreator>(ServiceLifetime.Singleton).Build();
}
}
//啟動(dòng)服務(wù)
public class ShardingCoreComponent:IServiceComponent
{
public void Load(IServiceCollection services, ComponentContext componentContext)
{
services.AddControllers();
services.AddEndpointsApiExplorer();
services.AddSwaggerGen();
services.AddDatabaseAccessor(options =>
{
// 配置默認(rèn)數(shù)據(jù)庫(kù)
options.AddDb<MyDbContext>(o =>
{
o.UseDefaultSharding<MyDbContext>(ShardingCoreProvider.ShardingRuntimeContext);
});
});
//依賴注入
services.AddSingleton<IShardingRuntimeContext>(sp => ShardingCoreProvider.ShardingRuntimeContext);
}
}
public class CustomerDbContextCreator:ActivatorDbContextCreator<MyDbContext>
{
public override DbContext GetShellDbContext(IShardingProvider shardingProvider)
{
var dbContextOptionsBuilder = new DbContextOptionsBuilder<MyDbContext>();
dbContextOptionsBuilder.UseDefaultSharding<MyDbContext>(ShardingCoreProvider.ShardingRuntimeContext);
return new MyDbContext(dbContextOptionsBuilder.Options);
}
}
public class UseShardingCoreComponent:IApplicationComponent
{
public void Load(IApplicationBuilder app, IWebHostEnvironment env, ComponentContext componentContext)
{
//......
app.ApplicationServices.UseAutoShardingCreate();
var serviceProvider = app.ApplicationServices;
using (var scope = app.ApplicationServices.CreateScope())
{
var defaultShardingDbContext = scope.ServiceProvider.GetRequiredService<MyDbContext>();
if (defaultShardingDbContext.Database.GetPendingMigrations().Any())
{
defaultShardingDbContext.Database.Migrate();
}
}
// app.Services.UseAutoTryCompensateTable();
}
}
//Program
using TodoApp;
Serve.Run(RunOptions.Default
.AddComponent<ShardingCoreComponent>()
.UseComponent<UseShardingCoreComponent>());添加遷移文件
啟動(dòng)：
增刪改查：
集成WTM
之前也有一次繼承過(guò)之后也有因?yàn)檫w移過(guò)于麻煩所以這邊ShardingCore出了更加完善遷移方案并且使用起來(lái)code-first更加無(wú)感
添加依賴
添加依賴 ShardingCore 6.6.0.3 MySql
//請(qǐng)安裝最新版本目前x.6.0.5+,第一個(gè)版本號(hào)6代表efcore的版本號(hào) Install-Package ShardingCore -Version 6.6.0.5 Install-Package Microsoft.EntityFrameworkCore.Tools -Version 6.0.6
新增分表分庫(kù)路由
//分庫(kù)路由
public class TodoDataSourceRoute:AbstractShardingOperatorVirtualDataSourceRoute<Todo,string>
{
/// <summary>
/// id的hashcode取模余3分庫(kù)
/// </summary>
/// <param name="shardingKey"></param>
/// <returns></returns>
/// <exception cref="InvalidOperationException"></exception>
public override string ShardingKeyToDataSourceName(object shardingKey)
{
if (shardingKey == null) throw new InvalidOperationException("sharding key cant null");
var stringHashCode = ShardingCoreHelper.GetStringHashCode(shardingKey.ToString());
return $"ds{(Math.Abs(stringHashCode) % 3)}";//ds0,ds1,ds2
}
private readonly List<string> _dataSources = new List<string>() { "ds0", "ds1", "ds2" };
public override List<string> GetAllDataSourceNames()
{
return _dataSources;
}
public override bool AddDataSourceName(string dataSourceName)
{
throw new NotImplementedException();
}
/// <summary>
/// id分庫(kù)
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataDataSourceBuilder<Todo> builder)
{
builder.ShardingProperty(o => o.Id);
}
public override Func<string, bool> GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator)
{
var t = ShardingKeyToDataSourceName(shardingKey);
switch (shardingOperator)
{
case ShardingOperatorEnum.Equal: return tail => tail == t;
default:
{
return tail => true;
}
}
}
}
//分表路由
public class TodoTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<Todo>
{
public TodoTableRoute() : base(2, 3)
{
}
/// <summary>
/// 正常情況下不會(huì)用內(nèi)容來(lái)做分片鍵因?yàn)樽鳛榉制I有個(gè)前提就是不會(huì)被修改
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataTableBuilder<Todo> builder)
{
builder.ShardingProperty(o => o.Name);
}
}創(chuàng)建DbContextCreator
public class WTMDbContextCreator:IDbContextCreator
{
public DbContext CreateDbContext(DbContext shellDbContext, ShardingDbContextOptions shardingDbContextOptions)
{
var context = new DataContext((DbContextOptions<DataContext>)shardingDbContextOptions.DbContextOptions);
context.RouteTail = shardingDbContextOptions.RouteTail;
return context;
}
public DbContext GetShellDbContext(IShardingProvider shardingProvider)
{
var dbContextOptionsBuilder = new DbContextOptionsBuilder<DataContext>();
dbContextOptionsBuilder.UseDefaultSharding<DataContext>(ShardingCoreProvider.ShardingRuntimeContext);
return new DataContext(dbContextOptionsBuilder.Options);
}
}遷移腳本
public class ShardingMySqlMigrationsSqlGenerator:MySqlMigrationsSqlGenerator
{
private readonly IShardingRuntimeContext _shardingRuntimeContext;
public ShardingMySqlMigrationsSqlGenerator(MigrationsSqlGeneratorDependencies dependencies, IRelationalAnnotationProvider annotationProvider, IMySqlOptions options,IShardingRuntimeContext shardingRuntimeContext) : base(dependencies, annotationProvider, options)
{
_shardingRuntimeContext = shardingRuntimeContext;
}
protected override void Generate(
MigrationOperation operation,
IModel model,
MigrationCommandListBuilder builder)
{
var oldCmds = builder.GetCommandList().ToList();
base.Generate(operation, model, builder);
var newCmds = builder.GetCommandList().ToList();
var addCmds = newCmds.Where(x => !oldCmds.Contains(x)).ToList();
MigrationHelper.Generate(_shardingRuntimeContext,operation, builder, Dependencies.SqlGenerationHelper, addCmds);
}
}靜態(tài)構(gòu)造IShardingRuntimeContext
因?yàn)閃TM在創(chuàng)建dbcontext并不是通過(guò)依賴注入創(chuàng)建的而是由其余的內(nèi)部實(shí)現(xiàn)所以為了兼容我們這邊只能通過(guò)靜態(tài)IShardingRuntimeContext注入
public class ShardingCoreProvider
{
private static ILoggerFactory efLogger = LoggerFactory.Create(builder =>
{
builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole();
});
private static readonly IShardingRuntimeContext instance;
public static IShardingRuntimeContext ShardingRuntimeContext => instance;
static ShardingCoreProvider()
{
instance=new ShardingRuntimeBuilder<DataContext>().UseRouteConfig(op =>
{
op.AddShardingTableRoute<TodoRoute>();
op.AddShardingDataSourceRoute<TodoDataSourceRoute>();
})
.UseConfig((sp,op) =>
{
op.UseShardingQuery((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.UseShardingTransaction((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.AddDefaultDataSource("ds0", "server=127.0.0.1;port=3306;database=wtm0;userid=root;password=root;");
op.AddExtraDataSource(sp=>new Dictionary<string, string>()
{
{"ds1", "server=127.0.0.1;port=3306;database=wtm1;userid=root;password=root;"},
{"ds2", "server=127.0.0.1;port=3306;database=wtm2;userid=root;password=root;"}
});
op.UseShardingMigrationConfigure(b =>
{
b.ReplaceService<IMigrationsSqlGenerator, ShardingMySqlMigrationsSqlGenerator>();
});
}).ReplaceService<IDbContextCreator, WTMDbContextCreator>(ServiceLifetime.Singleton).Build();
}
}創(chuàng)建抽象分片DbContext
因?yàn)檫^(guò)于長(zhǎng)所以這邊只顯示主要部分其余通過(guò)demo查看
public abstract class AbstractShardingFrameworkContext:FrameworkContext, IShardingDbContext, ISupportShardingReadWrite
{
protected IShardingDbContextExecutor ShardingDbContextExecutor
{
get;
}
public AbstractShardingFrameworkContext(CS cs)
: base(cs)
{
ShardingDbContextExecutor =new ShardingDbContextExecutor(this);
IsExecutor = false;
}
public AbstractShardingFrameworkContext(string cs, DBTypeEnum dbtype)
: base(cs, dbtype)
{
ShardingDbContextExecutor =new ShardingDbContextExecutor(this);
IsExecutor = false;
}
public AbstractShardingFrameworkContext(string cs, DBTypeEnum dbtype, string version = null)
: base(cs, dbtype, version)
{
ShardingDbContextExecutor =new ShardingDbContextExecutor(this);
IsExecutor = false;
}
public AbstractShardingFrameworkContext(DbContextOptions options) : base(options)
{
var wrapOptionsExtension = options.FindExtension<ShardingWrapOptionsExtension>();
if (wrapOptionsExtension != null)
{
ShardingDbContextExecutor =new ShardingDbContextExecutor(this);;
}
IsExecutor = wrapOptionsExtension == null;
}
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
if (this.CSName!=null)
{
base.OnConfiguring(optionsBuilder);
optionsBuilder.UseDefaultSharding<DataContext>(ShardingCoreProvider.ShardingRuntimeContext);
}
}
public DbContext GetDbContext(string dataSourceName, CreateDbContextStrategyEnum strategy, IRouteTail routeTail)
{
return ShardingDbContextExecutor.CreateDbContext(strategy, dataSourceName, routeTail);
}
}修改dbcontext
public class DataContextFactory : IDesignTimeDbContextFactory<DataContext>
{
public DataContext CreateDbContext(string[] args)
{
var virtualDataSource = ShardingCoreProvider.ShardingRuntimeContext.GetVirtualDataSource();
var defaultConnectionString = virtualDataSource.DefaultConnectionString;
return new DataContext(defaultConnectionString, DBTypeEnum.MySql);
}
}注入ShardingCore
移除掉了之前的多余代碼
public void ConfigureServices(IServiceCollection services){
//....
services.AddSingleton<IShardingRuntimeContext>(sp => ShardingCoreProvider.ShardingRuntimeContext);
}
public void Configure(IApplicationBuilder app, IOptionsMonitor<Configs> configs)
{
IconFontsHelper.GenerateIconFont();
// using (var scope = app.ApplicationServices.CreateScope())
// {
// var requiredService = scope.ServiceProvider.GetRequiredService<WTMContext>();
// var requiredServiceDc = requiredService.DC;
// }
//定時(shí)任務(wù)
app.ApplicationServices.UseAutoShardingCreate();
using (var dbconContext=new DataContextFactory().CreateDbContext(new string[0]))
{
dbconContext.Database.Migrate();
}
//補(bǔ)齊表防止iis之類的休眠導(dǎo)致按天按月的表沒(méi)有新建
//app.ApplicationServices.UseAutoTryCompensateTable();
//....
}遷移：
啟動(dòng)程序：
crud：
到此這篇關(guān)于.Net極限生產(chǎn)力之分表分庫(kù)全自動(dòng)化Migrations Code-First的文章就介紹到這了,更多相關(guān).Net自動(dòng)化Migrations Code-First內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！
ASP.Net防止刷新自動(dòng)觸發(fā)事件的解決方案
ASP.Net防止刷新自動(dòng)觸發(fā)事件的解決方案...
2006-09-09 
ASP.NET Core自定義中間件如何讀取Request.Body與Response.Body的內(nèi)容詳解
這篇文章主要給大家介紹了關(guān)于在ASP.NET Core自定義中間件中如何讀取Request.Body與Response.Body的內(nèi)容，文中通過(guò)示例代碼介紹的非常詳細(xì)，對(duì)大家學(xué)習(xí)或者使用ASP.NET Core具有一定的參考學(xué)習(xí)價(jià)值，需要的朋友們下面來(lái)一起學(xué)習(xí)學(xué)習(xí)吧
2020-05-05 
Asp.net中處理一個(gè)站點(diǎn)不同Web應(yīng)用共享Session的問(wèn)題
Asp.net中處理一個(gè)站點(diǎn)不同Web應(yīng)用共享Session的問(wèn)題...
2006-09-09 
這篇文章主要介紹了ASP.NET MVC中的視圖生成的過(guò)程,以實(shí)例形式詳細(xì)分析了控制器、布局與視圖的操作方法與技巧,非常具有實(shí)用價(jià)值,需要的朋友可以參考下
2014-12-12 
點(diǎn)擊提交按鈕后DropDownList的值變?yōu)槟J(rèn)值實(shí)現(xiàn)分析
在點(diǎn)擊提交按鈕后，頁(yè)面上所有的綁定到數(shù)據(jù)庫(kù)的控件值都恢復(fù)到默認(rèn)值，下面與大家分享下DropDownList的值變?yōu)槟J(rèn)值
2013-05-05 
asp.net session的使用與過(guò)期實(shí)例代碼
本文章來(lái)簡(jiǎn)單的介紹asp.net中session常見(jiàn)兩種用法，一種是session使用如何創(chuàng)建，另一種是告訴你如何判斷session過(guò)期了，有需要了解的朋友可以參考一下
2013-08-08 
asp.net 防止用戶通過(guò)后退按鈕重復(fù)提交表單
經(jīng)過(guò)一番仔細(xì)的尋尋覓覓之后，我發(fā)現(xiàn)仍舊無(wú)法找出真正能夠完全禁用瀏覽器后退按鈕的辦法。所有這里介紹的方法都能夠在不同程度上、以不同的方式禁止用戶返回前一頁(yè)面，但它們都有各自的局限。
2009-11-11 
不錯(cuò)的應(yīng)用，大家可以拓展到，判斷遠(yuǎn)程文件是否存在等功能
2008-09-09 
ASP.NET Core模仿中間件方式實(shí)現(xiàn)列表過(guò)濾功能
這篇文章介紹了ASP.NET Core模仿中間件方式實(shí)現(xiàn)列表過(guò)濾功能的方法，文中通過(guò)示例代碼介紹的非常詳細(xì)。對(duì)大家的學(xué)習(xí)或工作具有一定的參考借鑒價(jià)值，需要的朋友可以參考下
2022-07-07