Redis數(shù)據(jù)遷移RedisShake的實現(xiàn)方法
一、基本功能
redis-shake它支持解析、恢復、備份、同步四個功能
恢復restore:將RDB文件恢復到目的redis數(shù)據(jù)庫。
備份dump:將源redis的全量數(shù)據(jù)通過RDB文件備份起來。
解析decode:對RDB文件進行讀取,并以json格式解析存儲。
同步sync:支持源redis和目的redis的數(shù)據(jù)同步,支持全量和增量數(shù)據(jù)的遷移,支持單節(jié)點、主從版、集群版之間的互相同步。
同步rump:支持源redis和目的redis的數(shù)據(jù)同步,僅支持全量的遷移,采用scan和restore命令進行遷移,支持不同云廠商不同redis版本的遷移。
二、基本原理
三、RedisShake同步原理
1.源Redis服務實例相當于主庫,Redis-shake相當于從庫,它會發(fā)送psync指令給源Redis服務實例。
2.源Redis實例先把RDB文件傳輸給 Redis-shake ,Redis-shake 會把RDB文件發(fā)送給目的實例。
3.源實例會再把增量命令發(fā)送給 Redis-shake ,Redis-shake負責把這些增量命令再同步給目的實例。
四、RedisShake安裝
確保您在本地機器上設置了 Golang 環(huán)境。
4.1、release包下載
Releases · tair-opensource/RedisShake · GitHub
4.2、解壓
tar -zxvf redis-shake-linux-amd64.tar.gz -C /home/redisshake/
解壓完了之后有兩個文件:
4.3、修改shake.toml配置文件
function = "" [sync_reader] cluster = false # set to true if source is a redis cluster address = "127.0.0.1:6379" # when cluster is true, set address to one of the cluster node username = "" # keep empty if not using ACL password = "" # keep empty if no authentication is required tls = false sync_rdb = true # set to false if you don't want to sync rdb sync_aof = true # set to false if you don't want to sync aof prefer_replica = true # set to true if you want to sync from replica node #[scan_reader] #cluster = false # set to true if source is a redis cluster #address = "127.0.0.1:6379" # when cluster is true, set address to one of the cluster node #username = "" # keep empty if not using ACL #password = "" # keep empty if no authentication is required #tls = false #dbs = [] # set you want to scan dbs such as [1,5,7], if you don't want to scan all #scan = true # set to false if you don't want to scan keys #ksn = false # set to true to enabled Redis keyspace notifications (KSN) subscription #count = 1 # number of keys to scan per iteration # [rdb_reader] # filepath = "/tmp/dump.rdb" # [aof_reader] # filepath = "/tmp/.aof" # timestamp = 0 # subsecond [redis_writer] cluster = false # set to true if target is a redis cluster sentinel = false # set to true if target is a redis sentinel master = "" # set to master name if target is a redis sentinel address = "192.168.72.129:6379" # when cluster is true, set address to one of the cluster node username = "" # keep empty if not using ACL password = "" # keep empty if no authentication is required tls = false off_reply = false # ture off the server reply [advanced] dir = "data" ncpu = 0 # runtime.GOMAXPROCS, 0 means use runtime.NumCPU() cpu cores pprof_port = 0 # pprof port, 0 means disable status_port = 0 # status port, 0 means disable # log log_file = "shake.log" log_level = "info" # debug, info or warn log_interval = 5 # in seconds # redis-shake gets key and value from rdb file, and uses RESTORE command to # create the key in target redis. Redis RESTORE will return a "Target key name # is busy" error when key already exists. You can use this configuration item # to change the default behavior of restore: # panic: redis-shake will stop when meet "Target key name is busy" error. # rewrite: redis-shake will replace the key with new value. # ignore: redis-shake will skip restore the key when meet "Target key name is busy" error. rdb_restore_command_behavior = "panic" # panic, rewrite or skip # redis-shake uses pipeline to improve sending performance. # This item limits the maximum number of commands in a pipeline. pipeline_count_limit = 1024 # Client query buffers accumulate new commands. They are limited to a fixed # amount by default. This amount is normally 1gb. target_redis_client_max_querybuf_len = 1024_000_000 # In the Redis protocol, bulk requests, that are, elements representing single # strings, are normally limited to 512 mb. target_redis_proto_max_bulk_len = 512_000_000 # If the source is Elasticache or MemoryDB, you can set this item. aws_psync = "" # example: aws_psync = "10.0.0.1:6379@nmfu2sl5osync,10.0.0.1:6379@xhma21xfkssync" # destination will delete itself entire database before fetching files # from source during full synchronization. # This option is similar redis replicas RDB diskless load option: # repl-diskless-load on-empty-db empty_db_before_sync = false [module] # The data format for BF.LOADCHUNK is not compatible in different versions. v2.6.3 <=> 20603 target_mbbloom_version = 20603
官方使用指導手冊:Sync Reader | RedisShake
4.4、啟動RediShake
./redis-shake shake.toml
4.5、測試數(shù)據(jù)遷移
在192.168.72.128這臺機器上插入幾個key
可以看到RedisShake工具在實施監(jiān)聽key,有新增的就會把新增的key遷移到另外一機器的redis中;看打印的日志是寫了4條數(shù)據(jù)
在192.168.72.129這臺機器上查看遷移的數(shù)據(jù)
4.6、數(shù)據(jù)校驗
通過info keyspace命令,查看所有的key和過期key的數(shù)量。
到此這篇關于Redis數(shù)據(jù)遷移RedisShake的實現(xiàn)方法的文章就介紹到這了,更多相關Redis數(shù)據(jù)遷移RedisShake內容請搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關文章希望大家以后多多支持腳本之家!
相關文章
基于 Redis 的 JWT令牌失效處理方案(實現(xiàn)步驟)
當用戶登錄狀態(tài)到登出狀態(tài)時,對應的JWT的令牌需要設置為失效狀態(tài),這時可以使用基于Redis 的黑名單方案來實現(xiàn)JWT令牌失效,本文給大家分享基于 Redis 的 JWT令牌失效處理方案,感興趣的朋友一起看看吧2024-03-03redis分布式鎖的go-redis實現(xiàn)方法詳解
這篇文章主要介紹了redis分布式鎖的go-redis實現(xiàn)方法,本文給大家介紹的非常詳細對大家的學習或工作具有一定的參考借鑒價值,需要的朋友可以參考下2020-12-12