在Linux中刪除超大(100-200GB)文件的實現(xiàn)方式
一、IO調(diào)度策略
1.1、idle
空閑磁盤調(diào)度;
該調(diào)度策略是在當前系統(tǒng)沒有其他進程需要進行磁盤IO時,才能進行磁盤;
因此該策略對當前系統(tǒng)的影響基本為0;
當然,該調(diào)度策略不能帶有任何優(yōu)先級參數(shù);
1.2、Best effort
是缺省的磁盤IO調(diào)度策略;
(1)該調(diào)度策略可以指定優(yōu)先級參數(shù)(范圍是0~7,數(shù)值越小,優(yōu)先級越高);
(2)針對處于同一優(yōu)先級的程序?qū)⒉蓃ound-robin方式;
(3)對于best effort調(diào)度策略,8個優(yōu)先級等級可以說明在給定的一個調(diào)度窗口中時間片的大??;
(4)目前,普調(diào)用戶(非root用戶)是可以使用該調(diào)度策略;
(5)在內(nèi)核2.6.26之前,沒有設(shè)置IO優(yōu)先級的進程會使用“none”作為調(diào)度策略,但是這種策略使得進程看起來像是采用了best effort調(diào)度策略,因為其優(yōu)先級是通過關(guān)于 cpu nice有關(guān)的公式計算得到的:io_priority = (cpu_nice + 20) /5
(6)在內(nèi)核2.6.26之后,如果當前系統(tǒng)使用的是CFQ調(diào)度器,那么如果進程沒有設(shè)置IO優(yōu)先級級別,將采用與內(nèi)核2.6.26之前版本同樣的方式,推到出io優(yōu)先級級別;
1.3、Real time
實時調(diào)度策略,如果設(shè)置了該磁盤IO調(diào)度策略,則立即訪問磁盤,不管系統(tǒng)中其他進程是否有IO。
因此使用實時調(diào)度策略,需要注意的是,該訪問策略可能會使得其他進程處于等待狀態(tài);
二、ionice命令-設(shè)置排程優(yōu)先級
2.1、安裝
[root@docker][19:43:12][OK] ~ #yum provides */ionice Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * epel: mirror.01link.hk * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com epel/x86_64/filelists_db | 12 MB 00:00:08 extras/7/x86_64/filelists_db | 276 kB 00:00:01 updates/7/x86_64/filelists_db | 10 MB 00:00:42 util-linux-2.23.2-65.el7.i686 : A collection of basic system utilities Repo : base Matched from: Filename : /usr/bin/ionice Filename : /usr/share/bash-completion/completions/ionice util-linux-2.23.2-65.el7.x86_64 : A collection of basic system utilities Repo : base Matched from: Filename : /usr/bin/ionice Filename : /usr/share/bash-completion/completions/ionice util-linux-2.23.2-65.el7_9.1.i686 : A collection of basic system utilities Repo : updates Matched from: Filename : /usr/bin/ionice Filename : /usr/share/bash-completion/completions/ionice util-linux-2.23.2-65.el7_9.1.x86_64 : A collection of basic system utilities Repo : updates Matched from: Filename : /usr/bin/ionice Filename : /usr/share/bash-completion/completions/ionice util-linux-2.23.2-65.el7.x86_64 : A collection of basic system utilities Repo : @anaconda Matched from: Filename : /usr/bin/ionice Filename : /usr/share/bash-completion/completions/ionice
安裝
yum install util-linux-2.23.2-65.el7_9.1.x86_64 -y
2.2、用法
[root@docker][19:35:01][OK] ~ #ionice --help ionice - sets or gets process io scheduling class and priority. Usage: ionice [OPTION] -p PID [PID...] ionice [OPTION] COMMAND Options: -c, --class <class> scheduling class name or number 0: none, 1: realtime, 2: best-effort, 3: idle -n, --classdata <num> scheduling class data 0-7 for realtime and best-effort classes -p, --pid=PID view or modify already running process -t, --ignore ignore failures -V, --version output version information and exit -h, --help display this help and exit
2.3、示例
—- Deleting Huge Files in Linux —– # ionice -c 3 rm /var/logs/syslog # ionice -c 3 rm -rf /var/log/apache
注意:要使用安全的方法刪除大檔案,我們可以使 shred、wipe 以及 secure-deletion 工具包中的不同工具,而不是 rm 命令。
三、Shred -覆蓋文件以隱藏內(nèi)容
3.1、安裝軟件包
yum install coreutils-8.22-24.el7_9.2.x86_64 -y
3.2、命令及參數(shù)選項
[root@192_168_209_128 ~]# shred --help Usage: shred [OPTION]... FILE... Overwrite the specified FILE(s) repeatedly, in order to make it harder for even very expensive hardware probing to recover the data. Mandatory arguments to long options are mandatory for short options too. -f, --force change permissions to allow writing if necessary -n, --iterations=N overwrite N times instead of the default (3) --random-source=FILE get random bytes from FILE -s, --size=N shred this many bytes (suffixes like K, M, G accepted) -u, --remove[=HOW] truncate and remove file after overwriting; See below -v, --verbose show progress -x, --exact do not round file sizes up to the next full block; this is the default for non-regular files -z, --zero add a final overwrite with zeros to hide shredding --help display this help and exit --version output version information and exit
選項說明:
- -z- 添加一個帶有零的最終覆蓋以隱藏切碎。
- -v– 可以顯示操作進度。
- -u- 覆蓋后截斷和刪除文件。
- -n– 指定覆蓋文件內(nèi)容的次數(shù)(默認為 3)。
3.3、示例
shred覆蓋文件以隱藏其內(nèi)容,也可以選擇刪除它。
[root@docker][19:51:03][OK] ~ #shred -zvu -n 5 cuttingLog_bak shred: cuttingLog_bak: pass 1/6 (random)... shred: cuttingLog_bak: pass 2/6 (ffffff)... shred: cuttingLog_bak: pass 3/6 (random)... shred: cuttingLog_bak: pass 4/6 (000000)... shred: cuttingLog_bak: pass 5/6 (random)... shred: cuttingLog_bak: pass 6/6 (000000)... shred: cuttingLog_bak: removing shred: cuttingLog_bak: renamed to 00000000000000 shred: 00000000000000: renamed to 0000000000000 shred: 0000000000000: renamed to 000000000000 shred: 000000000000: renamed to 00000000000 shred: 00000000000: renamed to 0000000000 shred: 0000000000: renamed to 000000000 shred: 000000000: renamed to 00000000 shred: 00000000: renamed to 0000000 shred: 0000000: renamed to 000000 shred: 000000: renamed to 00000 shred: 00000: renamed to 0000 shred: 0000: renamed to 000 shred: 000: renamed to 00 shred: 00: renamed to 0 shred: cuttingLog_bak: removed
四、Wipe —在 Linux 中安全擦除文件
介紹:Linux wipe命令可以安全地從磁存儲器中擦除文件,從而無法恢復(fù)。
4.1、安裝軟件包
rpm -ivh wipe-2.2.0-1.2.el7.rf.x86_64.rpm
4.2、命令及參數(shù)選項
[root@docker][19:58:11][OK] ~ #wipe -h Wipe v2.2.0 - released January 10, 2004 by Tom Vier <nester@users.sf.net> Usage is wipe [options] [file-list] Options: Default: wipe -ZdntVAkO -S512 -C4096 -l1 -x8 -p1 -h -- help - display this screen -u -- usage -c -- show copyright and license -w -- show warranty information -i and -I -- enable (-i) or disable (-I) interaction - overrides force -f -- force file wiping and override interaction -r and -R -- recursion - traverse subdirectories -s -- silent - disable percentage and error reporting -v -- force verbose - always show percentage -V -- verbose - show percentage if file is >= 25K -d and -D -- delete (-d) or keep (-D) after wiping -n and -N -- delete (-n) or skip (-N) special files -k and -K -- lock (-k) or don't lock (-K) files -z -- zero-out file - single pass of zeroes -Z -- perform normal wipe passes -t and -T -- enable (-t) or disable (-T) static passes -a and -A -- write until out of space (-a) or don't (-A) -o[size] -O -- write to stdout (-o) or use files (-O) -B(count) -- block device sector count -S(size) -- block device sector size - default 512 bytes or stdout write length when used with -A -C(size) -- chunk size - maximum file buffer size in kilobytes (2^10) -l[0-2] -- sets wipe secure level -x[1-32] -X -- sets number of random passes per wipe or disables -p(1-32) -- wipe file x number of times -b(0-255) -- overwrite file with this value byte
4.3、示例
[root@docker][19:59:50][OK] ~ #wipe -fi cuttingLog_bak wipe: destroy file `cuttingLog_bak'? y
五、Linux 安全刪除工具包
Secure-delete是一個安全文件刪除工具的集合,其中包含srm (secure_deletion) 工具,用于安全刪除文件。
5.1、安裝軟件包
yum -y install srm-1.2.15-1.el7.x86_64
5.2、命令及參數(shù)選項
[root@docker][20:05:39][OK] ~ #srm --help Usage: srm [OPTION]... [FILE]... Overwrite and remove (unlink) the files. By default use the 35-pass Gutmann method to overwrite files. -d, --directory ignored (for compatability with rm(1)) -f, --force ignore nonexistant files, never prompt -i, --interactive prompt before any removal -x, --one-file-system do not cross file system boundaries -s, --simple overwrite with single pass using 0x00 (default) -P, --openbsd overwrite with three passes like OpenBSD rm -D, --dod overwrite with 7 US DoD compliant passes -E, --doe overwrite with 3 US DoE compliant passes -G, --gutmann overwrite with 35-pass Gutmann method -C, --rcmp overwrite with Royal Canadian Mounted Police passes -r, -R, --recursive remove the contents of directories -v, --verbose explain what is being done -h, --help display this help and exit -V, --version display version information and exit
5.3、示例
#srm -vf cuttingLog_bal srm: removing cuttingLog_bal
總結(jié)
以上為個人經(jīng)驗,希望能給大家一個參考,也希望大家多多支持腳本之家。
相關(guān)文章
linux VPS之間網(wǎng)站數(shù)據(jù)的備份與恢復(fù)(網(wǎng)站遷移教程)
有時候我們需要網(wǎng)站遷移或者網(wǎng)站數(shù)據(jù)需要備份或恢復(fù),那么就需要一些操作了,linux下操作都是命令下執(zhí)行的,特分享下,方便需要的朋友2014-02-02Linux多臺服務(wù)器之間免密登錄的實現(xiàn)步驟
在日常的后端開發(fā)和運維工作中,我們經(jīng)常需要頻繁登錄不同的服務(wù)器進行部署、排查問題或上傳文件,傳統(tǒng)的基于用戶名和密碼的登錄方式既不安全又不高效,為此,我們可以利用 SSH 公鑰認證的方式,實現(xiàn)無密碼(免密)登錄遠程服務(wù)器,下面小編給大家詳細說說2025-04-04linux防火墻iptables規(guī)則的查看、添加、刪除和修改方法總結(jié)
本文介紹了如何對linux防火墻iptables規(guī)則進行查看、添加、刪除和修改的操作,大家可以參考一下2018-03-03Linux應(yīng)用程序使用寫文件調(diào)試程序的方法
今天小編就為大家分享一篇關(guān)于Linux應(yīng)用程序使用寫文件調(diào)試程序的方法,小編覺得內(nèi)容挺不錯的,現(xiàn)在分享給大家,具有很好的參考價值,需要的朋友一起跟隨小編來看看吧2018-12-12Linux系統(tǒng)中CPU占用率較高問題排查思路與解決方法
這篇文章主要給大家介紹了關(guān)于Linux系統(tǒng)中CPU占用率較高問題排查思路與解決方法,文中通過示例代碼介紹的非常詳細,對大家學(xué)習或者使用Linux具有一定的參考學(xué)習價值,需要的朋友們下面來一起學(xué)習學(xué)習吧2019-07-07CentOS 7.4下安裝Oracle 11.2.0.4數(shù)據(jù)庫的方法
本篇文章主要介紹了CentOS 7.4下安裝Oracle 11.2.0.4數(shù)據(jù)庫的方法,小編覺得挺不錯的,現(xiàn)在分享給大家,也給大家做個參考。一起跟隨小編過來看看吧2017-12-12MySQL/MariaDB/Percona數(shù)據(jù)庫升級腳本
這篇文章主要介紹了MySQL/MariaDB/Percona數(shù)據(jù)庫升級腳本的相關(guān)資料,需要的朋友可以參考下2016-12-12