欧美bbbwbbbw肥妇,免费乱码人妻系列日韩,一级黄片

實例講解臨時處理去重 80w 數(shù)據(jù)時夯死現(xiàn)象

 更新時間:2015年09月02日 09:55:30   投稿:lijiao  
這篇文章主要介紹了臨時處理去重 80w 數(shù)據(jù)時夯死現(xiàn)象,需要的朋友可以參考下

近日,在對一張百萬數(shù)據(jù)的業(yè)務(wù)表進(jìn)行去重時,去重操作竟然夯住了。下面就來簡單回憶一下。

1、查詢業(yè)務(wù)表數(shù)據(jù)量,查看到總共有200多w條

SQL> select count(*) from tb_bj_banker_etl;

2552381

2、查詢表內(nèi)應(yīng)該去掉的重復(fù)數(shù)據(jù)量,共80多w條

SQL> select count(*) from tb_bj_banker_etl where (id) in (select id from tb_bj_banker_etl group by id having count(*)>1) and rowid not in(select max(rowid) from tb_bj_banker_etl group by id having count(*)>1);

830099

3、于是,在晚上下班前,執(zhí)行了下面的語句腳本,為了去重

SQL> delete from tb_bj_banker_etl where(id) in (select id from tb_bj_banker_etl group by id having count(*)>1) and rowid not in(select max(rowid) from tb_bj_banker_etl group by id having count(*)>1);

SQL> commit;

4、第二天,到達(dá)現(xiàn)場時,發(fā)現(xiàn)PL/SQL Developer工具中昨天晚上執(zhí)行的語句仍在執(zhí)行中

首先察覺,80多w的去重數(shù)據(jù)跑了一個晚上也沒跑完?這肯定是哪里出了問題?

懷疑有鎖表。

于是查詢是否有鎖表的用戶。

SELECT 
 A.OWNER,            --OBJECT所屬用戶 
 A.OBJECT_NAME,         --OBJECT名稱 
 B.XIDUSN, 
 B.XIDSLOT, 
 B.XIDSQN, 
 B.SESSION_ID,          --鎖表用戶的session 
 B.ORACLE_USERNAME,       --鎖表用戶的Oracle用戶名 
 B.OS_USER_NAME,         --鎖表用戶的操作系統(tǒng)登陸用戶名 
 B.PROCESS, 
 B.LOCKED_MODE, 
 C.MACHINE,           --鎖表用戶的計算機(jī)名稱 
 C.STATUS,            --鎖表狀態(tài) 
 C.SERVER, 
 C.SID, 
 C.SERIAL#, 
 C.PROGRAM            --鎖表用戶所用的數(shù)據(jù)庫管理工具 
FROM 
 ALL_OBJECTS A, 
 V$LOCKED_OBJECT B, 
 SYS.GV_$SESSION C 
WHERE 
 A.OBJECT_ID = B.OBJECT_ID 
 AND B.PROCESS = C.PROCESS 
ORDER BY 1,2 

在下面結(jié)果中可以看到,鎖表的只是去重語句的發(fā)起會話,并沒有其它用戶造成鎖表,這說明語句仍然在執(zhí)行嘛?帶著疑問,開始嘗試解決。

1 BJHYL tb_bj_banker_ETL 15 18 9000 913 BJHYL Administrator 4036:972 3 WORKGROUP\BACKDB ACTIVE DEDICATED 913 3381 plsqldev.exe

2 BJHYL tb_bj_banker_ETL 15 18 9000 913 BJHYL Administrator 4036:972 3 WORKGROUP\BACKDB INACTIVE DEDICATED 649 41791 plsqldev.exe

3 BJHYL tb_bj_banker_ETL 15 18 9000 913 BJHYL Administrator 4036:972 3 WORKGROUP\BACKDB INACTIVE DEDICATED 817 27777 plsqldev.exe

4 BJHYL tb_bj_banker_ETL 15 18 9000 913 BJHYL Administrator 4036:972 3 WORKGROUP\BACKDB INACTIVE DEDICATED 841 1981 plsqldev.exe

5、采用分批次,解決去重夯住問題

由于直接去重?zé)o法順利進(jìn)行,于是想到了分批次去重的方法,試一下。

第一次: 
delete from tb_bj_banker_etl where(id) in (select id from tb_bj_banker_etl group by id having count(*)>1) and rowid not in(select max(rowid) from tb_bj_banker_etl group by id having count(*)>1) and rownum<=100000; 
commit; 
 
第二次: 
delete from tb_bj_banker_etl where(id) in (select id from tb_bj_banker_etl group by id having count(*)>1) and rowid not in(select max(rowid) from tb_bj_banker_etl group by id having count(*)>1) and rownum<=100000; 
commit; 
 
。。。。。。。 
。。。。。。。 
。。。。。。。 
 
第八次: 
delete from tb_bj_banker_etl where(id) in (select id from tb_bj_banker_etl group by id having count(*)>1) and rowid not in(select max(rowid) from tb_bj_banker_etl group by id having count(*)>1); 
commit; 

結(jié)果:通過將80多萬數(shù)據(jù)劃分成以10w數(shù)據(jù)為單次進(jìn)行去重操作,總共用時140多秒,完成了去重80萬數(shù)據(jù)的目的。但為何直接處理出現(xiàn)夯死情況,有待后續(xù)跟蹤分析。

以上就是臨時處理去重80w數(shù)據(jù)時夯死現(xiàn)象的全部過程,希望可以幫到大家。

相關(guān)文章

最新評論