PostgreSQL索引失效會(huì)發(fā)生什么
前段時(shí)間碰到個(gè)奇怪的索引失效的問(wèn)題,實(shí)際情況類(lèi)似下面這樣:
bill=# begin;
BEGIN
bill=*# create index idx_t1 on t1(id);
CREATE INDEX
bill=*# explain select * from t1 where id = 1;
QUERY PLAN
----------------------------------------------------
Seq Scan on t1 (cost=0.00..25.88 rows=6 width=36)
Filter: (id = 1)
(2 rows)
bill=*# end;
COMMIT
bill=# explain select * from t1 where id = 1;
QUERY PLAN
---------------------------------------------------------------------
Bitmap Heap Scan on t1 (cost=1.50..7.01 rows=6 width=36)
Recheck Cond: (id = 1)
-> Bitmap Index Scan on idx_t1 (cost=0.00..1.50 rows=6 width=0)
Index Cond: (id = 1)
(4 rows)
很顯然的問(wèn)題就是,我在事務(wù)中創(chuàng)建了索引,卻沒(méi)辦法使用。但是當(dāng)事務(wù)提交了后便可以正常使用了,這是什么情況呢?
這個(gè)其實(shí)和pg_index中indcheckxmin屬性有關(guān),關(guān)于這個(gè)字段的解釋如下:
If true, queries must not use the index until the xmin of this pg_index row is below their TransactionXmin event horizon, because the table may contain broken HOT chains with incompatible rows that they can see
經(jīng)檢查也確實(shí)如此:
bill=*# select indcheckxmin from pg_index where indexrelid = 'idx_t1'::regclass; indcheckxmin -------------- t (1 row)
那么問(wèn)題來(lái)了,什么情況下創(chuàng)建索引時(shí)會(huì)將索引的該屬性設(shè)置為true呢?
從前面官方文檔對(duì)于該字段的解釋?zhuān)绻碇邪琤roken HOT chains 則會(huì)為true,那什么是broken HOT chains ?似乎和HOT機(jī)制有關(guān)。那是不是只有存在broken HOT chains 才會(huì)設(shè)置為true呢?
這里就不賣(mài)關(guān)子了,直接給出結(jié)論,然后我們?cè)偃ヒ灰或?yàn)證。
經(jīng)測(cè)試發(fā)現(xiàn),以下兩種情況會(huì)導(dǎo)致索引的indcheckxmin設(shè)置為true:
- 當(dāng)前事務(wù)中表上存在broken HOT chains,即官方文檔中所說(shuō);
- 當(dāng)old_snapshot_threshold被設(shè)置時(shí)。
場(chǎng)景一:broken HOT chains
這種情況,只要在當(dāng)前事務(wù)中表中存在HOT更新的行時(shí)就會(huì)存在。那么什么時(shí)候會(huì)進(jìn)行HOT更新呢??jī)蓚€(gè)前提:
- 新的元組和舊元組必須在同一個(gè)page中;
- 索引字段不能進(jìn)行更新。
既然如此,實(shí)際中常見(jiàn)的兩種情況就是:
- 對(duì)表上最后一個(gè)page進(jìn)行更新;
- 表設(shè)置了fillfactor,即每個(gè)page上有預(yù)留的空閑空間。
例子:
表中插入10條數(shù)據(jù),自然只有1個(gè)page:
bill=# insert into t1 select generate_series(1,10),md5(random()::text); INSERT 0 10
進(jìn)行更新:
bill=# update t1 set info = 'bill' where id = 10; UPDATE 1
查看發(fā)現(xiàn)的確是HOT更新:
關(guān)于t_infomask2字段的解釋這里就不再贅述。

接下來(lái)我們創(chuàng)建索引:
可以發(fā)現(xiàn)indcheckxmin被設(shè)置為true,在當(dāng)前事務(wù)中索引不可用。

經(jīng)過(guò)驗(yàn)證,在index_build階段,判斷到BrokenHotChain,便將indcheckxmin修改為true。

具體的修改代碼如下:
/*此時(shí)indexInfo->ii_BrokenHotChain已被修改為true */
if ((indexInfo->ii_BrokenHotChain || EarlyPruningEnabled(heapRelation)) &&
!isreindex &&
!indexInfo->ii_Concurrent)
{
Oid indexId = RelationGetRelid(indexRelation);
Relation pg_index;
HeapTuple indexTuple;
Form_pg_index indexForm;
pg_index = table_open(IndexRelationId, RowExclusiveLock);
indexTuple = SearchSysCacheCopy1(INDEXRELID,
ObjectIdGetDatum(indexId));
if (!HeapTupleIsValid(indexTuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
indexForm = (Form_pg_index) GETSTRUCT(indexTuple);
/* If it's a new index, indcheckxmin shouldn't be set ... */
Assert(!indexForm->indcheckxmin);
/*將indcheckxmin修改為true */
indexForm->indcheckxmin = true;
CatalogTupleUpdate(pg_index, &indexTuple->t_self, indexTuple);
heap_freetuple(indexTuple);
table_close(pg_index, RowExclusiveLock);
}
同樣我們也可以驗(yàn)證得知,的確是因?yàn)閎rokenhotchains導(dǎo)致的indcheckxmin被設(shè)置為true。
場(chǎng)景二:old_snapshot_threshold
先來(lái)看例子:
最簡(jiǎn)單的場(chǎng)景,完全的一張空表,在事務(wù)中創(chuàng)建索引indcheckxmin就會(huì)被設(shè)置為true,果然索引也是不可用。
bill=# drop table t1;
DROP TABLE
bill=# create table t1(id int,info text);
CREATE TABLE
bill=# begin;
BEGIN
bill=*# create index idx_t1 on t1(id);
CREATE INDEX
bill=*# select indcheckxmin from pg_index where indexrelid = 'idx_t1'::regclass;
indcheckxmin
--------------
t
(1 row)
bill=*# explain select * from t1 where id = 1;
QUERY PLAN
----------------------------------------------------
Seq Scan on t1 (cost=0.00..25.88 rows=6 width=36)
Filter: (id = 1)
(2 rows)
那么為什么old_snapshot_threshold會(huì)產(chǎn)生這樣的影響呢?
經(jīng)過(guò)跟蹤發(fā)現(xiàn),當(dāng)開(kāi)啟該參數(shù)時(shí),在事務(wù)中創(chuàng)建索引的snapshotdata結(jié)構(gòu)如下:
(SnapshotData) $6 = {
snapshot_type = SNAPSHOT_MVCC
xmin = 856
xmax = 856
xip = 0x00007fd55c804fc0
xcnt = 0
subxip = 0x00007fd55ad5d000
subxcnt = 0
suboverflowed = false
takenDuringRecovery = false
copied = false
curcid = 1
speculativeToken = 0
vistest = NULL
active_count = 0
regd_count = 0
ph_node = {
first_child = NULL
next_sibling = NULL
prev_or_parent = NULL
}
whenTaken = 691752041261069
lsn = 208079736
}
而禁用該參數(shù)呢?
(SnapshotData) $7 = {
snapshot_type = SNAPSHOT_MVCC
xmin = 828
xmax = 828
xip = 0x00007fad31704780
xcnt = 0
subxip = 0x00007fad3155d000
subxcnt = 0
suboverflowed = false
takenDuringRecovery = false
copied = false
curcid = 1
speculativeToken = 0
active_count = 0
regd_count = 0
ph_node = {
first_child = NULL
next_sibling = NULL
prev_or_parent = NULL
}
whenTaken = 0
lsn = 0
}
可以看到,區(qū)別在于不使用該參數(shù)時(shí),創(chuàng)建snapshotdata不會(huì)設(shè)置whenTaken和lsn,那么這兩個(gè)參數(shù)是干嘛的呢?
先來(lái)看看snapshotdata的結(jié)構(gòu):
typedef struct SnapshotData
{
SnapshotType snapshot_type; /* type of snapshot */
/*
* The remaining fields are used only for MVCC snapshots, and are normally
* just zeroes in special snapshots. (But xmin and xmax are used
* specially by HeapTupleSatisfiesDirty, and xmin is used specially by
* HeapTupleSatisfiesNonVacuumable.)
*
* An MVCC snapshot can never see the effects of XIDs >= xmax. It can see
* the effects of all older XIDs except those listed in the snapshot. xmin
* is stored as an optimization to avoid needing to search the XID arrays
* for most tuples.
*/
TransactionId xmin; /* all XID < xmin are visible to me */
TransactionId xmax; /* all XID >= xmax are invisible to me */
/*
* For normal MVCC snapshot this contains the all xact IDs that are in
* progress, unless the snapshot was taken during recovery in which case
* it's empty. For historic MVCC snapshots, the meaning is inverted, i.e.
* it contains *committed* transactions between xmin and xmax.
*
* note: all ids in xip[] satisfy xmin <= xip[i] < xmax
*/
TransactionId *xip;
uint32 xcnt; /* # of xact ids in xip[] */
/*
* For non-historic MVCC snapshots, this contains subxact IDs that are in
* progress (and other transactions that are in progress if taken during
* recovery). For historic snapshot it contains *all* xids assigned to the
* replayed transaction, including the toplevel xid.
*
* note: all ids in subxip[] are >= xmin, but we don't bother filtering
* out any that are >= xmax
*/
TransactionId *subxip;
int32 subxcnt; /* # of xact ids in subxip[] */
bool suboverflowed; /* has the subxip array overflowed? */
bool takenDuringRecovery; /* recovery-shaped snapshot? */
bool copied; /* false if it's a static snapshot */
CommandId curcid; /* in my xact, CID < curcid are visible */
/*
* An extra return value for HeapTupleSatisfiesDirty, not used in MVCC
* snapshots.
*/
uint32 speculativeToken;
/*
* For SNAPSHOT_NON_VACUUMABLE (and hopefully more in the future) this is
* used to determine whether row could be vacuumed.
*/
struct GlobalVisState *vistest;
/*
* Book-keeping information, used by the snapshot manager
*/
uint32 active_count; /* refcount on ActiveSnapshot stack */
uint32 regd_count; /* refcount on RegisteredSnapshots */
pairingheap_node ph_node; /* link in the RegisteredSnapshots heap */
TimestampTz whenTaken; /* timestamp when snapshot was taken */
XLogRecPtr lsn; /* position in the WAL stream when taken */
/*
* The transaction completion count at the time GetSnapshotData() built
* this snapshot. Allows to avoid re-computing static snapshots when no
* transactions completed since the last GetSnapshotData().
*/
uint64 snapXactCompletionCount;
} SnapshotData;如上所示,TimestampTz表示snapshot何時(shí)產(chǎn)生的,為什么啟用old_snapshot_threshold時(shí)會(huì)設(shè)置該值呢?
因?yàn)樵撝嫡怯脕?lái)判斷快照是否過(guò)舊的:
/*
* Implement slower/larger portions of TestForOldSnapshot
*
* Smaller/faster portions are put inline, but the entire set of logic is too
* big for that.
*/
void
TestForOldSnapshot_impl(Snapshot snapshot, Relation relation)
{
if (RelationAllowsEarlyPruning(relation)
&& (snapshot)->whenTaken < GetOldSnapshotThresholdTimestamp())
ereport(ERROR,
(errcode(ERRCODE_SNAPSHOT_TOO_OLD),
errmsg("snapshot too old")));
}這樣我們也比較好理解為什么設(shè)置了該參數(shù)時(shí)創(chuàng)建的索引在當(dāng)前事務(wù)中不可用:
因?yàn)槲覀儾辉O(shè)置該參數(shù)時(shí),在事務(wù)中創(chuàng)建索引是可以保證MVCC的一致性,那么索引便是安全可用的。
而使用參數(shù)時(shí),由于TimestampTz被設(shè)置,數(shù)據(jù)庫(kù)會(huì)對(duì)其進(jìn)行判斷該行數(shù)據(jù)是否已經(jīng)過(guò)期,如果過(guò)期了那便會(huì)被清理掉,這樣對(duì)于索引來(lái)說(shuō)便是不安全的,沒(méi)法保證數(shù)據(jù)的一致性,對(duì)于不是hot-safe的索引,自然要將其indcheckxmin設(shè)置為true,防止在事務(wù)中創(chuàng)建索引后數(shù)據(jù)實(shí)際已經(jīng)過(guò)期被刪除的情況。
/*
* At this moment we are sure that there are no transactions with the
* table open for write that don't have this new index in their list of
* indexes. We have waited out all the existing transactions and any new
* transaction will have the new index in its list, but the index is still
* marked as "not-ready-for-inserts". The index is consulted while
* deciding HOT-safety though. This arrangement ensures that no new HOT
* chains can be created where the new tuple and the old tuple in the
* chain have different index keys.
*
* We now take a new snapshot, and build the index using all tuples that
* are visible in this snapshot. We can be sure that any HOT updates to
* these tuples will be compatible with the index, since any updates made
* by transactions that didn't know about the index are now committed or
* rolled back. Thus, each visible tuple is either the end of its
* HOT-chain or the extension of the chain is HOT-safe for this index.
*/
總結(jié)
當(dāng)pg_index的indcheckxmin字段被設(shè)置為true時(shí),直到此pg_index行的xmin低于查詢(xún)的TransactionXmin視界之前,查詢(xún)都不能使用此索引。
而產(chǎn)生這種現(xiàn)象主要有兩種情況:
1. 表上在當(dāng)前事務(wù)中存在broken HOT chains;
2. old_snapshot_threshold被設(shè)置時(shí)。
到此這篇關(guān)于PostgreSQL索引失效會(huì)發(fā)生什么的文章就介紹到這了,更多相關(guān)PostgreSQL索引失效內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家!
相關(guān)文章
PostgreSQL 禁用全表掃描的實(shí)現(xiàn)
這篇文章主要介紹了PostgreSQL 禁用全表掃描的實(shí)現(xiàn)操作,具有很好的參考價(jià)值,希望對(duì)大家有所幫助。一起跟隨小編過(guò)來(lái)看看吧2021-01-01
PostgreSQL數(shù)據(jù)庫(kù)中DISTINCT關(guān)鍵字的四種用法詳解
PostgreSQL 不但高度兼容 SQL 標(biāo)準(zhǔn),同時(shí)還對(duì)很多語(yǔ)法進(jìn)行了擴(kuò)展,可以用于實(shí)現(xiàn)一些特殊的功能,今天我們就來(lái)介紹一下 PostgreSQL 數(shù)據(jù)庫(kù)中 DISTINCT 關(guān)鍵字的 4 種不同用法,需要的朋友可以參考下2024-04-04
postgresql高級(jí)應(yīng)用之行轉(zhuǎn)列&匯總求和的實(shí)現(xiàn)思路
這篇文章主要介紹了postgresql高級(jí)應(yīng)用之行轉(zhuǎn)列&匯總求和的實(shí)現(xiàn)思路,本文給大家介紹的非常詳細(xì),對(duì)大家的學(xué)習(xí)或工作具有一定的參考借鑒價(jià)值,需要的朋友可以參考下2021-05-05
PostgreSQL查詢(xún)修改max_connections(最大連接數(shù))及其它配置詳解
postgresql數(shù)據(jù)庫(kù)最大連接數(shù)是系統(tǒng)允許的最大連接數(shù),當(dāng)數(shù)據(jù)庫(kù)并發(fā)用戶(hù)超過(guò)該連接數(shù)后,會(huì)導(dǎo)致新連接無(wú)法建立或者連接超時(shí),這篇文章主要給大家介紹了關(guān)于PostgreSQL查詢(xún)修改max_connections(最大連接數(shù))及其它配置的相關(guān)資料,需要的朋友可以參考下2024-01-01
PostgreSQL13基于流復(fù)制搭建后備服務(wù)器的方法
這篇文章主要介紹了PostgreSQL13基于流復(fù)制搭建后備服務(wù)器,后備服務(wù)器作為主服務(wù)器的數(shù)據(jù)備份,可以保障數(shù)據(jù)不丟,而且在主服務(wù)器發(fā)生故障后可以提升為主服務(wù)器繼續(xù)提供服務(wù)。需要的朋友可以參考下2022-01-01
PostgreSQL流復(fù)制參數(shù)max_wal_senders的用法說(shuō)明
這篇文章主要介紹了PostgreSQL流復(fù)制參數(shù)max_wal_senders的用法說(shuō)明,具有很好的參考價(jià)值,希望對(duì)大家有所幫助。一起跟隨小編過(guò)來(lái)看看吧2020-12-12
postgreSQL中的row_number() 與distinct用法說(shuō)明
這篇文章主要介紹了postgreSQL中的row_number() 與distinct用法說(shuō)明,具有很好的參考價(jià)值,希望對(duì)大家有所幫助。一起跟隨小編過(guò)來(lái)看看吧2021-01-01
PostgreSQL創(chuàng)建新用戶(hù)所遇見(jiàn)的權(quán)限問(wèn)題以及解決辦法
這篇文章主要給大家介紹了關(guān)于PostgreSQL創(chuàng)建新用戶(hù)所遇見(jiàn)的權(quán)限問(wèn)題以及解決辦法, 在PostgreSQL中創(chuàng)建一個(gè)新用戶(hù)非常簡(jiǎn)單,但可能會(huì)遇到權(quán)限問(wèn)題,需要的朋友可以參考下2023-09-09

