Skip to content

Fix rowid datum memory leak#40

Open
Serge-sudo wants to merge 81 commits intoorioledb:patches17from
Serge-sudo:fixed-rowid-datum-leak
Open

Fix rowid datum memory leak#40
Serge-sudo wants to merge 81 commits intoorioledb:patches17from
Serge-sudo:fixed-rowid-datum-leak

Conversation

@Serge-sudo
Copy link
Contributor

Add cleanup for rowid datum allocated on each insert. The datum was never freed, causing a significant memory leak. During bulk inserts of large row counts this led to excessive RAM usage and could trigger the OOM killer.

This fixes #39

akorotkov and others added 30 commits September 4, 2025 13:23
Discussion: https://postgr.es/m/CAPpHfdua-YFw3XTprfutzGp28xXLigFtzNbuFY8yPhqeq6X5kg%40mail.gmail.com
Reviewed-by: Aleksander Alekseev, Pavel Borisov, Vignesh C, Mason Sharp
Reviewed-by: Andres Freund, Chris Travers
Snapshot have two pairing heap nodes: for data and system undos.
 * Added SearchCatCacheInternal_hook, SearchCatCacheList_hook
 * Added SysCacheGetAttr_hook
IsFatalError()
have_backup_in_progress()
SnapBuildNextPhaseAt()
DoLocalLockExist()
Outline-atomics is a gcc compilation flag that enables runtime detection
of CPU support for atomic instructions.
Performance on CPUs that do support atomic instructions is improved,
while compatibility and performance on CPUs without atomic instructions
is not hurt.

Discussion: https://postgr.es/m/flat/099F69EE-51D3-4214-934A-1F28C0A1A7A7%40amazon.com
Author: Tsahi Zidenberg
They are allowed to stay during shutdown checkpointing and help checkpointer
do its work.
To use curl during shared_preload_libraries initialization.
- added option --extension for pg_rewind
- extracted SimpleXLogRead from extractPageMap for generic wal iteration in pg_rewind
akorotkov and others added 29 commits September 4, 2025 15:39
Also, get xmin out of RetainUndoLocationPHNode.
Github Actions deprecated usage of Ubuntu 20.04
… cluster by providing invalid databaseId

For bgworkers CountOtherDBBackends doesn't add it to nbackends, but check if they are prepared transactions
to add to nprepared
This is required to have rewind functionality on heap tables
Fixes "rules" regress test for running with orioledb
…backward

OrioleDB table AM doesn't support BACKWARD SCAN in all cases. A user
needs to declare a cursor with SCROLL option.
…ation methods.

This allows to output size metrics for relations and indexes provided by OrioleDB extension using existing PG functions.

If extension function doesn't support the requested method it should output negative value and thus asking to fall
back to using PG internal calculation e.g.

- Orioledb relation_size outputs -1 for bridged indexes to fallback to counting them as PG indexes
- PG table_block_relation_size outputs -1 for any method except the only DEFAULT_SIZE that it supports.

This API relies on table AM extensibility and doesn't use index AM extensibility yet, which could look more logical,
but also more complicated.
Useful when extension performs some async operations in the WAL record
redo function.
 * RunningTransactionsExtension structure as a part of xl_running_xacts and
   RunningTransactionsData to be filled by extensions.
 * getRunningTransactionsExtension hook to fill RunningTransactionsExtension.
 * Add nextXid field to CSNSnapshotData.
 * SnapBuildGetCSNSnaphot() function to modify CSNSnapshotData in the
   SnapBuild.
 * waitSnapshotHook hook to wait for transactions inside the snapshot
   builder.
 * Use patchset number is there is an exact match to the tag
 * Use commit hash otherwise
Transactions need to be marked as RBTXN_DISTR_SKIP_CLEANUP to be skipped in
ReorderBufferAbortOld().
Add cleanup for rowid datum allocated on each insert. The datum was never freed,
causing a significant memory leak. During bulk inserts of large row counts this
led to excessive RAM usage and could trigger the OOM killer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants