[go: up one dir, main page]

Skip to content

fetch/receive: use batched reference updates

The git-fetch(1) and git-receive-pack(1) commands update references as part of the flow. Each reference update is treated as a single entity and a transaction is created for each update.

This can be really slow, specifically in reference backends where there are optimizations which ensure a single transaction with 'N' reference update perform much faster than 'N' individual transactions. Also having 'N' individual transactions has buildup and teardown costs. These costs add up in repositories with a large number of references.

Also specifically in the reftable backend, 'N' individual transactions would also trigger auto-compaction for every transaction.

The reasoning for using individual transactions is because we want to allow partial updates of references in these commands. Using a single transaction would be an all-or-nothing scenario.

Recently we introduced an in-between solution called batched reference updates in 23fc8e4f (refs: implement batch reference update support, 2025-04-08). This allows us to batch a set of reference updates, where individual updates can pass/fail without affecting the batch.

This patch series, modifies both 'git-fetch(1)' and 'git-receive-pack(1)' to use this mechanism. With this, we see a significant performance boost:

+---------------------+---------------+------------------+

| | files backend | reftable backend |

+---------------------+---------------+------------------+

| git-fetch(1) | 1.25x | 22x |

| git-receive-pack(1) | 1.21x | 18x |

+---------------------+---------------+------------------+

The first and third patch handle the changes for 'git-fetch(1)' and 'git-receive-pack(1)' respectively. The second patch fixes a small memory leak I encountered while working on this series.

This is based on top of master: 7a1d2bd0 (Merge branch 'master' of https://github.com/j6t/gitk, 2025-05-09). There were no conflicts observed with next or seen.

Edited by Karthik Nayak

Merge request reports

Loading