[go: up one dir, main page]
Skip to content

reftable: Small fixes/changes to the opportunistic migrator

Karthik Nayakrequested to merge
6738-optimize-the-opportunistic-migrator-with-learnings-from-staging into master

There are a few learnings from enabling the opportunistic reftable migrator in staging, this MR makes changes to accommodate the following:

  1. Don't log an error message when the repository doesn't exist. Since the migrator registers a migration for all read RPCs, there are often times when a migration is registered when a repository is being created. This is expected, so let's not log an error message here. Reducing the log surface, improves readability and ability to capture actual errors.
  2. Currently, we only register a migration for all incoming read requests and trigger a cancellation to cancel any ongoing migrations during a write request. This means that write heavy repositories often may not be migrated. So let's also add a migration trigger towards the end of a write request. This ensures that as a write request is being completed, we can start a migration.
  3. Fix a bug where the cancellation function was not correctly assigned when there was already a previous state.

I also thought about writing the state to KV, this would help to ensure we don't retry migration for already migrated repositories. But this is probably okay, since

  1. The migration code runs as a goroutine and doesn't affect RPC latency
  2. It's much simpler without creation of a new transaction for reading/writing to the KV

Closes #6738 (closed)

Edited by Karthik Nayak

Merge request reports