raft: Implement persistent routing table for Raft node membership
This MR introduced a static in-memory routing table. To support production use-case we need a dynamic node membership management with persistent storage.
The flow as discussed on Slack:
- The routing table is shared across all partitions of storage and is persisted into the K/V database. We have a hidden location for storage-wide information called partition 1.
- When a partition is bootstrapped, first leader inserts itself, including address to the routing table.
- Whenever a replica joins/leaves, the leader updates the routing table record accordingly.
- All replicas must have an identical record in the same term, even if gossiping is not implemented. We can rely on conf change entry. This entry allows free-form context.
- Each record uses a combination of (Term, Index) to invalidate a stale record.