Extract log management out of TransactionManager

For #6509 (closed)

The TransactionManager should primarily concentrate on concurrency control. However, it has taken on numerous additional tasks that fall outside its main responsibilities, such as housekeeping and log management. Log management, which used to be relatively straightforward, is now becoming increasingly complicated. This is particularly true if Raft integration enters the scene.

To simplify and improve the Raft integration process, it is beneficial to separate log management from the TransactionManager. The log should be accessed through a distinct type within the TransactionManager. This approach allows us to abstract the Raft specifics, ensuring the TransactionManager remains unaware of the various log implementations and the underlying consensus algorithm. Currently, there are several complexities regarding Raft, such as rolling back logs.

This MR refactors the log management within the TransactionManager by extracting it into a separate LogManager component. The LogManager is responsible for handling the write-ahead log (WAL), including committing log entries, pruning obsoleted entries, accounting consumer positions, and managing in-memory entry references.

Due to the complexity of TransactionManager, the size of this MR is fairly big. A large portion of the changes is for adding more tests. As it does not make sense to draft a massive commit, I follow a subtle approach:

• First, create a LogManager object inside storagemgr/partition package and extract codes there. All the tests are kept intact.
• The following commits strengthen the boundary between LogManager and TransactionManager. Along the way, unit tests of LogManager are added. The counterparts in the TransactionManager are simplified to assert the outcomes instead of internal states.
• Shift the interactions between Log Consumer and Transaction Manager to Log Manager.
• Finally, move the new LogManager to storage/wal package.