CN118885537B

CN118885537B - First data synchronization method, system, device and storage medium without using snapshot

Info

Publication number: CN118885537B
Application number: CN202411320328.1A
Authority: CN
Inventors: 黄传波; 洪杰鑫; 王东; 杨亮; 杨咏雪; 周钰寒
Original assignee: Chengdu Vinchin Science And Technology Co
Current assignee: Chengdu Vinchin Science And Technology Co
Priority date: 2024-09-23
Filing date: 2024-09-23
Publication date: 2024-12-03
Anticipated expiration: 2044-09-23
Also published as: CN118885537A

Abstract

The invention relates to a first data synchronization method, a system, equipment and a storage medium without using a snapshot, belonging to the technical field of data disaster recovery backup. The method comprises the steps of creating a first synchronization table of a disk, setting a first synchronization thread, an IO blocking thread and an IO issuing thread in an IO filtering driver, intercepting a write request by the IO blocking thread when the first synchronization thread is executed and a disk write request is generated by a system, reading a target data block from the disk, updating the progress state of the first synchronization table, checking the first synchronization table by the IO issuing thread, and releasing the write request once the synchronization of the data block is confirmed. The system comprises a first synchronous table creation module, a thread setting module, a write IO blocking request module, a write target data sending module, and a write IO inquiring and issuing module. The method and the device realize the first synchronization under the condition of not using the snapshot, ensure the data consistency during the first synchronization and reduce the occupied storage space of the production end.

Description

Method, system, device and storage medium for first-time data synchronization without using snapshot

Technical Field

The invention belongs to the technical field of data disaster recovery backup, and relates to a first data synchronization method, a system, equipment and a storage medium without using snapshot.

Background

Real-time synchronization refers to the process by which data is copied to another location or system immediately after creation or modification. This synchronization may be in the same network, or may be across networks or across geographic locations. The purpose of real-time synchronization is to ensure consistency and availability of data, and to enable quick recovery even when the original data source fails.

The first data synchronization is the first step in the real-time synchronization scheme, and aims to completely synchronize the initial data of the production end into the service end. In the process of first data synchronization, the production end is still continuously generating IO, the data can be continuously updated, in order to ensure the consistency of the synchronous data, a snapshot technology is generally adopted to carry out the first data synchronization, however, snapshot files tend to occupy a certain storage space of the production end, and meanwhile, the snapshot function is not supported by the disk types of all scenes. In the prior art, the first data synchronization technology which does not use snapshot under real-time synchronization is almost not available.

Therefore, how to help the client complete the first data synchronization without the snapshot function and save the storage space of the production end is a technical problem which needs to be solved currently.

Disclosure of Invention

The invention aims to solve the technical problems in the background art and provides a method, a system, equipment and a storage medium for first-time data synchronization without using snapshot.

The technical scheme for solving the technical problems is as follows:

In a first aspect, there is provided a method of first data synchronization without using snapshots, the method comprising the steps of:

Creating a first synchronization table of the disk in the memory, wherein elements of the first synchronization table correspond to data blocks of the disk one by one, and the elements record the first synchronization progress state of the data blocks;

setting a first synchronous thread, an IO blocking thread and an IO issuing thread in an IO filtering drive;

when the first synchronous thread is executing and generates a write IO request to a disk, judging whether the data of the write target data block is synchronous, if yes, sending the write IO request by an IO issuing line Cheng Xia, otherwise, obtaining the write IO request by an IO blocking thread, and preventing the write IO request from being issued;

According to the IO writing request, the IO blocking thread reads the data of the writing target data block from the disk, then sends the read data, and in the executing process, immediately updates the first synchronous progress state of the corresponding element on the first synchronous table and transfers the IO writing request to the IO issuing thread;

And the IO issuing line polls the first synchronization table, and if the first synchronization of the write target data block is finished, the IO issuing line sends and issues a write IO request.

In a second aspect, there is provided a first data synchronization system that does not use snapshots, the system comprising:

The first synchronization table creating module is used for creating a first synchronization table of the disk in the memory, wherein elements of the first synchronization table correspond to data blocks of the disk one by one, and the elements record the first synchronization progress state of the data blocks;

the thread setting module is used for setting a first synchronous thread, an IO blocking thread and an IO issuing thread in the IO filtering drive;

the block write IO request module is used for judging whether the write target data block data is synchronous when the first synchronous thread is executing and generates a write IO request to a disk, if yes, an IO issuing line Cheng Xia issues the write IO request, if not, the IO block thread obtains the write IO request and prevents the write IO request from being issued;

the IO blocking thread reads the data of the write target data block from the disk according to the write IO request, and then transmits the read data, and in the executing process, the first synchronous progress state of the corresponding element on the first synchronous table is updated immediately and the write IO request is transferred to the IO issuing thread;

and the polling and issuing write IO request module is used for polling the first synchronization table by the IO issuing thread and sending and issuing a write IO request if the first synchronization of the write target data block is completed.

In a third aspect, there is provided an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing a method of first data synchronization without using snapshots as defined in any one of the preceding claims when the program is executed.

In a fourth aspect, a computer readable storage medium is provided having stored thereon a computer program which, when executed by a processor, implements a first data synchronization method as described above without using snapshots.

The beneficial effects of the invention are as follows:

Once the primary synchronization task is started, the primary synchronization thread sequentially reads each data block on the disk, however, because primary synchronization is a time-consuming process, a new IO request is generated by a production end in the period, and data on the disk is modified, at the moment, the new IO writing request is processed in advance by introducing the IO blocking thread and the IO issuing thread, and the processed IO writing request is recorded and issued on time in the primary synchronization table, so that primary synchronization is realized, data consistency in the primary synchronization period is ensured, and storage space of the production end is reduced.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of a first data synchronization method without using a snapshot in an embodiment of the present invention.

Fig. 2 is a working schematic diagram of an IO blocking thread and an IO issuing thread in an embodiment of the present invention.

Fig. 3 is a schematic diagram of a first data synchronization system without using a snapshot in an embodiment of the present invention.

FIG. 4 is a schematic diagram of a module for sending write target data according to an embodiment of the present invention.

Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

In the drawings, the list of components represented by the various numbers is as follows:

310. the method comprises the steps of creating a first synchronous table module, setting a thread module, blocking a write IO request module, sending a write target data module, requesting and issuing the write IO request module, and obtaining a read IO request module, wherein the first synchronous table module is created 320, setting a thread module, the write target data module is blocked 330, the write target data module is sent 340, the write IO request module is sent 350, the read IO request module is acquired 360, the read IO request issuing module is judged 370, the position unit of a write target data block is calculated 3401, the write target data unit is read 3402, the write target data unit is sent 3403, the write target data unit is sent 410, the processor is sent 420, the communication interface is sent 430, the memory is stored 440, and the communication bus is used.

Detailed Description

The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

A Snapshot (snap shot) is a fully available copy of specified data, the copy resulting from the copy containing an image of the data at some point in time. The snapshot can quickly capture and save the state of the data at a specific time point without interrupting the operation of the system, and generate consistent data copies of the data at the time point. The existing real-time synchronization scheme is that a snapshot is created on a disk at a production end when the first synchronization is started, snapshot data are read and written into a server to finish the first synchronization, and because the first-time synchronization data are all data when a task is started and are not updated, consistency of the synchronization data is guaranteed, however, the snapshot is firstly stored at the production end and then sent to the server, the snapshot occupies a storage space of the production end, the more the snapshot occupies along with changes of a subsequent IO request, and further the performance of the production end is affected. Therefore, how to help the customer complete the first data synchronization without the snapshot function and save the storage space of the production end is a technical problem which needs to be solved currently

In view of the above problems, an embodiment of the present invention provides a first data synchronization method that does not use snapshots. Fig. 1 is a schematic flow chart of a first data synchronization method without using a snapshot according to an embodiment of the present invention, where, as shown in fig. 1, the method includes:

step S101, a first synchronization table of the disk is created in the memory, elements of the first synchronization table correspond to data blocks of the disk one by one, and the elements record the first synchronization progress state of the data blocks.

As shown in table 1, the present embodiment divides the disk into a plurality of data blocks with equal sizes, such as Block 0, block 1, block 2, block 3, block N, and the first synchronization and the subsequent real-time synchronization all operate with the data Block as the minimum unit. During the first synchronization, a first synchronization table is maintained in the memory, as shown in table 2, and the table is used for identifying the first synchronization progress state of each data block on the disk, and the number of elements is equal to the number of data blocks and corresponds to the number of data blocks one by one. Generally, the first synchronization progress state is set such that the first synchronization is not started, the first synchronization is being performed, and the first synchronization is completed. However, by introducing some extra states, various conditions in the first synchronization process can be monitored and managed more comprehensively, and the reliability and flexibility of the data synchronization can be improved, for example, the first synchronization process state can be that if the first synchronization operation is not completed beyond a preset time, the state of the data block can be marked as timeout, which can trigger a retry mechanism or error handling, and the synchronization operation of the data block is completed, but some confirmation steps are needed. The embodiment of the present invention is not particularly limited thereto.

TABLE 1 schematic construction of magnetic disk

Block 0	Block 1	Block 2	Block 3
				....	....	....	....
....	....	....	Block N

TABLE 2 structural schematic of first synchronization Table

2	2	2	1
				0	....	....	0
1	....	0	0

In addition, as shown in table 2, the value of each element is set to 0, 1 or 2, where 0 represents that the first synchronization of the corresponding data block is not started, 1 represents that the first synchronization of the corresponding data block is being performed, and 2 represents that the first synchronization of the corresponding data block is completed. However, the first sync status may also be identified by english, such as X, Y and Z. The embodiment of the present invention is not particularly limited thereto.

The first synchronization table in the embodiment is arranged in the memory, and has the advantages of (1) improving the performance, namely, the access speed of the memory is far higher than that of the disk, so that the access delay can be obviously reduced, the inquiry and update speed of the data block state can be accelerated, the overall synchronization efficiency can be improved, and (2) reducing the disk IO, avoiding frequent disk read-write operation, reducing the pressure on the disk IO, and being beneficial to reducing the abrasion of the disk and improving the response speed of the system.

Step S102, a first synchronous thread, an IO blocking thread and an IO issuing thread are set in the IO filtering drive.

The IO filter driver is a software component between an application program and a storage system, and has the main functions of monitoring and filtering IO requests and the like. In this embodiment, the IO filter driver helps to coordinate data consistency during the first synchronization process.

And the first synchronization thread is responsible for executing the first synchronization task, sequentially reading the data blocks on the disk, and synchronizing the data blocks to the server. The primary goal of the first synchronization thread is to ensure that all data blocks are copied entirely once.

IO blocking threads-when new IO requests (write operations) arrive, this thread is responsible for suspending or delaying these requests temporarily to avoid modifying the data blocks being synchronized during the first synchronization, thereby ensuring data consistency, while also being responsible for handling write target data block data ahead of time.

IO issuing thread after the first synchronization thread or IO blocking thread completes synchronization of specific data block, the thread is responsible for releasing the IO request suspended before and allowing them to continue executing.

It should be noted that the execution mode of the three threads depends on the CPU architecture of the system and the scheduling policy of the operating system. The operating system will determine how to schedule the threads based on current system load, number of CPU cores, thread priority, etc. For example, on a single core processor, all threads are concurrently executing, the operating system simulates multitasking by time slicing, even though they appear to be running simultaneously, in effect on a fast switch execution, and on a multi-core processor, threads may execute in parallel. Each core can be allocated one or more threads to realize real simultaneous operation, the system load is higher, and an operating system can switch between the threads more frequently so as to ensure that all threads can be processed. The embodiment of the present invention is not particularly limited thereto.

Step S103, when the first synchronous thread is executing and generates a write IO request to the disk, judging whether the data of the write target data block is synchronous, if yes, sending the write IO request by the IO issuing line Cheng Xia, and if not, obtaining the write IO request by the IO blocking thread and preventing the write IO request from issuing.

Step S104, according to the IO writing request, the IO blocking thread reads the data of the writing target data block from the disk, and then sends the read data, and in the executing process, the first synchronous progress state of the corresponding element on the first synchronous table is updated immediately, and the IO writing request is transferred to the IO issuing thread.

When the first synchronization thread is executing, it is responsible for reading the data blocks on disk and synchronizing them to the server. This process needs to ensure consistency of the data, i.e. the synchronized data should be in the data state at the beginning of the first synchronization. If the data on the disk is modified during the first synchronization, the synchronized data may not be consistent with the original data and may not meet the requirements.

The IO blocking thread is introduced, so that the write IO request to the disk can be temporarily prevented during the first synchronization period, the data of the write target data block is synchronized in advance, once the IO blocking thread completes the data synchronization to the write target data block, the IO blocking thread transfers the write IO request to the IO issuing thread to be released, and the IO issuing thread is allowed to continuously execute.

Meanwhile, by means of the first synchronization table, the first synchronization thread, the IO blocking thread and the IO issuing thread can timely know the first synchronization progress state of the target data block, so that mutual coordination work among the first synchronization thread, the IO blocking thread and the IO issuing thread is ensured, and synchronous data cannot be repeated.

In step S105, the IO issuing process polls the first synchronization table, and if the first synchronization of the write target data block is completed, sends and issues a write IO request.

It will be appreciated that polling is a method of proactive checking in which the IO outgoing line periodically checks the first synchronization table to determine the synchronization status of the data block. By polling it is possible to determine when the first synchronization of a data block has been completed, so that a previously blocked write IO request can be issued securely. Polling is typically performed in a loop, and the IO issue line Cheng Meige looks up the first synchronization table for a certain period of time to check if the status of any data block has been updated to "done" or "2".

It will also be appreciated that the frequency of polling may be adjusted according to the performance requirements and load conditions of the system. Higher polling frequencies may respond faster to changes in synchronization status, but may increase CPU usage. The polling interval is the time between two polls. Shorter intervals may process write IO requests faster, but may increase the load on the system, longer intervals may reduce the system load, but the response time to process requests may become longer. The embodiment of the present invention is not particularly limited thereto.

However, in order to avoid having to directly access the first synchronization table every poll, a cache may be used to store the most recent state change, which may reduce the number of accesses to the first synchronization table and increase efficiency.

Compared with the prior art, the method and the device have the advantages that the first synchronization is realized under the condition that the snapshot is not used, the data consistency during the first synchronization is ensured, and the occupied storage space of the production end is reduced.

Based on the above embodiment, in the method, the step S103 further includes:

step S1031, receiving a first synchronous task starting instruction;

Step S1032, according to the starting instruction, the first synchronization thread sequentially reads data of the data blocks from the disk, then writes the read data into the sending queue, and immediately updates the first synchronization progress state of the corresponding elements on the first synchronization table in the execution process;

step S1033, when generating a write IO request for modifying a disk, judging whether the data of the write target data block is synchronous, if yes, sending the write IO request by an IO issuing line Cheng Xia, if not, capturing the write IO request by an IO filtering driver and transferring the write IO request to an IO blocking thread, and executing step S1034;

In step S1034, the IO blocking thread blocks the write IO request from being issued.

Based on the above embodiment, in the method, the step S104 further includes:

step S1041, the IO blocking thread acquires an offset field and a data length field of a write IO request, and calculates a write target data block start block number and a write target data block end block number;

step S1042, according to the settlement result, the IO blocking thread starts to read the data of the writing target data block from the disk, then instantly adjusts the first synchronization progress state of the corresponding element on the first synchronization table to be in the first synchronization, and transfers the writing IO request to the IO issuing thread;

In step S1043, when the reading of the data of the write target data block by the IO blocking thread is completed, the IO blocking thread inserts the read data package into the sending queue, and then immediately adjusts the first synchronization progress status of the corresponding element on the first synchronization table to be the first synchronization completion.

It should be noted that each IO request structure has two important fields, an offset field (offset) and a data length field (length), and therefore, in order to more accurately read a target data block, it is preferable to calculate a target start block number and a target end block number, specifically, in such a way that the start block number=offset/block_size, and the end block number= (offset+length)/block_size, and the calculation results are all rounded down, and the block_size is the data block size.

As shown in fig. 2, the IO blocking thread blocks the write IO request, and after calculating the target block number of the write IO request, the IO blocking thread sequentially initiates a read request for data from the start block number to the end block number. After the data of the write IO target data Block (Block K) is read, the corresponding element in the first synchronization table is required to be set from 0 to 1, and the blocked write IO request is transferred to the IO issuing thread.

As also shown in fig. 2, the IO blocking thread encapsulates the read data into a first synchronization packet, inserts the first synchronization packet into the transmit queue, and sets the corresponding element in the first synchronization table to 2.

As further shown in fig. 2, after receiving the write IO request, the IO issuing thread polls the first synchronization table to check whether the target data block of the current write IO request completes the first synchronization (i.e. whether the corresponding element value on the table is 2), if the target block completes the first synchronization, copies the current write IO request into a real-time synchronization packet, inserts the real-time synchronization packet into the transmission queue, and issues the current write IO request.

It should be noted that, in this embodiment, the transmission mode preferably uses a pre-existing transmission queue to send the service end, and this mode uses the transmission queue as a buffer area to temporarily store real-time synchronous data packets, so as to cope with network fluctuation or change of processing capacity of the backup service end, ensure that the data packets are transmitted according to a specific sequence, keep consistency and integrity of data, and avoid overload by controlling the size and transmission rate of the transmission queue. Other sending modes can also adopt direct sending, sending through memory cache and sending through callback mechanism, and the embodiment of the invention is not limited in particular.

Different from the method in the previous embodiment, the method in the embodiment can more accurately identify the position of the target data block, orderly arrange data transmission, manage the load of the server and improve the synchronization efficiency.

Preferably, the step S1043 further includes:

Step S10431, setting a data sending thread in the IO filtering drive;

Step S10432, when the IO blocking thread finishes reading the data of the writing target data block, the IO blocking thread stores the read data into a local cache queue, and then the first synchronization progress state of the corresponding element on the first synchronization table is adjusted to be the first synchronization finish in real time;

step S10433, receiving a sending instruction, caching the encapsulated data locally by the data sending thread, and inserting the encapsulated data into a sending queue.

It should be noted that, for the case that the data of the write target data block is judged to have completed synchronization, the data can be transferred to the IO issuing process for issuing (transferred to a special IO thread, the burden of a main thread can be reduced, the responsiveness and throughput of the system are improved, disorder is avoided), a write cache can be copied, and then transferred to the IO issuing process for issuing (through a cache mechanism, the write request can be temporarily stored, the influence on the main thread is reduced, and the write operation can be processed in batches through the IO thread, so that the efficiency is improved). The embodiment of the present invention is not particularly limited thereto.

In this embodiment, considering that the bandwidth of the transmission network may fluctuate, the write target data block data is first stored in the local buffer queue, and then the encapsulated data is inserted into the transmission queue by the data transmission thread according to the transmission instruction, so that the influence of the bandwidth deficiency of the network on the service system can be reduced.

Based on the above embodiment, the method includes:

step S201, a first synchronization table of a disk is created in a memory, elements of the first synchronization table correspond to data blocks of the disk one by one, and the elements record the first synchronization progress state of the data blocks;

Step S202, setting a first synchronous thread, an IO blocking thread and an IO issuing thread in an IO filtering driver;

step S203, when the first synchronous thread is executing and generates a write IO request to the disk, judging whether the data of the write target data block is synchronous, if yes, transmitting the write IO request by an IO issuing line Cheng Xia, otherwise, acquiring the write IO request by an IO blocking thread, and preventing the write IO request from being issued;

Step S204, according to the IO writing request, the IO blocking thread reads the data of the writing target data block from the disk, and then sends the read data, and in the executing process, the first synchronous progress state of the corresponding element on the first synchronous table is updated immediately, and the IO writing request is transferred to the IO issuing thread;

step S205, the IO issuing line polls the first synchronization table, and if the first synchronization of the writing target data block is finished, the IO issuing line sends and issues a writing IO request;

Step S206, when a read IO request for reading a disk is generated, the first synchronous thread acquires the read IO request;

Step S207, according to the read IO request, the first synchronization line queries the first synchronization table, judges the current first synchronization proceeding state of the read target data block, if the first synchronization is proceeding, waits for the first synchronization to finish, issues the read IO request, and executes the read operation, if the first synchronization is not started or the first synchronization is finished, directly issues the read IO request, and executes the read operation.

Unlike the previous embodiment method, considering that if the target data block is executing synchronization, directly issuing the read IO request may cause problems of consistency and integrity of data during the read operation, the present embodiment checks the synchronization state of the target data block before the read operation, if synchronization is in progress, waits for the synchronization to complete and then executes the read operation, and the processing manner of the read IO is simpler and more intuitive, easy to implement and maintain, and reduces additional synchronization logic, such as not introducing complicated lock mechanism or version control, and can provide reasonable performance without sacrificing the data consistency.

As shown in fig. 3, a first data synchronization system that does not use snapshots, the system further comprising:

A first synchronization table creating module 310, configured to create a first synchronization table of the disk in the memory, where elements of the first synchronization table correspond to data blocks of the disk one by one, and the elements record a first synchronization progress state of the data blocks;

a thread setting module 320, configured to set a first synchronization thread, an IO blocking thread, and an IO issuing thread in the IO filtering driver;

The block write IO request module 330 is configured to determine whether the write target data block data is synchronized when the first synchronization thread is executing and a write IO request to the disk is generated, if yes, send the write IO request by the IO issuing line Cheng Xia;

The write target data sending module 340 is configured to read write target data block data from the disk by the IO blocking thread according to the write IO request, send the read data, update the first synchronization progress state of the corresponding element on the first synchronization table in real time in the execution process, and transfer the write IO request to the IO issuing thread;

the polling and issuing write IO request module 350 is configured to poll the first synchronization table by the IO issuing thread, and send and issue a write IO request if the first synchronization of the write target data block is completed;

the read IO request obtaining module 360 is configured to obtain, when a read IO request for reading a disk is generated, the read IO request by the first synchronization thread;

The read IO request issuing module 370 is configured to query the first synchronization table according to the read IO request, determine a current first synchronization progress status of the read target data block, if the first synchronization is being performed, issue the read IO request after the first synchronization is completed, execute the read operation, and if the first synchronization is not started or the first synchronization is completed, directly issue the read IO request, and execute the read operation.

Based on the above embodiment, as shown in fig. 4, the sending write target data module 340 further includes:

a write target data block position calculating unit 3401, configured to obtain an offset field and a data length field of a write IO request by an IO blocking thread, and calculate a write target data block start block number and a write target data block end block number;

the read-write target data unit 3402 is used for starting to read the data of the write target data block from the disk by the IO blocking thread according to the settlement result, then immediately adjusting the first synchronization progress state of the corresponding element on the first synchronization table to be in progress for first synchronization, and transferring the write IO request to the IO issuing thread;

And the sending write target data unit 3403 is used for when the IO blocking thread finishes reading the write target data block data, the IO blocking thread inserts the read data package into a sending queue, and then the first synchronization progress state of the corresponding element on the first synchronization table is immediately adjusted to be the first synchronization finish.

Compared with the prior art, the first synchronization thread, the IO blocking thread and the IO issuing thread can be coordinated with each other under the help of the first synchronization table even if a new IO writing request is generated, so that the consistency of synchronous data and the stability of a system are ensured, and the copy of the whole disk is not required to be stored.

Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, as shown in fig. 5, the electronic device may include a processor 410, a communication interface 420, a memory 430, and a communication bus 440, where the processor 410, the communication interface 420, and the memory 430 complete communication with each other through the communication bus 440. The processor 410 may call a logic instruction in the memory 430 to execute a method of creating a first synchronization table of a disk in the memory, wherein elements of the first synchronization table are in one-to-one correspondence with data blocks of the disk, the elements record a first synchronization progress state of the data blocks, setting a first synchronization thread, an IO blocking thread and an IO issuing thread in an IO filtering driver, judging whether data of a write target data block is synchronous or not when the first synchronization thread is executing and generates a write IO request to the disk, if yes, issuing the write IO request by an IO issuing line Cheng Xia, if not, obtaining the write IO request by the IO blocking thread and preventing the write IO request from issuing, reading the write target data block data from the disk according to the write IO request, then sending the read data, updating the first synchronization progress state of the corresponding elements on the first synchronization table in real time and transferring the write IO request to the IO issuing thread in an execution process, and sending and polling the first synchronization table by the IO issuing thread if the write target data block is inquired about the first synchronization is completed.

Further, the logic instructions in the memory 430 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. The storage medium includes a U disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, an optical disk, or other various media capable of storing program codes.

In this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements the API under-framework multi-version control method described in the above embodiment.

The computer storage media of embodiments of the invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the computer-readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

The computer program code for carrying out operations of the present invention may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++, ruby, go and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.

Claims

1. A method of first data synchronization without using snapshots, the method comprising the steps of:

2. The method for first-time data synchronization without using a snapshot according to claim 1, wherein when the first-time synchronization thread is executing and a write IO request to a disk is generated, determining whether the step of synchronizing the write target data block data is completed, further comprises:

receiving a first synchronous task starting instruction;

According to the starting instruction, the first synchronization thread sequentially reads data block data from the disk, and then transmits the read data, and the first synchronization progress state of the corresponding element on the first synchronization table is updated in real time in the execution process;

When a write IO request for modifying a disk is generated, judging whether the data of a write target data block is synchronous or not, if yes, sending the write IO request by an IO issuing line Cheng Xia, if not, capturing the write IO request by an IO filtering driver, transferring the write IO request to an IO blocking thread, and executing the next step;

The IO blocking thread prevents write IO requests from being issued.

3. The method for synchronizing first data without using snapshot according to claim 1, wherein the step of reading the data of the write target data block from the disk by the IO blocking thread according to the write IO request, retransmitting the read data, updating the first synchronization progress status of the corresponding element on the first synchronization table in real time during execution, and transferring the write IO request to the IO issuing thread further comprises:

the IO blocking thread acquires an offset field and a data length field of the write IO request, and calculates a start block number of a write target data block and an end block number of the write target data block;

according to the calculation result, the IO blocking thread starts to read the data of the write target data block from the disk, then the first synchronization progress state of the corresponding element on the first synchronization table is instantly adjusted to be in first synchronization, and the write IO request is transferred to the IO issuing thread;

And when the IO blocking thread finishes reading the data of the write target data block, the IO blocking thread inserts the read data package into a transmission queue, and then the first synchronous progress state of the corresponding element on the first synchronous table is immediately adjusted to be the first synchronous finish.

4. The method for first-time data synchronization without using snapshot according to claim 3, wherein when the IO blocking thread finishes reading the data of the write target data block, the IO blocking thread inserts the read data package into the transmission queue, and then instantly adjusts the first-time synchronization progress status of the corresponding element on the first-time synchronization table to be first-time synchronization finishing, further comprising:

setting a data sending thread in the IO filtering drive;

When the IO blocking thread finishes reading the data of the target data block, the IO blocking thread stores the read data into a local cache queue, and then the first synchronous progress state of the corresponding element on the first synchronous table is immediately adjusted to be the first synchronous finish;

and receiving a sending instruction, locally caching encapsulated data by a data sending thread, and then inserting the encapsulated data into a sending queue.

5. The method for first-time data synchronization without using snapshots of claim 1, further comprising:

when a read IO request for reading a disk is generated, a first synchronous thread acquires the read IO request;

According to the read IO request, the first synchronization thread queries the first synchronization table, judges the current first synchronization proceeding state of the read target data block, if the first synchronization is proceeding, waits for the first synchronization to finish, issues the read IO request, executes the read operation, and if the first synchronization is not started or the first synchronization is finished, directly issues the read IO request, and executes the read operation.

6. A system for first-time data synchronization without using snapshots, the system comprising:

7. The system for first-time data synchronization without using a snapshot of claim 6, wherein said sending write target data module further comprises:

the method comprises the steps of calculating a write target data block position unit, which is used for an IO blocking thread to acquire an offset field and a data length field of a write IO request, and calculating a write target data block initial block number and a write target data block terminal block number;

the IO blocking thread starts to read the data of the write target data block from the disk according to the calculation result, then immediately adjusts the first synchronization progress state of the corresponding element on the first synchronization table to be in progress for first synchronization, and transfers the write IO request to the IO issuing thread;

And the sending target data unit is used for inserting the read data package into a sending queue by the IO blocking thread when the IO blocking thread finishes reading the target data block data, and then immediately adjusting the first synchronization progress state of the corresponding element on the first synchronization table to finish the first synchronization.

8. The first-time data synchronization system without using snapshots of claim 6, said system further comprising:

the read IO request acquisition module is used for acquiring the read IO request by the first synchronous thread when the read IO request for reading the disk is generated;

The read IO request issuing module is used for inquiring the first synchronization table according to the read IO request, judging the current first synchronization proceeding state of the read target data block, if the first synchronization is proceeding, issuing the read IO request after the first synchronization is completed, executing the read operation, and if the first synchronization is not started or the first synchronization is completed, directly issuing the read IO request, and executing the read operation.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the first data synchronization method of any one of claims 1 to 5 without using snapshots when executing the program.

10. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the first data synchronization method of any of claims 1 to 5 without using snapshots.