CN108810150B - Data replication method for application-level disaster recovery system of collaborative office system - Google Patents

Abstract

The invention discloses a data replication method for an application-level disaster recovery system of a collaborative office system, comprising: the collaborative office system is provided with a replication mechanism according to the data type, and according to different usage requirements of local files of the disaster recovery center, asynchronous replication of intermediate files is adopted respectively. Copy with the application file redundant upload method; there are many local files for task collaboration, but due to the low requirement of real-time data copying due to regional isolation, the asynchronous copying method of files between centers is adopted; official document transmission and data exchange belong to the data exchange center. Using the method of redundant uploading of application files, the pressure of real-time replication is evenly distributed in each business operation process. The key to realizing the dual-active disaster recovery of the collaborative office system in the present invention lies in the unification of the back-end data of the two centers.

Description

Data replication method for application-level disaster recovery system of collaborative office system

technical field

The invention relates to the technical field of computers, in particular to a data replication method for an application-level disaster recovery system of a collaborative office system.

Background technique

Business continuity assurance is the goal of application-level disaster recovery construction in collaborative office systems. The goal of the design and construction of the long-distance active-active disaster recovery for the collaborative office of the State Grid Corporation belongs to the highest level of disaster recovery construction. Through the classification of business interruption events, it can be known that there must be corresponding emergency countermeasures for various events. Only by achieving rapid takeover and recovery for global disasters, and at the same time taking into account the emergency response to local local failures, can an all-round and powerful fault emergency response system be constructed, and can provide a strong guarantee for the business continuity of the enterprise . The key to the active-active disaster recovery of the collaborative office system lies in the unification of the back-end data of the two centers, which is the focus of application switching (takeover). If there is no data consistency guarantee, then the application of active-active is empty talk. The collaborative office system has many data types and complex applications. Each type of data has its own mode and mechanism for replication, including structured data (such as ORACLE), unstructured data (such as data generated by DOMINO application system, SOLR index data) and various image data files), cannot cover all data in one way of reproduction. Therefore, different business data such as DOMINO, ORACLE, Solr and other files need to be replicated separately.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a data replication method for an application-level disaster recovery system of a collaborative office system, so as to solve the data replication design problem of two disaster recovery centers for remote and remote dual-active disaster recovery in a pilot unit of a power grid collaborative office system. Purpose.

In order to achieve the above object, the present invention is realized through the following technical solutions:

A data replication method for an application-level disaster recovery system of a collaborative office system, comprising: in order to realize that the disaster recovery center at any end has a full amount of business logic and business data, and satisfy the ability of carrying the whole network business after the system is switched, any one of the disaster recovery The central business change data is asynchronously replicated to the peer disaster recovery center according to different business data types.

The collaborative office system has a corresponding replication mechanism according to the data type. According to the different requirements for the use of local files in the disaster recovery center, the asynchronous replication of intermediate files and the redundant upload of application files are used for replication.

There are many local files for task collaboration, but since regional isolation does not require high real-time data replication, the asynchronous file replication method between centers is adopted.

Official document transmission and data exchange belong to the data exchange center, and the method of applying redundant uploading of files is adopted to spread the pressure of real-time replication evenly in each business operation process.

Preferably, the method for asynchronously copying files between centers includes the following process: synchronously initializing the application-level disaster recovery system components of the collaborative office system:

Set the task collaboration async file start incremental sync flag before starting sync.

Develop automated scripts and use special tools to synchronize the data center task collaboration-related directory data of the normal disaster recovery center to the host directory corresponding to the data center of the initialized disaster recovery center; set the task collaboration data synchronization, and generate incremental synchronization messages.

Kafka distributed message middleware clusters were added to the two disaster recovery centers for asynchronous and synchronous control.

When the client uses the task cooperation module to operate the attachment and allows the start of incremental synchronization, the task cooperation processing module generates the local file of the disaster recovery center and generates the file synchronization kafka message request. The message content includes the file name, file path, file Size, information of the host where the file is located.

Synchronous message processing component.

Connect to the Kafka message middleware cluster corresponding to the data center of the peer disaster recovery center in the client mode through the kafka interface, obtain unprocessed message requests from the queue, and process them one by one until all message processing is completed.

Parse the synchronization message request, obtain the file name, file path, host where the file is located, and file size information to be synchronized, and synchronize the corresponding file from the directory specified by the peer disaster recovery center to the local disaster recovery center through ftp.

Preferably, the method for redundant uploading and copying of application files includes the following process: by using an automated shell script, using a special tool to synchronize the data center official document transmission and data exchange in the normal disaster recovery center to the corresponding directory data of the shared file system to the initialization disaster recovery center The data center corresponds to the host directory.

When the collaborative office subsystem sends documents online, it calls the document transmission component of the disaster recovery center at the local end to upload the document data to the shared file system, and at the same time calls the document transmission component service of the disaster recovery center at the opposite end to upload the document data to the peer. In the data center shared directory of the end disaster recovery center; the failure to call the official document transmission service of the local disaster recovery center or the peer disaster recovery center belongs to the same transaction. If one of the disaster recovery center service calls fails, the entire business processing fails.

When the collaborative office subsystem performs data exchange operations, it calls the data exchange component of the disaster recovery center at the local end to upload the temperature control data to the shared file system, and at the same time calls the data exchange and transmission component of the disaster recovery center at the opposite end to transfer the exchange file to the opposite end. In the data center shared file system of the disaster recovery center, two service calls belong to the same transaction. If one of the disaster recovery center service calls fails, the entire business process fails.

Preferably, the above-mentioned two disaster recovery centers use the same hardware resources to deploy two sets of the same SolrCloud unified search cluster, and the two center SolrCloud clusters run at the same time in daily situations.

The Solr index data replication method is used to realize the synchronization of Solr cluster data in the data centers of the two disaster recovery centers. The synchronization network between the clusters is created separately by using a virtual local area network, which is separated from the production network, and the synchronization network bandwidth is limited to ensure that the data is synchronized when , does not affect the normal operation of production applications; Solr index data replication method includes: cluster synchronization initialization component, set the SolrCloud cluster start incremental synchronization flag before starting synchronization.

Through automated shell scripts, use special tools to synchronize the index data directory data under the SolrCloud cluster host of the normal data center to the corresponding host directory of the data center of the initialized disaster recovery center;

After the data of the entire SolrCloud cluster is successfully synchronized, start the SolrCloud unified search service application in the data center of the initialization disaster recovery center.

Set SolrCloud cluster initialization complete flag; message middleware cluster component.

The two disaster recovery centers use multiple virtual Linux machines to build Kafka distributed message middleware clusters.

The first-level deployment application is transformed. In the file management, file center, and office automation modules, while generating SolrCloud cluster index data, the index-related data is simultaneously generated into the Kafka message middleware cluster for processing by the synchronization program.

Synchronous message processing component.

After the SolrCloud cluster is initialized, unprocessed message requests are obtained from the Kafka distributed message middleware queue in the data center of the peer disaster recovery center, and processed one by one until all messages are processed.

Parse the synchronization message to be processed, and write the message to the SolrCloud cluster of the local disaster recovery center according to the content of the request.

Preferably, the special tool is a data mirror backup tool under a Unix-like system under Linux.

Compared with the prior art, the present invention has the following advantages:

The disaster recovery system can effectively reduce the impact on the company's core IT system, speed up the recovery of damaged information systems, and then enhance the risk management capability of the entire power network for safe power supply, effectively promoting sustainable economic and social development. In order to maintain the healthy and stable development of the enterprise IT information system, the construction of a disaster recovery system is an effective method to ensure the good scalability and sustainable development of the business. State Grid Corporation of China has a broad business scope, and the external environment is complex. The construction of the disaster recovery system can effectively prevent risks, reduce losses, improve business continuity and high availability, and improve the ability of the IT information management system to respond to various changes at any time. When a disaster occurs in a certain province, it will bring immeasurable economic losses. The disaster recovery system ensures the security of data and minimizes losses. Compared with the self-built disaster recovery computer room model in provinces and cities, building a centralized disaster recovery center can improve resource utilization and save a lot of resources. A large amount of investment is saved in the supporting area of the computer room, air conditioners, power cabinets, UPS, cabinets, IT equipment, shared resources in the emergency verification area, and software licenses. The centralized disaster recovery center model can save equipment maintenance costs. Due to the reduction of equipment, energy consumption is reduced, human resource costs are saved, personnel training costs are reduced, and disaster recovery management costs are reduced. In addition, in terms of data replication, as a key point in the construction of active-active disaster recovery, according to different data types and application characteristics, build logical asynchronous replication strategies including Domino document data, Oracle database, files, Solr index data, etc. Logical isolation of data access scope, realizing dual read-write data and application active-active strategy.

Description of drawings

1 is a schematic diagram of the architecture of an application-level active-active disaster recovery system of a collaborative office system according to the present invention;

Fig. 2 is a flow chart of the method for asynchronously copying files between centers in the present invention;

Fig. 3 is a flow chart of the method for redundant uploading and copying of application files in the present invention;

FIG. 4 is a flowchart of a method for replicating Solr index data in the present invention.

Detailed ways

The present invention will be further elaborated below by describing a preferred specific embodiment in detail with reference to the accompanying drawings.

As shown in FIG. 1 , the present invention is applicable to an application-level active-active disaster recovery system of a collaborative office system, and the system includes:

The collaborative office system is split, the business of the entire region is distributed to the two disaster recovery centers in A and B, and the business change data is asynchronously replicated to the opposite end according to different business data types. In this way, the system at either end has a full amount of business logic and business data, which satisfies the ability to carry the entire network services after the system is switched.

Each disaster recovery center is provided with the same server group, and the above-mentioned server group includes an NTP time synchronization server, an ORACLE database replication server, multiple application server resource pool hosts and multiple database application servers.

The NTP time synchronization server is used to provide a unified synchronization time for each application server and network in the collaborative office system. Network management and operation and maintenance monitoring provide time benchmarks. At present, the NTP server deployed at the first level of the State Grid is used to provide the clock synchronization source for the collaborative office system. Each application server and network system of the collaborative office system obtains the correct time from the NTP server. Specifically, the NTP clock source of the disaster recovery center in Area B is selected as the benchmark.

The two disaster recovery centers of this system are respectively equipped with a storage system and a continuous data protector (CDP). The storage system is used to store local data in the collaborative office system and read and write operations to local data. The main function of the Continuous Data Protector (CDP) and its auxiliary flash memory is to provide local data protection against emergency recovery from local storage failures and logical failures. The amount of data that needs to be protected only involves core data (such as ORACLE database data).

The ORACLE database replication server is equipped with database replication software, which is used to perform the following operations: Adopt non-agent mode, obtain SQL statements by analyzing log logs, and realize real-time replication of data in the database.

The database replication software is mainly aimed at the ORACLE database. If the disaster preparedness center in Area B and Area A are far apart and there are many data transmission network nodes, it should have better fault tolerance for network failures; The performance of the database is affected. The above database replication software has the following advantages. Network failure: ensure normal operation in the case of long outages, short outages and intermittent network interruptions. Database failure: When the target database fails, the source database is not affected. When the target database is repaired, the software continues to work. In the case of network failure, server hardware failure, database failure, etc., it can continue to work after the failure is recovered, and support breakpoint resuming; the replication software ensures the accuracy of data, and the consistency of data is verified in real time during the replication process. , if an abnormality is found, an alarm function is provided.

Active-active application design is mainly aimed at the specific planning and design of DOMINO application system, unified login system, unified view, national network document transmission, database synchronization application, database data writing sequence application, database switching notification application and dynamic domain name application to ensure active-active application. unity.

The key to the active-active disaster recovery of the collaborative office system lies in the unification of the back-end data of the disaster recovery centers in Areas A and B. Disaster takeover and active-active reconstruction are two processes: first, when a system disaster occurs, the application is switched (taken over) from the end of the disaster to the other end, while the active-active reconstruction is the recovery of the application after the end of the disaster is restored to production. All, it involves the classification criteria of the danger level, decision-making principles, response methods, response processes, etc., especially the completion of the data supplementary recording link in the active-active reconstruction stage, which is the sign of real production recovery.

The active-active disaster recovery system of the collaborative office system is associated with many peripheral applications, including portal systems, unified permissions, digital archives, electronic files, unstructured platforms, video and teleconferencing, and IMS systems, which generate interaction and interaction of various application data. transfer. For the Domino node group in the system, the node group is merged on the application server, so that each distributed Domino node can provide business services and data management capabilities of more than two nodes. The server of the Disaster Preparedness Center in Area or Area B provides local business services, but also manages the business data of the Disaster Preparedness Center in Area B or Area A.

In the active-active operation, according to the actual business access, some business directories provide business access. In fact, the business directory provides business access in another data, but provides replication and writing on the local end. That is, as shown in Figure 2, when the application-level active-active disaster recovery system of the collaborative office system is in normal operation, users in the disaster recovery center in area A read and write data in area A. And the disaster recovery center in area A asynchronously replicates the data in the disaster recovery center in area B to the disaster recovery center in area A. On the other hand, users in the disaster recovery center in area B write and read the data in area B, and the disaster recovery center in area B asynchronously copies the data from the disaster recovery center in area A to the disaster recovery center in area B. When the disaster recovery center in Area A or B fails, after the disaster recovery system starts the disaster recovery operation mode, users in Area A and Area B can read and write operations from the Disaster Recovery Center in Area B or Area A. Disaster Recovery in Area B or Area A center data.

The disaster recovery centers in Area A and Area B have built the same unified login service, and the functions of the services are also completely the same. However, under normal circumstances, only the unified login service of the disaster recovery center in area B is enabled. Although the unified login service in area A is enabled, it is not actually enabled. Combined with the DNS server, the service switching when the disaster recovery is started is realized. That is, although these few services of the login portal in the active-active system of the collaborative office system are also enabled at the same time, they are not used at the same time.

Nodes with local file data have task collaboration, data exchange and document transmission. All applications use Weblogic middleware, and file data and applications are separated. Task collaboration has the characteristics of regional access and a large amount of data. Data exchange and official document transmission are both data exchange centers built to achieve smooth government decrees and integrated applications. They belong to the services shared by the entire network, and require high timeliness for data replication. The collaborative office system has a corresponding replication mechanism according to the data type. The data types in the collaborative office system of the present invention include: for structured data (such as ORACLE), unstructured data (such as data generated by DOMINO application system, SOLR index data and various image data files).

According to different usage requirements of local files, intermediate file asynchronous replication and application file redundancy upload strategy are used for replication.

There are many local files for task collaboration, but since regional isolation does not require high real-time data replication, an asynchronous file replication strategy between centers is adopted.

Official document transmission and data exchange belong to the data exchange center. Since the proportion of integrated services is not high, the amount of data is not large, but the real-time requirements are high. The redundant uploading method of application files can be used to spread the pressure of real-time replication to each secondary business operations.

As shown in Figure 2, the inter-center file asynchronous replication strategy includes the following processes: Synchronous initialization components:

Set the task collaboration asynchronous file start incremental synchronization flag before starting synchronization;

Develop automated scripts, and use special tools (data mirror backup tools under Unix-like systems under Linux) to synchronize the normal data center task collaboration-related directory data to the host directory corresponding to the initialization data center; set up task collaboration data synchronization. Incremental synchronization message generation.

Kafka distributed message middleware clusters are added to the disaster recovery centers in Area B and Area A, respectively, for asynchronous and synchronous control.

When the client uses the task cooperation module to operate the attachment and allows the start of incremental synchronization, the task cooperation processing module generates the local file of the disaster recovery center and generates the file synchronization kafka message request. The message content includes the file name, file path, file Size, host where the file is located, etc.

Synchronous message processing component.

Connect to the Kafka message middleware cluster corresponding to the peer data center in the client mode through the kafka interface, obtain unprocessed message requests from the queue, and process them one by one until all message processing is completed.

Parse the synchronization message request, obtain the file name, file path, host where the file is located, file size and other information to be synchronized, and synchronize the corresponding file from the specified directory of the peer center to the center through ftp.

As shown in Figure 3, the initialization of the application file redundant upload and replication strategy includes the following process: through the automated shell script, use a special (data mirror backup tool under Linux-like Unix system) tool to transfer the normal data center official documents and data exchange shared files The system corresponding directory data is synchronized to the corresponding host directory in the initialization data center.

When the collaborative office subsystem sends documents online, it calls the document transmission component of the disaster recovery center to upload the document data to the shared file system, and at the same time calls the document transmission component service of the peer disaster recovery center to upload the document data to the opposite end at the same time. In the shared directory of the data center; the failure to call the official document transmission service of the disaster recovery center or the peer disaster recovery center belongs to the same transaction. If the service call of a disaster recovery center fails, the entire business processing fails.

When the collaborative office subsystem performs data exchange operations, it calls the data exchange component of the disaster recovery center to upload the temperature control data to the shared file system, and at the same time calls the data exchange and transmission component of the peer disaster recovery center to transfer the exchange file to the peer data In the central shared file system, two service calls belong to the same transaction. If a service call of a disaster recovery center fails, the entire business process fails.

As shown in Figure 4, two sets of the same SolrCloud unified search cluster are deployed in the disaster recovery centers of Area B and Area A using the same hardware resources. In daily situations, the two central SolrCloud clusters run at the same time;

Through the central data synchronization component, the data synchronization of the data center Solr cluster in the disaster recovery center of the B area and the A area is realized. Does not affect the normal operation of production applications.

Cluster synchronization initialization components:

Set the SolrCloud cluster start incremental synchronization flag before starting synchronization.

Through an automated shell script, use a dedicated tool (data mirror backup tool under Unix-like systems under Linux) to synchronize the index data directory data under the SolrCloud cluster host in the normal data center to the corresponding host directory in the initialization data center.

After the entire SolrCloud cluster data is successfully synchronized, start the SolrCloud unified search service application in the initialization data center.

Set the SolrCloud cluster initialization complete flag.

Message middleware cluster components:

Set up Kafka distributed message middleware clusters on multiple virtual Linux machines in the disaster recovery centers in Area B and Area A.

The first-level deployment application is transformed. In the file management, file center, and office automation modules, while generating SolrCloud cluster index data, the index-related data is simultaneously generated into the Kafka message middleware cluster for processing by the synchronization program.

Synchronous message processing components:

After the SolrCloud cluster is initialized, unprocessed message requests are obtained from the Kafka distributed message middleware queue in the peer data center, and processed one by one until all messages are processed.

Analyze the synchronization message to be processed, and write the message to the SolrCloud cluster of the center according to the request content.

While the content of the present invention has been described in detail by way of the above preferred embodiments, it should be appreciated that the above description should not be construed as limiting the present invention. Various modifications and alternatives to the present invention will be apparent to those skilled in the art upon reading the foregoing. Accordingly, the scope of protection of the present invention should be defined by the appended claims.

Claims (4)

1. A data replication method for an application-level disaster recovery system of a collaborative office system is characterized by comprising the following steps: in order to realize that any one end of the disaster recovery backup center has full service logic and service data and meet the capability of carrying the whole network service after system switching, the service change data of any one disaster recovery backup center is asynchronously copied to the opposite end disaster recovery backup center according to different service data types;

the cooperative office system is correspondingly provided with a copying mechanism according to the data type, and copies are respectively carried out by adopting a method of asynchronous copying of files among centers and redundant uploading of application files according to different requirements of local files of the disaster recovery center;

copying a local file of task cooperation by adopting an inter-center file asynchronous copying method;

the official document transmission and data exchange belong to a data exchange center, and the real-time copying pressure is leveled in each business operation process by adopting an application file redundancy uploading method;

the method for asynchronously copying the files among the centers comprises the following processes: synchronously initializing application-level disaster recovery system components of the cooperative office system:

setting a task collaboration asynchronous file start increment synchronous mark before starting synchronization;

developing an automation script, and synchronizing data of a data center task cooperation related directory of a normal disaster recovery center to a host directory corresponding to a data center of an initialization disaster recovery center by using a special tool; setting task cooperation data synchronization and generating an increment synchronization message;

respectively adding a Kafka distribution message middleware cluster in the two disaster recovery centers for asynchronous and synchronous control;

when a client uses a task cooperation module to operate an attachment and allows increment synchronization to start, a task cooperation processing module generates a local file of the disaster recovery center and simultaneously generates a file synchronization kafka message request, wherein the message content comprises a file name, a file path, a file size and information of a host where the file is located;

a synchronization message processing component;

connecting to a kafka message middleware cluster corresponding to a data center of an opposite-end disaster recovery center in a client mode through a kafka interface, acquiring unprocessed message requests from a queue, and processing the message requests one by one until all message processing is completed;

and analyzing the synchronization message request, acquiring the file name and the file path which need to be synchronized, the host where the file is located and the file size information, and synchronizing the corresponding file from the specified directory of the opposite-end disaster recovery center to the local-end disaster recovery center in an ftp mode.

2. The data replication method of a collaborative office system application-level disaster recovery system according to claim 1, wherein the application file redundancy upload replication method comprises the following processes: synchronizing the directory data corresponding to the data center official document transmission and data exchange shared file system of the normal disaster recovery center to the host directory corresponding to the data center of the initialized disaster recovery center by using a special tool through an automatic shell script;

when the cooperative office subsystem carries out networking text sending, calling a document transmission assembly of the home-end disaster recovery center to upload document data to a shared file system, calling service of the document transmission assembly of the opposite-end disaster recovery center at the same time, and uploading the document data to a data center shared directory of the opposite-end disaster recovery center at the same time; the failure of the local or opposite disaster recovery center document transmission service call belongs to the same transaction, and if one of the disaster recovery centers fails in service call, the whole service processing fails;

when the cooperative office subsystem carries out data exchange operation, the data exchange assembly of the local disaster recovery center is called to upload temperature control data to the shared file system, the data exchange transmission assembly of the opposite disaster recovery center is called at the same time, the exchange file is transmitted to the data center shared file system of the opposite disaster recovery center, service calling belongs to the same transaction twice, and if one disaster recovery center fails in service calling, the whole service processing fails.

3. The data replication method of the cooperative office system application-level disaster recovery system as claimed in claim 2, wherein the two disaster recovery centers use the same hardware resources to deploy two sets of the same SolrCloud unified search clusters, and the two SolrCloud clusters in the two centers run simultaneously in daily conditions;

the Solr index data replication method is adopted to realize the Solr cluster data synchronization of the data centers of the two disaster recovery centers, a synchronization network between clusters is independently and newly established in a virtual local area network mode and is separated from a production network, the bandwidth of the synchronization network is limited, and the normal operation of production application is not influenced when the data are synchronized;

the Solr index data copying method comprises the following steps: the cluster synchronization initialization component sets a SolrCloud cluster start increment synchronization mark before starting synchronization;

through an automatic shell script, index data directory data under a SolrCloud cluster host of a normal data center are synchronized to a host directory corresponding to a data center of an initialization disaster recovery center by using a special tool;

when the whole SolrCloud cluster data are completely and successfully synchronized, starting and initializing SolrCloud unified search service application in a data center of the disaster recovery center;

setting a SolrCloud cluster initialization completion flag; message middleware cluster component:

respectively building Kafka distributed message middleware clusters in a plurality of virtual Linux machines used in two disaster recovery centers;

transforming a first-level deployment application, generating SolrCloud cluster index data and simultaneously generating index related data into a Kafka message middleware cluster at the same time in a file management, file center and office automation module for processing by a synchronization program;

a synchronization message processing component:

after the SolrCloud cluster is initialized, obtaining unprocessed message requests from a Kafka distributed message middleware queue of a data center of an opposite-end disaster recovery center, and processing the messages one by one until all the messages are processed;

and analyzing the synchronous message to be processed, and writing the message into the SolrCloud cluster of the local disaster recovery center according to the request content.

4. The data replication method of the cooperative office system application level disaster recovery system as recited in claim 1, 2 or 3, wherein the special tool is a data mirror backup tool under a Linux-like system under Linux.

Images