CN114741577B - Business data management method, system, electronic device and readable storage medium - Google Patents

Abstract

The present invention relates to the technical field of data management, and discloses a business data management method, system, electronic equipment and a readable storage medium. The method obtains multiple business data types and business data corresponding to each business data type, according to each described The business data corresponding to the business data type is integrated to obtain single-type data, and then the business data and/or single-type data are processed to obtain legal data or illegal data. If there is illegal data, the repair data corresponding to the illegal data is generated. Then determine the system data corresponding to each of the business systems from legal data and/or repair data, realize the unification of business data by integrating business data, and then perform quality inspection and verification on business data and single-type data after business data integration. Repair, avoid the situation that the low data quality of business data or single-type data cannot be used, and then improve the interaction efficiency between various business systems.

Description

Service data management method, system, electronic device and readable storage medium

technical field

The invention relates to the technical field of product detection, in particular to a business data management method, system, electronic equipment and a readable storage medium.

Background technique

At present, with the advancement of enterprise management intelligence and the expansion of enterprise development scale, various applications for supporting business operations have emerged. Because they have exclusive data storage and data definition methods, the core data standards between business systems are inconsistent and the definition is inaccurate, and "data islands" appear, which in turn causes continuous interaction and maintenance costs between business systems. improve.

Therefore, establishing a single data source by integrating the business data of various business systems to realize the unification and standardization of enterprise business data has become the main means for many enterprises to manage business data. However, due to the ever-changing business data formats and complex numerical types, data integration is difficult. As a result, the data quality of business data and its integrated data cannot be guaranteed, and there are cases where it cannot be used, which in turn reduces the interaction efficiency between various business systems.

Contents of the invention

In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is presented below. The summary is not intended to be an extensive overview nor to identify key/important elements or to delineate the scope of these embodiments, but rather serves as a prelude to the detailed description that follows.

In view of the above-mentioned shortcomings of the prior art, the present invention discloses a business data management method, system, electronic equipment and readable storage medium, so as to improve the data quality of business data and its integrated data, and further improve the interoperability between various business systems. interaction efficiency.

The invention discloses a business data management method, which is characterized in that it includes: obtaining a business data set from multiple business systems, the business data set includes multiple business data types and business data corresponding to each business data type, and Integrate at least a part of the business data corresponding to each of the business data types to obtain single-type data corresponding to each business data type; perform data processing on the business data and/or the single-type data according to preset data quality inspection rules Processing to obtain a quality inspection data set, wherein the quality inspection data set includes legal data and/or illegal data; if there is illegal data in the quality inspection data set, generate repair data corresponding to the illegal data; System data corresponding to each of the service systems is determined in the data, wherein the master data includes legal data and/or repair data.

Optionally, generating repair data corresponding to the illegal data includes: repairing the illegal data according to a preset repair rule script, and judging whether the repair of the illegal data is successful according to the repair result; if the repair of the illegal data is successful , then determine the repaired illegal data as the repaired data corresponding to the illegal data; if the illegal data is not repaired successfully, then obtain the quality inspection report corresponding to the illegal data, according to the illegal data and the report to generate a repair request, and send the repair request to a preset client; if a repair instruction corresponding to the repair request is received, determine the user update data corresponding to the illegal data according to the repair instruction, and according to the data The quality inspection rules reclassify the user update data, and judge whether the user update data is legal data according to the reclassification result; if the user update data is legal data, then determine the user update data as corresponding to the illegal data. repair data.

Optionally, acquiring business data sets from multiple business systems includes: collecting business data from multiple business systems and data source information corresponding to the business data; assigning business data to the business data according to the data source information Data type, determining the business data type and the business data corresponding to the business data type as a business data set; where business data is collected from the business system by at least one of the following methods, and the target system is determined from the business system , and collect business data from the target system; regularly collect business data in the business system according to the preset time interval corresponding to the business system; judge the status of the business system according to the system log corresponding to the business system Whether the business data changes, if the business data of the business system changes, collect the business data from the business system; connect the new system through the preset connection mode, and determine the new system as a new business system , and collect business data from the new business system.

Optionally, after the repair data corresponding to the illegal data is generated, before determining the system data corresponding to each of the business systems from the master data, the method further includes: from the master data according to the preset sensitive data list Determine sensitive data, generate an interception request based on the sensitive data, and send the interception request to a preset client; if an interception instruction corresponding to the interception request is received, determine whether the sensitive data exists according to the interception instruction intercepting data; if intercepting data exists in the sensitive data, deleting the intercepting data from the master data.

Optionally, the method further includes at least one of the following: adding reference data to master data, the reference data including at least one of administrative codes, company information, industry information, and client information; The target data is determined in the master data corresponding to the type, and the target data is fused based on a preset type fusion algorithm to obtain multi-type data, and the multi-type data is added to the master data.

Optionally, determining the system data corresponding to each of the business systems from the master data also includes at least one of the following: before determining the system data corresponding to each of the business systems from the master data, assigning a data identifier to the master data , and encode the master data according to the preset encoding format; after determining the system data corresponding to each of the business systems from the master data, configure the data acquisition method for each of the business systems, and encode the system data according to the corresponding The data acquisition method is sent to the corresponding business system.

Optionally, performing data processing on the business data and/or the single-type data according to preset data quality inspection rules includes: extracting type feature data, wherein the data to be quality inspected is the business data or the single-type data; input the type feature data into a preset data classification model to obtain a model output result, wherein the data classification The model is obtained by training the preset neural network model through feature sample data with classification labels; based on the output result of the model, it is determined that the data to be inspected is legal data or illegal data.

The invention discloses a business data management system, comprising: an acquisition module, used to acquire a business data set from multiple business systems, the business data set includes multiple business data types and business data corresponding to each business data type, Integrating at least a part of the business data corresponding to each of the business data types respectively to obtain single-type data corresponding to each business data type; the classification module is used to classify the input data to be inspected according to preset data quality inspection rules, Obtaining legal data and/or illegal data, wherein the data to be inspected includes the business data and/or the single-type data; a generating module, configured to generate repair data corresponding to the illegal data, and use the repair data as New data to be quality inspected; a determining module, configured to determine system data corresponding to each of the business systems from master data, wherein the master data includes legal data obtained by the quality inspection module.

The invention discloses an electronic device, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the above method.

The invention discloses a computer-readable storage medium, on which a computer program is stored: when the computer program is executed by a processor, the above-mentioned method is realized.

Beneficial effects of the present invention:

By acquiring multiple business data types and the business data corresponding to each business data type, integrating the business data corresponding to each of the business data types to obtain single-type data, and then performing data processing on the business data and/or single-type data to obtain legal data or illegal data, if there is illegal data, generate repair data corresponding to the illegal data, and then determine system data corresponding to each of the business systems from the legal data and/or repair data. In this way, the unification of business data is realized through the integration of business data, and then the quality inspection and repair of business data and single-type data after business data integration are performed, so as to avoid unusable business data or single-type data due to low data quality. situation, thereby improving the interaction efficiency between various business systems.

Description of drawings

Fig. 1 is a schematic flow chart of a business data management method in an embodiment of the present invention;

Fig. 2 is a schematic flow chart of another business data management method in an embodiment of the present invention;

Fig. 3 is a schematic structural diagram of a business data management system in an embodiment of the present invention;

Fig. 4 is a schematic structural diagram of another business data management system in an embodiment of the present invention

Fig. 5 is a schematic structural diagram of an electronic device in an embodiment of the present invention.

Detailed ways

Embodiments of the present invention are described below through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation modes, and various modifications or changes can be made to the details in this specification based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, in the case of no conflict, the following embodiments and sub-samples in the embodiments may be combined with each other.

It should be noted that the diagrams provided in the following embodiments are only schematically illustrating the basic ideas of the present invention, and only the components related to the present invention are shown in the diagrams rather than the number, shape and shape of the components in actual implementation. Dimensional drawing, the type, quantity and proportion of each component can be changed arbitrarily during actual implementation, and the component layout type may also be more complicated.

In the following description, numerous details are discussed in order to provide a more thorough explanation of embodiments of the invention, however, it will be apparent to those skilled in the art that embodiments of the invention can be practiced without these specific details, In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail in order to avoid obscuring the embodiments of the invention.

The terms "first", "second" and the like in the description and claims of the embodiments of the present disclosure and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It should be understood that the data so used may be interchanged under appropriate circumstances so as to facilitate the embodiments of the disclosed embodiments described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion.

Unless stated otherwise, the term "plurality" means two or more.

In the embodiments of the present disclosure, the character "/" indicates that the preceding and following objects are an "or" relationship. For example, A/B means: A or B.

The term "and/or" is an associative relationship describing objects, indicating that there can be three relationships. For example, A and/or B means: A or B, or, A and B, these three relationships.

As shown in FIG. 1, an embodiment of the present disclosure provides a business data management method, including:

Step S101, acquiring business data sets from multiple business systems, integrating at least a part of the business data corresponding to each business data type respectively, to obtain single-type data corresponding to each business data type;

Wherein, the business data set includes multiple business data types and business data corresponding to each business data type;

Step S102, perform data processing on business data and/or single-type data according to preset data quality inspection rules to obtain a quality inspection data set;

Among them, the quality inspection data set includes legal data and/or illegal data;

Step S103, if there is illegal data in the quality inspection data set, generate repair data corresponding to the illegal data;

Step S104, determining the system data corresponding to each business system from the master data;

Wherein, the master data includes legal data and/or repair data.

By adopting the business data management method provided by the embodiments of the present disclosure, multiple business data types and the business data corresponding to each business data type are obtained, and the business data corresponding to each business data type is integrated to obtain single-type data, and then the business data and /or single-type data is processed to obtain legal data or illegal data. If there is illegal data, the repair data corresponding to the illegal data is generated, and then the system data corresponding to each business system is determined from the legal data and/or repair data. In this way, the unification of business data is realized through the integration of business data, and then the quality inspection and repair of business data and single-type data after business data integration are performed, so as to avoid unusable business data or single-type data due to low data quality. situation, thereby improving the interaction efficiency between various business systems.

In some embodiments, the business system includes one or more of business systems such as an OA (Office Automation, office automation) system, a CRM (Customer Relationship Management, customer relationship management) system, and an ERP (Enterprise Resource Planning, enterprise resource planning) system. Types of business data include one or more of system data, customer data, enterprise data, employee data, supplier data, maintenance data, material data, production data, annual data, and monthly data.

Optionally, acquiring business data sets from multiple business systems includes: collecting business data and data source information corresponding to business data from multiple business systems; assigning business data types to business data according to data source information, and classifying business data The business data corresponding to the type and the business data type is determined as a business data set.

Optionally, the business data collection method includes database (DB, database) collection, message queue (MQ, Message Queue) collection, REST API (Representational State Transfer Application Programming Interface, an application programming interface that follows the REST architecture specification) collection, One or more of document collection, etc.

Optionally, collecting business data from a business system includes: determining a target system from the business system, and collecting business data from the target system.

Optionally, collecting business data from the business system includes: regularly collecting business data from the business system according to preset time intervals corresponding to the business system. In this way, according to the flexible configuration of the data acquisition strategy, different levels of business data delay acquisition requirements can be met.

In some embodiments, timed incremental/full data synchronization is performed on the business data in the form of a database according to a preset time interval, and the preset time interval includes 1-72 hours.

Optionally, collecting business data from the business system includes: judging whether the business data of the business system has changed according to the system log corresponding to the business system, and collecting the business data from the business system if the business data of the business system has changed.

In some embodiments, the system log corresponding to the business system includes binlog (binary log), monitors the binlog of the database, and if the business data of the database changes, real-time data stream synchronization is performed to collect business data through the database.

Optionally, collecting business data from the business system includes: connecting to the newly added system through a preset connection mode, determining the newly added system as a new business system, and collecting business data from the new business system.

In some embodiments, the preset connection mode includes one or more of connection modes such as SPI (Serial Peripheral Interface, serial peripheral interface), bluetooth, LAN (Local Area Network, local area network) interface, WIFI, wherein, Based on SPI, it supports secondary development horizontal expansion, quickly adds new access methods, and reduces development costs.

In this way, the dynamic configuration of pulling, pushing, and collecting business data from the business system in various ways not only realizes the synchronization of the data layer, but also completely decouples from the business data, so that each business system can realize data unification in a non-sensing state.

Optionally, integrating at least a part of the business data corresponding to each business data type to obtain single-type data corresponding to each business data type includes: separately integrating at least a part of the business data corresponding to each business data type through preset integration rules Integration to obtain single-type data corresponding to each business data type, wherein the preset integration rules include one or more of rolling update, character splicing, taking the same value, taking a specified value, taking a valid value, and filling in values.

In some embodiments, at least a part of the business data corresponding to each business data type is respectively integrated, and after obtaining the single-type data corresponding to each business data type, the obtained single-type data is connected to a unified data access pool, and then shielded Structural differences of heterogeneous data prevent subsequent data processing exceptions.

Optionally, performing data processing on business data and/or single-type data according to preset data quality inspection rules, including: extracting type feature data from the data to be inspected according to the type of business data corresponding to the data to be inspected, wherein, The data to be inspected is business data or single-type data; input the type characteristic data into the preset data classification model to obtain the model output result, wherein the data classification model compares the preset neural network model with the characteristic sample data with classification labels Obtained through training; based on the output result of the model, it is determined whether the data to be inspected is legal data or illegal data.

In some embodiments, the data classification model needs to input multiple types of feature data, and the type feature data corresponds to the type of business data. For example, if the type of business data is user data, the corresponding type feature data includes the number of characters in the name, place of work, address, The user corresponds to one or more types of customer service, etc., and the business data type is enterprise data, and the corresponding type feature data includes one or more types of enterprise employee numbers, financial statement parameters, and the like.

Optionally, if data such as business data, single-type data, legal data, illegal data, or master data is updated, the updated data is classified according to data quality inspection rules.

Optionally, the data quality inspection rules include one or more of null value detection, data type detection, length detection, value range detection, outlier value detection, reference integrity detection, specification detection, and the like.

Optionally, generating repair data corresponding to the illegal data includes: repairing the illegal data according to a preset repair rule script, and judging whether the repair of the illegal data is successful according to the repair result; if the repair of the illegal data is successful, the repaired illegal data It is determined to be the repair data corresponding to the illegal data; if the illegal data is not repaired successfully, obtain the quality inspection report corresponding to the illegal data, generate a repair request according to the illegal data and the quality inspection report, and send the repair request to the preset client; if received Get the repair instruction corresponding to the repair request, determine the user update data corresponding to the illegal data according to the repair instruction, reclassify the user update data according to the data quality inspection rules, and judge whether the user update data is legal data according to the reclassification result; if the user update data If the data is legitimate, the user update data is determined to be repair data corresponding to the illegal data. In this way, illegal data is automatically repaired through repair rule scripts, and manual repair channels are provided for illegal data that fail to be automatically repaired, which expands the scope of repair, thereby reducing the situation that master data cannot be used due to low data quality.

Optionally, repairing illegal data according to a preset repair rule script includes: determining benchmark data from business data and/or single-type data according to a preset repair rule script, repairing illegal data based on the benchmark data, repairing The rule script includes at least one of average value, specified value, median, large number padding and the like.

Optionally, if the user update data is legal data, the method further includes at least one of the following: deleting illegal data corresponding to the user update data; updating a quality inspection report corresponding to the user update data.

In some embodiments, the repair request includes illegal data and/or a quality inspection report, and the repair request is in Excel format; the maintenance personnel manually enters the manual repair data corresponding to the illegal data through the quality inspection report in the repair request, and according to the manual repair data Generate repair instructions and feedback.

Optionally, after the repair data corresponding to the illegal data is generated, before determining the system data corresponding to each business system from the master data, the method further includes: determining the sensitive data from the master data according to the preset sensitive data list, and generating Intercept the request and send the interception request to the preset client; if the interception instruction corresponding to the interception request is received, determine whether the interception data exists in the sensitive data according to the interception instruction; if there is interception data in the sensitive data, delete the interception from the main data data. In this way, by intercepting the master data according to the sensitive data list, the leakage of data confidentiality of different business data is prevented, and the data security is improved.

Optionally, the sensitive data list includes at least one of sensitive data types, sensitive data values, sensitive fields, sensitive character strings, and the like.

In some embodiments, through pre-configured data blacklist rules (for example, a certain field cannot contain a specific string), data interception can be performed on sensitive information in the data, and manual release and data repair are provided. etc. for data recovery.

Optionally, the method further includes: adding reference data to the master data, where the reference data includes at least one of administrative codes, company information, industry information, and client information. In this way, repeated entry of reference data in various business systems is avoided, and at the same time, the reference data can be maintained uniformly, which not only improves the interaction efficiency between various business systems, but also reduces maintenance costs.

Optionally, the method further includes: determining the target data from the master data corresponding to each business data type, merging the target data based on a preset type fusion algorithm to obtain multi-type data, and adding the multi-type data to the master data. In this way, to solve the data island problem of each business system, at least part of the business data corresponding to each business data type is firstly integrated to obtain the single-type data corresponding to each business data type, and then the target is determined from the master data corresponding to each business data type Data, based on the preset type fusion algorithm, the target data is fused to obtain multi-type data, which not only realizes the unification of business data through single-type data, but also realizes the association of master data at different levels through multi-type data, and improves the standardization data, making the master data more comprehensive.

Optionally, before determining the system data corresponding to each business system from the master data, assign a data identifier to the master data, and encode the master data according to a preset encoding format. In this way, assigning data identifiers to master data facilitates traceability of master data, and then manages the entire life cycle of master data. Moreover, uniform data coding is performed on master data to provide unified data business coding, which not only realizes the standardization of enterprise data , and also reduce the cost of coding and translation between systems.

In some embodiments, the preset encoding format includes at least one of a serial number, a fixed character string, a specified field, and the like.

Optionally, before the system data corresponding to each business system is determined from the master data, the subject data and index data are determined according to the master data, and the data warehouse is obtained by layering according to the master data, subject data and index data.

Optionally, after determining the system data corresponding to each business system from the master data, configure the data acquisition method for each business system, and send the system data to the corresponding business system according to the corresponding data acquisition method. In this way, data standardization and unification of various business systems are realized.

Optionally, the master data is sent to the downstream system, and the downstream system is triggered to perform data analysis, data statistics, data display, etc.

In some embodiments, if there is access to a business system, configure data acquisition methods for the accessed business system, such as RestAPI, database, message queue, etc.; when the master data is updated, the update information is sent to the business system; the Business data provides business operations through its own business requirements, and business operations include one or more of data acquisition, data addition, data deletion, data modification, and data query.

Optionally, the method further includes: if a rule modification instruction is received, modifying the system rules according to the rule modification instruction, wherein the system rules include data quality inspection rules, preset time intervals, preset connection methods, preset integration One or more of rules, repair rule scripts, sensitive data lists, reference data, type fusion algorithms, etc.

Optionally, the system rules are modified through the rule modification instructions written by the user on the page through dynamic loading. In this way, the flexibility of data management is improved through user-defined system rules.

In some embodiments, a hot-swappable rule expansion mechanism is provided through JAVA custom class loader and dynamic class loading, which supports users to customize data cleaning scripts according to their own needs, and users can directly write data on the page The quality inspection specification script and the corresponding parameter script can be applied to the quality inspection task in real time after the compilation is passed.

Optionally, the method further includes: obtaining the user modification instruction and the user account corresponding to the user modification instruction, and determining whether the user modification instruction has modification authority according to the user account; if the user modification instruction has modification authority, performing Revise.

In some embodiments, user instruction control is supported through the approval form. Through the user-defined approval process node and the person in charge, the addition, deletion, modification, and query instructions of the master data on the client end need to be submitted for approval before they can take effect. .

Optionally, the method further includes: generating a corresponding data processing report by monitoring the status of data access, quality inspection, and distribution processes. In this way, maintenance personnel can see the data access and distribution quantity in real time, which is convenient for data maintenance confirmation.

As shown in FIG. 2, an embodiment of the present disclosure provides a business data management method, including:

Step S201, acquiring business data sets from multiple business systems;

Wherein, the business data set includes multiple business data types and business data corresponding to each business data type;

Among them, the system rules are modified through the dynamic loading method and the rule modification instructions written by the user on the page. The system rules include data quality inspection rules, preset time intervals, preset connection methods, preset integration rules, repair rule scripts, sensitive One or more of data lists, reference data, type fusion algorithms, etc.;

Step S202, integrating at least a part of business data corresponding to each business data type to obtain single-type data corresponding to each business data type;

Step S203, perform data processing on business data and/or single-type data according to preset data quality inspection rules to obtain a quality inspection data set;

Step S204, judge whether there is illegal data in the quality inspection data set, if so, go to step S205, if not, go to step S209;

Step S205, repair the illegal data according to the preset repair rule script, and obtain the repair result;

Step S206, judging whether the illegal data has been repaired successfully according to the repair result, if so, go to step S209, if not, go to step S207;

Step S207, generate a repair request, receive a repair instruction corresponding to the repair request, and jump to step S208;

Step S208, determine the user update data corresponding to the illegal data according to the repair instruction, determine the user update data as new business data, and jump to step S203;

Step S209, judging whether there is sensitive data in the master data, if yes, skip to step S210, if not, skip to step S213;

Step S210, generating an interception request, and receiving an interception instruction corresponding to the interception request;

Step S211, judging whether there is interception data in the sensitive data, if yes, skip to step S212, if not, skip to step S213;

Step S212, delete the intercepted data from the main data, and jump to step S213;

Step S213, determining the target data from the master data corresponding to each business data type, merging the target data to obtain multi-type data and adding it to the master data;

Step S214, determine the system data corresponding to each business system from the master data, and send the system data to the corresponding business system according to the corresponding data acquisition method.

By adopting the business data management method provided by the embodiments of the present disclosure, multiple business data types and the business data corresponding to each business data type are obtained, and the business data corresponding to each business data type is integrated to obtain single-type data, and then the business data and /or single-type data is processed to obtain legal data or illegal data. If there is illegal data, the repair data corresponding to the illegal data is generated, and then the system data corresponding to each business system is determined from the legal data and/or repair data, which has the following advantages :

First, realize the unification of business data through the integration of business data, and then perform quality inspection and repair of business data and single-type data after business data integration, so as to avoid low data quality of business data or single-type data that cannot be used situation, thereby improving the interaction efficiency between various business systems;

Second, pull, push, and collect business data from the business system through dynamic configuration in various ways, which not only realizes the synchronization of the data layer, but also completely decouples from the business data, so that each business system can realize data unification in a senseless state;

Third, in order to solve the data island problem of each business system, first integrate at least a part of the business data corresponding to each business data type to obtain the single-type data corresponding to each business data type, and then determine from the master data corresponding to each business data type The target data is fused based on the preset type fusion algorithm to obtain multi-type data, which not only realizes the unification of business data through single-type data, but also realizes the association of master data at different levels through multi-type data, improving the Standardize data to make master data more comprehensive;

Fourth, automatically repair illegal data through repair rule scripts, and provide manual repair channels for illegal data that fail to be automatically repaired, expanding the scope of repair, thereby reducing the situation that master data cannot be used due to low data quality;

Fifth, through data interception of master data according to the list of sensitive data, data confidentiality leakage of different business data is prevented, and data security is improved;

Sixth, assign data identifiers to master data, which facilitates traceability of master data, and then manages the entire life cycle of master data. Moreover, unified data coding is performed on master data to provide unified data business coding, which not only realizes corporate data Standardization also reduces the cost of coding and translation between systems;

Seventh, through user-defined system rules, the flexibility of data management is improved.

As shown in FIG. 3 , an embodiment of the present disclosure provides a service data management system, including an acquisition module 301 , a classification module 302 , a generation module 303 and a determination module 304 . The acquiring module 301 is used to acquire a business data set from multiple business systems, the business data set includes multiple business data types and business data corresponding to each business data type, and integrates at least a part of the business data corresponding to each business data type, Obtain the single-type data corresponding to each business data type; the classification module 302 is used to classify the input data to be inspected according to the preset data quality inspection rules to obtain legal data and/or illegal data, wherein the data to be inspected includes business data and/or single-type data; the generation module 303 is used to generate repair data corresponding to illegal data, and uses the repair data as new data to be inspected; the determination module 304 is used to determine the system data corresponding to each business system from the master data, Among them, the master data includes legal data obtained by the quality inspection module.

Using the business data management system provided by the embodiments of the present disclosure, by acquiring multiple business data types and the business data corresponding to each business data type, integrating the business data corresponding to each business data type to obtain single-type data, and then analyzing the business data and /or single-type data is processed to obtain legal data or illegal data. If there is illegal data, the repair data corresponding to the illegal data is generated, and then the system data corresponding to each business system is determined from the legal data and/or repair data. In this way, the unification of business data is realized through the integration of business data, and then the quality inspection and repair of business data and single-type data after business data integration are performed, so as to avoid unusable business data or single-type data due to low data quality. situation, thereby improving the interaction efficiency between various business systems.

As shown in FIG. 4 , an embodiment of the present disclosure provides a service data management system, including an upstream service end 401 , a service management end 402 and a downstream service end 403 .

The upstream business end 401 includes one or more of business systems such as OA system, CRM system, and ERP system;

The business management terminal 402 includes an acquisition module 301, a classification module 302, a generation module 303 and a determination module 304; the acquisition module 301 is used to acquire a business data set from an upstream business end, and the business data set includes multiple business data types and each business data type For the corresponding business data, at least a part of the business data corresponding to each business data type is respectively integrated to obtain single-type data corresponding to each business data type; the classification module 302 is used to classify the input data to be quality-checked according to preset data quality-checking rules Classify to obtain legal data and/or illegal data, wherein the data to be inspected includes business data and/or single-type data; the generation module 303 is used to generate repair data corresponding to the illegal data, and use the repair data as a new quality inspection Data; the determination module 304 is used to determine the system data corresponding to each business system from the master data, and send the master data and/or system data to the downstream business end, wherein the master data includes legal data obtained by the quality inspection module;

The downstream business end 403 includes business systems such as OA system, CRM system, and ERP system, as well as data processing systems such as data analysis system, data statistics system, and data display system.

Using the business data management system provided by the embodiments of the present disclosure, by acquiring multiple business data types and the business data corresponding to each business data type, integrating the business data corresponding to each business data type to obtain single-type data, and then analyzing the business data and /or single-type data is processed to obtain legal data or illegal data. If there is illegal data, the repair data corresponding to the illegal data is generated, and then the system data corresponding to each business system is determined from the legal data and/or repair data. In this way, the unification of business data is realized through the integration of business data, and then the quality inspection and repair of business data and single-type data after business data integration are performed, so as to avoid unusable business data or single-type data due to low data quality. situation, thereby improving the interaction efficiency between various business systems.

As shown in FIG. 5 , an embodiment of the present disclosure provides an electronic device, including: a processor (processor) 500 and a memory (memory) 501; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to Make the terminal execute any method in this embodiment. Optionally, the electronic device may further include a communication interface (Communication Interface) 502 and a bus 503 . Wherein, the processor 500 , the communication interface 502 , and the memory 501 can communicate with each other through the bus 503 . Communication interface 502 may be used for information transfer. The processor 500 may invoke logic instructions in the memory 501 to execute the methods in the foregoing embodiments.

Optionally, the electronic device includes an audio device, a computer, a smart phone, a tablet computer, a server, etc., wherein, if the electronic device is an audio device, the above method is executed by a processor (processor) 500 to realize self-testing of the audio device .

In addition, the above logic instructions in the memory 501 may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as an independent product.

The memory 501, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions/modules corresponding to the methods in the embodiments of the present disclosure. The processor 500 executes the program instructions/modules stored in the memory 501 to execute functional applications and data processing, that is, implement the methods in the above-mentioned embodiments.

The memory 501 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created according to the use of the terminal device, and the like. In addition, the memory 501 may include a high-speed random access memory, and may also include a non-volatile memory.

Using the electronic device provided by the embodiments of the present disclosure, by acquiring multiple business data types and the business data corresponding to each business data type, and integrating the business data corresponding to each of the business data types to obtain single-type data, then the business data and /or data processing of single-type data to obtain legal data or illegal data, if there is illegal data, generate repair data corresponding to the illegal data, and then determine system data corresponding to each of the business systems from the legal data and/or repair data . In this way, the unification of business data is realized through the integration of business data, and then the quality inspection and repair of business data and single-type data after business data integration are performed, so as to avoid unusable business data or single-type data due to low data quality. situation, thereby improving the interaction efficiency between various business systems.

An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, any method in the present embodiment is implemented.

As for the computer-readable storage medium in the embodiments of the present disclosure, those skilled in the art can understand that all or part of the steps in the above-mentioned method embodiments can be completed by hardware related to computer programs. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above-mentioned method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

The electronic device disclosed in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver to complete mutual communication. The memory is used to store computer programs, and the communication interface is used to perform For communication, the processor and the transceiver are used to run the computer program to make the electronic equipment perform the steps of the above method.

In this embodiment, the memory may include a random access memory (Random Access Memory, RAM for short), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

Above-mentioned processor can be general-purpose processor, comprises central processing unit (Central Processing Unit, be called for short CPU), graphics processing unit (Graphics Processing Unit, be called for short GPU), network processor (Network Processor, be called for short NP) etc.; Signal Processor (Digital Signal Processing, DSP for short), Application Specific Integrated Circuit (ASIC for short), Field Programmable Gate Array (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gates or Transistor logic devices, discrete hardware components.

The above description and drawings sufficiently illustrate the embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may incorporate structural, logical, electrical, procedural, and other changes. The examples merely represent possible variations. Individual components and functions are optional unless explicitly required, and the order of operations may vary. Portions and subsamples of some embodiments may be included in or replace portions and subsamples of other embodiments. Also, the terms used in the present application are used to describe the embodiments only and are not used to limit the claims. As used in the examples and description of the claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well unless the context clearly indicates otherwise . Similarly, the term "and/or" as used in this application is meant to include any and all possible combinations of one or more of the associated listed ones. Additionally, when used in this application, the term "comprise" and its variants "comprises" and/or comprising (comprising) etc. refer to stated sub-samples, integers, steps, operations, elements, and/or or component, but does not preclude the presence or addition of one or more other sub-samples, integers, steps, operations, elements, components and/or groupings of these. Without further limitations, an element defined by the statement "comprising a ..." does not exclude the presence of additional identical elements in the process, method or apparatus comprising said element. Herein, what each embodiment focuses on may be the difference from other embodiments, and the same and similar parts of the various embodiments may refer to each other. For the method, product, etc. disclosed in the embodiment, if it corresponds to the method part disclosed in the embodiment, then the relevant part can refer to the description of the method part.

Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software may depend on the specific application and design constraints of the technical solution. Said artisans may implement the described functions using different methods for each particular application, but such implementation should not be regarded as exceeding the scope of the disclosed embodiments. The skilled person can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device and unit can be the corresponding process in the aforementioned method embodiment, and will not be repeated here.

In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units may only be a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined Or it can be integrated into another system, or some sub-samples can be ignored, or not implemented. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiments of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to embodiments of the disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or part of code that includes one or more Executable instructions. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than that disclosed in the description, and sometimes there is no specific agreement between different operations or steps. order. For example, two consecutive operations or steps may, in fact, be performed substantially concurrently, or they may sometimes be performed in the reverse order, depending upon the functionality involved. Each block in the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by dedicated hardware implemented in combination with computer instructions.

Claims (8)

1. A business data management method, comprising:

acquiring a service data set from a plurality of service systems, wherein the service data set comprises a plurality of service data types and service data corresponding to each service data type, and integrating at least one part of service data corresponding to each service data type to obtain single type data corresponding to each service data type;

performing data processing on the service data and/or the single type data according to a preset data quality inspection rule to obtain a quality inspection data set, wherein the quality inspection data set comprises legal data and/or illegal data;

if illegal data exist in the quality inspection data set, generating repair data corresponding to the illegal data;

determining system data corresponding to each service system from main data, wherein the main data comprises legal data and/or repair data;

determining target data from main data corresponding to each service data type, fusing the target data based on a preset type fusion algorithm to obtain multi-type data, and adding the multi-type data into the main data;

carrying out data processing on the service data and/or the single type data according to a preset data quality inspection rule, wherein the data processing comprises extracting type characteristic data from the data to be inspected according to the service data type corresponding to the data to be inspected, and the data to be inspected is the service data or the single type data; inputting the type characteristic data into a preset data classification model to obtain a model output result, wherein the data classification model is obtained by training a preset neural network model through characteristic sample data with classification labels; based on the model output result, determining that the data to be inspected is legal data or illegal data;

Generating repair data corresponding to the illegal data, wherein the repair data comprises repairing the illegal data according to a preset repair rule script, and judging whether the illegal data is successfully repaired according to a repair result; if the illegal data is successfully repaired, the repaired illegal data is determined to be the repair data corresponding to the illegal data; if the illegal data is not repaired successfully, acquiring a quality inspection report corresponding to the illegal data, generating a repair request according to the illegal data and the quality inspection report, and transmitting the repair request to a preset client; if a repair instruction corresponding to the repair request is received, determining user update data corresponding to the illegal data according to the repair instruction, reclassifying the user update data according to the data quality inspection rule, and judging whether the user update data is legal data according to a reclassifying result; and if the user updating data is legal data, determining the user updating data as repair data corresponding to the illegal data.

2. The method of claim 1, wherein obtaining a set of business data from a plurality of business systems comprises:

Collecting service data from a plurality of service systems and data source information corresponding to the service data;

distributing service data types to the service data according to the data source information, and determining the service data types and the service data corresponding to the service data types as a service data set;

wherein the service data is collected from the service system by at least one of the following methods,

determining a target system from the service system, and collecting service data from the target system;

respectively and regularly acquiring service data in the service system according to a preset time interval corresponding to the service system;

judging whether the service data of the service system is changed according to the system log corresponding to the service system, and if the service data of the service system is changed, acquiring the service data from the service system;

and connecting the new system through a preset connection mode, determining the new system as a new service system, and collecting service data from the new service system.

3. The method of claim 1, wherein after generating the repair data corresponding to the illegal data, before determining the system data corresponding to each service system from the main data, the method further comprises:

Determining sensitive data from the main data according to a preset sensitive data list, generating an interception request according to the sensitive data, and sending the interception request to a preset client;

if an interception instruction corresponding to the interception request is received, determining whether interception data exists in the sensitive data according to the interception instruction;

and if the interception data exist in the sensitive data, deleting the interception data from the main data.

4. The method of claim 1, further comprising at least one of:

and adding reference data into the main data, wherein the reference data comprises at least one of administrative codes, company information, industry information and user side information.

5. The method of claim 1, wherein determining system data corresponding to each of the service systems from the master data, further comprises at least one of:

before system data corresponding to each service system are determined from main data, data identification is allocated to the main data, and the main data are encoded according to a preset encoding format;

after determining system data corresponding to each service system from the main data, configuring a data acquisition mode for each service system, and sending the system data to the corresponding service system according to the corresponding data acquisition mode.

6. A business data management system, comprising:

the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a service data set from a plurality of service systems, the service data set comprises a plurality of service data types and service data corresponding to each service data type, and at least one part of service data corresponding to each service data type is respectively integrated to obtain single type data corresponding to each service data type;

the classification module is used for classifying the input data to be inspected according to a preset data quality inspection rule to obtain legal data and/or illegal data, wherein the data to be inspected comprises the service data and/or the single type data;

the generating module is used for generating repair data corresponding to illegal data, and taking the repair data as new data to be inspected;

the determining module is used for determining system data corresponding to each service system from main data, wherein the main data comprises legal data obtained by the quality inspection module;

determining target data from main data corresponding to each service data type, fusing the target data based on a preset type fusion algorithm to obtain multi-type data, and adding the multi-type data into the main data;

Carrying out data processing on the service data and/or the single type data according to a preset data quality inspection rule, wherein the data processing comprises extracting type characteristic data from the data to be inspected according to the service data type corresponding to the data to be inspected, and the data to be inspected is the service data or the single type data; inputting the type characteristic data into a preset data classification model to obtain a model output result, wherein the data classification model is obtained by training a preset neural network model through characteristic sample data with classification labels; based on the model output result, determining that the data to be inspected is legal data or illegal data;

generating repair data corresponding to the illegal data, wherein the repair data comprises repairing the illegal data according to a preset repair rule script, and judging whether the illegal data is successfully repaired according to a repair result; if the illegal data is successfully repaired, the repaired illegal data is determined to be the repair data corresponding to the illegal data; if the illegal data is not repaired successfully, acquiring a quality inspection report corresponding to the illegal data, generating a repair request according to the illegal data and the quality inspection report, and transmitting the repair request to a preset client; if a repair instruction corresponding to the repair request is received, determining user update data corresponding to the illegal data according to the repair instruction, reclassifying the user update data according to the data quality inspection rule, and judging whether the user update data is legal data according to a reclassifying result; and if the user updating data is legal data, determining the user updating data as repair data corresponding to the illegal data.

7. An electronic device, comprising: a processor and a memory;

the memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory, to cause the electronic device to perform the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized by:

the computer program implementing the method according to any of claims 1 to 5 when executed by a processor.