CN113961565A - Data detection method, system, computer system and readable storage medium - Google Patents
Data detection method, system, computer system and readable storage medium Download PDFInfo
- Publication number
- CN113961565A CN113961565A CN202111302699.3A CN202111302699A CN113961565A CN 113961565 A CN113961565 A CN 113961565A CN 202111302699 A CN202111302699 A CN 202111302699A CN 113961565 A CN113961565 A CN 113961565A
- Authority
- CN
- China
- Prior art keywords
- data
- detection
- tested
- value
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The present disclosure provides a data detection method, including: responding to a data detection request, determining a plurality of data to be detected corresponding to equipment to be detected and a data grade of each data to be detected, wherein the data to be detected are stored in a database, the data grades comprise a core grade, an important grade and a common grade, and each data grade corresponds to a weight value; detecting each data to be detected according to a target detection rule group, and outputting a first detection value of each data to be detected, wherein each data to be detected corresponds to the target detection rule group, and the target detection rule group comprises at least one target detection rule; and generating a second detection value of the equipment to be detected according to the first detection value and the weight value corresponding to each piece of data to be detected, so as to determine the data quality of the data to be detected corresponding to the equipment to be detected according to the second detection value. The present disclosure also provides a data detection system, a computer system and a readable storage medium.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a data detection method, a data detection system, a computer system, a readable storage medium, and a computer program.
Background
A Configuration Management Database (CMDB) is a logical Database containing information of the full life cycle of Configuration items and relationships between Configuration items (including physical relationships, real-time communication relationships, non-real-time communication relationships, and dependency relationships).
In implementing the disclosed concept, the inventors found that there are at least the following problems in the related art: as the data in the database is gradually increased, the quality of the data needs to be verified, but in the related art, when the data is verified, the verification efficiency is low, and the accuracy of the verification result is poor.
Disclosure of Invention
In view of the above, the present disclosure provides a data detection method, system, computer system, readable storage medium, and computer program.
One aspect of the present disclosure provides a data detection method, including:
responding to a data detection request, determining a plurality of data to be detected corresponding to equipment to be detected and a data level of each data to be detected, wherein the data to be detected are stored in a database, the data levels comprise a core level, an important level and a common level, each data level corresponds to a weight value, the weight value of the core level is greater than that of the important level, and the weight value of the important level is greater than that of the common level;
detecting each data to be detected according to a target detection rule group, and outputting a first detection value of each data to be detected, wherein each data to be detected corresponds to the target detection rule group, and the target detection rule group comprises at least one target detection rule;
and generating a second detection value of the equipment to be detected according to the first detection value and the weight value corresponding to each piece of data to be detected, so as to determine the data quality of the data to be detected corresponding to the equipment to be detected according to the second detection value.
According to an embodiment of the present disclosure, the data detection method further includes:
acquiring the data type of the data to be detected;
determining at least one target detection rule in a plurality of detection rules according to the data type of each piece of data to be detected;
and generating the target detection rule group corresponding to each piece of data to be detected according to each target detection rule.
According to an embodiment of the present disclosure, the detection rule includes at least one of: null determination, digit detection, range detection, date detection, canonical detection, and dictionary detection.
According to an embodiment of the present disclosure, the determining a plurality of pieces of data to be tested corresponding to devices to be tested includes:
acquiring device data associated with the device to be tested;
and screening the equipment data according to a preset screening rule, and determining the plurality of data to be tested corresponding to the equipment.
According to an embodiment of the present disclosure, the filtering rules include rules that filter the device data according to frequency of use, data source, and data content.
According to an embodiment of the present disclosure, generating the second detection value of the device under test according to the first detection value and the weight value corresponding to each piece of the data under test includes:
respectively generating a core data detection value, an important data detection value and a common data detection value corresponding to each data level according to the first detection value and the weight value by a preset method;
generating the second detection value from the core data detection value, the important data detection value, and the normal data detection value.
According to an embodiment of the present disclosure, the data detection method further includes:
and generating a statistical form according to the first detection values of the data to be detected so as to display the detection results of the data to be detected according to the statistical form, wherein the statistical form comprises the detection statistical results corresponding to each type of data group to be detected.
According to an embodiment of the present disclosure, the data detection method further includes:
acquiring the second detection values of a plurality of devices under test associated with the database;
and generating a data detection result of all the data stored in the database according to the plurality of second detection values.
Another aspect of the present disclosure provides a data detection system, including:
the response module is used for responding to a data detection request, determining a plurality of data to be detected corresponding to equipment to be detected and a data level of each data to be detected, wherein the data to be detected are stored in a database, the data levels comprise a core level, an important level and a common level, each data level corresponds to a weight value, the weight value of the core level is greater than that of the important level, and the weight value of the important level is greater than that of the common level;
the detection module is used for detecting each data to be detected according to a target detection rule group and outputting a first detection value of each data to be detected, wherein each data to be detected corresponds to the target detection rule group, and the target detection rule group comprises at least one target detection rule;
the first generation module is used for generating a second detection value of the equipment to be detected according to the first detection value and the weight value corresponding to each piece of data to be detected, so that the data quality of a plurality of pieces of data to be detected corresponding to the equipment to be detected can be determined according to the second detection value.
Another aspect of the present disclosure provides a computer system comprising: one or more processors; memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the methods of embodiments of the present disclosure.
Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions for implementing the method of the embodiments of the present disclosure when executed.
Another aspect of the present disclosure provides a computer program comprising computer executable instructions for implementing the method of the embodiments of the present disclosure when executed.
According to the embodiment of the disclosure, a plurality of pieces of data to be detected corresponding to the equipment to be detected and the data grade of each piece of data to be detected are determined in response to the data detection request, and each piece of data to be detected is detected according to the target detection rule set, so that a first detection value of each piece of data to be detected is obtained. And then obtaining a second detection value of the equipment to be detected according to the first detection value and the weight value corresponding to the data level of the data to be detected. Because the weighted value of the data to be detected is determined by the data grade, the data to be detected is graded, so that the data to be detected is more consistent with the scene of data use, and the efficiency of data detection and the accuracy of the detection result can be effectively improved.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:
fig. 1 schematically illustrates an exemplary system architecture 100 to which a data detection method may be applied, according to an embodiment of the present disclosure.
Fig. 2 schematically shows a flow chart of a data detection method according to an embodiment of the present disclosure.
Fig. 3 schematically illustrates a flowchart of a method for determining a target detection rule set according to an embodiment of the present disclosure.
Fig. 4 schematically shows a schematic diagram of a data detection method according to an embodiment of the present disclosure.
Fig. 5 schematically illustrates a block diagram of a data detection system according to an embodiment of the present disclosure.
Fig. 6 schematically shows a block diagram of a computer system suitable for implementing the above described method according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
A Configuration Management Database (CMDB) is a logical Database containing information of the full life cycle of Configuration items and relationships between Configuration items (including physical relationships, real-time communication relationships, non-real-time communication relationships, and dependency relationships).
The CMDB plays a crucial role in controlling and managing a large number of and a large variety of IT (Internet Technology) devices and IT services, but the data information in the CMDB is not perfect, which has an effect on controlling, maintaining, accident procedures, and changing procedures of the IT devices and the IT services.
Taking the absence of the accessory information of the device as an example, the overall performance of the device cannot be known in this case, and the state of the device cannot be accurately monitored. Moreover, when the accessories are abnormal and need to be replaced, the operation and maintenance cannot be implemented without specific parameters and models of the accessories. The quality of the CMDB data is verified, and the integrity, accuracy, legality and the like of the current CMDB data are judged, so that a basis can be provided for improving the service quality and efficiency of the CMDB.
In the related technology, when the quality of CMDB data is checked, the integrity and the accuracy of the data are detected by checking whether fields in a database table are missing or legal.
In implementing the disclosed concept, the inventors found that there are at least the following problems in the related art: in the related art, when data verification is performed, the verification efficiency is low, and the accuracy of the verification result is poor.
Embodiments of the present disclosure provide a data detection method, system, computer system, readable storage medium, and computer program. The data detection method comprises the following steps: responding to a data detection request, determining a plurality of data to be detected corresponding to equipment to be detected and a data level of each data to be detected, wherein the data to be detected are stored in a database, the data levels comprise a core level, an important level and a common level, each data level corresponds to a weight value, the weight value of the core level is greater than that of the important level, and the weight value of the important level is greater than that of the common level; detecting each data to be detected according to a target detection rule group, and outputting a first detection value of each data to be detected, wherein each data to be detected corresponds to the target detection rule group, and the target detection rule group comprises at least one target detection rule; and generating a second detection value of the equipment to be detected according to the first detection value and the weight value corresponding to each piece of data to be detected, so as to determine the data quality of the data to be detected corresponding to the equipment to be detected according to the second detection value.
Fig. 1 schematically illustrates an exemplary system architecture 100 to which a data detection method may be applied, according to an embodiment of the present disclosure. It should be noted that fig. 1 is only an example of a system architecture to which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, and does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios.
As shown in fig. 1, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired and/or wireless communication links, and so forth.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have installed thereon various communication client applications, such as a shopping-like application, a web browser application, a search-like application, an instant messaging tool, a mailbox client, and/or social platform software, etc. (by way of example only).
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (for example only) providing support for websites browsed by users using the terminal devices 101, 102, 103. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that the data detection method provided by the embodiment of the present disclosure may be generally executed by the server 105. Accordingly, the data detection system provided by the embodiments of the present disclosure may be generally disposed in the server 105. The data detection method provided by the embodiment of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the data detection system provided by the embodiment of the present disclosure may also be disposed in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Alternatively, the data detection method provided by the embodiment of the present disclosure may also be executed by the terminal device 101, 102, or 103, or may also be executed by another terminal device different from the terminal device 101, 102, or 103. Accordingly, the data detection system provided by the embodiment of the present disclosure may also be disposed in the terminal device 101, 102, or 103, or in another terminal device different from the terminal device 101, 102, or 103.
For example, the data detection request may be obtained by any one of the terminal devices 101, 102, or 103 (for example, the terminal device 101 is not limited thereto), and then the terminal device 101 may locally perform the data detection method provided by the embodiment of the present disclosure, or send the data detection request to be processed to another terminal device, server, or server cluster, and perform the data detection method provided by the embodiment of the present disclosure by another terminal device, server, or server cluster that receives the data detection request.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Fig. 2 schematically shows a flow chart of a data detection method according to an embodiment of the present disclosure.
As shown in fig. 2, the method includes operations S201 to S203.
In operation S201, in response to a data detection request, a plurality of data to be detected corresponding to a device to be detected and a data level of each data to be detected are determined, where the plurality of data to be detected are stored in a database, the data levels include a core level, an important level, and a normal level, each data level corresponds to a weight value, the weight value of the core level is greater than that of the important level, and the weight value of the important level is greater than that of the normal level.
According to an embodiment of the present disclosure, a device under test may refer to, for example, an IT device associated with a database, which may include, for example, a computer, a router, and the like. The application does not limit the types of the equipment to be tested. The database may include, for example, CMDB or the like.
According to the embodiment of the present disclosure, the data level of each data to be measured may be divided into a core level, an importance level, and a normal level according to a level division method. Each data level may also correspond to a different weight value, for example, the weight values for the core level, importance level, and normal level may be 5: 3: 2. Further, the data level of the data to be tested is not limited to the core level, the importance level and the normal level, and may include any number of data levels.
The data level of the data to be measured can be determined according to factors such as the use frequency of the data, the feedback information of the external interface, the data attribute and the like. For example, the data to be tested in the system may be sorted according to the frequency of use, and the data level of the data to be tested is divided into a core level, an important level and a normal level by setting a proper threshold; or acquiring error reporting information of the external interface, and dividing the data level of the data to be tested associated with the error reporting information into a core level, an important level and a common level according to the emergency degree of the error reporting information.
According to an embodiment of the present disclosure, the data detection request may be generated according to a trigger operation of the user, and the trigger operation may include a click or a slide. Further, the data detection request may also be generated according to a trigger condition of the data detection request, for example, the trigger condition may include a preset time threshold and the like.
According to the embodiment of the disclosure, the data detection efficiency can be improved by classifying the to-be-detected data, all fields in the table do not need to be checked, and the check logic is executed in a targeted manner according to the importance degree of the data.
In operation S202, each data to be detected is detected according to a target detection rule set, and a first detection value of each data to be detected is output, where each data to be detected corresponds to a target detection rule set, and the target detection rule set includes at least one target detection rule.
According to the embodiment of the disclosure, the first detection value may be used to characterize whether the data to be detected passes the inspection, for example. The first detection value may be represented by a number, for example, the first detection value may include "0" and "1", where "0" is used to represent that the data to be detected does not pass the detection, and "1" is used to represent that the data to be detected passes the detection.
According to the embodiments of the present disclosure, the target detection rule set may include one target detection rule, or may include a plurality of target detection rules. When a plurality of target detection rules are included in the target detection rule group, each target detection rule is different.
In operation S203, a second detection value of the device under test is generated according to the first detection value and the weight value corresponding to each piece of data under test, so as to determine data quality of a plurality of pieces of data under test corresponding to the device under test according to the second detection value.
According to the embodiment of the disclosure, the second detection value of the device to be detected can be obtained according to the first detection value of each piece of data to be detected corresponding to the device to be detected and the weight value of each piece of data to be detected. The data quality of the multiple pieces of data to be tested of the device to be tested can be determined according to the second detection value, for example, the larger the second detection value is, the higher the data quality is.
According to the embodiment of the disclosure, a plurality of pieces of data to be detected corresponding to the equipment to be detected and the data grade of each piece of data to be detected are determined in response to the data detection request, and each piece of data to be detected is detected according to the target detection rule set, so that a first detection value of each piece of data to be detected is obtained. And then obtaining a second detection value of the equipment to be detected according to the first detection value and the weight value corresponding to the data level of the data to be detected. Because the weighted value of the data to be detected is determined by the data grade, the data to be detected is graded, so that the data to be detected is more consistent with the scene of data use, and the efficiency of data detection and the accuracy of the detection result can be effectively improved.
Fig. 3 schematically illustrates a flowchart of a method for determining a target detection rule set according to an embodiment of the present disclosure.
As shown in fig. 3, the method includes operations S301 to S303.
In operation S301, a data type of data to be measured is acquired.
In operation S302, at least one target detection rule is determined among a plurality of detection rules according to a data type of each data to be detected.
In operation S303, a target detection rule group corresponding to each data to be detected is generated according to each target detection rule.
According to the embodiment of the present disclosure, the data type of the data to be measured may be divided by the content and form of the data, for example, may be divided into time type data or the like according to the content of the data.
According to an embodiment of the present disclosure, the detection rule may be generated, for example, by being configured in advance, and the generated detection rule is stored in a rule base. Further, the detection rule may include, for example, null judgment, number check, range check, date check, regular check, dictionary check, and the like.
Different data types may be detected, for example, according to different detection rules. For example, when the date lamp data is verified, the data can be detected by a detection rule in the third of null value judgment, date verification and range verification. Wherein the control judgment can detect whether the data is a null value; the date check may detect whether the data format of the data is date format data, for example; the range check may, for example, detect whether the data exceeds a preset time range.
According to the embodiment of the disclosure, when data is detected, different detection rules can be combined and matched for use, so that the multiplexing of the detection rules is realized, and the effect of the application range of the detection rules is improved. Furthermore, the rule is adaptive to any type of data verification through the combination of the rules, and the application range and the repeated utilization rate of the rules are improved. According to an embodiment of the present disclosure, determining a plurality of data under test corresponding to a device under test includes:
device data associated with a device under test is acquired. And screening the equipment data according to a preset screening rule, and determining a plurality of data to be tested corresponding to the equipment.
When the database service is developed to a certain stage, the database tables are increased, and the data volume is also accumulated greatly. In this case, since the effect of some attribute data on the business is not important, it is inefficient to check the fields in the database table, and it is difficult to determine the quality of the data as a whole.
According to embodiments of the present disclosure, the device data may include, for example, data associated with the device under test in a database and data in an external interface associated with the device under test. Further, the device data may further include subjective definition data of the user, and the supervisor definition data may include remark data and the like, for example.
According to an embodiment of the present disclosure, the screening rules include rules that screen the device data according to frequency of use, data source, and data content.
According to the embodiment of the disclosure, when the device data is screened, the screening can be performed according to the use frequency of the device data, for example, the device data with lower use frequency can be filtered in the database; the device data may also be filtered according to the data source, for example, the data from the external interface may be determined as the data to be tested, etc. Furthermore, the screening can be performed in a subjective definition manner of the user, for example, remark data of the user to the device can be determined as data to be tested, and the like.
According to an embodiment of the present disclosure, generating a second detection value of the device under test according to the first detection value and the weight value corresponding to each data under test includes:
and respectively generating a core data detection value, an important data detection value and a common data detection value corresponding to each data level according to the first detection value and the weight value by a preset method. A second detection value is generated based on the core data detection value, the important data detection value, and the normal data detection value.
According to an embodiment of the present disclosure, the preset method may include, for example: and respectively calculating a core data detection value, an important data detection value and a common data detection value according to the weight value corresponding to each data level and the first detection value of each data to be detected in each data level.
For example, the core data detection value may be calculated according to equation (1):
M1＝A1×A2×…×An (1)
where a1 to An may represent the first detection value of the core data, and α may represent the weight value of the core data.
The important data detection value can be calculated according to the formula (2):
where B1 to Bn may represent the first detection value of the important data, β may represent the weight value of the important data, and Σ B may represent the number of the important data. The common data detection value can be calculated according to the formula (3):
where C1 to Cn may represent the first detection value of the normal data, γ may represent a weight value of the normal data, and Σ C may represent the number of the normal data.
For example, according to the data levels, the detection values of different data levels may be calculated first, and then the second detection value may be generated based on the obtained detection values.
The second detection value may be calculated, for example, according to equation (4):
where a1 to An may represent a first detected value of core data, α may represent a weight value of the core data, B1 to Bn may represent a first detected value of important data, β may represent a weight value of important data, Σ B may represent the number of important data, C1 to Cn may represent a first detected value of normal data, γ may represent a weight value of normal data, and Σ C may represent the number of normal data.
According to the embodiment of the disclosure, if the first detection value of one important data in the data to be detected of a certain device to be detected does not pass, the first detection value of the important data is 0. The second detection value of the device to be detected is in direct proportion to the data quality of the device to be detected, and the higher the second detection value is, the higher the data quality of the device to be detected is.
According to an embodiment of the present disclosure, the data detection method further includes:
and generating a statistical form according to the first detection values of the data to be detected so as to display the detection results of the data to be detected according to the statistical form, wherein the statistical form comprises the detection statistical results corresponding to each data group to be detected.
According to the embodiment of the disclosure, according to specific implementation requirements, the first detection value of each data to be detected can be counted, and a statistical report is generated so that a user can analyze the first detection value. Furthermore, when the first detection value is counted, classified counting can be performed according to the data level of each data to be detected, so that the information displayed by the statistical table is more comprehensive.
According to an embodiment of the present disclosure, the data detection method further includes:
a second test value of the plurality of devices under test associated with the database is obtained. A data detection result of all the data stored in the database is generated based on the plurality of second detection values.
According to the embodiment of the disclosure, after the data to be detected of each device to be detected in the database is detected, the second detection value of each device to be detected is obtained, and the data quality in the database can be obtained according to each second detection value.
For example, the score of each device under test may be averaged to obtain a data quality score for the entire database. The score of the formula is compared to the standard score to derive the quality of the data in the library. The closer the data is to the standard score, the better the data quality is represented. The standard score may be set according to specific implementation requirements, and the disclosure is not limited in this respect.
Fig. 4 schematically shows a schematic diagram of a data detection method according to an embodiment of the present disclosure.
As shown in fig. 4, the method includes operations S401 to S405.
In operation S401, the data in the database is screened to determine the data to be tested. In operation S402, a target detection rule is determined in a rule base according to data to be detected, and a target detection rule group is generated. In operation S403, the data to be detected is detected using the target detection rule set. In operation S404, after the detection result is obtained, the detection result is stored. In operation S405, a statistical form is generated according to the detection result and displayed.
Fig. 5 schematically illustrates a block diagram of a data detection system according to an embodiment of the present disclosure.
As shown in FIG. 5, the data detection system 500 may include a response module 501, a detection module 502, and a first generation module 503.
The response module 501 is configured to determine, in response to a data detection request, a plurality of pieces of data to be detected corresponding to a device to be detected and a data level of each piece of data to be detected, where the plurality of pieces of data to be detected are stored in a database, the data levels include a core level, an important level, and a normal level, each data level corresponds to a weight value, the weight value of the core level is greater than the weight value of the important level, and the weight value of the important level is greater than the weight value of the normal level.
The detecting module 502 is configured to detect each data to be detected according to a target detection rule set, and output a first detection value of each data to be detected, where each data to be detected corresponds to a target detection rule set, and the target detection rule set includes at least one target detection rule.
A first generating module 503, configured to generate a second detection value of the device under test according to the first detection value and the weight value corresponding to each piece of data under test, so as to determine data quality of the plurality of pieces of data under test corresponding to the device under test according to the second detection value.
According to the embodiment of the disclosure, a plurality of pieces of data to be detected corresponding to the equipment to be detected and the data grade of each piece of data to be detected are determined in response to the data detection request, and each piece of data to be detected is detected according to the target detection rule set, so that a first detection value of each piece of data to be detected is obtained. And then obtaining a second detection value of the equipment to be detected according to the first detection value and the weight value corresponding to the data level of the data to be detected. Because the weighted value of the data to be detected is determined by the data grade, the data to be detected is graded, so that the data to be detected is more consistent with the scene of data use, and the efficiency of data detection and the accuracy of the detection result can be effectively improved.
According to an embodiment of the present disclosure, the data detection system 500 further includes a first obtaining module, a determining module, and a second generating module.
The first acquisition module is used for acquiring the data type of the data to be detected.
And the determining module is used for determining at least one target detection rule in the plurality of detection rules according to the data type of each piece of data to be detected.
And the second generation module is used for generating a target detection rule group corresponding to each data to be detected according to each target detection rule.
According to an embodiment of the present disclosure, the detection rule includes at least one of: null determination, digit detection, range detection, date detection, canonical detection, and dictionary detection.
According to an embodiment of the present disclosure, the response module 501 includes an obtaining unit and a screening unit.
An acquisition unit for acquiring device data associated with a device under test.
And the screening unit is used for screening the equipment data according to a preset screening rule and determining a plurality of data to be tested corresponding to the equipment.
According to an embodiment of the present disclosure, the screening rules include rules that screen the device data according to frequency of use, data source, and data content.
According to an embodiment of the present disclosure, the first generating module 503 includes a first generating unit and a second generating unit.
And a first generation unit for generating a core data detection value, an important data detection value, and a normal data detection value corresponding to each data level, respectively, by a preset method.
A second generation unit for generating a second detection value based on the core data detection value, the important data detection value, and the normal data detection value.
According to an embodiment of the present disclosure, the data detection system 500 further comprises a third generation module.
And the third generation module is used for generating a statistical form according to the first detection values of the data to be detected so as to display the detection results of the data to be detected according to the statistical form, wherein the statistical form comprises the detection statistical results corresponding to each data group to be detected.
According to an embodiment of the present disclosure, the data detection system 500 further includes a second obtaining module and a fourth generating module.
And the second acquisition module is used for acquiring second detection values of the plurality of devices to be detected associated with the database.
And the fourth generation module is used for generating a data detection result of all the data stored in the database according to the plurality of second detection values.
It should be noted that, the embodiments of the system part of the present disclosure correspond to the same or similar embodiments of the method part of the present disclosure, and the detailed description of the present disclosure is omitted here.
Any number of modules, sub-modules, units, or at least part of the functionality of any number thereof according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units according to the embodiments of the present disclosure may be implemented by being split into a plurality of modules. Any one or more of the modules, sub-modules, units according to the embodiments of the present disclosure may be implemented at least partially as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in any other reasonable manner of hardware or firmware by integrating or packaging the circuit, or in any one of three implementations of software, hardware, and firmware, or in any suitable combination of any of them. Alternatively, one or more of the modules, sub-modules, units according to embodiments of the disclosure may be implemented at least partly as computer program modules, which, when executed, may perform corresponding functions.
For example, any plurality of the response module 501, the detection module 502 and the first generation module 503 may be combined and implemented in one module/unit/sub-unit, or any one of the modules/units/sub-units may be split into a plurality of modules/units/sub-units. Alternatively, at least part of the functionality of one or more of these modules/units/sub-units may be combined with at least part of the functionality of other modules/units/sub-units and implemented in one module/unit/sub-unit. According to an embodiment of the present disclosure, at least one of the response module 501, the detection module 502, and the first generation module 503 may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or may be implemented by any one of or a suitable combination of software, hardware, and firmware. Alternatively, at least one of the response module 501, the detection module 502 and the first generation module 503 may be at least partly implemented as a computer program module, which when executed may perform a corresponding function.
Fig. 6 schematically shows a block diagram of a computer system suitable for implementing the above described method according to an embodiment of the present disclosure. The computer system illustrated in FIG. 6 is only one example and should not impose any limitations on the scope of use or functionality of embodiments of the disclosure.
As shown in fig. 6, a computer system 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. Processor 601 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 601 may also include onboard memory for caching purposes. Processor 601 may include a single processing unit or multiple processing units for performing different actions of a method flow according to embodiments of the disclosure.
In the RAM603, various programs and data necessary for the operation of the computer system 600 are stored. The processor 601, the ROM602, and the RAM603 are connected to each other via a bus 604. The processor 601 performs various operations of the method flows according to the embodiments of the present disclosure by executing programs in the ROM602 and/or RAM 603. It is to be noted that the programs may also be stored in one or more memories other than the ROM602 and RAM 603. The processor 601 may also perform various operations of the method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the present disclosure, computer system 600 may also include an input/output (I/O) interface 605, input/output (I/O) interface 605 also connected to bus 604. The system 600 may also include one or more of the following components connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
According to embodiments of the present disclosure, method flows according to embodiments of the present disclosure may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program, when executed by the processor 601, performs the above-described functions defined in the system of the embodiments of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples may include, but are not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM602 and/or RAM603 described above and/or one or more memories other than the ROM602 and RAM 603.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.
The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.
Claims (11)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111302699.3A CN113961565A (en) | 2021-11-04 | 2021-11-04 | Data detection method, system, computer system and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111302699.3A CN113961565A (en) | 2021-11-04 | 2021-11-04 | Data detection method, system, computer system and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113961565A true CN113961565A (en) | 2022-01-21 |
Family
ID=79469143
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111302699.3A Pending CN113961565A (en) | 2021-11-04 | 2021-11-04 | Data detection method, system, computer system and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113961565A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114595216A (en) * | 2022-03-21 | 2022-06-07 | 京东科技信息技术有限公司 | Data verification method, device, storage medium and electronic device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105701199A (en) * | 2016-01-08 | 2016-06-22 | 广东电网有限责任公司信息中心 | Method and device for detecting data quality of data dependence |
| CN110737650A (en) * | 2019-09-27 | 2020-01-31 | 北京明略软件系统有限公司 | Data quality detection method and device |
| CN110866259A (en) * | 2019-11-14 | 2020-03-06 | 杭州安恒信息技术股份有限公司 | Method and system for calculating potential safety hazard score based on multi-dimensional data |
| CN112686527A (en) * | 2020-12-28 | 2021-04-20 | 山东众阳健康科技集团有限公司 | Service data quality checking method and system |
| US20210144167A1 (en) * | 2018-07-20 | 2021-05-13 | Huawei Technologies Co., Ltd. | Apparatus and method for detecting an anomaly in a dataset and computer program product therefor |
| CN113469589A (en) * | 2021-09-03 | 2021-10-01 | 阿里云计算有限公司 | Carbon emission control method, level detection method, computing device and medium |
-
2021
- 2021-11-04 CN CN202111302699.3A patent/CN113961565A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105701199A (en) * | 2016-01-08 | 2016-06-22 | 广东电网有限责任公司信息中心 | Method and device for detecting data quality of data dependence |
| US20210144167A1 (en) * | 2018-07-20 | 2021-05-13 | Huawei Technologies Co., Ltd. | Apparatus and method for detecting an anomaly in a dataset and computer program product therefor |
| CN110737650A (en) * | 2019-09-27 | 2020-01-31 | 北京明略软件系统有限公司 | Data quality detection method and device |
| CN110866259A (en) * | 2019-11-14 | 2020-03-06 | 杭州安恒信息技术股份有限公司 | Method and system for calculating potential safety hazard score based on multi-dimensional data |
| CN112686527A (en) * | 2020-12-28 | 2021-04-20 | 山东众阳健康科技集团有限公司 | Service data quality checking method and system |
| CN113469589A (en) * | 2021-09-03 | 2021-10-01 | 阿里云计算有限公司 | Carbon emission control method, level detection method, computing device and medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114595216A (en) * | 2022-03-21 | 2022-06-07 | 京东科技信息技术有限公司 | Data verification method, device, storage medium and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10171335B2 (en) | Analysis of site speed performance anomalies caused by server-side issues | |
| US9807116B2 (en) | Methods and apparatus to identify priorities of compliance assessment results of a virtual computing environment | |
| CN109543891B (en) | Method and apparatus for establishing capacity prediction model, and computer-readable storage medium | |
| CN110471821A (en) | Abnormal alteration detection method, server and computer readable storage medium | |
| CN112181767A (en) | Software system abnormality determination method, device and storage medium | |
| CN113392010B (en) | Public assembly testing method and device, electronic equipment and storage medium | |
| CN113051183B (en) | A test data recommendation method, system, electronic device and storage medium | |
| CN110245684B (en) | Data processing method, electronic device, and medium | |
| CN111158926B (en) | Service request analysis method, device and equipment | |
| CN110888811B (en) | Code coverage rate information processing method and device, electronic equipment and medium | |
| US11797366B1 (en) | Identifying a root cause of an error | |
| CN108804501A (en) | A kind of method and device of detection effective information | |
| CN113961565A (en) | Data detection method, system, computer system and readable storage medium | |
| CN105893237A (en) | Test data driving method and equipment | |
| CN111858287A (en) | Code performance evaluation method and device, electronic device and storage medium | |
| CN114356703A (en) | Root cause analysis method and device | |
| CN114090407A (en) | Interface performance early warning method and related equipment based on linear regression model | |
| CN108376155A (en) | A method and device for determining geographic location information | |
| CN116578911A (en) | Data processing method, device, electronic device and computer storage medium | |
| KR102464688B1 (en) | Method and apparatus for detrmining event level of monitoring result | |
| CN114579398A (en) | Log storage method, device, equipment and storage medium | |
| CN108536604B (en) | Method and terminal for testing response time of WEB page | |
| CN113051128B (en) | Power consumption detection method and device, electronic equipment and storage medium | |
| US12323448B2 (en) | System and method for identifying security threats based on compliance failures and infrastructure activity | |
| CN112884497A (en) | Method and device for determining user type, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |