CN113064890A - Quality evaluation method, device, server and medium for operator data - Google Patents
Quality evaluation method, device, server and medium for operator data Download PDFInfo
- Publication number
- CN113064890A CN113064890A CN202110367597.3A CN202110367597A CN113064890A CN 113064890 A CN113064890 A CN 113064890A CN 202110367597 A CN202110367597 A CN 202110367597A CN 113064890 A CN113064890 A CN 113064890A
- Authority
- CN
- China
- Prior art keywords
- data
- cluster
- files
- base stations
- file storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Factory Administration (AREA)
Abstract
本发明公开了一种运营商数据的质量评估方法、装置、服务器及介质，包括：采集数据源、接口机、集群、共享服务器环节的MR数据；判断数据源、接口机和集群的文件个数、文件存储大小三者是否相等，上报基站数量是否与工参基站数量相等；若文件个数的偏差大于第一预设阈值，则发出告警信息，若文件存储大小的偏差大于第二预设阈值，则发出告警信息，若上报基站数量少于工参基站数量的差值超过第三预设阈值，则发出告警信息；根据是否发出告警信息以及告警信息的类型次数分析运营商的数据是否存在异常、出现异常的环节以及各个判别异常的原因，并输出数据质量评估结果。本发明的数据质量监控，实现监控全程化、规则配置化和检测实时化。
The invention discloses a quality evaluation method, device, server and medium for operator data, including: collecting MR data of data source, interface machine, cluster and shared server links; judging the number of files of data source, interface machine and cluster , whether the file storage size is equal, and whether the number of reported base stations is equal to the number of industrial parameter base stations; if the deviation of the number of files is greater than the first preset threshold, an alarm message will be sent, and if the deviation of the file storage size is greater than the second preset threshold , then send out alarm information, if the difference between the number of reported base stations less than the number of industrial parameter base stations exceeds the third preset threshold, then send out alarm information; according to whether the alarm information is sent and the type and number of alarm information, analyze whether there is any abnormality in the operator's data , the abnormal link and the reasons for each abnormal identification, and output the data quality evaluation results. The data quality monitoring of the present invention realizes whole-process monitoring, rule configuration and real-time detection.
Description
Technical Field
The embodiment of the invention relates to the field of data processing, in particular to a method, a device, a server and a medium for evaluating the quality of operator data.
Background
Information resources, which are strategic resources and production elements, are becoming the basis for the normal operation of enterprises, and abnormal data such as missing, error and the like necessarily produce wrong or inaccurate processing results, so that wrong or inaccurate decisions are made, and serious or even fatal consequences can be caused to the enterprises. Thus, data quality appears to be critical to the enterprise.
The data quality assurance work currently has the following problems:
(1) the data source is complex, the integrity is difficult to guarantee and identify
A. The data volume is large: the operator data is in the amount of ten thousand G per day. B. The data is of a wide variety: the operator data comprises O-domain wireless data, perception DPI data, fixed network DPI data, mobile internet logs, B-domain data, M-domain data and the like. C. The data sources are many: the operator data source comprises OMC equipment, SCA equipment, core network equipment, provincial terminal unified gateway equipment and the like. Based on the situations of large data quantity, multiple types and multiple data sources, the integrity of the data is difficult to guarantee and identify.
(2) Long data processing chain and low problem positioning efficiency
The data processing chain is long, and comprises a data source link, a transmission link, an acquisition link, a processing link, a warehousing link, a shared interface link and the like, the whole data processing process is closed, data is difficult to locate when a problem occurs, the links with the problem cannot be rapidly identified, and an automatic early warning mechanism is not provided. Most of the methods rely on manual positioning and screening, the operation and maintenance cost is high, the automation degree is low, and the current situation of high cost and low efficiency of maintenance is caused.
(3) Lack of standardized and platformized data quality detection system
The whole operator data lacks a standardized data quality monitoring guarantee system, data quality detection points are configured, the whole-process monitoring of the data quality is supported, a comprehensive and timely data quality report is provided, and configured alarm setting is provided.
Disclosure of Invention
The embodiment of the invention provides a quality evaluation method, a quality evaluation device, a quality evaluation server and a quality evaluation medium for operator data, so as to realize whole monitoring, rule configuration and real-time detection.
In a first aspect, an embodiment of the present invention provides a method for evaluating quality of operator data, including:
the method comprises the following steps of firstly, acquiring MR data of a data source link, an interface machine link, a cluster link and a shared server link, wherein the MR data comprises: A. the number of files of the data source, the number of files of the interface machine and the number of files of the cluster; B. the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster; C. reporting the number of base stations by the cluster and reporting the number of base stations by the sharing server;
judging whether the number of the files of the data source, the number of the files of the interface machine and the number of the files of the cluster are equal, whether the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster are equal, and whether the number of the cluster reporting base stations and the number of the sharing server reporting base stations are equal to the number of the working parameter base stations;
step three, if the deviation of the number of the files of the data source, the number of the files of the interface machine and the number of the files of the cluster is larger than a first preset threshold value, alarm information is sent out, if the deviation of the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster is larger than a second preset threshold value, the alarm information is sent out, and if the difference value of the number of the cluster reporting base stations and the number of the sharing server reporting base stations which are respectively smaller than the number of the industrial parameter base stations exceeds a third preset threshold value, the alarm information is sent out;
and step four, analyzing whether the data of the operator is abnormal or not, links in which the data are abnormal and reasons of all judgment abnormalities exist according to whether the alarm information is sent or not and the type frequency of the alarm information, and outputting a data quality evaluation result.
Optionally, the MR data further includes: D. call drop rate, imsi effective rate, agps reporting rate.
Optionally, in the second step, the method further includes:
and judging whether the call drop rate is less than 0.3 percent, the imsi effective rate is more than 90 percent, and the agps reporting rate is between 2 and 5 percent.
Optionally, in step three, the method further includes:
and if the call drop rate exceeds 0.3 percent, or the imsi effective rate is less than 90 percent, or the agps reporting rate is not between 2 and 5 percent, sending out alarm information.
Optionally, in step three, if an abnormal judgment occurs: a. the number of cluster files is less than that of the interface machine files; b. the storage size of the cluster file is smaller than that of the interface machine file; c. the number of the cluster reporting base stations is less than the number of the engineering parameter base stations; and (4) judging the abnormality, namely a, b and c belong to the abnormality judgment of the cluster link, and determining that the problem occurs in the transmission process from the interface machine file to the cluster file and the cluster data is lost.
Optionally, in step three, if an abnormal judgment occurs: d. and if the number of the shared service reporting base stations is less than the data volume of the engineering parameter base stations, performing source tracing analysis on the judgment d, and determining that the judgment d is abnormal due to the loss of the cluster link data.
Optionally, in step four, if the data is determined to be lost, the evaluation data is substantially available when the data loss rate is less than 1%.
In a second aspect, an embodiment of the present invention further provides an apparatus for evaluating quality of operator data, including:
the acquisition unit is used for acquiring MR data of a data source link, an interface machine link, a cluster link and a shared server link, wherein the MR data comprises: A. the number of files of the data source, the number of files of the interface machine and the number of files of the cluster; B. the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster; C. reporting the number of base stations by the cluster and reporting the number of base stations by the sharing server;
the judging unit is used for judging whether the number of the files of the data source, the number of the files of the interface machine and the number of the files of the cluster are equal, whether the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster are equal, and whether the number of the cluster reporting base stations and the number of the sharing server reporting base stations are equal to the number of the industrial parameter base stations;
the alarm unit is used for sending alarm information if the deviation of the number of the files of the data source, the number of the files of the interface machine and the number of the files of the cluster is larger than a first preset threshold value, sending alarm information if the deviation of the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster is larger than a second preset threshold value, and sending alarm information if the difference value of the number of the cluster reporting base stations and the number of the sharing server reporting base stations, which is respectively smaller than the number of the industrial parameter base stations, exceeds a third preset threshold value;
and the analysis unit is used for analyzing whether the data of the operator has abnormity, abnormal links and various abnormal reasons according to the alarm information and the type frequency of the alarm information, and outputting a data quality evaluation result.
In a third aspect, an embodiment of the present invention further provides a server, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor, when executing the computer program, implements the method for evaluating the quality of the operator data in any of the foregoing embodiments.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the quality assessment method for operator data in any of the foregoing embodiments.
The technical scheme of the embodiment of the invention provides powerful support for data quality management work through a data quality monitoring and guaranteeing system, and realizes the following system construction targets: (1) the monitoring is completed. And corresponding data quality monitoring means are provided for a data source link, a transmission link, an acquisition link, a processing link, a warehousing link, an interface sharing link and the like, so that the data quality monitoring of the whole process is realized. (2) And (5) rule configuration. And formulating a data quality evaluation general rule and a user-defined quality detection rule to realize the configuration of the detection rule and the alarm rule. (3) The detection is real-time. The online real-time diagnosis is realized, and the normalizations, the online and the real-time performances of fault discovery, fault positioning, influence analysis, problem tracing and the like of the data quality problem are realized.
Drawings
Fig. 1 is a schematic flowchart of a method for evaluating quality of operator data according to a first embodiment of the present invention;
fig. 2 is a schematic structural diagram of an apparatus for evaluating quality of operator data according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a server in the third embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the steps as a sequential process, many of the steps can be performed in parallel, concurrently or simultaneously. In addition, the order of the steps may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.
Furthermore, the terms "first," "second," and the like may be used herein to describe various orientations, actions, steps, elements, or the like, but the orientations, actions, steps, or elements are not limited by these terms. These terms are only used to distinguish one direction, action, step or element from another direction, action, step or element. For example, the first preset threshold may be referred to as a second preset threshold, and similarly, the second preset threshold may be referred to as a first preset threshold, without departing from the scope of the present application. Both the first preset threshold and the second preset threshold are preset thresholds, but they are not the same preset threshold. The terms "first", "second", etc. are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Example one
Fig. 1 is a schematic flow chart of a method for evaluating quality of operator data according to an embodiment of the present invention, which is applicable to data quality monitoring. The method of the embodiment of the invention can be executed by a quality evaluation device of operator data, which can be realized by software and/or hardware, and can be generally integrated in a server or a terminal device. Referring to fig. 1, the method for evaluating the quality of operator data according to the embodiment of the present invention specifically includes the following steps:
step S110, collecting MR data of a data source link, an interface machine link, a cluster link and a shared server link, wherein the MR data comprises: A. the number of files of the data source, the number of files of the interface machine and the number of files of the cluster; B. the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster; C. the cluster reports the number of the base stations, and the sharing server reports the number of the base stations.
Specifically, data quality management is performed on MR data, acquisition information of the MR data is configured in the acquisition process of a data quality management center, the data quality management center mainly performs management and issuing of various rule configurations, a data quality information acquisition module is mainly responsible for acquiring quality original information of the data, and is equivalent to a data quality embedding point, the acquisition rule is set according to the data quality management center for acquiring the data information, for example, information fields of which links are acquired, and a user can complete configuration through an interface of the data quality management center. In this embodiment, the data monitoring step includes: a data source link, an interface machine link, a cluster link and a shared server link. The acquired MR data includes: A. the number of files of the data source, the number of files of the interface machine and the number of files of the cluster; B. the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster; C. the cluster reports the number of the base stations, and the sharing server reports the number of the base stations. As an alternative embodiment, the MR data further comprises: D. call drop rate, imsi effective rate, agps reporting rate.
For example, the collected data information is: the number of files of the data source is 94282, the file storage size of the data source is 864954M, the number of files of the interface machine is 94282, the file storage size of the interface machine is 864954M, the number of cluster reporting base stations is 135687, the call drop rate is 0.09%, the effective rate of imsi is 99.5%, and the report rate of agps is 2.98%.
Step S120, judging whether the number of the files of the data source, the number of the files of the interface machine and the number of the files of the cluster are equal, whether the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster are equal, and whether the number of the cluster reporting base stations and the number of the sharing server reporting base stations are equal to the number of the working parameter base stations.
Specifically, after data quality information of each link is collected, index judgment is carried out on the information, the judgment standard of the index judgment is also from a data quality management center, and data quality evaluation judgment is carried out according to the rule set by the data quality management center for judging each index. In this embodiment, the data quality evaluation requirement determines whether the number of files of the data source, the number of files of the interface machine, and the number of files of the cluster are equal; whether the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster are equal or not is judged; and whether the number of the cluster reporting base stations and the number of the sharing server reporting base stations are equal to the number of the working parameter base stations or not is judged. As an alternative embodiment, in step S120, the method further includes: and judging whether the call drop rate is less than 0.3 percent, the imsi effective rate is more than 90 percent, and the agps reporting rate is between 2 and 5 percent.
Step S130, if the deviation of the number of the files of the data source, the number of the files of the interface machine and the number of the files of the cluster is larger than a first preset threshold value, alarm information is sent out, if the deviation of the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster is larger than a second preset threshold value, the alarm information is sent out, and if the difference value of the number of the cluster reporting base stations and the number of the sharing server reporting base stations, which is respectively smaller than the number of the industrial parameter base stations, exceeds a third preset threshold value, the alarm information is sent out.
For example, the deviation of the number of files of the data source, the number of files of the interface machine and the number of files of the cluster is 0.01%, the deviation of the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster is 0.1%, the number of the cluster reporting base stations and the number of the sharing server reporting base stations are less than 5% of the number of the industrial parameter base stations, and the alarm is generated. As an alternative embodiment, in step S130, the method further includes: and if the call drop rate exceeds 0.3 percent, or the imsi effective rate is less than 90 percent, or the agps reporting rate is not between 2 and 5 percent, sending out alarm information.
Step S140, analyzing whether the data of the operator has abnormity, abnormal links and reasons of various abnormal judgments according to whether the alarm information is sent out and the type frequency of the alarm information, and outputting a data quality evaluation result.
For example, according to the discrimination discovery of step S130: a. the number of cluster files is less than that of the interface machine files; b. the storage size of the cluster file is smaller than that of the interface machine file; c. the number of the cluster reporting base stations is less than the number of the engineering parameter base stations; d. the number of the base stations reported by the shared service is less than the data volume of the working parameter base stations. Further analysis was as follows: the abnormity discrimination a, b and c belong to abnormity discrimination occurring in a cluster link, normalization analysis is carried out on the abnormity discrimination a, b and c, the three abnormity discrimination results from the same problem, namely the problem occurs in the transmission process from the interface machine file to the cluster file, and the cluster data is lost. Judging d belongs to a link of a shared server, an upstream link of the shared server is a cluster link, tracing analysis is carried out on the judging d, and the abnormal judging d is found to be caused by data loss of the cluster link, and the problem root is the cluster link.
And finally obtaining a problem analysis conclusion after the reason of discriminant analysis is passed, wherein the four abnormal discriminants are caused by data loss in a cluster link, and the data is not completely put in storage due to file packet damage and data processing task error reporting according to the historical processing result suggestion. And processing the proposal to negotiate a data source manufacturer for data retransmission and performing additional recording on the retransmitted data. And (4) evaluating the data quality, obtaining a conclusion according to the indexes, losing 1% of the data quantity, not influencing the overall use effect, and displaying other normal detection indexes through a visual report chart.
The technical scheme of the embodiment of the invention provides powerful support for data quality management work through a data quality monitoring and guaranteeing system, and realizes the following system construction targets: (1) the monitoring is completed. And corresponding data quality monitoring means are provided for a data source link, a transmission link, an acquisition link, a processing link, a warehousing link, an interface sharing link and the like, so that the data quality monitoring of the whole process is realized. (2) And (5) rule configuration. And formulating a data quality evaluation general rule and a user-defined quality detection rule to realize the configuration of the detection rule and the alarm rule. (3) The detection is real-time. The online real-time diagnosis is realized, and the normalizations, the online and the real-time performances of fault discovery, fault positioning, influence analysis, problem tracing and the like of the data quality problem are realized.
Example two
The operator data quality evaluation device provided by the embodiment of the invention can execute the operator data quality evaluation method provided by any embodiment of the invention, has corresponding functional modules and beneficial effects of the execution method, can be realized in a software and/or hardware (integrated circuit) mode, and can be generally integrated in a server or terminal equipment. Fig. 2 is a schematic structural diagram of an operator data quality evaluation apparatus according to a second embodiment of the present invention. Referring to fig. 2, the quality evaluation apparatus 200 for operator data according to the embodiment of the present invention may specifically include:
an acquisition unit 210, configured to acquire MR data of a data source link, an interface machine link, a cluster link, and a shared server link, where the MR data includes: A. the number of files of the data source, the number of files of the interface machine and the number of files of the cluster; B. the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster; C. reporting the number of base stations by the cluster and reporting the number of base stations by the sharing server;
the judging unit 220 is configured to judge whether the number of the files of the data source, the number of the files of the interface machine, and the number of the files of the cluster are equal, whether the file storage size of the data source, the file storage size of the interface machine, and the file storage size of the cluster are equal, and whether the number of the cluster reporting base stations and the number of the sharing server reporting base stations are equal to the number of the working parameter base stations;
the alarm unit 230 is configured to send alarm information if a deviation between the number of files in the data source, the number of files in the interface machine, and the number of files in the cluster is greater than a first preset threshold, send alarm information if a deviation between the file storage size of the data source, the file storage size of the interface machine, and the file storage size of the cluster is greater than a second preset threshold, and send alarm information if a difference between the number of reporting base stations in the cluster and the number of reporting base stations in the shared server, which is respectively less than the number of working parameter base stations, exceeds a third preset threshold;
and the analysis unit 240 is configured to analyze whether the data of the operator has an abnormality, a link in which the abnormality occurs, and reasons for each abnormal judgment according to whether the alarm information is sent and the type frequency of the alarm information, and output a data quality evaluation result.
Optionally, the MR data further includes: D. call drop rate, imsi effective rate, agps reporting rate.
Optionally, the determining unit 220 is further configured to:
and judging whether the call drop rate is less than 0.3 percent, the imsi effective rate is more than 90 percent, and the agps reporting rate is between 2 and 5 percent.
Optionally, the alarm unit 230 is further configured to:
and if the call drop rate exceeds 0.3 percent, or the imsi effective rate is less than 90 percent, or the agps reporting rate is not between 2 and 5 percent, sending out alarm information.
Optionally, the alarm unit 230 is further configured to: if the abnormity occurs, judging: a. the number of cluster files is less than that of the interface machine files; b. the storage size of the cluster file is smaller than that of the interface machine file; c. the number of the cluster reporting base stations is less than the number of the engineering parameter base stations; and (4) judging the abnormality, namely a, b and c belong to the abnormality judgment of the cluster link, and determining that the problem occurs in the transmission process from the interface machine file to the cluster file and the cluster data is lost.
Optionally, the alarm unit 230 is further configured to: if the abnormity occurs, judging: d. and if the number of the shared service reporting base stations is less than the data volume of the engineering parameter base stations, performing source tracing analysis on the judgment d, and determining that the judgment d is abnormal due to the loss of the cluster link data.
Optionally, the analysis unit 240 is further configured to: if the data is determined to be lost, the evaluation data is basically available when the data loss rate is less than 1%.
The technical scheme of the embodiment of the invention provides powerful support for data quality management work through a data quality monitoring and guaranteeing system, and realizes the following system construction targets: (1) the monitoring is completed. And corresponding data quality monitoring means are provided for a data source link, a transmission link, an acquisition link, a processing link, a warehousing link, an interface sharing link and the like, so that the data quality monitoring of the whole process is realized. (2) And (5) rule configuration. And formulating a data quality evaluation general rule and a user-defined quality detection rule to realize the configuration of the detection rule and the alarm rule. (3) The detection is real-time. The online real-time diagnosis is realized, and the normalizations, the online and the real-time performances of fault discovery, fault positioning, influence analysis, problem tracing and the like of the data quality problem are realized.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a server according to a third embodiment of the present invention, as shown in fig. 3, the server includes a processor 310, a memory 320, an input device 330, and an output device 340; the number of the processors 310 in the server may be one or more, and one processor 310 is taken as an example in fig. 3; the processor 310, the memory 320, the input device 330 and the output device 340 in the server may be connected by a bus or other means, and the bus connection is taken as an example in fig. 3.
The memory 320 is a computer-readable storage medium, and can be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the quality evaluation method of the operator data in the embodiment of the present invention (for example, the acquisition unit 210, the judgment unit 220, the alarm unit 230, and the analysis unit 240 in the quality evaluation device of the operator data). The processor 310 executes various functional applications of the server and data processing by executing software programs, instructions, and modules stored in the memory 320, that is, implements the above-described quality evaluation method of the operator data.
Namely:
the method comprises the following steps of firstly, acquiring MR data of a data source link, an interface machine link, a cluster link and a shared server link, wherein the MR data comprises: A. the number of files of the data source, the number of files of the interface machine and the number of files of the cluster; B. the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster; C. reporting the number of base stations by the cluster and reporting the number of base stations by the sharing server;
judging whether the number of the files of the data source, the number of the files of the interface machine and the number of the files of the cluster are equal, whether the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster are equal, and whether the number of the cluster reporting base stations and the number of the sharing server reporting base stations are equal to the number of the working parameter base stations;
step three, if the deviation of the number of the files of the data source, the number of the files of the interface machine and the number of the files of the cluster is larger than a first preset threshold value, alarm information is sent out, if the deviation of the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster is larger than a second preset threshold value, the alarm information is sent out, and if the difference value of the number of the cluster reporting base stations and the number of the sharing server reporting base stations which are respectively smaller than the number of the industrial parameter base stations exceeds a third preset threshold value, the alarm information is sent out;
and step four, analyzing whether the data of the operator is abnormal or not, links in which the data are abnormal and reasons of all judgment abnormalities exist according to whether the alarm information is sent or not and the type frequency of the alarm information, and outputting a data quality evaluation result.
Of course, the processor of the server provided in the embodiment of the present invention is not limited to execute the method operations described above, and may also execute the relevant operations in the method for evaluating the quality of the operator data provided in any embodiment of the present invention.
The memory 320 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 320 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 320 may further include memory located remotely from processor 310, which may be connected to a server over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 330 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the server. The output device 340 may include a display device such as a display screen.
The technical scheme of the embodiment of the invention provides powerful support for data quality management work through a data quality monitoring and guaranteeing system, and realizes the following system construction targets: (1) the monitoring is completed. And corresponding data quality monitoring means are provided for a data source link, a transmission link, an acquisition link, a processing link, a warehousing link, an interface sharing link and the like, so that the data quality monitoring of the whole process is realized. (2) And (5) rule configuration. And formulating a data quality evaluation general rule and a user-defined quality detection rule to realize the configuration of the detection rule and the alarm rule. (3) The detection is real-time. The online real-time diagnosis is realized, and the normalizations, the online and the real-time performances of fault discovery, fault positioning, influence analysis, problem tracing and the like of the data quality problem are realized.
Example four
A fourth embodiment of the present invention further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, perform a method for quality assessment of operator data, the method including:
the method comprises the following steps of firstly, acquiring MR data of a data source link, an interface machine link, a cluster link and a shared server link, wherein the MR data comprises: A. the number of files of the data source, the number of files of the interface machine and the number of files of the cluster; B. the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster; C. reporting the number of base stations by the cluster and reporting the number of base stations by the sharing server;
judging whether the number of the files of the data source, the number of the files of the interface machine and the number of the files of the cluster are equal, whether the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster are equal, and whether the number of the cluster reporting base stations and the number of the sharing server reporting base stations are equal to the number of the working parameter base stations;
step three, if the deviation of the number of the files of the data source, the number of the files of the interface machine and the number of the files of the cluster is larger than a first preset threshold value, alarm information is sent out, if the deviation of the file storage size of the data source, the file storage size of the interface machine and the file storage size of the cluster is larger than a second preset threshold value, the alarm information is sent out, and if the difference value of the number of the cluster reporting base stations and the number of the sharing server reporting base stations which are respectively smaller than the number of the industrial parameter base stations exceeds a third preset threshold value, the alarm information is sent out;
and step four, analyzing whether the data of the operator is abnormal or not, links in which the data are abnormal and reasons of all judgment abnormalities exist according to whether the alarm information is sent or not and the type frequency of the alarm information, and outputting a data quality evaluation result.
Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the operations of the method described above, and may also perform related operations in the method for evaluating the quality of the operator data provided by any embodiment of the present invention.
The computer-readable storage media of embodiments of the invention may take any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or terminal. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The technical scheme of the embodiment of the invention provides powerful support for data quality management work through a data quality monitoring and guaranteeing system, and realizes the following system construction targets: (1) the monitoring is completed. And corresponding data quality monitoring means are provided for a data source link, a transmission link, an acquisition link, a processing link, a warehousing link, an interface sharing link and the like, so that the data quality monitoring of the whole process is realized. (2) And (5) rule configuration. And formulating a data quality evaluation general rule and a user-defined quality detection rule to realize the configuration of the detection rule and the alarm rule. (3) The detection is real-time. The online real-time diagnosis is realized, and the normalizations, the online and the real-time performances of fault discovery, fault positioning, influence analysis, problem tracing and the like of the data quality problem are realized.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110367597.3A CN113064890A (en) | 2021-04-06 | 2021-04-06 | Quality evaluation method, device, server and medium for operator data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110367597.3A CN113064890A (en) | 2021-04-06 | 2021-04-06 | Quality evaluation method, device, server and medium for operator data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113064890A true CN113064890A (en) | 2021-07-02 |
Family
ID=76566025
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110367597.3A Pending CN113064890A (en) | 2021-04-06 | 2021-04-06 | Quality evaluation method, device, server and medium for operator data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113064890A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115372029A (en) * | 2022-08-25 | 2022-11-22 | 武汉迪昌科技有限公司 | A fault diagnosis method, system and terminal equipment for EMU signal collection |
| CN116609801A (en) * | 2023-04-04 | 2023-08-18 | 北京讯腾智慧科技股份有限公司 | Main and standby service system and method for base station observation data |
| CN119322925A (en) * | 2024-10-15 | 2025-01-17 | 中国科学院空间应用工程与技术中心 | Method and system for supporting space science and application data quality analysis and evaluation |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104883298A (en) * | 2015-06-12 | 2015-09-02 | 中国通信建设集团设计院有限公司 | Business quality detection method and router |
| CN109062754A (en) * | 2018-06-26 | 2018-12-21 | 平安科技（深圳）有限公司 | Data monitoring and alarm method, device, storage medium and server |
| CN109714196A (en) * | 2018-12-11 | 2019-05-03 | 中国联合网络通信集团有限公司 | Data monitoring method and platform |
| CN110913426A (en) * | 2019-12-27 | 2020-03-24 | 树蛙信息科技(南京)有限公司 | Passenger flow big data signaling acquisition point information automatic detection synchronization method |
-
2021
- 2021-04-06 CN CN202110367597.3A patent/CN113064890A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104883298A (en) * | 2015-06-12 | 2015-09-02 | 中国通信建设集团设计院有限公司 | Business quality detection method and router |
| CN109062754A (en) * | 2018-06-26 | 2018-12-21 | 平安科技（深圳）有限公司 | Data monitoring and alarm method, device, storage medium and server |
| CN109714196A (en) * | 2018-12-11 | 2019-05-03 | 中国联合网络通信集团有限公司 | Data monitoring method and platform |
| CN110913426A (en) * | 2019-12-27 | 2020-03-24 | 树蛙信息科技(南京)有限公司 | Passenger flow big data signaling acquisition point information automatic detection synchronization method |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115372029A (en) * | 2022-08-25 | 2022-11-22 | 武汉迪昌科技有限公司 | A fault diagnosis method, system and terminal equipment for EMU signal collection |
| CN116609801A (en) * | 2023-04-04 | 2023-08-18 | 北京讯腾智慧科技股份有限公司 | Main and standby service system and method for base station observation data |
| CN116609801B (en) * | 2023-04-04 | 2023-12-22 | 北京讯腾智慧科技股份有限公司 | Main and standby service system and method for base station observation data |
| CN119322925A (en) * | 2024-10-15 | 2025-01-17 | 中国科学院空间应用工程与技术中心 | Method and system for supporting space science and application data quality analysis and evaluation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108763957B (en) | Database security audit system, method and server | |
| CN111126824B (en) | Multi-index correlation model training method and multi-index anomaly analysis method | |
| CN113064890A (en) | Quality evaluation method, device, server and medium for operator data | |
| CN107294808B (en) | Interface test method, device and system | |
| CN112905548B (en) | Security audit system and method | |
| CN113495820B (en) | Anomaly information collecting and processing method and device and anomaly monitoring system | |
| CN104979908B (en) | Substation network online failure analysis method | |
| US8144599B2 (en) | Binary class based analysis and monitoring | |
| CN110929896A (en) | A safety analysis method and device for system equipment | |
| CN113671909A (en) | Safety monitoring system and method for steel industrial control equipment | |
| CN105471932A (en) | Front-end application monitoring method, front-end application and front-end application monitoring system | |
| CN104574219A (en) | System and method for monitoring and early warning of operation conditions of power grid service information system | |
| CN117056109B (en) | Data operation and maintenance fault analysis system and method | |
| CN114726708A (en) | Network element equipment fault prediction method and system based on artificial intelligence | |
| CN105589800A (en) | Application system for predicting faults of complex system | |
| CN100544476C (en) | The gprs service intelligence control method | |
| CN117319279A (en) | Network transmission performance test system and test method | |
| CN113760634A (en) | A data processing method and device | |
| CN110647417B (en) | Energy internet abnormal data processing method, device and system | |
| CN109818808B (en) | Fault diagnosis method and device and electronic equipment | |
| CN101197714B (en) | A method for centralized collection of mobile data service status | |
| CN115242610A (en) | Link quality monitoring method, apparatus, electronic device and computer-readable storage medium | |
| CN119383115A (en) | A business operation and maintenance method and system based on cloud service architecture | |
| CN113254313A (en) | Monitoring index abnormality detection method and device, electronic equipment and storage medium | |
| CN116743618B (en) | Data collection and analysis methods, equipment and media for factory and station telecontrol equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210702 |
|
| RJ01 | Rejection of invention patent application after publication |