CN117473225B - Log data management method and device, electronic equipment and readable storage medium - Google Patents
Log data management method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN117473225B CN117473225B CN202311347841.5A CN202311347841A CN117473225B CN 117473225 B CN117473225 B CN 117473225B CN 202311347841 A CN202311347841 A CN 202311347841A CN 117473225 B CN117473225 B CN 117473225B
- Authority
- CN
- China
- Prior art keywords
- log data
- data
- abnormal
- trained
- corresponding relation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a log data management method, a log data management device, electronic equipment and a readable storage medium, wherein the method comprises the following steps: collecting current log data of firewall equipment, and obtaining a data corresponding relation; the data correspondence is determined based on historical log data preceding the current log data; and managing the current log data based on the data corresponding relation. It will be appreciated that the present application treats current log data by a data correspondence determined by historical log data preceding the current log data, rather than by a fixed data blood-address. The historical log data reflects some characteristics of the firewall equipment corresponding to the current log data, so that the data corresponding relation corresponds to the firewall equipment and is not a general data treatment basis in the industry, namely the embodiment of the application improves the pertinence of data treatment.
Description
Technical Field
The present application relates to the field of big data technologies, and in particular, to a method and apparatus for log data management, an electronic device, and a readable storage medium.
Background
With the development of network technology, data security is increasingly emphasized. In particular, the host, which is the main bearing industry of the key infrastructure, is often faced with a plurality of attack methods and combinations thereof, such as large-scale denial of service attack, advanced sustainable attack, exploit, phishing attack, supply chain attack, etc., and if the attack is successful, the attack can form a fatal hit to the key information infrastructure.
The security device is well resistant to the attacks described above. At present, the attack can be better prevented by carrying out correlation analysis on log data of the security device. Because of the huge volume of data, the log data needs to be subjected to data management before the relevant analysis, thereby facilitating the completion of the analysis. However, current data governance is generally implemented by a fixed data blood edge, which is a common data governance basis in the industry, and is not targeted.
Therefore, in practical application, a scheme with pertinence in data management is needed.
Disclosure of Invention
In view of the above, the present application provides a log data management method, device, electronic apparatus and readable storage medium, which aims to solve the technical problem of how to improve pertinence in data management.
In order to achieve the above object, the present application provides a log data management method, comprising the steps of:
collecting current log data of the security equipment, and acquiring a data corresponding relation; the data correspondence is determined based on historical log data preceding the current log data;
And managing the current log data based on the data corresponding relation.
Illustratively, the acquiring the data correspondence includes:
Inputting the current log data to a relation prediction model to obtain a data corresponding relation; the relation prediction model is obtained by carrying out iterative training on a model to be trained based on a log training data set; the log training dataset includes a plurality of the historical log data.
Illustratively, the inputting the current log data into the relationship prediction model, before obtaining the data corresponding relationship, includes:
acquiring a log training data set and acquiring a model to be trained;
Performing iterative training on the model to be trained based on the log training data set to obtain an updated model to be trained, and determining whether the updated model to be trained meets a preset iteration ending condition;
If the updated model to be trained meets the preset iteration ending condition, the updated model to be trained is used as the relation prediction model;
And if the updated model to be trained does not meet the iteration ending condition, returning to an iteration training step for the model to be trained based on the log training data set until the updated model to be trained meets the iteration ending condition.
Exemplary, the inputting the current log data into the relationship prediction model, after obtaining the data corresponding relationship, includes:
Continuously acquiring updated log data after the current log data;
and updating the relation prediction model based on the update log data when the update log data meets preset conditions.
Illustratively, the acquiring the data correspondence includes:
Acquiring history log data of the security device;
Extracting abnormal log data from the history log data;
And carrying out association analysis on the abnormal log data to obtain a data corresponding relation.
The method for performing association analysis on the abnormal log data to obtain a data corresponding relationship includes:
Determining safety equipment corresponding to each abnormal log data;
Counting the occurrence times of the security devices corresponding to the security events by taking the security events as dimensions;
determining the data corresponding relation of each abnormal log data based on the occurrence times;
in the process of determining the data corresponding relation of each abnormal log data, the operation of determining the data corresponding relation of any abnormal log data is as follows:
Determining the security event corresponding to the largest occurrence number as a target security event corresponding to any abnormal log data;
And associating the target security event with any abnormal log data to obtain a data corresponding relation.
For example, one portion of the abnormal log data corresponds to one or more security events, and the performing association analysis on the abnormal log data to obtain a data corresponding relationship includes:
obtaining the office factors corresponding to the abnormal log data;
matching factors of each of the office departments to obtain one or more corresponding target security events;
And associating each target security event with the corresponding abnormal log data to obtain a data corresponding relation.
To achieve the above object, the present application also provides a log data management device, including:
the acquisition module is used for acquiring current log data of the safety equipment and acquiring a data corresponding relation; the data correspondence is determined based on historical log data preceding the current log data;
and the treatment module is used for treating the current log data based on the data corresponding relation.
To achieve the above object, the present application also provides an electronic device, including: a memory, a processor, and a log data governance program stored on the memory and executable on the processor, the log data governance program configured to implement the steps of the log data governance method as described above.
Illustratively, to achieve the above object, the present application also provides a computer-readable storage medium having stored thereon a log data governance program which, when executed by a processor, implements the steps of the log data governance method as described above.
In the embodiment of the application, the current log data is managed through the data corresponding relation, and the data corresponding relation is determined through the historical log data before the current log data, but not the fixed data blood-edge. The historical log data reflects some characteristics of the safety equipment corresponding to the current log data, so that the data corresponding relation corresponds to the safety equipment, and is not a general data management basis in the industry, namely the embodiment of the application improves the pertinence in data management.
Drawings
FIG. 1 is a flow chart of a log data management method according to a first embodiment of the present application;
FIG. 2 is a flow chart of a log data management method according to a second embodiment of the present application;
FIG. 3 is a flow chart of a log data management method according to a third embodiment of the present application;
Fig. 4 is a schematic structural diagram of a hardware running environment according to an embodiment of the present application.
The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The application provides a log data management method, referring to fig. 1, fig. 1 is a flow chart of a first embodiment of the log data management method of the application.
Embodiments of the present application provide embodiments of log data governance methods, it being noted that although a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in a different order than that illustrated herein. The log data management method is applied to electronic equipment, and the electronic equipment comprises but is not limited to terminal equipment and a server. The electronic equipment is provided with a government affair big data platform, the government affair big data platform collects videos from office departments, social institutions, the Internet and other information resources, such as sensing equipment, social media, positioning information and the like, processes the collected multi-source heterogeneous data, provides big data application environments for urban cross departments, cross systems and cross business linkage, provides technical support for fusion innovation of government and social information resources and government and civil interaction, promotes the electronic government affairs to be upgraded into intelligent government affairs, and forms a novel government affair operation mode characterized by thorough sensing, quick response, fine management, scientific decision, business linkage and active service. The embodiment of the application aims at managing the acquired multi-source heterogeneous data, and the log data management method comprises the following steps:
Step S110, collecting current log data of the safety equipment and obtaining a data corresponding relation; the data correspondence is determined based on historical log data preceding the current log data.
It should be noted that the security device corresponds to a house department, that is, different house departments have their own security devices. The office generally includes government affairs, traffic, homeland, education, medical treatment, travel, culture, environmental ecology, electronic commerce, communication, etc. Accordingly, through the business data of the management departments, support can be provided for urban operation index systems, public credit evaluation, population flow analysis, supervision risk early warning, market price analysis, government public opinion monitoring, traffic flow prediction, commercial environment optimization, urban layout adjustment, public safety behavior analysis and the like.
The security device can effectively prevent information of a stealer department from interfering with government operation and even destroy the behavior of an infrastructure. The security devices include, but are not limited to, hardware firewalls, software firewalls, cloud firewalls, intrusion detection, database firewalls, intrusion protection, log auditing.
The current log data is the log data of the safety equipment collected in real time, and the history log data is the log data of the safety equipment which is collected before the current log data is collected.
Wherein the log data is information generated by the security device that records network communication activity, including but not limited to: 1. a source IP address and a destination IP address, wherein the log data provides the source IP address and the destination IP address associated with the network communication, which addresses can be used to determine the source and destination of the communication; 2. a time stamp, wherein each piece of log data should contain a time stamp for determining a specific time when the abnormal event occurs; 3. communication protocols, wherein log data may indicate the communication protocol used, including but not limited to TCP (Transmission Control Protocol ), UDP (User Datagram Protocol, user datagram protocol) or ICMP (Internet Control Message Protocol, network control message protocol); 4. port numbers, where log data may record both source and destination ports, are important for determining the particular service or application used in network communications; 5. actions, wherein the log data may indicate actions for execution of network communications, including but not limited to, permit, reject, or discard; 6. rules or policies, where the log data may provide information of which firewall rules or policies match a particular network communication; 7. a packet size, wherein the log data may contain information about the size of the transmitted packet, which may be useful for detecting abnormal traffic or network attacks; 8. abnormal events, wherein if a firewall detects abnormal activity, including but not limited to malicious traffic of an intrusion attempt or rejection, log data may record the abnormal events.
The data corresponding relation is the corresponding relation between the current log data and the security event, and the security event is caused by the attack. For example, medical treatment in a house is attacked and, after having successfully stolen medical-related data, may trigger a medical fraudulent security event. Accordingly, the security device of the medical department may record log data before the security event occurs, and thus there may be a data correspondence between the current log data of the security device and the security event of medical fraud. However, since the medical related data may also be used by an attacker for other security events, the security event, which is also not directly recognized as medical fraud, must correspond to the current log data of the security device of the medical department.
And step S120, treating the current log data based on the data corresponding relation.
The role of data governance is to provide the guidance needed to manage data as an asset. The core is decision right allocation and responsibility division of data asset management. Overall, data governance is to improve data quality, while reducing risk, maximizing data asset value. Data governance generally includes: constructing a flexible, standardized and modularized multi-source heterogeneous data resource access system; constructing a standardized, procedural and intelligent data processing system; constructing a data fine treatment system and an organized data resource fusion classification system; and constructing a unified scheduling, accurate service and safe and available information sharing service system.
It should be noted that, the embodiment of the present application aims at creating a data resource fusion classification system of a data refinement management system and an organization, that is, managing current log data based on a data correspondence is essentially classifying the current log data in a security event dimension, that is, determining attribution of the current log data on a security event.
It should be noted that the treated data can be used for safety evaluation, i.e. scoring the safety device, thereby determining the safety level of the safety device; or for tracking the source of the attack.
It will be appreciated that in embodiments of the present application, the current log data is managed by a data correspondence determined by historical log data preceding the current log data, rather than by a fixed data blood-edge. The historical log data reflects some characteristics of the safety equipment corresponding to the current log data, so that the data corresponding relation corresponds to the safety equipment, and is not a general data management basis in the industry, namely the embodiment of the application improves the pertinence in data management.
In order to clarify the data corresponding relation, the current log data with the data corresponding relation determined can be better utilized later. The embodiment of the application provides three modes for defining the corresponding relation of the data.
Mode one: on the basis of the first embodiment, a second embodiment of the log data management method of the present application is provided, and referring to fig. 2, fig. 2 is a schematic flow chart of the second embodiment of the log data management method of the present application. In this second embodiment, the acquiring the data correspondence relationship includes:
Step S220, inputting the current log data to a relation prediction model to obtain a data corresponding relation; the relation prediction model is obtained by carrying out iterative training on a model to be trained based on a log training data set; the log training dataset includes a plurality of the historical log data.
In this embodiment, the model to be trained may be constructed by algorithms such as a random forest, an artificial neural network or a support vector machine, and taking the random forest as an example, the random forest is a forest constructed by a random manner, the forest is composed of a plurality of decision trees, the decision trees are not associated with each other, after the current log data is input into the random forest, each decision tree classifies the current log data independently, and then an expected security event is determined according to the probability corresponding to each security event. For example, the security event includes security event 1, security event 2 and security event 3, after the random forest is input to the corresponding to-be-classified medical path feature, the probability of the security event 1 is 80%, the probability of the security event 2 is 14% and the probability of the security event 3 is 6%, so that the probability of the security event 1 can be determined to be the largest, that is, the expected security event is the security event 1, and accordingly, the data corresponding relationship is that there is a corresponding relationship between the current log data and the security event 1.
It should be noted that, the model prediction process is an automated processing process, and compared with manually determining the data correspondence, the prediction result has higher accuracy.
Further, before the obtaining the relationship prediction model, the method includes: acquiring a log training data set and acquiring a model to be trained; performing iterative training on the model to be trained based on the training data set to obtain an updated model to be trained, and determining whether the updated model to be trained meets a preset iteration ending condition; performing iterative training on the model to be trained based on the log training data set to obtain an updated model to be trained, and determining whether the updated model to be trained meets a preset iteration ending condition; if the updated model to be trained meets the preset iteration ending condition, the updated model to be trained is used as the relation prediction model; and if the updated model to be trained does not meet the iteration ending condition, returning to an iteration training step for the model to be trained based on the log training data set until the updated model to be trained meets the iteration ending condition.
In the embodiment, iterative training is performed on a model to be trained based on a log training data set to obtain an updated model to be trained, and whether the updated model to be trained meets a preset iteration ending condition is determined; if the updated model to be trained meets the preset iteration ending condition, the updated model to be trained is used as a relation prediction model; if the updated model to be trained does not meet the iteration ending condition, continuing to perform iteration training updating on the updated model to be trained until the updated model to be trained meets the iteration ending condition.
Specifically, the model to be trained is subjected to iterative training through the log training data set, so that an updated model to be trained is obtained. After each time the updated model to be trained is obtained, determining whether the updated model to be trained meets a preset iteration ending condition, if the updated model to be trained meets the preset iteration ending condition, ending the iteration, and taking the last updated model to be trained as a relation prediction model; if the updated model to be trained does not meet the iteration ending condition, the updated model to be trained is not met the using condition, and the updated model to be trained is continuously subjected to iteration training and updated until the updated model to be trained meets the iteration ending condition.
It should be noted that, the iterative training is a process of training the model to be trained through the history log data in the log training data set for multiple times, and generally, the relationship prediction model obtained from the model to be trained needs to be updated through multiple rounds of training. It should be noted that, when the preset iteration end condition is that the model to be trained is input or the model to be trained after being updated is that the model prediction accuracy reaches the preset accuracy threshold, the iteration is ended.
It should be noted that, in addition to the log training data set, the log test data set needs to be used in the model training process, where the data amount between the log training data set and the log test data set is proportional to a certain ratio, for example, 9: 1. 7:3, etc.
It should be noted that, for the iterative training process, in order to improve the sensitivity of the relational prediction model, multiple training may be performed on the model to be trained, and specifically, the iterative training process includes multiple training and multiple testing, for example, a process of testing 1 time after training 10 times and cycling the training and testing until the iteration is ended.
Further, in order to ensure accuracy of the prediction result of the relational prediction model, the step of inputting the current log data into the relational prediction model to obtain the data corresponding relationship includes: continuously acquiring updated log data after the current log data; and updating the relation prediction model based on the update log data when the update log data meets preset conditions.
It can be understood that the updated log data is the log data of the security device that is acquired after the current log data is acquired. The process of updating the relationship prediction model by updating the log data is substantially the same as the process of training the relationship prediction model by using the history log data, and will not be described in detail herein.
It can be appreciated that, as time goes by, the security event may be updated, that is, the business data of a management department may be used by an attacker for other purposes, so that other security events occur, and therefore, by continuously acquiring updated log data after the current log data, it is beneficial to ensure accuracy of the prediction result of the relationship prediction model.
For mode two and mode three:
on the basis of the first embodiment, a third embodiment of the log data management method of the present application is provided, and referring to fig. 3, fig. 3 is a schematic flow chart of the third embodiment of the log data management method of the present application. In this third embodiment, the acquiring the data correspondence relationship includes:
step S310, acquiring history log data of the safety equipment;
Step S320, extracting abnormal log data from the history log data;
and step S330, carrying out association analysis on the abnormal log data to obtain a data corresponding relation.
It should be noted that, the history log data includes normal log data and abnormal log data, where no abnormal event is recorded in the normal log data, and an abnormal event is recorded in the abnormal log data. It will be appreciated that if the exception event is not found by the security device, it indicates that the attacker attack was successful. Therefore, the abnormal log data and the security event have a corresponding relation, but the normal log data and the security event have no corresponding relation, namely, the association analysis is not needed through the normal log data, namely, the analysis efficiency of the association analysis can be improved by only carrying out the association analysis through the abnormal log data.
In an embodiment, the performing association analysis on the abnormal log data to obtain a data corresponding relationship includes: determining safety equipment corresponding to each abnormal log data; counting the occurrence times of the security devices corresponding to the security events by taking the security events as dimensions; determining the data corresponding relation of each abnormal log data based on the occurrence times; in the process of determining the data corresponding relation of each abnormal log data, the operation of determining the data corresponding relation of any abnormal log data is as follows: determining the security event corresponding to the largest occurrence number as a target security event corresponding to any abnormal log data; and associating the target security event with any abnormal log data to obtain a data corresponding relation.
In the embodiment of the application, the security event corresponding to the abnormal log data is determined according to the occurrence times, and the data corresponding relation is further obtained. The service data that may cause the security event is known, and accordingly, a certain correspondence exists between the security device corresponding to the service data and the security event, that is, a certain correspondence exists between the anomaly log data of the security device and the security event.
It should be noted that, there may be a correspondence between the anomaly log data and a plurality of security events, and in order to facilitate subsequent analysis, the security event with the largest occurrence number may be determined from the plurality of security events by the occurrence number as the target security event. It can be appreciated that the maximum number of occurrences indicates that the stronger the association of the anomaly log data with the target security event, and correspondingly, the higher the analysis value.
In another embodiment, one piece of the abnormal log data corresponds to one or more security events, and the performing association analysis on the abnormal log data to obtain a data corresponding relationship includes: obtaining the office factors corresponding to the abnormal log data; matching factors of each of the office departments to obtain one or more corresponding target security events; and associating each target security event with the corresponding abnormal log data to obtain a data corresponding relation.
Wherein the office factor is a unique identifier of the office. In the embodiment of the application, as long as the abnormal log data is associated with the safety event, the abnormal log data is associated with the corresponding safety event, thereby being beneficial to comprehensively analyzing the safety event.
In addition, when an attacker attacks a house department, the attacker may attack a plurality of house departments at the same time in order to achieve the corresponding purpose. For the security event corresponding to the situation, after the data corresponding relation is obtained and the corresponding security event is determined, when the data management is carried out, the current log data of the security equipment of other management departments can be directly and rapidly obtained through the determined association relation between the security event and the other management departments. Therefore, during data management, the current log data of other office departments and the current log data of the current office departments are bound. Therefore, joint analysis can be carried out in the subsequent data application process so as to accurately determine the attack purpose, better determine the attack source and the like.
In addition, the application also provides a log data management device, which comprises:
the acquisition module is used for acquiring current log data of the safety equipment and acquiring a data corresponding relation; the data correspondence is determined based on historical log data preceding the current log data;
and the treatment module is used for treating the current log data based on the data corresponding relation.
Illustratively, the acquisition module is specifically configured to:
Inputting the current log data to a relation prediction model to obtain a data corresponding relation; the relation prediction model is obtained by carrying out iterative training on a model to be trained based on a log training data set; the log training dataset includes a plurality of the historical log data.
Illustratively, the log data administration device further comprises:
The first acquisition module is used for acquiring a log training data set and acquiring a model to be trained;
the training module is used for carrying out iterative training on the model to be trained based on the log training data set to obtain an updated model to be trained, and determining whether the updated model to be trained meets a preset iteration ending condition or not; if the updated model to be trained meets the preset iteration ending condition, the updated model to be trained is used as the relation prediction model; and if the updated model to be trained does not meet the iteration ending condition, returning to an iteration training step for the model to be trained based on the log training data set until the updated model to be trained meets the iteration ending condition.
Illustratively, the log data administration device further comprises:
The second acquisition module is used for continuously acquiring updated log data after the current log data;
And the updating module is used for updating the relation prediction model based on the update log data when the update log data accords with a preset condition.
Illustratively, the acquisition module is specifically configured to:
Acquiring history log data of the security device;
Extracting abnormal log data from the history log data;
And carrying out association analysis on the abnormal log data to obtain a data corresponding relation.
For example, the anomaly log data is multiple, and the acquisition module is specifically configured to:
Determining safety equipment corresponding to each abnormal log data;
Counting the occurrence times of the security devices corresponding to the security events by taking the security events as dimensions;
determining the data corresponding relation of each abnormal log data based on the occurrence times;
in the process of determining the data corresponding relation of each abnormal log data, the operation of determining the data corresponding relation of any abnormal log data is as follows:
Determining the security event corresponding to the largest occurrence number as a target security event corresponding to any abnormal log data;
And associating the target security event with any abnormal log data to obtain a data corresponding relation.
Illustratively, one piece of anomaly log data corresponds to one or more security events, and the acquisition module is specifically configured to:
obtaining the office factors corresponding to the abnormal log data;
matching factors of each of the office departments to obtain one or more corresponding target security events;
And associating each target security event with the corresponding abnormal log data to obtain a data corresponding relation.
The specific implementation of the log data management device is basically the same as the above embodiments of the log data management method, and will not be described herein.
In addition, the application also provides electronic equipment. As shown in fig. 4, fig. 4 is a schematic structural diagram of a hardware running environment according to an embodiment of the present application.
As shown in fig. 4, the electronic device may include a processor 401, a communication interface 402, a memory 403 and a communication bus 404, where the processor 401, the communication interface 402 and the memory 403 complete communication with each other through the communication bus 404, and the memory 403 is used for storing a computer program; the processor 401 is configured to implement the steps of the log data management method when executing the program stored in the memory 403.
The communication bus 404 referred to above for the electronic devices may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, or the like. The communication bus 404 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface 402 is used for communication between the electronic device and other devices described above.
The Memory 403 may include a random access Memory (Random Access Memory, RMD) or may include a Non-Volatile Memory (NM), such as at least one disk Memory. Optionally, the memory 403 may also be at least one storage device located remotely from the aforementioned processor 401.
The processor 401 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but may also be a digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components.
The specific implementation manner of the electronic device of the present application is basically the same as that of each embodiment of the log data management method, and will not be repeated here.
In addition, the embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium is stored with a log data management program, and the log data management program realizes the steps of the log data management method when being executed by a processor.
The specific implementation manner of the computer readable storage medium of the present application is basically the same as the above embodiments of the log data management method, and will not be described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element. The terms "first" and "second" are used solely for distinction and do not represent other meanings, such as that the first characteristic information and the second characteristic information both pertain to the characteristic information.
The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present application.
The foregoing description is only of the preferred embodiments of the present application, and is not intended to limit the scope of the application, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (8)
1. The log data management method is characterized by comprising the following steps of:
collecting current log data of the security equipment, and acquiring a data corresponding relation; the data correspondence is determined based on historical log data preceding the current log data;
the obtaining the data correspondence includes:
Acquiring history log data of the security device;
Extracting abnormal log data from the history log data;
performing association analysis on the abnormal log data to obtain a data corresponding relation;
The abnormal log data is multiple, the association analysis is carried out on the abnormal log data to obtain a data corresponding relation, and the method comprises the following steps:
Determining safety equipment corresponding to each abnormal log data;
Counting the occurrence times of the security devices corresponding to the security events by taking the security events as dimensions;
determining the data corresponding relation of each abnormal log data based on the occurrence times;
in the process of determining the data corresponding relation of each abnormal log data, the operation of determining the data corresponding relation of any abnormal log data is as follows:
determining the security event corresponding to the largest occurrence number as a target security event corresponding to any abnormal log data, wherein the security event corresponding to the largest occurrence number represents the highest association between the abnormal log data and the target security event;
associating the target security event with any abnormal log data to obtain a data corresponding relation;
And managing the current log data based on the data corresponding relation.
2. The log data management method as set forth in claim 1, wherein the acquiring the data correspondence includes:
Inputting the current log data to a relation prediction model to obtain a data corresponding relation; the relation prediction model is obtained by carrying out iterative training on a model to be trained based on a log training data set; the log training dataset includes a plurality of the historical log data.
3. The log data management method according to claim 2, wherein before the current log data is input to the relationship prediction model to obtain the data correspondence, the method comprises:
acquiring a log training data set and acquiring a model to be trained;
Performing iterative training on the model to be trained based on the log training data set to obtain an updated model to be trained, and determining whether the updated model to be trained meets a preset iteration ending condition;
If the updated model to be trained meets the preset iteration ending condition, the updated model to be trained is used as the relation prediction model;
And if the updated model to be trained does not meet the iteration ending condition, returning to an iteration training step for the model to be trained based on the log training data set until the updated model to be trained meets the iteration ending condition.
4. The log data management method according to claim 2, wherein the step of inputting the current log data into a relationship prediction model to obtain the data correspondence, comprises:
Continuously acquiring updated log data after the current log data;
and updating the relation prediction model based on the update log data when the update log data meets preset conditions.
5. The log data management method as claimed in claim 1, wherein one piece of the abnormal log data corresponds to one or more security events, and the performing association analysis on the abnormal log data to obtain a data correspondence includes:
obtaining the office factors corresponding to the abnormal log data;
matching factors of each of the office departments to obtain one or more corresponding target security events;
And associating each target security event with the corresponding abnormal log data to obtain a data corresponding relation.
6. A log data administration device, characterized in that the log data administration device comprises:
an acquisition module for acquiring current log data of the security device
The data acquisition module is used for acquiring the history log data of the safety equipment;
the extraction module is used for extracting abnormal log data from the history log data;
The determining module is used for determining the safety equipment corresponding to each abnormal log data;
the statistics module is used for counting the occurrence times of the security devices corresponding to the security events by taking the security events as dimensions;
The security event determining module is used for determining that the security event corresponding to the largest occurrence number is a target security event corresponding to any abnormal log data, wherein the security event corresponding to the largest occurrence number represents that the association between the abnormal log data and the target security event is strongest;
the association module is used for associating the target security event with any abnormal log data to obtain a data corresponding relation;
and the treatment module is used for treating the current log data based on the data corresponding relation.
7. An electronic device, the electronic device comprising: a memory, a processor, and a log data governance program stored on the memory and executable on the processor, the log data governance program configured to implement the steps of the log data governance method of any one of claims 1 to 5.
8. A computer readable storage medium, wherein a log data governance program is stored on the computer readable storage medium, which when executed by a processor, implements the steps of the log data governance method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311347841.5A CN117473225B (en) | 2023-10-17 | 2023-10-17 | Log data management method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311347841.5A CN117473225B (en) | 2023-10-17 | 2023-10-17 | Log data management method and device, electronic equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117473225A CN117473225A (en) | 2024-01-30 |
| CN117473225B true CN117473225B (en) | 2024-10-01 |
Family
ID=89637022
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311347841.5A Active CN117473225B (en) | 2023-10-17 | 2023-10-17 | Log data management method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117473225B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112235327A (en) * | 2020-12-16 | 2021-01-15 | 中移(苏州)软件技术有限公司 | Abnormal log detection method, apparatus, device, and computer-readable storage medium |
| CN114598513A (en) * | 2022-02-24 | 2022-06-07 | 烽台科技（北京）有限公司 | Industrial control threat event response method and device, industrial control equipment and medium |
| CN115622867A (en) * | 2022-10-08 | 2023-01-17 | 华能国际电力股份有限公司 | Method and system for early warning and classification of security incidents in industrial control system |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7644365B2 (en) * | 2003-09-12 | 2010-01-05 | Cisco Technology, Inc. | Method and system for displaying network security incidents |
| CN106209826A (en) * | 2016-07-08 | 2016-12-07 | 瑞达信息安全产业股份有限公司 | A kind of safety case investigation method of Network Security Device monitoring |
| CN110647446B (en) * | 2018-06-26 | 2023-02-21 | 中兴通讯股份有限公司 | Log fault association and prediction method, device, equipment and storage medium |
| CN109241461B (en) * | 2018-08-10 | 2020-05-22 | 新华三信息安全技术有限公司 | User portrait construction method and device |
| CN111177095B (en) * | 2019-12-10 | 2023-10-27 | 中移（杭州）信息技术有限公司 | Log analysis method, device, computer equipment and storage medium |
| CN111190876A (en) * | 2019-12-31 | 2020-05-22 | 天津浪淘科技股份有限公司 | Log management system and operation method thereof |
| CN115269304A (en) * | 2021-04-29 | 2022-11-01 | 超聚变数字技术有限公司 | Log anomaly detection model training method, device and equipment |
| CN113486334A (en) * | 2021-05-25 | 2021-10-08 | 新华三信息安全技术有限公司 | Network attack prediction method and device, electronic equipment and storage medium |
| CN114205143B (en) * | 2021-12-09 | 2024-08-16 | 国家电网有限公司信息通信分公司 | A method and system for intelligent collaborative defense of heterogeneous security devices |
| CN115630404A (en) * | 2022-10-26 | 2023-01-20 | 中国电子科技集团公司第三十研究所 | Data security management service method |
-
2023
- 2023-10-17 CN CN202311347841.5A patent/CN117473225B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112235327A (en) * | 2020-12-16 | 2021-01-15 | 中移(苏州)软件技术有限公司 | Abnormal log detection method, apparatus, device, and computer-readable storage medium |
| CN114598513A (en) * | 2022-02-24 | 2022-06-07 | 烽台科技（北京）有限公司 | Industrial control threat event response method and device, industrial control equipment and medium |
| CN115622867A (en) * | 2022-10-08 | 2023-01-17 | 华能国际电力股份有限公司 | Method and system for early warning and classification of security incidents in industrial control system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117473225A (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112637220B (en) | Industrial control system safety protection method and device | |
| Apruzzese et al. | On the effectiveness of machine and deep learning for cyber security | |
| CN108471429B (en) | Network attack warning method and system | |
| CN103026345B (en) | For the dynamic multidimensional pattern of event monitoring priority | |
| CN110602029B (en) | Method and system for identifying network attack | |
| CN114338064B (en) | Method, device, system, equipment and storage medium for identifying network traffic type | |
| CN109831459B (en) | Method, device, storage medium and terminal equipment for secure access | |
| CN109274632A (en) | Method and device for identifying website | |
| CN113852625B (en) | A weak password monitoring method, device, equipment and storage medium | |
| CN115795330A (en) | Medical information anomaly detection method and system based on AI algorithm | |
| CN110313161A (en) | The detection based on IPFIX to the amplification attack on database | |
| CN110365636B (en) | Method and device for discriminating data source of industrial control honeypot attack | |
| CN119696931B (en) | Intelligent protection method and system for heterogeneous industrial control networks based on large models | |
| CN111159702A (en) | Process list generation method and device | |
| CN118337484A (en) | A network information security analysis method and system based on big data | |
| CN115378713A (en) | Block chain application early warning defense method, storage medium and electronic equipment | |
| CN117473225B (en) | Log data management method and device, electronic equipment and readable storage medium | |
| CN111625700B (en) | Anti-grabbing method, device, equipment and computer storage medium | |
| CN118611949A (en) | Malicious Internet Protocol address analysis method, device, equipment and readable storage medium | |
| CN110224975B (en) | Method and device for determining APT information, storage medium, and electronic device | |
| CN117650938A (en) | Industry industrial system network threat handling method and device based on data association analysis | |
| CN114205146B (en) | Processing method and device for multi-source heterogeneous security log | |
| CN114065225B (en) | A business vulnerability protection method and system | |
| CN116961926A (en) | Abnormal traffic attack identification method and device, electronic equipment and storage medium | |
| CN118200022B (en) | Data encryption method and system based on malicious attacks on big data networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |