[go: up one dir, main page]

CN119135949B - Digital video monitoring method - Google Patents

Digital video monitoring method Download PDF

Info

Publication number
CN119135949B
CN119135949B CN202411254155.8A CN202411254155A CN119135949B CN 119135949 B CN119135949 B CN 119135949B CN 202411254155 A CN202411254155 A CN 202411254155A CN 119135949 B CN119135949 B CN 119135949B
Authority
CN
China
Prior art keywords
flow
time
abnormal
digital video
event
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202411254155.8A
Other languages
Chinese (zh)
Other versions
CN119135949A (en
Inventor
吴天俊
唐元正
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Jueming Artificial Intelligence Co ltd
Original Assignee
Shenzhen Jueming Artificial Intelligence Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Jueming Artificial Intelligence Co ltd filed Critical Shenzhen Jueming Artificial Intelligence Co ltd
Priority to CN202411254155.8A priority Critical patent/CN119135949B/en
Publication of CN119135949A publication Critical patent/CN119135949A/en
Application granted granted Critical
Publication of CN119135949B publication Critical patent/CN119135949B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23412—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8547—Content authoring involving timestamps for synchronizing content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to the technical field of digital monitoring and discloses a digital video monitoring method which comprises the steps of carrying out digital processing on video content, identifying real-time digital video streams corresponding to the digital video content, identifying picture content, sound content and video flow corresponding to the real-time digital video streams, detecting video behavior data of the real-time digital video streams according to the picture content and the sound content, identifying abnormal flow events corresponding to the video flow according to abnormal flow triggering events, identifying abnormal behavior events corresponding to the video behavior data according to the abnormal flow triggering events, generating abnormal data stream time sequences corresponding to the real-time digital video streams according to abnormal flow time points corresponding to the abnormal flow events and abnormal behavior time points corresponding to the abnormal behavior events, generating two-dimensional monitoring fusion graphs of the real-time digital video streams through the abnormal data stream time sequences, and analyzing monitoring states of the real-time digital video streams according to the two-dimensional monitoring fusion graphs. The invention can be used for digital video monitoring accuracy.

Description

Digital video monitoring method
Technical Field
The invention relates to the technical field of digital monitoring, in particular to a digital video monitoring method.
Background
With the development of urban, public safety demands and technical progress, remote monitoring becomes more feasible and efficient, and the demands of digital video monitoring are also continuously increased, so that the wide application of digital video monitoring is promoted, but in order to improve the accuracy of digital video monitoring, abnormal states in videos need to be monitored from multiple aspects, so that the accurate monitoring of the digital video is realized.
The existing digital video monitoring technology is to identify abnormal data in a video aiming at sudden changes by identifying sudden changes in a scene. In practical applications, identifying abnormal video content through only a single scene change may result in excessive one-sided identification of abnormal states for digital video monitoring, and thus, the accuracy in performing digital video monitoring is low.
Disclosure of Invention
The invention provides a digital video monitoring method, which mainly aims to solve the problem of lower accuracy in digital video monitoring.
In order to achieve the above object, the present invention provides a digital video monitoring method, comprising:
Acquiring video content in real time, performing digital processing on the video content to obtain digital video content, and identifying a real-time digital video stream corresponding to the digital video content;
Identifying picture content and sound content corresponding to the real-time digital video stream, detecting video flow of the real-time digital video stream, and detecting video behavior data of the real-time digital video stream according to the picture content and the sound content;
The method comprises the steps of generating a flow time sequence corresponding to the video flow according to a preset timestamp, and calculating a flow factor corresponding to the video flow according to each video flow in the flow time sequence and a flow average value in the flow time sequence, wherein the flow factor calculation formula is as follows:
Wherein G r is a flow factor corresponding to the r real-time digital video stream, F r is a video flow corresponding to the r real-time digital video stream, F is the flow mean value, Q is a constant, and δ is a flow standard deviation in the flow time sequence;
when the flow factor is a preset first factor value, extracting video flow corresponding to the flow factor, and identifying a video event corresponding to the video flow; the video event is matched with an event in a preset abnormal flow triggering event to obtain an abnormal flow event;
generating an abnormal data stream time sequence corresponding to the real-time digital video stream according to the abnormal flow time point corresponding to the abnormal flow event and the abnormal behavior time point corresponding to the abnormal behavior event;
And generating a two-dimensional monitoring fusion graph of the real-time digital video stream through the abnormal data stream time sequence, and analyzing the monitoring state of the real-time digital video stream according to the two-dimensional monitoring fusion graph.
Optionally, the digitizing the video content to obtain digital video content includes:
Converting an original video signal corresponding to the video content into a digital signal;
performing signal enhancement processing on the digital signal to obtain a digital enhanced signal;
and compressing the digital enhanced signal to obtain digital video content.
Optionally, the identifying the real-time digital video stream corresponding to the digital video content includes:
carrying out video frame decomposition on the digital video content according to a preset frame rate to obtain a digital frame image;
generating a frame sequence corresponding to the digital frame image according to a preset time stamp;
the frame sequence is determined as the real-time digital video stream.
Optionally, the identifying the picture content and the sound content corresponding to the real-time digital video stream includes:
Identifying image data corresponding to each real-time digital video stream, and extracting picture events corresponding to the image data;
generating picture content corresponding to each real-time digital video stream according to the picture event;
Extracting audio characteristics corresponding to each real-time digital video stream, and determining sound events and sound emotions corresponding to each real-time digital video stream according to the audio characteristics;
And generating sound content corresponding to each real-time digital video stream according to the sound event and the sound emotion.
Optionally, the detecting video behavior data of the real-time digital video stream according to the picture content and the sound content includes:
Extracting a stream sequence number corresponding to the real-time digital video stream;
generating a first event index of a picture event corresponding to the picture content according to the stream sequence number;
generating a second event index of the sound event corresponding to the sound content according to the stream sequence number;
calculating the association degree between the picture event and the sound event according to the index identifier corresponding to the first event index and the second event index:
Wherein S is the association degree, w k is the association weight corresponding to the kth index pair, v ik is the picture event vector corresponding to the ith index identifier in the kth index pair, a jk is the sound event vector corresponding to the jth index identifier in the kth index pair, and n is the number of index pairs;
and fusing the picture event and the sound event into video behavior data corresponding to the real-time digital video stream according to the association degree.
Optionally, the matching the video event with an event in a preset abnormal flow triggering event to obtain an abnormal flow event includes:
extracting abnormal event type data in a preset abnormal flow triggering event;
Classifying the video event to obtain video event class data;
vector conversion is carried out on the abnormal event type data to obtain an abnormal event type vector;
vector conversion is carried out on the video event type data to obtain video event type vectors;
Calculating the matching degree between the abnormal event category vector and the video event category vector;
and selecting the abnormal flow triggering event with the highest matching degree as an abnormal flow event.
Optionally, the identifying the abnormal behavior event corresponding to the video behavior data according to the preset abnormal behavior trigger event includes:
Extracting behavior characteristics of each real-time digital video stream in the video behavior data, and generating a behavior time sequence corresponding to the video behavior data according to a preset event stamp and the behavior characteristics;
Calculating a behavior factor corresponding to the video behavior data according to each behavior feature in the behavior time sequence and a behavior feature standard deviation in the behavior time sequence, wherein the behavior factor calculation formula is as follows:
Wherein H r is a behavior factor corresponding to the r-th real-time digital video stream, sigma rα is a feature value corresponding to the alpha-th feature in behavior features in the r-th real-time digital video stream, m is a feature quantity in the behavior features, P is a constant, Standard deviation of the behavior characteristic;
When the behavior factor is a preset first factor value, extracting video behavior data corresponding to the behavior factor, and identifying a behavior event corresponding to the video behavior data;
and matching the behavior event with an event in a preset abnormal behavior trigger event to obtain an abnormal behavior event.
Optionally, the generating the abnormal data stream time sequence corresponding to the real-time digital video stream according to the abnormal traffic time point corresponding to the abnormal traffic event and the abnormal behavior time point corresponding to the abnormal behavior event includes:
Marking an abnormal flow time point corresponding to the abnormal flow event on a preset flow time axis;
Marking the abnormal behavior time point corresponding to the abnormal behavior event on a preset behavior time axis;
And sequencing the event points on the flow time axis and the time points on the behavior time axis to obtain an abnormal data stream time sequence corresponding to the real-time digital video stream.
Optionally, the generating the two-dimensional monitoring fusion map of the real-time digital video stream through the abnormal data stream time sequence includes:
Determining a value corresponding to a time point on a flow time axis in the abnormal data flow time sequence as a preset first abnormal value, and generating a first curve according to the first abnormal value;
Determining a value corresponding to a time point on a behavior time axis in the abnormal data flow time sequence as a preset second abnormal value, and generating a second curve according to the second abnormal value;
and fusing the first curve and the second curve into a two-dimensional monitoring fusion diagram of the real-time digital video stream according to a preset coordinate system.
Optionally, the analyzing the monitoring state of the real-time digital video stream according to the two-dimensional monitoring fusion graph includes:
counting the number of abnormal points corresponding to the real-time digital video stream according to the curve trend of the first curve and the second curve in the two-dimensional monitoring fusion graph;
Determining an abnormal grade corresponding to the real-time digital video stream according to the abnormal point quantity and a preset abnormal point threshold value;
And determining the monitoring state of the real-time digital video stream through the abnormal grade.
The embodiment of the invention can enable the digital monitoring to respond immediately by acquiring the video content in real time and performing digital processing, can monitor and accurately analyze events and behaviors occurring in a monitoring area by identifying the picture and sound content of the digital video content, can quickly identify and respond to potential security threats and abnormal conditions by detecting video behavior data and abnormal flow events, thereby improving the safety and early warning capability of a monitoring system, can intuitively know the abnormal conditions and trends of the monitoring area by generating an abnormal data flow time sequence and a two-dimensional monitoring fusion graph, can evaluate the monitoring state in real time by analyzing the two-dimensional monitoring fusion graph, and can improve the management efficiency and response speed of the monitoring area, and thus, the monitoring management is more efficient and reliable. Therefore, the digital video monitoring method provided by the invention can solve the problem of lower accuracy in the process of digital video monitoring.
Drawings
Fig. 1 is a flow chart of a digital video monitoring method according to an embodiment of the invention;
FIG. 2 is a flow chart of identifying digital video content according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating the identification of abnormal traffic events according to an embodiment of the present invention;
fig. 4 is a functional block diagram of a digital video monitoring system according to an embodiment of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The embodiment of the application provides a digital video monitoring method. The execution subject of the digital video monitoring method includes, but is not limited to, at least one of a server, a terminal, and the like, which can be configured to execute the method provided by the embodiment of the application. In other words, the digital video monitoring method may be performed by software or hardware installed in a terminal device or a server device, and the software may be a blockchain platform. The server side comprises, but is not limited to, a single server, a server cluster, a cloud server or a cloud server cluster and the like. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.
Referring to fig. 1, a flow chart of a digital video monitoring method according to an embodiment of the invention is shown.
In this embodiment, the digital video monitoring method includes:
S1, acquiring video content in real time, performing digital processing on the video content to obtain digital video content, and identifying a real-time digital video stream corresponding to the digital video content.
In the embodiment of the present invention, the video content refers to all information and data contained in the video file, and the visual portion of the video includes image data of each frame, where the image frames are combined together to form a video sequence, and the video may include sound portions, such as dialogue, music, background sound effects, etc., and the audio track is usually played synchronously with the video frame, where the video content may be obtained in real time from a pre-stored storage area through computer sentences (such as Java sentences, python sentences, etc.) with a data grabbing function, where the storage area includes, but is not limited to, a database, and a blockchain.
Further, in order to analyze the video content in real time, the video content needs to be converted into a digital format, and the video content in a specific time period or event can be quickly searched through the digital format, so that the query efficiency is improved.
In the embodiment of the invention, the digital video content refers to video content obtained through digital processing, and video data represented in a digital form comprises data such as images, audio and the like.
In the embodiment of the present invention, the digitizing the video content to obtain digital video content includes:
Converting an original video signal corresponding to the video content into a digital signal;
performing signal enhancement processing on the digital signal to obtain a digital enhanced signal;
and compressing the digital enhanced signal to obtain digital video content.
In detail, an original video signal is obtained by using an acquisition device, the original video signal is usually an analog signal, the analog video signal is converted into a digital signal through a mode converter (ADC), namely, a continuous value of the signal is converted into a discrete value, so that a digital data stream is generated, noise in a video is removed through a filter, so that the image definition is improved, contrast and brightness of the digital video signal are adjusted, the visual effect of the image is enhanced, the detail and edge definition of the image are improved by adopting a sharpening algorithm, the enhanced digital signal is compressed through a video compression algorithm (such as H.264 and H.265), the volume of video data can be reduced by compression so as to facilitate storage and transmission, the compressed code rate is adjusted according to requirements, the image quality and the file size are balanced, the enhanced digital video content is obtained, the monitoring system is ensured to provide clear and reliable video content, and the best balance is achieved in the aspects of storage and transmission.
Further, in order to be able to view and analyze the video stream in real time, the video content needs to be analyzed frame by frame in order to quickly respond to potential security threats or abnormal conditions, abnormal behaviors are detected and an alarm is given, and monitoring efficiency is improved, so that the video content needs to be divided into video streams.
In the embodiment of the invention, the real-time digital video stream refers to a digital video data stream which is continuously transmitted and processed, wherein video frames are rapidly transmitted according to time sequence, and the video stream is updated in real time, so that a user can see the latest picture content for instant watching and analysis.
In an embodiment of the present invention, the identifying a real-time digital video stream corresponding to the digital video content includes:
carrying out video frame decomposition on the digital video content according to a preset frame rate to obtain a digital frame image;
generating a frame sequence corresponding to the digital frame image according to a preset time stamp;
the frame sequence is determined as the real-time digital video stream.
In detail, the digital video content is decomposed into a series of digital frame images according to a preset frame rate, namely, the video content is cut into individual frame images according to a set time interval, the number of frames processed per second (namely, the frame rate) determines the smoothness of the video, a continuous video stream can be decomposed into discrete image frames, further, a time stamp is allocated to each digital frame image and recorded at a specific position of a video clock, the time stamp is helpful for the subsequent synchronization and retrieval of the frame images, if the video frame rate is 30 frames per second, the time stamp of each frame is allocated according to 30 time points per second, each frame image allocated with the time stamp is arranged in time sequence to form an ordered frame sequence, the frame sequence represents the time progress of the video, the generated frame sequence can be correctly played or analyzed according to the time sequence of the video, the image frames are arranged in time sequence to form a continuous video stream, and the video data can be continuously and synchronously transmitted and displayed.
Furthermore, the video frame images are precisely associated with time to form a dynamic video stream, so that efficient video playing and real-time processing can be realized, but in order to timely detect abnormal content in video, frame-by-frame analysis is required to be performed on video content corresponding to the real-time digital video stream.
S2, identifying the picture content and the sound content corresponding to the real-time digital video stream, detecting the video flow of the real-time digital video stream, and detecting the video behavior data of the real-time digital video stream according to the picture content and the sound content.
In the embodiment of the invention, the picture content refers to visual information captured in a video stream, including scenes, characters, objects, actions and the like, which describe the composition of the picture and the events shown, and the sound content refers to audio information in the video stream, including sound events (such as dialogue and background noise), emotion states (such as happiness and sadness) and audio features (such as tone and rhythm), which describe the properties and the context of sound.
In an embodiment of the present invention, referring to fig. 2, the identifying the picture content and the sound content corresponding to the real-time digital video stream includes:
s21, identifying image data corresponding to each real-time digital video stream, and extracting picture events corresponding to the image data;
s22, generating picture content corresponding to each real-time digital video stream according to the picture event;
S23, extracting audio characteristics corresponding to each real-time digital video stream, and determining sound events and sound emotions corresponding to each real-time digital video stream according to the audio characteristics;
S24, generating sound content corresponding to each real-time digital video stream according to the sound event and the sound emotion.
In detail, the images in each real-time video stream are analyzed through computer vision technology (such as object detection and scene recognition), events (such as people, objects and behaviors) in the images are identified and marked, detailed information of the events is extracted, the related objects, actions, time and places and the like are included, the identified event information is integrated, detailed image content description is generated, namely, each frame or each period of the images in the video stream is analyzed, the events are identified, such as the fact that a car passes through, a person walks into a room or a person makes a specific action is detected, the identified events are classified into different types, such as traffic events (car and pedestrian), behavior events (running and jumping), environmental events (sunny days and raining) and the like, and the image content description is generated according to the identified events. For example, if an event is identified that a person enters a room, a description is generated, a person walks into the room, wears a red jacket, and integrates multiple event information in the video stream to provide an overall summary of the picture content, for example, in a meeting room, one person wearing a blue suit is speaking, and another person is recording notes, so that the picture event is combined with the context information (e.g., time, place) of the video stream to ensure that the generated picture content accurately reflects the actual situation.
Specifically, audio processing techniques (e.g., MFCC, audio signal analysis) are utilized to extract audio features, analyze audio features, identify sound events, such as recognition dialogue, background music, traffic noise, etc., and classify the identified sound events into different types, such as human voice, ambient noise, mechanical sound, etc., and determine emotion states in the sound, such as anger, happiness, sadness, etc., through emotion semantic analysis, tag the sound events with corresponding emotion information, such as anger dialogue or happy music, and generate detailed sound content descriptions based on the sound events and emotion. For example, in video streams, where the background has a jerky traffic noise, several angry conversations occur, a comprehensive sound content description is created in conjunction with sound events and emotions, and contextual information is provided, e.g., where several people in the conference are in vigorous disputes, accompanied by occasional background noise and music, to obtain sound content corresponding to each real-time digital video stream.
Further, not only the picture content and the sound content in the real-time digital video stream need to be identified, but also the flow corresponding to the real-time digital video stream needs to be detected, and the flow fluctuation needs to be monitored in real time, so that the problem in video playing can be solved in time, wherein the video flow of the real-time digital video stream is detected, that is, the real-time data including the bit rate, the resolution, the frame rate and the like are obtained from the video stream through a network flow monitoring tool, so that the video flow in the real-time digital video stream is determined according to the flow (bit/second) =resolution×frame rate×bit depth×compression, wherein the resolution refers to the width×height, the frame rate refers to the frame number per second, and the bit depth refers to the bit number per pixel, so that the video flow corresponding to the real-time digital video stream is obtained.
Further, specific events (e.g., music, speech, ambient noise) in the audio are identified, emotion components (e.g., anger, happiness) of the sound are analyzed, voiceprint features of the speaker are extracted and analyzed, and the picture and the sound content are synchronized, so that the events in the picture are associated with the events in the sound for more comprehensive understanding.
In the embodiment of the invention, the video behavior data refers to behavior data after synchronizing video picture content and sound content, and visual information and auditory information in video are combined to more comprehensively understand and analyze the behavior patterns in the video, so that more accurate information is provided for video monitoring.
In an embodiment of the present invention, the detecting video behavior data of the real-time digital video stream according to the picture content and the sound content includes:
Extracting a stream sequence number corresponding to the real-time digital video stream;
generating a first event index of a picture event corresponding to the picture content according to the stream sequence number;
generating a second event index of the sound event corresponding to the sound content according to the stream sequence number;
calculating the association degree between the picture event and the sound event according to the index identifier corresponding to the first event index and the second event index:
Wherein S is the association degree, w k is the association weight corresponding to the kth index pair, v ik is the picture event vector corresponding to the ith index identifier in the kth index pair, a jk is the sound event vector corresponding to the jth index identifier in the kth index pair, and n is the number of index pairs;
and fusing the picture event and the sound event into video behavior data corresponding to the real-time digital video stream according to the association degree.
In detail, a stream sequence number corresponding to each frame image or each section of audio is extracted from the real-time digital video stream, and the stream sequence number can be used to uniquely identify video and audio events, and an event index of a picture event is generated according to the stream sequence number, then a first event index of the picture event refers to a specific picture content corresponding to the uniquely identified video event, and an event index of a sound event is generated according to the stream sequence number, then a second event index of the sound event refers to a specific sound content corresponding to the uniquely identified audio event, and the stream sequence number corresponding to the real-time digital video stream is { a 1,A2,…,At }, then the first event index corresponding to the picture event is {1, 2..times., t }, and the second event index corresponding to the sound event is {1, 2...
Specifically, the index corresponding to the first event index is identified as {1,2, & gt, t }, the index corresponding to the second event index is identified as {1,2, & gt, t }, the association degree between real-time digital video streams corresponding to different stream numbers is calculated according to the picture event and the sound event corresponding to the index identification, the picture event and the sound event are vectorized, the picture event and the sound event can be converted into picture event vectors and sound event vectors through a vector conversion model (such as a BERT model), and an association weight is provided between each index pair, the index corresponding to the first event index is identified as {1,2, & gt, t } and the index corresponding to the second event index are identified as {1,2, & gt, t } to construct an index identification matrix, if the matrix value corresponding to the index 1 in the first event index identifier and the index 1 in the second event index identifier in the index identifier matrix is the picture event vector and the video event vector corresponding to the stream serial number a 1, the weight corresponding to the index item with the same index identifier is set to x/(x+y), if the association weight corresponding to the first index 1 and the second index 2 is 1, the association weight corresponding to the first index 1 and the second index 2 is 1/(1+2), so as to obtain the association weight corresponding to each index item, the importance of the index pair is identified, so that the inner product of the picture event vector and the sound event vector of each index pair is multiplied by the weight w k of the index pair, then all index pairs are summed, the value reflects the correlation between the weighted picture event and the sound event, namely, the weighted cosine similarity is calculated, the similarity degree between the picture event vector and the sound event vector is measured, and then the picture event and the video event with the highest association degree are used as a matching event, namely, feature vectors of the picture event and the sound event are directly combined into a more comprehensive feature vector which is used for representing video behavior data corresponding to the real-time digital video stream, if the association degree between the first index 1 and the second index 1is the highest, the picture event vector and the video event vector corresponding to the stream serial number A 1 are combined into a more comprehensive feature vector which is used for representing the video behavior data corresponding to the stream serial number A 1.
Further, according to the detected video flow and the detected video behavior data, abnormal events in video monitoring can be identified according to the video flow and the video behavior data, so that the intelligent level and the real-time response capability of the video monitoring are improved.
S3, identifying an abnormal flow event corresponding to the video flow according to a preset abnormal flow triggering event, and identifying an abnormal behavior event corresponding to the video behavior data according to a preset abnormal behavior triggering event.
In the embodiment of the invention, the abnormal traffic event refers to the situation that the video traffic is abnormal in the digital monitoring process, and indicates that the video behavior corresponding to the traffic deviates from the normal mode, such as the traffic suddenly increases or decreases greatly, and exceeds the normal fluctuation range, such as the network attack, the equipment fault or the obvious change of the user behavior, such as the abnormal fluctuation or the irregularity of the traffic data deviates from the data mode or the expected behavior.
In the embodiment of the present invention, referring to fig. 3, the identifying an abnormal traffic event corresponding to the video traffic according to a preset abnormal traffic trigger event includes:
S31, generating a flow time sequence corresponding to the video flow according to a preset time stamp;
s32, calculating a flow factor corresponding to the video flow according to each video flow in the flow time sequence and a flow average value in the flow time sequence, wherein the flow factor calculation formula is as follows:
Wherein G r is a flow factor corresponding to the r real-time digital video stream, F r is a video flow corresponding to the r real-time digital video stream, F is the flow mean value, Q is a constant, and δ is a flow standard deviation in the flow time sequence;
S33, when the flow factor is a preset first factor value, extracting video flow corresponding to the flow factor, and identifying a video event corresponding to the video flow;
And S34, matching the video event with an event in a preset abnormal flow triggering event to obtain an abnormal flow event.
In detail, the video traffic detected at each moment is generated into a traffic sequence, if the video traffic detected at t 1 is L 1, the video traffic detected at t 2 is L 2, the video traffic detected at t n is L n, the traffic sequence is { L t1,Lt2,…,Ltn }, thus, the traffic factor is calculated according to each video traffic in the traffic sequence and the traffic means corresponding to all video traffic in the traffic sequence, that is, when the absolute value of the difference between a certain video traffic F r and the traffic means F in the traffic sequence is greater than the product of the traffic standard deviation delta corresponding to all video traffic in the traffic sequence and the constant Q, the traffic factor is defined as 1, which indicates that there is video traffic abnormality, and when the absolute value of the difference between a certain video traffic F r and the traffic means F in the traffic sequence is less than or equal to the product of the traffic standard deviation delta corresponding to all video traffic in the traffic sequence Q, the traffic factor is defined as 0, which indicates that there is no video traffic abnormality, wherein the traffic factor is used for indicating whether the video traffic is a normal or not, and the traffic factor Q is set as a normal threshold value, which is set to be used for determining whether the normal value of the traffic is a normal value, and the normal value is set to be 35, and the normal value is set to be combined with the normal value.
Specifically, when the flow factor is a first factor value (numerical value 1), corresponding video flows with the corresponding flow factor of 1 are collected, if the flow factors corresponding to the 1 st and 5 th real-time digital video flows are 1, the video flows corresponding to the 1 st and 5 th real-time digital video flows are extracted, and the video events corresponding to the extracted abnormal video flow data are identified, that is, the content in the frames is checked by using image processing or computer vision technology, any significant change or activity is identified, for example, a motion detection algorithm can be applied to find abnormal activity, and then the analysis result is compared with a predefined event category to classify the identified abnormal event, for example, the abnormal activity may be classified as "flow sudden increase", "flow decrease" or "flow fluctuation", and the like.
In the embodiment of the present invention, the matching the video event with an event in a preset abnormal flow triggering event to obtain an abnormal flow event includes:
extracting abnormal event type data in a preset abnormal flow triggering event;
Classifying the video event to obtain video event class data;
vector conversion is carried out on the abnormal event type data to obtain an abnormal event type vector;
vector conversion is carried out on the video event type data to obtain video event type vectors;
Calculating the matching degree between the abnormal event category vector and the video event category vector;
and selecting the abnormal flow triggering event with the highest matching degree as an abnormal flow event.
In detail, the video event is matched with an event in a preset abnormal flow triggering event, wherein the abnormal flow triggering event comprises a flow surge, the flow suddenly increases to be far higher than a normal level and possibly represents an attack (such as DDoS attack) or system abnormality, the flow suddenly drops and possibly represents service faults or network problems, the flow fluctuates frequently in a short time and possibly represents unstable system or problems, an abnormal mode occurs, flow data does not accord with the behavior of the normal mode, such as flow increase in an unusual period, and if the video event is the flow surge, the abnormal flow event is the attack.
The video event category data comprises flow sudden increase, flow decrease, flow fluctuation and abnormal modes, the video event category data comprises the flow sudden increase, the flow decrease, the flow fluctuation and the abnormal modes, the video event category data is further converted into vector representation, the video event category data is also converted into vector representation through word embedding (Word Embedding) or other characteristic vector representation, the video event category data is ensured to be consistent with the representation mode of the abnormal event category vector, the matching degree between the abnormal event category vector and the video event category vector is measured by utilizing an algorithm (such as cosine similarity, euclidean distance and the like) for calculating the similarity, and the abnormal flow trigger event which is most matched with the video event is selected according to the calculated matching degree and is determined as the current abnormal flow event.
Further, in order to more comprehensively analyze abnormal events in digital video monitoring, not only abnormal traffic events in digital video monitoring are identified, but also abnormal behavior events in digital video monitoring need to be analyzed.
In the embodiment of the invention, the abnormal behavior event refers to an event that the behavior characteristic deviates from a normal mode or a preset standard in video monitoring or data analysis, and shows abnormal or abnormal behavior, including an unexpected activity mode, behavior not conforming to the conventional condition and the like, and may indicate a system problem, security threat or other abnormal condition needing attention.
In the embodiment of the present invention, the identifying the abnormal behavior event corresponding to the video behavior data according to the preset abnormal behavior trigger event includes:
Extracting behavior characteristics of each real-time digital video stream in the video behavior data, and generating a behavior time sequence corresponding to the video behavior data according to a preset event stamp and the behavior characteristics;
Calculating a behavior factor corresponding to the video behavior data according to each behavior feature in the behavior time sequence and a behavior feature standard deviation in the behavior time sequence, wherein the behavior factor calculation formula is as follows:
Wherein H r is a behavior factor corresponding to the r-th real-time digital video stream, sigma rα is a feature value corresponding to the alpha-th feature in behavior features in the r-th real-time digital video stream, m is a feature quantity in the behavior features, P is a constant, Standard deviation of the behavior characteristic;
When the behavior factor is a preset first factor value, extracting video behavior data corresponding to the behavior factor, and identifying a behavior event corresponding to the video behavior data;
and matching the behavior event with an event in a preset abnormal behavior trigger event to obtain an abnormal behavior event.
In detail, the behavior features include motion features, gesture features, sound features, wherein the motion features describe how an object or an individual in a video moves and changes, including speed, direction and motion trajectory, then a motion vector field of the object in the video is analyzed by an optical flow technique, describing the direction and speed of the motion of the object on an image plane, then the motion features are represented by the speed, the gesture features describe the position and posture of the individual or the object in space, including the relative position and angle of the body parts of the individual, then key points (such as head, shoulder, elbow, knee, etc.) of the human body are identified by a deep learning model, and the relative positions thereof are estimated, then the gesture features are represented by coordinate values, the sound features describe audio information accompanying in the video, including speech, background noise or other sound effects, then by extracting the mel spectrum features of audio signals for speech recognition and sound classification, then the sound features are represented by the frequency of each audio segment, then behavior features detected at each moment are generated into behavior time sequence, such as behavior feature detected at moment t 1 is H 11,H12,H13, behavior feature detected at moment 2 is H 21,H22,H23, behavior feature detected at moment n is behavior feature detected at moment n1,Hn2,Hn3 is time sequence HWherein the method comprises the steps ofAnd calculating the behavior factors for the motion characteristic H n1, the gesture characteristic H n2 and the sound characteristic H n3 in the behavior characteristics at the time t n according to each behavior characteristic in the behavior time sequence and the behavior mean value corresponding to all the behavior characteristics in the behavior time sequence.
Specifically, when the absolute value of the difference between the feature value σ rα of the alpha-th feature in a certain behavior feature in the behavior sequence and the behavior mean of the alpha-th feature is larger than the behavior feature standard deviation corresponding to all the behavior features of the alpha-th feature in the behavior sequenceThe product of the motion characteristic, the gesture characteristic and the sound characteristic is required to be standardized respectively, the data is converted into a distribution with the mean value of 0 and the standard deviation of 1, if the behavior characteristic corresponding to the 1 st moment is H 11,H12,H13, the mean value and the standard deviation of H 11,H12,H13 are calculated, then the calculated mean value and standard deviation are analyzed and calculated with the behavior characteristic corresponding to a certain moment, so as to determine the behavior factor corresponding to the behavior characteristic, wherein the motion characteristic, the gesture characteristic and the sound characteristic are required to be standardized respectively because of different units, the data are converted into a distribution with the mean value of 0 and the standard deviation of 1, if the standardized speed is 0.63, the standardized gesture characteristic is (0.26,0.53), the standardized sound characteristic is 0.78, and the gesture characteristic is taken as a valueAnd when the absolute value of the difference between the characteristic value sigma rα of the alpha-th characteristic in a certain behavior characteristic in the behavior time sequence and the behavior mean value of the alpha-th characteristic is smaller than or equal to the product of the standard deviation phi of the behavior characteristics corresponding to all the behavior characteristics of the alpha-th characteristic in the behavior time sequence and a constant P, the behavior factor is defined as 0 and represents the video content without video behavior abnormality, wherein the behavior factor is used for indicating whether the behavior in the video is normal or not, P is a constant and is used for setting an abnormal threshold range, and the value of the behavior factor H r is determined by combining the characteristic value sigma rα in the behavior time sequence so as to identify whether the video behavior is abnormal or not, and the constant P is generally configured as a numerical value of 0.5.
Further, when the behavior factor is a first factor value (numerical value 1), corresponding video behavior data with the corresponding behavior factor of 1 is collected, if the corresponding behavior factor of 1 st and 5 th real-time digital video streams is 1, the corresponding video behavior data of 1 st and 5 th real-time digital video streams is extracted, and the corresponding behavior events are identified for the extracted abnormal video behavior data, namely, the extracted characteristics are analyzed by using a machine learning model or a pattern recognition algorithm, specific behavior events are identified, such as walking, running, calling and the like, and then the analysis results are compared with predefined event categories to classify the identified abnormal events, such as the abnormal behavior events are classified into face recognition anomalies, behavior anomalies, scene changes, audio anomalies, intrusion detection and the like, and the video events are matched with events in preset abnormal behavior triggering events, wherein the abnormal behavior triggering events comprise face recognition anomalies, behavior anomalies, scene changes, audio anomalies, intrusion detection and the like, the abnormal behaviors comprise false identity recognition and unknown person appearance, the abnormal behaviors comprise normal behavior detection, the abnormal behavior departure from the scene changes comprise the detection, the abnormal scene changes comprise the detection of the abnormal sound and the intrusion detection, and the abnormal environment detection comprises the abnormal sound and the intrusion detection.
Furthermore, in order to timely respond at the time point of occurrence of the abnormality, the time points of the abnormal flow event and the abnormal behavior event are integrated into one time sequence, so that the background and causal relationship of the abnormal event can be more comprehensively analyzed, and the abnormal event in the real-time digital video stream can be effectively managed and responded.
S4, generating an abnormal data stream time sequence corresponding to the real-time digital video stream according to the abnormal flow time point corresponding to the abnormal flow event and the abnormal behavior time point corresponding to the abnormal behavior event.
In the embodiment of the invention, the abnormal data flow time sequence is used for ordering and displaying the abnormal events in the video by taking the events as axes, and mainly displaying the abnormal activities in the data flow through a time sequence so as to help identify and analyze the modes and trends of the abnormal events in the video.
In the embodiment of the present invention, the generating the abnormal data stream time sequence corresponding to the real-time digital video stream according to the abnormal flow time point corresponding to the abnormal flow event and the abnormal behavior time point corresponding to the abnormal behavior event includes:
Marking an abnormal flow time point corresponding to the abnormal flow event on a preset flow time axis;
Marking the abnormal behavior time point corresponding to the abnormal behavior event on a preset behavior time axis;
And sequencing the event points on the flow time axis and the time points on the behavior time axis to obtain an abnormal data stream time sequence corresponding to the real-time digital video stream.
In detail, the traffic time axis is a time point for marking abnormal traffic time, the behavior time axis is a time point for marking abnormal behavior events, then the event point corresponding to the abnormal traffic event is recorded on the traffic time axis, the time point of the abnormal behavior event is recorded on the behavior time axis, the abnormal time point on the traffic time axis and the abnormal time point on the behavior time axis are combined into a unified time point set, the time point set comprises time information of all abnormal events, whether traffic is abnormal or behavior is abnormal, the combined time points are ordered according to the time sequence, so as to obtain a time sequence of abnormal data streams, namely, all the time points are stored in a list and ordered in ascending order, the time sequence of the abnormal data streams is created according to the ordered time points, the abnormal data stream time sequence can be expressed as a time sequence, wherein each time point corresponds to one abnormal event, the source (traffic abnormality or behavior abnormality) and the specific time of the occurrence of the abnormal event are marked, and the abnormal traffic and the abnormal behavior corresponding to the same time point are regarded as the same time point.
For example, assuming that the time point of the traffic anomaly event is [ T1, T2, T3], the time point of the behavior anomaly event is [ T3, T4], the combined time point set is [ T1, T2, T3, T4], and the ordered time point is [ T1, T2, T4, T3], the generated anomaly data stream timing is T1 and is traffic anomaly, T2 and T3 are traffic anomaly and behavior anomaly, and T4 is behavior anomaly.
Further, an abnormal data stream time sequence is generated based on the flow event and the behavior event, so that the abnormal event in the digital video monitoring can be analyzed more intuitively, visual information in the video stream and the time sequence information in the abnormal data stream are integrated, and deeper insight and real-time monitoring capability are provided.
S5, generating a two-dimensional monitoring fusion graph of the real-time digital video stream through the abnormal data stream time sequence, and analyzing the monitoring state of the real-time digital video stream according to the two-dimensional monitoring fusion graph.
In the embodiment of the invention, the two-dimensional monitoring fusion graph refers to a graph for integrating and visualizing abnormal flow and abnormal behavior in an abnormal data flow time sequence in a two-dimensional coordinate system, analyzes how different types of data are related to each other in time or space, and comprises an X-axis, a time axis, time distribution of display data and events, a Y-axis, different data values such as flow abnormal values and behavior abnormal values, and curves for displaying the change trend of flow and behavior abnormal.
In the embodiment of the present invention, the generating the two-dimensional monitoring fusion map of the real-time digital video stream through the abnormal data stream time sequence includes:
Determining a value corresponding to a time point on a flow time axis in the abnormal data flow time sequence as a preset first abnormal value, and generating a first curve according to the first abnormal value;
Determining a value corresponding to a time point on a behavior time axis in the abnormal data flow time sequence as a preset second abnormal value, and generating a second curve according to the second abnormal value;
and fusing the first curve and the second curve into a two-dimensional monitoring fusion diagram of the real-time digital video stream according to a preset coordinate system.
In detail, a time point on a flow time axis is extracted from an abnormal data flow time sequence and is determined to be a first abnormal value, the time point corresponding to an abnormal flow event is determined to be the first abnormal value, if an abnormal condition exists, the value corresponding to the time point is1, otherwise, the first abnormal value is1, a first curve is generated according to the abnormal value corresponding to the time point, the first curve is generated by the abnormal value on the flow time axis, the change condition of flow data with time is displayed and the video flow change is reflected, the time point on the action time axis is extracted from the abnormal data flow time sequence and is determined to be a second abnormal value, the time point corresponding to the abnormal action event is determined to be the second abnormal value, if the abnormal condition exists, the value corresponding to the time point is1, otherwise, the second abnormal value is1, the second curve is generated according to the abnormal value corresponding to the time point, the second curve is generated by the abnormal value on the action time axis, the change condition of the action data with time is displayed and the video action change is reflected, the first curve and the second curve are fused in a preset coordinate system, and the two-dimensional video data is formed.
Specifically, the abnormal flow event and the abnormal behavior event are displayed on one graph, and can be represented by lines with different colors, if the flow and the behavior corresponding to the same time point are normal, the two curves are overlapped, the time and the numerical variation of different abnormal data are observed by drawing the first curve and the second curve in the same coordinate system, meanwhile, the comprehensive understanding of the event is provided by combining the visual information in the video stream, and the screenshot or the related mark of the real-time video stream can be overlapped in a two-dimensional chart to display the relationship between the condition occurring in the video and the curve variation in the abnormal data stream.
Further, time and numerical changes of different abnormal data can be intuitively observed through the two-dimensional monitoring fusion graph, and meanwhile comprehensive understanding of events is provided by combining visual information in a video stream, so that the number of 0 and 1 in the two-dimensional monitoring fusion graph is counted and used for analyzing the monitoring running state of digital video monitoring.
In the embodiment of the invention, the monitoring state refers to the current health and stability condition in the monitoring process, the abnormal condition is judged and processed, and the monitoring state can be classified as normal, slightly abnormal, moderately abnormal or severely abnormal according to the number and the severity of abnormal points.
In the embodiment of the present invention, the analyzing the monitoring state of the real-time digital video stream according to the two-dimensional monitoring fusion graph includes:
counting the number of abnormal points corresponding to the real-time digital video stream according to the curve trend of the first curve and the second curve in the two-dimensional monitoring fusion graph;
Determining an abnormal grade corresponding to the real-time digital video stream according to the abnormal point quantity and a preset abnormal point threshold value;
And determining the monitoring state of the real-time digital video stream through the abnormal grade.
In detail, the abnormal event refers to a point with a value of 1 in the curve, the abnormal point identification and statistics are respectively performed on the first curve (flow data) and the second curve (behavior data), the number of abnormal points in each curve is respectively counted, the total number of abnormal points is calculated, the flow abnormal points and the behavior abnormal points can be respectively counted, the total number of abnormal points is obtained through combination, the abnormal point number is compared with an abnormal point threshold, and the abnormal point threshold is a standard for determining abnormality, for example, the threshold can be a certain percentage or absolute value of the total number of abnormal points.
Specifically, the anomaly level is determined that the slight anomaly is a certain standard (such as 50% of the threshold value) with the number of the anomaly points lower than the threshold value, the moderate anomaly is that the number of the anomaly points is in a range (such as 50% -100% of the threshold value), the serious anomaly is that the number of the anomaly points exceeds the threshold value (such as 100% of the threshold value), then a monitoring state of the real-time digital video stream is determined according to the anomaly level, if the monitoring state corresponding to the slight anomaly is that the monitoring is normal, the anomaly may be an occasional event, the monitoring state corresponding to the moderate anomaly is that a certain anomaly condition exists, attention may be needed, the monitoring state corresponding to the serious anomaly is that the anomaly condition is serious, the safety risk or the monitoring system fault is represented, if the number of the abnormal points of the flow data is assumed to be 15, the number of the abnormal points of the behavior data is 10, the total abnormal point number of the abnormal points is 25, the preset abnormal point threshold value is 20, the number of the abnormal points exceeds the threshold value, and the monitoring state of the real-time digital video stream is that the serious anomaly is determined.
The embodiment of the invention can enable the digital monitoring to respond immediately by acquiring the video content in real time and performing digital processing, can monitor and accurately analyze events and behaviors occurring in a monitoring area by identifying the picture and sound content of the digital video content, can quickly identify and respond to potential security threats and abnormal conditions by detecting video behavior data and abnormal flow events, thereby improving the safety and early warning capability of a monitoring system, can intuitively know the abnormal conditions and trends of the monitoring area by generating an abnormal data flow time sequence and a two-dimensional monitoring fusion graph, can evaluate the monitoring state in real time by analyzing the two-dimensional monitoring fusion graph, and can improve the management efficiency and response speed of the monitoring area, and thus, the monitoring management is more efficient and reliable. Therefore, the digital video monitoring method provided by the invention can solve the problem of lower accuracy in the process of digital video monitoring.
Fig. 4 is a functional block diagram of a digital video monitoring system according to an embodiment of the present invention.
The digital video monitoring system 100 of the present invention may be installed in an electronic device. Depending on the functions implemented, the digital video surveillance system 100 may include a real-time digital video stream identification module 101, a video behavior data detection module 102, an abnormal behavior event identification module 103, an abnormal data stream timing generation module 104, and a surveillance status analysis module 105. The module of the invention, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the electronic device, capable of being executed by the processor of the electronic device and of performing a fixed function.
In the present embodiment, the functions concerning the respective modules/units are as follows:
the real-time digital video stream identification module 101 is configured to obtain video content in real time, digitize the video content to obtain digital video content, and identify a real-time digital video stream corresponding to the digital video content;
The video behavior data detection module 102 is configured to identify a picture content and a sound content corresponding to the real-time digital video stream, detect a video flow of the real-time digital video stream, and detect video behavior data of the real-time digital video stream according to the picture content and the sound content;
The abnormal behavior event identification module 103 is configured to identify an abnormal traffic event corresponding to the video traffic according to a preset abnormal traffic trigger event, and identify an abnormal behavior event corresponding to the video behavior data according to a preset abnormal behavior trigger event;
The abnormal data flow time sequence generating module 104 is configured to generate an abnormal data flow time sequence corresponding to the real-time digital video flow according to an abnormal flow time point corresponding to the abnormal flow event and an abnormal behavior time point corresponding to the abnormal behavior event;
The monitoring state analysis module 105 is configured to generate a two-dimensional monitoring fusion graph of the real-time digital video stream according to the abnormal data stream timing sequence, and analyze a monitoring state of the real-time digital video stream according to the two-dimensional monitoring fusion graph.
In detail, each module in the digital video monitoring system 100 in the embodiment of the present invention adopts the same technical means as the digital video monitoring method described in fig. 1 to 3, and can produce the same technical effects, which are not described herein.
In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods may be implemented in other ways. For example, the system embodiments described above are merely illustrative, e.g., the division of the modules is merely a logical function division, and other manners of division may be implemented in practice.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.
The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the foregoing description, and all changes which come within the meaning and range of equivalency of the scope of the invention are therefore intended to be embraced therein.
The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Wherein artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) is the theory, method, technique, and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend, and expand human intelligence, sense the environment, acquire knowledge, and use knowledge to obtain optimal results.
Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. Multiple units or systems as set forth in the system claims may also be implemented by means of one unit or system in software or hardware. The terms first, second, etc. are used to denote a name, but not any particular order.
Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.

Claims (8)

1. A digital video monitoring method, the method comprising:
Acquiring video content in real time, performing digital processing on the video content to obtain digital video content, and identifying a real-time digital video stream corresponding to the digital video content;
Identifying picture content and sound content corresponding to the real-time digital video stream, detecting video flow of the real-time digital video stream, and detecting video behavior data of the real-time digital video stream according to the picture content and the sound content;
The method comprises the steps of generating a flow time sequence corresponding to the video flow according to a preset timestamp, and calculating a flow factor corresponding to the video flow according to each video flow in the flow time sequence and a flow average value in the flow time sequence, wherein the flow factor calculation formula is as follows:
Wherein G r is a flow factor corresponding to the r real-time digital video stream, F r is a video flow corresponding to the r real-time digital video stream, F is the flow mean value, Q is a constant, and δ is a flow standard deviation in the flow time sequence;
when the flow factor is a preset first factor value, extracting video flow corresponding to the flow factor, and identifying a video event corresponding to the video flow; the video event is matched with an event in a preset abnormal flow triggering event to obtain an abnormal flow event;
Marking the abnormal flow time points corresponding to the abnormal flow events on a preset flow time axis, marking the abnormal behavior time points corresponding to the abnormal behavior events on the preset behavior time axis, and sequencing the event points on the flow time axis and the time points on the behavior time axis to obtain an abnormal data flow time sequence corresponding to the real-time digital video flow;
And determining a numerical value corresponding to a time point in the abnormal data stream time sequence as an abnormal value, taking the time point as an abscissa, taking the abnormal value as an ordinate, generating a two-dimensional monitoring fusion graph, and analyzing the monitoring state of the real-time digital video stream according to the two-dimensional monitoring fusion graph.
2. The digital video monitoring method according to claim 1, wherein the digitizing the video content to obtain digital video content comprises:
Converting an original video signal corresponding to the video content into a digital signal;
performing signal enhancement processing on the digital signal to obtain a digital enhanced signal;
and compressing the digital enhanced signal to obtain digital video content.
3. The digital video monitoring method according to claim 1, wherein the identifying the real-time digital video stream to which the digital video content corresponds comprises:
carrying out video frame decomposition on the digital video content according to a preset frame rate to obtain a digital frame image;
generating a frame sequence corresponding to the digital frame image according to a preset time stamp;
the frame sequence is determined as the real-time digital video stream.
4. The digital video monitoring method according to claim 1, wherein the identifying the picture content and the sound content corresponding to the real-time digital video stream comprises:
Identifying image data corresponding to each real-time digital video stream, and extracting picture events corresponding to the image data;
generating picture content corresponding to each real-time digital video stream according to the picture event;
Extracting audio characteristics corresponding to each real-time digital video stream, and determining sound events and sound emotions corresponding to each real-time digital video stream according to the audio characteristics;
And generating sound content corresponding to each real-time digital video stream according to the sound event and the sound emotion.
5. The digital video monitoring method according to claim 1, wherein the detecting video behavior data of the real-time digital video stream from the picture content and the sound content comprises:
Extracting a stream sequence number corresponding to the real-time digital video stream;
generating a first event index of a picture event corresponding to the picture content according to the stream sequence number;
generating a second event index of the sound event corresponding to the sound content according to the stream sequence number;
calculating the association degree between the picture event and the sound event according to the index identifier corresponding to the first event index and the second event index:
Wherein S is the association degree, w k is the association weight corresponding to the kth index pair, v ik is the picture event vector corresponding to the ith index identifier in the kth index pair, a jk is the sound event vector corresponding to the jth index identifier in the kth index pair, and n is the number of index pairs;
and fusing the picture event and the sound event into video behavior data corresponding to the real-time digital video stream according to the association degree.
6. The method for digital video surveillance of claim 1, wherein the identifying the abnormal behavior event corresponding to the video behavior data according to the preset abnormal behavior trigger event comprises:
Extracting behavior characteristics of each real-time digital video stream in the video behavior data, and generating a behavior time sequence corresponding to the video behavior data according to a preset event stamp and the behavior characteristics;
Calculating a behavior factor corresponding to the video behavior data according to each behavior feature in the behavior time sequence and a behavior feature standard deviation in the behavior time sequence, wherein the behavior factor calculation formula is as follows:
Wherein H r is a behavior factor corresponding to the r-th real-time digital video stream, sigma rα is a feature value corresponding to the alpha-th feature in behavior features in the r-th real-time digital video stream, m is a feature quantity in the behavior features, P is a constant, Standard deviation of the behavior characteristic;
When the behavior factor is a preset first factor value, extracting video behavior data corresponding to the behavior factor, and identifying a behavior event corresponding to the video behavior data;
and matching the behavior event with an event in a preset abnormal behavior trigger event to obtain an abnormal behavior event.
7. The digital video monitoring method according to claim 1, wherein analyzing the monitoring state of the real-time digital video stream according to the two-dimensional monitoring fusion graph comprises:
counting the number of abnormal points corresponding to the real-time digital video stream according to the curve trend of the first curve and the second curve in the two-dimensional monitoring fusion graph;
Determining an abnormal grade corresponding to the real-time digital video stream according to the abnormal point quantity and a preset abnormal point threshold value;
And determining the monitoring state of the real-time digital video stream through the abnormal grade.
8. A digital video monitoring system for performing the digital video monitoring method of any of claims 1-7, the system comprising:
The real-time digital video stream identification module is used for acquiring video content in real time, carrying out digital processing on the video content to obtain digital video content, and identifying a real-time digital video stream corresponding to the digital video content;
The video behavior data detection module is used for identifying picture content and sound content corresponding to the real-time digital video stream, detecting video flow of the real-time digital video stream and detecting video behavior data of the real-time digital video stream according to the picture content and the sound content;
The abnormal behavior event identification module is used for identifying an abnormal flow event corresponding to the video flow according to a preset abnormal flow triggering event, and comprises the steps of generating a flow time sequence corresponding to the video flow according to a preset time stamp, and calculating a flow factor corresponding to the video flow according to each video flow in the flow time sequence and a flow average value in the flow time sequence, wherein the flow factor calculation formula is as follows:
Wherein G r is a flow factor corresponding to the r real-time digital video stream, F r is a video flow corresponding to the r real-time digital video stream, F is the flow mean value, Q is a constant, and δ is a flow standard deviation in the flow time sequence;
when the flow factor is a preset first factor value, extracting video flow corresponding to the flow factor, and identifying a video event corresponding to the video flow; the video event is matched with an event in a preset abnormal flow triggering event to obtain an abnormal flow event;
The abnormal data flow time sequence generation module is used for marking the abnormal flow time points corresponding to the abnormal flow events on a preset flow time shaft, marking the abnormal behavior time points corresponding to the abnormal behavior events on the preset behavior time shaft, and sequencing the event points on the flow time shaft and the time points on the behavior time shaft to obtain the abnormal data flow time sequence corresponding to the real-time digital video flow;
And the monitoring state analysis module is used for determining a numerical value corresponding to a time point in the abnormal data stream time sequence as an abnormal value, taking the time point as an abscissa, taking the abnormal value as an ordinate, generating a two-dimensional monitoring fusion graph, and analyzing the monitoring state of the real-time digital video stream according to the two-dimensional monitoring fusion graph.
CN202411254155.8A 2024-09-09 2024-09-09 Digital video monitoring method Active CN119135949B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202411254155.8A CN119135949B (en) 2024-09-09 2024-09-09 Digital video monitoring method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202411254155.8A CN119135949B (en) 2024-09-09 2024-09-09 Digital video monitoring method

Publications (2)

Publication Number Publication Date
CN119135949A CN119135949A (en) 2024-12-13
CN119135949B true CN119135949B (en) 2025-05-06

Family

ID=93760002

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202411254155.8A Active CN119135949B (en) 2024-09-09 2024-09-09 Digital video monitoring method

Country Status (1)

Country Link
CN (1) CN119135949B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110895861A (en) * 2018-09-13 2020-03-20 杭州海康威视数字技术股份有限公司 Abnormal behavior early warning method and device, monitoring equipment and storage medium
CN111641629A (en) * 2020-05-28 2020-09-08 腾讯科技（深圳）有限公司 Abnormal behavior detection method, device, equipment and storage medium
CN113630389A (en) * 2021-07-22 2021-11-09 北京明略软件系统有限公司 User abnormal behavior identification method, system, electronic equipment and storage medium

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8189905B2 (en) * 2007-07-11 2012-05-29 Behavioral Recognition Systems, Inc. Cognitive model for a machine-learning engine in a video analysis system
KR20160093253A (en) * 2015-01-29 2016-08-08 쿠도커뮤니케이션 주식회사 Video based abnormal flow detection method and system
AU2020272887B2 (en) * 2019-04-09 2023-03-02 Motorola Solutions, Inc. Anomaly detection method, system and computer readable medium
CN110245062B (en) * 2019-06-19 2021-09-10 山东大学 Abnormal monitoring behavior analysis method based on log file
CN113673311B (en) * 2021-07-05 2025-05-30 浙江大华技术股份有限公司 A method, device and computer storage medium for detecting abnormal traffic events
CN116488939B (en) * 2023-06-16 2023-08-25 江西科技学院 Computer information security monitoring method, system and storage medium

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110895861A (en) * 2018-09-13 2020-03-20 杭州海康威视数字技术股份有限公司 Abnormal behavior early warning method and device, monitoring equipment and storage medium
CN111641629A (en) * 2020-05-28 2020-09-08 腾讯科技（深圳）有限公司 Abnormal behavior detection method, device, equipment and storage medium
CN113630389A (en) * 2021-07-22 2021-11-09 北京明略软件系统有限公司 User abnormal behavior identification method, system, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN119135949A (en) 2024-12-13

Similar Documents

Publication Publication Date Title
CN113850229A (en) Method, system and computer equipment for early warning of abnormal human behavior based on video data machine learning
KR102149832B1 (en) Automated Violence Detecting System based on Deep Learning
CN118552882B (en) Edge calculation optimization method and system of intelligent video monitoring system
US20230186634A1 (en) Vision-based monitoring of site safety compliance based on worker re-identification and personal protective equipment classification
CN106355154B (en) Method for detecting frequent passing of people in surveillance video
CN109033476B (en) Intelligent spatio-temporal data event analysis method based on event cue network
Wang et al. Video anomaly detection method based on future frame prediction and attention mechanism
CN113239766A (en) Behavior recognition method based on deep neural network and intelligent alarm device
WO2023128186A1 (en) Multi-modal video captioning-based image security system and method
CN119229337A (en) A multimodal intelligent analysis method and system based on law enforcement supervision
CN109086717B (en) Violent behavior detection system and method based on human skeleton and motion signal characteristics
CN119672933B (en) Real-time warning method and system for abnormal behavior based on intelligent security
CN114222163A (en) Intelligent video analysis system for business hall
CN117079351A (en) Method and system for analyzing personnel behaviors in key areas
CN120145312A (en) An intelligent security management system based on AI big model
CN116842217A (en) Monitoring video intelligent service platform based on video content analysis and retrieval
CN120111283A (en) A dynamic intelligent monitoring and early warning system for live broadcast smart phones
CN119339318B (en) Campus abnormal behavior detection system based on artificial intelligence
CN119135949B (en) Digital video monitoring method
CN113837066A (en) Behavior recognition method and device, electronic equipment and computer storage medium
CN119068549A (en) A method for detecting abnormal campus behavior based on complexity theory
CN118694899A (en) Property safety prevention and control method and system based on artificial intelligence
CN117765609A (en) System for detecting factory man-object interaction violation
TWI784718B (en) Method and system for processing alarm event in factory
JP2007114885A (en) Classification method and apparatus based on image similarity

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant