CN115527166A - Image processing method, computer-readable storage medium, and electronic device - Google Patents

Abstract

The application discloses an image processing method, a computer-readable storage medium and electronic equipment. Wherein, the method includes: acquiring a monitoring image of the area to be monitored, wherein the monitoring image contains a target object; performing density estimation on the monitoring image to obtain a density estimation result of the target object, wherein the density estimation result contains a plurality of density values for Characterize the probability of the target object being present in multiple pixels in the monitoring image; determine the target pixel point from multiple pixel points based on the density estimation result, wherein the target pixel point has the target object; based on the position of the target pixel point in the monitoring image, A target positioning result of the target object in the area to be monitored is obtained. The present application solves the technical problem that it is difficult for an algorithm in the related art to output more information about a target object in a monitoring image.

Description

Image processing method, computer-readable storage medium, and electronic device

technical field

The present application relates to the field of data processing, and in particular, to an image processing method, a computer-readable storage medium, and electronic equipment.

Background technique

Traditional group situational awareness often only needs to estimate the number of overall objects in the video screen in real time, and then grasp the possible change trend of the number of overall objects. Accurate perception needs. For some monitoring images containing a large number of objects, the current algorithm can only estimate the total number of objects in the image, the output information is limited, and the granularity is not fine enough.

For the above problems, no effective solution has been proposed yet.

Contents of the invention

Embodiments of the present application provide an image processing method, a computer-readable storage medium, and an electronic device to at least solve the technical problem that it is difficult for an algorithm in the related art to output more information about a target object in a monitoring image.

According to an aspect of an embodiment of the present application, an image processing method is provided, including: acquiring a monitoring image of an area to be monitored, wherein the monitoring image contains a target object; performing density estimation on the monitoring image to obtain a density estimation result of the target object, Among them, the multiple density values contained in the density estimation result are used to represent the probability of the target object being present in multiple pixel points in the monitoring image; based on the density estimation result, the target pixel point is determined from multiple pixel points, wherein the target pixel point has the target object Object: Based on the position of the target pixel in the monitoring image, the target positioning result of the target object in the area to be monitored is obtained.

According to an aspect of an embodiment of the present application, an image processing method is provided, including: monitoring an active area through a monitoring device to obtain a monitoring image, wherein the monitoring image contains a target population; performing density estimation on the monitoring image to obtain a density estimation of the target population As a result, the multiple density values contained in the density estimation result are used to characterize the probability of the presence of the target crowd in multiple pixels in the monitoring image; based on the density estimation result, the target pixel is determined from the multiple pixel points, wherein the target pixel There is a target group; based on the position of the target pixel in the monitoring image, the target positioning result of the target group in the active area is obtained.

According to an aspect of an embodiment of the present application, an image processing method is provided, including: responding to an input instruction acting on the operation interface, displaying a monitoring image of the area to be monitored on the operation interface, wherein the monitoring image contains a target object; responding The positioning instruction acts on the operation interface, and the target positioning result of the target object in the area to be monitored is displayed on the operation interface, wherein the target positioning result is displayed in the monitoring image through the target pixel points determined from the multiple pixel points in the monitoring image The position of the target is determined, and the target pixel is determined based on the density estimation result of the target object. The density estimation result is obtained by performing density estimation on the monitoring image. The multiple density values contained in the density estimation result are used to represent the probability of the target object existing at multiple pixel points.

According to an aspect of an embodiment of the present application, an image processing method is provided, including: monitoring a monitoring image of an area to be monitored by a monitoring device, wherein the monitoring image contains a target object; presenting the image on a virtual reality VR device or an augmented reality AR device The monitoring image is displayed on the screen; the density estimation of the monitoring image is performed to obtain the density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used to represent the probability of the target object existing in multiple pixels in the monitoring image; based on the density Estimating the result, determining the target pixel point from multiple pixels, wherein the target pixel point has the target object; based on the position of the target pixel point in the monitoring image, the target positioning result of the target object in the area to be monitored is obtained; drive the VR device Or AR device rendering to display the target positioning results.

According to an aspect of an embodiment of the present application, an image processing method is provided, including: obtaining a monitoring image of an area to be monitored by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is monitoring An image, the monitoring image contains a target object; density estimation is performed on the monitoring image to obtain a density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used to represent the probability that the target object exists at multiple pixels in the monitoring image; Based on the density estimation result, a target pixel point is determined from a plurality of pixel points, wherein the target pixel point has a target object; based on the position of the target pixel point in the monitoring image, the target positioning result of the target object in the area to be monitored is obtained; by calling the second interface to output the target positioning result, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target positioning result.

In the embodiment of the present application, firstly, the monitoring image of the area to be monitored, wherein the monitoring image contains the target object; density estimation is performed on the monitoring image to obtain the density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used It is used to characterize the probability of the target object in multiple pixels in the monitoring image; based on the density estimation result, determine the target pixel from multiple pixels, wherein the target pixel has the target object; based on the position of the target pixel in the monitoring image , the target location result of the target object in the area to be monitored is obtained, and the location of the object in the monitoring image is realized. It is easy to notice that the density estimation of the monitoring image can be performed to obtain the density estimation result of the target object, and the pixel points with the target object can be determined from multiple pixel points based on the density estimation result, avoiding the detection of pixels that do not contain the target object. In order to improve the accuracy of the positioning result of the target object, by obtaining the positioning result of the target object, the effect of increasing the output information of the target object can be achieved, thereby solving the problem that the algorithm of the related technology is difficult to output more accurate information of the target object in the monitoring image. Multi-information technical issues.

Description of drawings

The drawings described here are used to provide a further understanding of the application and constitute a part of the application. The schematic embodiments and descriptions of the application are used to explain the application and do not constitute an improper limitation to the application. In the attached picture:

FIG. 1 is a schematic diagram of a hardware environment of a virtual reality device according to an image processing method according to an embodiment of the present application;

Fig. 2 is a structural block diagram of a computing environment of an image processing method according to an embodiment of the present application;

FIG. 3 is a flowchart of an image processing method according to Embodiment 1 of the present application;

FIG. 4 is a flowchart of an image processing process according to an embodiment of the present application;

FIG. 5 is a flowchart of another image processing method according to an embodiment of the present application;

FIG. 6 is a flowchart of an image processing method according to Embodiment 2 of the present application;

FIG. 7 is a flowchart of an image processing method according to Embodiment 3 of the present application;

FIG. 8 is a flowchart of an image processing method according to Embodiment 4 of the present application;

FIG. 9 is a flowchart of an image processing method according to Embodiment 5 of the present application;

FIG. 10 is a schematic diagram of an image processing device according to Embodiment 6 of the present application;

FIG. 11 is a schematic diagram of an image processing device according to Embodiment 7 of the present application;

FIG. 12 is a schematic diagram of an image processing device according to Embodiment 8 of the present application;

FIG. 13 is a schematic diagram of an image processing device according to Embodiment 9 of the present application;

FIG. 14 is a schematic diagram of an image processing device according to Embodiment 10 of the present application;

Fig. 15 is a structural block diagram of a computer terminal according to an embodiment of the present application.

detailed description

In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiment of the application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the application. Obviously, the described embodiment is only It is an embodiment of a part of the application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application.

It should be noted that the terms "first" and "second" in the description and claims of the present application and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

First of all, some nouns or terms that appear during the description of the embodiments of the present application are applicable to the following explanations:

Crowd situation: Crowd situation analysis refers to the use of automated algorithms to obtain quantitative indicators such as the number, density, and flow direction of the crowd, and based on the analysis of the above indicators, the crowd control and management model in the monitoring area is derived.

Group situation: including crowd situation, animal group situation, robot group situation, etc. The content of this application is not limited to the above-mentioned group situations, and may include group situation calculations formed by any similar objects.

Dense Counting: The purpose is to count the number of people in the scene. Crowd counting is widely used in video monitoring, traffic monitoring, public safety, urban planning, etc., such as monitoring the number of people in an area where people tend to gather, and preventing stampedes and other events.

Density map: According to the known position of each target, the size of the target at the position can be estimated, and the coverage area of the target can be obtained, and then the area can be converted into a possible target in the area by some methods (geometric adaptive Gaussian kernel) Probability, the area probability sum is 1 (or indicates how many people may be in each pixel), thus the target distribution density in the picture can be obtained.

Hill Climbing: Can be a local search algorithm that moves continuously in directions of increasing height/value to find local peaks.

N-ary tree: In a tree-like data structure, if each parent node allows more than two child nodes, then the tree is called an N-ary tree.

Depth First Search (DFS for short): It can be an algorithm for traversing a search tree or graph, which searches the branches of the tree as deep as possible. When all edges of node v have been explored, the search will backtrack to the start node of the edge where node v was found. This process continues until all nodes reachable from the source node have been found.

At present, since the total estimated number of people in a dense scene may reach thousands of objects, if the algorithm only outputs the total number of objects, it is difficult for the monitoring personnel of the crowd situation to perceive the advantages and disadvantages of the algorithm recognition effect at the first time. Therefore, an algorithm is needed It can provide the specific pixel position of each object in the picture. At the same time, when combined with the 3D modeling of the event venue, it is difficult for the density distribution map of the crowd to fit the 3D model finely, and the algorithm's perception ability is mapped to the 3D model.

In related technologies, the above problems are solved in the following ways:

(1) The crowd counting and localization network (SCALNet) realizes the transformation from the density map to the scattered coordinates. This scheme only applies a 3×3 maximum pooling filter to scan the density map, and uses a selection based on the verification set. However, the disadvantage of this scheme is that it is difficult to ensure that the number of target coordinates obtained matches the estimated number of people in the density map, resulting in a corresponding error in the number of people. This method is more sensitive to the threshold.

(2) Crowd Density Estimation (FIDTM) realizes the transformation from density map to scattered coordinates. This scheme is roughly the same as SCALNet. The difference is that the threshold is fixed at the maximum value in the density map and scaled to 100/255, avoiding the threshold value. However, it is also difficult for this scheme to ensure that the number of target coordinates obtained coincides with the estimated number of people in the density map.

The disadvantage of the above two schemes is that the assumption that there is at most one peak point in the 3×3 area is used, which may cause the omission of the target coordinates in an extremely dense situation.

This application provides an image processing method that can combine the density clustering algorithm and the depth-first search algorithm to produce more accurate target coordinate positions from the density map and improve the accuracy of the output target coordinate positions.

Example 1

According to an embodiment of the present application, an image processing method is also provided. It should be noted that the steps shown in the flow charts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although in the flow chart The figures show a logical order, but in some cases the steps shown or described may be performed in an order different from that shown or described herein.

Fig. 1 is a schematic diagram of a hardware environment of a virtual reality device according to an image processing method according to an embodiment of the present application. As shown in Figure 1, the virtual reality device 104 is connected to the terminal 106, and the terminal 106 is connected to the server 102 through the network. The virtual reality device 104 is not limited to: virtual reality helmets, virtual reality glasses, virtual reality all-in-one machines, etc. The above-mentioned terminal 104 is not limited to PC, mobile phone, tablet computer, etc., and the server 102 may be a server corresponding to a media file operator, and the above-mentioned network includes but not limited to: a wide area network, a metropolitan area network or a local area network.

Optionally, the virtual reality device 104 in this embodiment includes: a memory, a processor, and a transmission device. The memory is used to store an application program, and the application program can be used to perform: acquiring a monitoring image of the area to be monitored, wherein the monitoring image contains a target object; performing density estimation on the monitoring image to obtain a density estimation of the target object As a result, the multiple density values included in the density estimation result are used to characterize the probability that the target object exists at multiple pixel points in the monitoring image; based on the density estimation result, from the multiple pixel points determining a target pixel, wherein the target pixel exists the target object; based on the position of the target pixel in the monitoring image, obtaining a target positioning result of the target object in the area to be monitored, Therefore, the technical problem that it is difficult for the algorithm in the related art to output more information about the target object in the monitoring image is solved.

Optionally, the eye-tracking HMD (Head Mount Display, head-mounted display) head-mounted display and the eye-tracking module of the virtual reality device 104 of this embodiment have the same functions as those in the above-mentioned embodiment, that is, the HMD head-mounted display The screen is used to display real-time images, and the eye tracking module in the HMD is used to obtain the real-time movement trajectory of the user's eyeballs. The terminal in this embodiment obtains the position information and motion information of the user in the real three-dimensional space through the tracking system, and calculates the three-dimensional coordinates of the user's head in the virtual three-dimensional space, and the direction of the user's field of view in the virtual three-dimensional space.

The block diagram of the hardware structure shown in FIG. 1 can not only be used as an exemplary block diagram of the above-mentioned AR/VR device (or mobile device), but also can be used as an exemplary block diagram of the above-mentioned server. In an optional embodiment, FIG. 2 uses a block diagram An embodiment of using the above-mentioned AR/VR device (or mobile device) shown in FIG. 1 as a computing node in the computing environment 201 is shown. Fig. 2 is a structural block diagram of a computing environment of an image processing method according to an embodiment of the present application. As shown in Fig. 2, the computing environment 201 includes multiple (210-1, 210- 2, ..., to show) computing nodes (such as servers). Each computing node contains local processing and memory resources, and end users 202 can remotely run applications or store data in computing environment 201 . Applications may be provided as a plurality of services 220-1, 220-2, 220-3, and 220-4 in computing environment 301, representing services "A," "D," "E," and "H," respectively.

End user 202 may offer and access services through a web browser or other software application on a client, and in some embodiments, end user 202 offers and/or requests may be provided to ingress gateway 230 . Ingress gateway 230 may include a corresponding agent to handle offers and/or requests for services 220 (one or more services provided in computing environment 201).

Services 220 are provided or deployed according to various virtualization technologies supported by computing environment 201 . In some embodiments, service 220 may be provided according to virtual machine (VM)-based virtualization, container-based virtualization, and/or the like. Virtual machine-based virtualization can emulate a real computer by initializing a virtual machine to execute programs and applications without directly touching any actual hardware resources. While virtual machines virtualize machines, according to container-based virtualization, containers can be launched to virtualize an entire operating system (OS) so that multiple workloads can run on a single OS instance.

In an embodiment based on container virtualization, several containers of the service 220 may be assembled into a POD (eg, Kubernetes POD). For example, as shown in FIG. 2, the service 220-2 may be equipped with one or more PODs 240-1, 240-2, . . . , 240-N (collectively referred to as POD 240). Each POD 240 may include an agent 245 and one or more containers 242-1, 242-2, . . . , 242-M (collectively containers 242). One or more containers 242 in the POD 240 process requests related to one or more corresponding functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, and the like. Other services 220 may also accompany PODs similar to POD 240 .

During operation, executing a user request from an end user 202 may require invoking one or more services 220 in the computing environment 201. To execute one or more functions of one service 220, you need to invoke one or more functions of another service 220. function. As shown in FIG. 2, service "A" 220-1 receives a user request from end user 202 from ingress gateway 230, service "A" 220-1 may call service "D" 220-2, and service "D" 220-2 may Service "E" 220-3 is requested to perform one or more functions.

The above-mentioned computing environment may be a cloud computing environment, and the allocation of resources is managed by the cloud service provider, allowing the development of functions without considering the implementation, adjustment or expansion of servers. This computing environment allows developers to execute code that responds to events without building or maintaining complex infrastructure. Services can be partitioned to perform a set of functions that can be automatically and independently scaled, rather than scaling a single hardware appliance to handle the underlying load.

Under the above operating environment, the present application provides an image processing method as shown in FIG. 3 . It should be noted that the image processing method in this embodiment may be executed by the mobile terminal in the embodiment shown in FIG. 1 . FIG. 3 is a flowchart of an image processing method according to Embodiment 1 of the present application. As shown in Figure 3, the method may include the following steps:

Step S302, acquiring a monitoring image of the area to be monitored.

Wherein, the monitoring image contains the target object.

The number of target objects in the above monitoring image can be a preset number. When the preset number is greater than the preset value, it means that the monitoring image is an ultra-dense image, and the preset number can be obtained by estimating the number of target objects in the monitoring image. , where the preset value can be set according to the actual situation.

The above-mentioned to-be-monitored area may be a large-scale event scene, a large-scale gathering place, and other areas with high crowd density. The area to be monitored may also be an area with a high density of objects, for example, a forest, a habitat of animals living in groups, and the like. The area to be monitored may also be any area that needs to be monitored, which is not limited here.

The aforementioned monitoring images may be images collected when the number of target objects in the area to be monitored is greater than a preset number. Optionally, the monitoring image can be obtained through the shooting device; the video information of the area to be monitored can also be obtained through the camera, and the above-mentioned monitoring image can be obtained by cutting the video information, wherein the number of target objects in the video information can be greater than A preset number of video frames are used as monitoring images.

The aforementioned monitoring images may also be remote sensing images. The above monitoring image may also be a thermal distribution map of the crowd.

The above-mentioned target object may be a human, an animal, a plant, an object, etc., which is not limited here, and the present application takes the target object as an example.

The above-mentioned preset quantity can be set independently. If the number of target objects in the monitoring image is greater than the preset number, it means that the number of target objects in the to-be-monitored area is large, that is, the target objects are relatively dense, and it is necessary to monitor the target objects in the to-be-monitored area.

In an optional embodiment, the preset number can be determined according to the area to be monitored. For example, the preset number can be determined according to the area of the area to be monitored. If the area of the area to be monitored is larger, it can accommodate more target objects. At this time, the preset number can be set larger. When the number of target objects is greater than When the preset number is reached, the area to be monitored will be densely populated with target objects. If the area to be monitored is small, it can accommodate smaller target objects. At this time, the preset number can be set smaller. As long as the number of target objects is greater than the preset number, it is determined that a target has appeared in the area to be monitored. dense objects.

Step S304, performing density estimation on the monitoring image to obtain a density estimation result of the target object.

Wherein, the multiple density values included in the density estimation result are used to characterize the probability that the target object exists in multiple pixel points in the monitoring image.

In an optional embodiment, a density estimation model may be used to perform density estimation on the monitoring image to obtain a density estimation result of the target object. Wherein, the density estimation model may be a general density estimation model, which is not limited here.

In the case that the surveillance image contains dense crowds, the density estimation result can be an approximate number of dense crowds.

The above-mentioned density value can be used to indicate the probability of the target object in the area. The larger the density value is, the more the number of corresponding target objects is. The smaller the number of corresponding target objects, the less likely there are target objects in this area.

Through the above density estimation results, the probabilities of target objects in multiple regions in the monitoring image can be obtained, so as to determine the overall density of the monitoring image.

其中，i，j为矩阵M的行和列；h，w为图像的长和宽。In another optional embodiment, the current crowd density estimation model can be used to perform density estimation on high-density monitoring images to obtain the density estimation result. Optionally, for a high-density surveillance image captured from a video stream The resolution color image (Red, Green, Blue, referred to as RGB image) has a shape of (H, W, 3). Through a crowd density estimation model, a low-resolution density map can be obtained, that is, a shape of The matrix M of (H/4, W/4), the elements in the matrix are floating-point numbers, representing the probability that the target object may appear at each pixel position, therefore, the target object in the monitoring image can be obtained by summing the elements in the matrix The total number of K, that is

Among them, i, j are the rows and columns of the matrix M; h, w are the length and width of the image.

Step S306, based on the density estimation result, determine the target pixel point from the plurality of pixel points.

Wherein, there is a target object in the target pixel.

In an optional embodiment, target pixel points with larger density values may be determined according to multiple density values in the density estimation result.

In another optional embodiment, multiple pixel points may be divided into multiple first pixel sets according to the density estimation result, and the first pixel set whose quality exceeds the first preset quality is split to obtain multiple a second set of pixels, determine the target pixel points from the second set of pixels, and establish multiple target tree structures, wherein the sum of the density values of all nodes contained in each target tree structure in the density estimation result is less than the first As a preset value, the target tree structure may contain multiple nodes, and each node may contain a target object.

The aforementioned target tree structure may be an N-ary tree structure.

The above-mentioned first preset value may be 1.

The target tree structure in the dense crowd can be established according to the density estimation result, wherein the multiple nodes contained in the target tree structure can correspond to multiple pixel points, and the target pixel point can be used as the parent node in the target tree structure, for Other pixel points can be determined as child nodes of the parent node according to the distance between them and the target pixel point.

In an optional embodiment, a local density hill-climbing algorithm and a depth-first quality aggregation method may be used to build multiple target tree structures according to the density estimation results. Among them, the sum of the density values of the nodes contained in the target tree structure is less than 1 in the density estimation result, indicating that the node represents a target object, and avoiding the occurrence of one node representing multiple target objects, which will cause multiple target objects to share one anchor point.

Step S308, based on the position of the target pixel in the monitoring image, the target positioning result of the target object in the area to be monitored is obtained.

In an optional embodiment, the position of the target pixel in the monitoring image can be used to determine the target positioning result of the target object in the area to be monitored.

In another optional embodiment, in the multiple target tree structures built, it can be determined that the target pixel point corresponds to the target root node, and it can be determined according to the radiation range of the target root node that the target object is in the area to be monitored Targeting results in . The above-mentioned number of target root nodes may be approximately the same as the number of target objects, or the number of target root nodes may be the same as the number of target objects. If the sum of the density values of all nodes included in each target tree structure in the density estimation result is less than the first preset value, it means that the number of target objects corresponding to this node is approximately one. Based on the target root node contained in the above-mentioned target tree structure corresponding to the pixel position in the monitoring image, the roughly corresponding target positioning result of each person in the dense crowd can be determined.

The above target positioning result may be the coordinate information of the target object in the monitoring image.

In an optional embodiment, since the target root node represents the target object in the monitoring image, it can be determined according to the pixel position of the target root node contained in the multiple target tree structures corresponding to the monitoring image. The target positioning result of the target object in the monitoring result.

The above method of this application can use the density climbing algorithm and the depth optimization algorithm to sample the specific positions of pedestrians from the fuzzy crowd thermal distribution map in dense scenes, and provide important position information for 3D modeling and trend prediction of dense crowds.

Taking the monitoring images of crowded scenes as an example to illustrate, you can first obtain the monitoring images corresponding to crowded scenes, such as the scene of large-scale concerts and large-scale events, and estimate the density of the monitoring images to obtain the density estimation results of the crowd, that is, you can Roughly estimate the number of individuals in a densely populated scene. According to the density estimation result, the target pixel point can be determined from multiple pixel points. According to the position of the target pixel point in the monitoring image, the position of the target object in the area to be monitored can be obtained. Targeting results. Optionally, the initial tree structure corresponding to the crowd density cluster can be established, and the points with lower mass in the crowd density cluster can be superimposed on the central node of the crowd density cluster to obtain multiple aggregated tree structures. For multiple clusters The target tree structure whose crowd density value exceeds 1 in the tree structure contains multiple individuals, so it is necessary to split the target tree structure to obtain multiple target initial tree structures, which can be based on multiple target trees The root node of the target contained in the shape structure corresponds to the position of the target pixel in the monitoring image, and the target positioning position of each individual in the monitoring result in the crowded scene is obtained, so as to obtain the positioning result consistent with the number of people in the crowded scene . It should be noted that the crowd density cluster may be a crowd density cluster formed centering on a location with higher density in the monitoring image, and there may be one or more crowd density clusters.

Taking the monitoring images of dense animal scenes as an example, you can first obtain the monitoring images corresponding to animal dense scenes, such as animal migration scenes, animal group living scenes, and estimate the density of the monitoring images to obtain the animal density estimation results, that is, you can roughly Estimate the number of individuals in the animal-intensive scene, and determine the target pixel point from multiple pixels according to the density estimation result, and obtain the target location of the animal in the area to be monitored according to the position of the target pixel point in the monitoring image result. Optionally, the initial tree structure corresponding to the animal density cluster can be established, and the points with lower mass in the animal density cluster can be superimposed on the central node in the animal density cluster to obtain multiple aggregated tree structures. For multiple aggregates The target tree structure whose animal density value exceeds 1 in the tree structure contains multiple animal individuals, so it is necessary to split the target tree structure to obtain multiple target initial tree structures, which can be based on multiple targets The target root node contained in the tree structure corresponds to the position of the target pixel in the monitoring image, and the target positioning position of each individual animal in the monitoring results in the animal-intensive scene is obtained, so as to obtain the number of animals in the animal-intensive scene. positioning results. It should be noted that the animal density cluster may be an animal density cluster formed centering on a position with a higher density in the monitoring image, and there may be one animal density cluster or multiple animal density clusters.

Taking the monitoring image of a scene with dense objects as an example to illustrate, you can first obtain the monitoring image corresponding to the scene with dense objects, such as a scene containing multiple objects of the same type, and perform density estimation on the monitoring image to obtain the density estimation result of the object, that is, you can Roughly estimate the number of individuals in the object-intensive scene. According to the density estimation result, the target pixel can be determined from multiple pixels. According to the position of the target pixel in the monitoring image, the target of the object in the area to be monitored can be obtained. positioning results. Optionally, the initial tree-like structure corresponding to the object density cluster can be established, and the points with lower mass in the object density cluster can be superimposed on the central node in the object density cluster to obtain multiple aggregated tree structures. For multiple aggregates The target tree structure with an object density value exceeding 1 in the tree structure contains multiple objects, so it is necessary to split the target tree structure to obtain multiple target initial tree structures, which can be based on multiple target trees The target root node contained in the shape structure corresponds to the position of the target pixel in the monitoring image, and the target positioning position of each object in the monitoring result in the object-intensive scene is obtained, so as to obtain the positioning result consistent with the number of objects in the object-intensive scene . It should be noted that the object density cluster may be an object density cluster formed centering on a position with higher density in the monitoring image, and there may be one object density cluster or multiple object density clusters.

Through the above steps, firstly, the monitoring image of the area to be monitored, wherein the monitoring image contains the target object; density estimation is performed on the monitoring image to obtain the density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used to represent the monitoring The probability of the target object existing in multiple pixels in the image; based on the density estimation result, determine the target pixel point from multiple pixel points, where the target pixel point has the target object; based on the position of the target pixel point in the monitoring image, get the target The target positioning result of the object in the area to be monitored realizes the positioning of the object in the monitoring image. It is easy to notice that the density estimation of the monitoring image can be performed to obtain the density estimation result of the target object, and the pixel points with the target object can be determined from multiple pixel points based on the density estimation result, avoiding the detection of pixels that do not contain the target object. In order to improve the accuracy of the positioning result of the target object, by obtaining the positioning result of the target object, the effect of increasing the output information of the target object can be achieved, thereby solving the problem that the algorithm of the related technology is difficult to output more accurate information of the target object in the monitoring image. Multi-information technical issues.

In the above embodiments of the present application, determining the target pixel from multiple pixel points based on the density estimation result includes: dividing the multiple pixel points into multiple first pixel sets based on the density estimation result, wherein different first pixel There are different objects in the pixels in the set; aggregate the density values corresponding to the pixels in each first pixel set to obtain the quality corresponding to each first pixel set; based on the quality corresponding to multiple first pixel sets The preset pixel set in the first pixel set is split to obtain multiple second pixel sets, wherein the quality corresponding to the preset pixel set is greater than the first preset quality; based on the quality corresponding to the multiple second pixel sets, from Target pixel points are determined from the plurality of second pixel sets.

The multiple first pixel sets mentioned above may be in the form of multiple initial tree structures, and the quality of each node contained in each initial tree structure is the density value of the node in the density estimation result; the multiple initial tree structures The quality of all nodes in the structure is aggregated to obtain multiple aggregated tree structures, that is, the quality corresponding to each first pixel set mentioned above; the target tree structure in the multiple aggregated tree structures is split to obtain multiple The first tree structure, wherein, the target tree structure is the above-mentioned preset pixel structure, the first tree structure is the above-mentioned first pixel set, and the quality of the first root node of the first tree structure is greater than the first preset quality.

The aforementioned density values may be floating point density values. The above-mentioned first preset quality may be 1, but it is not limited thereto, and may be set according to actual needs.

In dense crowds, multiple initial tree structures can be tree structures corresponding to density clusters in dense crowds, where the pixel with a larger density value of the density cluster can correspond to the parent node in the tree structure. Other pixels in the range can be descendant child nodes of the parent node.

In an optional embodiment, after obtaining the density estimation result, the elements in the density matrix M can be scanned one by one to determine whether the element is the maximum floating-point density value point in its own 3×3 field, and if so, set It is pushed into the stack S of the candidate point, if not, it is pushed into the child node stack C_S of the peak point C in its neighborhood, where the pixel coordinates of each element (i, j ) and its floating-point density value V in M, and store the radiation range of each element as its own coordinates, where the radiation range refers to the coordinate range of each element as the descendant node that is the successor of the parent node.

Furthermore, since the quality of the candidate point is only its own quality, the quality of all descendant nodes (including child nodes, child nodes of child nodes, etc.) of the candidate point can be added to itself through the depth-first search algorithm, and updated as At the same time, the radiation range value can be updated to the upper left and lower right positions farthest from the coordinates of the candidate node among all descendant nodes, that is, the radiation range of the candidate point can be updated to the maximum range of all descendant nodes.

In another optional embodiment, multiple pixel points may be divided into multiple first pixel sets according to the density estimation result, and there are different objects in the pixel points in different first pixel sets. Optionally, The density estimation result is centered on the pixel points corresponding to one or more areas with larger density values, and the multiple pixel points are divided into multiple first pixel sets.

Further, the density values corresponding to the pixels in each first pixel set may be aggregated, optionally, all the pixels in each first pixel set may be scanned, and the pixel with the largest density value among them may be As the target pixel, the density value of other pixels within the radiation range of the pixel can be superimposed on the pixel with the highest quality, and the quality of the pixel with the highest density value can be updated, among which, the density value of the pixel can be superimposed The process is essentially the process of aggregating the density values of the pixels, so as to aggregate the density values corresponding to the pixels in each first pixel set, and obtain the quality corresponding to each first pixel set.

Still further, since the quality corresponding to the preset pixel set is greater than the first preset quality, it means that the number of target objects corresponding to the preset pixel set exceeds 1. At this time, it is necessary to perform a classification operation on the preset pixel set to obtain multiple first A collection of two pixels.

In the tree structure, multiple initial tree structures can be established based on the density estimation results. The quality of each node contained in each initial tree structure is the density value of the node in the density estimation result, and each All nodes in an initial tree structure are scanned, and the node with the highest quality is obtained as a candidate node. The quality of the child nodes within the radiation range of the candidate node can be superimposed on the node with the highest quality, and the quality of the node with the highest quality can be updated. , wherein the process of superimposing the node quality is essentially the process of aggregating the node quality, so as to aggregate the qualities of all nodes in multiple initial tree structures to obtain multiple aggregated tree structures.

In another optional embodiment, for the target tree structure whose parent node quality exceeds 1, the split operation can be performed recursively, so that the number of coordinates corresponding to each parent node is exactly equal to the rounding of the total quality, so that Avoid the problem of losing the number of target objects in extremely dense areas.

In the above embodiments of the present application, based on the density estimation result, dividing a plurality of pixel points into a plurality of first pixel sets includes: traversing each pixel point in the monitoring image, and determining whether the density value corresponding to the pixel point in the density estimation result is is the maximum density value among the density values corresponding to all pixels in the preset area, where the preset area is used to represent the area in the monitoring image centered on the pixel and adjacent to the pixel; When the density value corresponding to the pixel point is the maximum density value, a new set is established, and the pixel point is stored in the new set, wherein the new set is used to generate a plurality of first pixel sets; the density corresponding to the pixel point If the value is not the maximum density value, the pixel point is stored in the set in which the maximum density value is stored among the plurality of first pixel sets.

In an optional embodiment, each pixel in the monitoring image can be traversed to determine whether the density value corresponding to the pixel in the density estimation result is the maximum density among the density values corresponding to all pixels in the preset area value, when the density value is greater than the maximum density value in the preset area, a new set can be established, and the pixel point can be stored in the new set, which is shown in the dense crowd in the monitoring image. Density clusters.

The aforementioned preset area may be a 3×3 area, or an area of other sizes, and the area of the preset area is not limited here. The preset area of the pixel does not contain the density value of the pixel.

In an optional embodiment, the density value of each pixel in the monitoring image can be traversed to determine whether the density value is the maximum density value in the preset area, if the density value is in the preset area The maximum density value of , you can build a new set based on the density value.

In the case that the density value is not the maximum density value in the preset area, the pixel point can be stored in a set with the maximum density value stored in a plurality of first pixel sets, the

In another optional embodiment, the maximum density value in the preset area during the traversal process can be used as the initial root node, and other density values smaller than the maximum density value in the preset area can be used as child nodes of the initial root node, Thereby establishing multiple initial tree structures. Optionally, the maximum density value can be pushed into the stack S of the candidate point, and other density values can be pushed into the child node stack C_S of the peak point C in its neighborhood. During the stacking process, each The pixel value and pixel coordinates corresponding to each pixel.

In the above-mentioned embodiments of the present application, each first set of pixels adopts a tree structure, and the pixels in each first set of pixels correspond to nodes in the tree structure, and the relationship between the pixels in each first set of pixels The neighbor relationship corresponds to the connection relationship between nodes in the tree structure, wherein the density values corresponding to the pixel points in each first pixel set are aggregated to obtain the quality corresponding to each first pixel set, including: determining the tree structure The density value corresponding to the parent node and the density value corresponding to the child node in the structure, where the child node is the descendant node of the parent node in the tree structure; aggregate the density value corresponding to the parent node and the density value corresponding to the child node to obtain the parent node quality; determine the quality of the root node in the tree structure as the quality corresponding to each first pixel set.

The above-mentioned parent node may be the node with the highest quality in the tree structure, and the above-mentioned child node may be the node with the quality less than the maximum quality in the initial tree structure.

In an optional embodiment, each tree structure contains a parent node and a child node, and the density value of the child node can be superimposed on the density value of the parent node to obtain the quality of the parent node, thus achieving The purpose of aggregating the density value of the parent node and the density value of the child node is to improve the quality of the parent node by aggregating the density value corresponding to the parent node and the density value corresponding to the child node, thereby increasing the probability of the target object in the area, which can be The quality of the parent node in the tree structure is determined to be the quality corresponding to each first pixel set, thereby increasing the possibility that there is a pixel corresponding to the target object in the first pixel set.

In another optional embodiment, the quality of the parent node is only its own density value, and the density value of all descendant child nodes of the parent node can be added to the parent node through the depth-first search algorithm to obtain the quality of the parent node , at the same time, the radiation range value can be updated to all descendant nodes, the farthest from the upper left and lower right positions of the target parent node's own coordinates, that is, the radiation range of the parent node can be updated to the maximum range of all its descendant nodes.

In the above-mentioned embodiments of the present application, based on the qualities corresponding to the multiple first pixel sets, the preset pixel sets in the multiple first pixel sets are split to obtain multiple second pixel sets, including: step A, obtaining preset pixels The radiation range of the set, wherein the radiation range is used to characterize the coordinate range of all pixels in the preset pixel set; step B, determine the pixel corresponding to the maximum density value in the radiation range, and obtain candidate pixels; step C, based on the candidate pixels A split operation is performed on the preset pixel set to obtain the split first set and the second set, wherein the split first set includes candidate pixels and associated pixels associated with the candidate pixels, and the split second set includes Other pixels in the preset pixel set except the pixels in the split first set; step D, when the quality corresponding to the split second set is greater than the second preset value, update the radiation range to split coordinate range of all pixels in the second set after splitting, and repeat the above step B and step C until the quality corresponding to the second set after splitting is less than or equal to the second preset value, and multiple second sets of pixels are obtained , wherein the second preset value is smaller than the first preset value.

The density value of the above candidate pixels may be a value greater than 1.

The above-mentioned first preset value may be 1, and the above-mentioned second preset value may be 0, but not limited thereto, and the specific first and second preset values may be set according to requirements.

The density value of the parent node in the above preset pixel set is greater than 1. If the density value of the parent node is greater than 1, it means that the preset pixel set represents multiple target objects. Therefore, the preset pixel set needs to be split. So that each target object can correspond to a set of pixels. Mapping in the monitoring image is to aggregate the position information of multiple target objects into one position information. Therefore, it is necessary to split the preset pixel set to obtain a set corresponding to each target object, so as to realize the dense target audience for targeting purposes.

In an optional embodiment, the radiation range of the preset pixel set can be obtained, wherein the radiation range of the preset pixel set is the coordinate range of all pixels in the preset pixel set, and the maximum density within the radiation range can be determined Values corresponding to the pixel points to obtain candidate pixels, the preset pixel set can be split according to the density value of the candidate pixel to obtain the first set and the second set after splitting. At this time, the candidate pixels contained in the first set and Candidate pixels have an associated relationship. If the quality corresponding to the second set after splitting is greater than the second preset value, it means that the second set represents multiple target objects. At this time, the radiation range can be updated to the second set after splitting. Coordinate ranges of all pixels in the set, and repeat the above operations on the second set until the quality corresponding to the split second set is less than or equal to the second preset value, and multiple second pixel sets are obtained. If the second set If the quality corresponding to the two sets is less than or equal to the second preset value, it means that only one target object is represented in the second set. At this time, there is no need to split the second set again, and the second set can accurately represent its corresponding The location of the target object.

In the above-mentioned embodiments of the present application, the preset pixel set is split based on the candidate pixels to obtain the split first set and the second set, including: multiple times of diffusion centered on the candidate pixel, and each of the preset pixel sets The density value corresponding to the pixel point at the second diffusion position is accumulated with the accumulation result of the previous diffusion position to obtain the accumulation result of the diffusion position, wherein the accumulation result of the first diffusion position is obtained by adding the pixel point corresponding to the first diffusion position The density value of is obtained by accumulating the density value corresponding to the candidate pixel. Each diffusion position is the position of the pixel adjacent to the previous diffusion position in the monitoring image and far away from the candidate pixel. The first diffusion The position is the position of the pixel adjacent to the candidate pixel; if the cumulative result of the diffusion position is greater than the first preset value, based on the candidate pixel and the pixels at all diffusion positions before the diffusion position, the post-split The first set of ; the pixel points in the first set after splitting are deleted from the preset pixel set to obtain the second set after splitting.

In an optional embodiment, multiple diffusions can be performed with the candidate pixel as the center, and the density value corresponding to the pixel point at each diffusion position in the preset pixel set is accumulated with the cumulative result of the previous diffusion position to obtain The accumulation result of the diffusion position can gradually split the pixels representing multiple target objects in the preset pixel set by accumulating the diffusion position. When the accumulation result of the diffusion position is greater than the first preset value, it can be based on The candidate pixel and the pixel points at all diffusion positions before the diffusion position are obtained to obtain the first set after splitting, and the pixels in the first set after splitting can be deleted from the preset pixel set, that is, the preset pixel set can be deleted. Set the density value of the pixel points in the first set in the set to 0, so as to obtain the second set after splitting, so that the quality of the second set after splitting can be reduced, and continue to split the second set until the second set The quality of the two sets is less than 0, indicating that the split is complete.

It should be noted that since the first set after splitting is the candidate pixel and the pixels at all diffusion positions before the diffusion position, it does not add the diffusion positions exceeding the first preset value, therefore, the first set after splitting The accumulation result of a set should be smaller than the first preset value, that is, the first set after splitting represents a target object.

In another optional embodiment, after the quality aggregation step, for a candidate node whose quality exceeds 1.0 (that is, the above-mentioned preset pixel set), it means that there is more than one target object within the radiation range of the candidate node , so it is necessary to perform a split operation on the candidate node. Optionally, the candidate node can be popped out of the stack first, and then the position C' with the highest density in the radiation range of the candidate node (that is, the position corresponding to the above-mentioned candidate pixel) As the starting position, gradually spread to the surrounding, these diffused positions can be used as child nodes of C', and the mass sum of all child nodes can be added to C' (the first set) until the quality of C' exceeds 1.0, You can add C' to the candidate node stack, and set the position density value belonging to C' in the range of the original candidate node C (preset pixel set) to 0, so the total mass of node C is correspondingly reduced and repeat the previous step, Until the quality sum of the original node C is lower than 0, it means that all candidate nodes have been extracted from the original node C at this time, and this process represents rounding up.

In the above-mentioned embodiments of the present application, each first pixel set adopts a tree structure, and after splitting the preset pixel set based on the candidate pixels to obtain the split first set and second set, the candidate pixel points correspond to The quality of the nodes is set to a third preset value.

The above-mentioned third preset value may be 0, but not limited thereto, and may be set according to requirements.

In an optional embodiment, after splitting the preset pixel set according to the candidate pixels to obtain the split first set and the second set, the quality of the node corresponding to the candidate pixel is set to the third The preset value, that is, the quality of the node corresponding to the candidate pixel node within the radiation range can be set to 0, indicating that the candidate pixel node has been extracted from the preset pixel set at this time.

In the above embodiments of the present application, based on the quality corresponding to the multiple second pixel sets, determining the target pixel point from the multiple second pixel sets includes: according to the quality corresponding to the multiple second pixel sets Perform sorting to obtain a sorting result; obtain a preset number of second pixel sets in the sorting result to obtain a target pixel set, wherein the preset number is the number of target objects; determine the corresponding maximum density value from the pixel points in the target pixel set The pixels of are the target pixels.

In an optional embodiment, the plurality of second pixel sets may be sorted from small to large according to the quality corresponding to the plurality of second pixel sets, and the second pixel set with higher quality indicates that the second pixel set has a higher probability of containing the target object. is large, at this time, the target positioning result of the target object can be determined according to the second pixel set, and the number of the second pixel set to be retained can be determined according to the roughly estimated number of target objects, that is, the above-mentioned target pixel set Quantity, the pixel point corresponding to the maximum density value can be determined from the pixels in the target pixel set as the target pixel point. Since the number of target objects is equal to the preset number, the obtained target positioning result needs to be consistent with the number of target objects, so that Obtain the pixel point corresponding to the maximum density value in the sorting result as the target pixel point, and determine the target pixel point as the target positioning result of the target object.

Through the above steps, relatively accurate positioning results of the target object can be extracted from the density estimation results, and the total number of positioning results is basically consistent with the number of target objects in the monitoring image.

Fig. 4 is a flowchart of an image processing process according to an embodiment of the present application. As shown in Figure 4, the monitoring image is first obtained, where the monitoring image can be a dense image containing dense crowds, and the crowd density estimation model can be used to estimate the crowd density of the monitoring image to obtain the density estimation result, and then use the density climbing algorithm The method traverses each density value in the density estimation result to determine whether the density value is the maximum density value in the preset area, and if the density value is greater than the maximum density value in the preset area, a new set is established, and The pixel points are stored in a new set, and multiple first pixel sets are generated according to the new set. The center of the vortex in the figure can be the position of the pixel point with the maximum density value, and the density value corresponding to the pixel point is not the maximum density value. In this case, the pixel point can be stored in the set with the maximum density value stored in the plurality of first pixel sets, where the pixel point can be a node in an area other than the center point of the vortex in the figure, and the density value corresponding to the pixel point can be Establish multiple first pixel sets; each first pixel set can be aggregated to obtain the quality corresponding to each first pixel set, because multiple first pixel sets will have a quality greater than 1, that is, a first pixel set A set of pixels may represent multiple target objects. Therefore, it is necessary to split multiple sets of first pixels to obtain multiple sets of second pixels. The quality of multiple second sets of pixels can be sorted, and the selected quality is in the monitoring image. The node coordinates of the estimated total number of people are used as the target positioning results of the crowd in the monitoring image.

Fig. 5 is a flowchart of another image processing method according to an embodiment of the present application. As shown in Fig. 5, the method includes:

Step S501, obtaining the density estimation output of a general crowd density estimation model on a high-density image.

The above-mentioned high-intensity images may be monitoring images.

Optionally, for a high-resolution RGB image intercepted from a video stream, its shape is (H, W, 3), through a crowd density estimation model, a low-resolution density map is generally obtained, that is, a A matrix M with a shape of (H/4, W/4), the elements in the matrix are floating-point numbers, representing the probability that a target may appear at each pixel position, so the total number of people in the screen can be obtained by summing the elements in the matrix K, namely

Step S502, scan the elements in the density matrix M one by one, judge whether the element is the maximum floating-point density value point in its own 3x3 neighborhood, if it is, push it into the stack S of candidate points, if not, push it into the other In the child node stack C_S of the peak point C in the neighborhood.

The aforementioned maximum floating point density value point may be the maximum density value in a preset area.

The aforementioned candidate points may be candidate pixels.

Here, when pushing into the stack, store the pixel coordinates (i, j) of each element and its floating-point density value v in M (hereinafter referred to as quality), and store the radiation range of each element as itself Coordinates, where the radiation range refers to the coordinate range of the descendant nodes that each element is the successor of the parent node.

Step S503, the quality of the candidate point is only its own quality, and the quality of all descendant nodes of the candidate point can be added to itself through the depth-first search algorithm, and updated as its own quality v';

All the descendant nodes mentioned above may include child nodes, child nodes of child nodes, and so on.

At the same time, the radiation range value is updated to the upper left and lower right positions farthest from the candidate node's own coordinates among all descendant nodes, that is, the radiation range of the candidate point is updated to the maximum range of all descendant nodes of itself, which provides for extracting candidate nodes in step S504 Coordinate basis.

Step S504, starting from the position C' with the highest density in the radiation range of the candidate node, gradually spreading to the surroundings, taking these diffused positions as the child nodes of C', adding their mass to C', until If the quality sum of C' exceeds 1.0, add C' to the candidate node stack, and set the density value of the position belonging to C' in the range of the original node C to 0. At this time, the total mass of node C is correspondingly reduced, and repeat the process Steps until the sum of the original node C's quality is lower than 0.

If the quality sum of the original node C is lower than 0, it means that a candidate node has been extracted from the original node C at this time

After the quality aggregation step, a candidate node whose quality exceeds 1.0 (for example, the quality is V) indicates that there is more than one target within the radiation range of the candidate node. Therefore, it is necessary to perform a split operation on the candidate node. Specifically, the candidate node C may be popped out of the stack first.

Step S505, sort all the candidate points by quality, and select the coordinates whose quality is at the TOPK node.

The above K is the estimated total number of people in the high-density image.

This application uses the local density hill-climbing algorithm, establishes a strict node tree structure from the density map, divides the density map into density groups with different quality, and uses the depth-first algorithm to aggregate the node quality. Each quality contains the quality of all its child nodes. For each parent node whose quality exceeds 1 (the number of people exceeds 1), the split operation can be performed recursively, so that the number of coordinates corresponding to each parent node is exactly equal to the rounding of its total quality, thus avoiding loss in extremely dense areas The problem with the number of targets. After the aggregation and splitting operations are completed, the quality of all candidate nodes can be sorted. If the total number of people is N, the first N candidate nodes can be selected as the output of the final coordinates, thus avoiding the problem that the number of coordinates does not match the total number of people, and at the same time Threshold sensitivity issues are avoided.

In this application, based on the self-designed density climbing algorithm and depth-first quality aggregation operation, more accurate coordinate positions of target objects are produced from the density estimation results. The number of target objects in this application is the same as the number of coordinates in the density estimation results. Consistent, it provides a more refined structured output for crowd density monitoring, and provides a more intuitive display form and a wider range of application scenarios for the algorithm.

It should be noted that for the foregoing method embodiments, for the sake of simple description, they are expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence. Depending on the application, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification belong to preferred embodiments, and the actions and modules involved are not necessarily required by this application.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, disk, CD) contains several instructions to make a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) execute the method of each embodiment of the present application.

Example 2

According to an embodiment of the present application, an image processing method is also provided. It should be noted that the steps shown in the flow charts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although in the flow chart The figures show a logical order, but in some cases the steps shown or described may be performed in an order different from that shown or described herein.

Fig. 6 is a flow chart of an image processing method according to Embodiment 2 of the present application. As shown in Fig. 6, the method may include the following steps:

Step S602, monitor the active area through the monitoring device to obtain a monitoring image.

Wherein, the monitoring image contains the target population.

Step S604, performing density estimation on the monitoring image to obtain a density estimation result of the target population.

Wherein, the multiple density values contained in the density estimation result are used to represent the probability that the target crowd exists in multiple pixel points in the monitoring image.

Step S606, based on the density estimation result, determine the target pixel point from the plurality of pixel points.

Wherein, there is a target crowd in the target pixel.

Step S608, based on the position of the target pixel in the monitoring image, the target positioning result of the target crowd in the activity area is obtained.

It should be noted that the preferred implementations involved in the above examples of the present application are the same as those provided in Example 1, as well as the application scenarios and implementation process, but are not limited to the solutions provided in Example 1.

Example 3

According to an embodiment of the present application, an image processing method is also provided. It should be noted that the steps shown in the flow charts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although in the flow chart The figures show a logical order, but in some cases the steps shown or described may be performed in an order different from that shown or described herein.

Fig. 7 is a flowchart of an image processing method according to Embodiment 3 of the present application. As shown in Fig. 7, the method may include the following steps:

Step S702, in response to an input instruction acting on the operation interface, displaying a monitoring image of the area to be monitored on the operation interface.

Wherein, the monitoring image contains the target object.

Step S704, responding to the positioning instruction acting on the operation interface, displaying the target positioning result of the target object in the area to be monitored on the operation interface.

Among them, the target positioning result is determined by the position of the target pixel in the monitoring image determined from a plurality of pixels in the monitoring image, and the target pixel is determined based on the density estimation result of the target object, and the density estimation result is obtained by performing density estimation on the monitoring image It is obtained that the multiple density values contained in the density estimation result are used to characterize the probability that the target object exists at multiple pixel points.

It should be noted that the preferred implementations involved in the above examples of the present application are the same as those provided in Example 1, as well as the application scenarios and implementation process, but are not limited to the solutions provided in Example 1.

Example 4

According to the embodiment of the present application, an image processing method that can be applied to virtual reality scenarios such as virtual reality VR devices and augmented reality AR devices is also provided. It should be noted that the steps shown in the flow chart of the accompanying drawings can be and, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.

FIG. 8 is a flowchart of an image processing method according to Embodiment 4 of the present application. As shown in Figure 8, the method may include the following steps:

Step S802, monitor the monitoring image of the area to be monitored through the monitoring device.

Wherein, the monitoring image contains the target object.

Step S804, displaying the monitoring image on the presentation screen of the virtual reality VR device or the augmented reality AR device.

Step S806, based on the density estimation result, determine the target pixel point from the plurality of pixel points.

Wherein, there is a target object in the target pixel.

Step S808, based on the position of the target pixel in the monitoring image, the target positioning result of the target object in the area to be monitored is obtained.

Step S810, driving the VR device or the AR device to render and display the target positioning result.

Optionally, in this embodiment, the above image processing method may be applied in a hardware environment composed of a server and a virtual reality device. The target positioning results are displayed on the presentation screen of the virtual reality VR device or the augmented reality AR device. The server can be the server corresponding to the media file operator. The above-mentioned network includes but not limited to: wide area network, metropolitan area network or local area network. The above-mentioned virtual reality device does not Not limited to: virtual reality helmets, virtual reality glasses, virtual reality all-in-one machines, etc.

Optionally, the virtual reality device includes: a memory, a processor, and a transmission device. The memory is used to store an application program that can be used to perform:

Monitor the monitoring image of the area to be monitored through the monitoring equipment, wherein the monitoring image contains the target object; display the monitoring image on the presentation screen of the virtual reality VR device or the augmented reality AR device; perform density estimation on the monitoring image to obtain the density estimation of the target object As a result, the multiple density values contained in the density estimation result are used to characterize the probability of the target object at multiple pixels in the monitoring image; based on the density estimation result, the target pixel is determined from the multiple pixels, wherein the target pixel There is a target object; based on the position of the target pixel in the monitoring image, the target positioning result of the target object in the area to be monitored is obtained; the VR device or AR device is driven to render and display the target positioning result.

It should be noted that the above-mentioned image processing method applied in a VR device or an AR device of this embodiment may include the method of the embodiment shown in FIG. 8 , so as to realize the purpose of driving the VR device or AR device to display the target positioning result.

Optionally, the processor in this embodiment may call the application program stored in the above-mentioned memory through a transmission device to perform the above-mentioned steps. The transmission device can receive the media file sent by the server through the network, and can also be used for data transmission between the processor and the memory.

Optionally, in the virtual reality device, there is a head-mounted display with eyeball tracking, the screen in the HMD head-mounted display is used to display the displayed video picture, and the eyeball tracking module in the HMD is used to obtain real-time information about the user's eyeballs. Motion trajectory, tracking system, used to track the position information and motion information of the user in the real three-dimensional space, and a calculation processing unit, used to obtain the real-time position and motion information of the user from the tracking system, and calculate the position of the user's head in the virtual three-dimensional space The three-dimensional coordinates in the virtual three-dimensional space, and the direction of the user's field of view in the virtual three-dimensional space.

In the embodiment of this application, the virtual reality device can be connected to the terminal, and the terminal and the server are connected through the network. The above-mentioned virtual reality device is not limited to: virtual reality helmet, virtual reality glasses, virtual reality all-in-one machine, etc. The above-mentioned terminal is not limited to Not limited to PCs, mobile phones, tablet computers, etc., the server may be a server corresponding to a media file operator, and the above-mentioned network includes but is not limited to: a wide area network, a metropolitan area network or a local area network.

It should be noted that the preferred implementations involved in the above examples of the present application are the same as those provided in Example 1, as well as the application scenarios and implementation process, but are not limited to the solutions provided in Example 1.

Example 5

According to an embodiment of the present application, an image processing method is also provided. It should be noted that the steps shown in the flow charts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although in the flow chart The figures show a logical order, but in some cases the steps shown or described may be performed in an order different from that shown or described herein.

Fig. 9 is a flowchart of an image processing method according to Embodiment 5 of the present application. As shown in Fig. 9, the method may include the following steps:

Step S902, acquiring the monitoring image of the area to be monitored by calling the first interface.

Wherein, the first interface includes a first parameter, and a parameter value of the first parameter is a monitoring image, and the monitoring image includes a target object.

Step S904, performing density estimation on the monitoring image to obtain a density estimation result of the target object.

Wherein, the multiple density values included in the density estimation result are used to characterize the probability that the target object exists in multiple pixel points in the monitoring image.

Step S906, based on the density estimation result, determine the target pixel point from the plurality of pixel points.

Wherein, there is a target object in the target pixel.

Step S908, based on the position of the target pixel in the monitoring image, the target positioning result of the target object in the area to be monitored is obtained.

Step S910, outputting a target positioning result by calling the second interface.

Wherein, the second interface includes a second parameter, and a parameter value of the second parameter is a target positioning result.

It should be noted that the preferred implementations involved in the above examples of the present application are the same as those provided in Example 1, as well as the application scenarios and implementation process, but are not limited to the solutions provided in Example 1.

Example 6

According to an embodiment of the present application, an image processing device for implementing the above image processing method is also provided. FIG. 10 is a schematic diagram of an image processing device according to Embodiment 6 of the present application. As shown in FIG. 10 , the device 1000 It includes: an acquisition module 1002 , an estimation module 1004 , a determination module 1006 and a generation module 1008 .

Wherein, the acquisition module is used to obtain the monitoring image of the area to be monitored, wherein the monitoring image contains the target object; the estimation module is used to perform density estimation on the monitoring image to obtain the density estimation result of the target object, wherein the density estimation result contains multiple The density value is used to represent the probability of the target object being present in multiple pixels in the monitoring image; the determination module is used to determine the target pixel point from multiple pixel points based on the density estimation result, wherein the target pixel point has the target object; the generation module uses Based on the position of the target pixel in the monitoring image, the target positioning result of the target object in the area to be monitored is obtained. .

It should be noted here that the acquisition module 1002, estimation module 1004, determination module 1006, and generation module 1008 correspond to steps S302 to S308 in Embodiment 1, and the examples and application scenarios realized by the four modules and corresponding steps The same, but not limited to the content disclosed in Embodiment 1 above. It should be noted that, as a part of the device, the above modules can run in the computer terminal provided in Embodiment 1.

In the above embodiments of the present application, the determining module includes: a dividing unit, an aggregating unit, a splitting unit, and a first determining unit.

Wherein, the division unit is used for dividing multiple pixel points into multiple first pixel sets based on the density estimation result, wherein, there are different objects in the pixel points in different first pixel sets; the aggregation unit is used for each first pixel The density values corresponding to the pixel points in the set are aggregated to obtain the quality corresponding to each first pixel set; the splitting unit is used to split the preset pixel set in the multiple first pixel sets based on the quality corresponding to the multiple first pixel sets Operation, to obtain a plurality of second pixel sets, wherein the quality corresponding to the preset pixel set is greater than the first preset quality; the first determination unit is used to obtain from the plurality of second pixel sets Determine the target pixel in .

In the above embodiments of the present application, the division unit is also used to traverse each pixel in the monitoring image, and determine whether the density value corresponding to the pixel in the density estimation result is the maximum density among the density values corresponding to all pixels in the preset area Value, wherein, the preset area is used to represent the area composed of pixels adjacent to the pixel point in the monitoring image centered on the pixel point; the division unit is also used to set the maximum density value at the density value corresponding to the pixel point In the case of , a new set is established and the pixel point is stored in the new set, wherein the new set is used to generate a plurality of first pixel sets; the division unit is also used when the density value corresponding to the pixel point is not the maximum density value In the case of , the pixel point is stored in the set in which the maximum density value is stored among the plurality of first pixel sets.

In the above embodiments of the present application, the aggregation unit is also used to determine the density value corresponding to the parent node in the tree structure and the density value corresponding to the child node, wherein the child node is a descendant node of the parent node in the tree structure; the aggregation unit is also used for The density value corresponding to the parent node and the density value corresponding to the child node are aggregated to obtain the quality of the parent node; the aggregation unit is also used to determine that the quality of the parent node in the tree structure is the quality corresponding to each first pixel set.

In the above-mentioned embodiments of the present application, the splitting unit is also used to perform step A to obtain the radiation range of the preset pixel set, wherein the radiation range is used to represent the coordinate range of all pixel points in the preset pixel set; step B is to determine the radiation range The pixel point corresponding to the maximum density value within the range is obtained as a candidate pixel; step C, based on the candidate pixel, the preset pixel set is split to obtain the first set and the second set after splitting, wherein the first set after splitting contains Candidate pixels and associated pixels that have an association relationship with the candidate pixels, the second set after splitting contains other pixels in the preset pixel set except the pixels in the first set after splitting; step D, after splitting When the quality corresponding to the second set is greater than the second preset value, the radiation range is updated to the coordinate range of all pixels in the second set after splitting, and the above steps B and C are repeated until the second set after splitting The quality corresponding to the set is less than or equal to a second preset value to obtain a plurality of second pixel sets, wherein the second preset value is smaller than the first preset value.

In the above-mentioned embodiments of the present application, the splitting unit is also used to perform multiple diffusions centering on the candidate pixel, and accumulate the density value corresponding to the pixel point at each diffusion position in the preset pixel set with the accumulation result of the previous diffusion position, The accumulation result of the diffusion position is obtained, wherein the accumulation result of the first diffusion position is obtained by accumulating the density value corresponding to the pixel point at the first diffusion position and the density value corresponding to the candidate pixel point, and each diffusion position is a monitoring In the image, the pixel point adjacent to the previous diffusion position and away from the pixel point of the candidate pixel is located, and the first diffusion position is the position of the pixel adjacent to the candidate pixel; the splitting unit is also used in the When the cumulative result of the diffusion position is greater than the first preset value, based on the candidate pixel and the pixel points at all diffusion positions before the diffusion position, the first set after splitting is obtained; the splitting unit is also used to split the first The pixels in the set are deleted from the preset pixel set to obtain the second set after splitting.

In the above-mentioned embodiments of the present application, each first pixel set adopts a tree structure, and after splitting the preset pixel set based on the candidate pixels to obtain the split first set and second set, the candidate pixel points correspond to The quality of the nodes is set to a third preset value.

In the above embodiments of the present application, the determination module includes: a sorting unit, an acquisition unit, and a second determination unit.

Wherein, the sorting unit is used to sort the plurality of second pixel sets according to the quality corresponding to the multiple second pixel sets to obtain a sorting result; the obtaining unit is used to obtain a preset number of second pixel sets in the sorting result to obtain the target pixel The set, wherein the preset number is the number of target objects; the second determination unit is configured to determine the pixel point corresponding to the maximum density value from the pixel points in the target pixel set as the target pixel point.

It should be noted that the preferred implementations involved in the above examples of the present application are the same as those provided in Example 1, as well as the application scenarios and implementation process, but are not limited to the solutions provided in Example 1.

Example 7

According to an embodiment of the present application, an image processing device for implementing the above image processing method is also provided. FIG. 11 is a schematic diagram of an image processing device according to Embodiment 7 of the present application. As shown in FIG. 11 , the device 1100 It includes: a monitoring module 1102 , an estimating module 1104 , a determining module 1106 , a generating module 1108 , and an output module 1110 .

Wherein, the monitoring module obtains the monitoring image by monitoring the active area of the monitoring equipment, wherein the monitoring image contains the target population; the estimation module is used to perform density estimation on the monitoring image to obtain the density estimation result of the target population, wherein the density estimation result contains multiple The density value is used to characterize the probability of the target group being present in multiple pixels in the monitoring image; the determination module is used to determine the target pixel point from multiple pixel points based on the density estimation result, wherein the target pixel point has the target group; the generation module uses Based on the position of the target pixel in the monitoring image, the target positioning result of the target crowd in the active area is obtained.

It should be noted here that the monitoring module 1102, estimation module 1104, determination module 1106, generation module 1108, and output module 1110 correspond to steps S602 to S608 in Embodiment 2, and the four modules and corresponding steps realize The examples and application scenarios are the same, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that, as a part of the device, the above modules can run in the computer terminal provided in Embodiment 1.

It should be noted that the preferred implementations involved in the above examples of the present application are the same as those provided in Example 1, as well as the application scenarios and implementation process, but are not limited to the solutions provided in Example 1.

Example 8

According to an embodiment of the present application, an image processing device for implementing the above image processing method is also provided. FIG. 12 is a schematic diagram of an image processing device according to Embodiment 8 of the present application. As shown in FIG. 12 , the device 1200 It includes: a first display module 1202 and a second display module 1204 .

Wherein, the first display module is used to display the monitoring image of the area to be monitored on the operation interface in response to the input instruction acting on the operation interface, wherein the monitoring image contains the target object; the second display module is used to

In response to the positioning instruction acting on the operation interface, the target positioning result of the target object in the area to be monitored is displayed on the operation interface, wherein the target positioning result is displayed on the monitoring image through the target pixel points determined from the multiple pixel points in the monitoring image In the position determination, the target pixel is determined based on the density estimation result of the target object, and the density estimation result is obtained by performing density estimation on the monitoring image, and the multiple density values contained in the density estimation result are used to represent the probability of the target object existing in multiple pixel points .

It should be noted here that the above-mentioned first display module 1202 and second display module 1204 correspond to steps S702 to S704 in Embodiment 3, and the examples and application scenarios implemented by the two modules are the same as the corresponding steps, but not It is limited to the content disclosed in the above-mentioned embodiment 1. It should be noted that, as a part of the device, the above modules can run in the computer terminal provided in Embodiment 1.

It should be noted that the preferred implementations involved in the above examples of the present application are the same as those provided in Example 1, as well as the application scenarios and implementation process, but are not limited to the solutions provided in Example 1.

Example 9

According to an embodiment of the present application, an image processing device for implementing the above image processing method is also provided. FIG. 13 is a schematic diagram of an image processing device according to Embodiment 9 of the present application. As shown in FIG. 13 , the device 1300 It includes: a monitoring module 1302 , a display module 1304 , an estimation module 1306 , a determination module 1308 , a generation module 1310 , and a driving module 1312 .

Wherein, the monitoring module is used to monitor the monitoring image of the area to be monitored through the monitoring device, wherein the monitoring image contains the target object; the display module is used to display the monitoring image on the presentation screen of the virtual reality VR device or the augmented reality AR device; the estimation module uses It is used to perform density estimation on the monitoring image to obtain the density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used to represent the probability of the target object at multiple pixel points in the monitoring image; the determination module is used for density estimation based on As a result, the target pixel is determined from a plurality of pixels, wherein the target pixel has a target object; the generation module is used to obtain a target positioning result of the target object in the area to be monitored based on the position of the target pixel in the monitoring image; The driver module is used to drive VR devices or AR devices to render and display target positioning results.

It should be noted here that the monitoring module 1302, the display module 1304, the estimation module 1306, the determination module 1308, the generation module 1310, and the driving module 1312 correspond to steps S802 to S812 in Embodiment 4, and the five modules correspond to the corresponding The examples and application scenarios implemented by the steps are the same, but are not limited to the content disclosed in the above-mentioned embodiment 1. It should be noted that, as a part of the device, the above modules can run in the computer terminal provided in Embodiment 1.

It should be noted that the preferred implementations involved in the above examples of the present application are the same as those provided in Example 1, as well as the application scenarios and implementation process, but are not limited to the solutions provided in Example 1.

Example 10

According to an embodiment of the present application, an image processing device for implementing the above image processing method is also provided. FIG. 14 is a schematic diagram of an image processing device according to Embodiment 10 of the present application. As shown in FIG. 14 , the device 1400 It includes: calling module 1402 , estimating module 1404 , determining module 1406 , generating module 1408 , and outputting module 1410 .

Wherein, the calling module is used to obtain the monitoring image of the area to be monitored by calling the first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter is a monitoring image, and the monitoring image contains a target object; the estimation module is used to Perform density estimation on the monitoring image to obtain a density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used to represent the probability of the target object existing at multiple pixels in the monitoring image; the determination module is used to determine based on the density estimation result, Determining a target pixel point from a plurality of pixels, wherein the target pixel point has a target object; the generation module is used to obtain the target positioning result of the target object in the area to be monitored based on the position of the target pixel point in the monitoring image; the output module It is used to output the target positioning result by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target positioning result.

It should be noted here that the above calling module 1402, estimation module 1404, determination module 1406, generation module 1408, and output module 1410 correspond to steps S902 to S910 in Embodiment 5, and the five modules and corresponding steps realize The examples and application scenarios are the same, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that, as a part of the device, the above modules can run in the computer terminal provided in Embodiment 1.

It should be noted that the preferred implementations involved in the above examples of the present application are the same as those provided in Example 1, as well as the application scenarios and implementation process, but are not limited to the solutions provided in Example 1.

Example 11

Embodiments of the present application may provide an electronic device, and the electronic device may be any electronic device in a group of electronic devices. Optionally, in this embodiment, the foregoing electronic device may also be replaced with a terminal device such as a mobile terminal.

Optionally, in this embodiment, the foregoing electronic device may be located in at least one network device among multiple network devices in the computer network.

In this embodiment, the above-mentioned electronic device can execute the program codes of the following steps in the image processing method: acquire the monitoring image of the area to be monitored, wherein the monitoring image contains the target object; perform density estimation on the monitoring image to obtain the density estimation of the target object As a result, the multiple density values contained in the density estimation result are used to characterize the probability of the target object at multiple pixels in the monitoring image; based on the density estimation result, the target pixel is determined from the multiple pixels, wherein the target pixel There is a target object; based on the position of the target pixel in the monitoring image, the target positioning result of the target object in the area to be monitored is obtained.

Optionally, FIG. 15 is a structural block diagram of a computer terminal according to an embodiment of the present application. As shown in FIG. 15 , the computer terminal A may include: one or more (only one is shown in the figure) processors and memory.

Wherein, the memory can be used to store software programs and modules, such as the program instructions/modules corresponding to the image processing method and device in the embodiment of the present application, and the processor executes various functional applications by running the software programs and modules stored in the memory. And data processing, that is, realizing the above-mentioned image processing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the terminal A through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The processor can call the information stored in the memory and the application program through the transmission device to perform the following steps: obtain the monitoring image of the area to be monitored, wherein the monitoring image contains the target object; perform density estimation on the monitoring image to obtain the density estimation of the target object As a result, the multiple density values contained in the density estimation result are used to characterize the probability of the target object at multiple pixels in the monitoring image; based on the density estimation result, the target pixel is determined from the multiple pixels, wherein the target pixel There is a target object; based on the position of the target pixel in the monitoring image, the target positioning result of the target object in the area to be monitored is obtained.

Optionally, the above-mentioned processor may also execute the program code of the following steps: based on the density estimation result, divide the multiple pixel points into multiple first pixel sets, wherein different objects exist in the pixel points in different first pixel sets; Aggregating the density values corresponding to the pixel points in each first pixel set to obtain the quality corresponding to each first pixel set; performing a splitting operation to obtain multiple second pixel sets, wherein the quality corresponding to the preset pixel set is greater than the first preset quality; based on the quality corresponding to the multiple second pixel sets, determine the target pixel from the multiple second pixel sets point.

Optionally, the above-mentioned processor can also execute the program code of the following steps: traverse each pixel in the monitoring image, and determine whether the density value corresponding to the pixel in the density estimation result is the density value corresponding to all pixels in the preset area The maximum density value in , where the preset area is used to characterize the area of the monitoring image centered on the pixel point and adjacent to the pixel point; the density value corresponding to the pixel point is equal to the maximum density value case, establish a new set, and store the pixel in the new set, wherein the new set is used to generate a plurality of first pixel sets; in the case that the density value corresponding to the pixel is not the maximum density value, the The pixel points are stored in the set with the maximum density value stored in the plurality of first pixel sets.

Optionally, the above-mentioned processor can also execute the program code of the following steps: determine the density value corresponding to the parent node in the tree structure and the density value corresponding to the child node, wherein the child node is a descendant node of the parent node in the tree structure; The density value corresponding to the parent node and the density value corresponding to the child node are aggregated to obtain the quality of the parent node; the quality of the parent node in the tree structure is determined to be the quality corresponding to each first pixel set.

Optionally, the above-mentioned processor may also execute the program code of the following steps: Step A, obtain the radiation range of the preset pixel set, wherein the radiation range is used to represent the coordinate range of all pixel points in the preset pixel set; Step B , determine the pixel point corresponding to the maximum density value within the radiation range, and obtain candidate pixels; step C, perform splitting operation on the preset pixel set based on the candidate pixels, and obtain the first and second sets after splitting, wherein, the first set after splitting A set includes candidate pixels and associated pixels that have an association relationship with the candidate pixels, and the second set after splitting includes other pixels in the preset pixel set except the pixels in the first set after splitting; step D, in When the quality corresponding to the second set after splitting is greater than the second preset value, update the radiation range to the coordinate range of all pixels in the second set after splitting, and repeat the above steps B and C until after the split The quality corresponding to the second set of is less than or equal to the second preset value to obtain a plurality of second pixel sets, wherein the second preset value is smaller than the first preset value.

Optionally, the above-mentioned processor can also execute the program code of the following steps: performing multiple diffusions with the candidate pixel as the center, and accumulating the density value corresponding to the pixel point at each diffusion position in the preset pixel set with the previous diffusion position The results are accumulated to obtain the accumulation result of the diffusion position, wherein the accumulation result of the first diffusion position is obtained by accumulating the density value corresponding to the pixel point at the first diffusion position and the density value corresponding to the candidate pixel point, each time The diffusion position is the position of the pixel adjacent to the previous diffusion position in the monitoring image and far away from the candidate pixel, and the first diffusion position is the position of the pixel adjacent to the candidate pixel; When the cumulative result of the position is greater than the first preset value, based on the candidate pixel and the pixels at all diffusion positions before the diffusion position, the first set after splitting is obtained; the pixels in the first set after splitting are obtained from The preset pixel set is deleted to obtain the second set after splitting.

Optionally, the above-mentioned processor can also execute the program code in the following steps: each first set of pixels adopts a tree structure, and performs a split operation on the preset set of pixels based on the candidate pixels to obtain the first set and the second set after splitting. After the collection, the quality of the node corresponding to the candidate pixel is set to a third preset value.

Optionally, the above-mentioned processor can also execute the program code of the following steps: sort the multiple second pixel sets according to the quality corresponding to the multiple second pixel sets to obtain the sorting result; obtain the preset number of second pixel sets in the sorting result. A pixel set to obtain a target pixel set, wherein the preset number is the number of target objects; and the pixel point corresponding to the maximum density value is determined from the pixel points in the target pixel set as the target pixel point.

The processor can call the information stored in the memory and the application program through the transmission device to perform the following steps: monitor the active area through the monitoring equipment to obtain a monitoring image, wherein the monitoring image contains the target population; perform density estimation on the monitoring image to obtain the target population Density estimation results, wherein the multiple density values contained in the density estimation results are used to characterize the probability of the presence of target groups in multiple pixel points in the monitoring image; based on the density estimation results, the target pixel points are determined from multiple pixel points, wherein the target There is a target crowd in the pixel; based on the position of the target pixel in the monitoring image, the target positioning result of the target crowd in the active area is obtained.

The processor can call the information stored in the memory and the application program through the transmission device to perform the following steps: in response to the input instruction acting on the operation interface, display the monitoring image of the area to be monitored on the operation interface, wherein the monitoring image contains the target object ; In response to the positioning instruction acting on the operation interface, the target positioning result of the target object in the area to be monitored is displayed on the operation interface, wherein the target positioning result is monitored by the target pixel points determined from the multiple pixel points in the monitoring image The position in the image is determined, and the target pixel is determined based on the density estimation result of the target object. The density estimation result is obtained by performing density estimation on the monitoring image. The multiple density values contained in the density estimation result are used to represent the presence of the target object at multiple pixel points. probability.

The processor can call the information stored in the memory and the application program through the transmission device to perform the following steps: monitor the monitoring image of the area to be monitored through the monitoring device, wherein the monitoring image contains the target object; in the virtual reality VR device or the augmented reality AR device The monitoring image is displayed on the presentation screen; density estimation is performed on the monitoring image to obtain a density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used to represent the probability that the target object exists at multiple pixels in the monitoring image; Based on the density estimation result, determine the target pixel point from multiple pixel points, wherein the target pixel point has a target object; based on the position of the target pixel point in the monitoring image, obtain the target positioning result of the target object in the area to be monitored; drive The VR device or AR device renders and displays the target positioning results.

The processor can call the information stored in the memory and the application program through the transmission device to perform the following steps: obtain the monitoring image of the area to be monitored by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter In order to monitor the image, the monitoring image contains the target object; the density estimation is performed on the monitoring image to obtain the density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used to represent the existence of the target object at multiple pixel points in the monitoring image Probability; based on the density estimation result, determine the target pixel point from multiple pixel points, wherein the target pixel point has the target object; based on the position of the target pixel point in the monitoring image, obtain the target positioning result of the target object in the area to be monitored ; Outputting the target positioning result by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target positioning result.

Using the embodiment of the present application, firstly, the monitoring image of the area to be monitored, wherein the monitoring image contains the target object; density estimation is performed on the monitoring image to obtain the density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used for Characterize the probability of the target object existing in multiple pixels in the monitoring image; based on the density estimation result, determine the target pixel point from multiple pixel points, wherein, the target pixel point has the target object; based on the position of the target pixel point in the monitoring image, The target positioning result of the target object in the area to be monitored is obtained, and the positioning of the object in the monitoring image is realized. It is easy to notice that the density estimation of the monitoring image can be performed to obtain the density estimation result of the target object, and the pixel points with the target object can be determined from multiple pixel points based on the density estimation result, avoiding the detection of pixels that do not contain the target object. In order to improve the accuracy of the positioning result of the target object, by obtaining the positioning result of the target object, the effect of increasing the output information of the target object can be achieved, thereby solving the problem that the algorithm of the related technology is difficult to output more accurate information of the target object in the monitoring image. Multi-information technical issues.

Those of ordinary skill in the art can understand that the structure shown in Figure 15 is only schematic, and the computer terminal can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, an applause computer, and a mobile Internet device (Mobile Internet Devices, MID) , PAD and other terminal equipment. FIG. 15 does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may also include more or fewer components (eg, network interface, display device, etc.) than those shown in FIG. 15 , or have a configuration different from that shown in FIG. 15 .

Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium can be Including: a flash disk, a read-only memory (Read-Only Memory, ROM), a random access device (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.

Example 12

The embodiment of the present application also provides a computer-readable storage medium. Optionally, in this embodiment, the above-mentioned computer-readable storage medium may be used to store the program code executed by the image processing method provided in Embodiment 1 above.

Optionally, in this embodiment, the above-mentioned computer-readable storage medium may be located in any computer terminal in the AR/VR equipment terminal group in the AR/VR equipment network, or in any mobile terminal in the mobile terminal group .

Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for performing the following steps: acquiring a monitoring image of the area to be monitored, wherein the monitoring image contains a target object; performing density estimation on the monitoring image , to obtain the density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used to characterize the probability of the target object existing in multiple pixels in the monitoring image; based on the density estimation result, determine the target pixel from the multiple pixel points Points, where there is a target object in the target pixel point; based on the position of the target pixel point in the monitoring image, the target positioning result of the target object in the area to be monitored is obtained.

Optionally, the above-mentioned storage medium is further configured to store program codes for performing the following steps: based on the density estimation result, dividing multiple pixel points into multiple first pixel sets, wherein pixels in different first pixel sets There are different objects in the points; the density values corresponding to the pixel points in each first pixel set are aggregated to obtain the quality corresponding to each first pixel set; based on the quality corresponding to multiple first pixel sets, multiple first pixel sets Perform a split operation on the preset pixel set to obtain multiple second pixel sets, wherein the quality corresponding to the preset pixel set is greater than the first preset quality; based on the quality corresponding to the multiple second pixel sets, from the multiple second pixel sets Determine the target pixel in the set.

Optionally, the above-mentioned storage medium is also configured to store program codes for performing the following steps: traverse each pixel in the monitoring image, and determine whether the density value corresponding to the pixel in the density estimation result is all pixels in the preset area The maximum density value among the density values corresponding to a point, wherein, the preset area is used to represent the area composed of pixels adjacent to the pixel point in the monitoring image centered on the pixel point; the density value corresponding to the pixel point In the case of the maximum density value, a new set is established, and the pixel point is stored in the new set, wherein the new set is used to generate a plurality of first pixel sets; the density value corresponding to the pixel point is not the maximum density value In some cases, the pixel point is stored in the set in which the maximum density value is stored among the plurality of first pixel sets.

Optionally, the above-mentioned storage medium is also configured to store program codes for performing the following steps: determining the density value corresponding to the parent node in the tree structure and the density value corresponding to the child node, wherein the child node is the parent node in the tree structure descendant nodes; aggregate the density value corresponding to the parent node and the density value corresponding to the child node to obtain the quality of the parent node; determine the quality of the parent node in the tree structure as the quality corresponding to each first pixel set.

Optionally, the above-mentioned storage medium is also configured to store program codes for performing the following steps: Step A, obtaining the radiation range of the preset pixel set, wherein the radiation range is used to characterize all pixel points in the preset pixel set Coordinate range; step B, determine the pixel point corresponding to the maximum density value in the radiation range, and obtain candidate pixels; step C, perform splitting operation on the preset pixel set based on the candidate pixels, and obtain the first set and the second set after splitting, wherein , the first set after splitting contains candidate pixels and associated pixels that have an association relationship with the candidate pixels, and the second set after splitting contains other pixels in the preset pixel set except for the pixels in the first set after splitting ; Step D, when the quality corresponding to the split second set is greater than the second preset value, update the radiation range to the coordinate range of all pixels in the split second set, and repeat the above step B and step C. Obtain a plurality of second pixel sets until the quality corresponding to the split second set is less than or equal to a second preset value, wherein the second preset value is smaller than the first preset value.

Optionally, the above-mentioned storage medium is also configured to store program codes for performing the following steps: performing multiple diffusions with the candidate pixel as the center, and comparing the density value corresponding to the pixel point at each diffusion position in the preset pixel set with the previous Accumulate the accumulation results of the first diffusion position to obtain the accumulation result of the diffusion position, wherein the accumulation result of the first diffusion position is calculated by comparing the density value corresponding to the pixel point on the first diffusion position with the density value corresponding to the candidate pixel point Accumulated, each diffusion position is the position of the pixel adjacent to the pixel on the previous diffusion position in the monitoring image and far away from the candidate pixel, and the first diffusion position is the pixel adjacent to the candidate pixel. position; in the case where the cumulative result of the diffusion position is greater than the first preset value, based on the candidate pixel and the pixels on all diffusion positions before the diffusion position, the first set after splitting is obtained; the first set after splitting The pixels in are deleted from the preset pixel set to obtain the second set after splitting.

Optionally, the above-mentioned storage medium is also configured to store program codes for performing the following steps: each first set of pixels adopts a tree structure, and performs a split operation on the preset set of pixels based on the candidate pixels to obtain the split first set of pixels. After the first set and the second set, the quality of the node corresponding to the candidate pixel is set to a third preset value.

Optionally, the above-mentioned storage medium is also configured to store program codes for performing the following steps: sort the multiple second pixel sets according to the quality corresponding to the multiple second pixel sets to obtain the sorting result; obtain the preset Set the number of second pixel sets to obtain the target pixel set, wherein the preset number is the number of target objects; determine the pixel point corresponding to the maximum density value from the pixel points in the target pixel set as the target pixel point.

Optionally, in this embodiment, the storage medium is configured to store program codes for performing the following steps: monitoring the active area through monitoring equipment to obtain a monitoring image, wherein the monitoring image contains target groups of people; performing density estimation on the monitoring image, Obtain the density estimation result of the target population, wherein the multiple density values contained in the density estimation result are used to represent the probability of the target population being present in multiple pixels in the monitoring image; based on the density estimation result, determine the target pixel from the multiple pixels , where there is a target crowd in the target pixel; based on the position of the target pixel in the monitoring image, the target positioning result of the target crowd in the active area is obtained.

Optionally, in this embodiment, the storage medium is configured to store program codes for performing the following steps: responding to an input instruction acting on the operation interface, displaying a monitoring image of the area to be monitored on the operation interface, wherein the monitoring The image contains the target object; in response to the positioning instruction acting on the operation interface, the target positioning result of the target object in the area to be monitored is displayed on the operation interface, wherein the target positioning result is determined by the target from multiple pixels in the monitoring image The position of the pixel point in the monitoring image is determined, and the target pixel point is determined based on the density estimation result of the target object. The density estimation result is obtained by performing density estimation on the monitoring image, and the multiple density values contained in the density estimation result are used to represent multiple pixel points The probability that the target object exists.

Optionally, in this embodiment, the storage medium is set to store program codes for performing the following steps: monitor the monitoring image of the area to be monitored through the monitoring device, wherein the monitoring image contains the target object; in the virtual reality VR device or The monitoring image is displayed on the presentation screen of the augmented reality AR device; the density estimation of the monitoring image is performed to obtain the density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used to represent the presence of targets in multiple pixel points in the monitoring image The probability of the object; based on the density estimation result, determine the target pixel point from multiple pixels, wherein the target pixel point has the target object; based on the position of the target pixel point in the monitoring image, the target object in the area to be monitored is obtained Positioning results; drive VR devices or AR devices to render and display target positioning results.

Optionally, in this embodiment, the storage medium is configured to store program codes for performing the following steps: acquiring a monitoring image of the area to be monitored by calling a first interface, wherein the first interface includes a first parameter, the first The parameter value of the parameter is the monitoring image, and the monitoring image contains the target object; the density estimation is performed on the monitoring image to obtain the density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used to represent multiple pixel points in the monitoring image The probability of the existence of the target object; based on the density estimation result, determine the target pixel point from multiple pixel points, wherein the target pixel point has the target object; based on the position of the target pixel point in the monitoring image, obtain the target object in the area to be monitored the target positioning result; output the target positioning result by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target positioning result.

Using the embodiment of the present application, firstly, the monitoring image of the area to be monitored, wherein the monitoring image contains the target object; density estimation is performed on the monitoring image to obtain the density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used for Characterize the probability of the target object existing in multiple pixels in the monitoring image; based on the density estimation result, determine the target pixel point from multiple pixel points, wherein, the target pixel point has the target object; based on the position of the target pixel point in the monitoring image, The target positioning result of the target object in the area to be monitored is obtained, and the positioning of the object in the monitoring image is realized. It is easy to notice that the density estimation of the monitoring image can be performed to obtain the density estimation result of the target object, and the pixel points with the target object can be determined from multiple pixel points based on the density estimation result, avoiding the detection of pixels that do not contain the target object. In order to improve the accuracy of the positioning result of the target object, by obtaining the positioning result of the target object, the effect of increasing the output information of the target object can be achieved, thereby solving the problem that the algorithm of the related technology is difficult to output more accurate information of the target object in the monitoring image. Multi-information technical issues.

The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments.

In the above-mentioned embodiments of the present application, the descriptions of each embodiment have their own emphases, and for parts not described in detail in a certain embodiment, reference may be made to relevant descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed technical content can be realized in other ways. Wherein, the device embodiments described above are only illustrative, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or can be Integrate into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of units or modules may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions for enabling a computer device (which may be a personal computer, server or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes. .

The above description is only the preferred embodiment of the present application. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present application, some improvements and modifications can also be made. These improvements and modifications are also It should be regarded as the protection scope of this application.

Claims (14)

1. An image processing method, characterized in that, comprising: Acquiring a monitoring image of the area to be monitored, wherein the monitoring image includes a target object; performing density estimation on the monitoring image to obtain a density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used to represent the existence of the target object at multiple pixel points in the monitoring image probability; determining a target pixel point from the plurality of pixel points based on the density estimation result, wherein the target pixel point has the target object; Based on the position of the target pixel in the monitoring image, a target positioning result of the target object in the area to be monitored is obtained. 2. The method according to claim 1, wherein, based on the density estimation result, determining the target pixel from the plurality of pixels comprises: Based on the density estimation result, divide the plurality of pixel points into a plurality of first pixel sets, wherein different objects exist in the pixel points in different first pixel sets; Aggregating the density values corresponding to the pixel points in each first pixel set to obtain the quality corresponding to each first pixel set; Based on the qualities corresponding to the plurality of first pixel sets, the preset pixel sets in the plurality of first pixel sets are split to obtain a plurality of second pixel sets, wherein the quality corresponding to the preset pixel sets is greater than the first preset quality; Based on the quality corresponding to the multiple second pixel sets, determine the target pixel point from the multiple second pixel sets. 3. The method according to claim 2, wherein, based on the density estimation result, dividing the plurality of pixel points into a plurality of first pixel sets comprises: traverse each pixel in the monitoring image, and determine whether the density value corresponding to the pixel in the density estimation result is the maximum density value among the density values corresponding to all pixels in the preset area, wherein the preset The area is used to characterize the area composed of pixels adjacent to the pixel with the pixel as the center in the monitoring image; In the case that the density value corresponding to the pixel point is the maximum density value, a new set is established, and the pixel point is stored in the new set, wherein the new set is used to generate the plurality of first pixels gather; If the density value corresponding to the pixel point is not the maximum density value, the pixel point is stored in a set in which the maximum density value is stored among the plurality of first pixel sets. 4. The method according to claim 2, wherein each first pixel set adopts a tree structure, and the pixels in each first pixel set correspond to nodes in the tree structure , the adjacency relationship between pixels in each first pixel set corresponds to the connection relationship between nodes in the tree structure, wherein the density value corresponding to the pixel point in each first pixel set is performed Aggregate to obtain the quality corresponding to each first pixel set, including: determining a density value corresponding to a parent node in the tree structure and a density value corresponding to a child node, wherein the child node is a descendant node of the parent node in the tree structure; Aggregating the density value corresponding to the parent node and the density value corresponding to the child node to obtain the quality of the parent node; Determine the quality of the parent node in the tree structure as the quality corresponding to each first pixel set. 5. The method according to claim 2, characterized in that, based on the quality corresponding to the plurality of first pixel sets, the preset pixel set in the plurality of first pixel sets is split to obtain a plurality of second Pixel collection, including: Step A, obtaining the radiation range of the preset pixel set, wherein the radiation range is used to characterize the coordinate range of all pixel points in the preset pixel set; Step B, determining the pixel point corresponding to the maximum density value within the radiation range to obtain a candidate pixel; Step C, performing a split operation on the preset pixel set based on the candidate pixels to obtain a split first set and a second set, wherein the split first set includes the candidate pixels and the The candidate pixels are associated pixels having an association relationship, and the second set after splitting includes other pixel points in the preset pixel set except the pixels in the first set after splitting; Step D, when the quality corresponding to the split second set is greater than a second preset value, update the radiation range to the coordinate range of all pixels in the split second set, and execute repeatedly Steps B and C described above, until the quality corresponding to the second set after splitting is less than or equal to the second preset value, to obtain the plurality of second pixel sets, wherein the second preset value is less than the first preset value. 6. The method according to claim 5, characterized in that, performing a split operation on the preset pixel set based on the candidate pixels to obtain a split first set and a second set, comprising: performing multiple diffusions with the candidate pixel as the center, accumulating the density value corresponding to the pixel point at each diffusion position in the preset pixel set and the cumulative result of the previous diffusion position to obtain the cumulative result of the diffusion position, Wherein, the accumulation result of the first diffusion position is obtained by accumulating the density value corresponding to the pixel point at the first diffusion position and the density value corresponding to the candidate pixel point, and each diffusion position is The position of the pixel adjacent to the pixel on the previous diffusion position and away from the candidate pixel, the first diffusion position is the position of the pixel adjacent to the candidate pixel; In the case where the cumulative result of the diffusion position is greater than the first preset value, based on the candidate pixel and pixels at all diffusion positions before the diffusion position, the first set after splitting is obtained; The pixel points in the first split set are deleted from the preset pixel set to obtain the second split set. 7. The method according to claim 5, wherein each first pixel set adopts a tree structure, and the preset pixel set is split based on the candidate pixels to obtain the split first pixel set After the first set and the second set, the quality of the node corresponding to the candidate pixel is set to a third preset value. 8. The method according to claim 2, wherein, based on the quality corresponding to the plurality of second pixel sets, determining the target pixel point from the plurality of second pixel sets comprises: sorting the multiple second pixel sets according to the quality corresponding to the multiple second pixel sets, to obtain a sorting result; Acquiring a preset number of second pixel sets in the sorting result to obtain a target pixel set, wherein the preset number is the number of the target object; Determine the pixel point corresponding to the maximum density value from the pixel points in the target pixel set as the target pixel point. 9. An image processing method, characterized in that, comprising: Obtaining a monitoring image by monitoring an active area with a monitoring device, wherein the monitoring image includes a target group of people; performing density estimation on the monitoring image to obtain a density estimation result of the target population, wherein the multiple density values contained in the density estimation result are used to represent the presence of the target population in multiple pixel points in the monitoring image probability; Determining a target pixel point from the plurality of pixel points based on the density estimation result, wherein the target population exists in the target pixel point; Based on the position of the target pixel in the monitoring image, a target positioning result of the target crowd in the activity area is obtained. 10. An image processing method, characterized in that, comprising: Responding to an input instruction acting on the operation interface, displaying a monitoring image of the area to be monitored on the operation interface, wherein the monitoring image includes a target object; In response to a positioning instruction acting on the operation interface, a target positioning result of the target object in the area to be monitored is displayed on the operation interface, wherein the target positioning result is obtained by multiple The position of the target pixel determined among the pixels in the monitoring image is determined, the target pixel is determined based on the density estimation result of the target object, and the density estimation result is obtained by performing density estimation on the monitoring image , the multiple density values contained in the density estimation result are used to represent the probability that the target object exists in the multiple pixel points. 11. An image processing method, characterized in that, comprising: monitoring a monitoring image of the area to be monitored by monitoring equipment, wherein the monitoring image includes a target object; displaying the monitoring image on a presentation screen of a virtual reality VR device or an augmented reality AR device; performing density estimation on the monitoring image to obtain a density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used to represent the existence of the target object at multiple pixel points in the monitoring image probability; determining a target pixel point from the plurality of pixel points based on the density estimation result, wherein the target pixel point has the target object; Obtaining a target positioning result of the target object in the area to be monitored based on the position of the target pixel in the monitoring image; Driving the VR device or the AR device to render and display the target positioning result. 12. An image processing method, characterized in that, comprising: Obtaining a monitoring image of an area to be monitored by calling a first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter is the monitoring image, and the monitoring image includes a target object; performing density estimation on the monitoring image to obtain a density estimation result of the target object, wherein the multiple density values contained in the density estimation result are used to represent the existence of the target object at multiple pixel points in the monitoring image probability; determining a target pixel point from the plurality of pixel points based on the density estimation result, wherein the target pixel point has the target object; Obtaining a target positioning result of the target object in the area to be monitored based on the position of the target pixel in the monitoring image; Outputting the target positioning result by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter is the target positioning result. 13. A computer-readable storage medium, characterized in that the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute claims 1 to 12 The image processing method described in any one. 14. An electronic device, characterized in that it comprises: processor; A memory, connected to the processor, for providing the processor with instructions for processing the image processing method described in any one of claims 1-12.

Images