CN112785567B - Map detection method, device, electronic equipment and storage medium - Google Patents

Abstract

The disclosure discloses a map detection method, relates to the field of artificial intelligence, and particularly relates to the field of image processing. The specific implementation scheme is as follows: extracting a plurality of first images from the video and determining the positions of the plurality of first images in the video; determining a first image including a target map from the plurality of first images as a second image using the map classification model; determining a second image containing the erroneous target map as a target image using the map detection model; the position of the target image in the video is determined as the position of the wrong target map. The disclosure also discloses a map detection device, an electronic device and a storage medium.

Description

Map detection method, device, electronic equipment and storage medium

Technical field

The present disclosure relates to the field of artificial intelligence technology, and in particular to image processing technology. More specifically, the present disclosure provides a map detection method, device, electronic device and storage medium.

Background technique

With the rapid development of the Internet, users publish videos and images on the Internet every day. While people enjoy the convenience of obtaining information, they are also misled by wrong information on the Internet at any time, causing bad effects.

For example, in recent years, there have been frequent incidents of problematic maps on the Internet, which have had a negative impact on society. Therefore, auditing the compliance of maps in videos and images is an essential task.

Contents of the invention

The present disclosure provides a map detection method, device, equipment and storage medium.

According to an aspect of the present disclosure, a map detection method is provided, including: extracting a plurality of first images from a video and determining the positions of the plurality of first images in the video; using a map classification model to extract a plurality of first images from the plurality of first images. Determine the first image containing the target map as the second image; use the map detection model to determine the second image containing the wrong target map as the target image; determine the position of the target image in the video as the position of the wrong target map.

According to another aspect of the present disclosure, a map detection device is provided, including: an extraction module for extracting a plurality of first images from a video and determining the positions of the plurality of first images in the video; a first determination module, Used to use the map classification model to determine the first image containing the target map from the plurality of first images as the second image; the second determination module used to use the map detection model to determine the second image containing the wrong target map as the target image; The third determination module is used to determine the position of the target image in the video as the position of the wrong target map.

According to another aspect of the present disclosure, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are At least one processor executes to enable at least one processor to execute the method provided in accordance with the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute a method provided according to the present disclosure.

According to another aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a method provided in accordance with the present disclosure.

It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.

Description of the drawings

The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. in:

Figure 1 is a schematic diagram of an exemplary system architecture to which map inspection methods and devices can be applied according to an embodiment of the present disclosure;

Figure 2 is a flow chart of a map detection method according to one embodiment of the present disclosure;

Figure 3 is a schematic network structure diagram of a map classification model according to an embodiment of the present disclosure;

Figure 4 is a schematic flowchart of a method for using a map detection model to determine a second image containing an incorrect target map as a target image according to one embodiment of the present disclosure;

Figure 5 is a schematic diagram of a method of using a map detection model to calibrate multiple feature areas in a target map in a second image according to an embodiment of the present disclosure;

Figure 6 is a schematic flowchart of a method for determining whether the target map in the second image is an incorrect target map based on the calibrated feature area according to an embodiment of the present disclosure;

Figure 7 is a schematic network structure diagram of a map detection model according to an embodiment of the present disclosure;

Figure 8 is a block diagram of a map detection device according to an embodiment of the present disclosure;

FIG. 9 is a block diagram of an electronic device according to a map detection method according to one embodiment of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the present disclosure are included to facilitate understanding and should be considered to be exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

In recent years, some companies and organizations have used non-compliant maps in public, which has had a negative impact on the public. Management departments have announced and punished many companies that did not use complete and accurate maps. There are many Internet enterprise users and high social attention. If problematic maps are spread on the Internet, the negative impact will be more obvious. Therefore, how to identify non-compliant maps in massive images has become a practical problem that needs to be solved urgently. The technology to identify problem maps can be applied in picture and video websites, video software, security and other fields. Identifying problem maps and taking measures to avoid their spread on the Internet is an important part of reducing product security risks and ensuring product safety.

Problem maps are generally erroneous maps with incomplete boundary lines, missing key areas or islands, etc. Maps that can be detected by embodiments of the present disclosure may include national-level maps, administrative region-level maps, world maps, Asian maps, and Europe map etc.

Currently, the review of maps in videos often relies on manual labor, which is time-consuming and easy to miss. Specifically, because erroneous maps account for a small proportion of the massive map images, the attention paid to manual review may decrease as the standard time increases, and the annotation results will also vary depending on the reviewer and review environment. Subjective factors interfere seriously. In addition, manual review first requires the reviewers to be trained on standard maps, and then the reviewers complete the identification and processing of all images and videos, which consumes a lot of manpower and material resources, resulting in great pressure on labor costs.

Figure 1 is a schematic diagram of an exemplary system architecture to which a map detection method and device can be applied according to an embodiment of the present disclosure. It should be noted that Figure 1 is only an example of a system architecture to which embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiments of the present disclosure cannot be used in other applications. Device, system, environment or scenario.

As shown in FIG. 1 , the system architecture 100 according to this embodiment may include multiple terminal devices 101 , a network 102 , a server 103 and a server 104 . The network 102 is a medium used to provide a communication link between the terminal device 101 and the servers 103 and 104. Network 102 may include various connection types, such as wired and/or wireless communication links, and the like.

The terminal device 101 may be various electronic devices, including but not limited to smartphones, tablet computers, laptop computers, and the like. The server 103 may be an electronic device that provides map classification services, and the server 104 may be an electronic device that provides problem map detection services.

For example, the server 103 can collect a large number of positive target map samples and negative target map samples, where the positive target map samples can be correctly drawn target maps, and the negative target map samples can be non-target maps. Using target map positive samples and target map negative samples to train the neural network model, a map classification model can be obtained. The map classification model can detect whether the map to be classified is a target map or a non-target map.

For example, the server 104 can collect a large number of target map samples, use the target map samples to train a neural network model, and obtain a map detection model. The map detection model can detect whether there are incorrectly drawn boundary lines in the map to be reviewed. Missing areas and other issues.

According to embodiments of the present disclosure, the trained map classification model and map detection model can classify and detect images existing on the Internet or to be published, and can also classify and detect videos existing on the Internet or to be published. For example, the user publishes a video through the terminal device 101, and the terminal device 101 sends the video to the server 103. The server 103 can cut frames in the video, extract each frame of image in the video, and then identify whether there is enough content on each frame of the image. Containing a map, for the image containing the map, use the trained map classification model to identify whether the target map is included. For the identified image containing the target map, the frame position of each image can be marked. The identified image and frame location containing the target map are then sent to the server 104 .

The server 104 can use the map detection model to detect each image containing the target map, and identify whether the target map has problems such as misdrawing or missing drawings. If so, the server 104 will send the frame position information of the image to the terminal device 101 as a prompt. The wrong map that appears at a certain frame position in the user's video achieves the purpose of automatically reviewing the wrong map. Specifically, using the map detection model can also mark the specific drawing errors in the wrong map, such as marking missing areas, etc.

It should be noted that the map classification model and the map detection model can be trained and used in different servers, or can be trained and used in the same server, and this disclosure does not limit this.

Figure 2 is a flow chart of a map detection method according to one embodiment of the present disclosure.

As shown in Figure 2, the map detection method 200 may include operations S210 to S240.

In operation S210, a plurality of first images are extracted from the video and positions of the plurality of first images in the video are determined.

According to embodiments of the present disclosure, the video can be cut into frames, and multiple frames of images in the video can be extracted as the first image. Each frame of the first image can be 1 second. The frame position of each first image can also be marked as the position of the first image in the video.

In operation S220, a first image containing a target map is determined as a second image from a plurality of first images using a map classification model.

According to embodiments of the present disclosure, the map classification model may be trained using a large number of target map positive samples and target map negative samples. The trained map classification model can be used to identify whether each first image contains the target map, and the first image containing the target map can be used as the second image, and each second image can also be determined based on the frame position of each first image. frame position.

Specifically, a plurality of first images can be input into a trained map classification model, and the map classification model can output a probability that each first map contains a target map, if the output first image contains a probability that the target map is included. is greater than the first threshold (for example, 0.3), then the first image can be determined to be the first image containing the target map.

It can be understood that if the number of first images containing the target map is greater than or equal to 1, it is determined that the video contains the target map, and the video needs to be reviewed.

In operation S230, the second image containing the wrong target map is determined as the target image using the map detection model.

According to embodiments of the present disclosure, the map detection model may be trained using a large number of target map samples, and the map detection model may be used to identify whether the target map in the second image is an incorrect map. In addition, the second image containing the wrong target map is used as the target image, and the frame position of each target image can also be determined based on the frame position of each second image.

Specifically, multiple second images can be input into a trained map detection model, and the map detection model can calibrate multiple feature areas in each second image, and each feature area can correspond to a target map that is prone to drawing errors or Missing areas or islands, etc. The map detection model can output a score for each calibrated feature area, which represents the probability that the feature area is drawn correctly.

According to an embodiment of the present disclosure, if the number of feature areas in the calibrated target map is the same as the number of feature areas in the standard target map, and the score of each feature area is greater than a certain threshold (for example, 0.3), then it means The target map is drawn correctly. For example, if the map detection model is used to demarcate area A, area B, area C and area D of the target map in the second image, and the output score of area A is 0.4, the score of area B is 0.5, the area C has a score of 0.6 and area D has a score of 0.9. And if the standard target map also includes area A, area B, area C, and area D, it means that the target map in the second image is drawn correctly.

According to an embodiment of the present disclosure, if the number of feature areas in the calibrated target map is inconsistent with the number of feature areas in the standard target map, for example, is less than the number of feature areas in the standard target map, the target map can be determined A feature area is missing. For example, if a map detection model is used, for example, area A, area B, and area C of the target map are calibrated in the second image, and the output score of area A is 0.4, the score of area B is 0.5, and the score of area C is is 0.6. However, the standard target map includes area A, area B, area C and area D, which means that the target map in the second image omits area D.

According to an embodiment of the present disclosure, if the number of feature areas in the calibrated target map is consistent with the number of feature areas in the standard target map, but the score of a certain feature area is less than a threshold (for example, 0.3), then it means The calibrated characteristic area has a high probability of being drawn incorrectly, and the target map can be considered to be an incorrect map. For example, if the map detection model is used to demarcate area A, area B, area C and area D of the target map in the second image, and the output score of area A is 0.1, the score of area B is 0.5, the area C has a score of 0.6 and area D has a score of 0.9. And the standard target map also includes area A, area B, area C and area D, which means that the target map in the second image does not miss the characteristic area, but the probability of the boundary line of area A being drawn correctly is small, so it can be considered The target map in the second image is the wrong map.

In operation S240, the position of the target image in the video is determined as the position of the wrong target map.

According to an embodiment of the present disclosure, in operation S210, the frame position of each extracted first image may be marked, for example, as the 1st to 100th frames. In operation S220, the first image containing the target map is identified as the second image using the map classification model. The frame position of the second image may be determined according to the marked frame position of the first image, for example, the 5th to 10th frames and the 75th frame. The first image to frame 78 includes the target map, then the frame positions of the second image are frames 5 to 10 and frames 75 to 78. In operation S230, the map detection model is used to identify the second image containing the wrong target map as the target image. The frame position of the target map can be determined based on the frame position of the second image, for example, in the second image of frames 5 to 10. The target map is the correct target map, and the target map in the second image from frames 75 to 78 is the wrong target map, then the second image from frames 75 to 78 is the target image, and the frame position of the target image is from 75 to 78 frame. Then the reviewer can be notified that frames 75 to 78 in the video contain the wrong target map, improving map review efficiency and saving manpower.

According to an embodiment of the present disclosure, an image is extracted from a video, a map classification model is used to determine an image containing a target map, a map detection model is used to detect an image containing an erroneous target map, and a location in the video of an image containing an erroneous target map is determined, It can automatically detect whether the video contains an incorrect target map and determine the location of the incorrect target map, thereby improving the review efficiency of the target map, reducing labor cost pressure, and improving the accuracy of the review.

According to an embodiment of the present disclosure, before using the map classification model to classify the plurality of first images, the first images may also be scaled to a first preset size. The first preset size may be 224*224. The first The format of the image is RGB image. The map classification model classifies the first image with a size of 224*224, and obtains the probability that each first image contains the target map. Depending on the performance of the map classification model, the first image can also be scaled to other sizes.

According to embodiments of the present disclosure, when training a map classification model, target map positive samples and target map negative samples of different sizes can be used for training, so that the map classification model undergoes training of multi-size features, which can improve the robustness of the map classification model. Great sex.

According to embodiments of the present disclosure, when training a map classification model, a multi-scale feature pyramid can also be used for training. Multi-scale feature pyramid means that the neural network includes multiple convolutional layers, and the image is processed by different convolutional layers. The feature maps presented later have different sizes. After the map classification model is trained with a multi-scale feature pyramid, the robustness of the model can be improved.

Figure 3 is a schematic network structure diagram of a map classification model according to an embodiment of the present disclosure.

As shown in Figure 3, the network structure of the map classification model includes multiple convolutional layers 301, multiple maximum pooling layers 302, fully connected layers 303 and classification layers 304. Each processing layer of the map classification model will be described below using multiple first images in the video as input images of the map classification model.

As shown in Figure 3, the convolution layer 301 is used to extract the features of the input image. The output of each convolution layer 301 is connected to a maximum pooling layer 302. The maximum pooling layer 302 is used to output the convolution layer 301. Features are selected and the features with the greatest activation degree are selected. After each convolutional layer 301 performs feature extraction on the map sample, it will go through a maximum pooling layer 302 to select the features with the highest degree of activation. After being processed by multiple convolutional base layers 301 and maximum pooling layers 302, the fully connected layer 303 will integrate the features selected by the maximum pooling layer 302, and output the integrated features to the classification layer 304. The classification layer 304 is used to output classification results that conform to these features. Among them, the size of the feature image output after each processing by the convolutional base layer 301 and the maximum pooling layer 302 is different, which allows the trained model to adaptively process images of different sizes and improves the robustness of the map classification model.

4 is a flowchart of a method of using a map detection model to determine a second image containing an incorrect target map as a target image, according to one embodiment of the present disclosure.

As shown in Figure 4, the map detection method 400 may include operations S431 to S433.

In operation S431, a map detection model is used to demarcate a plurality of feature areas in the target map in each second image.

According to embodiments of the present disclosure, the plurality of characteristic areas may include a first area and at least one second area. For example, the target map can be a national map. The first area in the target map can be a rectangular area with the southernmost boundary point and the easternmost boundary point of the national border as vertices. The second area can be multiple identical areas in the national map. Or administrative regions at different levels, such as province A, province B, and city C.

Specifically, the rightmost point and the bottom point can be selected on the boundary of the target map, and an area of a preset shape is determined as the first area based on these two points. The two points are on the edge of the determined preset shape. superior. The shape may be a regular shape such as a rectangle or a circle, or an irregular shape. The first area is calibrated to further determine that the map in the second image is the target map to be reviewed based on the calibrated first area. Since the first area is determined by two points on the boundary of the map in the second image, the first area contains at least part of the map, and the second image can be further determined based on the characteristics of at least part of the map included in the first area. Whether the map in is a target map to be reviewed.

According to an embodiment of the present disclosure, the second area corresponds to a geographical area in the target map that meets preset conditions. For example, the location and outline of geographical area A in the standard target map are used as the first preset condition, and the geographical area B is in the standard target map. The location and outline in the map serve as the second preset condition. The geographical area A is marked on the target map according to the first preset condition, and the geographical area B is marked on the target map according to the second preset condition. The marked geographical area A and geographical area B are regarded as the second area. By calibrating the second area, it can be determined whether the target map misses certain geographical areas and whether the boundary lines are wrong. If the boundary lines are wrong, the second area may not be calibrated or may be calibrated incorrectly.

In operation S432, it is determined whether the target map in each second image is an incorrect target map according to the calibrated feature area.

According to an embodiment of the present disclosure, whether the target map in the second image is an incorrect map may be determined according to the number of calibrated second areas. Specifically, the preset standard target map may be marked with a first area and a plurality of second areas. The multiple second areas include, for example, area A, area B, area C, and area D. If the number of second areas in the target map in the calibrated second image is inconsistent with the number of second areas in the standard target map, for example, less than the number of second areas in the standard target map, the target can be determined The map misses a characteristic area. For example, if the map detection model is used to demarcate the first area of the target map and area A, area B, and area C in the second area in the second image, since the second area includes area A in the standard target map , area B, area C and area D, it means that the target map in the second image misses area D, which means that the target map in the second image is an incorrect map.

According to an embodiment of the present disclosure, while the map detection model calibrates the first region and at least one second region in each second image, it can also output a score for each calibrated region, which score represents the accuracy of drawing the feature region. The probability. Then it can be determined according to the score of the calibrated area whether the target map in the second image is an incorrect map.

Specifically, if the map detection model is used to calibrate the first area of the target map in the second image, as well as area A, area B, area C and area D in the second area, and the score of the output area A is 0.1, Area B has a score of 0.5, Area C has a score of 0.6, and Area D has a score of 0.9. Since the second area in the standard target map includes area A, area B, area C and area D, it means that the target map in the second image does not miss the second area, but the probability that the boundary line of area A is drawn correctly is small. , it can be considered that the target map in the second image is an error map.

In operation S433, the second image containing the wrong target map is used as the target image.

According to an embodiment of the present disclosure, the second image containing the wrong target map is used as the target image, and the frame position of the target map is determined to remind the reviewer of the position where the wrong target map appears in the video.

FIG. 5 is a schematic diagram of a method of using a map detection model to calibrate multiple feature areas in a target map in a second image according to an embodiment of the present disclosure.

As shown in Figure 5, the map detection model has calibrated five characteristic areas in the target map 500 in the second image, namely the first area 501, the second area 502, the second area 503, the second area 504 and the second area. Second area 505. The map detection model can also score the correctness of each feature area, and the correctness score of each area can represent the probability that the area is drawn correctly.

As shown in Figure 5, the first area 501 can be a rectangular frame determined based on the rightmost point and the bottom point on the boundary of the target map 500, while the second areas 502 to 505 can be based on designated areas in the target map. A rectangular box whose boundaries are determined.

FIG. 6 is a flowchart of a method for determining whether the target map in the second image is an incorrect target map according to the calibrated feature area according to an embodiment of the present disclosure.

As shown in Figure 6, the method includes operations S6321 to S6327.

In operation S6321, a map detection model is used to calibrate a first region and at least one second region in the target map in the second image, and score the correctness of the first region and each second region.

In operation S6322, it is determined whether the score of the calibrated first area is greater than a second threshold (such as 0.96). If the score of the first area is greater than 0.96, step S6323 is executed. Otherwise, it can be determined that the map in the second image is not the target. map, the process ends.

In operation S6323, it is determined whether the scores of at least one second area are all greater than a third threshold (for example, 0.3). If so, it can be determined that the target map in the second image is the correct target map, and the process ends. Otherwise, any first If the score of the second region is not greater than the third threshold, operation S6324 is performed.

In operation S6324, an area ratio of the first area in the second image and the second image is calculated.

In operation S6325, it is determined whether the area ratio is greater than a fourth threshold (for example, 0.5). If so, operation S6326 is performed. Otherwise, operation S6327 is performed.

In operation S6326, it is determined that the target map in the second image is the correct target map.

In operation S6327, the target map is intercepted from the second image and scaled to a preset size (such as 660*400), and operation S6321 is returned.

It can be understood that if the target map accounts for a small area in the second image, the area of the feature area in the second image will be smaller, which may affect the calibration of each feature area in the second image by the map detection model. Therefore, in the embodiment of the present disclosure, when it is determined that the score of any calibrated second area is less than the third threshold (that is, the probability of an error in drawing the target map is relatively high), the target map is intercepted from the second image. Specifically, It may be that the rectangular frame of the first area is enlarged to frame the entire target map, and then the target map is intercepted according to the enlarged rectangular frame of the first area. For the intercepted second map, the second map can be scaled to a second preset size, which can be 660*400, and the target image of the second preset size can be input into the map detection model. The detection model can calibrate and score the second area in the target map, and based on the calibrated score of the second area, it can be further determined whether the target map is an incorrect target map. This can avoid the problem of detection errors caused by the target map being too small and the characteristic area being difficult to identify, and can improve the accuracy of detecting the wrong target map.

According to an embodiment of the present disclosure, before using the map detection model to demarcate the plurality of feature areas in the target map in each second image, the second image may also be scaled to a second preset size, and the first preset size may be It is 660*400, and the format of the second image is RGB image. The map detection model calibrates and scores the first area and at least one second area in the second image with a size of 660*400, and determines whether the target map is an error based on the scores of the calibrated first area and at least one second area. map. Depending on the performance of the map detection model, the second image can also be scaled to other sizes.

According to embodiments of the present disclosure, when training a map detection model, target map samples of different sizes can be used for training, so that the map detection model undergoes training of multi-size features, which can improve the robustness of the map classification model.

According to an embodiment of the present disclosure, the target map sample may be marked with a first area and at least one second area. Specifically, the first area and at least one second area may be marked by a rectangular frame.

Figure 7 is a schematic network structure diagram of a map detection model according to an embodiment of the present disclosure.

As shown in Figure 7, the network structure of the map detection model includes an image feature extraction layer 701, a feature area calibration layer 702, and an error area identification layer 703. Each processing layer of the map detection model will be described below using the second image containing the target map as the input image of the map detection model.

According to embodiments of the present disclosure, the image feature extraction layer 701 may include multiple convolutional layers and maximum pooling layers, used to extract features of the input image, select features with the largest activation degree, and generate feature images. The image feature extraction layer 701 outputs the feature image to the feature area calibration layer 702 and the error area identification layer 703 respectively. The feature area calibration layer 702 is used to calibrate multiple feature areas in the feature image to obtain the calibration information of each feature area, and Score each feature area, where the calibration information may be the coordinate information of a rectangular box used to mark the feature area. The feature area calibration layer 702 outputs the calibration information and score of each feature area to the error area identification layer 703. The error area identification layer 703 is used to combine the feature image input by the image feature extraction layer 701 and each feature input by the feature area calibration layer 702. Calibration information and scores of the region, calibrate each feature area on the feature image, and determine whether each feature area is drawn incorrectly based on the calibrated feature area and the score of each feature area.

Figure 8 is a block diagram of a map detection device according to one embodiment of the present disclosure.

As shown in FIG. 8 , the map detection device 800 may include an extraction module 801 , a first determination module 802 , a second determination module 803 and a third determination module 804 .

The extraction module 801 is used to extract a plurality of first images from the video and determine the positions of the plurality of first images in the video.

The first determining module 802 is configured to use a map classification model to determine a first image containing a target map as a second image from the plurality of first images.

The second determination module 803 is configured to use the map detection model to determine the second image containing the wrong target map as the target image.

The third determination module 804 is used to determine the position of the target image in the video as the position of the wrong target map.

According to an embodiment of the present disclosure, the extraction module 801 is specifically configured to segment the video by frames, obtain a plurality of first images, and determine the frame position of each first image in the video as the position of the first image in the video.

According to an embodiment of the present disclosure, the first determination module 802 is specifically configured to use a map classification model to classify multiple first images to obtain the probability that each first image contains the target map; determine the first image with a probability greater than the first threshold as Second image.

According to an embodiment of the present disclosure, the map detection device 800 further includes a first zoom module.

The first scaling module is configured to scale the first images to a first preset size before the first determining module 802 uses the map classification model to classify the plurality of first images.

According to an embodiment of the present disclosure, the second determination module 803 includes a calibration unit, a first determination unit and a second determination unit.

The calibration unit is configured to use the map detection model to calibrate a plurality of feature areas in the target map in each second image.

The first determination unit is used to determine whether the target map in each second image is an incorrect target map according to the calibrated feature area.

The second determination unit is configured to use the second image containing the wrong target map as the target image.

According to an embodiment of the present disclosure, the first determination unit is specifically configured to determine that the target map in the second image is when the number of calibrated feature areas is not equal to the number of feature areas in the preset standard target map. Wrong target map.

According to an embodiment of the present disclosure, the first determination unit is specifically configured to determine missing feature areas in the wrong target map based on the feature areas in the preset standard target map.

According to an embodiment of the present disclosure, the plurality of characteristic areas include a first area and at least one second area, and the calibration unit is specifically configured to calibrate the first area in the target map based on at least two points on the boundary of the target map, and in the target map Successfully select at least one geographical area that meets the preset conditions as the second area.

According to an embodiment of the present disclosure, the first determining unit includes: a scoring subunit, a calculating subunit, and a determining subunit.

The scoring unit is used to score the correctness of the calibrated first region and the second region.

The calculation subunit is used to calculate the difference between the first area and the second area in the second image when it is determined that the correctness score of the first area is greater than the second threshold and the correctness score of any second area is less than or equal to the third threshold. The area ratio of the image.

The determination subunit is used to determine whether the target map in the second image is an incorrect target map according to the area ratio.

According to an embodiment of the present disclosure, the determination subunit is specifically configured to determine that the target map in the second image is an incorrect target map when it is determined that the area ratio is greater than the fourth threshold.

According to an embodiment of the present disclosure, the determination subunit is specifically configured to intercept the target map from the second image when it is determined that the area ratio is less than or equal to the fourth threshold; and return to the calibration unit.

According to an embodiment of the present disclosure, the map detection device 800 further includes a second zoom module.

The second scaling module is configured to scale the second image to a second preset size before the calibration unit uses the map detection model to calibrate a plurality of feature areas in the target map in each second image.

According to an embodiment of the present disclosure, the determination subunit is specifically further configured to determine an erroneous second area in the target map based on the correctness score of each second area in the second image.

According to an embodiment of the present disclosure, the determining subunit is specifically configured to determine the second image when it is determined that the correctness score of the first region is greater than the second threshold and the correctness scores of at least one second region are both greater than the third threshold. The target map in is the correct target map.

According to an embodiment of the present disclosure, the map detection device 800 further includes a first training module and a second training module.

The first training module is used to train the first neural network model using the first map sample set to obtain a map detection model; wherein the first map sample set includes a plurality of target map samples, and each target map sample has a calibrated first area and at least one calibrated second area.

The second training module is used to train the second neural network model using the second map sample set to obtain a map classification model; wherein the second map sample set includes multiple target map positive samples and multiple target map negative samples.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

Figure 9 shows a schematic block diagram of an example electronic device 900 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to refer to various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are examples only and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 9 , the device 900 includes a computing unit 901 that can execute according to a computer program stored in a read-only memory (ROM) 902 or loaded from a storage unit 908 into a random access memory (RAM) 903 Various appropriate actions and treatments. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. Computing unit 901, ROM 902 and RAM 903 are connected to each other via bus 904. An input/output (I/O) interface 905 is also connected to bus 904.

Multiple components in device 900 are connected to I/O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of displays, speakers, etc.; storage unit 908, such as magnetic disk, optical disk, etc. ; and communication unit 909, such as a network card, modem, wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information/data with other devices through computer networks such as the Internet and/or various telecommunications networks.

Computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processing processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs various methods and processes described above, such as the map detection method. For example, in some embodiments, the map detection method may be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 900 via ROM 902 and/or communication unit 909 . When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the map detection method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the map detection method in any other suitable manner (eg, by means of firmware).

Various implementations of the systems and techniques described above may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip implemented in a system (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or a combination thereof. These various embodiments may include implementation in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor The processor, which may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device. An output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions specified in the flowcharts and/or block diagrams/ The operation is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and may be provided in any form, including Acoustic input, voice input or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., A user's computer having a graphical user interface or web browser through which the user can interact with implementations of the systems and technologies described herein), or including such backend components, middleware components, or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communications network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

Computer systems may include clients and servers. Clients and servers are generally remote from each other and typically interact over a communications network. The relationship of client and server is created by computer programs running on corresponding computers and having a client-server relationship with each other.

It should be understood that various forms of the process shown above may be used, with steps reordered, added or deleted. For example, each step described in the present disclosure can be executed in parallel, sequentially, or in a different order. As long as the desired results of the technical solution disclosed in the present disclosure can be achieved, there is no limitation here.

The above-mentioned specific embodiments do not constitute a limitation on the scope of the present disclosure. It will be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions are possible depending on design requirements and other factors. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this disclosure shall be included in the protection scope of this disclosure.

Claims (16)

1. A map detection method, comprising:

extracting a plurality of first images from the video and determining the positions of the plurality of first images in the video;

determining a first image including a target map from the plurality of first images as a second image using a map classification model;

determining a second image containing the erroneous target map as a target image using the map detection model;

determining the position of the target image in the video as the position of an error target map;

wherein the determining, using the map detection model, the second image including the erroneous target map as the target image includes:

marking a plurality of feature areas in the target map in each second image using the map detection model;

determining whether the target map in each second image is an error target map according to the calibrated characteristic region;

Taking a second image containing an error target map as a target image;

wherein the plurality of feature regions includes a first region and at least one second region, and the marking the plurality of feature regions in the target map in each of the second images using the map detection model includes:

calibrating a first region in the target map based on at least two points on a boundary of the target map, and

marking at least one geographical area meeting preset conditions in the target map as a second area;

wherein the determining whether the target map in each second image is an error target map according to the calibrated feature area comprises:

scoring the correctness of the calibrated first area and second area;

calculating an area ratio of the first region in the second image to the second image under the condition that the correctness score of the first region is determined to be larger than a second threshold value and the correctness score of any second region is determined to be smaller than or equal to a third threshold value;

and determining whether the target map in the second image is an error target map according to the area ratio.

2. The method of claim 1, wherein the extracting the plurality of first images from the video and determining the locations of the plurality of first images in the video comprises:

Dividing the video according to frames to obtain a plurality of first images; and

the frame position of each first image in the video is determined as the position of the first image in the video.

3. The method of claim 1, wherein the determining at least one first image containing a target map from the plurality of first images as a second image using a map classification model comprises:

classifying the plurality of first images by using a map classification model to obtain the probability that each first image contains a target map;

a first image having a probability greater than a first threshold is determined as the second image.

4. The method of claim 3, further comprising, prior to classifying the plurality of first images using a map classification model:

the first image is scaled to a first preset size.

5. The method of claim 1, wherein the determining whether the target map in each second image is an erroneous target map based on the calibrated feature region comprises:

and under the condition that the number of the calibrated characteristic areas is not equal to the number of the characteristic areas in the preset standard target map, determining the target map in the second image as an error target map.

6. The method of claim 5, further comprising: and determining the missing characteristic region in the error target map according to the characteristic region in the preset standard target map.

7. The method of claim 1, wherein the determining whether the target map in the second image is an erroneous target map according to the area ratio comprises:

and determining that the target map in the second image is an error target map under the condition that the area ratio is determined to be larger than a fourth threshold value.

8. The method of claim 7, wherein the determining whether the target map in the second image is an erroneous target map according to the area ratio further comprises:

in the case that the area ratio is determined to be less than or equal to a fourth threshold value, cutting out the target map from the second image;

and enlarging the target map to a second preset size, and returning to the step of marking at least one geographic area meeting preset conditions in the target map as a second area.

9. The method of claim 8, further comprising, prior to using the map detection model to identify the plurality of feature regions in the target map in each second image:

Scaling the second image to the second preset size.

10. The method of claim 1, further comprising:

and determining the wrong second area in the target map according to the correctness score of each second area in the second image.

11. The method of claim 1, further comprising:

and determining that the target map in the second image is a correct target map under the condition that the correctness score of the first area is larger than a second threshold value and the correctness score of the at least one second area is larger than a third threshold value.

12. The method of claim 1, further comprising:

training a first neural network model by using a first map sample set to obtain the map detection model;

wherein the first set of map samples includes a plurality of target map samples, each target map sample having a calibrated first region and at least one calibrated second region.

13. The method of claim 1, further comprising:

training a second neural network model by using a second map sample set to obtain the map classification model;

wherein the second set of map samples includes a plurality of target map positive samples and a plurality of target map negative samples.

14. A map detection apparatus comprising:

an extraction module for extracting a plurality of first images from a video and determining the positions of the plurality of first images in the video;

a first determining module for determining a first image including a target map from the plurality of first images as a second image using a map classification model;

a second determining module for determining a second image containing the erroneous target map as a target image using the map detection model;

a third determining module, configured to determine a position of the target image in the video as a position of an erroneous target map;

the second determining module includes:

a calibration unit for calibrating a plurality of feature areas in the target map in each of the second images using the map detection model;

the first determining unit is used for determining whether the target map in each second image is an error target map according to the calibrated characteristic region;

a second determination unit configured to take a second image including an erroneous target map as a target image;

wherein the plurality of characteristic areas comprise a first area and at least one second area, the calibration unit is used for calibrating the first area in the target map based on at least two points on the boundary of the target map, and at least one geographic area meeting preset conditions is calibrated in the target map as the second area;

Wherein the first determining unit includes:

the scoring subunit is used for scoring the correctness of the calibrated first area and second area;

a calculating subunit, configured to calculate an area ratio of the first region in the second image to the second image when it is determined that the correctness score of the first region is greater than a second threshold and the correctness score of any second region is less than or equal to a third threshold;

and the determining subunit is used for determining whether the target map in the second image is an error target map according to the area ratio.

15. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 13.

16. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1 to 13.