CN109040604B - Shot image processing method and device, storage medium and mobile terminal - Google Patents

Abstract

The embodiment of the application discloses a processing method, a device, a storage medium and a mobile terminal for shot images, wherein the method comprises the following steps: firstly, if a moving interfering object exists in a preview picture, continuously shooting at least two image frames; then, determining the movement information of the moving interferent in the at least two image frames; and finally, determining whether the moving interfering object is reserved in the generated target shooting image or not according to the moving information of the moving interfering object. The influence of the moving interference object on the definition of the shot image can be avoided, and the shooting quality is improved.

Description

Image processing method, device, storage medium and mobile terminal

technical field

The embodiments of the present application relate to the technical field of mobile terminals, and in particular, to a method, an apparatus, a storage medium, and a mobile terminal for processing a captured image.

Background technique

At present, the photographing function has become a standard configuration of most mobile terminals, and terminal users can easily and quickly implement photographing operations through a mobile terminal that they carry with them.

When a terminal user uses a mobile terminal to take pictures, if an interfering object suddenly breaks into the lens, an artifact corresponding to the interfering object will appear in the captured image, causing the captured image to be unclear. Therefore, the image preprocessing function of the mobile terminal is still needs improvement.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a method, device, storage medium, and mobile terminal for processing a photographed image, which can improve photographing quality.

In a first aspect, an embodiment of the present application provides a method for processing a captured image, including:

If it is detected that there is a moving distraction in the preview screen, shoot at least two image frames continuously;

determining movement information of a moving distractor in the at least two image frames;

According to the movement information of the moving interference object, it is determined whether to retain the moving interference object in the generated target shot image.

In a second aspect, an embodiment of the present application provides an apparatus for processing a captured image, including:

an image capturing module, configured to continuously capture at least two image frames if it is detected that there is a moving interferer in the preview image;

an information determination module, configured to determine the movement information of the moving interferers in at least two image frames continuously captured by the image capturing module;

A reservation determining module is configured to determine whether to retain the moving distractor in the generated target photographed image according to the moving speed and/or the moving track information of the moving distractor determined by the information determining module.

In a third aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the method for processing a captured image according to the embodiment of the present application.

In a fourth aspect, an embodiment of the present application provides a mobile terminal, including a memory, a processor, and a computer program stored in the memory and executed by the processor, and the processor executes the computer program to achieve the same implementation as the present application The processing method of the captured image described in the example.

In the processing solution for capturing images provided in the embodiments of the present application, first, if it is detected that there is a moving interference object in the preview screen, at least two image frames are continuously captured; then, the movement of the moving interference object in the at least two image frames is determined. information; finally, according to the movement information of the moving interference object, it is determined whether to retain the moving interference object in the generated target shot image. It can avoid the influence of moving interference objects on the sharpness of the captured image, and improve the shooting quality.

Description of drawings

FIG. 1 is a schematic flowchart of a method for processing a captured image provided by an embodiment of the present application;

FIG. 2 is a schematic flowchart of another method for processing a captured image provided by an embodiment of the present application;

3 is a schematic flowchart of another method for processing a captured image provided by an embodiment of the present application;

4 is a schematic flowchart of another method for processing a captured image provided by an embodiment of the present application;

5 is a schematic flowchart of another method for processing a captured image provided by an embodiment of the present application;

6 is a structural block diagram of an apparatus for processing a captured image provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a mobile terminal according to an embodiment of the present application.

Detailed ways

The technical solutions of the present application will be further described below with reference to the accompanying drawings and through specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, the drawings only show some but not all the structures related to the present application.

Before discussing the exemplary embodiments in greater detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowchart depicts the steps as a sequential process, many of the steps may be performed in parallel, concurrently, or concurrently. Furthermore, the order of the steps can be rearranged. The process may be terminated when its operation is complete, but may also have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, subroutines, and the like.

At present, the photographing function has become a standard configuration of most mobile terminals, and terminal users can easily and quickly implement photographing operations through a mobile terminal that they carry with them. However, when a terminal user uses a mobile terminal to take pictures, if an interfering object suddenly breaks into the lens, an artifact corresponding to the interfering object will appear in the captured image, causing the captured image to be unclear. Therefore, the image preprocessing function of the mobile terminal Still needs improvement.

An embodiment of the present application provides a method for processing a captured image, which can continuously capture a plurality of image frames when there is a moving object in the preview screen to determine the movement information of the moving object, so as to determine whether to retain the object in the generated target captured image. Move the interference object, thereby avoiding the impact of the moving interference object on the clarity of the captured image and improving the shooting quality. The specific scheme is as follows:

FIG. 1 is a schematic flowchart of a method for processing a captured image according to an embodiment of the present application, and the method is suitable for a situation where there is a moving interferer in the photographing field of view during the photographing process. It is especially used in the case of taking a picture in the portrait shooting mode and there is a moving interference object. This method can be executed by a mobile terminal with a photographing function, and the mobile terminal can be a smart phone, a tablet computer, a wearable device (smart watch or smart glasses) ) etc., the method specifically comprises the following steps:

Step 101: If it is detected that there is a moving interference object in the preview image, at least two image frames are continuously captured.

The preview image may be an image displayed on the display screen of the mobile terminal after the shooting function is activated. The picture may be an image captured and displayed by the camera in real time after the shooting function is activated. The moving distractor can be a non-shooting target in the preview screen and an object in a moving state. For example, if the end user shoots in portrait mode, the portrait mode can locate multiple faces, but some faces are not what the user needs to shoot. For a target, such as a face that suddenly breaks into the shooting range, the face that breaks into the shooting range at this time is a moving disturbance.

Optionally, to detect whether there is a moving interferer in the preview screen, it may be to start real-time or every preset time interval (such as 1 second) after starting the shooting function of the mobile terminal to detect whether there is a mobile interferer in the preview screen, And the detection results are updated in real time; it can also be that when the end user clicks to trigger a photo (such as triggering by clicking the shooting button, voice trigger, gesture or expression trigger, etc.), the system starts to detect whether there is a moving interferer in the preview screen.

Optionally, it is detected that there is a moving interferer in the preview image, which may be if there is shooting content between at least two consecutive image frames or periodically extracted image frames (such as image frames extracted every 1 second) in the preview image. changes, it is detected that there is a moving object in the preview screen. Specifically, at least two adjacent image frames or periodically extracted image frames can be obtained for comparison. It means that there is a moving interferer in the preview screen; it can also be a difference operation between adjacent image frames or periodically extracted image frames. If the operation result is greater than the preset threshold, it means that there is a moving interferer. Because if there is no moving interference, the content between adjacent image frames or periodically extracted image frames is basically unchanged, so the result of the difference operation is small; when there is a moving interference, the content obtained by the camera is It will change due to the appearance of moving interference, and the result of the difference operation will be larger at this time.

Exemplarily, if it is detected that there is a moving interference object in the preview screen, the continuous shooting of at least two image frames may be, if it is detected that there is a moving interference object in the preview screen, the multi-frame continuous shooting mode is activated, and at least two image frames are continuously shot. image frame. Optionally, the shooting frequency when continuously shooting at least two image frames may be the sampling frequency of the camera configured on the mobile terminal, or may be preset according to requirements, which is not limited in this application.

Step 102: Determine the movement information of the moving interferer in at least two image frames.

Wherein, the movement information of the mobile interferer may be related information when the mobile interferer moves, including at least movement speed information and/or movement track information, optionally, the movement speed information may also include the movement speed of adjacent frames, the average movement speed Speed or movement speed trends, etc. The movement track information may further include: the length of the movement track, the range of the movement track, or the extension direction of the movement track, and the like.

Optionally, in this embodiment of the present application, determining the movement information of the moving distractors in the at least two image frames may be determining the movement information of the moving distractors in the at least two image frames according to the temporal correlation of the image frames, wherein the image The temporal correlation of the frames may be the temporal correlation between consecutively photographed adjacent image frames and the positional correlation between each feature pixel point between the image frames, and the like. For example, when shooting at least two image frames continuously, the shooting frequency is 10 image frames per second, then the corresponding moments of two adjacent image frames differ by one tenth of a second, and the moving interference occurs within this tenth of a second. Therefore, there is a certain variation relationship between the positions of the moving distractors in every two adjacent image frames, and the moving distractors in at least two image frames can be tracked to fit the moving distractors. Movement track information; according to the moving displacement in the fitted movement track information of the moving interference object and the interval time between image frames, the moving speed information of the moving interference object can be determined.

Step 103 , according to the movement information of the moving interference object, determine whether to retain the moving interference object in the generated target photographed image.

Optionally, in this embodiment of the present application, when determining whether to retain the mobile interferer in the generated target shooting image according to the movement information of the mobile interferer, it may be determined by setting a judgment rule based on the movement information of the mobile interferer in advance. Whether to keep moving distractors in the resulting target shot. Optionally, when setting the judgment rule, one type of information in the movement information may be selected for judgment, for example, the movement speed in the movement information may be greater than a preset speed threshold; It is judged that, for example, the moving speed of the moving interference object is greater than the preset threshold and the change trend of the moving speed does not decrease. When determining whether to retain the mobile interferer according to the interpretation rule, one judgment rule may be selected for determination, or a plurality of judgment rules may be combined to determine whether to retain the mobile interferer.

Optionally, when determining whether to retain the moving interference object in the generated target photographed image, it may be determined whether to retain the moving interference object in the generated target photographed image according to the moving speed and/or change trend of the moving interference object. and/or according to the length and/or the direction of the trajectory of the moving distractor, determine whether to retain the moving distractor in the generated image of the target. For example, when the moving speed of the moving object is greater than the preset speed threshold, and/or the change trend of the moving speed does not decrease, it can be determined that the moving object is not retained in the generated image of the target; The length of the moving track of the distractor is greater than the preset length threshold, and/or when the change of the moving track extends toward the edge of the image, it is determined that the moving distractor is not retained in the generated target shot image; When conditions are met, it is determined that the moving distractor is retained in the generated target shot image. The advantage of this setting is that the multi-dimensional disturbance object movement information and the judgment rules corresponding to the combined information are selected to determine whether to retain the moving object in the target shot image, thereby improving the accuracy of determining whether to retain the moving object, and then Improve shooting quality.

It should be noted that the magnitude of the moving speed of the above-mentioned moving interference object may be the magnitude of the average moving speed of the moving speed, or may be the magnitude of the instantaneous moving speed corresponding to every two image frames. When judging whether the moving speed is greater than the preset speed threshold, it can be judged whether the average moving speed exceeds the preset speed threshold; it can also be judged whether there is any instantaneous moving speed corresponding to each two image frames that exceeds the preset speed threshold. Instantaneous movement speed. This application is not limited to this.

Optionally, retaining moving distractors in the generated image of the target may be to perform fusion processing on each image frame containing the moving distractors to remove the artifacts of the moving distractors, and generate a final clear image containing the moving distractors. The target captures the image. No moving objects are retained in the generated target shot images. The moving objects in each image frame can be removed first, and then the image frames from which the moving objects have been removed are fused to generate a final image that does not contain moving objects. image with a clear target.

In the method for processing a captured image provided by the embodiment of the present application, firstly, if it is detected that there is a moving interference object in the preview screen, at least two image frames are continuously captured; then, the movement information of the moving interference object in the at least two image frames is determined. and finally, according to the movement information of the moving interference object, determine whether to retain the moving interference object in the generated target shot image. Compared with the related art, during the photographing process, if there is a moving object in the photographing field of view, the photographed image is blurred and the quality is poor. In this embodiment of the present application, when there is a moving object in the preview screen, a plurality of image frames can be continuously captured to determine the movement information of the moving object, so as to determine whether to retain the moving object in the generated target image, thereby avoiding the moving object. The impact on the sharpness of the captured image, improve the shooting quality.

FIG. 2 is a schematic flowchart of another method for processing a captured image provided by an embodiment of the present application. As a further description of the foregoing embodiment, the method includes the following steps:

Step 201: If it is detected that there is a moving interference object in the preview image, at least two image frames are continuously captured.

Step 202: Detect the area where the moving interference object is located in at least two image frames.

Wherein, the area where the moving disturbance object is located may be the area corresponding to the moving disturbance object in each image frame.

Exemplarily, in this embodiment of the present application, there are many methods for detecting the area where the moving interference object is located in at least two image frames, which is not limited in the present application. For example, when it is detected that there is a moving distractor in the preview image, the feature point of the moving distractor is extracted, and when the area where the moving distractor is located in each image frame is detected, the moving distractor with the feature point in each image frame is searched for , extract the contour of the moving distractor, and take the area where the contour is located in the image as the area where the moving distractor is located in the image. It is also possible to first determine the region where each object is located in the shooting content of each image frame, extract the feature points of each object, and then compare the region and feature points of each object corresponding to two adjacent image frames. , to detect whether there is a photographed object with the same feature points, but the area where the photographed object is located changes, if there is, the photographed object is regarded as a moving interference object, and the area where the photographed object is located is regarded as the moving interference object in the image frame. Optionally, in order to prevent the misjudgment of the area change caused by the calculation error, when the change of the position of the area where the photographed object is located is greater than the preset change threshold (for example, the moving pixel exceeds 10 pixels), it is determined that the area where the photographed object is located has occurred. Variety.

Step 203: Calculate the movement information of the moving disturbance object according to the shooting parameters and the area where the moving disturbance object is located in each image frame.

The shooting parameters may be setting parameters corresponding to the camera configured on the mobile terminal when continuously shooting images, and may include the sampling frequency, resolution, and shooting field size of the camera, and the like. When calculating the movement information of the moving interference object, one or more different shooting parameters may be selected for calculation according to the specific content of the movement information.

进行计算。在计算移动轨迹朝向时，可以是计算该移动干扰物与图像的某一边缘所呈的角度；也可以是计算该移动干扰物与拍摄目标所呈的角度，例如，移动干扰物的起点和拍摄目标所在的连线与移动干扰物轨迹所呈的角度。需要说明的是，上述方法计算出的轨迹长度为像素距离，为了提高本申请实施例判断是否保留移动干扰物的准确性，可以根据拍摄时摄像头的拍摄视场和分辨率之间的比例关系，将计算得到的像素距离转换为实际轨迹长度。例如，若像素拍摄视场和分辨率之间的比例关系为m/n，计算得到的像素轨迹长度为s，则实际轨迹长度为L＝s×m/n。Optionally, when calculating the movement track information of the moving distractor according to the shooting parameters and the area where the moving distractor is located in each image frame, the location coordinates of the region corresponding to the moving distractor in each image frame can be fitted to determine the location of the moving distractor. movement track. When calculating the length of the moving track, the length of the track corresponding to the position coordinates of the moving object in each two image frames can be calculated separately, and then the length of the moving track can be obtained by summing up. The coordinates of the center position of the area in one image frame are (a, b), and the coordinates of the center position of the area in the second image frame are (c, d). In the calculation of the two image frames, the position of the area of the moving object is moved. When the coordinate corresponds to the trajectory length s, it can be according to the formula

Calculation. When calculating the direction of the moving track, the angle between the moving object and an edge of the image can be calculated; it can also be calculated at the angle between the moving object and the shooting target, for example, the starting point and the shooting point of the moving object. The angle between the line where the target is located and the trajectory of the moving distractor. It should be noted that the trajectory length calculated by the above method is the pixel distance. In order to improve the accuracy of determining whether to retain the moving interference object in the embodiment of the present application, the proportional relationship between the shooting field of view and the resolution of the camera during shooting can be used. Convert the calculated pixel distance to the actual trajectory length. For example, if the proportional relationship between the pixel shooting field of view and the resolution is m/n, and the calculated pixel track length is s, the actual track length is L=s×m/n.

Optionally, when calculating the moving speed information of the interferer according to the shooting parameters and the area where the moving interferer is located in each image frame, the time interval between two adjacent image frames can be determined according to the sampling frequency of the camera; therefore, it can be The moving speed of the moving interferer is determined according to the length of the moving track corresponding to the moving interferer in the two adjacent image frames and the time interval between the two adjacent image frames. Optionally, the average speed value and the change trend of the speed corresponding to the moving interference object can also be determined by calculating the speed value corresponding to every two image frames.

Step 204 , according to the movement information of the moving interference object, determine whether to retain the moving interference object in the generated target photographed image.

The method for processing a captured image provided by the embodiment of the present application can calculate the movement information of the moving interference object by detecting the area of the moving interference object in each image frame after shooting multiple images continuously, in combination with the current shooting parameters, so as to determine whether the generation The moving interferer is retained in the target photographed image, which improves the accuracy of the calculation of the movement information of the mobile interferer, and provides a guarantee for accurately judging whether the mobile interferer is retained in the generated target photographed image, thereby improving the shooting quality.

FIG. 3 is a schematic flowchart of another method for processing a captured image provided by an embodiment of the present application. As a preferred example of the above embodiments, the method is suitable for capturing a group photo image of multiple people in a portrait capturing mode. case of processing. The method includes the following steps:

Step 301, start.

Step 302 , if it is detected that there is a moving interferer in the preview screen, determine whether the mobile interferer is the target interferer; if so, go to step 303 ; if not, go to step 307 .

The target distractors may be classified according to the types of distractors, for example, they may be divided into human distractors, animal distractors, vehicle distractors, and the like. The specific type of the target interferer can be determined according to different shooting modes.

Optionally, since this embodiment is mainly applicable to the portrait shooting mode, the portrait shooting mode can locate the faces of multiple people at the same time when taking a group photo, but maybe not all the people are the people who need to take a group photo. Therefore, this embodiment The main purpose is to process the moving person during the shooting process and determine whether to keep the moving person in the final generated target shooting image, so the target distractor in this embodiment can be a human distractor. Preferably, the character distractor may also be specific to the face of the character. It should be noted that the type of the target distractor can be changed according to the actual shooting mode and shooting requirements. For example, if the shooting target is an animal, it can be the animal distractor determined by the target distractor.

Exemplarily, when judging whether the mobile interferer is the target interferer, the feature of the target interferer can be extracted in advance, and it is judged whether the mobile interferer in the detection screen has the feature. 303 , if not, it means that it is not the target interferer, and step 307 is executed. For example, if the target distractor is a person, the facial features of the human face are extracted in advance, and it is determined whether the moving distractor in the detection screen has the facial features of the human face. 303, further determine whether the person needs to be retained, if it does not have the facial features of a human face, it means that the distractor is a non-person, and step 307 can be executed to directly not retain the moving distractor in the generated target photographed image.

It should be noted that, in the embodiment of the present application, if the moving distractor is not the target distractor, it can also be judged according to other rules whether to retain the moving distractor in the generated target shot image, and it is not limited to performing step 307 on the target. No moving distractions remain in the captured image.

Step 303: If the moving interferer is the target interferer, shoot at least two image frames continuously.

Step 304: Determine the movement information of the moving interferer in at least two image frames.

Step 305 , determine whether the length of the movement track of the moving interferer is greater than the preset length threshold, or whether the moving speed is greater than the preset speed threshold; if so, go to step 307 , if not, go to step 306 .

Exemplarily, in the process of taking a group photo, it is unavoidable that some people shake left and right or move slightly, but usually, the movement of the group photo person is usually small in terms of speed and movement trajectory, and sudden intrusion. The speed and range of the movement trajectory of the disturber of the lens will be relatively large. Therefore, it can be determined whether the length of the movement trajectory of the moving distractor is greater than the preset length threshold, or whether the movement speed is greater than the preset speed threshold. If so, then It means that the moving distractor is not one of the many people in the group photo, but a distractor who suddenly breaks into the lens. At this time, step 307 is performed, and the moving distractor is not retained in the generated target photographed image; otherwise, step 306 is performed, and the generated Keep moving distractors in the captured image of the target.

Step 306: Retain moving interference objects in the generated target photographed image.

Step 307: Do not retain moving distractors in the generated target shot image.

In the processing method for a captured image provided by the embodiment of the present application, when the captured image is a group photo image and the moving distractor is the target distractor, if the length of the moving trajectory is greater than the preset length threshold or the moving speed is greater than the preset speed threshold, then the generated The moving interference object is not retained in the target shooting image, which can retain the characters that move in a small range during the group photo, remove the interference of the non-group photo characters who move in a large amount, and improve the shooting quality.

FIG. 4 is a schematic flowchart of another method for processing a captured image provided by an embodiment of the present application. As a preferred example of the above-mentioned embodiments, the method is applicable to when the capturing mode is the portrait mode, and the moving interference object is another capturing mode. The target, which runs to the original shooting target to take a picture during the photographing process. The method includes the following steps:

Step 401, start.

Step 402 , if it is detected that there is a moving interferer in the preview screen, determine whether the mobile interferer is the target interferer; if so, go to step 403 ; if not, go to step 407 .

Step 403: If the moving interferer is the target interferer, shoot at least two image frames continuously.

Step 404: Determine the movement information of the moving interferer in at least two image frames.

Step 405 , determine whether the moving speed of the moving interference object is decreasing, and whether the moving trajectory of the moving interference object extends toward the shooting target direction, if yes, go to step 406 , if not, go to step 407 .

Exemplarily, this embodiment is applicable to the situation in which the moving interferer is not the interferer who accidentally broke into the camera, but the group photo who wants to take a group photo together. For example, user B outside the camera wants to take a group photo with user A. , run quickly to the situation next to user A. In view of this situation, users who run into the camera usually have two characteristics: (1) The moving speed of the intrusion camera will gradually decrease, and when it reaches the side of the original shooting target, the speed will be reduced to zero. (2) The movement trajectory will be directed towards the original shooting target, and finally stop around the original shooting target. Therefore, based on the above two characteristics, when the moving speed of the moving distractor tends to decrease, and the moving trajectory of the moving distractor extends toward the shooting target, it can be confirmed that the moving distractor is not a distractor who accidentally broke into the lens, Step 406 is performed, and the moving distractor is retained in the generated target photographed image; otherwise, it is confirmed that the moving distractor is an interferer who broke into the lens by mistake, and step 407 is performed, and the moving distractor is not retained in the generated target photographed image. .

Optionally, when judging whether the movement trajectory of the moving distractor extends in the direction of the shooting target, it can be judged whether the angle between the moving trajectory of the moving distractor and its moving starting point to the line where the shooting target is located gradually decreases, and the moving distractor reaches the target. Whether the distance of the shooting target gradually decreases, and if all are satisfied, it means that the movement trajectory of the moving interference object extends toward the shooting target.

Step 406: Retain moving interference objects in the generated target shot image.

Step 407: Do not retain moving interference objects in the generated target photographed image.

In the processing method for a captured image provided by the embodiment of the present application, when the captured image is a human image and the moving distractor is the target distractor, if the moving speed tends to decrease, and the moving trajectory extends toward the shooting target, then the generated target Retaining the moving interference object in the photographed image can avoid the situation of mistakenly deleting another photographing target next to the original photographing target running toward the lens, thereby improving the photographing quality.

FIG. 5 is a schematic flowchart of another method for processing a captured image provided by an embodiment of the present application. As a further description of the foregoing embodiment, the method includes the following steps:

Step 501: If it is detected that there is a moving interference object in the preview image, shoot at least two image frames continuously.

Step 502: Determine the movement information of the moving interferer in at least two image frames.

Step 503 , according to the movement information of the moving interference object, determine whether to retain the moving interference object in the generated target photographed image, if yes, go to step 504 , if not, go to step 505 .

Step 504 , if the moving interferer is retained in the generated target shot image, then according to the temporal correlation of the image frames, fuse at least two image frames containing the moving interferer to generate the target shot image.

Exemplarily, if moving distractors are retained in the generated target shot image, it is necessary to perform fusion processing on each image frame containing the moving distractors. The specific fusion processing process may be based on the temporal correlation between the image frames. , to determine the correlation between the locations of the moving interference objects in each image frame. First, fuse the images of the area where the moving distractors are located in each image frame, remove the artifacts of the moving distractors, and obtain a clear image of the target distractors, and then combine the specific shooting scenes to determine whether the fused moving distractors should be placed in the captured image. For example, if the moving distractor is a slight swaying among many people in the group photo, the average value of the area where the moving distractor is located in each image frame can be averaged, and the fused moving distractor can be placed in the average disturbance corresponding to each image frame. the area where the object is located. If the captured image is a shooting target that the intruding lens wants to take part in, the fused moving distractor may be placed in the area where the moving distractor is located in the last image frame.

Optionally, when merging at least two image frames containing moving distractors, it may be based on a pixel-level fusion algorithm and/or a feature-level fusion algorithm to fuse the processed at least two image frames to generate a target shot. image.

Among them, pixel-level fusion is also called data-level fusion, which may be a process of directly processing the collected image data containing moving interference objects to obtain a fusion image. Specifically, the pixel-level fusion algorithm can include spatial domain algorithms and transform domain algorithms, etc., and the spatial domain algorithms can include a variety of fusion rule methods, such as logical filtering methods, gray-scale weighted average methods, and contrast modulation methods; transform domain algorithms It can also include pyramid decomposition and fusion method and wavelet transform method. The advantage of using the pixel-level fusion algorithm is that it can restore as much detail information in the image as possible, such as the extraction of edges and textures. It can well judge the pixel point information of moving interference objects, so as to restore the blurred moving interference objects. In addition, as much image information of the non-moving interferers can be retained as possible, so that the fused image can ensure its clarity regardless of the area where the moving interferers are located or the areas where the non-moving interferers are located.

The feature-level image fusion algorithm can extract feature information from the collected images containing moving distractors, these feature information is the feature information of the area where the moving distractors are located, and then analyze, process and integrate these feature information to Get the fused image features. The advantage of using the feature-level fusion algorithm is that the fused mobile interferers have higher definition, and the feature-level fusion compresses the image information, and then analyzes and processes it with a computer, which consumes relatively little memory and time. The real-time performance of processing will be improved.

Optionally, when merging at least two image frames containing moving distractors, it may also be to remove the image where the unclear moving distractors containing artifacts are located from the at least two image frames, and then based on the at least two image frames. The above fusion method fuses images containing clear moving distractors.

Step 505 , if no moving interference objects are retained in the generated target captured image, remove the moving interference objects in at least two image frames according to the temporal correlation of the image frames to obtain a set of preprocessed images.

Exemplarily, the set of preprocessed images may be a set of image frames from which moving distractors are removed. According to the temporal correlation of the image frames, when removing the moving distractors in at least two image frames, the original shooting content corresponding to the area where the moving distractors are located in each image frame may be determined according to the temporal correlation between the image frames. , and replace the original shooting content in the area where the moving distractor is located in the image frame. For example, if the location of the moving distractor in the first image frame is S1, you can select any other frame except the first image frame to shoot The content in the corresponding S1 area in the image is the original shooting content corresponding to the moving interferer in the first image frame. replaces the area where the moving distractor is located in the first image frame.

Step 506 , fuse the preprocessed image set to generate the target shot image.

Exemplarily, when the preprocessed image set is fused, the at least two processed image frames may also be fused based on a pixel-level fusion algorithm and/or a feature-level fusion algorithm to generate a target shot image.

The method for processing a captured image provided by the embodiment of the present application can, according to the movement information of the moving interference object, fuse each image frame containing the moving interference object to generate the target captured image when it is determined to keep the moving interference object, and when it is determined that the moving interference object is not When the moving interference object is retained, the moving interference object is first removed from each image frame, and then the target shot image is generated by fusion. It can automatically analyze and process the moving interference during the shooting process, and can generate clear and high-quality shooting images regardless of whether the interference is retained.

FIG. 6 is a structural block diagram of a processing apparatus for capturing images provided by an embodiment of the present application. The apparatus may be implemented by software and/or hardware, and is generally integrated in a mobile terminal with a photographing function, and may execute the above-described embodiments. The processing method of the captured image. As shown in FIG. 6 , the apparatus includes: an image capturing module 601 , an information determining module 602 and a reservation determining module 603 .

An image capturing module 601, configured to continuously capture at least two image frames if it is detected that there is a moving object in the preview image;

an information determination module 602, configured to determine the movement information of the moving interferers in at least two image frames continuously captured by the image capturing module 601;

The reservation determination module 603 is configured to determine whether to retain the mobile interference object in the generated target photographed image according to the movement information of the mobile interference object determined by the information determination module 602 .

Further, the information determination module 602 is used for:

detecting the area where the moving distractor is located in the at least two image frames;

The movement information of the moving disturbance object is calculated according to the shooting parameters and the area where the moving disturbance object is located in each image frame.

Further, the image capturing module 601 is used for:

If it is detected that there is a mobile interferer in the preview screen, then determine whether the mobile interferer is a target interferer;

If the moving distractor is the target distractor, at least two image frames are continuously captured.

Further, the reservation determination module 603 is used for:

If the captured image is a group photo image, when the length of the moving track of the moving distractor is greater than the preset length threshold, or the moving speed is greater than the preset speed threshold, the moving distractor is not retained in the generated target photographed image.

Further, the reservation determination module 603 is used for:

If the shooting mode is the portrait mode, when the moving speed of the moving distractor tends to decrease, and the moving trajectory of the moving distractor extends toward the shooting target, the moving distraction is retained in the generated target shooting image thing.

Further, the above-mentioned device further includes an image fusion module, which is used for determining whether to retain the moving distractor in the generated target photographed image, if no moving distractor is retained in the generated target photographed image, then according to the image. The time domain correlation of the frames is used to remove the moving interference in the at least two image frames to obtain a set of pre-processed images; the set of pre-processed images is fused to generate a target shot image.

Further, the image fusion module is also used to determine whether to retain the moving distractor in the generated target photographed image, if the moving distractor is retained in the generated target photographed image, according to the time domain of the image frame. Correlation, at least two image frames containing moving distractors are fused to generate a target shot image.

In the processing device for capturing images provided by the embodiments of the present application, firstly, if the image capturing module 601 detects that there is a moving interference object in the preview screen, it continuously captures at least two image frames; then, the information determination module 602 determines at least two consecutively captured image frames. The movement information of the moving interference object in each image frame, and finally, the retention determining module 603 determines whether to keep the moving interference object in the generated target shot image according to the determined movement information of the moving interference object. Compared with the related art, during the photographing process, if there is a moving object in the photographing field of view, the photographed image is blurred and the quality is poor. In this embodiment of the present application, when there is a moving object in the preview screen, a plurality of image frames can be continuously captured to determine the movement information of the moving object, so as to determine whether to retain the moving object in the generated target image, thereby avoiding the moving object. The impact on the sharpness of the captured image, improve the shooting quality.

The above-mentioned apparatus can execute the methods provided by all the foregoing embodiments of the present application, and has corresponding functional modules and beneficial effects for executing the above-mentioned methods. For technical details not described in detail in this embodiment, reference may be made to the methods provided in all the foregoing embodiments of this application.

FIG. 7 is a schematic structural diagram of a terminal device provided by an embodiment of the present application. As shown in FIG. 7 , the terminal may include: a casing (not shown in the figure), a memory 701 , a central processing unit (Central Processing Unit, CPU) 702 (also known as a processor, hereinafter referred to as CPU), and storage on the memory 701 A computer program, a circuit board (not shown in the figure) and a power supply circuit (not shown in the figure) and may be executed on the processor 702 . The circuit board is arranged inside the space enclosed by the casing; the CPU 702 and the memory 701 are arranged on the circuit board; the power circuit is used to supply power to each circuit or device of the terminal; The memory 701 is used for storing executable program codes; the CPU 702 runs a program corresponding to the executable program codes by reading the executable program codes stored in the memory 701 .

The terminal further includes: a peripheral interface 703, an RF (Radio Frequency, radio frequency) circuit 705, an audio circuit 706, a speaker 711, a power management chip 708, an input/output (I/O) subsystem 709, a touch screen 712, and other inputs /Control device 710 and external ports 704, these components communicate through one or more communication buses or signal lines 707.

It should be understood that the illustrated terminal device 700 is merely an example of a terminal, and that the terminal device 700 may have more or fewer components than those shown in the figure, two or more components may be combined, or Different component configurations are possible. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

The following is a detailed description of a terminal device provided in this embodiment, where a smart phone is used as an example for the terminal device.

Memory 701, which can be accessed by CPU 702, peripheral interface 703, etc., said memory 701 can include high-speed random access memory, and can also include non-volatile memory, such as one or more disk storage devices, flash memory devices , or other volatile solid-state storage devices.

A peripheral interface 703 that can connect the input and output peripherals of the device to the CPU 702 and the memory 701 .

I/O subsystem 709 , which can connect input and output peripherals on the device, such as touch screen 712 and other input/control devices 710 , to peripherals interface 703 . The I/O subsystem 709 may include a display controller 7091 and one or more input controllers 7092 for controlling other input/control devices 710 . Wherein, one or more input controllers 7092 receive electrical signals from or send electrical signals to other input/control devices 710, which may include physical buttons (push buttons, rocker buttons, etc. ), dial pad, slide switch, joystick, click wheel. Notably, the input controller 7092 can be connected to any of the following: a keyboard, an infrared port, a USB interface, and a pointing device such as a mouse.

Wherein, according to the working principle of the touch screen and the classification of the medium for transmitting information, the touch screen 712 may be a resistive type, a capacitive induction type, an infrared type or a surface acoustic wave type. According to the type of installation, the touch screen 712 may be an external type, a built-in type or an integral type. Classified according to technical principles, the touch screen 712 may be: a vector pressure sensing technology touch screen, a resistive technology touch screen, a capacitive technology touch screen, an infrared technology touch screen or a surface acoustic wave technology touch screen.

The touch screen 712, which is an input interface and an output interface between the user terminal and the user, displays visual output to the user, and the visual output may include graphics, text, icons, videos, and the like. Optionally, the touch screen 712 sends electrical signals triggered by the user on the touch screen (eg, electrical signals on the contact surface) to the processor 702 .

Display controller 7091 in I/O subsystem 709 receives electrical signals from touch screen 712 or sends electrical signals to touch screen 712 . The touch screen 712 detects the contact on the touch screen, and the display controller 7091 converts the detected contact into interaction with the user interface object displayed on the touch screen 712, that is, to realize human-computer interaction, and the user interface object displayed on the touch screen 712 can be run. Icons for games, icons for connecting to the corresponding network, etc. It is worth noting that the device may also include a light mouse, which is a touch-sensitive surface that does not display visual output, or an extension of the touch-sensitive surface formed by a touch screen.

The RF circuit 705 is mainly used to establish the communication between the smart speaker and the wireless network (ie, the network side), so as to realize the data reception and transmission between the smart speaker and the wireless network. Such as sending and receiving text messages, e-mails, etc.

The audio circuit 706 is mainly used for receiving audio data from the peripheral interface 703 , converting the audio data into an electrical signal, and sending the electrical signal to the speaker 711 .

The speaker 711 is used to restore the voice signal received by the smart speaker from the wireless network through the RF circuit 705 into sound and play the sound to the user.

The power management chip 708 is used for power supply and power management for the hardware connected to the CPU 702, the I/O subsystem and the peripheral interface.

In this embodiment, the central processing unit 702 is used for:

If it is detected that there is a moving distraction in the preview screen, shoot at least two image frames continuously;

determining movement information of a moving distractor in the at least two image frames;

According to the movement information of the moving interference object, it is determined whether to retain the moving interference object in the generated target shot image.

Further, the determining of the movement information of the moving interferers in the at least two image frames includes:

detecting the area where the moving distractor is located in the at least two image frames;

The movement information of the moving disturbance object is calculated according to the shooting parameters and the area where the moving disturbance object is located in each image frame.

Further, if it is detected that there is a moving object in the preview image, at least two image frames are continuously captured, including:

If it is detected that there is a mobile interferer in the preview screen, then determine whether the mobile interferer is a target interferer;

If the moving distractor is the target distractor, at least two image frames are continuously captured.

Further, according to the movement information of the moving interference object, determining whether to retain the moving interference object in the generated target shot image includes:

If the captured image is a group photo image, when the length of the moving track of the moving distractor is greater than the preset length threshold, or the moving speed is greater than the preset speed threshold, the moving distractor is not retained in the generated target photographed image.

Further, according to the movement information of the moving interference object, determining whether to retain the moving interference object in the generated target shot image includes:

If the shooting mode is the portrait mode, when the moving speed of the moving distractor tends to decrease, and the moving trajectory of the moving distractor extends toward the shooting target, the moving distraction is retained in the generated target shooting image thing.

Further, after the determining whether to retain the moving distractor in the generated target shot image, the method further includes:

If no moving interference objects are retained in the generated target captured images, removing the moving interference objects in the at least two image frames according to the temporal correlation of the image frames, to obtain a set of preprocessed images;

The set of preprocessed images is fused to generate a target shot image.

Further, after the determining whether to retain the moving distractor in the generated target shot image, the method further includes:

If the moving interferer is retained in the generated target shot image, at least two image frames containing the moving interferer are fused according to the temporal correlation of the image frames to generate the target shot image.

Embodiments of the present application further provide a storage medium containing executable instructions of a terminal device, where the executable instructions of the terminal device are used to execute a method for processing a captured image when executed by a processor of the terminal device, and the method includes:

If it is detected that there is a moving distraction in the preview screen, shoot at least two image frames continuously;

determining movement information of a moving distractor in the at least two image frames;

According to the movement information of the moving interference object, it is determined whether to retain the moving interference object in the generated target shot image.

Further, the determining of the movement information of the moving interferers in the at least two image frames includes:

detecting the area where the moving distractor is located in the at least two image frames;

The movement information of the moving disturbance object is calculated according to the shooting parameters and the area where the moving disturbance object is located in each image frame.

Further, if it is detected that there is a moving object in the preview image, at least two image frames are continuously captured, including:

If it is detected that there is a mobile interferer in the preview screen, then determine whether the mobile interferer is a target interferer;

If the moving distractor is the target distractor, at least two image frames are continuously captured.

Further, according to the movement information of the moving interference object, determining whether to retain the moving interference object in the generated target shot image includes:

If the captured image is a group photo image, when the length of the moving track of the moving distractor is greater than the preset length threshold, or the moving speed is greater than the preset speed threshold, the moving distractor is not retained in the generated target photographed image.

Further, according to the movement information of the moving interference object, determining whether to retain the moving interference object in the generated target shot image includes:

If the shooting mode is the portrait mode, when the moving speed of the moving distractor tends to decrease, and the moving trajectory of the moving distractor extends toward the shooting target, the moving distraction is retained in the generated target shooting image thing.

Further, after the determining whether to retain the moving distractor in the generated target shot image, the method further includes:

If no moving interference objects are retained in the generated target captured images, removing the moving interference objects in the at least two image frames according to the temporal correlation of the image frames, to obtain a set of preprocessed images;

The set of preprocessed images is fused to generate a target shot image.

Further, after the determining whether to retain the moving distractor in the generated target shot image, the method further includes:

If the moving interferer is retained in the generated target shot image, at least two image frames containing the moving interferer are fused according to the temporal correlation of the image frames to generate the target shot image.

The computer storage medium of the embodiments of the present application may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples (a non-exhaustive list) of computer readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), Erasable Programmable Read Only Memory (EPROM or Flash), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

A computer-readable signal medium may include a propagated data signal in baseband or as part of a carrier wave, with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .

Program code embodied on a computer readable medium may be transmitted using any suitable medium, including - but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out the operations of the present application may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, but also conventional procedural languages, or a combination thereof. Programming Language - such as "C" language or similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through Internet connection).

Of course, a storage medium containing computer-executable instructions provided by the embodiments of the present application is not limited to the above-mentioned application recommendation operation, and the computer-executable instructions can also perform the processing of captured images provided by any embodiment of the present application. related operations in the method.

Note that the above are only preferred embodiments of the present application and applied technical principles. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present application. The scope is determined by the scope of the appended claims.

Claims (9)

1. A method of processing a captured image, comprising:

if the mobile interferent is detected to exist in the preview picture, continuously shooting at least two image frames;

determining movement information of a moving interferent in the at least two image frames;

determining whether the moving interfering object is remained in the generated target shooting image or not according to the moving information of the moving interfering object, and the method comprises the following steps:

if the shooting mode is the portrait mode, when the moving speed of the moving interfering object is in a decreasing trend and the moving track of the moving interfering object extends towards the shooting target direction, the moving interfering object is kept in the generated target shooting image.

2. The method of claim 1, wherein the determining movement information for moving interferers in the at least two image frames comprises:

detecting a region where a moving interferent is located in the at least two image frames;

and calculating the movement information of the moving interferent according to the shooting parameters and the area where the moving interferent is located in each image frame.

3. The method of claim 1, wherein the continuously capturing at least two image frames if the presence of the moving interferers in the preview screen is detected comprises:

if the preview picture is detected to have the mobile interferent, judging whether the mobile interferent is a target interferent;

and if the mobile interference object is the target interference object, continuously shooting at least two image frames.

4. The method according to claim 1 or 3, wherein the determining whether or not the moving interfering object remains in the generated target captured image according to the movement information of the moving interfering object further comprises:

if the shot image is a group photo image, when the length of the movement track of the movement interfering object is greater than a preset length threshold value or the movement speed is greater than a preset speed threshold value, the movement interfering object is not reserved in the generated target shot image.

5. The method of claim 1, wherein after said determining whether to retain the moving interfering object in the generated captured image of the target, further comprising:

if no mobile interferent is reserved in the generated target shooting image, removing the mobile interferent in the at least two image frames according to the time domain relevance of the image frames to obtain a preprocessed image set;

and fusing the preprocessed image set to generate a target shooting image.

6. The method of claim 1, wherein after said determining whether to retain the moving interfering object in the generated captured image of the target, further comprising:

and if the mobile interferent is reserved in the generated target shooting image, fusing at least two image frames containing the mobile interferent according to the time domain relevance of the image frames to generate the target shooting image.

7. A captured image processing apparatus, comprising:

the image shooting module is used for continuously shooting at least two image frames if the mobile interferent is detected to exist in the preview picture;

the information determining module is used for determining the movement information of the moving interferent in at least two image frames continuously shot by the image shooting module;

a retention determination module configured to determine whether to retain the mobile interfering object in the generated target captured image according to the movement information of the mobile interfering object determined by the information determination module, including:

if the shooting mode is the portrait mode, when the moving speed of the moving interfering object is in a decreasing trend and the moving track of the moving interfering object extends towards the shooting target direction, the moving interfering object is kept in the generated target shooting image.

8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method of processing a captured image according to any one of claims 1 to 6.

9. A mobile terminal, characterized in that it comprises a memory, a processor and a computer program stored on the memory and executable on the processor, said processor implementing the method for processing a captured image according to any one of claims 1 to 6 when executing said computer program.

Images