CN112367460B - Video anti-shake method and device - Google Patents

Abstract

The application aims to provide a video anti-shaking method, which comprises the following steps: responding to the recording operation of a user, and acquiring a frame picture to be processed, wherein the frame picture to be processed comprises a current frame picture and a previous frame picture of the current frame picture; determining offset information of the image information in the current frame picture relative to the image information in the previous frame picture according to the image information in the current frame picture and the previous frame picture; performing picture processing on the current frame picture according to the offset information to obtain a processed current frame picture; and combining the processed current frame picture and the previous frame picture according to the video frame sequence to generate a recorded video. The processed image information in the current frame picture and the processed image information in the previous frame picture are positioned on the same horizontal line, so that the recorded video is prevented from shaking and affecting the quality of the recorded video.

Description

A kind of video anti-shake method and device

technical field

The present application relates to the field of communications, and in particular, to a technology for video stabilization.

Background technique

Video generally refers to various technologies that capture, record, process, store, transmit and reproduce a series of static images in the form of electrical signals. When the continuous image changes exceed 24 frames per second, according to the principle of persistence of vision, the human eye cannot distinguish a single static image; it appears to be a smooth and continuous visual effect, and such a continuous image is called a video.

With the development of the times, more and more users will record their lives by shooting videos, or produce various film and television works. According to different applications, the requirements for video shooting are also different, but the most basic requirement is to try not to shake during the shooting process, which will seriously affect the effect of video shooting.

SUMMARY OF THE INVENTION

An object of the present application is to provide a video anti-shake method and device.

According to one aspect of the present application, a video anti-shake method is provided, the method comprising:

In response to a user's recording operation, obtain a frame picture to be processed, wherein the frame picture to be processed includes a current frame picture and a previous frame picture of the current frame picture;

Determine the offset information of the image information in the current frame picture relative to the image information in the previous frame picture in the target anti-shake direction according to the image information in the current frame picture and the previous frame picture ;

Perform picture processing on the current frame picture according to the offset information to obtain the processed current frame picture;

The processed picture of the current frame and the picture of the previous frame are combined according to the sequence of video frames to generate a recorded video.

According to an aspect of the present application, a video anti-shake device is provided, the device comprising:

A module, configured to obtain a frame picture to be processed in response to a user's recording operation, wherein the frame picture to be processed includes a current frame picture and a previous frame picture of the current frame picture;

Modules 1 and 2 are used to determine, according to the image information in the current frame picture and the previous frame picture, whether the image information in the current frame picture is in the target anti-shake relative to the image information in the previous frame picture Offset information in the direction;

One or three modules are used to perform picture processing on the current frame picture according to the offset information to obtain the processed current frame picture;

A fourth module, configured to combine the processed picture of the current frame and the picture of the previous frame according to the sequence of video frames to generate a recorded video.

According to an aspect of the present application, there is provided a video stabilization device, wherein the device includes:

processor; and

A memory arranged to store computer-executable instructions which, when executed, cause the processor to perform the operations of any of the methods described above.

According to one aspect of the present application, there is provided a computer-readable medium storing instructions that, when executed, cause a system to operate any of the methods described above.

Compared with the prior art, the present application obtains the frame picture to be processed in response to the user's recording operation; and determines the frame picture in the current frame picture according to the image information in the current frame picture and the previous frame picture. The offset information of the image information relative to the image information in the previous frame picture in the target anti-shake direction, and the current frame picture is processed according to the offset information, so that the processed current frame picture The image information in the picture is on the same horizontal line as the image information in the previous frame of picture, which prevents the recorded video from feeling jittery and affects the quality of the recorded video.

Description of drawings

Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

FIG. 1 shows a flowchart of a video anti-shake method according to an embodiment of the present application;

2 shows a schematic diagram of picture processing of a video anti-shake method according to an embodiment of the present application;

3 shows a device structure diagram of a video anti-shake device according to an embodiment of the present application;

FIG. 4 illustrates an exemplary system that may be used to implement various embodiments described in this application.

The same or similar reference numbers in the drawings represent the same or similar parts.

Detailed ways

The present application will be described in further detail below with reference to the accompanying drawings.

In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party all include one or more processors (for example, a central processing unit (CPU)), an input/output interface, a network interface, and Memory.

The memory may include non-persistent memory in computer readable media, random access memory (Random Access Memory, RAM) and/or non-volatile memory, such as read only memory (Read Only Memory, ROM) or flash memory (Flash). Memory). Memory is an example of a computer-readable medium.

Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (Phase-Change Memory, PCM), programmable random access memory (Programmable Random Access Memory, PRAM), static random access memory (Static Random-Access Memory, SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically-Erasable Programmable Read-Only Memory (Electrically-Erasable Programmable Read- Only Memory (EEPROM), flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridges, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

The devices referred to in this application include but are not limited to terminals, network devices, or devices formed by integrating terminals and network devices through a network. The terminal includes but is not limited to any mobile electronic product that can perform human-computer interaction with the user (for example, human-computer interaction through a touchpad), such as a smart phone, a tablet computer, etc., and the mobile electronic product can use any operating system. , such as Android operating system, iOS operating system, etc. The network device includes an electronic device that can automatically perform numerical calculation and information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs) ), Programmable Logic Device (PLD), Field Programmable Gate Array (Field Programmable Gate Array, FPGA), Digital Signal Processor (Digital Signal Processor, DSP), embedded devices, etc. The network device includes, but is not limited to, a computer, a network host, a single network server, multiple network server sets, or a cloud composed of multiple servers; here, a cloud is composed of a large number of computers or network servers based on cloud computing, Among them, cloud computing is a kind of distributed computing, a virtual supercomputer composed of a group of loosely coupled computer sets. The network includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, a wireless ad hoc network (Ad Hoc network), and the like. Preferably, the device may also be a program running on the terminal, a network device, or a device formed by a terminal and a network device, a network device, a touch terminal, or a network device and a touch terminal integrated through a network.

Of course, those skilled in the art should understand that the above-mentioned devices are only examples, and other existing or possible devices that may appear in the future, if applicable to this application, should also be included within the protection scope of this application, and are included in this application by reference. this.

In the description of this application, "plurality" means two or more, unless expressly and specifically defined otherwise.

FIG. 1 shows a method flowchart of a video anti-shake method according to an embodiment of the present application, and the method includes step S11 , step S12 , step S13 and step S14 . In step S11, the user equipment acquires a frame picture to be processed in response to the user's recording operation, wherein the frame picture to be processed includes a current frame picture and a previous frame picture of the current frame picture. In step S12, the user equipment determines, according to the image information in the current frame picture and the image information in the previous frame picture, whether the image information in the current frame picture is relative to the image information in the previous frame picture in the target defense Offset information in the dither direction. In step S13, the user equipment performs picture processing on the current frame picture according to the offset information to obtain a processed current frame picture. In step S14, the user equipment combines the processed picture of the current frame with the picture of the previous frame according to the sequence of video frames to generate a recorded video.

Specifically, in step S11, the user equipment obtains a frame picture to be processed in response to a user's recording operation, wherein the frame picture to be processed includes a current frame picture and a previous frame picture of the current frame picture. In some embodiments, the user equipment includes, but is not limited to, a mobile phone, a tablet computer, a computer, a video capture device (eg, a common camera, etc.), and the like. In some embodiments, the solution described in this application is implemented by a corresponding application installed in the user equipment, for example, the application is downloaded and installed in the mobile phone, tablet or computer, and obtained through the application The to-be-processed frame picture, and by comparing the current frame picture with the previous frame picture, image processing is performed on the current frame picture to generate the recorded video. In other embodiments, the solution described in this application is implemented by a corresponding module installed in the user equipment, for example, the corresponding module is installed in the video shooting device, and the The to-be-processed frame picture is compared, and the current frame picture is subjected to picture processing by comparing the current frame picture with the previous frame picture, so as to generate the recorded video. For example, recorded video information can be split into frame-by-frame pictures. In some embodiments, the current frame picture includes a frame picture currently obtained by the user equipment; the previous frame picture includes a previous frame picture obtained before the current frame picture. In some embodiments, the frame picture to be processed includes a start frame picture, the start frame picture needs to satisfy a start condition, and the start frame picture is used as the first frame picture of the generated recording frame video.

In step S12, the user equipment determines, according to the image information in the current frame picture and the image information in the previous frame picture, whether the image information in the current frame picture is relative to the image information in the previous frame picture in the target defense Offset information in the dither direction. In some embodiments, the image information includes picture content information of the current frame picture and the previous frame picture identified based on an image recognition technology (for example, the current frame picture or the previous frame picture table, chair, house, dog, etc. in the picture content information). In some embodiments, the user equipment determines, by comparing the position of the image information in the current frame picture and the position of the image information in the previous frame picture, that the image information in the current frame picture is relative to the image information in the previous frame picture. The offset information of the image information in the previous frame picture in the target anti-shake direction. In some embodiments, the target anti-shake direction includes, but is not limited to, landscape, portrait, landscape, and portrait. In some embodiments, the user equipment only compares the offset information of the current frame picture and the previous frame picture in the target anti-shake direction. In some embodiments, the offset information includes, but is not limited to, a specific offset value (eg, 5 cm, 6 cm, etc.). For example, the target anti-shake direction is landscape, and the user equipment determines by comparing the image information (for example, the dog) in the current frame picture with the image information (for example, the small dog) in the previous frame picture dog) offset information in the lateral direction. For example, if the target anti-shake direction is horizontal, and user A shakes his hand to the left by 5 cm when recording a video, the image information (for example, a dog) in the current frame picture obtained at this time is relative to that in the previous frame picture. In terms of image information (for example, the puppy), it is shifted to the right by 5 cm. The user equipment determines the offset information by comparing the image information in the current frame picture with the image information in the previous frame picture. In some embodiments, since the acquisition time of the current frame picture and the previous frame picture differs only by milliseconds, the image information of the current frame picture and the previous frame picture are almost the same, and the difference between the two is different. It may simply be a shift in the position of the image information due to jitter.

In step S13, the user equipment performs picture processing on the current frame picture according to the offset information to obtain a processed current frame picture. In some embodiments, the offset information includes an offset direction and an offset value (eg, a specific offset value). In some embodiments, the picture processing includes but is not limited to cropping and stitching. In some embodiments, the user equipment performs cropping and splicing of the current frame picture according to the offset information, so as to obtain the cropped and spliced current frame picture. For the specific picture processing process, please refer to the following embodiments, which will not be repeated here.

In step S14, the user equipment combines the processed picture of the current frame with the picture of the previous frame according to the sequence of video frames to generate a recorded video. For example, the user equipment combines the processed picture of the current frame with the picture of the previous frame in real time, so that each frame of the picture in the recorded video formed by the combination is a processed picture.

For example, user A is photographing a puppy on the ground, and the obtained current frame picture and the previous frame picture both include the puppy. The puppy in the current frame picture and the puppy in the previous frame picture are identified based on the image recognition technology. Further, by comparing the position of the puppy in the current frame picture in the current frame picture, and the position of the puppy in the previous frame picture in the previous frame picture, determine that the puppy is in the target anti-shake direction. offset information. For example, the position of the dog in the current frame is moved upward by 5mm relative to the position of the previous frame, wherein the offset information includes an upward offset of 5mm. Further, the current frame picture is cropped and spliced according to the offset information, so that the image information in the current frame picture and the image information in the previous frame picture are located on the same horizontal line. For example, the puppy in the current frame picture is kept on the same horizontal line as the puppy in the previous frame picture (or the position does not shift). Combine the cropped and spliced picture of the current frame with the picture of the previous frame according to the sequence of video frames to generate a recorded video. For example, the recorded video is generated by combining frame-by-frame frame pictures, and the frame-by-frame frame pictures are combined in the order of video frames based on the time obtained during the recording to generate the recorded video.

In some embodiments, the step S11 includes: the user equipment, in response to a user's recording operation, obtains a frame picture to be processed, wherein the frame picture to be processed includes a start frame picture, the current frame picture, and the The picture of the previous frame of the picture of the current frame, the picture of the starting frame satisfies the starting condition. Because in this solution, the offset information of the image information in the current frame picture relative to the image information in the previous frame picture needs to be determined by comparing the image information of the current frame picture and the previous frame picture, Therefore, there are certain requirements for the start frame picture of the entire recorded video, so that all subsequent frame pictures are compared on the basis of the start frame picture. For example, the image information of the initial frame picture needs to be in a natural arrangement state (for example, the edge of the table is horizontal to the edge of the picture).

In some embodiments, the method further includes step S15 (not shown). In step S15, the user equipment acquires a recording frame picture in response to the user's starting recording operation; and determines the starting frame picture from the recording frame picture, wherein the starting frame picture satisfies the starting frame picture condition. For example, when the user starts recording, the acquired first few frames of pictures are used as the recording frame pictures. In some embodiments, the recorded frame picture is not necessarily the to-be-processed frame picture, for example, the image information in the first few frames obtained when the user starts recording is in an unnatural placement state (for example, the edge of the table and the The angle between the edges of the picture is not 0 degrees, that is, it is not horizontal). In this embodiment, when the user starts the recording operation, the obtained recording frame picture is analyzed, and whether each frame of the recording frame picture satisfies the initial condition is detected, and if so, the recording frame picture is used as the initial frame picture. , if not, continue to detect the next frame of recording frame picture until a recording frame picture that satisfies the initial condition is detected.

For example, when user A starts recording, a recording frame picture 1 is obtained. If the first frame of recording frame picture 1 obtained satisfies the starting condition after the recording starts, the recording frame picture 1 is used as the starting frame. picture, further, take the recording frame picture 1 as the previous frame picture, take the acquired recording frame picture 2 as the current frame picture, and compare the image information in the current frame picture and the previous frame picture to determine the the offset information. If the recording frame picture 1 does not meet the initial condition, continue to acquire the recording frame picture 2, the recording frame picture 3, . . . until a recording frame picture that satisfies the initial condition is detected.

In some embodiments, the starting condition includes that the inclination angle of the recorded frame picture is equal to or less than an angle threshold; the determining the starting frame picture from the recorded frame picture includes step S16 (not shown). ) and step S17. In step S16, the user equipment determines the inclination angle of the recorded frame picture; in step S17, if the inclination angle of the recorded frame picture is equal to or less than the angle threshold, the user equipment determines that the recorded frame picture satisfies the initial condition , and use the recording frame picture as the starting frame picture. In some embodiments, an angle threshold of the tilt angle is preset in the user equipment, and when the tilt angle of the recorded frame picture is less than or equal to the angle threshold, it is determined that the recorded frame picture satisfies the initial condition . In some embodiments, the inclination angle of the recorded frame picture is the picture angle of the recorded frame picture (for example, the placement angle of the image information in the recorded frame picture); in other embodiments, the recording The tilt angle of the frame picture is obtained by calculation based on the picture angle of the recorded frame picture and the device angle of the user equipment.

In some embodiments, the step S16 includes: the user equipment determines a picture angle of the recorded frame picture, and uses the picture angle as the inclination angle of the recorded frame picture. For example, the inclination angle of the recorded frame picture is the picture angle of the recorded frame picture. In some embodiments, the picture angle includes an arrangement angle of image information in the recorded frame picture. For example, the picture angle includes the angle between the image information in the recording frame picture and the edge of the recording frame picture, for example, the image information includes a table, and the picture angle is the angle between the table and the picture edge of the recording frame picture .

In other embodiments, the step S16 includes: the user equipment determines the picture angle of the recorded frame picture, and uses the angle sensor to obtain the device angle of the user equipment; the user equipment calculates the picture angle and the device angle according to the picture angle and the device angle. The tilt angle of the recorded frame picture. For example, the user equipment includes a mobile phone, and the angle of the mobile phone is acquired through an angle sensor of the mobile phone, and the angle is used as the device angle. In some embodiments, the picture angle includes an angle between the image information in the recording frame picture and the edge of the recording frame picture, for example, the image information includes a table, and the picture angle is the angle between the table and the recording frame picture The angle of the edge of the picture. In this embodiment, the inclination angle of the recorded frame picture is calculated and obtained based on the picture angle of the recorded frame picture and the device angle of the user equipment.

In some embodiments, the determining the picture angle of the recorded frame picture includes: the user equipment recognizes image information of the recorded frame picture based on an image recognition technology, wherein the image information includes a target object; the user equipment determines the image information of the recorded frame picture. The included angle between the edge of the target object and the edge of the recorded frame picture is determined, and the included angle is determined as the picture angle of the recorded frame picture. For example, the target objects include target objects such as tables and buildings. The user equipment recognizes the target object based on an image recognition technology. and the included angle between the edge of the target object and the edge of the recorded frame picture is determined as the picture angle of the recorded frame picture.

In some embodiments, the picture angle corresponds to the first weight information; the device angle corresponds to the second weight information; the calculating the tilt angle of the recorded frame picture according to the picture angle and the device angle, The method includes: calculating and obtaining the inclination angle of the recorded frame picture according to the device angle, the inclination angle, the first weight information, and the second weight information. For example, different weights are assigned to the picture angle and the device angle, and the tilt angle of the recorded frame picture is calculated based on the different weights and the acquired specific picture angle and device angle. For example, the picture angle can better reflect the presentation state of the image information in the recorded frame picture. The first weight information of the picture angle is 0.3; the second weight information of the device angle is 0.1. For example, the picture angle is 3 degrees, the device angle is 5 degrees, 3*0.3+5*0.1=1.4.

In some embodiments, the method further includes step S18 (not shown), in which the user equipment obtains anti-shake configuration information about the recorded video, wherein the anti-shake configuration information includes the target Anti-shake direction. In some embodiments, the anti-shake configuration information is user configuration input. For example, when a user records a video while walking, it may be more necessary to prevent shaking in the vertical direction. Therefore, the user can set the target anti-shake direction to be vertical. For another example, when a user records a video in a sightseeing elevator, it may be more necessary to To prevent shaking in the horizontal direction, the user can set the target anti-shake direction to be horizontal; for another example, when the user records a puppy on the ground, he may need to prevent shaking in the horizontal and vertical directions, so , the user can set the target anti-shake direction as landscape and portrait. In this embodiment, the user can set the anti-shake configuration information according to his actual shooting needs. In some embodiments, the anti-shake configuration information further includes parameter information such as image depth.

In some embodiments, the target anti-shake direction includes at least one of the following:

Horizontal direction; for example, when the user takes a sightseeing elevator to record a video, the target anti-shake direction includes horizontal direction.

Portrait orientation; for example, when a user walks to record a video, the target anti-shake direction includes portrait orientation.

Landscape and portrait; for example, when the user is still recording a video of a house, the target anti-shake direction includes landscape and portrait.

In some embodiments, the offset information includes an offset direction and an offset value; the step S13 includes step S131 (not shown), step S132 and step S133. In step S131, the user equipment determines the cropping area of the current frame picture and the copy area of the previous frame picture according to the offset direction and the offset value respectively; in step S132, the user equipment copies the The copy area of the previous frame picture, and crop the crop area of the current frame picture; in step S133, the user equipment splices the copy area to the cropped splicing position of the current frame picture, wherein the The orientation of the splicing position in the picture of the current frame is opposite to the orientation of the cropped region in the picture of the current frame. In some embodiments, the cropped area is a picture area to be cropped out. In some embodiments, the copied region is to be copied and spliced onto the cropped picture of the current frame. In some embodiments, the size of the copied region is equal to the size of the cropped region. For example, the user equipment crops the cropped area of the current frame picture, and copies the duplicated area of the previous frame picture, and splices the duplicated area onto the cropped current frame picture. In some embodiments, the offset information includes an offset direction and an offset value, for example, the image information in the current frame picture is shifted to the left by 5 mm relative to the image information in the previous frame picture, Wherein, the offset direction is "left", and the offset value is "5mm".

In some embodiments, the determining the cropping region of the current frame picture according to the offset direction and the offset value includes: determining the current frame picture according to the offset direction and the offset value The cropping area, wherein the orientation of the cropping area in the current frame picture is opposite to the offset direction, the length of the cropping area is equal to the length or width of the current frame picture, the cropping area The width is equal to the offset value. For example, if the image information in the current frame picture is shifted to the left by 5mm relative to the image information in the previous frame picture, the right area of the current frame picture needs to be cropped, and the offset value is determined as the The width of the cropped area, where the length of the cropped area is equal to the width of the current frame picture. For another example, if the image information in the current frame picture is shifted upward by 5mm relative to the image information in the previous frame picture, the lower area of the current frame picture needs to be cropped, and the offset value is determined as the The width of the cropped area, where the length of the cropped area is equal to the length of the current frame picture.

In some embodiments, the determining the copy area of the previous frame picture according to the offset direction and the offset value includes: determining the previous frame according to the offset direction and the offset value a copy area of a frame picture, wherein the orientation of the copy area in the previous frame picture is the same as the offset direction, and the length of the copy area is equal to the length or width of the previous frame picture, The width of the copied area is equal to the offset value. For example, referring to FIG. 2 , FIG. 2( b ) is the picture of the current frame, FIG. 2( a ) is the picture of the previous frame, and FIG. 2( c ) is the picture of the current frame after cropping and splicing. The image information in the current frame picture is shifted upward by 5mm relative to the image information in the previous frame picture, and the length and width of the current frame picture and the previous frame picture are respectively 5cm*5cm. Then, it is determined that the cropping area of the current frame picture is an area with a bottom length of 5 cm and a width of 5 mm. An area with a length of 5 cm and a width of 5 mm in the upper part of the picture of the previous frame is determined as the copy area, the copy area is copied, and the copied area is spliced above the picture of the current frame. For another example, the image information in the current frame picture is shifted to the left by 5mm relative to the image information in the previous frame picture, and the length and width of the current frame picture and the previous frame picture are respectively 5cm*5cm. . Then, it is determined that the cropped area of the current frame picture is an area with a right length of 5 cm and a width of 5 mm. Determine the area with a length of 5 cm and a length of 5 mm on the left side of the previous frame picture as the copy area, copy the copy area, and splicing the copy area to the left side of the current frame picture.

In some embodiments, the step S14 includes: the user equipment combines the processed picture of the current frame and the picture of the previous frame according to the sequence of video frames; during the recording process of the user, repeating the above operations , to generate a recorded video. In some embodiments, the recorded video is composed of multiple frames of pictures. For example, each time the user equipment obtains a frame of the current frame picture, it determines the offset information according to the image information in the current frame picture and the image information in the previous frame picture; and based on the offset information, the offset information is determined. The current frame picture is subjected to picture processing, and the processed current frame picture is combined after the previous frame picture. Further, the user equipment obtains a new current frame picture, continues to compare with the previous frame picture before the new current frame picture, determines offset information, and performs the picture processing on the new current frame picture , until the recording ends. During the whole recording process, each frame picture will be compared and processed, so that the obtained images in the recorded video are on the same horizontal line, and the user will not feel that the images are shaken.

FIG. 3 shows a device structure diagram of a video anti-shake user equipment according to an embodiment of the present application, the device includes a one-module, a two-module, a three-module and a four-module. A module, configured to acquire a frame picture to be processed in response to a user's recording operation, wherein the frame picture to be processed includes a current frame picture and a previous frame picture of the current frame picture. Modules 1 and 2 are used to determine, according to the image information in the current frame picture and the previous frame picture, whether the image information in the current frame picture is in the target anti-shake relative to the image information in the previous frame picture Offset information in the direction. Modules 1 and 3 are configured to perform picture processing on the current frame picture according to the offset information to obtain the processed current frame picture. A fourth module, configured to combine the processed picture of the current frame and the picture of the previous frame according to the sequence of video frames to generate a recorded video.

Specifically, each module is configured to acquire a frame picture to be processed in response to a user's recording operation, wherein the frame picture to be processed includes a current frame picture and a previous frame picture of the current frame picture. In some embodiments, the user equipment includes, but is not limited to, a mobile phone, a tablet computer, a computer, a video capture device (eg, a common camera, etc.), and the like. In some embodiments, the solution described in this application is implemented by a corresponding application installed in the user equipment, for example, the application is downloaded and installed in the mobile phone, tablet or computer, and obtained through the application The to-be-processed frame picture, and by comparing the current frame picture with the previous frame picture, image processing is performed on the current frame picture to generate the recorded video. In other embodiments, the solution described in this application is implemented by a corresponding module installed in the user equipment, for example, the corresponding module is installed in the video shooting device, and the The to-be-processed frame picture is compared, and the current frame picture is subjected to picture processing by comparing the current frame picture with the previous frame picture, so as to generate the recorded video. For example, recorded video information can be split into frame-by-frame pictures. In some embodiments, the current frame picture includes a frame picture currently obtained by the user equipment; the previous frame picture includes a frame picture obtained before the current frame picture. In some embodiments, the frame picture to be processed includes a start frame picture, and the start frame picture needs to satisfy a start condition.

Modules 1 and 2 are used to determine, according to the image information in the current frame picture and the previous frame picture, whether the image information in the current frame picture is in the target anti-shake relative to the image information in the previous frame picture Offset information in the direction. In some embodiments, the image information includes picture content information of the current frame picture and the previous frame picture identified based on an image recognition technology (for example, the current frame picture or the previous frame picture table, chair, house, dog, etc. in the picture content information). In some embodiments, the user equipment determines, by comparing the position of the image information in the current frame picture and the position of the image information in the previous frame picture, that the image information in the current frame picture is relative to the image information in the previous frame picture. The offset information of the image information in the previous frame picture in the target anti-shake direction. In some embodiments, the target anti-shake direction includes, but is not limited to, landscape, portrait, landscape, and portrait. In some embodiments, the user equipment only compares the offset information of the current frame picture and the previous frame picture in the target anti-shake direction. In some embodiments, the offset information includes, but is not limited to, a specific offset value (eg, 5 cm, 6 cm, etc.). For example, the target anti-shake direction is horizontal. offset information. For example, if the target anti-shake direction is horizontal, and user A shakes his hand to the left by 5 cm when recording a video, the image information (for example, a dog) in the current frame picture obtained at this time is relative to that in the previous frame picture. In terms of image information (for example, the puppy), it is shifted to the right by 5 cm. The user equipment determines the offset information by comparing the image information in the current frame picture with the image information in the previous frame picture. In some embodiments, since the acquisition time of the current frame picture and the previous frame picture differs only by milliseconds, the image information of the current frame picture and the previous frame picture are almost the same, and the difference between the two is different. It may simply be a shift in the position of the image information due to jitter.

Modules 1 and 3 are configured to perform picture processing on the current frame picture according to the offset information to obtain the processed current frame picture. In some embodiments, the offset information includes an offset direction and an offset value (eg, a specific offset value). In some embodiments, the picture processing includes but is not limited to cropping and stitching. In some embodiments, the user equipment performs cropping and splicing of the current frame picture according to the offset information, so as to obtain the cropped and spliced current frame picture. For the specific picture processing process, please refer to the following embodiments, which will not be repeated here.

A fourth module, configured to combine the processed picture of the current frame and the picture of the previous frame according to the sequence of video frames to generate a recorded video. For example, the user equipment combines the processed picture of the current frame with the picture of the previous frame in real time, so that each frame of the picture in the recorded video formed by the combination is a processed picture.

For example, user A is photographing a puppy on the ground, and the obtained current frame picture and the previous frame picture both include the puppy. The puppy in the current frame picture and the puppy in the previous frame picture are identified based on the image recognition technology. Further, by comparing the position of the puppy in the current frame picture in the current frame picture, and the position of the puppy in the previous frame picture in the previous frame picture, determine that the puppy is in the target anti-shake direction. offset information. For example, the position of the dog in the current frame is moved upward by 5mm relative to the position of the previous frame, wherein the offset information includes an upward offset of 5mm. Further, the current frame picture is cropped and spliced according to the offset information, so that the image information in the current frame picture and the image information in the previous frame picture are located on the same horizontal line. For example, the puppy in the current frame picture is kept on the same horizontal line as the puppy in the previous frame picture (or the position does not shift). Combine the cropped and spliced picture of the current frame with the picture of the previous frame according to the sequence of video frames to generate a recorded video. For example, the recorded video is generated by combining frame-by-frame frame pictures, and the frame-by-frame frame pictures are combined in the order of video frames based on the time obtained during the recording to generate the recorded video.

In some embodiments, the one-to-one module is configured to acquire a frame picture to be processed in response to a user's recording operation, wherein the frame picture to be processed includes a start frame picture, the current frame picture, and the The picture of the previous frame of the picture of the current frame, the picture of the starting frame satisfies the starting condition.

Here, the specific implementation manner corresponding to the one-to-one modules is the same as or similar to the specific implementation manner of the step S11, and thus will not be repeated here, but is included here by reference.

In some embodiments, the apparatus further includes a five-module (not shown). A module 15 is configured to acquire a recording frame picture in response to the user's start-up recording operation; determine the starting frame picture from the recording frame picture, wherein the starting frame picture satisfies the starting condition . For example, when the user starts recording, the acquired first few frames of pictures are used as the recording frame pictures.

Here, the specific implementation manner corresponding to the one and five modules is the same as or similar to the specific implementation manner of the step S15, and thus will not be repeated, but is included here by reference.

In some embodiments, the initial condition includes that the inclination angle of the recorded frame picture is equal to or less than an angle threshold; the determining the initial frame picture from the recorded frame picture includes a six-module (not shown). out) and 17 modules. A six-module, for determining the inclination angle of the recorded frame picture; a seven-module, for if the inclination angle of the recorded frame picture is equal to or less than an angle threshold, the user equipment determines that the recorded frame picture satisfies the initial condition , and use the recording frame picture as the starting frame picture.

Here, the specific implementation manners corresponding to the one six modules and the one seven modules are the same as or similar to the specific implementation implementations of the steps S16 and S17 , and thus are not repeated here, but are incorporated herein by reference.

In some embodiments, the one-six modules are configured to determine the picture angle of the recorded frame picture, and use the picture angle as the inclination angle of the recorded frame picture.

Herein, the specific implementation manner corresponding to the six modules is the same as or similar to the specific implementation manner of the step S16, and thus will not be repeated, but is included herein by reference.

In other embodiments, the one-six modules are configured to determine the picture angle of the recorded frame picture, and use the angle sensor to determine the device angle of the user equipment; the user equipment calculates the picture angle and the device angle according to the picture angle and the device angle. The tilt angle of the recorded frame picture.

Herein, the specific implementation manner corresponding to the six modules is the same as or similar to the specific implementation manner of the step S16, and thus will not be repeated, but is included herein by reference.

In some embodiments, the determining the picture angle of the recorded frame picture includes: the user equipment recognizes image information of the recorded frame picture based on an image recognition technology, wherein the image information includes a target object; the user equipment determines the image information of the recorded frame picture. The included angle between the edge of the target object and the edge of the recorded frame picture is determined, and the included angle is determined as the picture angle of the recorded frame picture. For example, the target objects include target objects such as tables and buildings. The user equipment recognizes the target object based on an image recognition technology. and the included angle between the edge of the target object and the edge of the recorded frame picture is determined as the picture angle of the recorded frame picture.

In some embodiments, the picture angle corresponds to the first weight information; the device angle corresponds to the second weight information; the calculating the tilt angle of the recorded frame picture according to the picture angle and the device angle, The method includes: calculating and obtaining the inclination angle of the recorded frame picture according to the device angle, the inclination angle, the first weight information, and the second weight information. For example, different weights are assigned to the picture angle and the device angle, and the tilt angle of the recorded frame picture is calculated based on the different weights and the acquired specific picture angle and device angle. For example, the picture angle can better reflect the presentation state of the image information in the recorded frame picture. The first weight information of the picture angle is 0.3; the second weight information of the device angle is 0.1. For example, the picture angle is 3 degrees, the device angle is 5 degrees, 3*0.3+5*0.1=1.4.

In some embodiments, the device further includes an eighteen module (not shown), an eighteen module for acquiring anti-shake configuration information about the recorded video, wherein the anti-shake configuration information includes the target Anti-shake direction.

Here, the specific implementation manners corresponding to the eighth modules are the same as or similar to the specific implementation manners of the step S18 , and thus are not described again, but are incorporated herein by reference.

In some embodiments, the target anti-shake direction includes at least one of the following:

Horizontal direction; for example, when the user takes a sightseeing elevator to record a video, the target anti-shake direction includes horizontal direction.

Portrait orientation; for example, when a user walks to record a video, the target anti-shake direction includes portrait orientation.

Landscape and portrait; for example, when the user is still recording a video of a house, the target anti-shake direction includes landscape and portrait.

In some embodiments, the offset information includes an offset direction and an offset value; the one-three modules include a one-three-one module (not shown), a one-three-two module, and a one-three-three module. The one-three-one module is used to determine the cropping area of the current frame picture and the copy area of the previous frame picture according to the offset direction and the offset value respectively; the one-three-two module is used to copy all the The copy area of the previous frame picture, and cut out the crop area of the current frame picture; a three-three module is used for splicing the copy area to the splicing position of the cropped current frame picture, wherein the The orientation of the splicing position in the picture of the current frame is opposite to the orientation of the cropped region in the picture of the current frame.

Here, the specific implementations corresponding to the one-three-one modules, one-three-two modules, and one-three-three modules are the same as or similar to the specific implementations of the steps S131 , S132 and S133 , so they will not be repeated here, and will be cited for reference. way is included here.

In some embodiments, the determining the cropping region of the current frame picture according to the offset direction and the offset value includes: determining the current frame picture according to the offset direction and the offset value The cropping area, wherein the orientation of the cropping area in the current frame picture is opposite to the offset direction, the length of the cropping area is equal to the length or width of the current frame picture, the cropping area The width is equal to the offset value. For example, if the image information in the current frame picture is shifted to the left by 5mm relative to the image information in the previous frame picture, the right area of the current frame picture needs to be cropped, and the offset value is determined as the The width of the cropped area, where the length of the cropped area is equal to the width of the current frame picture. For another example, if the image information in the current frame picture is shifted upward by 5mm relative to the image information in the previous frame picture, the lower area of the current frame picture needs to be cropped, and the offset value is determined as the The width of the cropped area, where the length of the cropped area is equal to the length of the current frame picture.

In some embodiments, the determining the copy area of the previous frame picture according to the offset direction and the offset value includes: determining the previous frame according to the offset direction and the offset value a copy area of a frame picture, wherein the orientation of the copy area in the previous frame picture is the same as the offset direction, and the length of the copy area is equal to the length or width of the previous frame picture, The width of the copied area is equal to the offset value. For example, referring to FIG. 2 , FIG. 2( b ) is the picture of the current frame, FIG. 2( a ) is the picture of the previous frame, and FIG. 2( c ) is the picture of the current frame after cropping and splicing. The image information in the current frame picture is shifted upward by 5mm relative to the image information in the previous frame picture, and the length and width of the current frame picture and the previous frame picture are respectively 5cm*5cm. Then, it is determined that the cropping area of the current frame picture is an area with a bottom length of 5 cm and a width of 5 mm. An area with a length of 5 cm and a width of 5 mm in the upper part of the picture of the previous frame is determined as the copy area, the copy area is copied, and the copied area is spliced above the picture of the current frame. For another example, the image information in the current frame picture is shifted to the left by 5mm relative to the image information in the previous frame picture, and the length and width of the current frame picture and the previous frame picture are respectively 5cm*5cm. . Then, it is determined that the cropped area of the current frame picture is an area with a right length of 5 cm and a width of 5 mm. Determine the area with a length of 5 cm and a length of 5 mm on the left side of the previous frame picture as the copy area, copy the copy area, and splicing the copy area to the left side of the current frame picture.

In some embodiments, the one-fourth module is used to: combine the processed current frame picture and the previous frame picture according to the video frame sequence; during the recording process of the user, repeat the above operations , to generate a recorded video.

In addition to the methods and devices described in the above-mentioned embodiments, the present application also provides a computer-readable storage medium, where the computer-readable storage medium stores computer codes, when the computer codes are executed, as in any of the previous The described method is executed.

The present application also provides a computer program product, when the computer program product is executed by a computer device, the method according to any one of the preceding items is executed.

The present application also provides a computer device, the computer device comprising:

one or more processors;

memory for storing one or more computer programs;

The one or more computer programs, when executed by the one or more processors, cause the one or more processors to implement the method of any preceding item.

FIG. 4 illustrates an exemplary system that may be used to implement various embodiments described in this application;

In some embodiments, as shown in FIG. 4, system 300 can function as any of the devices in each of the described embodiments. In some embodiments, system 300 may include one or more computer-readable media (eg, system memory or NVM/storage device 320 ) having instructions and be coupled to the one or more computer-readable media and configured to execute Instructions to implement a module to perform one or more processors (eg, processor(s) 305 ) to perform the actions described herein.

For one embodiment, the system control module 310 may include any suitable interface controller to provide at least one of the processor(s) 305 and/or any suitable device or component in communication with the system control module 310 any appropriate interface.

The system control module 310 may include a memory controller module 330 to provide an interface to the system memory 315 . The memory controller module 330 may be a hardware module, a software module, and/or a firmware module.

System memory 315 may be used, for example, to load and store data and/or instructions for system 300 . For one embodiment, system memory 315 may include any suitable volatile memory, eg, suitable DRAM. In some embodiments, system memory 315 may include double data rate type quad synchronous dynamic random access memory (DDR4 SDRAM).

For one embodiment, system control module 310 may include one or more input/output (I/O) controllers to provide interfaces to NVM/storage device 320 and communication interface(s) 325 .

For example, NVM/storage device 320 may be used to store data and/or instructions. NVM/storage device 320 may include any suitable non-volatile memory (eg, flash memory) and/or may include any suitable non-volatile storage device(s) (eg, one or more hard drives ( HDD), one or more compact disc (CD) drives and/or one or more digital versatile disc (DVD) drives).

NVM/storage device 320 may include storage resources that are physically part of the device on which system 300 is installed, or it may be accessed by the device without necessarily being part of the device. For example, the NVM/storage device 320 is accessible via the communication interface(s) 325 over a network.

Communication interface(s) 325 may provide an interface for system 300 to communicate over one or more networks and/or with any other suitable device. System 300 may wirelessly communicate with one or more components of a wireless network in accordance with any of one or more wireless network standards and/or protocols.

For one embodiment, at least one of the processor(s) 305 may be packaged with the logic of one or more controllers of the system control module 310 (eg, the memory controller module 330 ). For one embodiment, at least one of the processor(s) 305 may be packaged with logic of one or more controllers of the system control module 310 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 305 may be integrated on the same die with the logic of one or more controllers of the system control module 310 . For one embodiment, at least one of the processor(s) 305 may be integrated on the same die with logic of one or more controllers of the system control module 310 to form a system on a chip (SoC).

In various embodiments, system 300 may be, but is not limited to, a server, workstation, desktop computing device, or mobile computing device (eg, laptop computing device, handheld computing device, tablet computer, netbook, etc.). In various embodiments, system 300 may have more or fewer components and/or different architectures. For example, in some embodiments, system 300 includes one or more cameras, keyboards, liquid crystal display (LCD) screens (including touchscreen displays), non-volatile memory ports, multiple antennas, graphics chips, application specific integrated circuits ( ASIC) and speakers.

It should be noted that the present application may be implemented in software and/or a combination of software and hardware, eg, an application specific integrated circuit (ASIC), a general purpose computer, or any other similar hardware device. In one embodiment, the software program of the present application may be executed by a processor to implement the steps or functions described above. Likewise, the software programs of the present application (including associated data structures) may be stored on a computer-readable recording medium, such as RAM memory, magnetic or optical drives or floppy disks, and the like. In addition, some steps or functions of the present application may be implemented in hardware, for example, as a circuit that cooperates with a processor to perform various steps or functions.

In addition, a part of the present application can be applied as a computer program product, such as computer program instructions, which when executed by a computer, through the operation of the computer, can invoke or provide methods and/or technical solutions according to the present application. Those skilled in the art should understand that the existing forms of computer program instructions in computer-readable media include but are not limited to source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include but are not limited to Limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding post-installation program. program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium that can be accessed by a computer.

Communication media includes media by which communication signals containing, for example, computer readable instructions, data structures, program modules or other data are transmitted from one system to another. Communication media may include conducted transmission media such as cables and wires (eg, fiber optic, coaxial, etc.) and wireless (unconducted transmission) media capable of propagating energy waves, such as acoustic, electromagnetic, RF, microwave, and infrared . Computer readable instructions, data structures, program modules or other data may be embodied, for example, as a modulated data signal in a wireless medium such as a carrier wave or similar mechanism such as embodied as part of spread spectrum technology. The term "modulated data signal" refers to a signal whose one or more characteristics are altered or set in a manner that encodes information in the signal. Modulation can be analog, digital or hybrid modulation techniques.

By way of example and not limitation, computer-readable storage media may include volatile and non-volatile, readable storage media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Removable and non-removable media. For example, computer-readable storage media include, but are not limited to, volatile memory, such as random access memory (RAM, DRAM, SRAM); and non-volatile memory, such as flash memory, various read-only memories (ROM, PROM, EPROM) , EEPROM), magnetic and ferromagnetic/ferroelectric memory (MRAM, FeRAM); and magnetic and optical storage devices (hard disks, tapes, CDs, DVDs); or other media now known or later developed capable of storing data for computer systems Computer readable information/data used.

Here, an embodiment according to the present application includes an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein, when the computer program instructions are executed by the processor, a trigger is The apparatus operates based on the aforementioned methods and/or technical solutions according to various embodiments of the present application.

It will be apparent to those skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, but that the present application may be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Accordingly, the embodiments are to be regarded in all respects as illustrative and not restrictive, and the scope of the application is to be defined by the appended claims rather than the foregoing description, which is therefore intended to fall within the scope of the claims. All changes within the meaning and scope of the equivalents of , are included in this application. Any reference signs in the claims shall not be construed as limiting the involved claim. Furthermore, it is clear that the word "comprising" does not exclude other units or steps and the singular does not exclude the plural. Several units or means recited in the device claims can also be realized by one unit or means by means of software or hardware. The terms first, second, etc. are used to denote names and do not denote any particular order.

Claims (15)

1. A video anti-shake method, applied to a user equipment side, wherein the method comprises: In response to the user's recording operation, a to-be-processed frame picture is obtained, wherein the to-be-processed frame picture includes a start frame picture, a current frame picture, and a previous frame picture of the current frame picture, and the start frame picture satisfies a starting condition, the starting condition includes that the image information of the starting frame picture is in a natural arrangement state; Determine the offset information of the image information in the current frame picture relative to the image information in the previous frame picture in the target anti-shake direction according to the image information in the current frame picture and the previous frame picture , wherein the acquisition time difference between the current frame picture and the previous frame picture is in milliseconds, and the acquisition time difference between the current frame picture and the previous frame picture is used to determine the offset information; Perform picture processing on the current frame picture according to the offset information to obtain the processed current frame picture; The processed picture of the current frame and the picture of the previous frame are combined according to the sequence of video frames to generate a recorded video, wherein the picture of the starting frame is used as the first picture of the recorded video. 2. The method of claim 1, wherein the method further comprises: In response to the user's start-up recording operation, obtain a recording frame picture; The starting frame picture is determined from the recorded frame picture, wherein the starting frame picture satisfies the starting condition. 3. The method according to claim 2, wherein the initial condition comprises that the inclination angle of the recorded frame picture is equal to or less than an angle threshold; The determining the starting frame picture from the recorded frame picture includes: determining the inclination angle of the recorded frame picture; If the inclination angle of the recorded frame picture is equal to or smaller than the angle threshold, it is determined that the recorded frame picture satisfies the initial condition, and the recorded frame picture is used as the initial frame picture. 4. The method according to claim 3, wherein the determining the inclination angle of the recorded frame picture comprises: The picture angle of the recorded frame picture is determined, and the picture angle is used as the inclination angle of the recorded frame picture. 5. The method according to claim 3, wherein the determining the inclination angle of the recorded frame picture comprises: Determine the picture angle of the recorded frame picture, and use the angle sensor of the device angle of the user equipment; The tilt angle of the recorded frame picture is calculated according to the picture angle and the device angle. 6. The method according to claim 4 or 5, wherein the determining the picture angle of the recorded frame picture comprises: Identify the image information of the recorded frame picture based on image recognition technology, wherein the image information includes a target object; Determine the included angle between the edge of the target object and the edge of the recorded frame picture, and determine the included angle as the picture angle of the recorded frame picture. 7. The method according to claim 5, wherein the picture angle corresponds to the first weight information; the device angle corresponds to the second weight information; The calculating the tilt angle of the recorded frame picture according to the picture angle and the device angle includes: The inclination angle of the recorded frame picture is obtained by calculating according to the device angle, the inclination angle, the first weight information, and the second weight information. 8. The method of claim 1, wherein the method further comprises: Obtain anti-shake configuration information about the recorded video, where the anti-shake configuration information includes the target anti-shake direction. 9. The method according to claim 1 or 8, wherein the target anti-shake direction comprises at least one of the following: horizontal; vertical; Landscape and portrait. 10. The method of claim 1, wherein the offset information includes an offset direction and an offset value; Performing picture processing on the current frame picture according to the offset information to obtain the processed current frame picture, including: Determine the cropping area of the current frame picture and the copy area of the previous frame picture according to the offset direction and the offset value respectively; Copy the copying area of the picture of the previous frame, and cut out the cropping area of the picture of the current frame; The copy area is spliced to the splicing position of the cropped current frame picture, wherein the stitching position is at the orientation of the current frame picture and the position of the cropped area in the current frame picture. Relative orientation. 11. The method according to claim 10, wherein the determining the cropping region of the current frame picture according to the offset direction and the offset value comprises: The cropping area of the current frame picture is determined according to the offset direction and the offset value, wherein the direction of the cropping area in the current frame picture is opposite to the offset direction, and the cropping area is opposite to the offset direction. The length of the region is equal to the length or width of the current frame picture, and the width of the cropped region is equal to the offset value. 12. The method according to claim 11, wherein the determining the copy area of the picture of the previous frame according to the offset direction and the offset value comprises: The copy area of the previous frame of picture is determined according to the offset direction and the offset value, wherein the position of the copy area in the previous frame of picture is the same as the offset direction, so The width of the copy area is equal to the width of the previous frame picture, and the height of the copy area is equal to the offset value. 13. The method according to claim 1, wherein the combining the processed current frame picture and the previous frame picture according to the video frame sequence to generate a recorded video, comprising: combining the processed current frame picture and the previous frame picture according to the video frame sequence; During the recording process of the user, the above operations are repeatedly performed to generate a recording video. 14. A video anti-shake device, wherein the device comprises: processor; and a memory arranged to store computer-executable instructions which, when executed, cause the processor to perform the operations of the method of any of claims 1 to 13. 15. A computer readable medium storing instructions that, when executed, cause a system to perform operations of the method of any of claims 1 to 13.

Images