CN102098527B - Method and device for transforming two dimensions into three dimensions based on motion analysis - Google Patents

Abstract

The invention discloses a plane-to-stereo method based on motion analysis, comprising the following steps: inputting a single image, and preprocessing the single image; reading each frame of the single image as the current frame, and The heterosexuality detection selects its adjacent n frames as frames to be matched; obtains a motion vector according to the frame to be matched and the current frame, and uses the motion vector to perform motion analysis and judgment to obtain a final matching frame; the current frame and the current frame performing image transformation on the final matching frame to obtain a first stereoscopic input image and a second stereoscopic input image; and performing stereoscopic synthesis on the first stereoscopic input image and the second stereoscopic input image to obtain a stereoscopic video, and outputting the stereoscopic video. The motion-analysis-based plane-to-stereo method of the embodiment of the present invention has the characteristics of wide application range, fast calculation speed, high image and video quality, and good stereo effect. The invention also discloses a plane-to-stereo device based on motion analysis.

Description

A method and device for converting plane to stereo based on motion analysis

technical field

The invention relates to the field of computer vision processing, in particular to a method and device for converting plane to stereo based on motion analysis.

Background technique

Stereoscopic video technology, as the development direction of future multimedia technology, is a new type of video technology that can provide stereoscopic effect. Compared with single-channel video, stereoscopic video generally has two video channels, and the amount of data is much larger than single-channel video, so efficient compression of stereoscopic video is particularly important. Specifically, the stereoscopic video not only contains the surface information about the scene of the traditional two-dimensional video, but also contains the three-dimensional information related to the specific position of the scene. Compared with traditional two-dimensional video, stereoscopic video is a more effective and realistic way of expression, which overcomes the shortcomings of one-sidedness and passivity of two-dimensional video, and can more fully meet people's visual sensory needs. Free Viewpoint Video (FVV), virtual reality, 3DTV, 3D games, live sports, advertising media and many other fields have broad application prospects.

In stereoscopic video technology, stereoscopic perception is achieved through binocular stereovision. Specifically, binocular stereo vision uses the principle of binocular imaging, and by simulating binocular perception imaging, the left and right images or videos are used as the input of the left and right eyes, and then through the fusion of the brain, the input two images and videos are reconstructed stereoscopic scenes to achieve a stereoscopic perception effect. The binocular imaging principle is based on the principle of binocular perception of the world. When the left and right eyes watch a specific scene, two left and right imaging images with a certain parallax are formed on the retinas of the left and right eyes, and through the fusion of the brain, the Construct an actual scene with a sense of front and rear layers.

With the rapid development of stereoscopic display technology, the lack of stereoscopic content seriously restricts the development of the entire stereoscopic video industry, and the existing stereoscopic video technology has a complex processing process and low imaging quality, and the overall stereoscopic display effect is poor.

Contents of the invention

The purpose of the present invention is to at least solve one of the above-mentioned technical defects, and especially propose a method and device for converting plane to stereo based on motion analysis. High quality and good three-dimensional effect.

In order to achieve the above-mentioned purpose, the embodiment of the first aspect of the present invention proposes a plane-to-stereo method based on motion analysis, including the following steps:

input a single-channel image, and preprocess the single-channel image;

Reading each frame of image in the single-channel image as a current frame, and selecting n frames adjacent to the current frame as frames to be matched by dissimilarity detection;

Calculating a motion vector according to the frame to be matched and each frame of the single-channel image, and using the motion vector to perform motion analysis and judgment to obtain a final matching frame;

performing image transformation on the current frame and the matching frame to obtain a first stereo input image and a second stereo input image; and

Stereoscopic synthesis is performed on the first stereoscopic input image and the second stereoscopic input image to obtain a stereoscopic video, and the stereoscopic video is output.

The plane-to-stereo method based on motion analysis according to the embodiment of the present invention has the following advantages:

(1) Wide application range: The motion analysis-based plane-to-stereo method of the embodiment of the present invention has different solutions for different image or video input conditions, thus having a relatively large application range.

(2) Fast calculation speed: the total amount of calculation is reduced through the selection method of adjacent frames and the block sampling method, and the motion analysis and judgment algorithm adopted by the present invention is relatively simple and practical and has high processing efficiency.

(3) High image and video quality: Since both the current frame and the matching frame come from the image sequence or the video itself, and the image transformation of the current frame and the matching frame is based on the preset area or the entire image, there is no image and video The loss of its own quality makes the image and video quality high.

(4) Good stereoscopic effect: Linear, nonlinear and image transformation methods based on preset areas make the stereoscopic video after stereoscopic synthesis have obvious stereoscopic effects, without obvious depth perception errors and frame-to-frame jitter.

The embodiment of the second aspect of the present invention proposes a stereoscopic image device, including: a single-channel input and preprocessing module for inputting a single-channel image and performing preprocessing on the single-channel image; a frame selection module to be matched, For each current frame in the single-channel image from the single-channel input and preprocessing module, select its adjacent n frames as the frame to be matched by dissimilarity detection; the motion analysis and judgment module is used for according to the The frame to be matched and each frame of the single image in the frame to be matched selection module obtain a motion vector, and use the motion vector to perform motion analysis and judgment to obtain a final matching frame; the image conversion module, For performing image transformation on the current frame and the matching frame from the motion analysis and judgment module to obtain a first stereo input image and a second stereo input image; and a stereo synthesis module, used to perform image transformation from the image transformation module performing stereo synthesis on the first stereo input image and the second stereo input image to obtain a stereo video, and outputting the stereo video.

According to the motion analysis-based plane-to-stereo device of the embodiment of the present invention, there are different solutions for different image or video input conditions, thus having a large application range, and reducing the total amount of calculation through the selection of adjacent frames and block sampling, At the same time, the motion analysis and judgment algorithm adopted by the motion analysis and judgment module of the present invention is simple, practical and has high processing efficiency. And because both the current frame and the matching frame come from the image sequence or the video itself, and the image transformation of the current frame and the matching frame is based on the preset area or the entire image, so there is no loss of image and video quality, so that the image and the matching frame The video quality is high. In addition, the linear, non-linear and image transformation based on the preset area applied in the present invention makes the stereoscopic video after the stereoscopic synthesis have obvious stereoscopic effect, so that there is no obvious depth perception error and frame-to-frame jitter.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and easy to understand from the following description of the embodiments in conjunction with the accompanying drawings, wherein:

FIG. 1 is a flow diagram of a motion analysis-based plane-to-stereo method according to an embodiment of the present invention; and

Fig. 2 is a structural block diagram of a motion analysis-based 2D-to-3D device according to an embodiment of the present invention.

Detailed ways

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

As shown in Figure 1, according to the embodiment of the present invention, the plane-to-stereo method based on motion analysis includes the following steps:

S101: Input a single-channel image, and perform preprocessing on the single-channel image;

First, input a single image, where the single image can be one of the following: single planar image, single image sequence or single video. Among them, the image format of single-channel plane image or single-channel image sequence can be RAW, BMP, PCX, TIFF, GIF, JPEG, TGA, EXIF, etc.; the video format of single-channel video can be MPEG, AVI, MOV, ASF, WMV , MKV, DIVX, Real Video, etc. Of course, those skilled in the art can understand that the single-channel planar image or single-channel image sequence can also be in other image formats, and the single-channel video can also be in other video formats. The input single-channel image is then preprocessed.

For the preprocessing of single-channel video, image coding technology is used to decode the video. In one embodiment of the present invention, a decoder is used to decode the input single-channel video, and output a code stream of YUV series or RGB series.

The preprocessing process also includes extracting image pixel values from the input single-channel image. Specifically, each frame in a single planar image, a single image sequence, or a single video decoding stream is read, and then pixel values in the corresponding color space are extracted.

For the RGB space, the extracted image pixel value can be one of the following three types: a triple array of RGB three-channel values; the average value of the RGB three-channel values; the weighted sum of the RGB three-channels with a certain proportional coefficient value. Among them, when the extracted image pixel value is the weighted value obtained by superimposing and summing the RGB three channels with a certain proportional coefficient, it is calculated by the following formula,

Gray＝R*0.3+G*0.59+B*0.11,

Among them, Gray is the weighted value. Of course, those skilled in the art can understand that when other coefficients are used for weighting to obtain weighted values, it also belongs to the protection scope of the present invention.

For the YUV space, the extracted image pixel values can be divided into one of the following three types: the Y component; the average value of the three YUV channels; the weighted value obtained by superimposing and summing the three YUV channels with a certain proportional coefficient.

S102: For each frame of image in the single-channel image, select an adjacent frame of the current frame as the frame to be matched;

Due to the use of the principle of double-sided imaging in the double-sided stereoscopic vision, two images on the left and right are required as initial input. Therefore, in addition to the input current frame, matching frames corresponding to the current frame are also required as two inputs.

When the single-channel image is a single-channel planar image, the matching frame is the current frame.

When the single-channel image is a single-channel image sequence or a single-channel video, it is necessary to obtain the frame to be matched before determining the matching frame. Specifically, for each frame of images in the single-channel image, it is necessary to select an initial sequence of frames to be matched from adjacent frames of the current frame. Wherein, the sequence of initial frames to be matched includes a plurality of initial frames to be matched. Wherein, the initial frame sequence to be matched is the first h frames and the last h frames of the current frame on the time axis. In one embodiment of the present invention, h may be 5. Of course, those skilled in the art can understand that h can also be other values.

In order to determine the dissimilarity between the current frame and the initial frame sequence to be matched, after obtaining the initial frame sequence to be matched, it is necessary to perform a dissimilarity detection on each initial frame to be matched in the initial frame sequence to be matched to obtain the frame sequence to be matched, including Follow the steps below:

A1: First, a horizontal midline segmentation and a vertical midline segmentation are performed on the current frame and an initial frame to be matched at the same time, so that the current frame and the initial frame to be matched are divided into four image blocks, and the four image blocks after segmentation The length and width are 1/2 of the length and width of the original image frame;

A2: Subtract the pixel values of the four image blocks corresponding to the current frame and the initial frame to be matched, and take the absolute value of the difference as the new pixel value to obtain a new image block, and calculate the four new image blocks Take the pixel mean and variance;

A3: Threshold judgment is performed on the pixel mean and variance of the four new image blocks respectively. When the number of the mean and variance exceeding the preset threshold in the four new image blocks is less than or equal to the threshold of the number of dissimilarities, the initial matching is judged The frame is the frame to be matched of the current frame; when the number of the four new image blocks whose mean value or variance exceeds the preset threshold is greater than the threshold of the number of dissimilarities, it is judged that the initial frame to be matched is not the frame to be matched of the current frame. Wherein, the preset threshold includes a pixel mean threshold and a variance threshold. In an embodiment of the present invention, the pixel mean value and variance threshold can be any value within [20, 30]. Wherein, the threshold of the number of dissimilarities may be 3.

Steps A1 to A3 are repeated, and each initial frame to be matched in the sequence of initial frames to be matched is subjected to a dissimilarity detection calculation with the current frame until all initial frames to be matched are subjected to dissimilarity detection to obtain n frames to be matched. where n is a positive integer.

S103: Obtain a motion vector, and use the motion vector to perform motion analysis and judgment to obtain a matching frame;

Since the matching frame of the single-channel planar graphics is the current frame, motion analysis and judgment are not required.

In this step, when the input is a single-channel image sequence or a single-channel video, motion detection is required to determine the motion form and size of the scene. Specifically, the motion vector is obtained for the frame to be matched and the current frame of the single-channel image sequence or single-channel video, and the motion analysis and judgment are performed using the motion vector to obtain the matching frame, including the following steps:

In the first step, the image data of the current frame is divided into blocks according to a predetermined size, and the divided image is uniformly sampled according to a predetermined sampling number to obtain multiple sampling blocks;

First, divide the image data into blocks with a predetermined size according to the size of the current frame image and the size of the blocks. Wherein, the size of each image block is preset. In an embodiment of the present invention, for a 640*480 image, the size of each image block is 4*4 pixels. Then uniform sampling of data blocks is performed on the divided image to obtain multiple sampling blocks. Wherein, the number of sampling points is preset. In one embodiment of the present invention, the number of sampling points is about 5% of the total number of image pixels.

In the second step, the motion offset between each sampling block in the current frame and the matching block in each frame to be matched is used as the motion vector of the sampling block corresponding to the frame to be matched;

For each sampling block in the current frame, search for the most similar block in a frame to be matched. Wherein, the search method includes full search, three-step search, logarithmic search or diamond search algorithm and the like. After the search is finished, the motion offset between each sample block in the current frame and the matching block in the frame to be matched is used as a motion vector of the sample block in the current frame corresponding to the frame to be matched. Through the above method, n motion vector graphics can be obtained, where n is a positive integer.

The third step is to carry out motion analysis and judgment to n vector graphics to obtain the quantity of zero motion vector graphics and the quantity of non-zero motion vector graphics;

It is judged whether there is a zero motion vector diagram among the obtained n motion vector diagrams. In one embodiment of the present invention, when the number of sample blocks corresponding to zero vector accounts for more than 90% of the total number of sample blocks, it is judged that the motion vector diagram is a zero motion vector diagram. Wherein, the number of zero-motion vector diagrams is m. Of course, those skilled in the art can understand that when the ratio of the number of sampling blocks corresponding to zero vectors to the total number of sampling blocks is other reasonable ratios, it also falls within the protection scope of the present invention.

According to the m zero motion vector diagrams obtained above, it can be known that the number of non-zero motion vector diagrams is n-m. A horizontal motion vector analysis is then performed on the non-zero motion vector diagrams. Non-zero motion vector diagrams include motion vector diagrams corresponding to foreground horizontal motion, motion vector diagrams corresponding to camera horizontal movement, and motion vector diagrams corresponding to remaining situations. The motion vector diagrams corresponding to the remaining situations are motion vector diagrams other than the motion vector diagram corresponding to the foreground horizontal motion and the motion vector diagram corresponding to the camera horizontal movement.

When, among the sampling blocks corresponding to the non-zero motion vector diagram, the sampling blocks whose motion vector direction is in the horizontal direction account for 70% to 90% of the non-zero vector sampling blocks, it is determined that the foreground is moving horizontally. Wherein, the number of motion vector diagrams corresponding to the foreground horizontal motion is p. Of course, those skilled in the art can understand that when the ratio of the sampling blocks whose motion vector direction is in the horizontal direction to the non-zero vector sampling blocks is other reasonable ratios, it also falls within the protection scope of the present invention.

When among the sampling blocks corresponding to the non-zero motion vector diagram, the sampling blocks whose motion vector direction is in the horizontal direction account for more than 90% of the non-zero vector sampling blocks, it is determined that the camera is moving horizontally. Wherein, the number of motion vector diagrams corresponding to the horizontal movement of the camera is q. Of course, those skilled in the art can understand that when the ratio of the sampling blocks whose motion vector direction is in the horizontal direction to the non-zero vector sampling blocks is other reasonable ratios, it also falls within the protection scope of the present invention.

It can be seen that the number of motion vector diagrams corresponding to the remaining cases is n-m-p-q.

Next, motion analysis and judgment are performed on the n-m non-zero motion vector diagrams. First, average the non-zero motion vectors of each non-zero motion vector diagram to obtain the average value A(i) of the motion vectors of each non-zero motion vector diagram. Among them, A is the average value, and i is a non-zero vector label. If the average value of the non-zero motion vector corresponding to the non-zero motion vector diagram is within the average threshold, the non-zero motion vector diagram is retained; otherwise, the non-zero motion vector diagram is discarded and the corresponding frame to be matched is discarded.

In one example of the present invention, the mean threshold is in the range of [5, 30] pixel distances. Of course, those skilled in the art can understand that, when the average threshold value is in other reasonable setting ranges, it also falls within the protection scope of the present invention.

Through the above motion analysis and judgment, it can be obtained that the number of zero-motion vector diagrams meeting the requirements is v1, the number of motion vector diagrams corresponding to foreground horizontal motion is v2, the number of motion vector diagrams corresponding to camera horizontal movement is v3, and the remaining cases correspond to The number of motion vector graphics for v4.

The fourth step is to obtain matching frames according to the number of zero motion vectors and the number of non-zero motion vectors.

(1) When the number v1 of the zero-motion vector diagram is greater than the first threshold R(v1), then it is judged that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a static scene, and all zero-motion vector graphics corresponding to the scene to be matched Select the farthest frame on the time axis from the current frame as the matching frame.

In an embodiment of the present invention, the first threshold R(v1) may be set to max(1, n-2), that is, v1>max(1, n-2). Of course, those skilled in the art can understand that when the first threshold is set to other reasonable values, it also falls within the protection scope of the present invention.

(2) When the number v2 of the motion vector diagram corresponding to the foreground horizontal motion is within the second threshold R(v2), it is judged that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a foreground motion scene, and v2 foreground levels are selected The motion vector map corresponding to the foreground horizontal motion with the smallest difference between the average value A(v2) of the motion vector map corresponding to the motion and the foreground standard preset threshold R (a2) is the motion vector graph corresponding to the final foreground horizontal motion, and the final foreground is selected The frame to be matched corresponding to the motion vector diagram corresponding to the horizontal motion is the matching frame.

In one embodiment of the present invention, R(v2)=[n/3,n], R(a2)=15. Of course, those skilled in the art can understand that when the second threshold R(v2) is set to other reasonable values, it also falls within the protection scope of the present invention.

(3) When the number v3 of the motion vector diagram corresponding to the horizontal movement of the camera is greater than the third threshold R(v3), it is judged that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a camera movement scene, and v3 cameras are selected to move horizontally The motion vector diagram corresponding to the foreground horizontal movement with the smallest difference between the average value A(v3) of the corresponding motion vector diagram and the camera standard preset threshold R(a3) is the motion vector diagram corresponding to the final camera horizontal movement, and the final camera horizontal movement is selected as The frame to be matched corresponding to the motion vector diagram corresponding to the movement is the matching frame.

In one embodiment of the present invention, R(v3)=[n/2,n], R(a3)=10. Of course, those skilled in the art can understand that when the third threshold R(v3) is set to other reasonable values, it also falls within the protection scope of the present invention.

(4) When the number v4 of the motion vector diagrams corresponding to the remaining situations is greater than the fourth threshold R(v4), then it is judged that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a scene other than the horizontal movement of the foreground and the horizontal movement of the camera Scenes. Select the motion vector diagram corresponding to the remaining situation with the smallest difference between the average A(v4) of the motion vector diagrams corresponding to v4 remaining situations and the standard preset threshold value R(a4) of the remaining situations as the motion vector diagram corresponding to the final remaining situation, The frame to be matched corresponding to the motion vector diagram corresponding to the final remaining situation is selected as the matching frame.

In one embodiment of the present invention, R(v4)=[n-2,n], R(a4)=5. Of course, those skilled in the art can understand that when the fourth threshold R(v4) is set to other reasonable values, it also falls within the protection scope of the present invention.

In one embodiment of the present invention, when none of the above four conditions are met, for example, a single planar image is input, the current frame is selected as the frame to be matched.

S104: For a foreground moving scene, obtain a disparity map according to the matching frame and the current frame, and update the matching frame according to the disparity map and the current frame;

When the scene corresponding to the current frame of a single-channel video or single-channel image sequence is a foreground motion scene, it is necessary to obtain a disparity map based on the matching frame and the current frame, and then obtain a new matching frame through the disparity map and the current frame, that is, update the matching frame . Wherein, the disparity map is obtained by one of the following algorithms, including an optical flow method based on pixel block operations, a block matching method, a depth matching method based on pixel point operations, and the like.

In one embodiment of the present invention, utilize current frame and disparity map to obtain new matching frame by DIBR (Depth Image BasedRendering) method.

S105: Perform image transformation on the current frame and the matching frame to obtain a first stereo input image and a second stereo input image;

When the single-channel image is a single-channel plane image or a single-channel video or a single-channel image sequence, and the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a static scene or a camera moving scene or a remaining situation, the current frame and Matching frames for image transformation to obtain a first stereoscopic input image and a second stereoscopic input image.

When the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a foreground motion scene, image transformation is performed on the current frame and the updated matching frame to obtain the first stereoscopic input image and the second stereoscopic input image.

Image transformation includes the following methods: linear transformation, nonlinear transformation and preset area transformation.

Linear transformation refers to the linear image transformation of the entire image only for the current frame or the matching frame. In an embodiment of the present invention, the linear transformation may be translation transformation, stretch transformation, compression transformation, or perspective transformation.

Non-linear transformation refers to image rotation transformation around X, Y, and Z axes, and a preset transformation operation for each row or column or pixel of the image.

Preset area transformation means that by presetting the area that the user is interested in, the part of the preset area is mainly linearly transformed or nonlinearly transformed, while the area outside the preset area is not transformed, or the area outside the preset area is not transformed. The area is transformed with a smaller transformation range than the preset area transformation.

In an embodiment of the present invention, the image transformation between the current frame and the matching frame can be a single type or a combined type, and only one frame of the image can be transformed, and the other frame of image can not be transformed.

S106: Perform stereoscopic synthesis on the first stereoscopic input image and the second stereoscopic input image to obtain a stereoscopic video, and output the stereoscopic video.

In one embodiment of the present invention, the first stereoscopic input image is used as the initial input of the left view, that is, the current frame is used as the initial input of the left view; the second stereoscopic input image is used as the initial input of the right view, corresponding to viewing with the left eye; That is, the matching frame is used as the initial input of the right view, which corresponds to viewing with the right eye. Of course, those skilled in the art can understand that using the current frame as the initial input of the right view and using the matching frame as the initial input of the left view also falls within the protection scope of the present invention.

Corresponding to different display terminals, different stereoscopic video synthesis formats and stereoscopic video output methods are used to perform stereoscopic synthesis of two input images. Wherein, synthesizing the two input images according to the stereoscopic video synthesizing format includes performing appropriate synthesizing on the left and right input images to obtain a single stereoscopic video output adapting to different stereoscopic display requirements. Wherein, the stereoscopic video synthesis format includes checkerboard format synthesis and horizontal interlaced format synthesis, and the like.

The stereoscopic video output method includes arranging the left and right input images side by side horizontally and vertically, or using the left and right input images as two output channels. In an embodiment of the present invention, the output stereoscopic video after synthesis may be a single-channel output display of stereoscopic synthesis of red-green, red-cyan, red-blue, etc.

The plane-to-stereo method based on motion analysis according to the embodiment of the present invention has the following advantages:

(1) Wide application range: The motion analysis-based plane-to-stereo method of the embodiment of the present invention has different solutions for different image or video input conditions, thus having a relatively large application range.

(2) Fast calculation speed: the total amount of calculation is reduced through the selection method of adjacent frames and the block sampling method, and the motion analysis and judgment algorithm adopted by the present invention is relatively simple and practical and has high processing efficiency.

(3) High image and video quality: Since both the current frame and the matching frame come from the image sequence or the video itself, and the image transformation of the current frame and the matching frame is based on the preset area or the entire image, there is no image and video The loss of its own quality makes the image and video quality high.

(4) Good three-dimensional effect: the linear, non-linear and image transformation based on the preset area of the embodiment of the present invention makes the three-dimensional video after stereo synthesis have obvious three-dimensional effect, without obvious depth perception error and frame-to-frame jitter.

Referring to FIG. 2 , a motion analysis-based 2D-to-3D device 200 according to an embodiment of the present invention will be described below.

As shown in FIG. 2 , according to the embodiment of the present invention, the device 200 for converting plane to stereo based on motion analysis includes a single-channel input and preprocessing module 210, wherein the single-channel input and pre-processing module 210 is used to input a single-channel image, and The frame to be matched is selected by the frame selection module 220, wherein the frame to be matched is selected by the frame to be matched for each current frame in the single-channel image from the single-channel input and the pre-processing module 210, and its neighbors are selected by the difference detection n frames are used as frames to be matched; motion analysis and judgment module 230, wherein motion analysis and judgment module 230 are used to obtain the motion vector according to the frame to be matched and the current frame from the frame to be matched selection module 220, and use the motion vector to carry out motion analysis and Judging to obtain the matching frame; image transformation module 250, wherein the image transformation module 250 is used to perform image transformation on the current frame and the matching frame from the motion analysis and judgment module 230 to obtain the first stereoscopic input image and the second stereoscopic input image; stereoscopic The synthesis module 260, wherein the stereo synthesis module 260 is used to perform stereo synthesis on the first stereo input image and the second stereo input image from the image transformation module 250 to obtain a stereo video, and output the stereo video.

In an embodiment of the present invention, the single-channel image input by the single-channel input and preprocessing module 210 may be one of the following: a single-channel planar image, a single-channel image sequence or a single-channel video. Among them, the image format of single-channel plane image or single-channel image sequence can be RAW, BMP, PCX, TIFF, GIF, JPEG, TGA, EXIF, etc.; the video format of single-channel video can be MPEG, AVI, MOV, ASF, WMV , MKV, DIVX, Real Video, etc. Of course, those skilled in the art can understand that the single-channel planar image or single-channel image sequence can also be in other image formats, and the single-channel video can also be in other video formats. The input single-channel image is then preprocessed.

For the preprocessing of single-channel video, the single-channel input and preprocessing module 210 uses image coding technology to decode the video. In one embodiment of the present invention, the single-channel input and preprocessing module 210 uses a decoder to decode the input single-channel video, and outputs a code stream of YUV series or RGB series.

The preprocessing by the single-channel input and preprocessing module 210 also includes extracting image pixel values from the input single-channel image. Specifically, the single-channel input and preprocessing module 210 reads each frame in a single-channel planar image, a single-channel image sequence or a single-channel video decoding stream, and then extracts pixel values in a corresponding color space.

For the RGB space, the image pixel value extracted by the single-channel input and preprocessing module 210 can be one of the following three types: a ternary array of RGB three-channel values; the average value of the RGB three-channel values; The weighted value obtained by superimposing and summing the coefficients. Among them, when the extracted image pixel value is the weighted value obtained by superimposing and summing the RGB three channels with a certain proportional coefficient, it is calculated by the following formula,

Gray＝R*0.3+G*0.59+B*0.11,

Among them, Gray is the weighted value. Of course, those skilled in the art can understand that when other coefficients are used for weighting to obtain weighted values, it also belongs to the protection scope of the present invention.

For the YUV space, the image pixel values extracted by the single input and preprocessing module 210 can be divided into one of the following three types: Y component; the average value of the YUV three channels; weighted value.

Due to the use of the principle of double-sided imaging in the double-sided stereoscopic vision, two images on the left and right are required as initial input. Therefore, in addition to the input current frame, matching frames corresponding to the current frame are also required as two inputs.

When the single-channel image is a single-channel planar image, the matching frame is the current frame.

When the single-channel image is a single-channel image sequence or a single-channel video, the frame-to-be-matched selection module 220 needs to obtain the frame-to-be-matched frame before determining the matched frame. Specifically, for each frame of images in the single-channel image, the frame-to-be-matched selection module 220 needs to select an initial sequence of frames to be matched from adjacent frames of the current frame. Wherein, the sequence of initial frames to be matched includes a plurality of initial frames to be matched. Wherein, the initial frame sequence to be matched is the first h frames and the last h frames of the current frame on the time axis. In one embodiment of the present invention, h may be 5. Of course, those skilled in the art can understand that h can also be other values.

In order to determine the dissimilarity between the current frame and the initial to-be-matched frame sequence, after obtaining the initial to-be-matched frame sequence, the to-be-matched frame selection module 220 needs to perform a dissimilarity detection on each initial to-be-matched frame in the initial to-be-matched frame sequence to obtain The frame sequence to be matched includes the following steps:

A1: The frame to be matched selection module 220 firstly performs a horizontal midline segmentation and a vertical midline segmentation on the current frame and an initial frame to be matched, so that the current frame and the initial frame to be matched are respectively divided into four image blocks. The length and width of the four image blocks are 1/2 of the length and width of the original image frame;

A2: The frame to be matched selection module 220 subtracts the image block pixel values from the four image blocks corresponding to the current frame and the initial frame to be matched, and takes the absolute value of the difference as a new pixel value to obtain a new image block. Calculate the pixel mean and variance of the four updated image blocks;

A3: The to-be-matched frame selection module 220 performs threshold judgment on the pixel mean and variance of the four new image blocks, and when the number of the four new image blocks with the mean and variance exceeding the preset threshold is less than or equal to the dissimilarity quantity threshold, Then it is judged that the initial frame to be matched is the frame to be matched of the current frame; when the number of the four updated image blocks whose mean value or variance exceeds the preset threshold is greater than the threshold of the number of dissimilarities, it is judged that the initial frame to be matched is not the current frame Frames to match. Wherein, the preset threshold includes a pixel mean threshold and a variance threshold. In an embodiment of the present invention, the pixel mean threshold and variance threshold can be any value within [20, 30]. Wherein, the threshold value of the number of dissimilarities may be 3. Steps A1 to A3 are repeated, and the to-be-matched frame selection module 220 performs a dissimilarity detection calculation on each initial to-be-matched frame in the initial to-be-matched frame sequence with the current frame until all initial to-be-matched frames have completed the dissimilarity detection to obtain n frames to be matched. where n is a positive integer.

Since the matching frame of the single-channel planar image is the current frame, the motion analysis and judgment module 230 is not required to perform motion analysis and judgment.

When the input is a single-channel image sequence or a single-channel video, the motion analysis and judgment module 230 needs to judge the motion form and size of the scene through motion detection. Specifically, the motion analysis and judgment module 230 obtains a motion vector for the frame to be matched and the current frame of the single-channel image sequence or single-channel video, and uses the motion vector to perform motion analysis and judgment to obtain a matching frame, including the following steps:

In the first step, the motion analysis and judgment module 230 divides the image data of the current frame into blocks according to a predetermined size, and uniformly samples the data blocks according to a predetermined number of samples to obtain a plurality of sample blocks;

First, divide the image data into blocks with a predetermined size according to the size of the current frame image and the size of the blocks. Wherein, the size of each image block is preset. In an embodiment of the present invention, for a 640*480 image, the size of each image block is 4*4 pixels. Then uniform sampling of data blocks is performed on the divided image to obtain multiple sampling blocks. Wherein, the number of sampling points is preset. In one embodiment of the present invention, the number of sampling points is about 5% of the total number of image pixels.

In the second step, the motion analysis and judgment module 230 uses the motion offset between each sample block in the current frame and the matching block in each frame to be matched as the motion vector of the sample block corresponding to the frame to be matched;

For each sampling block in the current frame, search for the most similar block in a frame to be matched. Wherein, the search method includes full search, three-step search, logarithmic search or diamond search algorithm and the like. After the search is finished, the motion offset between each sample block in the current frame and the matching block in the frame to be matched is used as a motion vector of the sample block in the current frame corresponding to the frame to be matched. Through the above method, n motion vector graphics can be obtained, where n is a positive integer.

In the third step, the motion analysis and judgment module 230 performs motion analysis and judgment on the n vector graphics to obtain the number of zero motion vector graphics and the number of non-zero motion vector graphics;

The motion analysis and judging module 230 judges whether there is a zero motion vector diagram among the obtained n motion vector diagrams. In one embodiment of the present invention, when the number of sampling blocks corresponding to zero vector accounts for more than 90% of the total number of sampling blocks, the motion analysis and judgment module 230 judges that the motion vector diagram is a zero motion vector diagram. Wherein, the number of zero motion vector diagrams is m. Of course, those skilled in the art can understand that when the ratio of the number of sampling blocks corresponding to zero vectors to the total number of sampling blocks is other reasonable ratios, it also falls within the protection scope of the present invention.

According to the m zero motion vector diagrams obtained above, it can be known that the number of non-zero motion vector diagrams is n-m. Then, the motion analysis and judgment module 230 performs horizontal motion vector analysis on the non-zero motion vector diagrams. Non-zero motion vector diagrams include motion vector diagrams corresponding to foreground horizontal motion, motion vector diagrams corresponding to camera horizontal movement, and motion vector diagrams corresponding to remaining situations. The motion vector diagrams corresponding to the remaining situations are motion vector diagrams other than the motion vector diagram corresponding to the foreground horizontal motion and the motion vector diagram corresponding to the camera horizontal movement.

When among the sampling blocks corresponding to the non-zero motion vector diagram, the sampling blocks whose motion vector direction is in the horizontal direction account for 70% to 90% of the non-zero vector sampling blocks, the motion analysis and judging module 230 judges that the foreground is moving horizontally. Wherein, the number of motion vector diagrams corresponding to the foreground horizontal motion is p. Of course, those skilled in the art can understand that when the ratio of the sampling blocks whose motion vector direction is in the horizontal direction to the non-zero vector sampling blocks is other reasonable ratios, it also falls within the protection scope of the present invention.

When among the sampling blocks corresponding to the non-zero motion vector diagram, the sampling blocks whose motion vector direction is in the horizontal direction account for more than 90% of the non-zero vector sampling blocks, the motion analysis and judgment module 230 judges that the camera moves horizontally. Wherein, the number of motion vector diagrams corresponding to the horizontal movement of the camera is q. Of course, those skilled in the art can understand that when the ratio of the sampling blocks whose motion vector direction is in the horizontal direction to the non-zero vector sampling blocks is other reasonable ratios, it also falls within the protection scope of the present invention.

It can be seen that the number of motion vector diagrams corresponding to the remaining cases is n-m-p-q.

Then, the motion analysis and judgment module 230 performs motion analysis and judgment on the n-m non-zero motion vector diagrams. First, the average of the non-zero motion vectors of each non-zero motion vector diagram is calculated, and the motion analysis and judgment module 230 obtains the average value A(i) of the motion vectors of each non-zero motion vector diagram. Among them, A is the average value, and i is a non-zero vector label. If the non-zero motion vector mean value corresponding to the non-zero motion vector diagram is within the mean value threshold, the motion analysis and judgment module 230 retains the non-zero motion vector diagram, otherwise discards the non-zero motion vector diagram and matches the non-zero motion vector diagram corresponding to it. Frame drops.

In one example of the present invention, the mean threshold is in the range of [5, 30] pixel distances. Of course, those skilled in the art can understand that, when the average threshold value is in other reasonable setting ranges, it also falls within the protection scope of the present invention.

Through the motion analysis and judgment of the motion analysis and judgment module 230, the number of zero motion vector diagrams meeting the requirements can be obtained as v1, the number of motion vector diagrams corresponding to the foreground horizontal motion is v2, and the number of motion vector diagrams corresponding to the horizontal movement of the camera is v3, and the number of motion vector diagrams corresponding to the remaining cases is v4.

In the fourth step, the motion analysis and judgment module 230 acquires matching frames according to the number of zero motion vectors and the number of non-zero motion vectors.

(1) When the number v1 of the zero-motion vector diagram is greater than the first threshold R(v1), the motion analysis and judgment module 230 judges that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a static scene, and all zero-motion Among the frames to be matched corresponding to the vector diagram, the frame farthest from the current frame on the time axis is selected as the matching frame.

In an embodiment of the present invention, the first threshold R(v1) may be set to max(1, n-2), that is, v1>max(1, n-2). Of course, those skilled in the art can understand that when the first threshold is set to other reasonable values, it also falls within the protection scope of the present invention.

(2) When the number v2 of the motion vector diagram corresponding to the foreground horizontal motion is within the second threshold R(v2), the motion analysis and judgment module 230 judges that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a foreground motion scene , select the motion vector graph corresponding to the foreground horizontal motion with the smallest difference between the average A(v2) of the motion vector graphs corresponding to the v2 foreground horizontal motions and the foreground standard preset threshold R(a2) as the motion corresponding to the final foreground horizontal motion Vector graph, select the frame to be matched corresponding to the motion vector graph corresponding to the final foreground horizontal motion as the matching frame.

In one embodiment of the present invention, R(v2)=[n/3,n], R(a2)=15. Of course, those skilled in the art can understand that when the second threshold R(v2) is set to other reasonable values, it also falls within the protection scope of the present invention.

(3) When the number v3 of the motion vector diagram corresponding to the horizontal movement of the camera is greater than the third threshold R(v3), the motion analysis and judgment module 230 judges that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a camera movement scene, Select the motion vector diagram corresponding to the foreground horizontal motion with the smallest difference between the average A(v3) of the motion vector diagrams corresponding to the v3 camera horizontal movement and the camera standard preset threshold R(a3) as the final motion vector corresponding to the horizontal camera movement , select the frame to be matched corresponding to the motion vector diagram corresponding to the final horizontal movement of the camera as the matching frame.

In one embodiment of the present invention, R(v3)=[n/2,n], R(a3)=10. Of course, those skilled in the art can understand that when the third threshold R(v3) is set to other reasonable values, it also falls within the protection scope of the present invention.

(4) When the quantity v4 of the motion vector diagrams corresponding to the remaining situations is greater than the fourth threshold R(v4), then the motion analysis and judgment module 230 judges that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a scene other than the foreground horizontal motion and Scenes outside of horizontal camera movement. Select the motion vector diagram corresponding to the remaining situation with the smallest difference between the average A(v4) of the motion vector diagrams corresponding to v4 remaining situations and the standard preset threshold value R(a4) of the remaining situations as the motion vector diagram corresponding to the final remaining situation, The frame to be matched corresponding to the motion vector diagram corresponding to the final remaining situation is selected as the matching frame.

In one embodiment of the present invention, R(v4)=[n-2,n], R(a4)=5. Of course, those skilled in the art can understand that when the fourth threshold R(v4) is set to other reasonable values, it also falls within the protection scope of the present invention.

In one embodiment of the present invention, when none of the above four conditions are met, for example, a single planar image is input, the current frame is selected as the frame to be matched.

As shown in FIG. 2 , the apparatus 200 for stereo-to-stereo conversion based on motion analysis further includes a matching frame update module 240 . When the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a foreground motion scene, the matching frame update module 240 needs to obtain a disparity map according to the matching frame and the current frame, and then obtain a new matching frame through the disparity map and the current frame , that is, update the matching frame. Wherein, the matching frame update module 240 obtains the disparity map through one of the following algorithms, including an optical flow method based on pixel block operations, a block matching method, and a depth matching method based on pixel point operations.

In one embodiment of the present invention, the matching frame update module 240 uses the current frame and the disparity map to obtain a new matching frame through the DIBR (Depth Image Based Rendering) method.

When the single-channel image is a single-channel plane image or a single-channel video or a single-channel image sequence, and the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a static scene or a camera moving scene or other situations, the image conversion module 250 Image transformation is performed on the current frame and the matching frame to obtain a first stereoscopic input image and a second stereoscopic input image.

When the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a foreground motion scene, the image transformation module 250 performs image transformation on the current frame and the updated matching frame to obtain the first stereoscopic input image and the second stereoscopic input image.

The image transformation performed by the image transformation module 250 includes the following manners: linear transformation, nonlinear transformation and preset area transformation.

Linear transformation refers to the linear image transformation of the entire image only for the current frame or the matching frame. In an embodiment of the present invention, the linear transformation may be translation transformation, stretch transformation, compression transformation, or perspective transformation.

Non-linear transformation refers to image rotation transformation around X, Y, and Z axes, and a preset transformation operation for each row or column or pixel of the image.

Preset area transformation means that by presetting the area that the user is interested in, the part of the preset area is mainly linearly transformed or nonlinearly transformed, while the area outside the preset area is not transformed, or the area outside the preset area is not transformed. The area is transformed with a smaller transformation range than the preset area transformation.

In one embodiment of the present invention, the image transformation of the current frame and the matching frame by the image transformation module 250 can be a single type or a combined type, and can only transform one of the frame images, while the other frame image Do not transform.

In one embodiment of the present invention, the first stereoscopic input image is used as the initial input of the left view, that is, the current frame is used as the initial input of the left view; the second stereoscopic input image is used as the initial input of the right view, corresponding to viewing with the left eye; That is, the matching frame is used as the initial input of the right view, which corresponds to viewing with the right eye. Of course, those skilled in the art can understand that using the current frame as the initial input of the right view and using the matching frame as the initial input of the left view also falls within the protection scope of the present invention.

Corresponding to different display terminals, the stereo synthesis module 260 performs stereo synthesis on two input images by using different stereo video synthesis formats and stereo video output modes. Among them, the stereo synthesis module 260 synthesizes the two input images according to the stereo video synthesis format, including performing proper synthesis of the left and right input images to obtain a single stereo video output that meets different stereo display requirements. Wherein, the stereoscopic video synthesis format includes checkerboard format synthesis and horizontal interlaced format synthesis, and the like.

The stereoscopic video output method includes arranging the left and right input images side by side horizontally and vertically, or using the left and right input images as two output channels. In an embodiment of the present invention, the stereoscopic video synthesized by the stereoscopic synthesis module 260 may be a single-channel output display of stereoscopic synthesis of red-green, red-cyan, red-blue, etc.

According to the motion analysis-based plane-to-stereo device of the embodiment of the present invention, there are different solutions for different image or video input conditions, thus having a large application range, and reducing the total amount of calculation through the selection of adjacent frames and block sampling, At the same time, the motion analysis and judgment algorithm adopted by the motion analysis and judgment module of the present invention is simple, practical and has high processing efficiency. And because both the current frame and the matching frame come from the image sequence or the video itself, and the image transformation of the current frame and the matching frame is based on the preset area or the entire image, so there is no loss of image and video quality, so that the image and the matching frame The video quality is high. In addition, the linear, non-linear, and preset region-based image transformations of the embodiments of the present invention make the stereoscopic video after stereoscopic synthesis have obvious stereoscopic effect, so that there is no obvious depth perception error and frame-to-frame jitter.

In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although the embodiments of the present invention have been shown and described, those skilled in the art can understand that various changes, modifications and substitutions can be made to these embodiments without departing from the principle and spirit of the present invention. and modifications, the scope of the invention is defined by the appended claims and their equivalents.

Claims (10)

1. A plane-to-stereo method based on motion analysis, is characterized in that, comprises the steps: Inputting a single-channel image, and performing preprocessing on the single-channel image, including: reading each frame in the single-channel image, and extracting a pixel value in a corresponding color space; Read each frame of image in the single-channel image as the current frame, when the single-channel image is a single-channel planar image, the final matching frame is the current frame; when the single-channel image is a single-channel image sequence or a single-channel video, Selecting n frames adjacent to the current frame as frames to be matched by dissimilarity detection, wherein n is a positive integer, and selecting n frames adjacent to the current frame as frames to be matched includes: selecting an initial frame from adjacent frames of the current frame A sequence of frames to be matched, wherein the initial sequence of frames to be matched includes a plurality of initial frames to be matched, and the initial sequence of frames to be matched is the first h frames and the last h frames of the current frame on the time axis, and Performing a dissimilarity detection on each of the initial frames to be matched to obtain n frames to be matched, wherein the dissimilarity detection on each of the initial frames to be matched includes the following steps: A1: Carry out a horizontal midline segmentation and a vertical midline segmentation on the current frame and an initial frame to be matched at the same time to obtain four image blocks respectively; A2: Subtract the pixel values of the current frame and the four image blocks corresponding to the corresponding initial frame to be matched and take the absolute value to obtain four new image blocks, and calculate the pixel mean value and variance of the four new image blocks; A3: Threshold judgment is performed on the pixel mean and variance of the four new image blocks, and when the number of the mean and variance of the four new image blocks exceeding the preset threshold is less than or equal to the dissimilarity threshold, then judge the The initial frame to be matched is a frame to be matched in the current frame, Repeat steps A1 to A3 until each initial frame to be matched in the sequence of initial frames to be matched completes the dissimilarity detection to obtain n frames to be matched; Obtaining a motion vector according to the frame to be matched and the current frame, and using the motion vector to perform motion analysis and judgment to obtain a final matching frame, including: dividing the image data of the current frame into blocks according to a predetermined size, and after the blocks are divided, The image is uniformly sampled according to the predetermined number of samples to obtain multiple sampling blocks; the motion offset between each sampling block in the current frame and the matching block in each frame to be matched is used as the corresponding sampling block The motion vector of the frame to be matched to obtain u vector graphics, wherein u is a positive integer; motion analysis and judgment are performed on the u vector graphics to obtain the number of zero motion vector graphics and the number of non-zero motion vector graphics, Wherein the non-zero motion vector diagram includes the motion vector diagram corresponding to the foreground horizontal motion, the motion vector diagram corresponding to the camera horizontal movement and the motion vector diagram corresponding to the remaining situation; and according to the number of the zero motion vector diagram and the non-zero motion vector diagram Quantity to get the final matching frame; performing image transformation on the current frame and the final matching frame to obtain a first stereoscopic input image and a second stereoscopic input image; and Stereoscopic synthesis is performed on the first stereoscopic input image and the second stereoscopic input image to obtain a stereoscopic video, and the stereoscopic video is output. 2. the plane-to-stereo method based on motion analysis as claimed in claim 1, is characterized in that, described according to the quantity of zero motion vector diagram and the quantity of non-zero motion vector diagram to obtain final matching frame, comprises the steps: When the number of the zero-motion vector graphics is greater than the first threshold, it is judged that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a static scene, and the frame to be matched corresponding to the zero-motion vector graphics is selected The frame farthest from the current frame on the time axis is used as the final matching frame; When the number of motion vector diagrams corresponding to the foreground horizontal motion is within the second threshold, it is judged that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a foreground motion scene, and the motion corresponding to the foreground horizontal motion is selected. The motion vector diagram corresponding to the foreground horizontal motion with the minimum difference between the non-zero motion vector average value and the foreground standard preset threshold value in the vector diagram is the motion vector diagram corresponding to the final foreground horizontal motion, and the motion vector corresponding to the final foreground horizontal motion is selected The frame to be matched corresponding to the graph is the final matching frame; When the number of motion vector diagrams corresponding to the horizontal movement of the camera is greater than the third threshold, it is judged that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a camera movement scene, and the motion vector corresponding to the horizontal movement of the camera is selected. In the figure, the motion vector diagram corresponding to the camera horizontal movement with the smallest difference between the non-zero motion vector average value and the camera standard preset threshold is the motion vector diagram corresponding to the final camera horizontal movement, and the motion vector diagram corresponding to the final camera horizontal movement is selected The corresponding frame to be matched is the final matching frame; When the number of motion vector diagrams corresponding to the remaining situation is greater than the fourth threshold, select the remaining situation in which the difference between the average value of non-zero motion vectors and the standard preset threshold of the remaining situation in the motion vector diagram corresponding to the remaining situation is the smallest The corresponding motion vector diagram is the motion vector diagram corresponding to the final remaining situation, and the frame to be matched corresponding to the motion vector diagram corresponding to the final remaining situation is selected as the final matching frame. 3. The plane-to-stereo method based on motion analysis as claimed in claim 2, wherein, when the scene corresponding to the single-channel video or single-channel image sequence is a foreground motion scene, according to the matching frame and the current frame to obtain Get the disparity map, and update the matching frame according to the disparity map and the current frame. 4. The plane-to-stereo method based on motion analysis as claimed in claim 3, wherein said performing image transformation on the current frame and the matching frame to obtain the first stereo input image and the second stereo input image comprises: When the single-channel image is a single-channel plane image or a single-channel video or a single-channel image sequence, and the scene corresponding to the single-channel video or single-channel image sequence is a static scene or a camera moving scene or a remaining situation, the current frame performing image transformation with the matching frame to obtain the first stereoscopic input image and the second stereoscopic input image; When the scene corresponding to the single-channel video or the single-channel image sequence is a foreground moving scene, image transformation is performed on the current frame and the updated matching frame to obtain the first stereoscopic input image and the second stereoscopic input image. 5. The method for converting plane to stereo based on motion analysis according to claim 4, wherein the image transformation includes the following methods: linear transformation, nonlinear transformation and preset area transformation. 6. A plane-to-stereo device based on motion analysis, characterized in that it comprises: The single-channel input and preprocessing module is used to input a single-channel image and perform preprocessing on the single-channel image, wherein the single-channel input and pre-processing module further reads each frame in the single-channel image, And extract the pixel value in the corresponding color space; The frame selection module to be matched is used to read each current frame in the single image from the single input and preprocessing module. When the single image is a single planar image, the frame to be matched is selected The module selects the current frame as the final matching frame; when the single-channel image is a single-channel image sequence or a single-channel video, for each frame of the image in the single-channel image, the frame selection module to be matched selects the selected frame through the dissimilarity detection The adjacent n frames of the current frame are frames to be matched, wherein n is a positive integer, and an initial frame sequence to be matched is selected from adjacent frames of the current frame, wherein the initial sequence of frames to be matched includes a plurality of initial frames to be matched , the frame sequence to be initially matched is the first h frames and the last h frames of the current frame on the time axis, and each initial frame to be matched is subjected to dissimilarity detection to obtain a frame sequence to be matched, wherein , the frame to be matched selection module performs a horizontal midline segmentation and a vertical midline segmentation on the current frame and an initial frame to be matched simultaneously to obtain four image blocks respectively, and subtracts the pixel values of the four image blocks respectively and obtains Absolute value, finally four new image blocks are obtained, and the pixel mean value and variance are calculated for the four new image blocks; threshold value judgment is performed on the pixel mean value and variance of the four new image blocks, when the four new image blocks When the number of blocks whose mean value and variance exceeds the preset threshold is less than or equal to the dissimilarity threshold, it is judged that the initial frame to be matched is a frame to be matched in the current frame, and the above steps are repeated until the initial frame to be matched is in the sequence of frames to be matched Complete the dissimilarity detection for each initial frame to be matched, and obtain n frames to be matched, where n is a positive integer; A motion analysis and judgment module, configured to obtain a motion vector from the frame to be matched and the current frame from the frame to be matched selection module, and use the motion vector to perform motion analysis and judgment to obtain a final matching frame, wherein the The motion analysis and judgment module divides the image data of the current frame into blocks according to a predetermined size, and uniformly samples the data blocks according to a predetermined number of samples to obtain a plurality of sample blocks. Each sample in the current frame The motion offset between the block and the matching block in each frame to be matched is used as the motion vector of the sampled block corresponding to the frame to be matched, so as to obtain u vector diagrams, where u is a positive integer, and for the u Carry out motion analysis and judgment on the vector graphics to obtain the number of zero motion vector graphics and the number of non-zero motion vector graphics, wherein the non-zero motion vector graphics include the motion vector graphics corresponding to the horizontal movement of the foreground, the motion vector graphics corresponding to the horizontal movement of the camera, and the remaining The motion vector diagram corresponding to the situation; and obtaining the final matching frame according to the quantity of the zero motion vector diagram and the quantity of the non-zero motion vector diagram; An image transformation module, configured to perform image transformation on the current frame and the final matching frame from the motion analysis and judgment module to obtain a first stereoscopic input image and a second stereoscopic input image; and A stereo synthesis module, configured to perform stereo synthesis on the first stereo input image and the second stereo input image from the image conversion module to obtain a stereo video, and output the stereo video. 7. The plane-to-stereo device based on motion analysis as claimed in claim 6, wherein said obtaining the final matching frame according to the quantity of zero motion vector diagram and the quantity of non-zero motion vector diagram comprises the steps of: When the number of the zero-motion vector graphics is greater than the first threshold, it is judged that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a static scene, and the frame to be matched corresponding to the zero-motion vector graphics is selected The frame farthest from the current frame on the time axis is used as the final matching frame; When the number of motion vector diagrams corresponding to the foreground horizontal motion is within the second threshold, it is judged that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a foreground motion scene, and the motion corresponding to the foreground horizontal motion is selected. The motion vector diagram corresponding to the foreground horizontal motion with the minimum difference between the non-zero motion vector average value and the foreground standard preset threshold value in the vector diagram is the motion vector diagram corresponding to the final foreground horizontal motion, and the motion vector corresponding to the final foreground horizontal motion is selected The frame to be matched corresponding to the graph is the final matching frame; When the number of motion vector diagrams corresponding to the horizontal movement of the camera is greater than the third threshold, it is judged that the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a camera movement scene, and the motion vector corresponding to the horizontal movement of the camera is selected. In the figure, the motion vector diagram corresponding to the camera horizontal movement with the smallest difference between the non-zero motion vector average value and the camera standard preset threshold is the motion vector diagram corresponding to the final camera horizontal movement, and the motion vector diagram corresponding to the final camera horizontal movement is selected The corresponding frame to be matched is the final matching frame; When the number of motion vector diagrams corresponding to the remaining situation is greater than the fourth threshold, select the remaining situation in which the difference between the average value of non-zero motion vectors and the standard preset threshold of the remaining situation in the motion vector diagram corresponding to the remaining situation is the smallest The corresponding motion vector diagram is the motion vector diagram corresponding to the final remaining situation, and the frame to be matched corresponding to the motion vector diagram corresponding to the final remaining situation is selected as the final matching frame. 8. The plane-to-stereo device based on motion analysis as claimed in claim 7, wherein the plane-to-stereo device based on motion analysis further comprises a matching frame update module, When the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a foreground motion scene, the matching frame update module is used to obtain a disparity map according to the matching frame and the current frame from the motion analysis and judgment module , and update the matching frame according to the disparity map and the current frame. 9. The plane-to-stereo device based on motion analysis as claimed in claim 8, characterized in that, When the single-channel image is a single-channel plane image or a single-channel video or a single-channel image sequence, and the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a static scene or a camera moving scene or other situations, the image transformation The module performs image transformation on the current frame and the matching frame from the motion analysis and judgment module to obtain the first stereoscopic input image and the second stereoscopic input image; When the scene corresponding to the current frame of the single-channel video or single-channel image sequence is a foreground motion scene, the image transformation module performs image transformation on the current frame and the updated matching frame from the matching frame updating module to obtain The first stereoscopic input image and the second stereoscopic input image are obtained. 10. The motion analysis-based 2D-to-3D device according to claim 9, wherein the image transformation performed by the image transformation module includes the following methods: linear transformation, nonlinear transformation and preset area transformation. the

Images