KR20250142725A - ideo synthesis and character generation system using chroma key technique - Google Patents

Abstract

The present invention relates to a system for synthesizing a subject image and a background image by applying a chroma key technique and extracting a user image from the captured image to create a character, the system comprising: a cash input unit for charging a fee to a user; a background screen installed behind a user to apply a chroma key technique; a shooting unit for shooting the user's motion and converting it into a digital image signal; a background image storage unit for storing background images to be synthesized with the user's motion; a background image selection unit for allowing the user to select a background image; a monitor for checking a state in which the user's motion and the selected background image are synthesized and displayed; an image processing unit for synthesizing the captured image and background image in real time to create a composite image file and extracting the user image from the captured image to create a user character file; an image editing unit for editing the files generated by the image processing unit; an image compression unit for compressing the files generated by the image processing unit; and a compressed composite image file.
It comprises a video output unit that outputs a synthetic video file or a user character file to a recording medium; a video transmission unit that transmits the compressed synthetic video file or the user character file to a dedicated web server via the Internet; a microphone that converts a user's voice signal into an electrical signal; and a sound synthesis unit that synthesizes the accompaniment sound of a song selected by the user and a voice signal from the microphone and outputs the synthesized signal to the recording medium; thereby enabling the production of various contents that reflect the user's individuality.

Description

Image synthesis and character generation system using chroma key technique

The present invention relates to a video and character generation system, and more particularly, to a system that synthesizes a subject image and a background image by applying a chroma key technique and extracts a user image from the captured image to generate a character.

In general, the karaoke machines installed in entertainment facilities such as karaoke rooms are designed so that users can simply sing along with the accompaniment to the song they have selected and enjoy it with a microphone, and do not provide a recording method to record the singing video and then watch it again whenever they want.

Recently, a service has been launched that synthesizes karaoke accompaniment and background video and records it on a CD or other storage medium. However, this service has the problem that users cannot select the desired video as the background because it is a service in which entertainment facility managers manage multiple karaoke machines installed independently through a single video management device.

Meanwhile, as personal mini-homepages and user-created contents (UCC) become more common, the commercial value of virtual characters (avatars) that act as users in cyberspace is rapidly increasing. However, most virtual characters are either created by simply using existing characters or by simply combining several images, which reduces realism and prevents sufficient self-expression.

The present invention was created to address the aforementioned issues, and provides a system capable of producing and recording video content containing the user's unique movements and voices, using a user-selected video as a background. Another object of the present invention is to provide a system that extracts the user's video during the content production process and utilizes it as a character representing the user online.

The character generation system through video synthesis and real-time subject extraction according to the present invention for solving the above-mentioned problem is as follows.

A cash machine for charging users a fee;

A background screen installed behind the user to apply the chroma key technique;

A shooting unit that shoots the user's movements and converts them into digital video signals;

A background image storage unit that stores background images to be synthesized with the user's actions;

A background image selection section for allowing the user to select a background image;

A monitor to check the state in which the user's actions and the selected background image are synthesized and displayed;

An image processing unit that synthesizes the above-described captured image and background image in real time to create a composite image file, and extracts a user image from the above-described captured image to create a user character file;

A video editing unit for editing files generated by the above video processing unit;

An image compression unit that compresses files generated by the image processing unit;

An image output unit that outputs the compressed synthetic image file or user character file to a recording medium;

An image transmission unit that transmits the compressed synthetic image file or user character file to a dedicated web server via the Internet;

A microphone that converts the user's voice signal into an electrical signal; and

It comprises a sound synthesis unit that synthesizes the accompaniment sound of a song selected by a user and a voice signal from a microphone and outputs the synthesized sound to the recording medium.

The above image processing unit preferably generates a composite image file by synthesizing a video signal of an image captured by a user and a video mask alpha signal of a selected background image in real time using a chroma key technique, and extracts a user image by codec processing and alpha channel file conversion after capturing the video signal and the video mask alpha signal in real time to generate a user character file. The above image processing unit preferably performs image processing by binarizing a brightness image captured by an infrared camera based on a threshold value to distinguish a subject area from a background area.

It is preferable that the above dedicated web server uploads the received file so that the website subscriber can download it via the Internet, and transmits the received file to the subscriber's mobile phone via a mobile network at the website subscriber's request.

The following effects can be expected by the character generation system through video synthesis and real-time subject extraction according to the present invention.

First, users can use their actual videos on personal homepages, UCC, mobile phones, etc., enabling the creation of diverse content that reflects the user's individuality.

Second, it can enrich the quality of life by allowing you to record videos of your loved ones in various forms to commemorate events or special days.

Third, we can generate significant revenue by commercializing video and character creation studios and related Internet services.

Fourth, you can select a background video to be synthesized with your video, allowing you to create a video in the format you want.

Figure 1 is an actual configuration diagram of a video production room according to one embodiment of the present invention.
Figure 2 is a configuration diagram of a video synthesis and character generation system according to one embodiment of the present invention.
Figure 3 is a detailed configuration diagram of a photographing unit in a system according to one embodiment of the present invention.
FIG. 4 is a diagram showing an image synthesis and subject image extraction process in an image processing unit of a system according to one embodiment of the present invention.
Figure 5 is a drawing showing the usage form of a file produced according to one embodiment of the present invention.

A character generation system through video synthesis and real-time subject extraction according to one embodiment of the present invention will be described in detail with reference to the attached drawings below.

Figure 1 is an actual configuration diagram of a video production room according to one embodiment of the present invention.

A background screen (120) is installed behind the back of the user (130) who is the subject to enable chroma keying to synthesize a background image during filming. Chroma keying is a technique in which a specific color (Chroma) is designated as a key for image synthesis, and then pixels corresponding to the key in the main image are replaced with various images. The background screen and the subject (or object) must be separated from each other based on chroma. In order for the background screen and the subject (or object) to be separated, the color of the subject, the color of the background, the lighting conditions, the camera technique, etc. must be planned in advance. In addition, the subject must be planned in advance so that it does not fall within the same range as the background by causing reflections, shadows, or changes in lighting conditions due to the appearance of another subject. In this embodiment, a blue screen that facilitates image editing using the chroma key technique is applied as the background screen.

The user wears the desired costume in front of a blue screen and performs a dance. In this embodiment, the user sings a song and moves accordingly, just like in a conventional karaoke room. A camera unit (310) is installed opposite the user, and captures the user's movements with the support of lighting (250) and converts them into digital signals. The camera unit

An LCD screen (340) is installed at the bottom so that the user can check the image that is a composite of the motion he is directing and the background image. An integrated processing unit for processing images and sounds is installed at the bottom of the LCD screen. Meanwhile,

Upon entering the production room, the user pays a fee to the cash register (110) and selects a video to use as a background image via the touch screen (320). The video created by the user is output to the DVD writer (600), allowing the user to take the created DVD with them.

FIG. 2 is a configuration diagram of a video synthesis and character generation system according to one embodiment of the present invention.

The background image storage unit (330) stores a number of images to be supported as backgrounds, and the background image selected by the user is transmitted to the image processing unit (410) together with the video of the subject captured by the camera unit (310). The image processing unit creates a composite video by synthesizing the captured video and the background image in real time using the chroma key technique, and extracts an image of only the subject from the captured video. The image editing unit (420) allows the user to edit NG scenes and insert various special effects while checking the produced video if desired. Since the video produced in this way has a large capacity, the image compression unit (430) converts it into a compressed file. Among the compressed files, the composite image file (510) is transmitted to the image output unit (440) and output to the recording medium (600) in 2D or 3D format, and the subject image character file (520) is transmitted to the image transmission unit (450) and transmitted to a dedicated web server (700) via the Internet.

Meanwhile, the microphone (210) converts the user's voice signal into an electric signal, and the accompaniment sound of a song selected by the user from among a number of songs stored in the accompaniment sound storage unit (220) is transmitted to the sound synthesis unit (230) together with the user's converted voice signal. The sound synthesis unit synthesizes the voice and accompaniment sound and outputs it to the speaker (240), while also outputting it to the recording medium (600).

The recording medium records the synthesized image and synthesized sound in synchronization.

Figure 3 is a detailed configuration diagram of a photographing unit in a system according to one embodiment of the present invention.

The camera unit is composed of a general camera (312), an infrared camera (313), and two reflective mirrors (311) that allow the two cameras to capture the same image. In the present invention, in order to smoothly separate the image into a subject portion and a background portion, a user image is captured using an infrared camera in addition to a general camera.

FIG. 4 is a diagram showing an image synthesis and subject image extraction process in an image processing unit of a system according to one embodiment of the present invention.

The image processing unit mixes in real time the screen image video (YUV) signal transmitted from the shooting unit and the video mask alpha signal of the background image transmitted from the background image storage unit, and then encodes the signal to create a composite image file.

Specifically, the grayscale image output from the infrared camera is binarized based on a threshold value, thereby separating the subject and the background. At this time, the grayscale image from the infrared camera is determined in real time to determine whether each pixel is greater or less than a pre-determined threshold value, thereby extracting the subject.

When a subject and background are captured by the camera, both regular and infrared cameras output a contrast image, varying in brightness or darkness depending on the amount of infrared light entering the lens. Since the subject (human) reflects less infrared light than the background, the image captured by the infrared camera appears brighter in the background. The subject and background areas are distinguished by performing binarization processing based on an appropriate threshold value on the contrast image. When P(x,y) is the output intensity image of the infrared camera, m(x,y) is the binarized output image, and (x,y) represents the coordinates, if P(x,y) is less than the threshold, m(x,y) has the value 1 and represents the subject, and if P(x,y) is greater than or equal to the threshold, m(x,y) has the value 0 and represents the background. At this time, if I(x,y) is the input image of a general camera and G(x,y) is the graphic image to be synthesized, it is determined whether the general camera input image I(x,y) or the graphic image G(x,y) is designated for the output image O(x,y) by determining whether each pixel in the input infrared camera image P(x,y) is greater than or less than a predetermined threshold. At this time, the threshold becomes the standard for distinguishing whether a pixel in the infrared image is the background or the subject. In the end, the output O(x,y) is created as a file by synthesizing the graphic image G(x,y) with the background part of I(x,y).

In addition, the image processing unit captures the screen image video (YUV) signal and the video mask alpha signal of the background image in real time through hardware, removes the background image, and then codec-processes it. By converting the alpha channel file created at this time, the user image is extracted and a user character file is created.

Figure 5 is a drawing showing the usage form of a file produced according to one embodiment of the present invention.

The generated composite image file (510) is stored on a recording medium such as a DVD or CD and provided to the user. The generated user character file (520) is transmitted to a dedicated web server (700) via the Internet. Here, the dedicated web server refers to a server of a website to which a user character file produced in a video production studio is transmitted. The transmitted character file is downloaded to the user's personal computer via the Internet at the request of the user who has registered on the website. The user can use the downloaded actual character file as an avatar to represent himself on his personal homepage, UCC, etc.

In addition, when a user who has registered on the website requests, the character file is transmitted to the subscriber's mobile phone via a mobile network, allowing the user to use his or her video file wherever he or she needs it.

In order to prevent unauthorized use of videos, when creating a user character file, the user's mobile phone number or resident registration number is used as an identification code so that only users who enter the matching identification code on the website can receive the character file.

While the present invention has been described above with reference to specific embodiments, it should be understood that various modifications are possible without departing from the scope of the present invention. For example, while the present embodiment has described filming a video of a user singing, it is not necessary to film the singing, and the video alone may be produced without audio. Furthermore, while it has been described that only a composite image is output to a recording medium and only a character image is transmitted over the Internet, it is self-evident that both the composite image and the character image may be output to a recording medium or transmitted over the Internet and stored on a web server through a simple design change. Therefore, the scope of protection of the present invention should not be limited to the above-described embodiments, but should be defined by the following claims and their equivalents.

Claims (6)

A background screen installed behind the user to apply the chroma key technique;
A shooting unit that shoots the user's movements and converts them into digital video signals;
A background image storage unit that stores background images to be synthesized with the user's actions;
A background image selection section for allowing the user to select a background image;
An image processing unit that synthesizes the above-described captured image and background image in real time to create a composite image file, and extracts a user image from the above-described captured image to create a user character file;
An image compression unit that compresses files generated by the image processing unit;
A system for creating a character through video synthesis and real-time subject extraction, characterized in that it comprises a video transmission unit that transmits the compressed synthetic video file or user character file to a dedicated web server via the Internet. A system for creating a character through video synthesis and real-time subject extraction, characterized in that in claim 1, the image processing unit synthesizes a video signal of an image of a user and a video mask alpha signal of a selected background image in real time using a chroma key technique to create a composite image file, and then extracts a user image through codec processing and alpha channel file conversion after capturing the video signal and the video mask alpha signal in real time to create a user character file. A system for creating a character through video synthesis and real-time subject extraction, characterized in that in claim 1, it further comprises a video output unit that outputs the compressed synthetic video file or user character file to a recording medium. A character generation system through video synthesis and real-time subject extraction, characterized in that in claim 1, it further comprises a microphone that converts a user's voice signal into an electrical signal; and a sound synthesis unit that synthesizes the accompaniment sound of a song selected by the user and the voice signal from the microphone and outputs the synthesized signal to the recording medium. A character generation system through video synthesis and real-time subject extraction, characterized in that in claim 1, a monitor is additionally included to check the state in which the user's motion and the selected background image are synthesized and displayed. A character generation system through video synthesis and real-time subject extraction, characterized in that the system further comprises a video editing unit for editing files generated by the video processing unit in the first paragraph.