KR20150001551A - Method of providing expanded support for target image handling in augmented reality by database multi-mounting in a single memory, and computer-readable recording medium for the same - Google Patents

Abstract

The present invention relates to a technique for enhancing the quality of an augmented reality by supporting a plurality of target images when an augmented reality is realized. In particular, in order to solve the problem that the quality of the augmented reality result is poor and the processing time is delayed because the target image that is allowed to be used when implementing the augmented reality in the electronic device is limited, a database is stored in a single memory space of the electronic device It is possible to utilize much more target images than conventional ones while satisfying the technical specifications related to the augmented reality, thereby improving the quality of the augmented reality and shortening the processing time and increasing the satisfaction of the augmented reality users .

Description

TECHNICAL FIELD [0001] The present invention relates to a method and apparatus for expanding a target image of augmented reality through a database multiple-mount of a single memory, and a computer readable recording medium therefor. -READABLE RECORDING MEDIUM FOR THE SAME}

The present invention relates to a technique for enhancing the quality of an augmented reality by supporting a plurality of target images when an augmented reality is realized.

In particular, in order to solve the problem that the quality of the augmented reality result is poor and the processing time is delayed because the target image which is allowed to be utilized when implementing the augmented reality in the electronic device is limited, a database is stored in a single memory space of the electronic device It is possible to utilize much more target images than conventional ones while satisfying the technical specifications related to the augmented reality, thereby improving the quality of the augmented reality and shortening the processing time and increasing the satisfaction of the augmented reality users .

With the explosion of high-performance camera-equipped mobile devices such as smart phones, attempts have been made to provide new services through Augmented Reality in such electronic devices.

This augmented reality is a technique of superimposing virtual objects on the real world seen by the user. That is, it is called Mixed Reality (MR) because the virtual world which has additional information through computer graphics processing is displayed on the real world by the user in real time.

In this way, the augmented reality is to add virtual world to the real world to provide useful additional information to the user. That is, the virtual graphic image created by the computer technology is overlapped with the information necessary for the real environment.

In order to provide an augmented reality in an electronic device, a target image is required to be added to a real world, that is, a real environment. That is, in order to superimpose and display a virtual image in an actual environment, a target image for overlapping display is required.

However, currently available augmented reality platforms have limited target images. That is, there is a limit that the designer of the virtual reality can not sufficiently utilize the target image as much as desired. This is because the augmented reality technology is designed for a specific service. For example, Starbucks, a famous coffee chain, is supposed to be used for a specific purpose of providing a store search service and a coupon providing service. It is preferable to restrict the target image because only the Starbucks logo image, the coffee image, and the coupon image are required.

However, the limitation of this target image is a result of limiting the scope of the augmented reality technology. Although a small number of target images are sufficient for realizing a specialized and simple service as described above, it is a great obstacle that the target image is limited if the service is widened. For example, applying augmented reality to a navigation service or a street monitoring system is a hindrance to many target images.

In the prior art, there is a problem that the application range of the augmented reality technology is narrowed as the target image is limited. When the image is forcedly implemented using a small number of target images, the quality of the augmented reality result is poorer than the original target, There was a problem getting out. Also, since the control program can not find an appropriate target image and search it repeatedly, there is a problem that the processing time is rather long even though the target image is small.

Therefore, there is a need for a technique that can solve these problems related to augmented reality and utilize the augmented reality technology efficiently.

An object of the present invention is to provide a technique for enhancing the quality of an augmented reality by supporting a plurality of target images when realizing an augmented reality.

In particular, it is an object of the present invention to solve the problem that the quality of the augmented reality result is poor and the processing time is delayed because the target image that is allowed to be utilized when implementing the augmented reality in the electronic device is limited, It is possible to utilize much more target images than existing ones while satisfying the technical specifications related to the augmented reality, thereby achieving the enhancement of the quality of the augmented reality and the shortening of the processing time, To increase the power of the system.

According to an aspect of the present invention, there is provided a method of extending and supporting a target image in augmented reality through a database multi-mount of a single memory, the method comprising: a first step of multi-mounting and loading a plurality of databases in a single memory provided in an electronic device; ; A second step of receiving an acquired image of a real world photographed by a camera; A third step of sequentially searching a plurality of databases for a target image matching an acquired image; A fourth step of acquiring, from the database, an enhancement image matching with the found target image when a target image matching the acquired image is found during the database search; And a fifth step of overlaying the acquired enhancement image on the position of the target image found in the acquired image.

In the present invention, each of the multiple-mounted databases in the first step has a target image amount that can be loaded in a single memory at a time, and the third step includes: finding a specific object from the acquired image; Searching for a target image corresponding to the object with respect to the plurality of databases, and performing a search by moving to the next database when the target image discovery fails in one database.

In addition, in the present invention, the database includes a plurality of internal databases previously stored in the electronic device and at least one external database externally connected through the wired / wireless network. In the third step, the search order of the external database is set to be subordinate to the internal database .

Also, in the present invention, it is preferable that a plurality of databases are connected via an XML file, and in the first step, all of the XML files mediating a plurality of databases are loaded into a single memory.

Meanwhile, a computer-readable recording medium according to the present invention records a program for executing a method of supporting a target image extension of augmented reality through a database multiple mount of a single memory as described above.

According to the present invention, an electronic device providing an augmented reality can utilize a significantly increased number of target images compared to the prior art, thereby enhancing the quality of the augmented reality result and shortening the processing time, thereby ultimately enhancing There is an advantage that the satisfaction of a real user is enhanced.

1 is an internal block diagram of a smart terminal having an augmented reality platform according to the present invention;
FIG. 2 is a flowchart showing a control procedure when an augmented reality is executed in a smart terminal according to the present invention;
3 is a control flowchart for processing a target image in an augmented reality mode according to the present invention; Fig.
4 is a diagram showing a concept of a target image call in an augmented reality platform,
5 is a diagram conceptually illustrating a process of searching for a target image in an augmented reality platform,
6 illustrates a concept of moving and searching a target image using a multi-mounted database in a single memory according to the present invention.
7 is a diagram illustrating the concept of the present invention for providing augmented reality service using a plurality of target images by multi-mounting a database in a single memory while satisfying a technical standard related to the augmented reality.
FIG. 8 is a diagram showing a program source code showing a process of moving and searching a target image in the present invention. FIG.

Hereinafter, the present invention will be described in detail with reference to the drawings. In the present invention, an electronic device includes a device for acquiring an image such as a camera and a device for displaying an acquired screen, such as a monitor, and may be a desktop computer, a laptop computer, a personal digital assistant, a tablet PC, a mobile phone, Smart phones, handheld devices, game consoles, electronic boards, electronic sign boards, and the like.

1 is an internal block diagram of a smart terminal 100 having an augmented reality platform according to the present invention.

First, the terminal communication unit 101 includes an antenna, a wireless processing module, a modem, and the like so that the smart terminal 100 can access a mobile communication network or various wired / wireless networks to perform digital data communication with the outside.

The camera module 110 includes a photographing unit 103 and an image processing unit 105. The photographing unit 103 includes a lens, a lens driving motor, and an image sensor. The image processing unit 105 amplifies the sensing signal, And a converting circuit. The image processing unit 105 may convert the photographing signal into graphic data and provide various additional information (for example, EXIF) to the terminal control unit 111.

The user input unit 107 includes a touch sensing sensor for sensing a touch input of a user and a key button for inputting a key of a button type. The user input unit 107 senses a user input through the key input unit and provides the sensing result to the terminal control unit 111.

The display unit 108 displays the state of the smart terminal 100 under the control of the terminal control unit 111 as an LCD panel or an LED panel and can combine the photographed image of the real world and the augmented image and display them on the screen.

The memory unit 109 is a module that provides a space for storing various data necessary for the operation of the smart terminal 100 and includes a RAM, a ROM, a flash memory, an MRAM (magnetic material RAM), a PRAM ), A FRAM (ferroelectric RAM), a web disk, and a cloud disk. In addition, the memory unit 109 may store various kinds of data for providing an augmented reality according to the present invention.

The terminal control unit 111 is a module for controlling the overall operation of the smart terminal 100, and performs augmented reality processing according to the present invention. The process of controlling the image data processing of the augmented reality according to the present invention by the terminal control unit 111 will be described in detail with reference to the following drawings.

FIG. 2 is a control flowchart when an augmented reality is executed in the smart terminal 100 according to the present invention.

First, in step S300, the terminal control unit 111 maintains a standby state. The standby state is a state in which a call is received by the smart terminal 100 or an input is provided through the user input unit 107. [

If there is an input while maintaining the standby state, the terminal control unit 111 proceeds to step S302 and determines whether a photographing mode is requested. If it is determined in step S302 that the photographing mode is requested, the terminal control unit 111 proceeds to step S304. Otherwise, the terminal control unit 111 proceeds to step S320 to perform the corresponding function.

The flowchart of FIG. 2 according to the present invention is a control flowchart for explaining a case where an augmented reality is applied, so a situation in which a shooting mode is not required in step S302 will not be described.

If it is determined in step S302 that the photographing mode is requested, the terminal control unit 111 proceeds to step S304 and drives the camera module 110. [ The driving of the camera module 110 means receiving the light transmitted through the lens of the photographing unit 103, acquiring the brightness and color information of the light by an electrical signal, (111). In step S304, the terminal control unit 111 displays the acquired image on the display unit 108 in advance.

Then, the terminal control unit 111 proceeds to step S306 and checks whether or not the setup of the augmented reality is required. The application of the augmented reality may be enabled by operating the setting menu of the smart terminal 100 before the user executes the shooting mode or during the shooting mode.

If it is determined in step S306 that the application of the augmented reality is required, the terminal control unit 111 proceeds to step S310 and proceeds to the augmented reality application mode. On the other hand, if it is determined in step S306 that the application of the augmented reality is not required, the process proceeds to step S308.

Here, the execution of the shooting mode refers to an operation for checking whether a specific still image is photographed or a video is recorded while continuously maintaining the preview form and performing the corresponding function, as described above.

[Figure 3] Figure 3 is a control flowchart for processing a target image in the augmented reality mode according to the present invention.

When the terminal control unit 111 proceeds to the augmented reality mode, the terminal control unit 111 sets all the xml (database of the target image) existing in the memory unit 109 as a search target in step S400, and determines the order of the xml to be searched. At this time, in the augmented reality mode, the object to be searched must be set in advance from the image acquired in the real world.

At this time, a method of setting an object to be searched may be a region in which a user sets a specific region in an acquired image, a region focused by a camera, and an acquired image by analyzing an acquired image in software dedicated to augmented reality. It is possible to enlarge or reduce the image obtained by itself in the image processing apparatus 100, or to find the object to be searched from the acquired image of the real world through various methods such as utilizing the current position information.

On the other hand, depending on the implementation, the search order of xml (the database of the target image) may not be determined in advance. That is, xml may be read from the memory unit 109 at that time and then sequentially retrieved one by one.

Then we first look at the target image used in augmented reality.

As described above, augmented reality requires a target image for outputting a virtual object or information to the smart terminal 100 using information acquired in the real world. In the augmented reality platform, which is currently used as a standard, the number of target images that can be used is limited.

We will discuss how to use the target image in a commonly used Vuforia augmented reality platform. In augmented reality, a target image can be used by adding an xml file including a target image to an augmented reality application, which will be described with reference to the accompanying drawings.

4 is an exemplary view showing a concept of a target image call in an augmented reality platform.

First, the augmented reality software can identify a piece of wood in the acquired image as a search target object, as shown in Fig. 4 (a), and retrieve the xml file as a target image. The target image is matched with the augmented image and stored in the xml file. For the target image, for example, a piece of wood of Fig. 4 (a), the porcelain kettle of Fig. 4 (b) May be stored. Conventionally, the target image and the enhancement image are generally matched and stored as an xml file.

When the user acquires an image using the smart terminal 100 as shown in FIG. 4 (c), the user searches the xml file to see if there is a target image corresponding to the image. As a result of searching the xml file, if there is a matching target image, the corresponding enhancement image is overlaid.

This will be described in more detail with reference to FIG. 4 (c). 4 (a)] is photographed by the smart terminal 100, the photographed image is displayed on the display unit of the smart terminal 100 in the same shape as the reference numeral 511. FIG.

At this time, if the augmented reality is requested from the smart terminal 100, an object called a piece of wood chopped in the image of 511 is identified and set as a target image as shown in Fig. 4 (a) do. If the target image of FIG. 4 (a) is present in the xml file, a porcelain kettle image of FIG. 4 (b) such as 502 may be obtained as an enhancement image. Thus, the augmented image is overlaid on the target image (Fig. 4 (c)).

It is provided to use various augmented reality in real life by using the latest augmented reality, such as acquiring the name of the street at a specific location or displaying the result of a review of a shop when a specific shop is photographed.

However, as described above, according to the current technical standard, the number of combinations of the target image and the enhancement image that match the target image in the augmented reality is limited to 100. That is, in the xml file, only up to 100 combinations of the target image and the enhancement image can be defined, and the background is as described above.

The present invention proposes a technique for multiple-mounting a plurality of databases in a single memory and loading an xml file for each of these databases from a methodology of extending and using a target image while obeying these constraints. This will be described with reference to FIG. 3 again.

In step S400, the terminal control unit 111 sets all the xmls existing in the memory unit 109 as search targets and assigns order numbers to the search targets xml. For example, if there are five xml's, they will be set as search targets. Thereafter, the order is set one by one for each xml. The xml to be searched first as the first database, the second searched xml as the second database, the third searched xml as the third database, the fourth searched xml as the fourth database, and the last searched xml And sets the sequence number to the fifth database.

At this time, an xml file that mounts a plurality of databases in a single memory unit 109 of the smart terminal 100 and defines a combination of a target image and an augmented image is loaded in each database.

Then, the terminal control unit 111 proceeds to step S402 and searches for the target image within the xml of the search object. That is, it is checked whether or not a target image exists in the acquired image. This retrieval method will be described with reference to FIG.

5 is a conceptual view illustrating a process of searching for a target image in an augmented reality platform. Each database shown in FIG. 5 may include up to 100 target images individually. That is, reference numerals 601, 602, ... , The target image is sequentially searched in any one of directions 611 and 612 for a maximum of 100 target images 603.

That is, as shown in FIG. 5, the terminal control unit 111 searches for a target image in the augmented reality platform in step S402. Then, in step S404, the terminal control unit 111 determines whether there is a target image corresponding to the search target object. If the target image exists, the terminal control unit 111 proceeds to step S408. Otherwise, the terminal control unit 111 proceeds to step S406 .

First, let us consider the case of proceeding to step S406. In step S406, the terminal control unit 1110 moves to the next database and continues searching for the target image in the xml file loaded in the corresponding database because the target image is not found in the xml file. If the target image is not found in the database, the terminal control unit 111 moves to the second database and sets the next xml file as a search target in step S406.

This will be described with reference to the accompanying FIG. 6 and FIG. 7. FIG. 6 is a diagram illustrating a concept of moving and searching a target image using a multi-mounted database in a single memory according to the present invention. FIG. 7 illustrates a concept of utilizing multiple target images by multi- FIG. 2 is a conceptual diagram showing the structure of the present invention for providing an augmented reality service.

As shown in FIGS. 6 and 7, each of the databases 710, 720 and 730 can upload up to 100 target images at a time. In addition, the search order of each of the databases 710, 720, and 730 is determined as described above. This search order can be set by the user, randomly selected, first or most recently applied database, or by a special algorithm.

Referring again to FIG. 3, the terminal control unit 111 sets the next xml as a search target in step S406, and then returns to step S402 to perform the procedure. If the target image is not searched, all the databases in the memory unit 109 can be sequentially searched.

The databases 710, 720 and 730 are not limited to an internal database stored in advance in the smart terminal 100, and may include an external database externally connected through a wired or wireless network such as a mobile communication network or a wireless LAN. However, it is desirable to search the internal database first and the external database later, considering the access speed.

In order to configure the database search structure as shown in FIG. 6, the same operation as the program source code of FIG. 8 is executed to load an xml file into an individual database. Then it searches the first xml file for the target image, and if it fails, it goes to the next xml file and examines the target image. This is done for all xml files.

If the target image exists in step S404, the process proceeds to step S408. In step S408, the terminal control unit 111 combines the enhancement image corresponding to the target image with the acquired image and displays the combined image on the display unit. At this time, the result screen displayed on the display unit 108 may have a form as shown in FIG. 4 (c).

Then, the terminal control unit 111 proceeds to step S410 and determines whether the augmented reality mode is canceled. If it is determined in step S410 that the augmented reality mode is canceled, the augmented reality mode is terminated.

On the other hand, if it is determined in step S410 that the augmented reality mode cancellation is not required, the flow advances to step S412 to check whether an image change occurs. That is, it is examined whether a change occurs in the target image for applying the augmented reality, such as a change in the photographed image. If it is determined in step S412 that there is an image change of the augmented reality, the process proceeds to step S414 to change the target image, and proceeds to step S400 to repeat the above-described series of operations.

However, if there is no image change as a result of the inspection in step S412, the terminal control unit 111 continues to perform step S408.

The present invention can also be embodied in the form of computer readable code on a computer readable recording medium. At this time, the computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage, and the like, and may be implemented in the form of a carrier wave . The computer-readable recording medium can also be stored and executed by a computer-readable code in a distributed manner on a networked computer system. And functional programs, codes, and code segments for implementing the present invention can be easily deduced by programmers skilled in the art to which the present invention belongs.

As described above, the embodiments of the present invention have been disclosed in the present specification and drawings, and although specific terms have been used, they have been used only in a general sense to easily describe the technical contents of the present invention and to facilitate understanding of the invention. And is not intended to limit the scope of the invention. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

100: smart terminal
101:
103:
105:
107: User input
108:
109:
110: camera module
111:
501: Subject
502: Augmented image
511: image obtained from the subject
601, 602, 603: target image
611, 621: search direction
710, 720, 730:
701, 702, 703: Move between databases

Claims (5)

A first step of multi-mounting and loading a plurality of databases in a single memory provided in an electronic device;
A second step of receiving an acquired image of a real world photographed by a camera;
A third step of sequentially searching the plurality of databases for a target image matched with the acquired image;
A fourth step of acquiring an enhancement image matching the found target image from the database if a target image matching the acquired image is found during the database search;
A fifth step of overlaying the acquired enhancement image on the acquired target image at the position of the found target image;
A method for supporting a target image extension of augmented reality through a database multiple-mount of a single memory.
The method according to claim 1,
Wherein each of the plurality of multi-mounted databases in the first step has a target image amount that can be loaded in a single memory at a time,
In the third step,
Finding a specific object from the acquired image;
Retrieving a target image corresponding to the object for the plurality of databases and performing a search if the target image is not found in one database, moving to the next database;
The method comprising: storing a plurality of images in a memory;
The method of claim 2,
Wherein the database comprises a plurality of internal databases previously stored in the electronic device and one or more external databases externally connected through a wire / wireless network,
Wherein the search order of the external database in the third step is set to be a subordinate to the internal database.
The method of claim 3,
Wherein the plurality of databases are connected via an XML file,
Wherein the step of loading the XML file mediating the plurality of databases into a single memory in the first step is performed.
A computer-readable recording medium recording a program for executing a method of supporting a target image extension of augmented reality through a database multiple-mount of a single memory according to any one of claims 1 to 4.

Images