CN102915234A - Method and device for realizing program interface in application program - Google Patents

Abstract

The invention discloses a method and device for realizing a program interface in an application program. After constructing a real scene interface and a mathematical interface, the binding relationship between the real scene coordinate system of the real scene interface and the mathematical coordinate system of the mathematical interface is established, and according to the binding Determine the relationship, convert the coordinates of the real scene interface to the mathematical coordinate system, and combine the real scene interface and the mathematical interface in the mathematical coordinate system to generate the program interface; on the one hand, compared with the prior art, because the method of the present invention is used to obtain As a part of the program interface, the real-scene interface of the present invention can achieve the effect of reducing the calculation amount generated by the program interface and saving terminal resource overhead; on the other hand, the present invention organically combines the real-scene interface and the mathematical interface together to achieve Realistic effect.

Description

Method and device for realizing program interface in application program

technical field

The invention relates to the technical field of terminals, in particular to a method and device for realizing a program interface in an application program.

Background technique

With the continuous development of terminal technology and hardware capabilities of the terminal, more and more various software (application programs) are applied to the terminal, and the display modes of program interfaces in various application programs are also developing rapidly.

At present, one implementation method of the program interface in the application program is obtained through software simulation calculation. Specifically, the calculation elements in the application program calculate the graphics to be displayed on the screen, and the three-dimensional graphics are displayed on the screen through simulation calculation. , Graphics of simulation effect. Especially in recent years, with the rapid development of graphics libraries (such as DirectX and OpenGL) dedicated to graphics processing, application developers can use these graphics libraries to develop applications with complex program interfaces. Can make exquisite simulation of the real environment, so that users have an immersive feeling. However, using the graphics library above to develop the program interface in the application program requires a lot of resource overhead for the terminal, which is not conducive to the development of the terminal application program.

In addition, another implementation method of the program interface in the existing application program is to use the image obtained by the camera directly as the background of the program interface, without any modification to the image obtained by using the camera, so as to present a transparent background effect; or It is to use the image obtained by the camera as the background of the program interface, collect the position information of the key graphics in the image, and apply the position information to the application program, for example, search for the position information of a specific piece of paper in the image obtained from the camera , and display the mathematical image of the basket on the position information. The above two methods, compared to the method of software simulation calculation simulation to construct the program interface, only use the image (real scene interface) acquired by the camera as the background of the application program or as information points (position information of key graphics), which are used as the program interface in the application program. The structure of the application provides a reference, thereby reducing the resources occupied by the application program on the terminal. However, the program interface produced by the above two methods is not closely combined with the real scene interface and the mathematical interface calculated by the application program, so the program interface effect presented Not real enough.

Contents of the invention

Embodiments of the present invention provide a method and device for implementing a program interface in an application program, which are used to solve the problems that the existing program interface generation technology has a large resource overhead on the terminal and the presentation effect is not realistic enough.

A method for implementing a program interface in an application program provided by an embodiment of the present invention includes:

Construct a real-scene interface according to the acquired real-scene image, and establish a real-scene coordinate system of the real-scene interface;

Construct the mathematical interface according to the mathematical image, and establish the mathematical coordinate system of the mathematical interface;

Establish a binding relationship between the real scene coordinate system and the mathematical coordinate system;

Calculate the position of the real-scene interface in the real-scene coordinate system in the mathematical coordinate system according to the binding relationship;

In the mathematical coordinate system, the real-scene interface is combined with the mathematical interface to generate a program interface.

A device for implementing a program interface in an application program provided by an embodiment of the present invention includes:

A real-scene module, configured to construct a real-scene interface according to the obtained real-scene image, and establish a real-scene coordinate system of the real-scene interface;

A mathematical module, configured to construct a mathematical interface according to a mathematical image, and establish a mathematical coordinate system of the mathematical interface;

A binding module, configured to establish a binding relationship between the real scene coordinate system and the mathematical coordinate system;

A calculation module, configured to calculate the position of the real-scene interface in the real-scene coordinate system in the mathematical coordinate system according to the binding relationship;

The program interface generation module is used to combine the real-scene interface with the mathematical interface in the mathematical coordinate system to generate the program interface.

The beneficial effects of the embodiments of the present invention include:

The embodiment of the present invention provides a method and device for implementing a program interface in an application program. After constructing the real scene interface and the mathematical interface, the binding relationship between the real scene coordinate system of the real scene interface and the mathematical coordinate system of the mathematical interface is established, and According to the binding relationship, the coordinates of the real scene interface are converted to the mathematical coordinate system, and the real scene interface and the mathematical interface are combined to generate the program interface in the mathematical coordinate system; on the one hand, compared with the prior art, due to the use of the method of the present invention The obtained real-scene interface is used as a part of the program interface, which can achieve the effect of reducing the calculation amount generated by the program interface and saving the resource overhead of the terminal; on the other hand, the present invention organically combines the real-scene interface and the mathematical interface to achieve the Realistic effect of the interface picture.

Description of drawings

Fig. 1 is one of the flowcharts of the implementation method of the program interface in the application program provided by the embodiment of the present invention;

FIG. 2 is a schematic diagram of a program interface of an application program provided by an embodiment of the present invention;

Fig. 3 is the second flow chart of the implementation method of the program interface in the application program provided by the embodiment of the present invention;

Fig. 4 is a schematic diagram of an embodiment provided by an embodiment of the present invention;

FIG. 5 is a structural diagram of an apparatus for implementing a program interface in an application program provided by an embodiment of the present invention.

Detailed ways

The specific implementation manners of a method and device for implementing a program interface in an application program provided by an embodiment of the present invention will be described in detail below in conjunction with the accompanying drawings.

A method for implementing a program interface in an application program provided by an embodiment of the present invention, as shown in FIG. 1 , the specific process includes:

S101. Construct a real-scene interface according to the acquired real-scene image, and establish a real-scene coordinate system of the real-scene interface;

S102. Construct a mathematical interface according to the mathematical image, and establish a mathematical coordinate system of the mathematical interface;

S103. Establish a binding relationship between the real scene coordinate system and the mathematical coordinate system;

S104. Calculate the position of the real-scene interface in the real-scene coordinate system in the mathematical coordinate system according to the binding relationship;

S105. In the mathematical coordinate system, combine the real-scene interface and the mathematical interface to generate a program interface.

Wherein, in step S101, hardware and/or software tools such as a camera on the terminal may be used to acquire the real-scene image, which is not limited in this embodiment of the present invention.

Preferably, in the above steps S101 and S102, various image filtering methods (for example: image displacement, shape, color change, blurring, sharpening, etc.) Interfaces with natural depth of field and light effects further enhance the sense of reality of the final generated program interface.

Specifically, as shown in the schematic diagram of the program interface in FIG. 2 , the establishment of the binding relationship between the real scene coordinate system and the mathematical coordinate system in the above step S103 may include the following steps:

Determine the coordinates of the origin of the real scene coordinate system in the mathematical coordinate system as O(p, q);

Determine the clockwise angle θ between the x' axis of the real scene coordinate system and the x axis of the mathematical coordinate system.

In the above step S104, according to the binding relationship, the position of the real scene interface located in the real scene coordinate system in the mathematical coordinate system is calculated, which can be realized through the following process:

Determine the coordinates of the real scene interface midpoint P in the real scene coordinate system as P(x', y');

Through coordinate transformation, the coordinates P(x, y) of the calculation point P in the mathematical coordinate system are:

$\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; Cos\&theta;</span>}& \mathrm{<span\; class="notranslate">\; Sin\&theta;</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; Sin\&theta;</span>}& \mathrm{<span\; class="notranslate">\; Cos\&theta;</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; p</span>}& \mathrm{<span\; class="notranslate">\; q</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; .</span>$

Preferably, when the mathematical interface is displaced relative to the real-scene interface, the method for implementing the program interface in the application program provided by the embodiment of the present invention also needs to perform the following operations: re-establish the binding relationship between the real-scene coordinate system and the mathematical coordinate system.

Preferably, the implementation method of the program interface in the application program provided by the embodiment of the present invention, as shown in Figure 3, may also perform the following steps:

S301. Determine whether the real scene interface is displaced, if yes, execute step S302, if not, execute step S304;

S302. Calculate the relative displacement between the real scene interface and the previous frame real scene interface;

S303. In the mathematical coordinates, move the mathematical interface to a corresponding position according to the calculated relative displacement between the real-scene interface and the previous frame of the real-scene interface;

S304, end.

Specifically, the specific calculation method for calculating the relative displacement between the real-scene interface and the last frame of the real-scene interface in the above step S302 is as follows:

Dividing the real scene interface and the previous frame real scene interface into 16×16 small blocks;

Select N small blocks in the real scene interface, and calculate the relative displacement between each small block in the selected N small blocks and each small block in the previous frame real scene interface;

Calculate the relative displacement D between the real scene interface and the previous frame real scene interface based on the following formula:

$\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; N</span>}}\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; ;</span>$

Wherein, d(n) is characterized as the relative displacement between the nth small block in the N small blocks and the nth small block in the real scene interface of the previous frame, and the value range of n is [1-N].

In the process of calculating the relative displacement between the real-scene interface and the previous frame of the real-scene interface, the specific implementation method for calculating the relative displacement between each small block in the selected N small blocks and each small block in the previous frame of the real-scene interface Belonging to the category of prior art, for example, the following two methods can be used:

The first one: For the image of the nth small block, after changing the whole image by displacement k, calculate the image vector data ik(n) of the nth small block in the N small blocks and the image vector data of the nth small block in the previous frame The error of i′(n) δ ² (n)=|i _k (n)-i′(n)| ² , where the value range of n is [1-N].

Determine that when the calculated error δ ² (n) is the smallest, the displacement k is the relative displacement d(n) between the nth small block in the N small blocks and the nth small block in the real scene interface of the previous frame.

The second method: for the image of the nth small block, after changing the whole image by displacement k, calculate the image vector data i _k (n) of the nth small block in the N small blocks and the image vector of the nth small block in the previous frame Correlation degree of data i′(n): arg max _k [i _k (n)-i′(n)] ² , where the value range of n is [1-N].

It is determined that when the calculated correlation value is the largest, the displacement k is the relative displacement d(n) between the nth small block in the N small blocks and the nth small block in the real scene interface of the previous frame.

In addition, other existing implementations can also be used, such as the three-step method mentioned in "Sub-pixel Motion Estimation Combining Two-Step Estimation and Motion Compensation" ("Computer Engineering and Applications", 2009.45(7), pp. 190-191) calculation method, which will not be repeated here.

The implementation method of the program interface in the application program provided by the embodiment of the present invention will be described in detail below through a specific embodiment.

The program interface of an application program is shown in Figure 4. The camera of the terminal device is used to obtain the real-scene interface (comprising the interface of the table) and establishes the mathematical interface (comprising the interface of the cup); After the binding relationship between the coordinate systems, the program interface is generated.

When the real scene interface is displaced (moving from the position of the solid line table to the position of the table shown in the dotted line in Fig. 4), the displacement a of the real scene interface is calculated; after that, coordinate transformation is carried out to the displacement a of the real scene interface to obtain in the mathematical coordinate system The displacement a' below; in the mathematical coordinate system, move the mathematical interface to the corresponding displacement to reach the position shown by the dotted line, that is, the displacement b between the position of the cup in the solid line and the position of the cup in the dotted line in the program interface equal to the displacement a'. In this way, no matter how the real-scene interface changes, the program interface can always present a realistic display effect that the cup is placed on the table as the real-scene.

In addition, if effect processing is performed on the mathematical interface (for example, the cup is broken), when the real scene interface is displaced, the mathematical interface also moves to the corresponding position, so the effect that the cup is always in a broken state can be presented.

Based on the same inventive concept, the embodiment of the present invention also provides a device for implementing a program interface in an application program. Since the problem-solving principle of the device is similar to that of the aforementioned method for implementing a program interface in an application program, the implementation of the device can Refer to the implementation of the method, and the repetition will not be repeated.

An implementation device for a program interface in an application program provided by an embodiment of the present invention, as shown in FIG. 5 , includes:

The real-scene module 501 is configured to construct a real-scene interface according to the acquired real-scene image, and establish a real-scene coordinate system of the real-scene interface;

A mathematical module 502, configured to construct a mathematical interface according to the mathematical image, and establish a mathematical coordinate system of the mathematical interface;

A binding module 503, configured to establish a binding relationship between the real scene coordinate system and the mathematical coordinate system;

Calculation module 504, for calculating the position of the real scene interface in the real scene coordinate system in the mathematical coordinate system according to the binding relationship;

The program interface generation module 505 is used to combine the real-scene interface and the mathematical interface in the mathematical coordinate system to generate the program interface.

Specifically, the binding module 503 in the above device provided by the embodiment of the present invention, as shown in FIG. 5 , specifically includes:

Determine the origin coordinate submodule 5031, which is used to determine the coordinates of the origin of the real scene coordinate system in the mathematical coordinate system as O(p, q);

Determine the included angle sub-module 5032, which is used to determine the clockwise included angle θ from the x' axis of the real scene coordinate system to the x axis of the mathematical coordinate system;

Specifically, the calculation module 504 in the above-mentioned device, as shown in FIG. 5 , specifically includes:

Determine the coordinate sub-module 5041, for determining the coordinates of the midpoint P in the real scene interface in the real scene coordinate system as P(x', y');

The coordinate transformation sub-module 5042 is used to calculate the coordinates P(x, y) of the point P in the mathematical coordinate system through coordinate transformation as $P(x,\mathrm{the\; y})=\left[\begin{array}{ccc}\mathrm{Cos\&theta;}& \mathrm{Sin\&theta;}& 0\\ -\mathrm{Sin\&theta;}& \mathrm{Cos\&theta;}& 0\\ p& q& 1\end{array}\right]P(x^{\&prime;},{\mathrm{the\; y}}^{\&prime;}).$

Preferably, the binding module 503 in the above device provided by the embodiment of the present invention is also used to re-establish the binding relationship between the real scene coordinate system and the mathematical coordinate system when the mathematical interface is displaced relative to the real scene interface.

Preferably, the implementation device of the program interface in the application program provided by the embodiment of the present invention, as shown in Figure 5, further includes:

The displacement calculation module 506 is used to calculate the relative displacement between the current real scene interface and the previous frame real scene interface when the real scene interface changes;

The moving module 507 is configured to move the mathematical interface to a corresponding position in mathematical coordinates according to the calculated relative displacement between the real-scene interface and the previous frame of the real-scene interface.

Specifically, the above-mentioned displacement calculation module 506, as shown in FIG. 5 , specifically includes:

The image block sub-module 5061 is used to divide the real scene interface and the previous frame real scene interface into 16×16 small blocks;

The displacement calculation sub-module 5062 is used to select N small blocks in the real scene interface, and calculates respectively the relative displacement between each small block in the selected N small blocks and each small block in the previous frame real scene interface; based on The following formula calculates the relative displacement D between the real scene interface and the previous frame real scene interface:

$\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; N</span>}}\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; ;</span>$

Among them, d(n) is characterized as the relative displacement between the nth small block in the N small blocks and the nth small block in the real scene interface of the previous frame, and the value range of n is [1-N].

The embodiment of the present invention provides a method and device for implementing a program interface in an application program. After constructing the real scene interface and the mathematical interface, the binding relationship between the real scene coordinate system of the real scene interface and the mathematical coordinate system of the mathematical interface is established, and According to the binding relationship, the coordinates of the real scene interface are converted to the mathematical coordinate system, and the real scene interface and the mathematical interface are combined to generate the program interface in the mathematical coordinate system; on the one hand, compared with the prior art, due to the use of the method of the present invention The obtained real-scene interface is used as a part of the program interface, which can achieve the effect of reducing the calculation amount generated by the program interface and saving the resource overhead of the terminal; on the other hand, the present invention organically combines the real-scene interface and the mathematical interface to achieve the Realistic effect of the interface picture.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims (10)

1. A method for implementing a program interface in an application program, comprising: Construct a real-scene interface according to the acquired real-scene image, and establish a real-scene coordinate system of the real-scene interface; Construct the mathematical interface according to the mathematical image, and establish the mathematical coordinate system of the mathematical interface; Establish a binding relationship between the real scene coordinate system and the mathematical coordinate system; Calculate the position of the real-scene interface in the real-scene coordinate system in the mathematical coordinate system according to the binding relationship; In the mathematical coordinate system, the real-scene interface is combined with the mathematical interface to generate a program interface. 2. The method according to claim 1, wherein establishing the binding relationship between the real scene coordinate system and the mathematical coordinate system specifically comprises: Determining that the coordinates of the origin of the real scene coordinate system in the mathematical coordinate system are O(p, q); Determine the clockwise included angle θ from the x' axis of the real scene coordinate system to the x axis of the mathematical coordinate system; According to the binding relationship, calculating the position of the real-scene interface located in the real-scene coordinate system in the mathematical coordinate system specifically includes: Determine the coordinates of the real scene interface midpoint P in the real scene coordinate system as P(x', y'); By coordinate transformation, calculate the coordinates P(x, y) of the point P in the mathematical coordinate system as $\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; Cos\&theta;</span>}& \mathrm{<span\; class="notranslate">\; Sin\&theta;</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; Sin\&theta;</span>}& \mathrm{<span\; class="notranslate">\; Cos\&theta;</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; p</span>}& \mathrm{<span\; class="notranslate">\; q</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; .</span>$ 3. The method according to claim 1 or 2, further comprising: when the mathematical interface is displaced relative to the real-scene interface, re-establishing the binding relationship between the real-scene coordinate system and the mathematical coordinate system. 4. The method according to claim 1 or 2, further comprising: When the real scene interface changes, calculate the relative displacement between the current real scene interface and the previous frame real scene interface; In the mathematical coordinates, the mathematical interface is moved to a corresponding position according to the calculated relative displacement between the real scene interface and the real scene interface of the previous frame. 5. The method according to claim 4, wherein the calculating the relative displacement between the real-scene interface and the last frame of the real-scene interface specifically comprises: Dividing the real scene interface and the previous frame real scene interface into 16×16 small blocks; Select N small blocks in the real scene interface, and calculate the relative displacement between each small block in the selected N small blocks and each small block in the previous frame real scene interface; Calculate the relative displacement D between the real scene interface and the previous frame real scene interface based on the following formula: $\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; N</span>}}\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; ;</span>$ Wherein, d(n) is characterized as the relative displacement between the nth small block in the N small blocks and the nth small block in the real scene interface of the previous frame, and the value range of n is [1-N]. 6. A device for implementing a program interface in an application program, characterized in that it comprises: A real-scene module, configured to construct a real-scene interface according to the obtained real-scene image, and establish a real-scene coordinate system of the real-scene interface; A mathematical module, configured to construct a mathematical interface according to a mathematical image, and establish a mathematical coordinate system of the mathematical interface; A binding module, configured to establish a binding relationship between the real scene coordinate system and the mathematical coordinate system; A calculation module, configured to calculate the position of the real-scene interface in the real-scene coordinate system in the mathematical coordinate system according to the binding relationship; The program interface generation module is used to combine the real-scene interface with the mathematical interface in the mathematical coordinate system to generate the program interface. 7. The device according to claim 6, wherein the binding module specifically comprises: Determine the origin coordinate submodule, used to determine the coordinates of the origin of the real scene coordinate system in the mathematical coordinate system as O(p, q); Determine the included angle submodule, which is used to determine the clockwise included angle θ from the x' axis of the real scene coordinate system to the x axis of the mathematical coordinate system; The calculation module includes: Determine the coordinate submodule, for determining the coordinates of the midpoint P in the real scene interface in the real scene coordinate system as P(x', y'); The coordinate transformation submodule is used to calculate the coordinates P(x, y) of the point P in the mathematical coordinate system by coordinate transformation as $P(x,\mathrm{the\; y})=\left[\begin{array}{ccc}\mathrm{Cos\&theta;}& \mathrm{Sin\&theta;}& 0\\ -\mathrm{Sin\&theta;}& \mathrm{Cos\&theta;}& 0\\ p& q& 1\end{array}\right]P(x^{\&prime;},{\mathrm{the\; y}}^{\&prime;}).$ 8. The device according to claim 6 or 7, wherein the binding module is further configured to re-establish the real scene coordinate system and the mathematical coordinate system when the mathematical interface is displaced relative to the real scene interface. The binding relationship of the system. 9. The device according to claim 6 or 7, further comprising: A displacement calculation module, configured to calculate the relative displacement between the current real-scene interface and the last frame of the real-scene interface when the real-scene interface changes; The moving module is configured to move the mathematical interface to a corresponding position in the mathematical coordinates according to the calculated relative displacement between the real-scene interface and the previous frame of the real-scene interface. 10. The device according to claim 11, wherein the displacement calculation module specifically comprises: The image block sub-module is used to divide the real-scene interface and the previous frame of the real-scene interface into 16×16 small blocks; The displacement calculation submodule is used to select N small blocks in the real-scene interface, and calculates the relative displacement between each small block in the selected N small blocks and each small block in the previous frame of the real-scene interface; Calculate the relative displacement D between the real scene interface and the previous frame real scene interface based on the following formula: $\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; N</span>}}\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; ;</span>$ Wherein, d(n) is characterized as the relative displacement between the nth small block in the N small blocks and the nth small block in the real scene interface of the previous frame, and the value range of n is [1-N].

Images