JP2001229408A - 3D virtual image experience system - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To provide a three-dimensional virtual image sensation system capable of obtaining a higher real sensation in a world of virtual realization. A three-dimensional virtual image display device (1) for displaying a three-dimensional virtual image in an image space, a force sense bodily sensation device (2) capable of experiencing a sense of force with respect to a virtual object, and a control operation device (3) for controlling these two devices
And a coordinate conversion unit 12 that matches the spatial coordinates for the image with the spatial coordinates for the haptic device on the haptic device side, and obtains a contact state between the virtual object and the three-dimensional virtual image. A contact state calculation unit 13; a reaction force calculation unit 14 that calculates the reaction force to the virtual object by inputting the contact state; and a deformation that calculates the amount of deformation of the stereoscopic virtual image by inputting the video data of the stereoscopic virtual image and the contact state. In addition to the provision of the amount calculation unit 15, the amount of deformation is reflected in the video data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional virtual image sensation system in which a force sense such as a reaction force to a three-dimensional virtual image projected in a space can be sensed by superimposing it on an image.

[0002]

2. Description of the Related Art Along with the remarkable development of a microprocessor in recent years, a haptic sensation system capable of experiencing a tactile sensation of a three-dimensional object by operating a hand unit of a robot apparatus has been developed. This force sensation system creates a virtual object in space, operates the hand part to make the tip virtually touch the virtual object, and transmits the reaction force to the tip of the hand part according to the mutual positional relationship. The operation target is displayed on the display device so that the operation can be visually checked.

[0003]

In the haptic system, the operation target is a virtual object, which is visually displayed on a display device distant from the operation location. When trying to acquire a technique using the system, there is a problem that the technique cannot be sufficiently acquired depending on the type of technique to be acquired. That is, in the world of virtual realization, a higher physical feeling is desired.

Accordingly, the present invention provides a three-dimensional virtual image sensation system in which an extremely high physical sensation is obtained by projecting a three-dimensional virtual image of the virtual object at the position of a virtual object which is an operation object for experiencing a force sensation. The purpose is to provide.

[0005]

According to a first aspect of the present invention, there is provided a three-dimensional virtual image display device for displaying a three-dimensional virtual image in a space, and an operation capable of performing a predetermined operation on a virtual object. A force sensation sensation device having a member and capable of experiencing a haptic sensation on a virtual object in accordance with the operation of the operation member, and a control operation device for controlling both of these devices. Data generating unit for inputting the position and external shape data to generate video data for displaying a stereoscopic virtual image in the video space, and inputting the position of the operation member of the haptic device and the haptic device A coordinate conversion unit that converts the spatial coordinates for haptics on the side to spatial coordinates for video, and a contact state performance that calculates the contact state between the virtual object, which is the haptic object displayed in the video space, and the three-dimensional virtual image A reaction force calculation unit that inputs a contact state obtained by the contact state calculation unit and calculates a reaction force against the virtual object; and a video data obtained by the video data creation unit and the contact state calculation unit. A deformation amount calculation unit that calculates the deformation amount of the stereoscopic virtual image by inputting the obtained contact state, and outputs the deformation amount calculated by the deformation amount calculation unit to the video data creation unit to perform the deformation. This is a three-dimensional virtual image sensation system configured to display a later three-dimensional virtual image.

A second means of the present invention has a three-dimensional virtual image display device for displaying a three-dimensional virtual image in a space, an operating member capable of performing a predetermined operation on a virtual object, and responding to the operation of the operating member. Haptic device that can sense the haptic sensation, a control operation device that controls these two devices, and the position of the coordinates of the image space by the stereoscopic virtual image display device and the coordinates of the haptic space by the haptic device. And a position matching means for performing alignment, and to the control arithmetic unit, inputting the position and outline data of the three-dimensional virtual image to create image data for displaying the three-dimensional virtual image in the space for images. A coordinate conversion unit for inputting the position of the operation member of the haptic device and converting the spatial coordinates for haptics on the haptic device side into spatial coordinates for video; and a haptic sense displayed in the video space. versus A contact state calculation unit for calculating the contact state between the virtual object and the stereoscopic virtual image, and a reaction force calculation unit for inputting the contact state obtained by the contact state calculation unit and calculating a reaction force on the virtual object A deformation amount calculation unit configured to calculate the deformation amount of the stereoscopic virtual image by inputting the video data obtained by the video data creation unit and the contact state obtained by the contact state calculation unit; The deformation amount calculated by the section is output to the video data creation section to display the stereoscopic virtual image after the deformation, and further, as the position matching means, a reference arranged at a predetermined position in the video space. This is a three-dimensional virtual image sensation system using a reference object having external dimensions or a reference three-dimensional virtual image displayed at a predetermined position.

According to each of the three-dimensional virtual image sensation systems described above, the space for force sensation that can sense haptics and the space for video that displays a three-dimensional virtual image are made the same space, and the coordinate system of the operation member and the three-dimensional object that is the operated part are formed. While matching the virtual image coordinate system,
Since the deformation of the operated portion caused by the operation of the operation member is reflected in the three-dimensional virtual image, the real feeling is extremely good, and a high immersion feeling can be obtained in the virtual reality world.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A stereoscopic virtual image sensation system according to an embodiment of the present invention will be described below with reference to FIGS.

As shown in FIGS. 1 and 2, a three-dimensional virtual image sensation system according to the present embodiment displays a three-dimensional virtual image in a three-dimensional space and displays the three-dimensional virtual image, and a reaction force ( (Including contact resistance), that is, a haptic sensation device 2 capable of experiencing (experiencing) a haptic sensation, a control operation device 3 for controlling both of these devices 1 and 2, and a three-dimensional image displayed by the three-dimensional virtual image display device 1. And a position matching means 4 for performing position matching between the virtual image and the virtual object virtually created by the haptic device 2.
These devices, the control operation device 3 and the position matching means 4 are connected to each other via a signal line 5.

The stereoscopic virtual image display device 1 includes an image display unit 1a such as a monitor and an image reflection unit (for example, a curved surface) for projecting an image output from the image display unit 1a in a three-dimensional space. 1b) (a special mirror is used).

As the haptic device 2, a robot device having an arm having a predetermined degree of freedom is used, and an operating member 2b is attached to the tip of the arm 2a. When the processing) is performed, when a reaction force (described later) generated in the virtual object corresponding to the operation amount is input, the reaction force is transmitted to the operation member 2b, and the person who is operating is operated. You can feel the reaction force. That is, the robot device is provided with a function capable of generating a reaction force.

Further, the control arithmetic unit 3 stores various data such as position data of the three-dimensional virtual image to be displayed, outer shape data, material characteristic data of the virtual object necessary for calculating the reaction force, and colors. Necessary data is input from a data storage unit, for example, an external storage device (which may be an internal storage unit) 10, processed according to predetermined conditions, and stored in a predetermined space (hereinafter also referred to as a video space). A video data generating unit 11 provided with a graphic display program for generating video data for displaying a stereoscopic virtual image, and a portion expressing the sense of force of the haptic device 2, that is, provided at the tip of the arm 2a of the robot. The user inputs the position of the operating member (hereinafter, also referred to as an operation part) 2b, and sits in a space (hereinafter, also referred to as a force sense space) representing a virtual object on the force sense device 2. Is converted into coordinates on the image space side in the stereoscopic virtual image display device 1, and the virtual object subjected to coordinate conversion by the coordinate conversion unit 12 and the stereoscopic virtual image displayed in the image space. A contact state calculating unit 13 for obtaining a contact state, and a built-in program for inputting the contact state obtained by the contact state calculating unit 13 and the material characteristic data from the external storage device 10 to generate a reaction force generated in the virtual object. A reaction force calculator 14 for calculating, and the video data obtained by the video data generator 11 and the contact state obtained by the contact state calculator 13 are input to calculate the amount of deformation of the three-dimensional virtual image and to calculate the calculated amount. And a deformation calculating unit 15 for outputting the deformation amount to the video data creating unit 11.

The position aligning means 4 includes a scale plate 21 disposed at a predetermined position in the image space, and a reference object having a predetermined shape and a predetermined size mounted on the scale plate 21 (for example, one side is fixed). The coordinate of the reference object 22 and the coordinate position of the operation part in the haptic device 2 are input to the control arithmetic unit 3 in the image space. And a correction amount calculation unit 16 that calculates a correction amount that is a difference between the coordinates of the force sense space and the coordinates of the force sense space. The position matching means 4 is provided with a position determination button for performing position alignment, or an instruction tool 23 such as a keyboard (not shown) for inputting an arrangement position. However, the reference object 22 in the image space
If the arrangement position is determined, if the position is input to the correction amount calculation unit 16 in advance, it is not necessary to provide the above-described pointing tool.

Next, a description will be given of a case where training of dental treatment is performed using the stereoscopic virtual image sensation system. First, the coordinates of the force sense space on the force sense feeling device 2 side are made to match the coordinates of the image space on the stereoscopic virtual image display device 1 side. That is, the scale plate 21 is arranged at a predetermined position in the image space, and the reference object 22 is placed at a predetermined position of the scale plate 21, and then provided at the tip of the arm portion 2 a of the haptic device 2. The distal end of the dental treatment instrument (of course, a pseudo treatment instrument) P, which is the operation member 2b, is sequentially moved to a reference position of the reference object 22, for example, each vertex of the reference object 22, and a position determination button or the like is moved. Pointer 2
3 is used to make the user recognize that each is a predetermined vertex.

Then, the coordinate data of the recognized reference position of the reference object 22 is input to the correction amount calculation unit 16 and the deviation from the coordinates in the video space is calculated. The calculated shift is input to the coordinate conversion unit 12 as a correction amount,
Arm 2a from haptic device 2 in subsequent processing
Are converted (coincided) from the spatial coordinates for force sense to the spatial coordinates for video, so that both are represented in the same coordinate space.

By the way, video data of the teeth of the subject (patient) is input to the video data creating unit 11 of the control arithmetic unit 3 and, for example, by operating a display switch, as shown in FIG. In the virtual image display device 1, a three-dimensional image of the tooth part, that is, a three-dimensional virtual image Q is displayed in the image space.

Then, the trainee grasps the dental treatment instrument P, which is the operation member 2b of the arm 2a of the haptic device 2,
The treatment (treatment) is performed on the tooth portion Q as a stereoscopic virtual image.
At this time, when the dental treatment tool P comes into contact with the tooth Q, the coordinate data of the operating portion of the arm 2a (which is also the tip position of the dental treatment tool) is always input to the contact state calculation unit 13 and at the same time. The image data of the tooth portion Q is input from the image data creation unit 11 to the contact state calculation unit 13, and the difference, that is, the presence or absence of the contact and the degree of the contact are output as the deformation amount.

The amount of deformation calculated by the contact state calculation unit 13 and the material property data from the external storage device 10 are input to the reaction force calculation unit 14, and the reaction force data representing the reaction force is input to the force sensation device 2. Input (feedback).

When the reaction force data is input to the haptic device 2, a force is applied to the arm 2a based on the reaction force data, and a predetermined reaction force is applied to the operation portion. At the same time, the deformation amount calculating unit 15 calculates the deformation amount of the operation part from the contact state, and sends the deformation amount to the video data creating unit 11 to generate the deformed tooth portion Q. An image is projected in the image space by the stereoscopic virtual image display device 1.

That is, the trainee can feel the force sense generated during the dental treatment, and at the same time, can see a stereoscopic virtual image of the tooth portion corresponding to the treatment at hand.
Special and advanced dental treatment techniques can be mastered without the use of costly tooth models. Further, by inputting a program that assumes an accident caused by a sudden movement of the subject, for example, a situation in which the tongue moves when swallowing the brim, training for avoiding the accident can be performed.

As described above, the space for force sensation capable of experiencing a force sensation and the space for video for displaying a three-dimensional virtual image are formed in the same coordinate space, and the operated member (operated portion) is operated by operating the operating member (operating portion). Since the deformation of the operation part) is also reflected on the three-dimensional virtual image, the real feeling is extremely good.

In the above embodiment, various data of the virtual object are stored in the external storage device, and necessary data is input from the external storage device to the video data creating section and the reaction force calculating section. However, for example, it is also possible to directly input manually with input means such as a keyboard, and the external storage device includes a computer device or a storage device that can be connected on a network.

In the above embodiment, the scale plate and the reference object are used for positioning the coordinates of the image space and the force sense space. For example, a stereoscopic virtual image for positioning is used. You can also.

In this case, for example, a cubic image as described in the above embodiment is used as a reference three-dimensional virtual image as a three-dimensional virtual image for positioning, and data for creating this image is video data creation. Input to the unit 11. Of course, the position where the reference stereoscopic virtual image is projected in the space for video is predetermined, and therefore, the operating member 2b of the arm 2a of the haptic device 2 is moved to a predetermined position of the reference stereoscopic virtual image, for example, each vertex. , And the position may be input to the correction amount calculation unit 16. Note that the size of the reference stereoscopic virtual image may be changed in, for example, several steps. In this case, the changed step is input to the correction amount calculation unit of the control calculation device using, for example, an indicator. Is done.

Further, in the above-described embodiment, the position matching means 4 is provided in addition to the stereoscopic virtual image display device 1 and the force sensation device 2, but, for example, the position matching means is not provided. The control operation device may be provided with a correction function for matching the coordinates of the force sense space with the coordinates.

That is, the control arithmetic unit is provided with an input means for directly inputting a correction amount, or a function capable of aligning an operation portion with a stereoscopic virtual image itself displayed by the stereoscopic virtual image display device. You may have it.

In the above-described embodiment, the case of performing training for dental treatment has been described. However, the present invention is not limited to this training. For example, training for adding operations and processing to three-dimensional objects, or playing games Applicable to equipment,
More specifically, the present invention can be applied to the production of crafts, for example, when experiencing the production of pottery and the like using clay, and when performing as a hobby, the immersion (substantial feeling) is extremely high.

[0028]

As described above, according to the three-dimensional virtual image sensation system of the present invention, the space for force sensation that can sense haptics and the space for video for displaying a three-dimensional virtual image are made the same space, and the coordinate system of the operating member is set. And the coordinate system of the three-dimensional virtual image that is the operated part, and the deformation of the operated part caused by the operation of the operating member is reflected on the three-dimensional virtual image, so that the real feeling is very good. That is, in the virtual reality world, an unprecedentedly high immersion feeling can be obtained, and therefore, it is possible to learn various advanced special skills in a short time in an environment equivalent to the real world.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a schematic configuration of a stereoscopic virtual image sensation system according to an embodiment of the present invention.

FIG. 2 is a perspective view illustrating a position alignment operation in the stereoscopic virtual image sensation system.

FIG. 3 is a perspective view illustrating training of dental treatment in the stereoscopic virtual image sensation system.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 stereoscopic virtual image sensation system 2 haptic sensation device 2a arm unit 2b operation member 3 control operation unit 4 position matching unit 11 video data creation unit 12 coordinate conversion unit 13 contact state operation unit 14 reaction force operation unit 15 deformation amount operation unit 16 correction Quantity calculation part P Dental treatment instrument Q Tooth part

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] March 22, 2000 (200.3.2)
2)

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] Explanation of sign

[Correction method] Change

[Correction contents]

[Description of Signs ] 1 stereoscopic virtual image display device 2 haptic sensory device 2a arm unit 2b operation member 3 control operation device 4 position matching means 11 video data creation unit 12 coordinate conversion unit 13 contact state operation unit 14 reaction force operation unit 15 deformation Amount calculation unit 16 Correction amount calculation unit P Dental treatment instrument Q Tooth

[Procedure amendment 2]

[Document name to be amended] Drawing

[Correction target item name] Figure 2

[Correction method] Change

[Correction contents]

FIG. 2

 ────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Haruyo Tamura 1-7-89 Minami Kohoku, Suminoe-ku, Osaka-shi, Osaka F-term in Hitachi Zosen Corporation (reference) 3F059 AA10 BA07 BC07 BC09 BC10 CA05 CA08 DA02 DA05 DA08 DB04 DB09 DC01 DC04 DD06 DE03 EA07 FA03 FA05 FA10 FB12 FB17 FB29 FC02 FC03 FC04 FC06 FC07 FC13 FC14 5B050 BA07 BA08 BA09 CA07 FA02 FA08 9A001 BB04 DD12 HH19 HH29 HH30 HH32 HH34

Claims (3)

[Claims] 1. A stereoscopic virtual image display device for displaying a stereoscopic virtual image in a space, an operation member capable of performing a predetermined operation on a virtual object, and a sense of force on the virtual object in response to the operation of the operation member. And a control operation device for controlling both of these devices. The control operation device inputs the position and outline data of the three-dimensional virtual image to display the three-dimensional virtual image in the video space. A video data creation unit that creates video data, a coordinate conversion unit that inputs the position of the operation member of the haptic device and converts the haptic space coordinates on the haptic device side into image space coordinates, A contact state calculation unit for calculating a contact state between the virtual object which is a haptic object displayed in the image space and the three-dimensional virtual image, and a contact state obtained by the contact state calculation unit being input to the virtual object versus A reaction force calculation unit for calculating a reaction force to be applied, and a deformation amount calculation for calculating a deformation amount of a stereoscopic virtual image by inputting the video data obtained by the video data creation unit and the contact state obtained by the contact state calculation unit. A stereoscopic virtual image sensation system, wherein the stereoscopic virtual image after deformation is displayed by outputting the deformation amount calculated by the deformation amount calculation unit to the video data creating unit. . 2. A three-dimensional virtual image display device for displaying a three-dimensional virtual image in a space, a force member having an operation member capable of performing a predetermined operation on a virtual object, and capable of experiencing a force sense in response to the operation of the operation member. It comprises a sensation device, a control operation device for controlling both of these devices, and a position matching unit for aligning the coordinates of the image space by the stereoscopic virtual image display device with the coordinates of the haptic space by the haptic device. A video data creation unit for inputting the position and outline data of the stereoscopic virtual image to the control arithmetic unit and generating video data for displaying the stereoscopic virtual image in the video space; and an operation member of the haptic device. A coordinate conversion unit for inputting a position and converting the spatial coordinates for haptics on the side of the haptic device to spatial coordinates for a video, and a virtual object which is a haptic object displayed in the video space and a three-dimensional virtual image. Contact A contact state calculator for calculating a touch state, a reaction force calculator for inputting the contact state obtained by the contact state calculator and calculating a reaction force on the virtual object, And a deformation amount calculation unit for calculating the deformation amount of the stereoscopic virtual image by inputting the video data and the contact state obtained by the contact state calculation unit, and calculating the deformation amount calculated by the deformation amount calculation unit. A configuration is used in which the stereoscopic virtual image after the transformation is displayed by outputting to the video data creating unit, and a reference object having a standard external dimension arranged at a predetermined position in the video space is used as the position matching means. A stereoscopic virtual image sensation system characterized by the following. 3. The position matching means according to claim 2, wherein a reference stereoscopic virtual image having a predetermined reference outer dimension is used instead of the reference object placed at a predetermined position in the video space. 3D virtual image experience system.