[go: up one dir, main page]

WO2008145169A1 - Dispositif mobile et procédé pour un affichage rétinien virtuel - Google Patents

Dispositif mobile et procédé pour un affichage rétinien virtuel Download PDF

Info

Publication number
WO2008145169A1
WO2008145169A1 PCT/EP2007/004843 EP2007004843W WO2008145169A1 WO 2008145169 A1 WO2008145169 A1 WO 2008145169A1 EP 2007004843 W EP2007004843 W EP 2007004843W WO 2008145169 A1 WO2008145169 A1 WO 2008145169A1
Authority
WO
WIPO (PCT)
Prior art keywords
mobile device
user
image
pose
face
Prior art date
Application number
PCT/EP2007/004843
Other languages
German (de)
English (en)
Inventor
Thomas Grosch
Original Assignee
Siemens Aktiengesellschaft
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens Aktiengesellschaft filed Critical Siemens Aktiengesellschaft
Priority to PCT/EP2007/004843 priority Critical patent/WO2008145169A1/fr
Publication of WO2008145169A1 publication Critical patent/WO2008145169A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/74—Projection arrangements for image reproduction, e.g. using eidophor
    • H04N5/7475—Constructional details of television projection apparatus
    • H04N5/7491—Constructional details of television projection apparatus of head mounted projectors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3141—Constructional details thereof
    • H04N9/3173—Constructional details thereof wherein the projection device is specially adapted for enhanced portability
    • H04N9/3176—Constructional details thereof wherein the projection device is specially adapted for enhanced portability wherein the projection device is incorporated in a camera

Definitions

  • the invention relates to a mobile device and a method for projecting an image on a retina of a user by means of a projection unit.
  • US 2007/0052672 A1 discloses a device for virtual retinal display (VNA) in the form of spectacles.
  • VNA virtual retinal display
  • a raster image can be projected directly onto the retina of a user's eye so that the user sees the raster image as projected onto a screen floating in the air in front of it.
  • the projection unit used for the display is rigidly coupled with respect to its position and position to the retina of the user.
  • tracking methods are known with which the position and position of a user or of a user-carried camera can be compared with respect to the real one he is considering Environment can be determined.
  • a method is known in which the position and orientation of a camera is determined by means of a video sequence recorded by the camera by means of electronic image processing.
  • the invention has for its object to provide a virtual retina display with a mobile device whose position and position is variable with respect to the user.
  • a mobile device having a tracking system for continuously determining the pose of the mobile device with respect to a user, a projection unit for projecting an image on a retina of a user, and
  • Control means for controlling the projection unit in accordance with the pose determined by the tracking system such that the image can be projected onto the retina.
  • the object is achieved by a method for projecting an image onto a retina of a user by means of a projection unit integrated in a mobile device, in which a tracking system of the mobile device continuously detects the pose of the mobile device with respect to the user, and the projection unit is controlled in accordance with the pose determined by the tracking system so that the image can be projected onto the retina.
  • the pose is understood as the combination of position and orientation in three-dimensional space. Accordingly, the position and orientation of the mobile device with respect to the user is understood here and in the entire document under the pose.
  • the invention is based on the recognition that a virtual retina display with a mobile device, which in no way rigid with a Eye of the user can be realized, if the position and orientation of the mobile device with respect to the user or its retina are constant. Since the position and the orientation can be changed during the display due to the mobile character of the device according to the invention, the pose is continuously detected with the aid of the tracking system. As soon as the pose changes, eg in which the user turns the mobile device or leads away from it, this is detected by the tracking system. With the help of the control means, this pose change is compensated, so that the image to be displayed can continue to be projected onto the retina of the user.
  • the mobile device is enabled to detect movements that it itself performs with respect to the user in three dimensions and to compensate for them with the aid of the control means. In this way, it is possible to dispense with the fixed positioning of the projection unit for the virtual retina display relative to the eye of the user.
  • the invention enables the incorporation of virtual retinal displays into any portable device, such as a mobile device. Mobile phones, PDAs, laptops, mobile control panels, etc.
  • the tracking system can use various tracking algorithms known, for example, from the area of augmented reality, in order to continuously determine the position and position of the mobile device in relation to the user.
  • the mobile device comprises a camera for recording the user's face and the mobile device has image recognition means for determining image features on the basis of the image.
  • the pose of the mobile device with respect to the user can be determined with the aid of image processing methods, such as those used in so-called markerless tracking in the augmented reality environment.
  • the mobile device has a display for simultaneously displaying the image characteristics and the face detected by means of the camera. In this way, the user is able to first put the mobile device in a so-called initial pose. Here it is assumed that the user has first created an image of his face with the help of the camera. Based on this image, the said image features are determined.
  • the camera first uses the camera to recapture his face and display it on the screen.
  • the previously extracted image features are shown in the display. The user now has the opportunity to change the mobile device in its position and its position until the image features are brought into line with the captured by the camera image of his face. Once this has been done, it is ensured that the mobile device is again almost in the initial position at which the user's face was first recorded.
  • the projection unit is set so that it can project the image onto the user's retina when the initial pose is taken, subsequent changes in the pose can be compensated with the aid of the tracking system as described above.
  • the subsequent continuous compensation of a Posen- change after the mobile device has been brought into the initial pose can be implemented in an advantageous embodiment of the invention in that the camera is designed for continuous detection of the face during the continuous determination of the pose and the tracking system Image processing means for continuously determining the pose by comparing the image features and the continuously detected face comprises.
  • the tracking system comprises motion sensors for the continuous determination of the pose.
  • the mobile device is designed as a mobile phone.
  • a mobile telephone which is equipped with the retina projection possibility according to the invention can take into account in an outstanding manner the general user wish of a portable device which is as small and handy as possible, nevertheless with good display characteristics.
  • the projection unit for the projection of the image is formed on both retinae of the user, so that the user can perceive the image 3-dimensionally.
  • the projection unit for the projection of the image is formed on both retinae of the user, so that the user can perceive the image 3-dimensionally.
  • the figure shows schematically a method in which an image is projected onto a retina of a user 2 with the aid of a mobile device 1 designed as a mobile telephone.
  • the mobile device 1 or the mobile phone comprises a
  • Projection unit 3 which is integrated in the mobile phone. Since the mobile phone is not in a fixed position to the eye of the user 2, it must be able to determine the position and the viewing direction of the user 2. This is done by using a markerless tracking method, as explained below.
  • a tracking system is implemented which uses features relevant to the tracking from the face of the user 2.
  • the user 2 has to take a picture of his face once with a camera integrated in the mobile device 1.
  • the mobile device 1 uses image processing algorithms known from the prior art to determine characteristic image features that can be used for later tracking.
  • the image features 4 are displayed in a conventional display of the mobile device 1.
  • the projection unit 3 of the mobile device 1 is calibrated so that it can project into a retina of the user in the pose in which the user 2 once takes the image of his face.
  • the user wants to use the mobile device 1 for virtual retina display, he will in turn record his face with the help of installed in the mobile device 1 camera. At this stage, his face is continuously picked up by the camera.
  • the image of its continuously recorded face is displayed together with the previously determined image features.
  • the user 2 now manually brings the mobile device 1 in the pose, in which the displayed image features are with his recorded face on the conventional display in cover. Once this has been done, it is ensured that the mobile device is again in the initial pose, in which the projection unit can project the image to be displayed onto the retina of the user 2. Subsequently, poses changes can be compensated with the help of the tracking system such that the virtual retina display continues to hit the user retina.
  • the distance of the mobile device 1 to the eye of the user can be determined optically based on the lens settings and the geometry of the camera. If the If the mobile device 1 leads the mobile device 1 farther away from its eye, the focal length of the camera objective can be adjusted in such a way that the image acquired by the user 2 continues to coincide with the superimposed image features. In turn, it is possible to draw conclusions about the extent to which the projection needs to be changed so that it can continue to project onto the retina of the user 2.
  • Movements of the mobile device 1 itself can be detected by a sensor installed in the mobile device 1.
  • the projection unit can be adapted to the movements performed by the mobile device 1 and thus compensate for them.
  • the user is projected a stable image on the retina, even if the distance to the mobile device 1 changes or the mobile device 1 itself changes its position or orientation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention concerne un dispositif mobile (1) et un procédé pour projeter une image sur la rétine d'un utilisateur (2). L'objectif de l'invention est de permettre un affichage rétinien virtuel grâce à un dispositif mobile (1), la position et l'orientation dudit dispositif par rapport à l'utilisateur (2) pouvant être modifiées. A cet effet, l'invention est caractérisée en ce que la position du dispositif mobile (1) par rapport à l'utilisateur (2) peut être déterminé de manière continue au moyen d'un système de poursuite appartenant au dispositif mobile (1), et que l'unité de projection (3) peut être commandée de manière à correspondre à la position déterminée par le système de poursuite, de sorte que l'image peut être projetée sur la rétine.
PCT/EP2007/004843 2007-05-31 2007-05-31 Dispositif mobile et procédé pour un affichage rétinien virtuel WO2008145169A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/EP2007/004843 WO2008145169A1 (fr) 2007-05-31 2007-05-31 Dispositif mobile et procédé pour un affichage rétinien virtuel

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2007/004843 WO2008145169A1 (fr) 2007-05-31 2007-05-31 Dispositif mobile et procédé pour un affichage rétinien virtuel

Publications (1)

Publication Number Publication Date
WO2008145169A1 true WO2008145169A1 (fr) 2008-12-04

Family

ID=38420629

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2007/004843 WO2008145169A1 (fr) 2007-05-31 2007-05-31 Dispositif mobile et procédé pour un affichage rétinien virtuel

Country Status (1)

Country Link
WO (1) WO2008145169A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5759044A (en) * 1990-02-22 1998-06-02 Redmond Productions Methods and apparatus for generating and processing synthetic and absolute real time environments
WO2001008414A1 (fr) * 1999-07-22 2001-02-01 Swisscom Mobile Ag Procede et dispositifs correspondant pour delivrer des informations utiles concernant des objets observes
US20030234823A1 (en) * 2002-05-17 2003-12-25 Shigemi Sato Image processing apparatus and image processing method, and image processing program and recording medium of the same
US20060028400A1 (en) * 2004-08-03 2006-02-09 Silverbrook Research Pty Ltd Head mounted display with wave front modulator
US20060197832A1 (en) * 2003-10-30 2006-09-07 Brother Kogyo Kabushiki Kaisha Apparatus and method for virtual retinal display capable of controlling presentation of images to viewer in response to viewer's motion

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5759044A (en) * 1990-02-22 1998-06-02 Redmond Productions Methods and apparatus for generating and processing synthetic and absolute real time environments
WO2001008414A1 (fr) * 1999-07-22 2001-02-01 Swisscom Mobile Ag Procede et dispositifs correspondant pour delivrer des informations utiles concernant des objets observes
US20030234823A1 (en) * 2002-05-17 2003-12-25 Shigemi Sato Image processing apparatus and image processing method, and image processing program and recording medium of the same
US20060197832A1 (en) * 2003-10-30 2006-09-07 Brother Kogyo Kabushiki Kaisha Apparatus and method for virtual retinal display capable of controlling presentation of images to viewer in response to viewer's motion
US20060028400A1 (en) * 2004-08-03 2006-02-09 Silverbrook Research Pty Ltd Head mounted display with wave front modulator

Similar Documents

Publication Publication Date Title
DE102017208806B3 (de) Externe Darstellung von Bildaufnahmen eines Fahrzeuginnenraums in einer VR-Brille
DE102014214516A1 (de) Vorrichtung und Verfahren zur Wiedergabe von Daten in einer erweiterten Realität für einen Insassen eines Fahrzeugs
EP3394708A1 (fr) Procédé de fonctionnement d'un système de réalité virtuelle et système de réalité virtuelle associé
DE102016212687A1 (de) Verfahren zum Betreiben einer Anzeigeeinrichtung eines Kraftfahrzeugs
DE102014213021A1 (de) Lokalisierung eines HMD im Fahrzeug
DE102016223179A1 (de) Koordinaten-Abgleichvorrichtung für ein Head-Up-Display
DE102014213285A1 (de) Kopfrichtungsabhängige Anzeige von Inhalten auf einer Datenbrille
DE112020002727T5 (de) Videoverteilsystem, videoverteilverfahren und display-terminal
DE102022113343A1 (de) Verfahren und Vorrichtung zum Betreiben eines Anzeigesystems in einer mobilen Einrichtung mit einer Datenbrille
DE102015204745A1 (de) Vorrichtung und Verfahren zur Wiedergabe von Daten in einer erweiterten Realität
DE102015003948A1 (de) Verfahren zum Betreiben einer in einem Kraftfahrzeug angeordneten Virtual-Reality-Brille und Virtual-Reality-System
DE102014013408A1 (de) Verfahren zur Darstellung von Umgebungsinformationen eines Fahrzeugs
DE102014009699B4 (de) Verfahren zum Betreiben einer Anzeigeeinrichtung und System mit einer Anzeigeeinrichtung
DE102019103360A1 (de) Verfahren und Vorrichtung zum Betreiben eines Anzeigesystems mit einer Datenbrille
WO2022218698A1 (fr) Procédé et dispositif de détermination d'une pose dans des lunettes intelligentes
DE102014018056A1 (de) Verfahren zum Betreiben einer Virtual-Reality-Brille und Virtual-Reality-Brille
DE102015006610B4 (de) Verfahren zum Betreiben einer Datenbrille in einem Kraftfahrzeug und System mit einer Datenbrille
DE102016225262A1 (de) Verfahren und Vorrichtung zum Betreiben eines Anzeigesystems mit einer Datenbrille
WO2019228780A1 (fr) Concept pour la commande d'un écran d'un appareil mobile à réalité augmentée
WO2008145169A1 (fr) Dispositif mobile et procédé pour un affichage rétinien virtuel
EP3129861A1 (fr) Détermination de la pose de lunettes de réalité augmentée dans le véhicule
EP3271774B1 (fr) Dispositif et procédé de reproduction de données dans une réalité étendue
DE102018211624A1 (de) Verfahren und Benutzerschnittstelle zur Erfassung einer Eingabe mittels Zeigegestik
DE102012202957B4 (de) Anzeigevorrichtung
DE102014105011B4 (de) System zur Visualisierung des Sichtfeldes einer optischen Vorrichtung

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07725724

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 07725724

Country of ref document: EP

Kind code of ref document: A1