CN106971368A - A kind of lock in time for virtual reality crimps computational methods - Google Patents
A kind of lock in time for virtual reality crimps computational methods Download PDFInfo
- Publication number
- CN106971368A CN106971368A CN201710030989.4A CN201710030989A CN106971368A CN 106971368 A CN106971368 A CN 106971368A CN 201710030989 A CN201710030989 A CN 201710030989A CN 106971368 A CN106971368 A CN 106971368A
- Authority
- CN
- China
- Prior art keywords
- time
- virtual reality
- content
- gpu
- lock
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/16—Indexing scheme for image data processing or generation, in general involving adaptation to the client's capabilities
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Processing Or Creating Images (AREA)
Abstract
Computational methods are crimped the present invention relates to a kind of lock in time for virtual reality, the present invention program separates the processing time at CPU ends in virtual reality system and the processing time at GPU ends, the time that i.e. CPU ends are operated is not take up the GPU ends operating time, add content of GPU and render the time operated with Time warp, it is used as the virtual reality lock in time curling solution that a scope of application is wider and system delay is lower, this programme ensure that the stable smooth of whole operation process, provide the user with the experience of good virtual reality immersion sense.
Description
Technical field
The present invention relates to a kind of Time warp computational methods, based on especially a kind of lock in time curling by virtual reality
Calculation method.
Background technology
Virtual reality, also referred to as virtual reality (Virtual Reality), abbreviation VR technologies refer to utilize computer simulation
Simulation of the virtual world there is provided user on sense organs such as vision, the sense of hearing, tactiles of a three dimensions is produced, user is allowed
As on the spot in person, the things in three dimensions can be observed in real time, without limitation.
A crucial technical indicator for influenceing virtual reality system feeling of immersion is system delay time, system delay time
Refer to that the virtual reality device attitude information worn from user changes to the image for reacting the attitude information to be shown in wear
Time interval (Motion-to-Photon) on display, this time interval is shorter, it is meant that picture that user sees with from
Body athletic posture is more matched, and user does not just allow to be also easy to produce spinning sensation, but is due to screen display, graphics calculations, data transfer, biography
Inevitably there is certain delay in the influence of the factors such as sensor response, virtual reality system.
According to prior art, if it is desired to obtain preferable Consumer's Experience, virtual reality system time delay has to small
In 20ms.In order to reduce system delay, when figure, which renders frame per second, does not reach corresponding requirements, it can used during graphics calculations
In Time warp (Time Warping, TW) technology, existing lock in time crimping techniques scheme, in a vertical synchronizing signal
It is interior in CPU (central processing unit) after the task of game engine distribution content generation when (Vertical Sync, VSYNC) arrives
Hold Rendering operations and Time warp operation is sequentially performed in same GPU (graphics processor) queue, i.e., only in completing
Hold after Rendering operations, just carry out Time warp operation, one of this scheme has the disadvantage that the operation of game engine distributed tasks can take
The rendering content time, and when a certain frame render time is long, tear occurs in picture.
Reprojection technology of the prior art can be when each vertical synchronizing signal arrives, to the rendering content of previous frame
Reprojection is once predicted, carrying out content by GPU afterwards is rendered, and reprojection is carried out to rendering content after the completion of rendering content
Operation, when before the arrival of next vertical synchronizing signal, if content is rendered operates completion with reprojection, reprojection is passed through in display
The rendering content of operation；When rendering content is time-consuming longer, do not completed when next vertical synchronizing signal arrives content render and
Reprojection is operated, then shows the previous frame content by prediction reprojection, and the method is collected when there is prediction reprojection operation
Virtual reality device attitude information and vertical synchronizing signal arrive after when show rendering content virtual reality device posture letter
Cease the weak point of existence time difference.
In order to tackle above mentioned problem, the present invention devises a kind of lock in time curling computational methods, to CPU ends game engine
Operation it is without any changes, and GPU ends are optimized, the GPU operation of all submissions are deferred to next vertical synchronization
The time point that signal is reached just starts to perform.The method advantage is whole lock in time crimping techniques operation not by GPU hardware
The limitation for the technical support that manufacturer is provided, and the task distribution operation of CPU game engines is not take up content of GPU and renders and the time
The time of crimp operation, the relatively sufficient time is reserved for content Rendering operations before vertical synchronizing signal arrives, it is ensured that whole
Content renders to the stable smooth of display process, can effectively solve the problems such as display delayed is longer, be provided the user with this
Good virtual reality immersion sense.
The content of the invention
The present invention seeks to：A kind of lock in time curling computational methods are devised, lock in time volume in the prior art is solved
Bent delay and picture tear problem, while avoiding dependence of the existing Time warp technology to hardware.
The technical scheme is that：A kind of lock in time curling computational methods are devised, are arrived in vertical synchronizing signal
When, CPU ends game engine distribution content generation task, and GPU ends the GPU operation of all submissions is deferred to it is next vertical same
Walk the time point execution that signal is reached, including content rendering task and Time warp operation.
Compared with prior art, technical solution of the present invention has the advantages that：The present invention program changes traditional
After CPU ends submission task, the operator scheme that GPU ends are immediately performed so that by the processing time at CPU ends in virtual reality system and
The processing time at GPU ends separates, i.e., the time that CPU ends are operated is not take up the GPU ends operating time, in vertical synchronization letter
Render to operate with Time warp for content before number arriving and reserve the relatively sufficient time, be used as a scope of application wider and system
The lower virtual reality lock in time curling solution of delay, this programme ensure that the stable smooth of whole operation process, give
User provides the experience of good virtual reality immersion sense.
Brief description of the drawings
Fig. 1 is that user's posture changes to the content output stream journey figure that matches；
Fig. 2 is existing lock in time crimping techniques implementation process figure；
Fig. 3 is technical solution of the present invention lock in time to crimp implementation process figure.
Embodiment
As described in the background art, there are still be limited by hardware to the Time warp method of prior art in actual applications
Structure, is delayed longer, picture tear phenomenon occurs in content when rendering time-consuming longer the problems such as.
In the present invention program, a kind of lock in time curling computational methods are devised, when vertical synchronizing signal arrives,
CPU ends game engine distribution content generation task, and the GPU operation of all submissions is deferred to next vertical synchronization by GPU ends
The time point that signal is reached performs, including content rendering task and Time warp operation, ensures to stay for content Rendering operations with this
Go out the relatively sufficient time.
It is understandable to enable the above objects, features and advantages of the present invention to become apparent, below in conjunction with the accompanying drawings to the present invention
Specific embodiment be described in detail.
Fig. 1 is that user's posture changes to the content output stream journey figure that matches；
Step 101：User's attitudes vibration causes virtual reality device posture to change；
Step 102：Inertial Measurement Unit and other location technologies gather new virtual reality device attitude information；
Generally sample and new athletic posture and PC (PC) host side is transferred to by USB cable usually requires 1ms
Time；
Step 103：Game engine distribution of images render instruction；
The new athletic posture of game engine processing simultaneously initiates new double buffering image rendering request (CPU ends), is taken
Between with game engine and application content depending on, generally in more than 2ms；
Step 104：GPU ends complete image rendering and operated with Time warp；
GPU performs new right and left eyes image render request, required time with application content scene GPU workloads and be
Unite depending on GPU computing capability, excursion is larger (several milliseconds to a few tens of milliseconds)；
Step 105：The new image of head-mounted display output display；
After the completion of the content Rendering operations of GPU ends, new picture material, which can be sent to, wears display device, and described wears
Display device immediately will not include it on screen when receiving new image, but can be connected to vertical synchronizing signal (on
One two field picture, which is shown, to be terminated) when just start to show new content；
Can be seen that the optimal situation of system delay from described whole flow process is that 5 steps described in accompanying drawing 1 can
To be completed in two neighboring vertical synchronizing signal time interval, therefore new image can just ensure in next vertical synchronization
Signal is shown in when arriving and worn on display device, and the neighboring vertical synchronizing signal time interval wears display by virtual reality
Depending on the refresh rate of equipment, by taking the refresh rate that the virtual reality of main flow wears display device 90Hz as an example, neighboring vertical is synchronously believed
Number time interval is exactly 11.11ms, and 5 steps that completed in this time interval described in accompanying drawing 1 are more difficult, especially
User for virtual reality applications image quality and resolution requirement more and more higher, even if using top CPU and GPU hardware at present
Configuration, can not also ensure all virtual reality applications can reach this require, it is therefore desirable to using Time warp technology come
Reduce the delay of step described in accompanying drawing 1.
Fig. 2 is existing lock in time crimping techniques implementation process figure；
Step 201：CPU ends are when vertical synchronizing signal arrives, game engine distribution of images render instruction (step 202, step
Rapid 203, step 204)；
Step 205：GPU ends are after the image rendering instruction that CPU ends game engine sends is received, and GPU is according to having collected
Virtual reality device attitude information carry out content generation, such as complete rendering content 1 and Time warp and operate 1 (step 206), wash with watercolours
Contaminate content 2 operates 2 (steps 207) and rendering content 3 to operate 3 (step 208) steps with Time warp with Time warp；
The step 206, step 207 are different the time required to being rendered with the content of step 208；
It is different the time required to the Time warp operation of the step 206, step 207 and step 208；
The Time warp operation is by virtual reality device posture of the described rendering content according to current instantaneous acquiring
Information carries out Time warp calculating；
In lock in time crimping techniques, content Rendering operations in the step 206, step 207 and step 208 and when
Between crimp operation be serial operation, i.e., after the completion of only corresponding content Rendering operations, just perform the Time warp behaviour of the content
Make；
Step 209：Display control step, i.e., receive vertical synchronizing signal (step 210, step in display control apparatus
211st, step 212, step 213) after, complete the display of the virtual reality content；
In lock in time crimping techniques, if display control apparatus receives the vertical synchronizing signal, GPU is not completed
Content render with Time warp calculation procedure (step 207), then virtual reality wear on display device keep display before in
Hold (step 212 and step 211 display content are all content 1), wait next vertical synchronizing signal, it is next vertical when receiving
During synchronizing signal (step 213), if CPU and GPU completed content render with Time warp calculation procedure (step 208),
Worn in virtual reality and new content (step 213) is shown on display device.
There is an obvious shortcoming, by taking double buffering as an example the, (step of rendering content 1 in such a Time warp calculation
206) represent with rendering content 2 (step 207) and wear the display content of left and right two that display device is shown as virtual reality, when
When the appearance render time of rendering content 2 is long, step 212 shows rendering content 1, and the picture that user sees occurs that tear is existing
As influenceing virtual reality experience.
Fig. 3 is technical solution of the present invention lock in time to crimp implementation process figure；
Step 301：CPU ends are when vertical synchronizing signal arrives, game engine distribution of images render instruction (step 302, step
Rapid 303, step 304)；
Step 305：GPU ends are not immediately performed after the image rendering instruction that CPU ends game engine is sent is received, and
It is that, until next vertical synchronizing signal arrives, GPU just carries out interior according to the virtual reality device attitude information collected
Hold and produce；
Such as CPU ends are after vertical synchronizing signal (step 310) is received, (the step of game engine distribution of images render instruction 2
302), and GPU ends do not carry out image rendering operations immediately when receiving image rendering instruction, but when next vertical
When synchronizing signal (step 311) arrives, start to perform rendering content 2 and Time warp 2 (steps 307) of operation；
The step 306, step 307 are different the time required to being rendered with the content of step 308；
It is different the time required to the Time warp operation of the step 306, step 307 and step 308；
The Time warp operation is by virtual reality device posture of the described rendering content according to current instantaneous acquiring
Information carries out Time warp calculating；
In technical solution of the present invention, content Rendering operations and time in the step 306, step 307 and step 308
Crimp operation is serial operation, i.e., after the completion of only corresponding content Rendering operations, just performs the Time warp operation of the content,
Because the time that CPU ends game engine distributes render instruction is not take up the time of GPU rendering contents, therefore it can ensure virtual existing
The content worn in fact shown by display device is real-time, farthest avoids the tear of picture；
Step 309：Display control step, i.e., receive vertical synchronizing signal (step 310, step in display control apparatus
311st, step 312, step 313) after, complete the display of the virtual reality content；
In technical solution of the present invention, because the time that CPU ends game engine distributes render instruction is not take up in GPU renders
The time of appearance, the time that farthest ensure that GPU has abundance carries out content Rendering operations and Time warp operation, therefore often
When individual vertical synchronizing signal arrives, the content that virtual reality wears shown by display device passes through real-time update；
In this programme in particular cases, if display control apparatus receives vertical synchronizing signal, GPU does not complete content
Render with Time warp calculation procedure, then virtual reality wear on display device keep display before content, wait it is next
Vertical synchronizing signal, when receiving next vertical synchronizing signal, if GPU has completed content and rendered and Time warp calculating step
Suddenly, then worn in virtual reality and new content is shown on display device.
The advantage of lock in time crimping techniques is that this scheme changes traditional CPU ends and submits task in the present invention program
Afterwards, the operator scheme that GPU ends are immediately performed, so that by virtual reality system during the processing at the processing time at CPU ends and GPU ends
Between separate, i.e., the time that CPU ends are operated is not take up the GPU ends operating time, vertical synchronizing signal arrival before be interior
Hold to render to operate with Time warp and reserve the relatively sufficient time, be used as a scope of application wider and lower void of system delay
Intend real lock in time curling solution, the present invention program farthest ensure that the stable smooth of whole operation process,
Provide the user with the experience of good virtual reality immersion sense.
Specifically, the virtual reality device disclosed in the present invention program includes but is not limited to virtual reality and wears display to set
Standby and operational control unit；
The virtual reality wears display device and includes but is not limited to virtual implementing helmet, virtual reality glasses；
The virtual reality operation control device includes but is not limited to virtual reality operation handle and various operation peripheral hardwares；
It is described above, the only specific embodiment of the invention.Protection scope of the present invention is not limited thereto, any to be familiar with
Those skilled in the art the invention discloses technical scope in, the change or replacement that can be readily occurred in should all be covered
Within protection scope of the present invention.Therefore, the protection domain that protection scope of the present invention should be defined using claim as
It is accurate.
Claims (6)
1. a kind of lock in time for virtual reality crimps computational methods, it is characterised in that including：
CPU ends：When receiving vertical synchronizing signal, game engine distribution of images render instruction；
GPU ends：After the image rendering instruction that CPU ends game engine is sent is received, wait vertical synchronizing signal to arrive, receive
After vertical synchronizing signal, GPU carries out content generation according to the virtual reality device attitude information collected；
Display control end：The rendering content that elapsed time curling is calculated is shown when vertical synchronizing signal is produced.
2. a kind of lock in time for virtual reality according to claim 1 crimps computational methods, it is characterised in that institute
State CPU ends game engine and distribute content render instruction immediately when receiving vertical synchronizing signal.
3. a kind of lock in time for virtual reality according to claim 1 crimps computational methods, it is characterised in that
GPU ends are received after the image rendering instruction that CPU ends game engine is sent, and wait vertical synchronizing signal to arrive, and are received vertical same
Walk after signal, GPU ends carry out content generation.
4. a kind of lock in time for virtual reality according to claim 1 crimps computational methods, it is characterised in that institute
State GPU ends Content Creation process and be divided into content Rendering operations and the Time warp meter carried out according to virtual reality device attitude information
Calculate operation.
5. a kind of lock in time for virtual reality according to claim 1 crimps computational methods, it is characterised in that institute
State GPU ends content Rendering operations and calculate the serial progress of operation with Time warp.
6. a kind of lock in time for virtual reality according to claim 1 crimps computational methods, it is characterised in that institute
State after content render instruction and vertical synchronizing signal that GPU ends receive the game engine distribution of CPU ends, can be in next vertical synchronization
Before signal arrives in preset time, rendering content current to GPU carries out Time warp operation, display content is more pressed close to virtually
The real-time attitude of real world devices.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710030989.4A CN106971368A (en) | 2017-01-18 | 2017-01-18 | A kind of lock in time for virtual reality crimps computational methods |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710030989.4A CN106971368A (en) | 2017-01-18 | 2017-01-18 | A kind of lock in time for virtual reality crimps computational methods |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106971368A true CN106971368A (en) | 2017-07-21 |
Family
ID=59334578
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710030989.4A Pending CN106971368A (en) | 2017-01-18 | 2017-01-18 | A kind of lock in time for virtual reality crimps computational methods |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106971368A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109727305A (en) * | 2019-01-02 | 2019-05-07 | 京东方科技集团股份有限公司 | Virtual reality system screen processing method, device and storage medium |
| CN109874003A (en) * | 2019-02-28 | 2019-06-11 | 京东方科技集团股份有限公司 | VR display control method, VR display control unit and display device |
| CN113596569A (en) * | 2021-07-22 | 2021-11-02 | 歌尔光学科技有限公司 | Image processing method, apparatus and computer-readable storage medium |
| CN114009035A (en) * | 2019-06-28 | 2022-02-01 | Ati科技无限责任公司 | Real-time GPU rendering with guaranteed power management performance |
| WO2022068477A1 (en) * | 2020-09-30 | 2022-04-07 | 华为技术有限公司 | Event processing method, and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020109697A1 (en) * | 2000-12-06 | 2002-08-15 | Gardiner Harold Dee | Method for reducing transport delay in a synchronous image generator |
| CN104915200A (en) * | 2014-10-20 | 2015-09-16 | 晶晨半导体(上海)有限公司 | Video frame treatment in mobile operating system |
| CN105635805A (en) * | 2015-12-18 | 2016-06-01 | 歌尔声学股份有限公司 | Method and apparatus for optimizing motion image in virtual reality scene |
| CN105898138A (en) * | 2015-12-18 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Panoramic video play method and device |
| CN106296566A (en) * | 2016-08-12 | 2017-01-04 | 南京睿悦信息技术有限公司 | A kind of virtual reality mobile terminal dynamic time frame compensates rendering system and method |
-
2017
- 2017-01-18 CN CN201710030989.4A patent/CN106971368A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020109697A1 (en) * | 2000-12-06 | 2002-08-15 | Gardiner Harold Dee | Method for reducing transport delay in a synchronous image generator |
| CN104915200A (en) * | 2014-10-20 | 2015-09-16 | 晶晨半导体(上海)有限公司 | Video frame treatment in mobile operating system |
| CN105635805A (en) * | 2015-12-18 | 2016-06-01 | 歌尔声学股份有限公司 | Method and apparatus for optimizing motion image in virtual reality scene |
| CN105898138A (en) * | 2015-12-18 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Panoramic video play method and device |
| CN106296566A (en) * | 2016-08-12 | 2017-01-04 | 南京睿悦信息技术有限公司 | A kind of virtual reality mobile terminal dynamic time frame compensates rendering system and method |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109727305A (en) * | 2019-01-02 | 2019-05-07 | 京东方科技集团股份有限公司 | Virtual reality system screen processing method, device and storage medium |
| CN109727305B (en) * | 2019-01-02 | 2024-01-12 | 京东方科技集团股份有限公司 | Virtual reality system picture processing method, device and storage medium |
| CN109874003A (en) * | 2019-02-28 | 2019-06-11 | 京东方科技集团股份有限公司 | VR display control method, VR display control unit and display device |
| CN109874003B (en) * | 2019-02-28 | 2021-09-14 | 京东方科技集团股份有限公司 | VR display control method, VR display control device and display device |
| CN114009035A (en) * | 2019-06-28 | 2022-02-01 | Ati科技无限责任公司 | Real-time GPU rendering with guaranteed power management performance |
| WO2022068477A1 (en) * | 2020-09-30 | 2022-04-07 | 华为技术有限公司 | Event processing method, and device |
| CN113596569A (en) * | 2021-07-22 | 2021-11-02 | 歌尔光学科技有限公司 | Image processing method, apparatus and computer-readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111641844B (en) | Live broadcast interaction method and device, live broadcast system and electronic equipment | |
| US12214277B2 (en) | Method and device for generating video frames | |
| CN106527713B (en) | Three-dimensional data rendering system and method for VR | |
| CN106971368A (en) | A kind of lock in time for virtual reality crimps computational methods | |
| US10607412B2 (en) | Mixed reality presentation system | |
| CN108273265A (en) | The display methods and device of virtual objects | |
| US10755675B2 (en) | Image processing system, image processing method, and computer program | |
| EP3691280B1 (en) | Video transmission method, server, vr playback terminal and computer-readable storage medium | |
| JP5652037B2 (en) | Simulated video generation device, method, and program | |
| CN108921951A (en) | Virtual reality image display methods and its device, virtual reality device | |
| CN106454322A (en) | VR Image processing system and method thereof | |
| CN106502427A (en) | Virtual reality system and its scene rendering method | |
| CN107479699A (en) | Virtual reality interaction method, device and system | |
| KR102142876B1 (en) | Method and apparatus for buffer management in cloud based virtual reallity services | |
| US20210082187A1 (en) | Haptic simulation of motion in virtual reality | |
| US20100013837A1 (en) | Method And System For Controlling Character Animation | |
| CN106683034A (en) | Asynchronous time warping calculation method for virtual reality | |
| WO2023240999A1 (en) | Virtual reality scene determination method and apparatus, and system | |
| JP6723743B2 (en) | Information processing apparatus, information processing method, and program | |
| CN105844581B (en) | Image rendering method, device and device | |
| CN109089098A (en) | Virtual reality device and operation method thereof | |
| CN115641392A (en) | Data processing method, device, server and medium for virtual reality system | |
| WO2023108016A1 (en) | Augmented reality using a split architecture | |
| EP3422701B1 (en) | Method for reproducing virtual reality image and program using same | |
| Bondarenko et al. | Latency visualization emulation scheme for interactive cloud xr gaming |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170721 |