CN106339094A - Interactive remote expert cooperation maintenance system and method based on augmented reality technology - Google Patents
Interactive remote expert cooperation maintenance system and method based on augmented reality technology Download PDFInfo
- Publication number
- CN106339094A CN106339094A CN201610801442.5A CN201610801442A CN106339094A CN 106339094 A CN106339094 A CN 106339094A CN 201610801442 A CN201610801442 A CN 201610801442A CN 106339094 A CN106339094 A CN 106339094A
- Authority
- CN
- China
- Prior art keywords
- image
- video
- information
- sent
- target area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B23/00—Testing or monitoring of control systems or parts thereof
- G05B23/02—Electric testing or monitoring
- G05B23/0205—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults
- G05B23/0259—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults characterized by the response to fault detection
- G05B23/0275—Fault isolation and identification, e.g. classify fault; estimate cause or root of failure
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Automation & Control Theory (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses an interactive remote expert cooperation maintenance system and a method based on an augmented reality technology. An on-site client side carries out video acquisition or image shooting on equipment to be detected and maintained, and transmits to a remote server side; the remote server side receives a video or an image sent by the on-site client side, receives drawing processing on the video or the image carried out by an expert, and sends the drawn target area image to the on-site client side; after receiving the drawn target area image, the on-site client side completes recognition, location and tracking of a target area in the target area image and an on-site video image through an object tracking algorithm; a detector finds out the target area image; a tracker tracks a target; the detection accuracy of the detector is improved continually through P-N Learning; a marked pattern or a text comment, and current equipment state information are overlapped at the position of the target area in the on-site video image, and an augmented reality image video is generated; the augmented reality technology is utilized to synthesize the augmented reality video, and the fault diagnosis and the maintenance of on-site equipment are completed with the cooperation of the expert.
Description
Technical field
The present invention relates to a kind of interactive remote expert cooperation examination and repair system based on augmented reality and method.
Background technology
Augmented reality (augmented reality, abbreviation ar) is increased by the information that computer system provides to be used
Family perceives to real world, by virtual Information application to real world, and computer is generated dummy object, scene or be
System information is added in real scene, thus reaching to real reinforced effects.The target of this technology is on screen
Virtual world is implanted real world and carries out interaction.At present, the product using ar Technical comparing maturation is MS
Hololens glasses, but hololens only supports that the effect generating computer is superimposed on real world, is supplied to list
Individual user is checked or is interacted therewith, and does not but enable the interaction between multiple user.And existing utilization ar skill
The product of art is used for the consumer field such as life, social activity, seldom even without answering towards industrial system Measuring error aspect
With.
At this stage, large scale industry system equipment is distributed each corner at home and abroad already, with 777 visitors of Boeing Co.
As a example machine, end in March, 2012, produced more than 800 framves, be distributed in China, the U.S., Britain, Japan etc. all over the world.Aircraft
Electromotor, wing, fuselage, driver's cabin are designed manufacturing by professional, are related to electronics, physicss, kinetics, Information Center
The multi-subject knowledges such as skill, artificial intelligence, pattern recognition.Carry out it is necessary to be reached the spot by special expert when aircraft goes wrong
Measuring error, the time not only increasing product repairing also considerably increases the operation cost of service provider.
Content of the invention
The purpose of the present invention is exactly to solve the above problems, and provides a kind of interactive remote based on augmented reality
Expert's cooperation examination and repair system and method, it utilizes augmented reality, by expert feedback information, on-site customer collection video or figure
The status information synthesis augmented reality video of picture and current device, liberates client's both hands, helps client to be quickly found out problem.
To achieve these goals, the present invention adopts the following technical scheme that
Interactive remote expert cooperation examination and repair system based on augmented reality, comprising: on-site customer end and long-range clothes
Business end；
The equipment that described on-site customer end is used for treating Measuring error carries out video acquisition or image taking, by the letter of collection
Cease and be transferred to remote service end, and receive the target area image delineated at remote service end, by the target area image delineated
Synthesized with the video of collection in worksite, client shows the target location of maintenance to be detected in real time at the scene；
Described remote service end is used for receiving video or the image that on-site customer end is sent, and receives expert to video or image
Delineate process, the target area image delineated is sent to on-site customer end.
On-site customer end, comprising:
Video acquisition unit, the equipment for treating Measuring error carries out video acquisition or image taking；Obtain to be detected
The video flowing of equipment of maintenance or image；The equipment video flowing of the maintenance to be detected of collection or image are sent to the first communication unit
Unit；
Video flowing or image, for receiving video flowing or image, are sent to far by the first communication unit by wireless network
Journey service end, and receive the feedback information at remote service end；Described feedback information includes voice messaging and markup information delineated by band
Target area image；
Ar processing unit, the collection information sent for feedback information that remote service end is sent, video acquisition unit
Synthesize augmented reality video with the status information of current device；It is sent to the first display unit after synthesis；Video acquisition unit is sent out
The collection information come includes equipment video flowing or the image of maintenance to be detected；
First display unit, for showing to the result of ar processing unit.
Remote service end, comprising:
Second communication unit, for receiving video flowing or the image that the first communication unit sends；By receive video flowing or
Image is sent to video image synthesis unit；Receive video image synthesis unit feedback information, feedback information is sent to first
Communication unit；Described feedback information includes voice messaging and the target area image of markup information delineated by band；
Video image synthesis unit, is synthesized with image for delineating endorsement information, band is delineated endorsement information
Target area image carries out cutting；Then the target area image of cutting is sent to the second communication unit；
Second display unit, for showing the video flowing or image that the first communication unit sends over, is additionally operable to show simultaneously
Show the result of video image synthesis unit；
Second input block, delineates markup information and voice messaging for receiving, and will delineate markup information and voice letter
Breath sends into video image synthesis unit.
Interactive remote expert cooperation repair method based on augmented reality, comprises the steps:
Step (1): the equipment treating Measuring error carries out video acquisition or image taking, obtains maintenance of equipment to be detected and regards
Frequency stream or image；
Step (2): the maintenance of equipment video flowing to be detected gathering or image are sent to remote service end by on-site customer end；
Step (3): remote service end receives described video flowing or image and shows, receives expert to doubtful Measuring error portion
Position carries out delineating the information of annotation；Receive the voice messaging of repair prompt；
Step (4): remote service end will be delineated endorsement information and be synthesized with image, band be delineated the target of endorsement information
Area image carries out cutting；Then the target area image of cutting and voice messaging are sent to on-site customer end.
Step (5): on-site customer end receives the feedback information that remote service end sends over；Described feedback information includes language
The target area image of markup information delineated by message breath and band；
Step (6): the target area image of markup information is delineated at on-site customer end to band, is completed by object tracking algorithm
Target area image and the identification of target area, positioning and tracking, target in video image area at the scene in live video image
Superposition indicia patterns or word annotation at the position in domain, and it is superimposed the status information of current device, thus generating augmented reality shadow
As video；
Step (7): augmented reality image video is shown by on-site customer end.
On-site customer end, comprising:
Video acquisition unit, the equipment for treating Measuring error carries out video acquisition or image taking；Obtain to be detected
The video flowing of maintenance of equipment or image；The maintenance of equipment video flowing to be detected of collection or image are sent to the first communication unit；
Video flowing or image, for receiving video flowing or image, are sent to far by the first communication unit by wireless network
Journey service end, and receive the feedback information at remote service end；Described feedback information includes voice messaging and markup information delineated by band
Target area image；
Ar processing unit, the collection information sent for feedback information that remote service end is sent, video acquisition unit
Synthesize augmented reality video with the status information of current device；It is sent to the first display unit after synthesis；Video acquisition unit is sent out
The collection information come includes equipment video flowing or the image of maintenance to be detected；
First display unit, for showing to the result of ar processing unit.
Described ar processing unit includes:
Particle filter tracker, for following the tracks of target, obtains position and the size of target area；
Support vector machine classifier, is used for detecting all possible target；
Study subelement, for using p-n learning learning strategy complete support vector machine classifier online more
Newly, the nicety of grading making support vector machine classifier steps up；
Object module, for being mated with sample to be sorted, finds matching degree highest sample；
Merge subelement, for by the tracking result of the classification results of support vector machine classifier and particle filter tracker
Blend, obtain the position of target.
Preferably, described video acquisition unit is video camera.
Further, on-site customer end, also includes: the first input block.
Preferably, described first input block includes the first contact panel and the first voice module, described first voice mould
Block includes the first mike and the first speaker.
Further, on-site customer end, also includes: the first power supply unit.
Preferably, described on-site customer end is smart mobile phone, panel computer or wearable device.
Preferably, described wearable device is ar intelligent glasses.
Remote service end, comprising:
Second communication unit, for receiving video flowing or the image that the first communication unit sends；By receive video flowing or
Image is sent to video image synthesis unit；Receive video image synthesis unit feedback information, feedback information is sent to first
Communication unit；Described feedback information includes voice messaging and the target area image of markup information delineated by band；
Video image synthesis unit, is synthesized with image for delineating endorsement information, band is delineated endorsement information
Target area image carries out cutting；Then the target area image of cutting is sent to the second communication unit；
Second display unit, for showing the video flowing or image that the first communication unit sends over, is additionally operable to show simultaneously
Show the result of video image synthesis unit；
Second input block, delineates markup information and voice messaging for receiving, and will delineate markup information and voice letter
Breath sends into video image synthesis unit.Described markup information of delineating refers to that expert delineates and maintenance built to punctuating of target area
The word annotation of view.
Described video image synthesis unit includes:
Synthesis subelement, for synthesizing to the target area image delineated and endorsement information；
Cut out subelement, for being cut out to the image information synthesizing, obtain required target image information.
Further, remote service end, also includes: the second input block；
Preferably, described second input block includes the second contact panel and the second voice module, described second voice mould
Block includes second microphone and the second speaker.
Further, remote service end, also includes: the second power supply unit.
The repair method at on-site customer end, comprises the steps:
Step (a-1): the equipment treating Measuring error carries out video acquisition or image taking, obtains maintenance of equipment to be detected
Video flowing or image；
Step (a-2): the video flowing of the maintenance of equipment to be detected of collection or image are sent to remote service end；
Step (a-3): receive the feedback information being sended over by remote service end；Described feedback information includes voice messaging
Delineate the target area image of markup information with band；
Step (a-4): target area image and target area in live video image are completed by object tracking algorithm
Identification, positioning and tracking, superposition indicia patterns or word annotation at the position in target in video image region at the scene, and be superimposed
The status information of current device, thus generate augmented reality image video；
Step (a-5): augmented reality image video is shown.
Described step (a-4) object tracking algorithm steps are as follows:
Step (a-4-1): identify the 1st frame of tracking process in video image；It is loaded into the target that markup information delineated by band
Area image is as tracking target p；
Step (a-4-2): initialization support vector machine classifier and particle filter tracker；
Step (a-4-3): in the subsequent frame of video is processed, it is loaded into the i-th frame video image, the span of i is 2～n,
N represents the number of frame in video；Detect all possible target p={ p using support vector machine classifier1,p2,...,pn,
Follow the tracks of target using particle filter tracker, obtain position and the size of target area；
Step (a-4-4): by the tracking result phase of the classification results of support vector machine classifier and particle filter tracker
Merge；Obtain after fusion detecting the target location of positioning；Enter step (a-4-6)；
Step (a-4-5): complete the online updating of support vector machine classifier using p-n learning learning strategy, return
Return step (a-4-3)；
Step (a-4-6): judge whether video has reached n-th frame, if so, then terminate；Otherwise, i=i+1；Proceed to step
Suddenly (a-4-3).
Described step (a-4-4) merge have five kinds may:
The first, support vector machine classifier and particle filter tracker have image block as output, but supporting vector
Machine grader has multiple similar image blocks to be determined out, the plurality of refers to more than one；And particle filter tracker is only
Find a target location, arest neighbors classification is carried out with normalized correlation coefficient to testing result, select and previous frame target
Similarity highest image block is as last testing result；
Second, particle filter tracker does not have image block to export, and support vector machine classifier has image block to export, that
Testing result carries out clustering splitting with normalized correlation coefficient again, is tied as merging using first cluster segmentation result
Really；
The third, particle filter tracker does not have image block to export, and support vector machine classifier has image block to export, such as
Fruit Clustering Decision-Making result is the maximum result of decision corresponding to correlation coefficient, but the result of decision exceedes the threshold value model of setting
Enclose, then adopt the result of decision as fusion results, then initialization is re-started to particle filter tracker, and lose and originally recognized
For correct sample；
4th kind, particle filter tracker has image block to export, and support vector machine classifier does not have output image block, that
The output result adopting particle filter tracker is as fusion results；
5th kind, if support vector machine classifier and particle filter tracker all do not have image block output then it is assumed that mesh
Mark disappears.
The online updating process of described step (a-4-5) support vector machine classifier is:
In p-n learning, define p-expert and n-expert；
P-expert find video sequence in time domain on Structural Characteristics: time structure, and assume target be along
Trajectory movement；
But p-expert finds out and is classified device and is categorized as negative requires the classification should to be according to structural constraints
The sample of positive；
N-expert finds the Structural Characteristics in the spatial domain in video sequence: space structure, and assumes target one
It is only possible in individual frame of video occur in a position；
But n-expert finds out and is classified device and is categorized as positive requires the classification should to be according to structural constraints
The sample of negative；
P-n learning generates training sample by p-expert and n-expert, forms the training sample of new mark
This collection, and the training sample set of new mark is added in support vector machine classifier, update online object module and prop up
Hold vector machine classifier.
As time goes on described online object module constantly carries out the renewal of model, and training sample is continuously increased, mould
Type is constantly updated, classification accuracy more and more higher, and the target of acquisition is more accurate, and target location also determines that.
The image information of demarcation that described step (a-3) is fed back or video information include the doubtful Measuring error position demarcated
Image, video or information of voice prompt.
The status information of described step (a-4) current device includes: the buying hour of equipment or repair and maintenance record etc.；
The status information of described step (a-4) current device, is on-site customer end and field device controller or intelligent network
Gutron crosses the status information of the real-time device of wireless telecommunications acquisition.
The repair method at remote service end, comprises the steps:
Step (b-1): receive video flowing or image and show；
Step (b-2): receive expert and doubtful Measuring error position is carried out delineating the information of annotation；Receive repair prompt
Voice messaging；
Step (b-3): endorsement information will be delineated and synthesized with image, and band be delineated the target area image of endorsement information
Carry out cutting；Then the target area image of cutting and voice messaging are sent to on-site customer end.
Beneficial effects of the present invention:
1 utilizes augmented reality, by the shape of expert feedback information, on-site customer collection video or image and current device
State information synthesizes augmented reality video, liberates client's both hands, helps client to be quickly found out problem.
2 clients, while watching video, can carry out real-time interactive voice with remote port expert, scene is gone out temporarily
Existing problem feeds back to remote port expert in time, and obtains prompting and the guidance of remote port expert in time.
Brief description
Fig. 1 is the structural representation of inventive algorithm；
Fig. 2 is the structural representation of the present invention；
Fig. 3 is the structural representation at on-site customer end；
Fig. 4 is on-site customer end flow chart；
Fig. 5 is remote service end flow chart.
Specific embodiment
The invention will be further described with embodiment below in conjunction with the accompanying drawings.
As shown in figure 1, Fig. 1 is the structural representation of algorithm, first have to obtain the position of target, that is, in video tracking
The the first frame delineation target processing, continually entering with video flowing, detector constantly detects target, and tracker is constantly followed the tracks of
Target, makes the precision more and more higher of detector by study, finally carries out merging acquisition by the result of tracker and detector
The position of target.
As shown in Fig. 2 Remote cooperation examination and repair system includes on-site customer end and remote service end, described floor trader
Row data communication is entered by wireless network in family end and remote service end.
Described on-site customer end includes video camera, ar processing unit, wireless communication module, contact panel, display, language
Sound module (including client mike and client speaker), supply module, as shown in figure 3, described on-site customer end can
To be smart mobile phone, panel computer or wearable device (as ar intelligent glasses), described client communication module communication link
It is connected to on-site customer end, and connected to service end by telecommunication network.
Described on-site customer end ar processing unit, the calculating for described augmented reality and process.
Described on-site customer end communication unit, for sending described video flowing or image to remote service end, receives institute
State the expert feedback information of remote service end return.
Described on-site customer end display unit, for show remote service end return expert demarcate image information or
Video information.
Described remote service end includes video image synthesis unit, service end wireless communication module, display, voice mould
Block (include service end mike and service end speaker), described service end communication module communication link then service end main frame,
And connected to client by telecommunication network.
Described remote service end video image synthesis unit, hooks at video image doubtful Measuring error position for expert
Draw mark image synthesis.
Described remote service end communication unit, for receiving video flowing or the image of client transmission, and to described existing
Field client sends expert feedback information.
Described remote service end display unit, for showing described video or image information.
Implementation steps are as follows:
First, maintainer carries out video acquisition or image using the video camera at on-site customer end to field scene at the scene
Shoot, on-site customer end sends the video of acquisition or image to remote service end by wireless telecommunications；
Secondly, the video that expert comes in the transmission of remote service end real time inspection on-site customer end, and using mouse in video
Or image doubtful maintenance position carries out delineating mark it is possible to carry out interactive voice by voice and field service personnel；
Finally, the feedback information that expert marks is returned to on-site customer end by remote service end, assists maintainer to complete
Service work to equipment.
Specific implementation step is as follows:
Step (1): as shown in Fig. 4 first frame, maintainer treats setting of Measuring error using on-site customer end at the scene
For carrying out video acquisition or image taking, obtain the video flowing needed for the expert of remote service end or image；Described on-site customer end
Move wearable device including smart mobile phone, panel computer and ar intelligent glasses etc.；
Step (2): as shown in Fig. 4 second frame, on-site customer end will be remote by radioing to described video flowing or image
Journey service end；
Step (3): as shown in Fig. 5 first and second frame, remote service end receives described video flowing or image and shows.Expert
As needed observation can be zoomed in and out to image or video at remote service end, and feelings are understood with Field Force by voice
Condition, expert provides maintenance direction suggestion according to priori.Illustrate without loss of generality, the video that expert's viewing client-side is sent or
Image information, carries out delineating mark it is also possible to carry out real-time language by mike and on-site customer at doubtful Measuring error position
Sound interacts, and real-time instruction Measuring error works；
Step (4): as shown in Fig. 5 the 3rd frame, the video at the doubtful Measuring error position that expert is demarcated by remote service end
Or image and described information of voice prompt are sent to on-site customer end；
Step (5): as shown in Fig. 4 the 3rd frame, the expert feedback information that the reception of on-site customer end is returned by remote service end,
Described expert feedback information includes the image at doubtful Measuring error position or video and the specialist speech information of expert's demarcation；
Step (6): as shown in Fig. 4 the 4th frame, described augmented reality processing unit will receive expert feedback information and visitor
The image video at family end scene, completes expert by object tracking algorithm and demarcates the identification of object in object and live video, determines
Position and tracking, the indicia patterns that at the position of object in image, superposition Remote is demarcated or word annotation, and it is superimposed current
The status information of equipment, the such as buying hour of equipment, repair and maintenance record etc., generate augmented reality image video；Described currently sets
Standby status information, is the real-time device that on-site customer end is obtained by wireless telecommunications with field device controller or intelligent network management
Status information.
Step (7): as shown in Fig. 4 the 5th frame, finally augmented reality image audio video synchronization is shown in Field Force and is worn
In the on-site customer end equipment worn.
By above-mentioned implementation steps, to reach the mesh of the interactive remote expert cooperation maintenance based on augmented reality
's.
Described step (6) object tracking algorithm steps are as follows:
Step (6-1): identify the 1st frame of tracking process in video image；It is loaded into the target area that markup information delineated by band
Area image is as tracking target p；
Step (6-2): initialization support vector machine classifier and particle filter tracker；
Step (6-3): in the subsequent frame of video is processed, be loaded into the i-th frame video image, the span of i is 2～n, n
Represent the number of frame in video；Detect all possible target p={ p using support vector machine classifier1,p2,...,pn,
Follow the tracks of target using particle filter tracker, obtain position and the size of target area；
Step (6-4): the classification results of support vector machine classifier are mutually melted with the tracking result of particle filter tracker
Close；Obtain after fusion detecting the target location of positioning；Enter step (6-6)；
Step (6-5): complete the online updating of support vector machine classifier using p-n learning learning strategy, return
Step (a-4-3)；
Step (6-6): judge whether video has reached n-th frame, if so, then terminate；Otherwise, i=i+1；Proceed to step
(6-3).
Described step (6-4) merge have five kinds may:
The first, support vector machine classifier and particle filter tracker have image block as output, but supporting vector
Machine grader has multiple similar image blocks to be determined out, the plurality of refers to more than one；And particle filter tracker is only
Find a target location, arest neighbors classification is carried out with normalized correlation coefficient to testing result, select and previous frame target
Similarity highest image block is as last testing result；
Second, particle filter tracker does not have image block to export, and support vector machine classifier has image block to export, that
Testing result carries out clustering splitting with normalized correlation coefficient again, is tied as merging using first cluster segmentation result
Really；
The third, particle filter tracker does not have image block to export, and support vector machine classifier has image block to export, such as
Fruit Clustering Decision-Making result is the maximum result of decision corresponding to correlation coefficient, but the result of decision exceedes the threshold value model of setting
Enclose, then adopt the result of decision as fusion results, then initialization is re-started to particle filter tracker, and lose and originally recognized
For correct sample；
4th kind, particle filter tracker has image block to export, and support vector machine classifier does not have output image block, that
The output result adopting particle filter tracker is as fusion results；
5th kind, if support vector machine classifier and particle filter tracker all do not have image block output then it is assumed that mesh
Mark disappears.
The online updating process of described step (6-5) support vector machine classifier is:
In p-n learning, define p-expert and n-expert.
P-expert find video sequence in time domain on Structural Characteristics be time structure, and assume target be along
Trajectory movement, i.e. mobile very little between adjacent video frames, and there is certain dependency.
But p-expert finds out those and is classified device and is categorized as negative requires its classification should according to structural constraints
It should be the sample of positive.
The Structural Characteristics that n-expert finds in the spatial domain in video sequence are space structure, and assume target one
It is only possible in individual frame of video occur in a position.
But n-expert finds out those and is classified device and is categorized as positive requires its classification should according to structural constraints
It should be the sample of negative.
P-n learning generates effective training sample by p-expert and n-expert, forms new mark
Training sample set, and this sample set is added in support vector machine classifier, update online object module and support vector machine
Grader.
Although the above-mentioned accompanying drawing that combines is described to the specific embodiment of the present invention, not model is protected to the present invention
The restriction enclosed, one of ordinary skill in the art should be understood that on the basis of technical scheme, and those skilled in the art are not
Need to pay the various modifications that creative work can make or deformation still within protection scope of the present invention.
Claims (10)
1. on-site customer end, is characterized in that, comprising:
Video acquisition unit, the equipment for treating Measuring error carries out video acquisition or image taking；Obtain maintenance to be detected
The video flowing of equipment or image；The maintenance of equipment video flowing to be detected of collection or image are sent to the first communication unit；
Video flowing or image, for receiving video flowing or image, are sent to long-range clothes by wireless network by the first communication unit
Business end, and receive the feedback information at remote service end；Described feedback information includes voice messaging and the mesh of markup information delineated by band
Mark area image；
Ar processing unit, the collection information sent for feedback information that remote service end is sent, video acquisition unit and working as
The status information synthesis augmented reality video of front equipment；It is sent to the first display unit after synthesis；Video acquisition unit is sent
Collection information includes equipment video flowing or the image of maintenance to be detected；
First display unit, for showing to the result of ar processing unit.
2. on-site customer end as claimed in claim 1, is characterized in that, described ar processing unit includes:
Particle filter tracker, for following the tracks of target, obtains position and the size of target area；
Support vector machine classifier, is used for detecting all possible target；
Study subelement, for being completed the online updating of support vector machine classifier using p-n learning learning strategy, is made
The nicety of grading of support vector machine classifier steps up；
Object module, for being mated with sample to be sorted, finds matching degree highest sample；
Merge subelement, for mutually melting the classification results of support vector machine classifier with the tracking result of particle filter tracker
Close, obtain the position of target.
3. remote service end, is characterized in that, comprising:
Second communication unit, for receiving video flowing or the image that the first communication unit sends；By the video flowing receiving or image
It is sent to video image synthesis unit；Receive video image synthesis unit feedback information, feedback information is sent to the first communication
Unit；Described feedback information includes voice messaging and the target area image of markup information delineated by band；
Video image synthesis unit, is synthesized with image for delineating endorsement information, band is delineated the target of endorsement information
Area image carries out cutting；Then the target area image of cutting is sent to the second communication unit；
Second display unit, for showing the video flowing or image that the first communication unit sends over, is additionally operable to display simultaneously and regards
The result of frequency image composing unit；
Second input block, delineates markup information and voice messaging for receiving, and will delineate markup information and voice messaging send
Enter video image synthesis unit.
4. the repair method at on-site customer end, is characterized in that, comprises the steps:
Step (a-1): the equipment treating Measuring error carries out video acquisition or image taking, obtains regarding of maintenance of equipment to be detected
Frequency stream or image；
Step (a-2): the video flowing of the maintenance of equipment to be detected of collection or image are sent to remote service end；
Step (a-3): receive the feedback information being sended over by remote service end；Described feedback information includes voice messaging and band
Delineate the target area image of markup information；
Step (a-4): by object tracking algorithm complete the identification of target area in target area image and live video image,
Positioning and tracking, superposition indicia patterns or word annotation at the position in target in video image region at the scene, and it is superimposed current
The status information of equipment, thus generate augmented reality image video；
Step (a-5): augmented reality image video is shown.
5. the repair method at on-site customer end as claimed in claim 4, is characterized in that, described step (a-4) object tracking is calculated
Method step is as follows:
Step (a-4-1): identify the 1st frame of tracking process in video image；It is loaded into the target area that markup information delineated by band
Image is as tracking target p；
Step (a-4-2): initialization support vector machine classifier and particle filter tracker；
Step (a-4-3): in the subsequent frame of video is processed, be loaded into the i-th frame video image, the span of i is 2～n, n table
Show the number of frame in video；Detect all possible target p={ p using support vector machine classifier1,p2,...,pn, profit
Follow the tracks of target with particle filter tracker, obtain position and the size of target area；
Step (a-4-4): the tracking result of the classification results of support vector machine classifier and particle filter tracker is blended；
Obtain after fusion detecting the target location of positioning；Enter step (a-4-6)；
Step (a-4-5): complete the online updating of support vector machine classifier using p-n learning learning strategy, return step
Suddenly (a-4-3)；
Step (a-4-6): judge whether video has reached n-th frame, if so, then terminate；Otherwise, i=i+1；Proceed to step (a-
4-3).
6. the repair method at remote service end, is characterized in that, comprises the steps:
Step (b-1): receive video flowing or image and show；
Step (b-2): receive expert and doubtful Measuring error position is carried out delineating the information of annotation；Receive the voice of repair prompt
Information；
Step (b-3): endorsement information will be delineated and synthesized with image, the target area image that band is delineated endorsement information is carried out
Cutting；Then the target area image of cutting and voice messaging are sent to on-site customer end.
7. based on augmented reality interactive remote expert cooperate examination and repair system, it is characterized in that, comprising: on-site customer end and
Remote service end；
The equipment that described on-site customer end is used for treating Measuring error carries out video acquisition or image taking, and the information of collection is passed
Be defeated by remote service end, and receive the target area image delineated at remote service end, by the target area image delineated with existing
The video of field collection is synthesized, and client shows the target location of maintenance to be detected in real time at the scene；
Described remote service end is used for receiving video or the image that on-site customer end is sent, and receives the hook to video or image for the expert
Picture is processed, and the target area image delineated is sent to on-site customer end.
8. examination and repair system as claimed in claim 7, is characterized in that, on-site customer end, comprising:
Video acquisition unit, the equipment for treating Measuring error carries out video acquisition or image taking；Obtain maintenance to be detected
The video flowing of equipment or image；The equipment video flowing of the maintenance to be detected of collection or image are sent to the first communication unit；
Video flowing or image, for receiving video flowing or image, are sent to long-range clothes by wireless network by the first communication unit
Business end, and receive the feedback information at remote service end；Described feedback information includes voice messaging and the mesh of markup information delineated by band
Mark area image；
Ar processing unit, the collection information sent for feedback information that remote service end is sent, video acquisition unit and working as
The status information synthesis augmented reality video of front equipment；It is sent to the first display unit after synthesis；Video acquisition unit is sent
Collection information includes equipment video flowing or the image of maintenance to be detected；
First display unit, for showing to the result of ar processing unit.
9. examination and repair system as claimed in claim 7, is characterized in that, remote service end, comprising:
Second communication unit, for receiving video flowing or the image that the first communication unit sends；By the video flowing receiving or image
It is sent to video image synthesis unit；Receive video image synthesis unit feedback information, feedback information is sent to the first communication
Unit；Described feedback information includes voice messaging and the target area image of markup information delineated by band；
Video image synthesis unit, is synthesized with image for delineating endorsement information, band is delineated the target of endorsement information
Area image carries out cutting；Then the target area image of cutting is sent to the second communication unit；
Second display unit, for showing the video flowing or image that the first communication unit sends over, is additionally operable to display simultaneously and regards
The result of frequency image composing unit；
Second input block, delineates markup information and voice messaging for receiving, and will delineate markup information and voice messaging send
Enter video image synthesis unit.
10. the interactive remote expert cooperation repair method based on augmented reality, is characterized in that, comprise the steps:
Step (1): the equipment treating Measuring error carries out video acquisition or image taking, obtains maintenance of equipment video flowing to be detected
Or image；
Step (2): the maintenance of equipment video flowing to be detected gathering or image are sent to remote service end by on-site customer end；
Step (3): remote service end receives described video flowing or image and shows, receives expert and doubtful Measuring error position is entered
Row delineates the information of annotation；Receive the voice messaging of repair prompt；
Step (4): remote service end will be delineated endorsement information and be synthesized with image, band be delineated the target area of endorsement information
Image carries out cutting；Then the target area image of cutting and voice messaging are sent to on-site customer end；
Step (5): on-site customer end receives the feedback information that remote service end sends over；Described feedback information includes voice letter
The target area image of markup information delineated by breath and band；
Step (6): the target area image of markup information is delineated at on-site customer end to band, completes target by object tracking algorithm
The identification of target area, positioning and follow the tracks of in area image and live video image, target in video image region at the scene
Superposition indicia patterns or word annotation at position, and it is superimposed the status information of current device, thus generate augmented reality image regarding
Frequently；
Step (7): augmented reality image video is shown by on-site customer end.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610801442.5A CN106339094B (en) | 2016-09-05 | 2016-09-05 | Interactive remote expert cooperation examination and repair system and method based on augmented reality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610801442.5A CN106339094B (en) | 2016-09-05 | 2016-09-05 | Interactive remote expert cooperation examination and repair system and method based on augmented reality |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106339094A true CN106339094A (en) | 2017-01-18 |
| CN106339094B CN106339094B (en) | 2019-02-26 |
Family
ID=57823708
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610801442.5A Active CN106339094B (en) | 2016-09-05 | 2016-09-05 | Interactive remote expert cooperation examination and repair system and method based on augmented reality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106339094B (en) |
Cited By (61)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107241579A (en) * | 2017-07-14 | 2017-10-10 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of Telemedicine Consultation |
| CN107240317A (en) * | 2017-07-14 | 2017-10-10 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of long-distance education |
| CN107277450A (en) * | 2017-07-14 | 2017-10-20 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of the live disaster relief of remote command |
| CN107277429A (en) * | 2017-07-14 | 2017-10-20 | 福建铁工机智能机器人有限公司 | A kind of method that teleconference is carried out using AR |
| CN107274431A (en) * | 2017-03-07 | 2017-10-20 | 阿里巴巴集团控股有限公司 | video content enhancement method and device |
| CN107277451A (en) * | 2017-07-14 | 2017-10-20 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of remote guide scene investigation failure |
| CN107329268A (en) * | 2017-08-03 | 2017-11-07 | 福建铁工机智能机器人有限公司 | A kind of utilization AR glasses realize the shared method in sight spot |
| CN107333063A (en) * | 2017-07-14 | 2017-11-07 | 福建铁工机智能机器人有限公司 | A kind of method that utilization AR shoots film |
| CN107426508A (en) * | 2017-07-14 | 2017-12-01 | 福建铁工机智能机器人有限公司 | A kind of method that long-range security inspection is realized using AR |
| CN107454365A (en) * | 2017-07-14 | 2017-12-08 | 福建铁工机智能机器人有限公司 | A kind of method that special reconnaissance is carried out using AR |
| CN107547554A (en) * | 2017-09-08 | 2018-01-05 | 北京枭龙科技有限公司 | A kind of smart machine remote assisting system based on augmented reality |
| CN107578110A (en) * | 2017-09-19 | 2018-01-12 | 北京枭龙科技有限公司 | A kind of repair message method for visualizing based on augmented reality |
| CN107872655A (en) * | 2017-11-21 | 2018-04-03 | 北京恒华伟业科技股份有限公司 | A kind of method and system for determining hidden danger point |
| CN107948591A (en) * | 2017-11-21 | 2018-04-20 | 北京恒华伟业科技股份有限公司 | A kind of bridge information processing method and system |
| CN107957778A (en) * | 2017-11-21 | 2018-04-24 | 北京恒华伟业科技股份有限公司 | A kind of method and system of hidden danger point mark |
| CN107977716A (en) * | 2017-11-21 | 2018-05-01 | 北京恒华伟业科技股份有限公司 | The method and system that a kind of hidden danger point is shown |
| CN108170273A (en) * | 2017-12-28 | 2018-06-15 | 南京华讯方舟通信设备有限公司 | A kind of expert's remote guide system and implementation method based on hololens glasses |
| CN108200331A (en) * | 2017-12-16 | 2018-06-22 | 苏州晨本智能科技有限公司 | It is a kind of based on the interactive robot of long-range explanation for following technology and VR technologies automatically |
| WO2018152685A1 (en) * | 2017-02-22 | 2018-08-30 | Tencent Technology (Shenzhen) Company Limited | Image processing in a vr system |
| CN108628286A (en) * | 2017-03-24 | 2018-10-09 | 郑芳田 | Board maintenance system and method |
| CN108632379A (en) * | 2018-05-11 | 2018-10-09 | 厦门柏讯信息科技有限公司 | A kind of automobile remote diagnostic and maintenance system based on panorama VR live streamings |
| CN108769517A (en) * | 2018-05-29 | 2018-11-06 | 亮风台（上海）信息科技有限公司 | A kind of method and apparatus carrying out remote assistant based on augmented reality |
| CN108776444A (en) * | 2018-05-25 | 2018-11-09 | 江西博异自动化科技有限公司 | Augmented reality man-machine interactive system suitable for CPS automatic control systems |
| CN108965791A (en) * | 2018-04-04 | 2018-12-07 | 广州高新兴机器人有限公司 | One kind passing through robot AR camera and internet of things equipment exchange method and system |
| CN108986556A (en) * | 2018-08-22 | 2018-12-11 | 高海龙 | A kind of aircraft remote technology clearance method, apparatus and smart machine |
| CN109034404A (en) * | 2018-06-27 | 2018-12-18 | 上海擎测机电工程技术有限公司 | A kind of thermal power plant's multi-person synergy examination and repair system based on AR glasses, method and a kind of AR glasses |
| CN109218794A (en) * | 2017-06-30 | 2019-01-15 | 全球能源互联网研究院 | Remote job guidance method and system |
| CN109300112A (en) * | 2018-08-27 | 2019-02-01 | 百度在线网络技术（北京）有限公司 | Instrument maintaining method and device based on AR |
| CN109635957A (en) * | 2018-11-13 | 2019-04-16 | 广州裕申电子科技有限公司 | A kind of equipment maintenance aid method and system based on AR technology |
| CN109714575A (en) * | 2018-12-29 | 2019-05-03 | 广州德毅维才软件技术有限公司 | A kind of repair method based on internet remote command |
| CN109725247A (en) * | 2018-12-18 | 2019-05-07 | 蒂姆维澳(上海)网络技术有限公司 | A kind of circuit board remote maintenance guarantee method and system |
| CN110061755A (en) * | 2019-04-30 | 2019-07-26 | 徐州重型机械有限公司 | Operation householder method and system, wearable device and engineering truck |
| CN110097573A (en) * | 2018-01-31 | 2019-08-06 | 深圳市掌网科技股份有限公司 | A kind of combined tracking method and system based on augmented reality |
| CN110138831A (en) * | 2019-03-29 | 2019-08-16 | 亮风台（上海）信息科技有限公司 | Method and equipment for remote assistance |
| CN110174423A (en) * | 2019-04-26 | 2019-08-27 | 云南电网有限责任公司电力科学研究院 | It is a kind of for X-ray detection it is long-range monitoring instruct system |
| CN110494887A (en) * | 2018-02-23 | 2019-11-22 | 弗隆蒂斯株式会社 | Server, method and the wearable device of military hardware maintenance are supported based on binary search tree in the general object identification based on augmented reality, virtual reality or mixed reality |
| CN110546677A (en) * | 2018-02-23 | 2019-12-06 | 弗隆蒂斯株式会社 | Server, method and wearable device for supporting military equipment maintenance in augmented reality technology using correlation rule mining |
| CN111031284A (en) * | 2019-12-12 | 2020-04-17 | 国网浙江余姚市供电有限公司 | Infrastructure construction site management and control method, system and terminal |
| CN111050112A (en) * | 2020-01-10 | 2020-04-21 | 北京首翼弘泰科技有限公司 | Method for remote operation command or guidance by displaying mark on screen |
| CN111260084A (en) * | 2020-01-09 | 2020-06-09 | 长安大学 | Remote system and method for collaborative assembly maintenance based on augmented reality |
| CN111294568A (en) * | 2020-03-13 | 2020-06-16 | 北京维卓致远医疗科技发展有限责任公司 | System and method for remotely controlling video fusion |
| WO2020135851A1 (en) * | 2018-12-29 | 2020-07-02 | 中兴通讯股份有限公司 | Method for realizing remote assistance and related device |
| CN111526118A (en) * | 2019-10-29 | 2020-08-11 | 南京翱翔信息物理融合创新研究院有限公司 | Remote operation guiding system and method based on mixed reality |
| CN111614922A (en) * | 2019-02-22 | 2020-09-01 | 中国移动通信有限公司研究院 | An information interaction method, network terminal and terminal |
| CN111784005A (en) * | 2020-06-17 | 2020-10-16 | 北京中铁电梯工程有限公司 | Elevator maintenance method, maintenance device, maintenance system and computer-readable storage medium based on Internet of things |
| CN111950506A (en) * | 2020-08-25 | 2020-11-17 | 中山大学 | A kind of auxiliary method and system for MOCVD equipment maintenance based on AR technology |
| CN112130666A (en) * | 2020-09-16 | 2020-12-25 | 博彦科技（上海）有限公司 | Data processing method and system |
| CN112399125A (en) * | 2019-08-19 | 2021-02-23 | 中国移动通信集团广东有限公司 | A remote assistance method, device and system |
| CN112492281A (en) * | 2020-12-10 | 2021-03-12 | 郑州捷安高科股份有限公司 | Remote auxiliary maintenance method, device, equipment and storage medium |
| CN112509148A (en) * | 2020-12-04 | 2021-03-16 | 全球能源互联网研究院有限公司 | Interaction method and device based on multi-feature recognition and computer equipment |
| CN112565844A (en) * | 2020-12-04 | 2021-03-26 | 维沃移动通信有限公司 | Video communication method and device and electronic equipment |
| CN113360057A (en) * | 2020-03-03 | 2021-09-07 | 中国移动通信集团贵州有限公司 | Remote interaction method and system based on augmented reality and electronic equipment |
| CN113542332A (en) * | 2020-04-22 | 2021-10-22 | 中移智行网络科技有限公司 | Customer service video interaction method and equipment based on positioning labels |
| CN113674426A (en) * | 2021-07-19 | 2021-11-19 | 中国电建集团西北勘测设计研究院有限公司 | Equipment maintenance method and device using AR technology |
| CN113780706A (en) * | 2020-10-27 | 2021-12-10 | 北京京东尚科信息技术有限公司 | A field operation and maintenance operation method and device based on visual enhancement |
| CN114063954A (en) * | 2021-11-11 | 2022-02-18 | 广西电网有限责任公司崇左供电局 | Interactive system for remote cooperation of communication operation and inspection operation |
| WO2022061858A1 (en) * | 2020-09-28 | 2022-03-31 | Orange | Method for assisting a first user provided with a terminal at a scene |
| CN115776577A (en) * | 2022-10-27 | 2023-03-10 | 中电信数智科技有限公司 | AR dynamic labeling method based on free circle drawing in remote collaboration scene |
| EP4358041A1 (en) * | 2022-10-19 | 2024-04-24 | Koninklijke Philips N.V. | Video-processing for equipment maintenance |
| CN119152405A (en) * | 2024-11-13 | 2024-12-17 | 广东电网有限责任公司中山供电局 | Bi-directional video fusion interactive overhaul AR (augmented reality) glasses system and overhaul method |
| CN119785165A (en) * | 2025-03-11 | 2025-04-08 | 深圳市贝铂智能科技有限公司 | Multi-mode interaction and image processing method for AI glasses, electronic device and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015060393A1 (en) * | 2013-10-25 | 2015-04-30 | 独立行政法人産業技術総合研究所 | Remote action guidance system and processing method therefor |
| CN104834379A (en) * | 2015-05-05 | 2015-08-12 | 江苏卡罗卡国际动漫城有限公司 | Repair guide system based on AR (augmented reality) technology |
| CN105814500A (en) * | 2013-06-27 | 2016-07-27 | Abb技术有限公司 | Method and data presentation device for assisting remote user to provide instructions |
-
2016
- 2016-09-05 CN CN201610801442.5A patent/CN106339094B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105814500A (en) * | 2013-06-27 | 2016-07-27 | Abb技术有限公司 | Method and data presentation device for assisting remote user to provide instructions |
| WO2015060393A1 (en) * | 2013-10-25 | 2015-04-30 | 独立行政法人産業技術総合研究所 | Remote action guidance system and processing method therefor |
| CN104834379A (en) * | 2015-05-05 | 2015-08-12 | 江苏卡罗卡国际动漫城有限公司 | Repair guide system based on AR (augmented reality) technology |
Cited By (75)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110036356A (en) * | 2017-02-22 | 2019-07-19 | 腾讯科技（深圳）有限公司 | Image procossing in VR system |
| US11003707B2 (en) | 2017-02-22 | 2021-05-11 | Tencent Technology (Shenzhen) Company Limited | Image processing in a virtual reality (VR) system |
| CN110036356B (en) * | 2017-02-22 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Image processing in VR systems |
| WO2018152685A1 (en) * | 2017-02-22 | 2018-08-30 | Tencent Technology (Shenzhen) Company Limited | Image processing in a vr system |
| CN107274431A (en) * | 2017-03-07 | 2017-10-20 | 阿里巴巴集团控股有限公司 | video content enhancement method and device |
| CN108628286B (en) * | 2017-03-24 | 2020-06-16 | 郑芳田 | Machine maintenance system and method |
| CN108628286A (en) * | 2017-03-24 | 2018-10-09 | 郑芳田 | Board maintenance system and method |
| CN109218794A (en) * | 2017-06-30 | 2019-01-15 | 全球能源互联网研究院 | Remote job guidance method and system |
| CN107333063A (en) * | 2017-07-14 | 2017-11-07 | 福建铁工机智能机器人有限公司 | A kind of method that utilization AR shoots film |
| CN107426508A (en) * | 2017-07-14 | 2017-12-01 | 福建铁工机智能机器人有限公司 | A kind of method that long-range security inspection is realized using AR |
| CN107454365A (en) * | 2017-07-14 | 2017-12-08 | 福建铁工机智能机器人有限公司 | A kind of method that special reconnaissance is carried out using AR |
| CN107277451A (en) * | 2017-07-14 | 2017-10-20 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of remote guide scene investigation failure |
| CN107241579A (en) * | 2017-07-14 | 2017-10-10 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of Telemedicine Consultation |
| CN107277429A (en) * | 2017-07-14 | 2017-10-20 | 福建铁工机智能机器人有限公司 | A kind of method that teleconference is carried out using AR |
| CN107277450A (en) * | 2017-07-14 | 2017-10-20 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of the live disaster relief of remote command |
| CN107240317A (en) * | 2017-07-14 | 2017-10-10 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of long-distance education |
| CN107329268A (en) * | 2017-08-03 | 2017-11-07 | 福建铁工机智能机器人有限公司 | A kind of utilization AR glasses realize the shared method in sight spot |
| CN107547554A (en) * | 2017-09-08 | 2018-01-05 | 北京枭龙科技有限公司 | A kind of smart machine remote assisting system based on augmented reality |
| CN107578110A (en) * | 2017-09-19 | 2018-01-12 | 北京枭龙科技有限公司 | A kind of repair message method for visualizing based on augmented reality |
| CN107977716A (en) * | 2017-11-21 | 2018-05-01 | 北京恒华伟业科技股份有限公司 | The method and system that a kind of hidden danger point is shown |
| CN107957778A (en) * | 2017-11-21 | 2018-04-24 | 北京恒华伟业科技股份有限公司 | A kind of method and system of hidden danger point mark |
| CN107948591A (en) * | 2017-11-21 | 2018-04-20 | 北京恒华伟业科技股份有限公司 | A kind of bridge information processing method and system |
| CN107872655A (en) * | 2017-11-21 | 2018-04-03 | 北京恒华伟业科技股份有限公司 | A kind of method and system for determining hidden danger point |
| CN108200331A (en) * | 2017-12-16 | 2018-06-22 | 苏州晨本智能科技有限公司 | It is a kind of based on the interactive robot of long-range explanation for following technology and VR technologies automatically |
| CN108170273A (en) * | 2017-12-28 | 2018-06-15 | 南京华讯方舟通信设备有限公司 | A kind of expert's remote guide system and implementation method based on hololens glasses |
| CN110097573A (en) * | 2018-01-31 | 2019-08-06 | 深圳市掌网科技股份有限公司 | A kind of combined tracking method and system based on augmented reality |
| CN110546677A (en) * | 2018-02-23 | 2019-12-06 | 弗隆蒂斯株式会社 | Server, method and wearable device for supporting military equipment maintenance in augmented reality technology using correlation rule mining |
| CN110494887A (en) * | 2018-02-23 | 2019-11-22 | 弗隆蒂斯株式会社 | Server, method and the wearable device of military hardware maintenance are supported based on binary search tree in the general object identification based on augmented reality, virtual reality or mixed reality |
| CN108965791B (en) * | 2018-04-04 | 2021-08-24 | 广州高新兴机器人有限公司 | Method and system for interacting with Internet of things equipment through robot AR camera |
| CN108965791A (en) * | 2018-04-04 | 2018-12-07 | 广州高新兴机器人有限公司 | One kind passing through robot AR camera and internet of things equipment exchange method and system |
| CN108632379A (en) * | 2018-05-11 | 2018-10-09 | 厦门柏讯信息科技有限公司 | A kind of automobile remote diagnostic and maintenance system based on panorama VR live streamings |
| CN108776444B (en) * | 2018-05-25 | 2021-09-03 | 江西博异自动化科技有限公司 | Augmented reality man-machine interaction system suitable for CPS automation control system |
| CN108776444A (en) * | 2018-05-25 | 2018-11-09 | 江西博异自动化科技有限公司 | Augmented reality man-machine interactive system suitable for CPS automatic control systems |
| CN108769517A (en) * | 2018-05-29 | 2018-11-06 | 亮风台（上海）信息科技有限公司 | A kind of method and apparatus carrying out remote assistant based on augmented reality |
| CN109034404A (en) * | 2018-06-27 | 2018-12-18 | 上海擎测机电工程技术有限公司 | A kind of thermal power plant's multi-person synergy examination and repair system based on AR glasses, method and a kind of AR glasses |
| CN108986556A (en) * | 2018-08-22 | 2018-12-11 | 高海龙 | A kind of aircraft remote technology clearance method, apparatus and smart machine |
| CN109300112A (en) * | 2018-08-27 | 2019-02-01 | 百度在线网络技术（北京）有限公司 | Instrument maintaining method and device based on AR |
| CN109635957A (en) * | 2018-11-13 | 2019-04-16 | 广州裕申电子科技有限公司 | A kind of equipment maintenance aid method and system based on AR technology |
| CN109725247B (en) * | 2018-12-18 | 2020-11-20 | 蒂姆维澳(上海)网络技术有限公司 | A kind of circuit board remote maintenance guarantee method and system |
| CN109725247A (en) * | 2018-12-18 | 2019-05-07 | 蒂姆维澳(上海)网络技术有限公司 | A kind of circuit board remote maintenance guarantee method and system |
| CN109714575A (en) * | 2018-12-29 | 2019-05-03 | 广州德毅维才软件技术有限公司 | A kind of repair method based on internet remote command |
| WO2020135851A1 (en) * | 2018-12-29 | 2020-07-02 | 中兴通讯股份有限公司 | Method for realizing remote assistance and related device |
| CN111381918A (en) * | 2018-12-29 | 2020-07-07 | 中兴通讯股份有限公司 | A method and related equipment for realizing remote assistance |
| CN111614922A (en) * | 2019-02-22 | 2020-09-01 | 中国移动通信有限公司研究院 | An information interaction method, network terminal and terminal |
| CN110138831A (en) * | 2019-03-29 | 2019-08-16 | 亮风台（上海）信息科技有限公司 | Method and equipment for remote assistance |
| CN110174423A (en) * | 2019-04-26 | 2019-08-27 | 云南电网有限责任公司电力科学研究院 | It is a kind of for X-ray detection it is long-range monitoring instruct system |
| CN110061755B (en) * | 2019-04-30 | 2021-04-16 | 徐州重型机械有限公司 | Operation assisting method and system, wearable device and engineering vehicle |
| CN110061755A (en) * | 2019-04-30 | 2019-07-26 | 徐州重型机械有限公司 | Operation householder method and system, wearable device and engineering truck |
| CN112399125B (en) * | 2019-08-19 | 2022-06-10 | 中国移动通信集团广东有限公司 | A remote assistance method, device and system |
| CN112399125A (en) * | 2019-08-19 | 2021-02-23 | 中国移动通信集团广东有限公司 | A remote assistance method, device and system |
| CN111526118A (en) * | 2019-10-29 | 2020-08-11 | 南京翱翔信息物理融合创新研究院有限公司 | Remote operation guiding system and method based on mixed reality |
| CN111031284A (en) * | 2019-12-12 | 2020-04-17 | 国网浙江余姚市供电有限公司 | Infrastructure construction site management and control method, system and terminal |
| CN111260084B (en) * | 2020-01-09 | 2024-03-15 | 长安大学 | Remote system and method based on augmented reality collaborative assembly and maintenance |
| CN111260084A (en) * | 2020-01-09 | 2020-06-09 | 长安大学 | Remote system and method for collaborative assembly maintenance based on augmented reality |
| CN111050112A (en) * | 2020-01-10 | 2020-04-21 | 北京首翼弘泰科技有限公司 | Method for remote operation command or guidance by displaying mark on screen |
| CN113360057A (en) * | 2020-03-03 | 2021-09-07 | 中国移动通信集团贵州有限公司 | Remote interaction method and system based on augmented reality and electronic equipment |
| CN111294568A (en) * | 2020-03-13 | 2020-06-16 | 北京维卓致远医疗科技发展有限责任公司 | System and method for remotely controlling video fusion |
| CN113542332A (en) * | 2020-04-22 | 2021-10-22 | 中移智行网络科技有限公司 | Customer service video interaction method and equipment based on positioning labels |
| CN113542332B (en) * | 2020-04-22 | 2023-04-07 | 中移智行网络科技有限公司 | Customer service video interaction method and equipment based on positioning labels |
| CN111784005A (en) * | 2020-06-17 | 2020-10-16 | 北京中铁电梯工程有限公司 | Elevator maintenance method, maintenance device, maintenance system and computer-readable storage medium based on Internet of things |
| CN111950506A (en) * | 2020-08-25 | 2020-11-17 | 中山大学 | A kind of auxiliary method and system for MOCVD equipment maintenance based on AR technology |
| CN112130666A (en) * | 2020-09-16 | 2020-12-25 | 博彦科技（上海）有限公司 | Data processing method and system |
| WO2022061858A1 (en) * | 2020-09-28 | 2022-03-31 | Orange | Method for assisting a first user provided with a terminal at a scene |
| CN113780706A (en) * | 2020-10-27 | 2021-12-10 | 北京京东尚科信息技术有限公司 | A field operation and maintenance operation method and device based on visual enhancement |
| WO2022116545A1 (en) * | 2020-12-04 | 2022-06-09 | 全球能源互联网研究院有限公司 | Interaction method and apparatus based on multi-feature recognition, and computer device |
| CN112565844A (en) * | 2020-12-04 | 2021-03-26 | 维沃移动通信有限公司 | Video communication method and device and electronic equipment |
| CN112509148A (en) * | 2020-12-04 | 2021-03-16 | 全球能源互联网研究院有限公司 | Interaction method and device based on multi-feature recognition and computer equipment |
| CN112492281A (en) * | 2020-12-10 | 2021-03-12 | 郑州捷安高科股份有限公司 | Remote auxiliary maintenance method, device, equipment and storage medium |
| CN113674426A (en) * | 2021-07-19 | 2021-11-19 | 中国电建集团西北勘测设计研究院有限公司 | Equipment maintenance method and device using AR technology |
| CN114063954A (en) * | 2021-11-11 | 2022-02-18 | 广西电网有限责任公司崇左供电局 | Interactive system for remote cooperation of communication operation and inspection operation |
| EP4358041A1 (en) * | 2022-10-19 | 2024-04-24 | Koninklijke Philips N.V. | Video-processing for equipment maintenance |
| CN115776577A (en) * | 2022-10-27 | 2023-03-10 | 中电信数智科技有限公司 | AR dynamic labeling method based on free circle drawing in remote collaboration scene |
| CN119152405A (en) * | 2024-11-13 | 2024-12-17 | 广东电网有限责任公司中山供电局 | Bi-directional video fusion interactive overhaul AR (augmented reality) glasses system and overhaul method |
| CN119785165A (en) * | 2025-03-11 | 2025-04-08 | 深圳市贝铂智能科技有限公司 | Multi-mode interaction and image processing method for AI glasses, electronic device and storage medium |
| CN119785165B (en) * | 2025-03-11 | 2025-06-17 | 深圳市贝铂智能科技有限公司 | Multi-mode interaction and image processing method for AI glasses, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106339094B (en) | 2019-02-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106339094A (en) | Interactive remote expert cooperation maintenance system and method based on augmented reality technology | |
| CN109657609A (en) | Face identification method and system | |
| CN108089696A (en) | For the virtual reality and augmented reality of industrial automation | |
| CN103941716B (en) | Building equipment monitoring method, device and system | |
| CN109543549A (en) | Image processing method and device, mobile end equipment, server for more people's Attitude estimations | |
| CN112132197A (en) | Model training method, image processing method, device, computer equipment and storage medium | |
| CN109115221A (en) | Indoor positioning, air navigation aid and device, computer-readable medium and electronic equipment | |
| CN112307886A (en) | Pedestrian re-identification method and device | |
| EP2920758A2 (en) | Rotation of an image based on image content to correct image orientation | |
| CN109902681B (en) | User group relation determining method, device, equipment and storage medium | |
| CN108491823A (en) | Method and apparatus for generating eye recognition model | |
| CN114048541B (en) | Asset space marking method and system based on digital twins | |
| CN111242704A (en) | Method and electronic equipment for superposing live character images in real scene | |
| CN106843243A (en) | The management method of UAS and unmanned plane route | |
| CN110188690A (en) | A kind of intelligent vision analysis system based on unmanned plane, intelligent vision analysis system and method | |
| CN107452018A (en) | Speaker's tracking and system | |
| CN108073851A (en) | A kind of method, apparatus and electronic equipment for capturing gesture identification | |
| CN110111346A (en) | Semantic Segmentation Method of Remote Sensing Image Based on Disparity Information | |
| CN115115785A (en) | Multi-machine collaborative 3D modeling system and method for search and rescue in wild mountain and forest environment | |
| CN113916244A (en) | Method and device for setting inspection position, electronic equipment and readable storage medium | |
| CN115437507A (en) | AR (augmented reality) -glasses-based equipment maintenance guiding method and system and related equipment | |
| CN110334620A (en) | Appraisal procedure, device, storage medium and the electronic equipment of quality of instruction | |
| CN110009407A (en) | A kind of advertisement sending method, device, advertisement playback terminal and storage medium | |
| CN109902923A (en) | A kind of method and apparatus of asset management | |
| US20220207832A1 (en) | Method and apparatus for providing virtual contents in virtual space based on common coordinate system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Shao Peng Inventor after: Zhang Zhen Inventor after: Zhu Chunjian Inventor after: Zhang Guodong Inventor after: Zhang Yan Inventor after: Liang Bo Inventor before: Zhang Zhen Inventor before: Zhu Chunjian Inventor before: Zhang Guodong Inventor before: Zhang Yan Inventor before: Liang Bo |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 250103 room 1-101, office building, 2269 development road, high tech Zone, Ji'nan, Shandong Patentee after: Shandong Wanteng Digital Technology Co.,Ltd. Address before: 250103 room 1-101, office building, 2269 development road, high tech Zone, Ji'nan, Shandong Patentee before: SHANDONG WANTENG ELECTRONIC TECHNOLOGY CO.,LTD. |
|
| CP01 | Change in the name or title of a patent holder |