WO2008018156A1

WO2008018156A1 - Fluctuation corrected image display method and mobile object remote control system using the method

Info

Publication number: WO2008018156A1
Application number: PCT/JP2006/316123
Authority: WO
Inventors: Yuji Katayama; Yutaka Kidawara; Mina Aoki; Kenji Tanaka
Original assignee: Sanritz Automation Co., Ltd.; National Institute Of Information And Communications Technology, Incorporated Administrative Agency
Priority date: 2006-08-10
Filing date: 2006-08-10
Publication date: 2008-02-14
Also published as: JPWO2008018156A1; JP5140889B2

Abstract

A image display method and a mobile object remote control system adopting such a method for making accurate remote control possible are provided for carrying out processing of fluctuation correction to obtain a video image without unnaturalness in moving picture transmission to a personal computer or the like connected with the mobile object through the internet, wherein the mobile object in which a camera is installed is a car model, a robot arm or the like set in a remote place. In an operation machine that carries out displaying a receiving image on a monitor in order to make the mobile object remotely controllable while looking at monitor display of the image received from the camera installed in the mobile object through a network, drawing a picture of the received image on the monitor is made at a prescribed drawing timing interval and the drawn picture is an estimated image produced by delay processing of the received image.

Description

Description Image display method by fluctuation correction and moving body remote control system using the method

The present invention relates to an image display method based on fluctuation correction and a motion using the method.

'In relation to the body remote control system, more specifically, image display method by fluctuation correction and operation of the operator of the operation machine using the method, model car, model airplane, model boat The present invention relates to a moving object remote control system having a remote control function for remotely controlling the speed of a moving object such as an arm of a robot or an industrial robot.

'

Background Art A remote power camera device and a remote monitoring system device that can change an imaging region by remote control have been proposed. As such a remote camera device, for example, a remote camera device that can correct the control overshoot amount of the monitoring camera unit due to transmission delay or signal processing delay has been proposed (Japanese Patent Laid-Open No. 6-3880). No. 7). As a remote monitoring system device, for example, the image displayed on the receiving side when the camera is operating is shifted and displayed so that it approaches the range that the force camera is currently capturing. A monitoring system device has been proposed in which the display range is processed so that it does not appear that the image is delayed, and the delay caused by the remote operation of the force sensor cannot be felt (Japanese Patent Laid-Open No. 2000). 2— 1 8 5 9 5 6).

However, these technologies are premised on video transmission via a wired line, and by correcting for a fixed delay time, it is possible to transmit a video with no sense of incompatibility. When video transmission is performed, fluctuation in the arrival time of video data occurs because a uniform communication speed cannot be obtained.

In other words, the timing of drawing varies due to variations in communication delays, and sometimes data loss occurs due to communication errors, resulting in a feeling of strangeness in moving images (Figs. 20 to 2 2). See)

For this reason, for example, when a camera is mounted on a moving body such as a model car and the camera is operated while watching the image from the camera on a monitor such as a personal computer, this fluctuation is a discontinuity of the video data. As a result, the operator cannot predict the movement of the model car, etc., making it difficult to perform accurate operations, resulting in poor remote control.

As a countermeasure, conventionally, a method has been adopted in which fluctuations are suppressed by accumulating video data in an amount that covers the fluctuation change range on the receiving side. However, with this method, there is a problem in that a video delay occurs due to the accumulation of video data.

Patent Document 1 Japanese Patent Laid-Open No. 6-3 8 0 8 7

Patent Document 2 Japanese Patent Laid-Open No. 2 0 0 2-1 8 5 9 5 6 · Disclosure of Invention

Problems to be solved by the invention

The present invention has been made with the object of solving the above problems, such as a model car equipped with a camera installed in a remote place, a personal computer connected via a Internet from a moving body such as a robot arm, etc. When video transmission is performed, an image display method that can obtain unnatural images by performing fluctuation correction processing, and by using this method, accurate remote operation becomes possible. The task is to provide a moving object remote control system.

Means for Solving the Problems' The invention according to claim 1 for solving the above-mentioned problems is characterized in that the moving object is viewed while viewing the monitor display of an image from a camera mounted on the moving object such as a model car or a robot arm. The monitor to the monitor to enable remote control In an operation machine that displays a received image, the received image is drawn on the monitor at a predetermined drawing timing interval, and the drawn image is delayed from the received image. This is an image display method by fluctuation correction, characterized by being a processed predicted image.

The invention described in claim 2 is the image display method according to claim 1, wherein the predicted image subjected to the delay process is received from the system time at the time of the drawing at that time. This is a pseudo image after the estimated delay time after subtracting the time stamp added to. ·

The invention according to claim 3 is the image display method by fluctuation correction according to claim 1, wherein the delay prediction image is the received image, the delay prediction time, the moving speed of the moving object, a swing angle, and a battery voltage. The image processing is based on sensor information such as values.

According to a fourth aspect of the present invention, in the image display method by fluctuation correction according to the third aspect of the invention, the image processing zooms the received image based on the sensor information and shifts it vertically, vertically, horizontally, or diagonally. It is characterized by that.

The invention according to claim 5 for solving the above-mentioned problem is a moving object remote equipped with an operation machine for remotely controlling the moving object based on image information from a moving object such as a model car or a mouth bot arm. A control system,

The operation machine is

Image drawing information receiving means for receiving image information and sensor information from a camera mounted on the moving body;

Image expansion means for expanding image data in the received image drawing information;

Image processing means for adding the decompressed image data based on the received sensor information;

Operation input means for converting the operation input value input by the controller into control information for the moving object; Control information transmitting means for transmitting the control information generated by the operation input means to the moving object,

The moving body is

An image compression means for adding the time of an internal clock as a time stamp to the image of the mounting force mera, and compressing the image as image information to be transmitted to the operation machine, and movement of the moving object from the speed sensor Sensor one capturing means for capturing the speed and the battery voltage value from the battery as sensor information to be transmitted to the operation machine, and the operation using the image 'information and the sensor information as image drawing information. Image drawing information transmitting means for transmitting to the machine, control information receiving means for receiving the control information from the operation machine, control processing means for controlling the servo and motor based on the received control information, and This is a moving body remote control system characterized by comprising: According to a sixth aspect of the present invention, in the moving object remote control system according to the fifth aspect, the image processing stage outputs the expanded image data based on the received sensor information. It is characterized in that it adjusts the timing to perform and the drawing area of the image to be displayed. '

The invention according to claim 7 is the moving object remote control system according to claim 6, wherein the image processing means calculates the time of the time dump stored in the expanded image data as the current system time. The drawing delay time subtracted from the image is calculated, the image enlargement ratio is calculated from the drawing delay time and the moving speed of the moving object, and the image shift in the up / down, left / right or diagonal direction from the drawing delay time and the shake position is calculated. An amount is calculated, and the drawing area is designated from the image enlargement ratio and the image shift amount.

The invention according to claim 8 is the moving object remote control system according to claim 5, wherein the sensor information includes a speed value from a speed sensor mounted on the moving object and a voltage value of a battery. It is characterized by inclusion. The invention's effect

-According to the image display method by fluctuation correction according to the present invention, the predicted images (pseudo images) obtained by image processing from the received images are displayed at regular intervals, and thus fluctuations are eliminated. When video transmission is performed from a moving body such as a model car equipped with a camera installed in a mobile phone or a personal computer connected via the Internet from a moving body such as a mouth bot arm, this fluctuation correction process makes it unnatural. This makes it possible to obtain accurate remote control. BEST MODE FOR CARRYING OUT THE INVENTION

DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The following explanation also applies to the case where the moving object is a model vehicle, or to a moving object such as an arm of another model airplane, model boat, or robot or industrial robot.

FIG. 1 shows an overall configuration of a moving object remote control system according to the present invention, which is composed of a model car 1 and an operation machine 2. This system consists of a model car 1 that is a moving body equipped with a camera, an operation machine 2 with a motor of an operator who operates the model car 1 at a distance from the model car 1, an internet 4 and so on. And communication means.

In operation machine 2, the camera image of model car 1 is output to the monitor. The operator can operate the model car 1 by inputting control information with a game controller such as a steering controller or joystick or other controller 3 while watching the camera image of the model car 1. Is possible. Then, in the model vehicle 1 that has received the control information, the control data is read from the control information by the in-vehicle computer, and the already driven servo (steering) and drive motor are controlled. Thus, the model car 1 can operate according to the control information input by the operation machine 2. FIG. 2 is a block diagram showing the configuration of the model automobile 1 in the present embodiment. The model car 1 in this embodiment is connected to a wireless LAN, and has the function of generating sensor information from the sensors in the model car 1 and the image information for adding a time stamp value to the camera capture image. In addition to a transmission information processing function such as a generation function, it also has a control function that performs predetermined processing on the servo and drive motor. In the model automobile 1 of the present embodiment, a camera 5 for realizing the image information generation function, a vehicle speed sensor 6 for realizing the sensor information generation function, a battery 7,

'The servo 8 and the drive motors 9 and 9a for realizing the control function, the gearboxes 10 and 10a, and the in-vehicle computer 11 that functions as various function processing means are provided.

The in-vehicle computer 1 1 is a CPU that controls each device and calculates data. The CPU that performs processing, the contents once written cannot be erased, but the contents are not lost even when the power is turned off. F-ROM, RAM using semiconductor elements , Internal clock 1 3, video capture section 1 2, image compression section, sensor input section 16, motor control section 15, wireless LAN communication section 14.

The in-vehicle computer 11 is connected with a camera 5, a vehicle speed sensor 6, a battery 7, a servo 8, and drive motors 9 and 9a (see Figs. 2 and 3), Fig. 4 As shown in the figure, the information sent to the operation machine 2 is processed and the information received from the operation machine 2 is processed.

Processing of information sent to the operation machine 2 includes image compression processing that compresses the image by adding the time stamp value of the internal clock 1 3 to the image from the video capture part 1 2, and the vehicle speed sensor 6. Therefore, there is a sensor acquisition process that acquires the vehicle speed and battery voltage value obtained as sensor information. The information received from the operation machine 2 is processed by a motor control process, and the servo 8 and the drive motors 9 and 9a are controlled according to the received control information.

The operation machine 2 of the present embodiment displays a predetermined drawing area of the video of the in-vehicle power camera 5 on the monitor. And to model car 1 The function of recognizing and determining the state of the model car 1 from the detection results of the installed sensors, etc., and outputting the vehicle power image according to the result, the speed meter display and the battery remaining capacity display, It is executed in conjunction with image information and sensor information from model car 1. For example, processing for receiving image drawing information such as “image data, time stamp, vehicle speed, battery voltage value” is performed, and the image information and sensor information included in this image drawing information are determined. A process corresponding to the next video is executed.

Here, the image display method by fluctuation correction according to the present invention employed in the above system will be described in more detail. The image display method displays the received image of the monitor for enabling the moving object to be remotely controlled while observing a monitor display of the image from the camera of the moving object such as a model car received via a network. In the operation machine which performs the drawing, the drawing of the received image on the monitor is performed at a predetermined drawing timing interval, and the drawn image is a predicted image obtained by delaying the received image. Features.

The predicted image subjected to the delay process is a pseudo image after the delay predicted time has elapsed, which is obtained by subtracting the time stamp added to the image information received at that time from the system time at the time of the drawing. The delay prediction image is obtained by subjecting the received image to image processing based on sensor information such as the delay prediction time, vehicle speed, steering angle (swing angle), battery voltage value, and the like. The received image is zoomed and / or shifted left and right based on sensor information.

This fluctuation correction method will be described in detail with reference to FIGS. In Fig. 5, S a, S b, S c, S d, Se in the stage of the sending image indicate the imaging time when the image is captured from the camera, and S a, S b, S in the stage of fluctuation. c, S d, and Se indicate time stamps in which image information is added to the image information at the time of image capture from force force, and Ta, T b, T c at the stage of drawing the received image , Td, Te indicate the drawing timing time. In receiving-side image drawing, delayed prediction images are drawn at equal intervals, so that there is no variation in drawing timing, in other words, there is no fluctuation and a smooth video can be obtained. The estimated delay time t is a time T 1 S obtained by subtracting the time stamp S added to the image information received at that time from the drawing timing time.

6 to 8 explain the drawing function of the delayed prediction image of the operation machine 2. FIG. As shown in FIG. 6, the operation machine 2 has a function of transferring the range to the video memory by designating a drawing area on the received image. This function makes it possible to display any area of the received image.

Fig. 7 explains the change in the drawing area when the model car 1 goes straight, which is used to monitor the operation machine 2 when the model car 1 goes straight from point A to point B. The change in the displayed area is shown. As you can see from this change in the drawing process, if the model car 1 goes straight ahead, you can zoom in on the area without changing the center of the viewpoint of the received image. . The zoom amount in this case is calculated from the vehicle speed, which is sensor information.

In addition, Fig. 8 explains the change in the drawing area of model car 1 when it turns right, where model car 1 moves from point A to point B to make a right curve. The change of the area displayed on the monitor of Operation Machine 2 is shown. As can be seen from the change in the drawing area, when the model car 1 is carved to the right, it is only necessary to zoom while gradually moving the center of the viewpoint to the right with respect to the received image. The zoom amount and shift amount in this case are calculated from the vehicle speed and the steering angle, which are sensor information.

Thus, in the image display method according to the present invention, the sensor information is used to generate a predicted delay image when the predicted delay time of the received image has elapsed, and this is drawn at the drawing timing time. Fluctuations are corrected, and real-time videos without unnaturalness can be obtained. Next, with reference to the flowchart of FIG. 9, the flow of a series of processing by the operation of the model car 1, the operation machine 2, and the operator of the system according to the present invention adopting the image display method described above is simply described. explain. First, in the model car 1, the camera image is time-stamped by the internal clock 13 and image-compressed, and this image information is used as the first data to be transmitted to the operation machine 2.

In addition, the vehicle speed value obtained by the vehicle speed sensor 6 and the battery voltage value are captured by the sensor capture process, and are used as sensor information.

'It is the second data sent to the machine 2.

The image information generated by the image compression process and the sensor information generated by the sensor one capture process are transmitted by the communication process as information transmitted to the operation machine 2. The operation machine 2 receives this information through communication processing, and passes the image information to the image expansion processing and the sensor information to the image processing processing. In the image expansion process, the image data in the image information box is expanded and transferred to the image processing process. In image processing, an image to be drawn on the motor is generated from the decompressed image data, time stamp, and sensor information and output to the monitor.

, The operator operates the game controller while viewing the video from the camera displayed on the monitor. The control information input by the operation of the game controller is transmitted to the model car 1 through communication processing. The model car 1 controls the servo 6 and the drive motors 7 and 7 a based on the received control information. To do.

Hereinafter, each process in the flowchart shown in FIG. 9 will be described in detail. First, the wireless LAN communication processing of the in-vehicle computer 11 according to the present invention will be described based on the flowchart of FIG. The wireless LAN communication unit 14 (see FIG. 3) of the in-vehicle computer 11 1 is waiting to receive the system connection information from the operation machine 2 during normal times (S 1 0 1).

When the information is received, it is confirmed whether it is the system connection information from the operation machine 2 (S 1 0 2). If it is the system connection information from the machine 2, check whether the authentication code is normal (S 1 0 3).

If the connection code is normal, connection response information is sent to the source operation machine 2 and the system status is switched to connected (S 1 0 4, S 1 0 5), and the internal clock 1 3 is connected to the system. Set to the system time data in the information '(S 1 0 6).

If the information received from operation machine 2 is not system connection information (S 1 0 2), if it is control information (S 1 0 7), check whether the system status is connected (S 1 0 8). If the system is connected, transfer control information to the motor control process

(S 1 0 9), internal clock timing processing is performed (S 1 1 0). If a control information reception error occurs (S 1 1 1), the system state is switched to unconnected (S 1 1 2).

Conversely, when system information is not received, it is checked whether or not there is transmission information to Operation Ma. Thin 2 (S1 1 3). If there is transmission information, whether or not the system status is connected. Check

(S1 1 4), send transmission information (S1 1 5).

The image compression processing of the in-vehicle computer 11 according to the present invention will be described with reference to the flowchart of FIG. The in-vehicle computer 1 1 determines whether or not it is connected to the operation machine 2 on the system during normal times (S 2 0 1), and if it is connected, starts the video capture (S 2 0 2 ) The captured image is compressed (S 2 0 3), and the time stamp generated from the internal clock 13 is added to create image information (S 2 0 4, S 2 0 5), and wireless LAN communication Passed to processing (S 2 0 6).

Next, the motor control processing and sensor input processing of the in-vehicle computer 11 according to the present invention will be described with reference to the flowchart of FIG. The motor control unit 15 is on standby in preparation for receiving control information from the operation machine 2 during normal times (S 3 0 1). When control information is received, according to the drive motor value and servo position information included in the control information, Outputs to control the drive motors 9, 9a and servo 8 (S3 02, S3 03).

In order to use the vehicle speed and battery voltage value immediately after being controlled by the motor control process as sensor information to be sent to the operation machine 2, the drive motors 9, 9a and servo 8 are controlled by the motor control process. Then, sensor input processing starts in conjunction with each other (S 3 0 4).

When the sensor input unit 16 receives a processing start command by the motor control processing of the motor control unit 15 (S 4 0 1), it captures the vehicle speed and the battery 7 voltage from the vehicle speed sensor 6 respectively. (S 4 0 2, S 4 0 3), create sensor information based on them (S 4 0 4), and send the sensor information to the operation machine 2 to send the sensor information to the wireless LAN communication unit 1 Deliver (S 4 0 5).

Next, the internal clock timing processing of the in-vehicle computer 11 according to the present invention will be described with reference to the flowchart of FIG. In the internal clock timing processing, the time difference between the system time of the internal clock 1 3 and the operation machine 2 is calculated from the control information transmission time and communication delay time, and the timing time is calculated (S5 0 1 to S 5 0 3 ) The timing is set to 1/20 of the time difference and absorbs sudden changes. This process is performed when operation machine 2 is connected to the system.

If the calculated timing is less than 1 ms, the internal clock 1 3 will not be reset, but if it is 1 ms or more, the internal clock 1 3 will be adjusted by adding the timing to the internal clock 1 3 time. Set again (S5 0 4, S 5 0 5).

The operation machine 2 is a personal computer in which a program for remotely operating the model car 1 is stored. An operator can operate the operation machine 2 to remotely control the model car 1.

As shown in Fig. 14 ', the processing performed by the operation machine 2 is roughly divided into processing based on data transmitted from the model car 1 and processing based on input from the game controller 3 by the operator. Is done. Processing based on the data sent from model car 1 consists of image expansion processing that expands image information received by wireless LAN communication processing, and images that are expanded by image expansion processing are monitored based on the received sensor information. This is an image processing process for generating a video signal to be projected on the screen.

On the other hand, the processing based on the input from the game controller 3 by the operator is shown as the operation input processing. However, the control information processed here is a model car by wireless LAN communication processing. Sent to.

The wireless LAN communication processing of the operation machine 2 according to the present invention is illustrated in FIG.

'Explain with reference to the flow chart of '15. In the wireless LAN communication processing of the operation machine 2, it is checked whether the connection with the model car 1 is maintained during normal times (S 6 0 1).

If the connection with model car 1 is not maintained, it is determined whether or not the specified time has elapsed since the transmission of the system connection information sent at the previous connection (S 6 0 2). If it is, the system connection information is transmitted again (S 6 0 3).

If the connection response information from the model car 1 is received (S 6 0 4), it is determined from the response code whether the connection is normal (S 6 0 5), and if the connection is normal, Switch the system status to connected (S 6 0 6).

When the connection with the model car 1 is maintained (S 6 0 1), it is always checked whether there is information received from the model car 1 (S 6 0 7). If the received information from the model car 1 is image information (S.6 0 8), the image information is transferred to the image expansion processing program (S 6 0 9).

If the received information is not image information but sensor information (S 6 1 0), the communication delay time is calculated from the current time and communication time (S 6 1 1), and the sensor information is transferred to the image processing process ( S 6 1 2), reset the communication monitoring timer (S 6 1 3).

When there is no information received from model car 1, it is checked whether there is information to send to model car 1 (S 6 1 4). If there is a report, the transmission information is transmitted (S 6 15), and the communication monitoring timer is reset (S 6 16). When the state where there is no reception information and no transmission information continues and the communication monitoring time is timed out (S 6 17), the system state is switched to the unconnected state (S 6 18).

The image expansion processing of the operation machine 2 according to the present invention will be described with reference to the flowchart shown in FIG. In the image expansion process of the operation machine 2, it is waiting in preparation for the reception of image information from the model car 1 during normal times (S 7 0 1). When the image information is received, the time stamp included in the image information is stored (S700), the image data is expanded, and one frame of image is transferred to the image processing process (S700). ) Subsequently, the image processing of the operation machine 2 according to the present invention will be described with reference to the flowchart of FIG. In the image processing processing of the operation machine 2, when an image that has been subjected to image expansion processing is updated while connected to the system (S 8 0 1, S 8 0 2), first, the image is stored in the updated image. The time obtained by subtracting the time of the time stamp from the current system time, ie, the drawing delay time is calculated (S 8 0 3). When the drawing delay time is calculated, the image enlargement ratio is calculated from the drawing delay time and the vehicle speed (S 8 0 4), and the image right / left shift amount is calculated from the drawing delay time and the steering position (S 8 0 5). )

Then, a drawing area is designated from the calculated image enlargement ratio and image left / right shift amount (S 8 06) and transferred to the video memory (S 8 0 7). On the other hand, based on the vehicle speed data and battery voltage data in the sensor information, the speed meter and the remaining battery capacity are overwritten (S 8 08).

The operation input process of the operation machine 2 according to the present invention will be described with reference to the flowchart of FIG. In the operation machine 2 in this embodiment, the system control cycle is set to 50 ms, and it is monitored whether or not the input from the game controller 3 is input at a fixed cycle of 50 ms (S 9 0 1). And if there is an input with a fixed period of 50 ms from the game controller 3 forces, the steering value, accelerator 'blur Take the key pedal value (S9002). Then, the steered steering value is used as the steering position, the accelerator / brake pedal value is converted as the drive motor value, and this is used as the control value for transmission to model car 1 (. S 9 0 3).

If there is no change in the converted control value (S900) and there is no transmission history for more than 500 ms since the last transmission of control information (S900), the control information is obtained from the current control value. Create and transfer to the wireless LAN communication process (S900) If there is a change in the converted control value, create control information from the control value and transfer it to the wireless LAN communication process (S9 0 6).

The communication data in the present invention is summarized as shown in FIG. In other words, the system connection information transmitted when the operation machine 2 is connected to the system is composed of the authentication code and the system time. Here, the authentication code is the code for operation machine 2 to connect to the system, and the system time is the time of operation machine 2, and the in-vehicle computer 1 1 internal clock 1 when the system is connected Used to time 3

Although the present invention has been described in some detail with respect to its most preferred embodiments, it is apparent that various embodiments can be constructed without departing from the spirit and scope of the present invention, so that the present invention is It is not limited to that particular embodiment, except as limited in the scope of. .

For example, as described above, a model automobile has been described as a moving object. However, the moving object to be controlled by the method according to the present invention is not limited to this, and a model airplane, a model boat ', a robot, It goes without saying that industrial robot arms, industrial equipment and other various moving objects are included.

In the case of a model car, the swing angle is the steering angle in the left-right direction. For example, in the case of an arm for an industrial robot, the swing direction may be in the up-down direction or the diagonal direction. In the image processing, the received image may be shifted vertically or diagonally. Brief Description of Drawings

FIG. 1 shows a system configuration diagram of a model car remote control system using an operation machine according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a model automobile in one embodiment of the present invention.

Fig. 3 is a block diagram of the in-vehicle computer.

Fig. 4 is a block diagram showing the processing of the in-vehicle computer.

FIG. 5 is a diagram for explaining fluctuation correction.

Fig. 6 is a diagram for explaining the drawing function of the operation machine. Fig. 7 is a diagram showing the change of the drawing work when the model car goes straight.

Fig. 8 is a diagram showing the change in the drawing work when the model car is on the right curve. Figure 9 Flow chart of a series of processing by hardware and operator.

Figure 10 is a flowchart of communication processing on the model car side.

FIG. 11 is a flowchart of image compression processing.

Fig. 1 Flow chart of 2 “Motor operation” process and sensor input process.

Figure 1 3 Flow chart of internal clock timing processing.

Fig. 14 is a block diagram showing the processing of the 4 operation machine.

Figure 15 Flowchart of communication processing on the operation machine side.

Fig. 6 Flow chart of image expansion processing.

Figure 17 is a flowchart of image processing.

Figure 1 8 Flow chart of operation input processing.

Fig. 19 is a block diagram of communication data for a remote control system for a model car.

FIG. 20 is a diagram for explaining fluctuations in communication. Figure 21 is a diagram for explaining the effect of frame fluctuation on the frame drawing timing.

• Fig. 22 This is a diagram for explaining the effect of frame errors on the frame drawing timing.

Claims

The scope of the claims

1 In an operation machine that displays the received image on the above-mentioned monitor to enable remote control of the moving object while watching the monitor display of the image from a camera mounted on the moving object such as a model car or robot arm. And

Drawing of the received image on the monitor is performed at a predetermined drawing timing interval, and the drawn image is a predicted image obtained by delaying the received image. Method.

2. The predicted image subjected to the delay process is a pseudo image after a delay predicted time has elapsed by subtracting a time stamp added to the image information received at that time from the system time at the time of the drawing timing. An image display method by fluctuation correction described in 1.

3. The fluctuation correction according to claim 1, wherein the delay prediction image is obtained by performing image processing on the received image based on the delay prediction time and sensor information such as a moving speed, a swing angle, and a battery voltage value of the moving object. Image display method by.

4. The image display method according to claim 3, wherein the image processing zooms the received image based on the sensor information and / or shifts the image vertically and horizontally and diagonally.

5 A moving body remote control system equipped with an operation machine for remotely controlling the moving body based on image information from moving bodies such as model cars and robot arms,

The operation machine is

An image drawing information receiving means for receiving image information and sensor information from a force mesa mounted on the moving body;

Image processing means for adding the decompressed image data based on the received sensor information; Operation input means for converting the operation input value input by the controller into control information for the moving object;

Control information transmission means for transmitting the control information generated by the operation input means to the moving body,

The moving body is

Image compression means for adding the time of the internal clock as a time stamp to the image of the mounted turtle and compressing the image as image information to be transmitted to the operation machine;

Sensor capturing means for capturing the moving speed of the moving body from the speed sensor and the battery voltage value from the battery as sensor information to be transmitted to the operation machine;

Image drawing information transmitting means for transmitting the image information and the sensor information as image drawing information to the operation machine;

Control information receiving means for receiving the control information from the operation machine;

Control processing means for controlling the servo and the motor based on the received control information;

And a moving body remote control system. 6. The moving object remote control according to claim 5, wherein the image processing means adjusts a timing for outputting the decompressed image data based on the received sensor information and a drawing process for an image to be displayed. system.

7 The image processing means calculates a drawing delay time obtained by subtracting a time stamp stored in the decompressed image data from a current system time, and enlarges the image from the drawing delay time and the moving speed of the moving object. And calculating the image shift amount in the up / down, left / right, or diagonal directions from the drawing delay time and the shake position, and the drawing area is calculated from the image enlargement rate and the image shift amount. The moving body remote control system according to claim 6 to be specified. 8. The moving object remote control system according to claim 5, wherein the sensor information includes a speed value from a speed sensor mounted on the moving object and a voltage value of a battery.