JP7547857B2 - Work vehicle voice operation system, work vehicle voice operation method and program - Google Patents

Description

The present invention relates to a work vehicle voice operation system, a work vehicle voice operation method, and a program.

Some work vehicles allow you to control various parts of the vehicle using voice commands.

For example, Patent Document 1 discloses that various parts of a crane, such as the boom and hoisting device, can be operated by voice using a voice operation device that recognizes voice instructions given by an operator.

JP 2017-160007 A

However, with a work vehicle, operation may be restricted depending on the position or condition of the work vehicle. For example, if the outrigger jack is not installed, boom operation is restricted and the boom cannot be operated.

However, the voice operation device described in Patent Document 1 only extracts operation information by performing voice recognition and voice analysis on the voice instructions, and performs operations based on the extracted operation information. Therefore, if the operator issues a voice instruction to perform a restricted operation without realizing that the operation is restricted due to the attitude and state of the work vehicle, there is a risk that the voice operation device described in Patent Document 1 will perform the operation using the operation information extracted from the voice instruction.

To ensure safety, operations using such voice-operated devices are blocked by the vehicle's controller, but in that case, the operator's voice instructions themselves become useless. Also, from the operator's perspective, some voice operations are executed and some are not, making voice operation difficult.

The present invention has been made to solve the above problems, and aims to provide a work vehicle voice operation system, a work vehicle voice operation method, and a program that allow easy voice operation even when the operation of the work vehicle is restricted.

In order to achieve the above object, a work vehicle voice operation system according to a first aspect of the present invention comprises:
a voice recognition result acquisition unit that acquires a voice recognition result from a voice recognition device that recognizes voice instructions uttered by an operator to operate the work vehicle;
an executable operation selection unit that selects an executable operation that can be executed under the attitude or state of the work vehicle based on the attitude or state detected by an attitude detector or state detector equipped in the work vehicle;
a response generation unit which determines whether the voice recognition result acquired by the voice recognition result acquisition unit includes an instruction for operating the work vehicle , and if it is determined that the instruction is not included, generates a response by referring to a response database in which responses to vocabulary other than the instruction included in the voice recognition result are stored, and if it is determined that the instruction is included, further determines whether the instruction is an instruction for a restricted operation that is restricted under the attitude or state, and if it is determined that the instruction is an instruction for the restricted operation, generates a response recommending the executable operation instead of the instruction , and if it is determined that the instruction is not an instruction for the restricted operation, generates a response to the effect that the operation of the instruction is to be executed;
a presentation unit that presents the response generated by the response generation unit;
Equipped with.

In addition, a work vehicle voice operation system according to a second aspect of the present invention comprises:
a voice recognition result acquisition unit that acquires a voice recognition result from a voice recognition device that recognizes voice instructions uttered by an operator to operate the work vehicle;
an executable operation selection unit that selects an executable operation that can be executed under the attitude or state of the work vehicle based on the attitude or state detected by an attitude detector or state detector equipped in the work vehicle;
a response generating unit which, when the voice recognition result acquired by the voice recognition result acquiring unit includes an instruction to operate the work vehicle and the instruction is an instruction for a restricted operation that is restricted under the attitude or state, generates a response recommending the executable operation instead of the instruction, and, when the voice recognition result acquired by the voice recognition result acquiring unit includes an instruction to operate the work vehicle and the instruction is not an instruction for the restricted operation, generates a response to execute the operation of the instruction;
a presentation unit that presents the response generated by the response generation unit;
a storage unit that stores the operation history by storing the operation content of the voice instruction every time the response generation unit generates a response to execute the operation instructed by the operator by voice;
Equipped with
When the voice recognition result acquired by the voice recognition result acquisition unit includes instructions for operating the work vehicle, and the instructions are instructions for the regulated operation, and there are multiple executable operations selected by the executable operation selection unit, the response generation unit generates a response recommending an executable operation among the multiple executable operations that corresponds to the most recent operation or the most frequent operation in the history data stored in the memory unit.

Furthermore, a voice operation method for a work vehicle according to a third aspect of the present invention includes:
a voice recognition step in which the first computer or the second computer recognizes voice instructions uttered by an operator to operate the work vehicle;
an executable operation selection step in which the second computer selects an executable operation that can be executed under the attitude or state of the work vehicle based on the attitude or state detected by an attitude detector or state detector equipped in the work vehicle;
a response generating step in which the second computer determines whether or not the voice recognition result obtained by voice recognition in the voice recognition step includes an instruction to operate the work vehicle, and if it determines that the instruction is not included, generates a response by referring to a response database in which responses to vocabulary other than the instruction included in the voice recognition result are stored, and if it determines that the instruction is included, further determines whether or not the instruction is an instruction for a restricted operation that is restricted under the attitude or state, and if it determines that the instruction is an instruction for the restricted operation, generates a response recommending the executable operation instead of the instruction, and if it determines that the instruction is not an instruction for the restricted operation, generates a response to the effect that the operation of the instruction is to be executed;
Equipped with .

Further, a program for operating a work vehicle according to a fourth aspect of the present invention comprises:
A program for voice-operating a work vehicle using a microphone that converts voice instructions uttered by an operator to operate the work vehicle into electrical signals and outputs the electrical signals as voice data, and a voice recognition device that performs voice recognition on the voice data, comprising:
On the computer,
a speech recognition result acquisition step of acquiring a speech recognition result from the speech recognition device;
an executable operation selection step of selecting an executable operation that can be executed under the attitude or state of the work vehicle based on the attitude or state detected by an attitude detector or state detector equipped in the work vehicle;
a response generating step of determining whether the voice recognition result acquired in the voice recognition result acquisition step includes an instruction for operating the work vehicle , and if it is determined that the instruction is not included, generating a response by referring to a response database in which responses to vocabulary other than the instruction included in the voice recognition result are stored, and if it is determined that the instruction is included, further determining whether the instruction is an instruction for a restricted operation that is restricted under the posture or state, and if it is determined that the instruction is an instruction for the restricted operation, generating a response recommending the executable operation instead of the instruction , and if it is determined that the instruction is not an instruction for the restricted operation, generating a response to execute the operation of the instruction ;
This is to enable the execution of the above.

Further, a voice operation method for a work vehicle according to a fifth aspect of the present invention comprises:
a voice recognition step in which the first computer or the second computer recognizes voice instructions uttered by an operator to operate the work vehicle;
an executable operation selection step in which the second computer selects an executable operation that can be executed under the attitude or state of the work vehicle based on the attitude or state detected by an attitude detector or state detector equipped in the work vehicle;
a response generating step in which, when the voice recognition result obtained by voice recognition in the voice recognition step includes an instruction to operate the work vehicle and the instruction is an instruction for a restricted operation that is restricted under the attitude or the state, the second computer generates a response recommending the executable operation instead of the instruction , and when the voice recognition result obtained by voice recognition in the voice recognition step includes an instruction to operate the work vehicle and the instruction is not an instruction for the restricted operation, the second computer generates a response to execute the operation of the instruction;
a storage step of storing a history of operations by the second computer by storing the operation content of the voice instruction in a storage device every time the response generating step generates a response to execute an operation instructed by the operator by voice;
Equipped with
In the response generation step, when the voice recognition result obtained in the voice recognition step includes an instruction to operate the work vehicle, and the instruction is an instruction to perform a regulated operation, and there are multiple executable operations selected in the executable operation selection step, a response is generated that recommends an executable operation among the multiple executable operations that corresponds to the most recent operation or the most frequent operation in the history data stored in the storage device.
Furthermore, a program for operating a work vehicle according to a sixth aspect of the present invention comprises:
A program for voice-operating a work vehicle using a microphone that converts voice instructions uttered by an operator to operate the work vehicle into electrical signals and outputs the electrical signals as voice data, a voice recognition device that performs voice recognition on the voice data, and a storage device, comprising:
On the computer,
a speech recognition result acquisition step of acquiring a speech recognition result from the speech recognition device;
an executable operation selection step of selecting an executable operation that can be executed under the attitude or state of the work vehicle based on the attitude or state detected by an attitude detector or state detector equipped in the work vehicle;
a response generating step of generating a response recommending the executable operation in place of the instruction when the voice recognition result acquired in the voice recognition result acquiring step includes an instruction to operate the work vehicle and the instruction is an instruction for a restricted operation that is restricted under the attitude or state, and generating a response to execute the operation of the instruction when the voice recognition result acquired in the voice recognition result acquiring step includes an instruction to operate the work vehicle and the instruction is not an instruction for the restricted operation;
a storage step of storing a history of operations by storing the operation contents of the voice instruction in the storage device every time a response to execute an operation instructed by the operator by voice is generated in the response generating step;
Run the command,
In the response generation step, when the voice recognition result acquired in the voice recognition result acquisition step includes instructions for operating the work vehicle, and the instructions are instructions for the regulated operation, and there are multiple executable operations selected in the executable operation selection step, a response is generated recommending an executable operation among the multiple executable operations that corresponds to the most recent operation or the most frequent operation in the history data stored in the storage device.

According to the configuration of the present invention, when the voice recognition result includes an instruction to operate a work vehicle and the instruction is an instruction to perform a restricted operation, the response generation unit generates a response recommending an executable operation in place of the instruction, and the presentation unit presents the generated response. Therefore, even if the operator mistakenly gives a voice instruction to perform a restricted operation, an executable operation in place of the operation is presented, allowing smooth voice operation. As a result, voice operation is easy.

1 is a system configuration diagram of a work vehicle voice operation system according to a first embodiment of the present invention. FIG. FIG. 2 is a side view of the aerial work vehicle that is the object of operation of the work vehicle voice operation system. 4 is a block diagram of a control unit of a mobile terminal provided in the work vehicle voice operation system. FIG. 13 is a diagram showing an example of a data structure of restricted operation information acquired by an executable operation selection unit included in the control unit. FIG. 4 is an example of a data configuration diagram of a work vehicle operation database stored in a storage unit. FIG. 4 is a diagram showing an example of a data structure of a command dictionary stored in a storage unit; 4 is a flowchart of a work vehicle voice operation process performed by the work vehicle voice operation system. 13 is a block diagram of a control unit of a mobile terminal provided in a work vehicle voice operation system according to a second embodiment of the present invention. FIG.

The following describes in detail the vehicle voice operation system, vehicle voice operation method, and program according to the embodiment of the present invention with reference to the drawings. Note that in the drawings, the same or equivalent parts are given the same reference numerals.

(Embodiment 1)
The vehicle voice operation system according to the first embodiment is a system for voice-operating a vehicle for work at height using a mobile terminal. First, the configuration of the entire system will be described with reference to Figs. 1 and 2.

Figure 1 is a system configuration diagram of a work vehicle voice operation system 1A according to embodiment 1 of the present invention. Figure 2 is a side view of an aerial work vehicle 5 that is the target of operation of the work vehicle voice operation system 1A.

As shown in FIG. 1, the work vehicle voice operation system 1A is composed of a mobile terminal 2A to which voice instructions for operating the aerial work vehicle 5 are input, a server 3 that performs voice recognition on the input voice, and a controller 4A that controls the aerial work vehicle 5 based on the recognition results of the voice recognition.

The mobile terminal 2A is a small and lightweight information terminal device that can be carried by an operator. For example, the mobile terminal 2A is a smartphone or a tablet computer. The mobile terminal 2A includes a microphone 21, a control unit 22, a speaker 23, a memory unit 24, and a communication unit 25.

The microphone 21 converts voice into an electrical signal and outputs it as voice data. When an operator carrying the mobile terminal 2A gives a voice instruction to operate the aerial work vehicle 5, the microphone 21 outputs the voice instruction as voice data. The microphone 21 transmits the voice data to the control unit 22.

The control unit 22 includes a microcomputer including a CPU (Central Processing Unit) that performs arithmetic processing, and memory including a ROM (Read Only Memory) and a RAM (Random Access Memory). The CPU controls each part of the mobile terminal 2A by reading a program stored in the ROM or the storage unit 24 into the RAM and executing it. For example, the control unit 22 performs a work vehicle voice operation process by the CPU executing a work vehicle voice operation program.

In detail, the control unit 22 controls the communication unit 25 to transmit voice data of the operator's voice instructions input to the microphone 21 to the server 3. It also receives the voice recognition result transmitted from the server 3. The control unit 22 generates a response to the operator's voice instructions based on the voice recognition result. Then, it outputs the generated response data to the speaker 23. The control unit 22 also generates an operation signal for operating the aerial work vehicle 5 based on the voice recognition result of the server 3, and transmits the generated operation signal to the communication unit 25.

The speaker 23 converts the electrical signal of the response data into sound and outputs it. When the speaker 23 receives the response data from the control unit 22, it converts the response data into sound and outputs it. In this way, the speaker 23 presents the generated response to the operator.

Meanwhile, the memory unit 24 has a storage device such as an EEPROM (Electronic Erasable Programmable Read-Only Memory) or flash memory, and stores the above-mentioned work vehicle voice operation program.

The communication unit 25 includes a wireless communication circuit and an antenna (not shown), and is connected to a network 100 such as the Internet, an in-house network, or an in-vehicle network, to wirelessly communicate data with the server 3 and the controller 4A. For example, the communication unit 25 transmits voice data of voice instructions received from the control unit 22 to the server 3. It also transmits operation signals received from the control unit 22 to the controller 4A.

The server 3 includes a control unit 31, a storage unit 32, and a communication unit 33. The control unit 31 has a CPU, ROM, and RAM (not shown), and the CPU reads out programs and data stored in the ROM and uses the RAM as a work area to perform overall control of the server 3. The storage unit 32 has an EEPROM or flash memory (not shown), and stores various programs for performing various processes, the data thereof, and various data that is generated and acquired. The communication unit 33 connects to the network 100 and communicates with the mobile terminal 2A and the controller 4A.

The server 3 performs voice recognition processing by the control unit 31 executing a voice recognition program stored in the memory unit 32. In detail, the server 3 receives voice data of the operator's voice instructions from the communication unit 25 of the mobile terminal 2A and performs voice recognition on the voice data. The server 3 then transmits the voice recognition result to the communication unit 25 of the mobile terminal 2A.

On the other hand, the controller 4A is provided in the aerial work vehicle 5 and controls each device of the aerial work vehicle 5.

In detail, the controller 4A is equipped with a computer (not shown) including a CPU that performs calculation processing and memory including ROM and RAM. The CPU of the controller 4A reads out and executes an operating program stored in the storage device 61 shown in FIG. 1 or in the ROM (not shown) into the RAM. In this way, the controller 4A drives each of the devices shown in FIG. 2, such as the outriggers 52A-52D, the rotating base 53, the boom 54, and the bucket 55.

More specifically, as shown in FIG. 1, the vehicle for working at height 5 includes an operating device 56 for operating the above-mentioned various devices, and a drive unit 57 for driving the above-mentioned various devices. The controller 4A acquires operation information output from levers, switches, etc. (not shown) included in the operating device 56 by the CPU executing the above-mentioned operating program, and operates the drive unit 57 based on the operation information. In this way, the controller 4A drives the above-mentioned various devices of the vehicle for working at height 5 in response to the operation of the lever of the operating device 56, the on/off of the switch, etc.

The controller 4A also determines the posture or state of the aerial work platform vehicle 5 based on the detection results obtained by the detectors 58A-58E. The controller 4A then determines the operation that should be restricted from the determined posture or state (hereinafter, referred to as the restricted operation) in order to ensure safety.

In detail, as shown in Figures 1 and 2, the aerial work platform 5 is equipped with a detector 58A that detects the extension and contraction of the beams and jacks of the outriggers 52A-52D, a detector 58B that detects the rotation angle of the swivel base 53, a detector 58C that detects the hoisting angle of the boom 54 relative to the vehicle 51, a detector 58D that detects the length of the boom 54 that changes as the boom 54 extends and contracts, and a detector 58E that detects the swing angle of the bucket 55 relative to the boom 54.

The controller 4A determines the installation state of the jacks of the outriggers 52A-52D, for example, based on the detection results of the detector 58A. If the determined installation state is a state in which the jacks are not installed (i.e., the float is not on the ground), the controller 4A restricts the operation of rotating, raising and lowering, and extending the boom 54 (hereinafter referred to as boom operation) to prevent tipping over.

The controller 4A also determines the attitude of the boom 54 based on the detection results of the detectors 58B-58E. If the determined attitude of the boom 54 is one in which the bucket 55 would interfere with the vehicle 51 if the boom 54 were rotated, the controller 4A restricts the boom operation to prevent such interference.

In addition, if the determined boom 54 posture is the working posture, i.e., the unstowed posture, the controller 4A restricts the extension and retraction of the beams of the outriggers 52A-52D and the extension and retraction of the jacks (hereinafter referred to as outrigger operation) to prevent the boom 54 from tipping over.

Furthermore, the controller 4A obtains the on/off state of a stop switch (not shown) from the output of the operating device 56. For example, if the obtained on/off state of the stop switch is the on state, the controller 4A treats it as being switched to a state in which the machine should be stopped in an emergency, etc., and restricts both the boom operation and the outrigger operation.

When such a restricted operation is performed, the controller 4A restricts the operation of the drive unit 57 linked to the lever of the operating device 56. This prevents the aerial work platform 5 from being operated by the restricted operation, thereby ensuring safety.

As shown in FIG. 1, the controller 4A is connected to a communication device 62 provided on the aerial work platform vehicle 5 in order to perform data communication with the above-mentioned portable terminal 2A. As a result, the controller 4A is connected to the network 100 via the communication device 62. Then, data communication is performed with the portable terminal 2A.

For example, the controller 4A receives an operation signal generated by the control unit 22 of the mobile terminal 2A to operate the vehicle for aerial work 5. When the controller 4A receives an operation signal, it activates the drive unit 57 based on the operation signal. This drives each device of the vehicle for aerial work 5, such as the outriggers 52A-52D, the swivel base 53, the boom 54, and the bucket 55. As a result, the vehicle for aerial work 5 is operated by the operation signal of the mobile terminal 2A.

The controller 4A also functions as an overload prevention device. That is, the controller 4A calculates the moment around the fulcrum of the swivel base 53 based on the output of the detectors 58A-58E, and if the calculated moment exceeds the limit moment, it stops the drive unit 57 to prevent overload.

In this way, in the work vehicle voice operation system 1A, when the operator's voice instructions are input to the microphone 21 of the mobile terminal 2A, the server 3 performs voice recognition on the voice data of the voice instructions, and the control unit 22 of the mobile terminal 2A generates a response to the voice instructions based on the voice recognition result. It also generates an operation signal based on the voice recognition result. The controller 4A then operates each device of the aerial work vehicle 5 based on the operation signal.

However, as described above, in order to ensure safety, the controller 4A prevents the aerial work vehicle 5 from being operated by restrictive operations that correspond to the posture or state of the aerial work vehicle 5. For this reason, if the operator issues a voice command to execute a restrictive operation without noticing the restrictive operation, the voice commanded operation may not be executed, which may confuse the operator. Furthermore, if the operator does not notice the restrictive operation, some voice commands will be executed and some will not, making voice operation difficult.

To solve this problem, the control unit 22 of the mobile terminal 2A performs a work vehicle voice operation process that, when an operation instructed by the operator through voice is a restricted operation, notifies the operator through voice a message that there is an alternative operation to the operation instructed through voice. In order to perform this work vehicle voice operation process, the control unit 22 has a processing block configured as software. Next, the processing block of the control unit 22 will be described with reference to Figures 3 to 6. The processing block of the control unit 31 of the server 3 will also be described.

Figure 3 is a block diagram of the control unit 22 and memory unit 24 of the mobile terminal 2A provided in the work vehicle voice operation system 1A. Figure 4 is an example of a data structure diagram of restricted operation information 71 acquired by the executable operation selection unit 222 provided in the control unit 22. Figure 5 is an example of a data structure diagram of the work vehicle operation database 241 stored in the memory unit 24. Figure 6 is an example of a data structure diagram of the command dictionary 242 stored in the memory unit 24. Note that Figure 3 also shows processing blocks of the control unit 31 and memory unit 32 of the server 3 to make it easier to understand the work vehicle voice operation processing.

As shown in FIG. 3, the control unit 22 of the mobile terminal 2A includes a voice recognition result acquisition unit 221, an executable operation selection unit 222, and a response generation unit 223.

The voice recognition result acquisition unit 221 transmits voice data to the server 3 and acquires the voice recognition results of the voice data from the server 3.

In detail, the voice recognition result acquisition unit 221 acquires voice data of the voice uttered by the operator from the microphone 21 provided in the mobile terminal 2A to obtain voice data for voice recognition. Specifically, when the operator speaks, the voice recognition result acquisition unit 221 starts acquiring voice data, and acquires voice data until the operator's speech is interrupted. In this way, the voice recognition result acquisition unit 221 acquires a series of voice data. Each time the voice recognition result acquisition unit 221 acquires a series of voice data, it transmits the acquired voice data to the control unit 31 of the server 3 via the communication units 25 and 33 shown in FIG. 1.

On the other hand, the control unit 31 of the server 3 includes a voice recognition unit 311 that performs voice recognition, as shown in FIG. 3. When the voice recognition unit 311 receives voice data from the voice recognition result acquisition unit 221, it refers to the voice recognition dictionary 321 stored in the memory unit 32, extracts vocabulary based on acoustically or linguistically plausible words from the voice data, and performs voice recognition. The voice recognition unit 311 transmits the recognition result to the voice recognition result acquisition unit 221 of the mobile terminal 2A shown in FIG. 3 via the communication units 33 and 25 shown in FIG. 1.

When the voice recognition result acquisition unit 221 receives the voice recognition result from the voice recognition unit 311, it transmits the voice recognition result to the response generation unit 223.

In response to this, the executable operation selection unit 222 selects an operation to replace the voice-instructed operation when the operation instructed by the operator's voice is a restricted operation. In order to select such an operation, the executable operation selection unit 222 identifies executable operations that can be performed by the aerial work vehicle 5 at the stage of performing voice recognition of the operator's voice.

In detail, each time the voice recognition result acquisition unit 221 acquires voice data from the microphone 21, the executable operation selection unit 222 acquires restricted operation information 71 shown in FIG. 4 from the controller 4A in order to identify the restricted operation. In addition, the storage unit 24 stores a work vehicle operation database 241 as shown in FIG. 3, and the executable operation selection unit 222 reads out the work vehicle operation database 241 from the storage unit 24 in order to obtain data for selecting an operation to replace the restricted operation.

In the above-mentioned restricted operation information 71, as shown in FIG. 4, the posture or state of the aerial work vehicle 5 is associated with the restricted objects (referred to as regulated objects) and restricted operation items of the restricted operation in that posture or state. In addition, in the work vehicle operation database 241, as shown in FIG. 5, the posture or state of the aerial work vehicle 5 is associated with the restricted objects and operation items of the operations that can be performed in that posture or state (hereinafter referred to as executable operations).

The executable operation selection unit 222 shown in FIG. 3 uses the read work vehicle operation database 241 to select an executable operation corresponding to the posture or state of the restricted operation information 71 acquired from the controller 4A. In this way, the executable operation selection unit 222 selects an executable operation for the restricted operation at the stage when the voice data is acquired from the microphone 21. The executable operation selection unit 222 then transmits data on the selected executable operation to the response generation unit 223 together with data on the posture or state of the aerial work vehicle 5 and data on the restricted operation.

The response generation unit 223 generates a response to the operator depending on whether the voice recognition result is an instruction (hereinafter referred to as a command) for operating the aerial work vehicle 5, and whether the voice recognition result is an instruction for a regulated operation.

In detail, the response generation unit 223 first receives the voice recognition result from the voice recognition result acquisition unit 221. Furthermore, in order to obtain data for generating a response, the response generation unit 223 receives data on the attitude or state of the vehicle 5, data on restricted operations, and data on executable operations from the executable operation selection unit 222. Furthermore, the memory unit 24 stores a command dictionary 242 in which the command object of the instruction (hereinafter referred to as a command) for operating the vehicle 5 shown in FIG. 6, the command content, the response to the command, and the operation signal of the vehicle 5 in response to the command are associated with each other, and the response generation unit 223 shown in FIG. 3 reads out the command dictionary 242.

The response generation unit 223 determines whether the received voice recognition result is related to a command by determining whether the voice recognition result contains words of the command object and command content in the command dictionary 242 or words similar to them.

When the response generation unit 223 determines that the received voice recognition result does not contain the words of the command object and command content in the command dictionary 242 or words similar thereto, it treats the voice recognition result as not related to a command. It then reads out the response database 243 stored in the storage unit 24. Although not shown, the response database 243 associates the vocabulary contained in the voice recognition result with a response when the voice recognition result is not related to a command. The response generation unit 223 generates a response by referring to the read response database 243. The response generation unit 223 outputs the generated response data to the speaker, and the speaker 23 converts the electrical signal of the response data into sound, thereby conveying the response to the operator.

On the other hand, if the response generation unit 223 determines that the voice recognition result contains words for the command object and command content or words similar thereto, it treats the voice recognition result as relating to a command. In that case, the response generation unit 223 determines whether the voice recognition result contains words for the restricted object and restricted items of the data for the restricted operation received from the executable operation selection unit 222 or words similar thereto, in order to determine whether the voice recognition result is related to a restricted operation.

When the response generation unit 223 determines that the received voice recognition result contains words of restricted objects and restricted items or words similar thereto, it determines that the voice recognition result is related to a restricted operation. Then, using the posture or state data and the executable operation data received from the executable operation selection unit 222, the response generation unit 223 creates a response sentence that suggests an executable operation as an operation to replace the voice-instructed operation.

In detail, the response generation unit 223 creates a response sentence stating that the aerial work vehicle 5 is in the posture or state data received from the executable operation selection unit 222, and therefore voice instructions are not possible, but that instead, the executable operation data received from the executable operation selection unit 222 can be operated.

The response generation unit 223 outputs the response data of the generated response sentence to the speaker 23. The response generation unit 223 presents to the operator that there is an executable operation that can be used instead of the operation instructed by voice, by the speaker 23 converting the electrical signal of the response data into voice.

On the other hand, if the response generation unit 223 determines that the received voice recognition result does not contain words for the restricted object and restricted item or words similar thereto, it determines that the voice recognition result does not relate to a restricted operation. The response generation unit 223 generates a response using a response to a command in the command dictionary 242 shown in FIG. 6 that is associated with the command object and command content in the command dictionary 242, which is determined to contain words for the command object and command content in the command dictionary 242 or words similar thereto, as described above. Then, the generated response is output to the speaker 23.

The generated response indicates that the command object will be executed according to the command content. The response generation unit 223 notifies the operator that the operation will be executed according to the voice-recognized command by having the speaker 23 convert the electrical signal of this response sentence into sound. The response generation unit 223 also outputs an operation signal from the command dictionary 242 associated with the command object and the words of the command content described above to the communication unit 25 shown in FIG. 1. This causes the operation signal to be sent to the controller 4A via the communication device 62. As a result, the response generation unit 223 operates the aerial work vehicle 5 according to the voice-recognized command.

Returning to FIG. 3, the memory unit 24 stores an operation history database 244 in which the transmission time is associated with the transmitted operation signal. Each time the response generation unit 223 outputs an operation signal to the controller 4A, the response generation unit 223 transmits the operation signal together with the clocked time to the operation history database 244 for storage. In other words, the operation content is transmitted together with the clocked time to the operation history database 244 for storage.

Depending on the posture or state of the restricted operation information 71, there may be multiple executable operations corresponding to the above-mentioned executable operations. As described above, when the response generation unit 223 determines that the received voice recognition result contains words for the restricted object and restricted item or words similar thereto, it creates a response sentence that suggests an executable operation instead of the voice-instructed operation. For this reason, in this case, there may be multiple suggested executable operations, and the operator may be confused as to which operation is desirable.

Therefore, in order to make the operation easier, when there are multiple executable operations, the response generation unit 223 uses the above-mentioned operation history database 244 to suggest one of the more desirable executable operations.

In detail, the response generation unit 223 reads the operation history database 244, and among the multiple executable operations, sets the most recently performed executable operation as the desired executable operation. Note that the response generation unit 223 may set the most frequently performed executable operation among the multiple executable operations as the desired executable operation. In this way, the response generation unit 223 generates a response that suggests one more desirable executable operation as an alternative to a voice instruction. As a result, voice operation is made easier.

Next, referring to FIG. 7, the work vehicle voice operation processing implemented by the work vehicle voice operation system 1A will be described. In the following description, it is assumed that when the mobile terminal 2A is started, an icon for starting the work vehicle voice operation program stored in the memory unit 24 is displayed on the display device of the mobile terminal 2A. In addition, after the work vehicle voice operation program is started, an end button for ending the work vehicle voice operation program is constantly displayed on the display device. Furthermore, it is assumed that a voice recognition program has been started in advance in the server 3, and voice recognition processing is being performed. And it is assumed that a remote operation selection button (not shown) is provided in the cabin of the aerial work vehicle 5.

Figure 7 is a flowchart of the work vehicle voice operation process performed by the work vehicle voice operation system 1A.

First, the operator of the aerial work vehicle 5 switches the remote operation selection button (not shown) to the on state. The operator then starts the mobile terminal 2A. Next, the operator taps the icon of the work vehicle voice operation program displayed on the display device. This causes the microcomputer of the control unit 22 to execute the work vehicle voice operation program. As a result, the work vehicle voice operation processing flow starts.

When the flow of the work vehicle voice operation processing is started, as shown in FIG. 7, the control unit 22 first determines whether or not there has been voice input to the microphone 21 (step S1). In this determination, if a sound that is louder than a certain volume is input to the microphone 21, it is determined that there has been voice input.

Next, if the control unit 22 determines that there has been voice input (Yes in step S1), it assumes that the voice input was made by the operator and acquires the voice data (step S2). In acquiring this voice data, a sound louder than the above-mentioned certain volume is made for a certain period of time until it is interrupted, and voice data of a continuous voice is acquired. On the other hand, if the control unit 22 determines that there has been no voice input (No in step S1), it returns to step S1 and repeats step S1.

After acquiring the voice, the control unit 22 transmits the acquired voice data to the server 3 (step S3). This transmission is performed by connecting the communication unit 25 of the mobile terminal 2A to the network 100.

In the server 3, the control unit 31 shown in FIG. 1 executes a voice recognition program to perform voice recognition processing. The control unit 31 receives voice data via the communication unit 33 and performs voice recognition on the received voice data. The control unit 31 then transmits the voice recognition result to the control unit 22 of the mobile terminal 2A via the communication unit 33.

Returning to FIG. 7, after transmitting the voice data to the server 3, the control unit 22 receives the voice recognition result transmitted from the server 3 (step S4).

Then, the control unit 22 selects an executable operation (step S5). In detail, the control unit 22 acquires the restricting operation information 71 from the controller 4A shown in FIG. 3, and further reads out the vehicle operation database 241 from the storage unit 24. As described above, the restricting operation information 71 associates the current attitude or current state of the vehicle 5 with the restricting operation at that attitude or state, and the vehicle operation database 241 associates each attitude or state of the vehicle 5 with an executable operation at that attitude or state. The control unit 22 uses the restricting operation information 71 and the vehicle operation database 241 to select an executable operation when the restricting operation of the acquired restricting operation information 71 is applied.

To give a specific example, in the example of restricted operation information 71 shown in Figure 4, the current posture or state of the aerial work vehicle 5 is a "jack not installed" state, with which the restricted operations of restricted object "boom" and restricted items "raise, lower, extend, retract, turn right, turn left" are associated. Meanwhile, in the example of work vehicle operation database 241 shown in Figure 5, the posture or state of the aerial work vehicle 5 in which "jack not installed" is associated with executable operations of operation object "outrigger" and operation item "extend", and executable operations of operation object "jack" and operation item "extend". In this case, the control unit 22 selects an operation with the operation object "outrigger" and the operation item "extension" as well as an operation with the operation object "jack" and the operation item "extension" as executable operations for the restricted operation information 71 with the restricted object "boom" and the restricted items "raise, lower, extend, reduce, turn right, turn left."

Returning to FIG. 7, following step S5, the control unit 22 determines whether the voice recognition result includes a command in the command dictionary 242 (step S6). In detail, the command dictionary 242 is stored in the storage unit 24, and the control unit 22 reads the command dictionary 242 and determines whether the voice recognition result includes a word of the command content in the command dictionary 242 or a similar word.

When the control unit 22 determines that the voice recognition result includes a command in the command dictionary 242 (Yes in step S6), it then determines whether or not the voice recognition result includes a word for a restricted operation (step S7). In detail, when the control unit 22 obtains data on restricted operations from the restricted operation information 71 in the selection of executable operations in step S5, it determines whether or not the data includes words for the restricted target and restricted items of the restricted operation or similar words.

For example, in the case of the example of the restricted operation information 71 in FIG. 4, the control unit 22 determines whether the words of the restricted object "boom" and the restricted items "raising, lowering, extending, reducing, turning right, turning left" or words similar thereto are included in the voice recognition result.

As shown in FIG. 7, when the control unit 22 determines that the voice recognition result contains a word for a restricted operation (Yes in step S7), it treats the operator's voice instruction as an instruction for a restricted operation, and generates a response proposing the executable operation selected in step S5 in place of the operation for which the voice instruction is made (step S8).

When generating this response, if there are multiple executable operations, the control unit 22 reads out the operation history database 244 from the storage unit 24, and uses the operation history database 244 to set the most recently performed executable operation among the multiple executable operations as the proposed executable operation. The control unit 22 then generates a response proposing the executable operation. Note that the proposed executable operation may be the executable operation that is most frequently found in the operation history database 244 among the multiple executable operations.

For example, if the control unit 22 determines that the voice recognition result includes the word "boom" for the restricted object and the word "extend" for the restricted item in the example of restricted operation information 71 shown in FIG. 4, the control unit 22 selects the operation of the operation object "outrigger" and the operation item "extend" and the operation of the operation object "jack" and the operation item "extend" in the work vehicle operation database 241 shown in FIG. 5 as executable operations to be proposed in the response. The control unit 22 then generates a response proposing the operation determined to be the most recent among the selected operations by referring to the operation history database 244 as a desirable executable operation to replace the voice-instructed operation, and proposing the other executable operations as the next desirable executable operations.

When the control unit 22 generates a response proposing an executable operation in step S8, the control unit 22 outputs the generated response data to the speaker 23 (not shown). This suggests to the operator that an executable operation be performed instead of the operation specified by voice.

On the other hand, as shown in FIG. 7, if the control unit 22 determines that the voice recognition result does not include a word for a restricted operation (No in step S7), it generates a response using a response sentence in the command dictionary 242 that corresponds to the command determined in step S6 (step S9).

In detail, in step S6, the control unit 22 determines whether or not the voice recognition result includes a command in the command dictionary 242, and reads out a response associated with the command determined to be included in the voice recognition result from the command dictionary 242. Then, a response is generated using the read response. Since the read response is a response to execute the command, the response generated by the control unit 22 is a response to execute the operation specified by the operator's voice. The control unit 22 outputs the data of this response to the speaker 23. This notifies the operator that the operation specified by voice will be executed.

For example, if the control unit 22 determines that the voice recognition result is "Extend the jack" and that the words "boom" and "extend" in the example of the restricted operation information 71 shown in FIG. 4 are not included, or similar words, it generates a response using the response "Yes, extend the jack" corresponding to the command object "jack" and the command content "extend" in the command dictionary 242 shown in FIG. 6, and outputs the generated response to the speaker 23.

The control unit 22 also reads out an operation signal associated with the command determined to be included in the voice recognition result from the command dictionary 242 (not shown), and outputs the operation signal to the communication unit 25. This causes the operation signal to be transmitted to the controller 4A via the communication device 62. As a result, the aerial work platform vehicle 5 operates according to the operation instructed by the operator through voice.

Returning to FIG. 7, once the control unit 22 generates the response in steps S8 and S9, it returns to step S1 to prepare for the next voice input by the operator.

On the other hand, if the control unit 22 determines in step S6 that the voice recognition result does not include a command in the command dictionary 242 (No in step S6), the control unit 22 generates a response by referring to the response database 243 in which responses to voice inputs other than commands are stored (step S10). Then, the control unit 22 outputs the generated response data to the speaker 23 to respond to the operator.

After generating a response in step S10, the control unit 22 returns to step S1 to prepare for the next voice input by the operator, similar to steps S8 and S9.

As described above, after the work vehicle voice operation program is launched, an end button for terminating the work vehicle voice operation program is constantly displayed on the display device of the mobile terminal 2A. When the operator wishes to terminate the work vehicle voice operation process, he or she presses the end button. This forcibly terminates the flow of the work vehicle voice operation process.

Of the configurations of the mobile terminal 2A and the server 3 described above, the speaker 23 is an example of a presentation unit as defined in this specification and the claims. The computer including the CPU provided in the control unit 31 of the server 3 is an example of a first computer as defined in this specification and the claims. The microcomputer provided in the control unit 22 of the mobile terminal 2A is an example of a second computer. The detectors 58A-58E are an example of a posture detector or state detector as defined in this specification and the claims. Furthermore, the steps of the server 3 performing voice recognition, steps S4, S5, and steps S6-S10, are examples of a voice recognition step, a voice recognition result acquisition step, an executable operation selection step, and a response generation step as defined in the claims.

As described above, in the work vehicle voice operation system 1A according to embodiment 1, when the voice recognition result includes a command for operating the work vehicle and also includes a word for a restricted operation, the response generation unit 223 generates a response recommending an executable operation instead of the operation instructed by the operator's voice. The speaker 23 then converts the generated response data into voice and presents the response to the operator. In the work vehicle voice operation system 1A, even if the operator mistakenly instructs a restricted operation by voice, an executable operation in place of the voice-instructed operation is presented to the operator, allowing smooth voice operation. Voice operation is also easy.

If the operator mistakenly issues a regulated operation by voice, the work vehicle voice operation system 1A will not execute the regulated operation but will instead present the operator with executable alternative operations, ensuring safety.

In addition, in the work vehicle voice operation system 1A, a voice recognition unit that recognizes the voice instructions of the operator is provided in the server 3. By having the server 3 execute the voice recognition process, which requires a large amount of calculations, the amount of calculations on the mobile terminal 2A can be reduced. In addition, calculations can be made faster.

(Embodiment 2)
In the first embodiment, the control unit 22 of the mobile terminal 2A includes the executable operation selection unit 222. However, the present invention is not limited to this. In the present invention, the controller 4A of the aerial work platform 5 may include the executable operation selection unit 222.

As described above, in the work vehicle voice operation system 1B according to the second embodiment, the controller 4B includes an executable operation selection unit 41. Below, the work vehicle voice operation system 1B will be described with reference to FIG. 8. In the second embodiment, the configuration different from the first embodiment will be described.

Figure 8 is a block diagram of the control unit 22 and controller 4B of the mobile terminal 2B provided in the work vehicle voice operation system 1B according to embodiment 2.

As shown in FIG. 8, in the work vehicle voice operation system 1B, the mobile terminal 2B includes a voice recognition result acquisition unit 221, a voice recognition result analysis unit 224, and a response generation unit 223. In addition, the controller 4B includes an executable operation selection unit 41.

As in the first embodiment, the voice recognition result acquisition unit 221 transmits the operator's voice data obtained from the microphone 21 to the voice recognition unit 311 of the server 3, and then acquires the voice recognition result from the voice recognition unit 311. The voice recognition result acquisition unit 221 then transmits the voice recognition result to the voice recognition result analysis unit 224 and the response generation unit 223.

The voice recognition result analysis unit 224 reads the command dictionary 242 from the storage unit 24, and determines whether the voice recognition result received from the voice recognition result acquisition unit 221 contains the command object and command content words in the command dictionary 242, or words similar thereto. If the voice recognition result analysis unit 224 determines that the command object and command content are included, it transmits the command object and command content to the executable operation selection unit 41 of the controller 4B. On the other hand, if the voice recognition result analysis unit 224 determines that the command object and command content are not included, it transmits an analysis result to the response generation unit 223 indicating that the voice recognition result is not related to a command.

As described in the first embodiment, the controller 4B determines the attitude or state of the aerial work platform 5 from the detection results detected by the detectors 58A-58E, and requests a restricting operation to ensure safety based on the determined attitude or state. At this time, although not described in the first embodiment, the controller 4B turns on a flag for data of an operation that corresponds to a restricting operation in order to identify whether the operation is a restricting operation or not. For example, when the jack is not installed, the controller 4B turns on a flag for data of boom operation.

When the executable operation selection unit 41 receives data on the command object and the command content from the voice recognition result analysis unit 224, it determines whether the operation specified by the command object and the command content is a restricted operation based on whether the flag is on. Then, the executable operation selection unit 41 transmits the determination result of whether the operation is a restricted operation to the response generation unit 223 of the mobile terminal 2B.

The executable operation selection unit 41 also extracts operations for which the flag is not on, i.e., the flag is off. In this way, the executable operation selection unit 41 selects executable operations at the stage of receiving data from the voice recognition result analysis unit 224. The executable operation selection unit 41 transmits data of the selected executable operation to the response generation unit 223 of the mobile terminal 2B.

When the response generation unit 223 receives an analysis result from the voice recognition result analysis unit 224 indicating that the voice recognition result is not related to a command, the response generation unit 223 reads out the response database 243 from the storage unit 24. As described in the first embodiment, the response database 243 stores responses in association with vocabulary included in the voice recognition result when the voice recognition result is not related to a command. The response generation unit 223 generates a response by referring to the response database 243, and conveys the response to the operator by outputting the data of the generated response to the speaker 23.

On the other hand, when the voice recognition result analysis unit 224 determines that the voice recognition result is related to a command, and the executable operation selection unit 41 determines whether the command is a restricted operation, the response generation unit 223 receives the determination result of whether the command is a restricted operation from the executable operation selection unit 41.

When the response generation unit 223 receives a determination result indicating that the operation is a restricted operation, it uses the executable operation data received from the executable operation selection unit 41 to create a response proposing an executable operation instead of the voice-instructed operation. Then, by outputting the generated response data to the speaker 23, it notifies the operator that there is an executable operation instead of the operation instructed by voice.

Furthermore, when the response generation unit 223 receives a determination result that the operation is not a restricted operation, it reads out the command dictionary 242 from the storage unit 24. In the command dictionary 242, as described in the first embodiment, responses to commands are associated with command targets and command contents. The response to the command is a response to the effect that the operation will be performed according to the command. The response generation unit 223 generates a response using the response to the command. The response generation unit 223 then outputs the generated response data to the speaker 23, thereby informing the operator that the operation will be performed according to the voice instructions.

As described above, in the work vehicle voice operation system 1B according to the second embodiment, similarly to the first embodiment, when the voice recognition result includes a command for operating the work vehicle and also includes a word for a restricted operation, the response generation unit 223 generates a response recommending an executable operation instead of the operation instructed by the operator's voice. Then, the response is presented to the operator. As a result, in the work vehicle voice operation system 1B, even if the operator mistakenly instructs a restricted operation by voice, the voice instruction is not executed and the operator is not confused. In addition, because executable operations in place of the voice-instructed operation are presented to the operator, voice operation is smooth and easy.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments. For example, in the first and second embodiments, the response data generated by the speaker 23 is converted into voice and the response is presented to the operator. However, the present invention is not limited to this. In the present invention, the presentation unit may present the response generated by the response generation unit 223. The mobile terminals 2A and 2B are equipped with a display device (not shown) such as a liquid crystal display or an organic EL (Electro Luminescence) display, and the display device may present the response in addition to the speaker 23. In this case, the response generated by the response generation unit 223 may be displayed as text information on the display device.

In addition, in the first and second embodiments, examples of restricted operations include boom operation when a jack is not installed, outrigger operation when the boom 54 is in a working position, and boom operation and outrigger operation when the stop switch is on, but the present invention is not limited to these. In the present invention, a restricted operation may be an operation that is restricted in a certain position or state of the aerial work vehicle 5. For example, a restricted operation may be boom operation when the boom 54 is about to exceed the working range.

In the first and second embodiments, the server 3 includes the voice recognition unit 311. However, the present invention is not limited to this. In the present invention, the control unit 22 of the mobile terminals 2A and 2B may include the voice recognition unit 311. In other words, instead of the computer included in the control unit 31 of the server 3 performing the voice recognition process, the microcomputer included in the control unit 22 of the mobile terminals 2A and 2B may perform the voice recognition process.

Although the server 3 is equipped with a voice recognition unit 311, the server 3 may be a dedicated device for performing voice recognition processing, i.e., a voice recognition device.

In the first embodiment, the control unit 22 of the mobile terminal 2A includes a voice recognition result acquisition unit 221, an executable operation selection unit 222, and a response generation unit 223. In the second embodiment, the control unit 22 of the mobile terminal 2B includes a voice recognition result acquisition unit 221, a voice recognition result analysis unit 224, and a response generation unit 223. However, the present invention is not limited to this. In the present invention, a system consisting of the mobile terminals 2A and 2B, the server 3, the aerial work vehicle 5, etc., may include the voice recognition result acquisition unit 221, the executable operation selection units 41 and 222, and the response generation unit 223. For example, the server 3 may include the voice recognition result acquisition unit 221, the executable operation selection units 41 and 222, and the response generation unit 223. In this case, the control unit 22 of the mobile terminals 2A and 2B may transmit input data from the microphone 21 to the server 3, receive response data transmitted from the server 3, and drive the speaker 23.

The mobile terminals 2A and 2B may also include a voice recognition unit 311 in addition to the voice recognition result acquisition unit 221, the executable operation selection units 41 and 222, and the response generation unit 223. Alternatively, a server separate from the server 3 may include the voice recognition result acquisition unit 221, the executable operation selection units 41 and 222, and the response generation unit 223, and the server 3 and the mobile terminals 2A and 2B may communicate with this server to carry out a series of processes from voice recognition to response generation.

Furthermore, in the first and second embodiments, the work vehicle operation database 241, the command dictionary 242, the response database 243, and the operation history database 244 are stored in the storage unit 24 of the mobile terminals 2A and 2B, but the work vehicle operation database 241, the command dictionary 242, the response database 243, and the operation history database 244 may be stored in the storage unit of the server 3 or a server other than the server 3, for example. In other words, the work vehicle operation database 241, the command dictionary 242, the response database 243, and the operation history database 244 may be stored in a storage unit connected via the network 100.

1A, 1B... Work vehicle voice operation system, 2A, 2B... Mobile terminal, 3... Server, 4A, 4B... Controller, 5... Aerial work vehicle, 21... Microphone, 22... Control unit, 23... Speaker, 24... Memory unit, 25... Communication unit, 31... Control unit, 32... Memory unit, 33... Communication unit, 41... Executable operation selection unit, 51... Vehicle, 52A-52D... Outrigger, 53... Swing base, 54... Boom, 55... Bucket, 56... Operation device, 5 7...Drive unit, 58A-58E...Detector, 61...Storage device, 62...Communication device, 71...Restricted operation information, 100...Network, 221...Voice recognition result acquisition unit, 222...Executable operation selection unit, 223...Response generation unit, 224...Voice recognition result analysis unit, 241...Work vehicle operation database, 242...Command dictionary, 243...Response database, 244...Operation history database, 311...Voice recognition unit, 321...Voice recognition dictionary

Claims (6)

a voice recognition result acquisition unit that acquires a voice recognition result from a voice recognition device that recognizes voice instructions uttered by an operator to operate the work vehicle;
an executable operation selection unit that selects an executable operation that can be executed under the attitude or state of the work vehicle based on the attitude or state detected by an attitude detector or state detector equipped in the work vehicle;
a response generation unit which determines whether the voice recognition result acquired by the voice recognition result acquisition unit includes an instruction for operating the work vehicle , and if it is determined that the instruction is not included, generates a response by referring to a response database in which responses to vocabulary other than the instruction included in the voice recognition result are stored, and if it is determined that the instruction is included, further determines whether the instruction is an instruction for a restricted operation that is restricted under the posture or state, and if it is determined that the instruction is an instruction for the restricted operation, generates a response recommending the executable operation instead of the instruction , and if it is determined that the instruction is not an instruction for the restricted operation, generates a response to the effect that the operation of the instruction will be executed ;
a presentation unit that presents the response generated by the response generation unit;
A work vehicle voice operation system equipped with: a voice recognition result acquisition unit that acquires a voice recognition result from a voice recognition device that recognizes voice instructions uttered by an operator to operate the work vehicle;
an executable operation selection unit that selects an executable operation that can be executed under the attitude or state of the work vehicle based on the attitude or state detected by an attitude detector or state detector equipped in the work vehicle;
a response generating unit which, when the voice recognition result acquired by the voice recognition result acquiring unit includes an instruction to operate the work vehicle and the instruction is an instruction for a restricted operation that is restricted under the attitude or state, generates a response recommending the executable operation instead of the instruction, and, when the voice recognition result acquired by the voice recognition result acquiring unit includes an instruction to operate the work vehicle and the instruction is not an instruction for the restricted operation, generates a response to execute the operation of the instruction;
a presentation unit that presents the response generated by the response generation unit;
a storage unit that stores the operation history by storing the operation content of the voice instruction every time the response generation unit generates a response to execute the operation instructed by the operator by voice ;
Equipped with
the response generation unit, when the voice recognition result acquired by the voice recognition result acquisition unit includes an instruction to operate the work vehicle, the instruction is an instruction to perform the regulated operation, and there are a plurality of executable operations selected by the executable operation selection unit, generates a response recommending, from among the plurality of executable operations, an executable operation that corresponds to a most recent operation or a most frequently performed operation in the history data stored in the storage unit.
Work vehicle voice operation system. a voice recognition step in which the first computer or the second computer recognizes voice instructions uttered by an operator to operate the work vehicle;
an executable operation selection step in which the second computer selects an executable operation that can be executed under the attitude or state of the work vehicle based on the attitude or state detected by an attitude detector or state detector equipped in the work vehicle;
a response generating step in which the second computer determines whether or not the voice recognition result obtained by voice recognition in the voice recognition step includes an instruction to operate the work vehicle, and if it determines that the instruction is not included, generates a response by referring to a response database in which responses to vocabulary other than the instruction included in the voice recognition result are stored, and if it determines that the instruction is included, further determines whether or not the instruction is an instruction for a restricted operation that is restricted under the attitude or state, and if it determines that the instruction is an instruction for the restricted operation , generates a response recommending the executable operation instead of the instruction , and if it determines that the instruction is not an instruction for the restricted operation, generates a response to the effect that the operation of the instruction is to be executed ;
A voice operation method for a work vehicle comprising: A program for voice-operating a work vehicle using a microphone that converts voice instructions uttered by an operator to operate the work vehicle into electrical signals and outputs the electrical signals as voice data, and a voice recognition device that performs voice recognition on the voice data, comprising:
On the computer,
a speech recognition result acquisition step of acquiring a speech recognition result from the speech recognition device;
an executable operation selection step of selecting an executable operation that can be executed under the attitude or state of the work vehicle based on the attitude or state detected by an attitude detector or state detector equipped in the work vehicle;
a response generating step of determining whether the voice recognition result acquired in the voice recognition result acquisition step includes an instruction for operating the work vehicle , and if it is determined that the instruction is not included, generating a response by referring to a response database in which responses to vocabulary other than the instruction included in the voice recognition result are stored, and if it is determined that the instruction is included, further determining whether the instruction is an instruction for a restricted operation that is restricted under the posture or state, and if it is determined that the instruction is an instruction for the restricted operation, generating a response recommending the executable operation instead of the instruction , and if it is determined that the instruction is not an instruction for the restricted operation, generating a response to execute the operation of the instruction ;
A program for executing. a voice recognition step in which the first computer or the second computer recognizes voice instructions uttered by an operator to operate the work vehicle;
an executable operation selection step in which the second computer selects an executable operation that can be executed under the attitude or state of the work vehicle based on the attitude or state detected by an attitude detector or state detector equipped in the work vehicle;
a response generating step in which, when the voice recognition result obtained by voice recognition in the voice recognition step includes an instruction to operate the work vehicle and the instruction is an instruction for a restricted operation that is restricted under the attitude or the state, the second computer generates a response recommending the executable operation instead of the instruction, and, when the voice recognition result obtained by voice recognition in the voice recognition step includes an instruction to operate the work vehicle and the instruction is not an instruction for the restricted operation, the second computer generates a response to execute the operation of the instruction;
a storage step of storing a history of operations by the second computer by storing the operation content of the voice instruction in a storage device every time the response generating step generates a response to the effect that the operation instructed by the operator by voice is to be executed;
Equipped with
In the response generation step, when the voice recognition result obtained by voice recognition in the voice recognition step includes an instruction to operate the work vehicle, and the instruction is an instruction to perform the restricted operation, and there are a plurality of executable operations selected in the executable operation selection step, a response is generated that recommends the executable operation that corresponds to the most recent operation or the most frequently performed operation in the history data stored in the storage device out of the plurality of executable operations.
How to operate a work vehicle with voice commands. A program for voice-operating a work vehicle using a microphone that converts voice instructions uttered by an operator to operate the work vehicle into an electrical signal and outputs the electrical signal as voice data, a voice recognition device that performs voice recognition on the voice data, and a storage device, comprising:
On the computer,
a speech recognition result acquisition step of acquiring a speech recognition result from the speech recognition device;
an executable operation selection step of selecting an executable operation that can be executed under the attitude or state of the work vehicle based on the attitude or state detected by an attitude detector or state detector equipped in the work vehicle;
a response generating step of generating a response recommending the executable operation in place of the instruction when the voice recognition result acquired in the voice recognition result acquiring step includes an instruction to operate the work vehicle and the instruction is an instruction for a restricted operation that is restricted under the attitude or state, and generating a response to execute the operation of the instruction when the voice recognition result acquired in the voice recognition result acquiring step includes an instruction to operate the work vehicle and the instruction is not an instruction for the restricted operation;
a storage step of storing a history of operations by storing the operation contents of the voice instruction in the storage device every time a response to execute an operation instructed by the operator by voice is generated in the response generating step;
Run the command,
In the response generating step, when the voice recognition result acquired in the voice recognition result acquiring step includes an instruction to operate the work vehicle, the instruction is an instruction to perform the restricted operation, and there are a plurality of executable operations selected in the executable operation selecting step, a response is generated that recommends the executable operation that corresponds to the most recent operation or the most frequently performed operation in the history data stored in the storage device out of the plurality of executable operations.
Program.

Images