JP7627485B2 - Input Devices - Google Patents

Description

Article 30, paragraph 2 of the Patent Act applies. 24th Symposium of the Information Processing Society of Japan (Interaction 2020) Held March 9-11, 2020 [Publications, etc.] The 38th Annual Meeting of the Robotics Society of Japan (RSJ2020) Online Proceedings Published October 8, 2020 [Publications, etc.] The 38th Annual Meeting of the Robotics Society of Japan (RSJ2020) Held October 9-11, 2020

This invention relates to an input device, and in particular to an input device that inputs operational data based on cheek movements into a computer.

One example of this type of conventional input device is disclosed in Patent Document 1. The gaze detection device disclosed in Patent Document 1 includes two first cameras for measuring the position of the pupil relative to a coordinate system, a light source for forming a corneal reflection point, and a second camera for acquiring data on the distance between the center of the pupil and the corneal reflection point and a predetermined angle of this distance relative to the coordinate axes of the coordinate system, and is equipped with a calculation means for calculating the gaze direction from information from each camera. In this gaze detection device, in the relational equation determination stage, the subject is asked to look at a known point, measurements are taken, and the relational equation is determined. In the gaze determination stage, the subject is measured again, and the gaze is determined using the relational equation. Patent Document 1 also discloses that this gaze detection device can be used as a gaze input means.

Patent No. 4517049

The gaze detection device disclosed in the above-mentioned Patent Document 1 has the problem that when determining the gaze, the subject or user must focus on a known point that they wish to input, which makes the operation cumbersome and makes it relatively difficult to input accurately. In addition, when confirming the input, the user must focus on the known point for a certain amount of time, which also makes it difficult to input quickly.

Therefore, the main object of this invention is to provide a novel input device.

Another object of the present invention is to provide an input device that is easy to operate and allows accurate and fast input.

A first invention is an input device comprising a deformable sheet-like covering portion covering at least a portion of a user's cheek, a detection portion attached to the side of the covering portion that comes into contact with the user's face and capable of detecting at least the movement of the user's left cheek and the movement of the right cheek, and a generation portion that generates operation data based on the movement of the user's left cheek and/or the movement of the right cheek detected by the detection portion.

The second invention is dependent on the first invention, and further includes a transmission unit that transmits the operation data generated by the generation unit to an external computer.

A third invention is according to the first or second invention, and the detection unit is two capacitive touch sensors, which are arranged on the left and right sides of the covering unit, respectively , and detect the movement of the user's left cheek and/or right cheek.

A fourth invention is according to the first or second invention, and the detection unit includes four capacitive touch sensors, the four capacitive sensors being respectively arranged on the top, bottom, left and right of the covering unit and detecting at least one of movements of the user's left cheek, right cheek, philtrum and jaw, and the generation unit generates at least one of operation data based on the movement of the user's left cheek, operation data based on the movement of the user's right cheek, operation data based on the movement of the user's philtrum and operation data based on the movement of the user's jaw detected by the four capacitive sensors.

The fifth invention is dependent on the third or fourth invention, and the capacitive touch sensor includes a conductive cloth formed by overlapping a first cloth formed by knitting conductive thread with a second cloth formed by knitting insulating thread, an insulating sheet having the same size as the first cloth and overlapping and affixed to the first cloth, and a capacitive sensor IC electrically connected to the first cloth.

The sixth invention is dependent on the fifth invention, and the conductive cloth has elasticity.

According to this invention, a user wearing a mask equipped with a touch detection unit can perform operations similar to those of operating a computer mouse button by moving their cheek. This makes the operation simple and allows characters to be input accurately and quickly.

The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the embodiments with reference to the drawings.

FIG. 1 is a block diagram showing the electrical configuration of an input device according to a first embodiment of the present invention. FIG. 2 is a block diagram showing the electrical configuration of the cheek movement detection device shown in FIG. FIG. 3 is a diagram showing the outline of the external configuration of the mask type detection unit. FIG. 4 is a diagram for explaining an outline of the configuration of the touch detection unit shown in FIG. 5(A) is a partial enlarged view of the conductive surface side of the conductive cloth constituting the touch detection unit shown in FIG. 4, and FIG. 5(B) is a partial enlarged view of the non-conductive surface side of the conductive cloth constituting the touch detection unit shown in FIG. 4. FIG. 6 is a partial enlarged view of the conductive cloth constituting the touch detection unit shown in FIG. 7(A) is a diagram showing an example of a state in which a user is not touching the touch detection unit of the cheek movement detection device shown in FIG. 1, and FIG. 7(B) is a diagram showing an example of a state in which a user is touching the touch detection unit of the cheek movement detection device shown in FIG. 1. FIG. 8 is a diagram showing an example of a memory map of a RAM built in the computer shown in FIG. FIG. 9 is a flow chart showing an example of the operation detection and operation data generation processing of the CPU built in the computer shown in FIG. FIG. 10 is a block diagram showing the electrical configuration of a character input system using the input device shown in FIG. FIG. 11 is a flow chart showing an example of a character input process of a CPU built in the user terminal shown in FIG. FIG. 12 is a block diagram showing the electrical configuration of the system according to the second embodiment. FIG. 13 is a diagram showing a schematic external configuration of the mask-type detection unit included in the cheek movement detection device shown in FIG. FIG. 14 is a block diagram showing an example of the electrical configuration of the wheelchair shown in FIG. FIG. 15 is a flowchart showing a part of an example of a wheelchair control process by a CPU built in the computer shown in FIG. FIG. 16 is a flow chart showing another part of the wheelchair control process by the CPU built in the computer shown in FIG. 14, which follows FIG. FIG. 17 is a flow chart showing another part of the wheelchair control process by the CPU built in the computer shown in FIG. 14, which follows FIG. 16. FIG. 18 is a flow chart showing an example of mouse operation processing by the CPU built in the user terminal of the third embodiment.

[First Example]
1, an input device 10 of the first embodiment includes a cheek motion detection device 12, which is communicatively connected to a computer 14. The cheek motion detection device 12 will be described in detail later.

The computer 14 is a general-purpose server or PC, and includes components such as a CPU 14a, a RAM 14b, and a communication device 14c. The computer 14 also includes other storage units configured with non-volatile memory such as an HDD, flash memory, or EEPROM, or semiconductor memory such as an SSD.

The CPU 14a is a processor that is responsible for the overall control of the computer 14. The RAM 14b is the main storage device of the computer 14, and functions as a buffer area and a work area for the CPU 14a. The communication device 14c is a communication module for communicating with the cheek movement detection device 12 or an external computer via a wired or wireless connection according to a communication method such as Ethernet or Wi-Fi.

Figure 2 is a block diagram showing the electrical configuration of the cheek movement detection device 12. The cheek movement detection device 12 includes a mask type detection unit 120 and a controller 130, and a touch detection unit 124 (described later) of the mask type detection unit 120 is electrically connected to the controller 130.

Figure 3 is a diagram showing an outline of the structure of the mask-type detection unit 120. The mask-type detection unit 120 includes a general-purpose mask 122, and touch detection units 124 are provided at the left and right ends of the mask 122, respectively.

In addition, in FIG. 3, a part of the user's face covered by the mask 122 is also shown with a solid line to clearly show the relative positions of the user's left and right cheeks and the two touch detection units 124. Also, in FIG. 3, a state in which the user does not have the strings of the mask 122 hanging from the ears is shown to clearly show the positions of the two touch detection units 124 arranged on the mask 122. Furthermore, in FIG. 3, the touch detection units 124 are painted gray. These are the same as in FIG. 13 of the second embodiment.

As an example, the mask 122 is a gauze-type, nonwoven fabric-type, or urethane-type household mask made of a deformable material, and the sheet-like covering portion covers part of the user's face. The part of the user's face specifically means all or part of the mouth, nose, chin, and each of the left and right cheeks.

In this first embodiment, the mask 122 used had a covering portion measuring 9 cm in length and 17 cm in width.

However, a medical mask (i.e., a surgical mask) can also be used as the mask 122. Also, the mask 122 does not need to be limited to a household mask or a medical mask, and a dedicated mask can be created.

As described below, in the first embodiment, it is sufficient to be able to detect the movement of the user's cheek, so it is sufficient for the mask 122 to cover at least a part of the user's cheek. Also, it is sufficient to be able to detect the movement of the user's cheek, and it is sufficient for the user to be able to visually confirm the content and position of input using the input device 10, so the mask 122 may cover the entire head except for the eyes.

The two touch detection units 124 are each formed in a strip shape and, as an example, fixed in a vertical orientation between the outer and inner (i.e., the user side wearing the mask 122) cloth of the layered mask 122 (i.e., the covering portion). However, the touch detection unit 124 may also be fixed to the surface of the outer or inner cloth of the mask 122. As described below, since the touch detection unit 124 is made of cloth, it can be sewn to the mask 122.

In addition, in the first embodiment, when viewed from the side of a user wearing the mask-type detection unit 120 (or mask 122), the touch detection unit 124a is attached to the left end of the fabric of the mask 122, and the touch detection unit 124b is attached to the right end of the fabric of the mask 122.

In the following, in this specification, when there is no need to particularly distinguish between touch detection unit 124a and touch detection unit 124b, they will simply be referred to as "touch detection unit 124."

In this first embodiment, the user wears the above-mentioned mask-type detection unit 120 and moves either the left cheek, the right cheek, or both, and the movement is detected by touch detection unit 124a and/or touch detection unit 124b. The user can inflate their cheek with air or press their cheek from the inside to the outside with their tongue to touch or press their cheek against touch detection unit 124 (hereinafter simply referred to as a "pressing action"). Also, in this specification, the user can perform a clicking action by performing a pressing action and then stopping the pressing action (i.e., moving the cheek away from mask 122).

In other words, by moving their cheeks, the user can perform operations similar to pressing the left and right buttons of a computer mouse, and then pressing and releasing the left and right buttons.

Figure 4 is a diagram showing an outline of the structure of the touch detection unit 124 shown in Figure 3. As shown in Figure 4, the touch detection unit 124 has a layered structure and includes a conductive cloth 200 and an insulating tape 220. Furthermore, the conductive cloth 200 is formed by bonding a conductive surface side cloth 200a (corresponding to the "first cloth") and a non-conductive surface side cloth 200b (corresponding to the "second cloth"). The insulating tape 220 is attached to the conductive surface side cloth 200a. Although not shown, the controller 130 shown in Figure 3 is connected by electric wires to the conductive surface side cloth 200a of each of the touch detection units 124a and 124b.

5(A) is a partial enlarged view of the conductive side cloth 200a, and FIG. 5(B) is a partial enlarged view of the non-conductive side cloth 200b. Also, FIG. 6 is a partial enlarged view of the conductive cloth 200 constituting the touch detection unit 124 of the cheek motion detection device 12. With reference to FIG. 5(A), FIG. 5(B) and FIG. 6, the conductive side cloth 200a, the non-conductive side cloth 200b and the conductive cloth 200 of this first embodiment will be specifically described. However, in FIG. 5(A), FIG. 5(B) and FIG. 6, the non-conductive side cloth 200b is shown by a hollow line in order to easily show the conductive side cloth 200a and the non-conductive side cloth 200b. Also, in FIG. 6, the insulating thread 200c is shown by a dotted line.

The conductive surface side cloth 200a is made by circular knitting conductive yarn. In this first embodiment, the conductive yarn is silver-plated nylon yarn. The non-conductive surface side cloth 200b is made by circular knitting insulating yarn. In this first embodiment, the insulating yarn is polyester yarn. In this way, the conductive surface side cloth 200a and the non-conductive surface side cloth 200b that make up the conductive cloth are both made by circular knitting, and therefore have high elasticity.

As shown in FIG. 6, the conductive cloth 200 is made by knitting the conductive side cloth 200a and the non-conductive side cloth 200b, and knitting an insulating thread 200c such as polyester thread between them, so that the conductive side cloth 200a and the non-conductive side cloth 200b are knit together. Therefore, the conductive cloth 200 has a three-layer structure consisting of the conductive side cloth 200a, the non-conductive side cloth 200b, and the insulating thread 200c between them. However, in FIG. 4, the layer of insulating thread 200c for knitting the conductive side cloth 200a and the non-conductive side cloth 200b is omitted.

In this first embodiment, the conductive cloth 200 is made by bonded jersey knitting, but it is also possible to make two pieces of cloth, the conductive side cloth 200a and the non-conductive side cloth 200b, separately and then attach them together with insulating thread 200c so that their surfaces meet.

In addition, in this first embodiment, in order to give the conductive cloth 200 elasticity, the conductive surface side cloth 200a and the non-conductive surface side cloth 200b are made by circular knitting, but this does not have to be limited to this. The conductive surface side cloth 200a and the non-conductive surface side cloth 200b may also be made by weft knitting or warp knitting. In such cases, the conductive cloth 200 can also be made elastic.

The insulating tape 220 is an insulating sheet with an adhesive applied to one side. The insulating sheet is made of nylon. As an example of the insulating tape 220, "Nylon Repair Sheet Seal Type" manufactured by KAWAGUCHI Corporation can be used.

The controller 130 is also called a capacitance sensor IC, and detects the capacitance generated between the conductive surface side cloth 200a and a conductive object at ground potential. A DC voltage is supplied to the controller 130 from a power source, and the controller 130 uses this voltage to detect the capacitance. In simple terms, the controller 130 detects the capacitance every predetermined period (e.g., 5 seconds), applies a predetermined DC voltage to the conductive surface side cloth 200a during the first period of the predetermined period (e.g., 2.5 seconds), and calculates the capacitance by detecting the voltage value when the conductive surface side cloth 200a discharges during the remaining period of the predetermined period. As an example of the controller 130 having the function of calculating the capacitance, a microcontroller (model number: PIC16F1847) manufactured by Microchip Technology Japan Co., Ltd. can be used.

However, in this first embodiment, the cloth 200a on the conductive surface side of the two touch detection units 124 is connected to different ports of the capacitance sensor IC using wires covered with an insulating resin such as polyvinyl chloride. Therefore, the capacitance generated between the cloth 200a on the conductive surface side of each touch detection unit 124 and a conductor at ground potential is detected individually.

The cheek motion detection device 12 configured as described above can function as a touch sensor. FIG. 7(A) shows an example of a state in which the touch detection unit 124 is not touched, and FIG. 7(B) shows an example of a state in which the touch detection unit 124 is touched. However, the controller 130 is omitted in FIG. 7(A) and FIG. 7(B). Also, for convenience of explanation, the user's fingers are shown in FIG. 7(A) and FIG. 7(B), but in reality, the touch detection unit 124 is touched by the user's cheek. Also, for convenience of explanation, the inner layer of fabric of the mask 122 is omitted in FIG. 7(A) and FIG. 7(B).

As shown in FIG. 7B, when a user at ground potential touches the insulating tape 220 of the touch detection unit 124, a capacitance C is generated between the conductive surface side of the cloth 200a (touch detection unit 124) and the user's finger.

Therefore, in the first embodiment, the touch detection unit 124 is fixed to the mask 122 so that the insulating tape 220 faces the inside (user side) of the mask 122 and the non-conductive cloth 20b faces the outside.

The capacitance C can be calculated using the following formula (Formula 1). In the formula, ε is the dielectric constant, which is determined by the material of the insulating tape 220 in the first embodiment. D is the distance between the touch detection unit 124 and the user's hand. S is the area of the touch detection unit 124, which is 1.5 cm x 4 cm in the first embodiment.

[Equation 1]
C = ε × S/D
In this manner, when the capacitance C occurs, it is detected that a conductive body such as a user is touching the touch detection unit 124 .

Therefore, in this first embodiment, the cheek movement of a user wearing the mask-type detection unit 120 is detected by a change in the capacitance C of the touch detection unit 124. However, when the user is wearing the mask-type detection unit 120, the distance D between the user's cheek and the touch detection unit 124 is about a few mm, so some capacitance C occurs even when the user does not move his or her cheek, i.e., when the user's cheek does not contact (touch) the touch detection unit 124. Therefore, the output of the controller 130 is calibrated to be zero or nearly zero when the user is wearing the mask 122 and the user's cheek is not in contact with the touch detection unit 124 (the cloth inside the mask 122).

As described above, since capacitance C occurs even when the user's cheek does not touch the touch detection unit 124, the computer 14 (CPU 14a) is configured to determine that the user's cheek has touched the touch detection unit 124, i.e., a pressing operation, when the output of the controller 130 (hereinafter sometimes referred to as the "sensor value") has changed by a predetermined threshold value or more. When the computer 14 (CPU 14a) determines that a pressing operation has occurred, it generates operation data for the pressing operation.

In the experiment, when the user's cheek touched the touch detection unit 124, the capacitance changed by about 300 F compared to before the user's cheek touched the touch detection unit 124, so the predetermined threshold was set between 150 F and 250 F.

In addition, when the computer 14 (CPU 14a) detects that the pressing action has stopped, if a pressing action was performed prior to the first predetermined time (100 msec in the first embodiment), it determines that a click action was performed and generates operation data for the click action.

When the computer 14 (CPU 14a) generates operation data, it transmits (or inputs) the generated operation data to the external computer to which the input device 10 is connected.

In this first embodiment, the length of time that the pressing action is performed is not relevant to determining whether it is a click action, but if the pressing action is relatively long (for example, 1 sec or longer), it may be determined to be a long press action rather than a click action.

Figure 8 shows an example of a memory map 300 of RAM 14b of computer 14 shown in Figure 1. As shown in Figure 8, RAM 14b includes a program memory area 302 and a data memory area 304.

The program memory area 302 stores the programs of the input device 10 (i.e., information processing programs), and the information processing programs include a communication program 302a, a sensor value detection program 302b, and an operation data generation program 302c, etc.

The communication program 302a is a program for communicating with the controller 130 and an external computer. The sensor value detection program 302b is a program for detecting a sensor value based on the output of the touch detection unit 124 transmitted from the controller 130. However, the sensor value based on the output of the touch detection unit 124a and the sensor value based on the output of the touch detection unit 124b are made distinguishable.

The operation data generation program 302c is a program for generating operation data based on the sensor values detected according to the sensor value detection program 302b. However, the operation data generation program 302c generates operation data for the left cheek movement based on the sensor value of the touch detection unit 124a, and generates operation data for the right cheek movement based on the sensor value of the touch detection unit 124b.

The data memory area 304 stores sensor value data 304a, operation data 304b, a pressing operation flag 304c, and a no-operation flag 304d.

The sensor value data 304a is data on the sensor values transmitted from the controller 130, and is stored so that the sensor values based on the output of the touch detection unit 124a and the sensor values based on the output of the touch detection unit 124b can be distinguished from each other. The sensor value data 304a is also stored in chronological order, and is deleted once it has been used by the CPU 14a in the process of generating operation data.

Operation data 304b is operation data generated based on sensor value data 304a, and is stored in an identifiable manner for each of touch detection units 124a and 124b.

The pressing action flag 304c is a flag for determining whether the action of the user's cheek is a pressing action. The pressing action flag 304c is turned on when a pressing action is detected, and is turned off when no action or a clicking action is detected.

The no-motion flag 304d is a flag for determining whether the user's cheek movement is no movement. The no-motion flag 304d is turned on when no movement is detected, and is turned off when a pressing movement is detected.

However, the pressing action flag 304c and the no-action flag 304d are set so as to be identifiable for each of the left and right cheeks, i.e., for each of the touch detection units 124a and 124b.

Figure 9 is a flow diagram of the motion detection and operation data generation process of the CPU 14a built into the computer 14. The motion detection and operation data generation process shown in Figure 9 is executed separately for each of the sensor values based on the output of the left touch detection unit 124a and the right touch detection unit 124b. That is, the motion of the left cheek is detected, and operation data corresponding to the left cheek motion (hereinafter sometimes referred to as "left cheek operation data") is generated, and the motion of the right cheek is detected, and operation data corresponding to the right cheek motion (hereinafter sometimes referred to as "right cheek operation data") is generated.

When the CPU 14a receives an instruction from a user or an instruction from an external computer connected for communication, it starts a process of detecting motion and generating operation data as shown in FIG. 9, and in step S1, it determines whether the sensor value based on the output of the touch detection unit 124 is equal to or greater than a predetermined threshold value (e.g., 150).

If step S1 is "YES", that is, if the sensor value is equal to or greater than the predetermined threshold, step S3 turns on the pressing action flag 304c, step S5 generates operation data for the pressing action, and step S7 turns off the no-action flag 304d and returns to step S1.

However, in step S5, when operation data is generated, the corresponding operation data 304b is stored in RAM 14b and input (or transmitted) to an external computer. This also applies to step S17, which will be described later.

If the pressing action flag 304c is already on, the process of step S3 is skipped. Similarly, if the no-action flag 304d is already off, the process of step S7 is skipped. The same applies to the cases where other flags are on or off.

If step S1 is "NO", that is, if the sensor value is less than the predetermined threshold, step S9 determines whether the pressing action flag 304c was on a first predetermined time period (e.g., 500 msec) ago. Although not shown, when the pressing action flag 304c is changed from on to off, the CPU 14a starts counting a timer provided in the RAM 14b, and determines whether the count value is equal to or less than the first predetermined time period.

If the answer is "NO" in step S9, that is, if the pressing action flag 304c from the first predetermined time ago is off, the no-action flag 304d is turned on in step S11, and the pressing action flag 304c is turned off in step S13, and the process returns to step S1.

On the other hand, if step S9 is "YES", that is, if the pressing action flag 304c from the first predetermined time ago is on, operation data for a click action is generated in step S15, and then the pressing action flag 304c is turned off in step S17, and the process returns to step S1.

The input device 10 of the present invention shown in FIG. 1 can be used as an input device for an external computer. As an example, FIG. 10 is a block diagram showing the electrical configuration of a system 400 in which the input device 10 is applied to an input system using an eye-gaze input device.

As shown in FIG. 10, the system 400 is composed of an input device 10, a user terminal 20, and a gaze input device 22. The user terminal 20 and the gaze input device 22 are communicatively connected to a computer 14.

The user terminal 20 is a general-purpose desktop PC or tablet terminal, and is equipped with components such as a CPU 20a, memory (RAM, ROM, HDD), a communication device, and a display.

The gaze input device 22 captures the user's face with a camera and calculates (or estimates) the user's gaze direction from the captured image. However, the position or area of the display surface relative to the gaze direction can be determined based on the relative positions of the user's face, the camera, and the display of the user terminal 20.

The gaze input device 22 is well known, and may be the gaze direction estimation device disclosed in Japanese Patent Application Laid-Open No. 2014-194617 or Japanese Patent Application Laid-Open No. 2012-216180, which have been filed by the applicants and have already been published. However, the computer 14 may also function as the computer body included in the gaze direction estimation device.

If the user terminal 20 is a tablet terminal, the camera of the tablet terminal can be used as the camera of the gaze input device 22.

The computer 14 integrates the detection result data (hereinafter referred to as "sensor value") input from the cheek movement detection device 12 and the gaze direction data input from the gaze input device 22, and inputs it to an external computer (here, the user terminal 20). The computer 14 synchronizes the detection result data with the gaze direction data. For this reason, the gaze input device 22 is connected to the computer 14.

The user terminal 20 receives operation data and/or directional input data input from the input device 10 and executes a predetermined process. Here, a case will be described in which the user terminal 20 executes a predetermined application for character input. However, since only the operation data of the click action is used in the character input process shown in FIG. 11, even if the user terminal 20 receives the operation data of the press action, it does not use it for the character input process. For this reason, the input device 10 may not transmit the operation data of the press action to the user terminal 20.

Figure 11 is a flow diagram of the character input process of the CPU 20a of the user terminal 20 shown in Figure 10. When the cheek movement and gaze-based character input function is executed, the character input process shown in Figure 11 is executed. Although not shown, a predetermined application for entering characters for document creation or e-mail, etc. is also executed at the same time in the user terminal 20.

As shown in FIG. 11, when the CPU 20a of the user terminal 20 starts character input processing, in step S31, a software keyboard is displayed on the display of the user terminal 20. In the following step S33, the mouse cursor is moved according to the movement of the line of sight. However, if the line of sight does not move, the mouse cursor does not move either.

In the next step S35, it is determined whether or not a left-side click action has been performed. Here, the CPU 20a determines whether or not operation data for a left cheek click action, i.e., operation data for a click action based on the output of the touch detection unit 124a, has been input from the computer 14. The same applies when determining whether or not there is another click action.

If step S35 is "NO," that is, if there is no left-click action, proceed to step S45. On the other hand, if step S35 is "YES," that is, if there is a left-click action, then in step S37, it is determined whether the mouse cursor is pointing to any key on the software keyboard.

If the answer is "NO" in step S37, that is, if the mouse cursor is not pointing to any key on the software keyboard, then in step S39 it is determined whether the mouse cursor is pointing to the end button of the specified application.

If step S39 is "NO", that is, if the mouse cursor is not pointing to the end button of the above-mentioned specified application, the process returns to step S33. On the other hand, if step S39 is "YES", that is, if the mouse cursor is pointing to the end button of the above-mentioned specified application, the process instructs the user to end the specified application in step S41, and ends the character input process.

Also, if step S37 is "YES", that is, if the mouse cursor is pointing to any key on the software keyboard, then in step S43, the character corresponding to the pointed-to key is input, and the process returns to step S33. That is, in step S43, the CPU 20a instructs a specific application to input the character corresponding to the pointed-to key. Therefore, in the specific application, the pointed-to character is input at the position where it should be input.

In addition, in step S45, it is determined whether a right-side click action has been performed. Here, the CPU 20a determines whether operation data for a right cheek click action has been input from the computer 14.

If step S45 is "NO", that is, if there is no right-side click, the process returns to step S33. On the other hand, if step S45 is "YES", that is, if there is a right-side click, the process deletes the previously entered character (deletes one character) in step S47, and the process returns to step S33.

According to this first embodiment, a user wearing a mask equipped with a touch detection unit can perform an operation similar to operating a button on a computer mouse by moving his/her cheek. This simplifies the operation and enables accurate and fast character input. Therefore, the user can shorten the time spent gazing at the character to be input, compared to inputting characters only by eye gaze input.

In addition, according to the first embodiment, the touch detection unit is composed of conductive cloth and insulating tape, so the cheek motion detection device, i.e., the sensor, is easy to manufacture. In addition, the conductive cloth is made of bonded jersey knit, so it is highly durable.

Furthermore, according to this first embodiment, the mask and the touch detection unit are made of cloth and can be washed. Therefore, they can be kept clean even after repeated use.

In this first embodiment, we have described the case of entering characters, but when another application is running, you can execute a command or cancel the execution of a command by clicking.

In addition, in this first embodiment, either the left side operation data or the right side operation data is received to execute the character input and single character deletion processes, but if both the left side operation data and the right side operation data are received simultaneously, another command (for example, a line break) may be executed. In this case, the user puffs out both cheeks at the same time to touch the two touch detection sections at the same time.

Furthermore, in this first embodiment, two touch detection units are provided, but one touch detection unit is sufficient to function as an input device, and three or more touch detection units may be provided if the user can perform the operation intended.

Furthermore, in this first embodiment, a touch detection unit made of cloth is provided, but other sensors made of sheet-like soft materials can also be used. As an example, a thin tactile sensor (product name "Shock Cube (TM)" manufactured by Touchence Inc. can be used.

[Second embodiment]
The second embodiment is a system 500 in which the input device 10 of the present invention is used to control an electric wheelchair 30. The system 500 will be described below, but since the input device 10 has been described in the first embodiment, a duplicated description will be omitted.

Figure 12 is a block diagram showing the electrical configuration of the system 500 of the second embodiment. As shown in Figure 12, the system 500 includes an input device 10 and a wheelchair 30, and the wheelchair 30 is electrically connected to a computer 14.

In the input device 10 of the second embodiment, the mask-type detection unit 120 has four touch detection units 124. In addition to the left touch detection unit 124a and the right touch detection unit 124b described in the first embodiment, an upper touch detection unit 124c and a lower touch detection unit 124d are provided.

As shown in FIG. 13, the upper touch detection unit 124c is positioned upward from the center of the fabric of the mask 122, and the lower touch detection unit 124d is positioned downward from the center of the fabric of the mask 122.

The shape and material of the touch detection units 124c and 124d are the same as those of the touch detection units 124a and 124b. Furthermore, the touch detection units 124c and 124d are fixed in a landscape orientation between the outer and inner fabrics of the layered mask 122.

The user can bring the philtrum into contact with the touch detection unit 124c by inflating the area between the upper lip and nose with air or pushing it out with the tongue. In other words, the upper touch detection unit 124c detects pressing and clicking actions on the upper side.

The user can also bring part of the chin into contact with the touch detection unit 124d by inflating the area between the lower lip and the chin with air or by pushing out the area with the tongue. In other words, the lower touch detection unit 124d detects pressing and clicking actions on the lower side.

Strictly speaking, movements of the philtrum and parts of the jaw are not considered cheek movements, but for the purposes of this specification, these movements are included in cheek movements.

Furthermore, in the following description, when there is no need to distinguish between the touch detection units 124a-124d, they will simply be referred to as "touch detection unit 124."

Figure 14 is a block diagram showing an example of the electrical configuration of the wheelchair 30. As shown in Figure 14, the wheelchair 30 includes a computer 32, which is provided with an input/output interface (hereinafter simply referred to as "interface") 34, motor drivers 36a, 36b, motors 38a, 38b, and encoders 40a, 40b.

The computer 32, interface 34 and motor drivers 36a, 36b are housed in a box that is located under or behind the seat of the wheelchair 30.

The computer 32 is a general-purpose PC and includes components such as a CPU 32a, memory (RAM, ROM, and HDD), and a communication device. This computer 32 is connected to the computer 14 of the input device 10 so as to be able to communicate with it.

The computer 32 is also connected to an interface 34. This interface 34 is connected to a left motor 38a via a motor driver 36a, and to a right motor 38b via a motor driver 36b. The interface 34 is also connected to encoders 40a and 40b.

Although not shown in the figure, the rotation shaft of the left motor 38a is connected to the rotation shaft of the left rear wheel of the wheelchair 30 using gears, and the rotation shaft of the right motor 38b is connected to the rotation shaft of the right rear wheel of the wheelchair 30 using gears.

The computer 32 controls the driving of the left motor 38a and the right motor 38b in response to the operation data from the input device 10, and controls the movement of the wheelchair 30. As an example, when an upward click motion is detected by the touch detection unit 124c and operation data corresponding to the upward click motion is input, the computer 32 rotates the left motor 38a and the right motor 38b at a first predetermined speed, and moves the wheelchair 30 forward. When a downward click motion is detected by the touch detection unit 124d and operation data corresponding to the click motion is input, the computer 32 stops the left motor 38a and the right motor 38b, and stops the wheelchair 30.

When the wheelchair 30 is moving forward, if the touch detection unit 124a detects a left-side click action and inputs operation data corresponding to the click action, the computer 32 increases the rotation speed of the right motor 38b from a first predetermined speed to a second predetermined speed, causing the wheelchair 30 to turn left. When the wheelchair 30 is moving forward, if the touch detection unit 124b detects a right-side click action and inputs operation data corresponding to the click action, the computer 32 increases the rotation speed of the left motor 38a from a first predetermined speed to a second predetermined speed, causing the wheelchair 30 to turn right.

Furthermore, when the wheelchair 30 is stopped, if the touch detection unit 124a detects a left-side click operation and inputs operation data corresponding to the click operation, the computer 32 rotates the right motor 38b at a third predetermined speed with the left motor 38a stopped, thereby turning the wheelchair 30 to the left. However, the third predetermined speed is set to be equal to or lower than the first predetermined speed. Also, when the wheelchair 30 is stopped, if the touch detection unit 124b detects a right-side click operation and inputs operation data corresponding to the click operation, the computer 32 rotates the left motor 38a at the third predetermined speed with the right motor 38b stopped, thereby turning the wheelchair 30 to the right.

In addition, when the touch detection unit 124d detects a downward click action and operation data corresponding to the click action is detected a predetermined number of times within a second predetermined time, the computer 32 ends the control process of the wheelchair 30.

Specifically, the CPU 32a built into the computer 32 of the wheelchair 30 executes the wheelchair control process shown in Figures 15 to 17. The flowcharts shown in Figures 15 to 17 are as follows.

The processing of the input device 10 is the same as in the first embodiment, so a duplicated description will be omitted. However, in the second embodiment, the mask-type detection unit 120 has four touch detection units 124, so the motion detection and operation data generation processing shown in FIG. 9 is executed for each of the four touch detection units 124, and operation data based on the detection results of each of the top, bottom, left, and right touch detection units 124 is input to the computer 32 in an identifiable manner.

15 to 17, only the operation data of the click action is used, so even if the user terminal 20 receives the operation data of the press action, the user terminal 20 does not use the operation data for the wheelchair control process. Therefore, the input device 10 may not transmit the operation data of the press action to the user terminal 20.

When the main power supply of the wheelchair 30 is turned on, as shown in FIG. 15, the CPU 32a starts the wheelchair control process, and in step S71, determines whether or not a downward click action has been performed. Here, the CPU 20a determines whether or not operation data for a chin click action, i.e., operation data for a click action based on the output of the touch detection unit 124d, has been input from the computer 14. The same applies when determining whether or not there is another click action.

If step S71 is "NO," that is, if there is no downward click action, proceed to step S89 shown in FIG. 16. On the other hand, if step S71 is "YES," that is, if there is a downward click action, then in step S73, it is determined whether the number of downward click actions (i.e., the number of clicks) is 0.

Although not shown in the figure, when the wheelchair control process starts, the counter that counts the number of clicks is reset (i.e., the count value = 0).

If the answer is "YES" in step S73, that is, if the number of clicks on the lower side is 0, the rotation speed of the left motor 38a and the right motor 38b is set to 0 (m/sec) in step S75, the number of clicks is incremented by 1 (number of clicks = 1) in step S77, the timer is started in step S79, and the process returns to step S71.

However, in step S75, the CPU 32a controls the motor drivers 36a and 36b via the interface 34 to set (control) the rotation speeds of the left motor 38a and the right motor 38b. The same applies below when setting the rotation speeds of the left motor 38a and the right motor 38b.

On the other hand, if "NO" in the step S73, that is, if the number of clicks on the lower side is not 0, in a step S81, it is determined whether the number of clicks is 3 or not.
If "NO" in the step S81, that is, if the number of clicks is 1 or 2, in a step S83, the number of clicks is incremented by 1, and the process returns to the step S71.

On the other hand, if step S81 returns "YES," i.e., if the number of clicks is 3, then step S83 determines whether the timer count has reached a second predetermined time (e.g., 1 second).

If step S83 is "YES", that is, if the timer count has passed the second predetermined time, the counter is reset (i.e., the number of clicks = 0) in step S87, and the process returns to step S71. In other words, if the downward click action is not performed three times within the second predetermined time, the wheelchair control process is not terminated. On the other hand, if step S83 is "NO", that is, if the timer count has not passed the second predetermined time, the wheelchair control process is terminated.

In step S89 shown in FIG. 16, it is determined whether or not an upward click motion has occurred. If the answer is "YES" in step S89, that is, if an upward click motion has occurred, the rotation speeds of the left motor 38a and the right motor 38b are set to a first predetermined speed in step S91, and the process proceeds to step S95. Therefore, the wheelchair 30 moves forward at a travel speed corresponding to the first predetermined speed.

On the other hand, if step S89 is "NO," that is, if there is no upward click action, in step S93, the rotation speed of the left motor 38a and the right motor 38b is set to 0, and the process proceeds to step S95. Therefore, if the wheelchair 30 is stopped, it remains stopped, and if it is moving, it decelerates or stops.

In step S95, it is determined whether or not a left click operation has occurred. If step S95 is "NO," that is, if there has been no left click operation, in step S97, the rotation speed of the right motor 38b is set to zero, and the process proceeds to step S105 shown in FIG. 17. However, if the rotation speed of the right motor 38b has already been set to zero, the process of step S97 is skipped.

On the other hand, if step S95 is "YES", that is, if there is a left click operation, then in step S99 it is determined whether the wheelchair 30 is moving forward. Here, the CPU 32a determines whether the left motor 38a and the right motor 38b are rotating. The same applies to step S109, which will be described later.

If step S99 is "YES", that is, if the wheelchair 30 is moving forward, then in step S101, the rotation speed of the right motor 38b is set to a second predetermined speed, and the process proceeds to step S105. Therefore, the wheelchair 30 turns left (i.e., turns left). On the other hand, if step S99 is "NO", that is, if the wheelchair 30 is stopped, then in step S103, the rotation speed of the right motor 38b is set to a third predetermined speed, and the process proceeds to step S105. Therefore, the wheelchair 30 turns left.

As shown in FIG. 17, in step S105, it is determined whether or not a right-side click operation has occurred. If the answer is "NO" in step S105, that is, if there has been no right-side click operation, in step S107, the rotation speed of the left motor 38a is set to zero, and the process returns to step S71 shown in FIG. 15. However, if the rotation speed of the left motor 38a has already been set to zero, the process of step S107 is skipped.

On the other hand, if step S105 is "YES", that is, if there is a right-side click operation, then in step S109 it is determined whether the wheelchair 30 is moving forward. If step S109 is "YES", then in step S111 the rotation speed of the left motor 38a is set to a second predetermined speed, and the process returns to step S71. Therefore, the wheelchair 30 turns left (i.e., turns left). On the other hand, if step S109 is "NO", that is, if the wheelchair 30 is stopped, then in step S113 the rotation speed of the left motor 38a is set to a third predetermined speed, and the process returns to step S71. Therefore, the wheelchair 30 turns right.

The second embodiment achieves the same effect as the first embodiment and also allows the movement of the wheelchair to be controlled.

The wheelchair control method shown in the second embodiment is an example and should not be limited to the above. In another example, instead of detecting a downward click motion three times within the second predetermined time, the wheelchair control process may be terminated when left and right click motions are detected simultaneously, or when upper, left and right click motions are detected simultaneously, or when lower, left and right click motions are detected simultaneously, or when upper, lower, left and right click motions are detected simultaneously.

[Third Example]
The third embodiment is a system in which the input device 10 of the present invention is used to control mouse operations. This system will be described below, but as the input device 10 has been described in the first and second embodiments, a duplicate description will be omitted.

Although not shown in the figures, the system of the third embodiment is the system 400 of the first embodiment in which the eye-gaze input device 22 has been removed and the input device 10 of the second embodiment is used.

The user terminal 20 therefore controls the position, i.e., movement, of the mouse cursor based on pressing and clicking operations from the input device 10, and executes commands based on left-clicking and right-clicking operations.

Specifically, in the third embodiment, the CPU 20a of the user terminal 20 executes the mouse operation process shown in FIG. 18. It is assumed that the software keyboard described in the first embodiment is displayed, but the software keyboard does not have to be displayed. In addition, the mouse operation process is executed in parallel with a predetermined application.

As shown in FIG. 18, when the CPU 20a starts mouse operation, in step S121 it determines whether the pressing of any of the up, down, left, and right buttons continues for a third predetermined time (e.g., one second) or more.

If the answer is "NO" in step S121, and if no pressing action to the top, bottom, left, or right continues for more than the third predetermined time, proceed to step S125. However, in step S121, if no pressing action to the top, bottom, left, or right is performed, it is also determined to be "NO".

On the other hand, if step S121 returns "YES," meaning that the pressing action in either the up, down, left, or right direction continues for a third predetermined time or longer, then in step S123, the mouse cursor is moved in the direction in which the pressing action continued for a third predetermined time or longer, and the process returns to step S121.

In step S125, the mouse cursor movement speed is set to 0. In the next step S127, it is determined whether or not a left click action has been performed. If the answer is "NO" in step S127, that is, if a left click action has not been performed, then in step S129, it is determined whether or not a right quick action has been performed.

If step S129 is "NO", that is, if there has been no right-click action, the process returns to step S121. On the other hand, if step S129 is "YES", that is, if there has been a right-click action, the process executes a right-click command in step S131 and returns to step S121. For example, in step S131, if character input is being performed, one character is deleted as described in the first embodiment, or if the mouse cursor is pointing to a button displayed on the execution screen of a specified application, the properties of that button are displayed on the display.

Also, if the answer is "YES" in step S127, that is, if a left click has been performed, then in step S133 it is determined whether the mouse cursor is over the end button of a specific application.

If the answer is "NO" in step S133, that is, if the mouse cursor is not on the end button of the specified application, a left click command is executed in step S135, and the process returns to step S121. For example, in step S135, characters are entered using a software key, or a function assigned to a button on the execution screen of the specified application is executed.

On the other hand, if the answer is "YES" in step S133, that is, if the mouse cursor is on the end button of a specific application, in step S137, an instruction to end the specific application is issued, and the mouse operation processing ends.

The third embodiment achieves the same effects as the first embodiment, and also allows commands to be executed by moving the mouse cursor and clicking.

Note that the specific numerical values and control methods shown in each of the above examples are merely examples and are not limited to these, and can be modified as appropriate depending on the actual product and the environment in which the product is used.

REFERENCE SIGNS LIST 10 Input device 12 Cheek movement detection device 14, 32, Computer 20 User terminal 22 Line-of-sight input device 30 Wheelchair 400, 500 System

Claims (6)

A deformable sheet-like covering portion that covers at least a portion of the cheek of the user;
an input device comprising: a detection unit attached to a side of the covering unit that comes into contact with the user's face, the detection unit being capable of detecting at least a left cheek movement and a right cheek movement of the user; and a generation unit that generates operation data based on the left cheek movement and/or the right cheek movement of the user detected by the detection unit. The input device according to claim 1, further comprising a transmission unit that transmits the operation data generated by the generation unit to an external computer. the detection unit is two capacitive touch sensors,
The input device according to claim 1 , wherein the two capacitive touch sensors are disposed on the left and right sides of the covering portion, respectively, and detect motion of the left cheek and/or the right cheek of the user. the detection unit includes four capacitive touch sensors;
The four capacitive touch sensors are arranged on the top, bottom, left and right of the covering portion, respectively, and detect at least one movement of the left cheek, right cheek, philtrum and chin of the user;
3. The input device according to claim 1, wherein the generation unit generates at least one of operation data based on the movement of the user's left cheek, operation data based on the movement of the user's right cheek, operation data based on the movement of the user's philtrum, and operation data based on the movement of the user's jaw detected by the four capacitive touch sensors. The capacitive touch sensor includes:
A conductive cloth in which a first cloth formed by knitting a conductive thread and a second cloth formed by knitting an insulating thread are overlapped with each other;
The input device according to claim 3 or 4, further comprising: an insulating sheet having the same size as the first cloth and attached to the first cloth; and a capacitance sensor IC electrically connected to the first cloth. The input device according to claim 5, wherein the conductive cloth is stretchable.

Images