JP2003518687A - Handwriting command - Google Patents

Abstract

A handheld electronic device adapted to perform at least one operation. The device can be controlled by the user using the device as a pen and writing a command to initiate the operation, such as "dial the following phone number". A method of initiating operation in an electronic device and software used to implement the method.

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD OF THE INVENTION The present invention relates to handheld electronic devices adapted to perform at least one operation. The invention also relates to a method of initiating operation of a handheld electronic device and its computer program.

[0002]

BACKGROUND OF THE INVENTION Handheld computers and other handheld electronic devices such as cell phones, PDAs, reading pens, scanners, typically assist the user in issuing various commands to the device or entering information used by the device. To do
It is controlled by one or several buttons or keys. With only a few buttons, controlling the device is usually cumbersome and time consuming. On the other hand, if there are a large number of buttons / keys, they are usually small due to the small amount of space provided in a handheld device, which makes them difficult to use,
Increases the risk of pressing the wrong button / key.

US Pat. No. 5,852,434 discloses a device that allows a user to enter handwritten information into a computer while the information is being written on the writing surface. The device consists of a writing surface on which a position code for coding the X / Y coordinates on the surface is arranged, and a special pen with a writing tip on which the user can write. Furthermore, the pen comprises a light source for illuminating the position code and a CCD sensor for forming an image of the position code.
The position information received by the CCD sensor is transferred to a computer for processing.

WO99 / 48268 discloses a handheld device that writes a telephone number and sends the number to a mobile telephone for auto dialing.

[0005]

SUMMARY OF THE INVENTION It is an object of the present invention to facilitate control and operation of handheld devices.

This object is realized wholly or partly by a device according to claim 1, a computer program according to claim 16 and a method according to claim 17 or 18. Preferred embodiments are defined in the dependent claims.

According to a first aspect, the invention relates to a handheld electronic device adapted to perform at least one operation, the device comprising:
It can be controlled by the user using the device as a pen and writing a command to initiate the action.

Therefore, a benefit of the present invention is that the user does not have to press any keys or buttons on the device. Instead, the user controls the device by writing commands for the actions he wants to perform. This makes the device more user friendly.

For example, a command written by a user can be "dial" if the user wants the device to dial a word indicating the action desired to be performed, eg, a number. It can also be a symbol, letter, number, or figure that indicates the action you want to perform.

The device can record commands written by the device in various ways, for example, using one or more sensors to detect movement of the device.

However, in a preferred embodiment, the device is adapted to record the command by detecting a position code located on the writing surface on which the command is written.

This embodiment is advantageous because it allows the accurate recording of handwritten commands and the determination of the absolute position of the device at the writing surface. More specifically, the position code can be continuously detected while the command is being written, resulting in a series of position indicators that define how the device is operated and thus what command was written.

The device may use a variety of sensors depending on the technique used to implement the position code. In the preferred embodiment, the position code is designed to be detectable by an optical sensor. In this case, the device preferably comprises an optical sensor adapted to record an image of the writing surface and a signal processing device adapted to use a position code in the image which provides a digital representation of the command.

This embodiment is advantageous because it is based on image processing, which is a known and well-developed technique.

The signal processor may include an interpretation function for recognizing command symbols. Thus, the user can define any symbol as a command symbol, and when writing the symbol, the device interprets the symbol as a command.

One example is the character @, which can be interpreted as a command that subsequent characters should be interpreted as an email address. By rewriting the command symbol @ after the email address (hence the third time)
, It is interpreted as the end of the email address input.

Further, the signal processor may include a character interpretation function adapted to convert the digital representation of the command into a character encoded form. This embodiment is advantageous because it facilitates the determination of which command the user wrote and available character recognition software can be used to interpret the command. The character interpretation operation can be executed by, for example, ICR software.

In addition, the device may be adapted to record message information used in operation and recorded in essentially the same manner as the command. In this way, the user will have the same interface for controlling the device and for entering message information. The device is not very expensive to manufacture if the same means are used to enter message information.

The message information can be, for example, text sent in an electronic mail message, a telephone number to dial, an item created in a calendar program, or some other similar information. It may include text, numbers, various characters, symbols, graphics, drawings, and other information that can be generated by "handwriting" defined as any manual movement of the device on a surface.

In the preferred embodiment, the message information is also recorded by detecting the position code of the writing surface.

The device can have at least two modes, one is a command mode for recording commands and the other is an information mode for recording information. By having different modes, the device knows how the data being written by the device should be processed, and there is no doubt as to whether the data is command or message information.

In one embodiment, the device assumes command mode when certain command symbols are written with the help of the device. The command symbol can be a graphic, one or several letters, a picture, and the like.

In another embodiment, the device assumes command mode when it detects that the writing surface has a special design. For example, the writing surface may comprise a special area for writing commands and another area for writing information, the device detecting in which area it is located with the aid of a position code.

The principles in accordance with the present invention can be used to cause a handheld electronic device to perform any action that can be initiated by a command. Examples of operations performed in handheld electronic devices include making phone calls, faxing, sending electronic messages, storing information, handling documents and files, and starting, controlling and ending programs. The action may or may not be an action that uses message information.

In the preferred embodiment, the device is a mobile phone, which allows the user to use the phone as a pen, eg, "dial 1234567.
Control by writing "," at which point the phone will automatically dial the number.

In an alternative embodiment, the device comprises an accelerometer for recording commands. Accelerometers record the movement of the device as commands are written.
This embodiment has the advantage that the commands can be written on any surface or even in the air without contact with the surface. Thus, there is no need for a writing surface with a position code, and even no writing surface, and vice versa, a command can be "written" on any arbitrary surface.

In yet another alternative embodiment, the device is a light sensor for recording an image with partially overlapping content, and by determining the relative position of the image the device is associated with writing commands. A signal processing device adapted to determine whether it has been moved.

This embodiment has the advantage that no position code is required, but, on the contrary, the user is either sufficiently patterned to allow the relative position of the image to be determined. You can "write" commands on the surface with the help of the device. A technique for determining what was written is disclosed in the published publication WO99.
/ 60467, the contents of which are incorporated herein by reference. This method of recording commands is described in "C-pen," in the form described in publication WO 98/20446, the contents of which are incorporated herein by reference.
It may suitably be used to control a reading pen or handheld scanner sold under the trademark "R".

The device does not have to be a single integrated device. In one embodiment, the device can have first and second parts that are separable and comprise transceivers for mutual wireless or wired communication, the device comprising: It is controllable by using the first part as the pen for writing commands.

According to a second aspect of the invention, the invention relates to a computer program, which is stored on a storage medium, readable by a computer and used by a computer as a pen. A command to detect a command written by the handheld electronic device and initiate a predetermined action in response to the command. The advantages of this software are clear from the above. The software is designed to be installed on the device for writing commands.

According to a third aspect of the invention, the invention relates to a method of initiating an operation of a handheld electronic device, which comprises using the device itself as a pen and writing commands for performing said operations. It is a thing. The benefits of the method are clear from the above. The invention is described in detail below in connection with the accompanying drawings.

[0032]

DESCRIPTION OF THE PREFERRED EMBODIMENT FIG. 1 discloses a handheld device in the form of a digital pen that can be used to write text and lines, patterns or symbols recorded in digital form. The device is controlled by the user by writing commands for actions performed with this pen.

The device comprises a casing 11 which is substantially the same shape as the pen. On one short side of the casing there is an opening 12 or window that is transparent to infrared light. This short side is intended to be located a short distance from or abut against the writing surface 3 during operation of the device (see FIG. 2).

The casing 11 essentially comprises an optical section, an electronic circuit section and a power supply. The optics consists of at least one light emitting diode 13 for illuminating the surface to be imaged and a photosensitive area sensor 14 for recording a two-dimensional image, such as a CCD or CMOS sensor. The user device may also include a lens system (not shown).

The power supply to the device is obtained from the battery 15, which is provided in a separate compartment of the casing 11. The battery can also be a rechargeable battery. Alternatively, the power source is obtained by cabling with a separate device that is the power source.

The electronic circuit section comprises a signal processing device 16. The signal processor 16 is embodied with the aid of a microprocessor and memory and comprises software which analyzes the image from the sensor in order to obtain a digital representation of what was written on the writing surface 3. In addition, it includes operating software for initiating and / or performing various operations in response to commands, as well as user software. Further, the signal processor 16 preferably comprises ICR software used to interpret the recorded characters so that the characters can be stored, manipulated and / or transferred in a character encoded form.

The casing 11 of the device is also provided with a nib 18, with which the user can write normal pigment-based writings on the writing surface 3. The pen tip 18 is retractable so that the user can control whether or not to use it. The pigment is preferably non-absorbent for infrared light.

In addition, the device is provided with a button 19 by means of which the device can be activated and controlled for operations not initiated by commands written by the device. Button 19, like the on / off switch, can also be used to cause the device to enter various modes.

The device may also comprise a transceiver 20 for wireless transfer of information over short distances, eg by infrared light, ultrasound or radio waves. This wireless transfer can be used when the device is unable to perform the entire operation initiated by itself and needs to transfer recorded information about the operation to perform to an external device. Finally, the device may comprise a display 21 for displaying the information.

The above-mentioned publication WO 98/20446 discloses a device for scanning and recording text. If the program was created in the right way,
The device can be used to record text and commands written on a writing surface provided with a position code and to initiate / execute an action in response to the handwriting command.

FIG. 2 shows a sheet of paper 1 forming the writing surface 3.
A position code 4 is printed on the writing surface 3, which is the first and the second.
Types 5a and 5b, more specifically, symbols 5 which are points of large diameter and points of small diameter corresponding to "1" and "0", respectively. For clarity, the symbol 5 has been enlarged and the position code 4 is only shown in a small part of the writing surface 3. In a practical embodiment, the position code is spread over the writing surface 3 and the symbols 5 are smaller to ensure a better position resolution.

The position code 4 can be created in various ways. One method of creating a position code 4 in which each position is encoded with one symbol 5 is described in US Pat.
, 852,434. However, in the present invention, each position is preferably encoded with the aid of a large number of symbols 5, such that the individual symbols 5 in the position code 4 contribute to the encoding of multiple positions, This is so-called floating coding. This means that two adjacent positions share some symbols 5, as indicated by the dashed areas 6a, 6b in FIG. In this way, higher resolution is obtained and detection is easier because the complexity of the individual symbols 5 is reduced. This type of overlap of floating position code 4 can be created with the aid of a computer.

A special method for generating such a floating position code is disclosed in WO 00/7398, the contents of which are incorporated herein by reference.
3 and their contents are incorporated herein by reference, both are 200
Patent application number PCT / SE00 / 001895, PCT / filed on October 2, 0
It is described in SE00 / 001897 and PCT / SE00 / 001898.

The operation of the apparatus shown in FIG. 1 will be described. Suppose the user wants to create an item in the device's calendar program. In this case, the user must first open the calendar program. To have the device perform this task, the user first writes a special command symbol on the paper 3 with the nib 17 and then writes the command letter "Calendar". The command symbol may be any symbol that the user has defined as a command symbol, for example, a small circle such as "o" is written inside a large circle such as "O". As with the above, it is possible to use only one stroke.

While the user is writing, the optical sensor 14 continuously captures the image of the writing surface 3 at the current position of the device. Each image contains a part of the position code 4. The signal processor 16 locates the position code 4 in each image and based on this code determines the absolute position coordinates of the writing surface 3 where each image was captured. In this way, a description of how the pen was moved on the paper is obtained in the form of multiple position indicators. This description constitutes a digital representation of the command symbols and commands. The position indicator is provided as an input signal to the ICR software, the latter determining the character to which the position corresponds so that the command symbol and command can be translated into a character coded form. The command can be terminated by retyping the command symbol.

When the processing unit detects a command symbol, it knows that the following character should be interpreted as a handwriting command. The processor then
To determine which command the user has written, the letter representing the recorded command is compared to a number of predetermined commands. If the processor determines that the command is for opening the calendar program, the processor performs the operation of opening the calendar program. The program can, for example, show the date of the calendar on the display 21 in the usual manner.

The user moves the cursor at the time when the item has to be created. After that, the user uses the pen tip 17 to write an item to be created on the paper 3. The device records the item as if it were a command. The processing unit is a command
Since it does not detect symbols, it knows that the written characters represent information. The characters are translated into a character encoding form with the aid of ICR software and stored in a calendar program.

Next, the user wants to save and close the program. To do so, the user writes the command "save" following the command symbol. The handwritten command is processed in the same manner as the command "open", resulting in the action of saving the written item.

After executing the command, the device returns to its normal input mode until the next command symbol is entered.

It should be noted that the nib 17 is only used to help the user understand what he is writing. It has no other functionality.

FIG. 3 diagrammatically shows another embodiment of the device according to the invention.
In this example, the device is new software, transceiver 3 for short range wireless communication.
1 and a normal mobile phone 3 supplemented with a pen 32 for controlling the phone
It is 0. The pen is placed in a holder 33 on the side of the phone when not in use. The holder may include contacts for connecting the cell phone battery and the pen to charge the pen battery. Alternatively, the pen battery can be charged at the same time as the phone battery is charged by the battery charger. The phone charging circuit can also be used to control the charging of the pen battery.

The holder may also be provided with a connector for connecting the pen to the electronic circuitry of the telephone, whereby the pen either downloads its stored content to the telephone or uses the telephone as a modem to store its stored content. Can be sent to the network.

The structure of the pen 32 is essentially the same as that of the device of FIG.
However, it has no display, no cell phone transceiver, and only part of the software of the device of FIG.

For example, when the user wants to make a phone call, the user removes the pen and writes "dial" on the paper with the position code using the pen. The user circles the word "dial" to indicate that it is a command. The user then writes the telephone number to dial. The pen records the image of the position code of the paper, and transfers these images to the transceiver 31 of the mobile phone by the intermediary of the transmitter 34 for short-range wireless communication. The received image is processed in the mobile phone as described above with respect to FIG. 1, so that the mobile phone performs the operation of dialing the indicated number.

Alternatively, the light sensor of the pen 32 can be integrated with the mobile phone and the entire mobile phone can be used as a pen for writing commands.

Another alternative is for the pen to perform all actions and operations, once the phone number is recognized and decoded into an ASCII code, the pen calls the phone number to initiate the call. Or send the phone number to the phone via the connector described above.

As an alternative to the above-described embodiment in which commands are indicated by special characters or graphics, the device can switch between command mode and information mode with the aid of button 19. If the number of possible commands is small, there is no need to specifically command the commands.

As mentioned above, the detection of handwriting commands may include recording images with partially overlapping content to determine movement of the accelerometer or device, such as
It can be based on other techniques other than position code detection. Those skilled in the art will have no difficulty in modifying the above-mentioned devices for the use of alternative techniques, so that their embodiments will not be detailed.

Finally, a preferred embodiment of the absolute position coding pattern is described here. For the sake of clarity, it will be described in relation to the form. This is PCT /
This corresponds to the absolute position coding pattern described in SE00 / 01895. Since the surface provided with the position code has the impression as a pattern, it is hereinafter referred to as a position-coding pattern.

FIG. 4 shows an enlarged portion of the sheet, the surface 102 of which is shown.
Is provided with a position coding pattern 105. The sheet has an X coordinate axis and a Y coordinate axis.

The position-coding pattern is a virtual raster which is invisible to the human eye and cannot be directly detected by the device for determining the position on this surface, and the four values "1" as described below. ˜4 ”and a plurality of symbols that can be assumed to be one of the above.

The position coding pattern is arranged so that the symbols on the partial surface of the sheet code the absolute coordinates of the positions on the virtual surface, which will be described later. The first and second partial surfaces 125a, 125b are indicated by dotted lines in FIG. That portion of the position-coding pattern found in the first partial surface 125a (in this example, a 4 × 4 symbol) encodes the coordinates of the first point of the virtual surface and in the second partial surface 125b. That part of the position-coding pattern that is found codes the coordinates of the second point. Therefore, the position-coding pattern is composed of adjacent first and second
Partly shared by the point of. Such a position-coding pattern is referred to as "floating" in this application.

FIGS. 5a-5d show one embodiment of the symbols used in the position-coding pattern. The symbol comprises a virtual raster point 130 represented by the intersection of the raster lines and a marking 106 in the shape of a point. The value of the symbol depends on where the marking is placed. In the example of FIG. 5, four types of positions are possible and each is on each raster line extended from the raster point. The deviations from the raster points are all the same. The symbols in FIG. 5a have the value 1, the value 2 in FIG. 5b, the value 3 in FIG. 5c, and the value 4 in FIG. In other words, there are four different forms of symbol. It is pointed out that these points may, of course, have different shapes. Therefore, each symbol can represent four values “1 to 4”. This means that the position coding pattern can be divided into a first position code for the X coordinate and a second position code for the Y coordinate. This division is as follows.

[0064]

[Table 1]

In this way, each symbol value is converted into a first number (in this case bits) for the X code and a second number (in this case bits) for the Y code.
In this way two completely independent bit patterns are obtained. This bit pattern can be integrated into the overall pattern which is graphically encoded by the symbols according to FIG.

The coordinates of each position are encoded by a plurality of symbols. In this example, a 4x4 symbol is used to encode the position in two dimensions, i.e., the X and Y coordinates.

The position code is composed of a number string of 1s and 0s, and the number string has a characteristic that the same 4-bit sequence appears only once in the number string. The sequence of numbers is cyclic, and the same applies to the case where the end of the sequence of numbers is joined to the beginning of the sequence of numbers. Therefore, the 4-bit array always has a uniquely fixed position in the number string.

If the number sequence has the characteristics described above for a 4 bit sequence, the number sequence can be up to 16 bits long. However, in this example, a 7-bit long digit string is used as follows. "0001010"

This sequence of numbers contains seven unique 4-bit sequences that encode positions within the sequence of numbers:

[0070]

[Table 2]

In X-coordinate encoding, a sequence of numbers is written in columns in sequence over the entire surface to be encoded. The encoding is based on the difference in the numbers between adjacent columns, ie the displacement of the positions. The magnitude of the difference depends on the position within the starting column number sequence (ie, which sequence is used). Specifically, a number that is encoded by a 4-bit sequence in the first column and therefore has values (positions) 0 to 6 and a corresponding number in the adjacent column (same "level" sequence). If we take the modulo 7 difference between, the result will be the same regardless of the position in the two columns being compared. Therefore, the difference between two columns can be used to encode an X coordinate that is constant for all Y coordinates.

In this example, since each position on the surface is coded using 4 × 4 symbols, as described above, the three differences (having values 0 to 6) are the X coordinates. It can be used for encoding. Then, the encoding is performed so that the three differences have one value that is always 1 or 2 and the other two values that are in the range of 3 to 6. As a result, zero differences are not allowed in the X code. In other words, the difference in the X code is
(3 to 6) (3 to 6) (1 to 2) (3 to 6) (3 to 6) (1 to 2) (3 to 6) (3 to 6) (1 to 2) ... Be built. Therefore, each X coordinate is encoded using two numbers between 3 and 6, followed by one number, 1 or 2. Subtracting 3 from the larger number and 1 from the smaller number gives the mixed radix number, which gives the position directly in the X direction. From this position, the X coordinate can then be determined directly, as shown in the example below.

According to the above-mentioned principle, X coordinates 0, 1, 2. ． ． Can be encoded using numbers representing the three differences. These differences are encoded using a bit pattern based on the above digit sequence. The bit pattern is finally graphically encoded using the symbols of FIG.

In many cases, when reading a 4.times.4 symbol, it is not possible to make a complete number that encodes the X coordinate, even if some of the two numbers can be made. However, the lowest part of these numbers is always 1 or 2, so the complete number can be easily reconstructed.

The Y coordinate is encoded according to the same principles used for the X coordinate. The circular digit sequence is written repeatedly in a horizontal row across the surface encoding the position. Just as for the X coordinate, this row can start at a different position within the digit sequence, ie, using a different sequence. However, it does not use the difference for the Y coordinate, but encodes the coordinate using a number based on the starting position of that number column in each row. After the X coordinate of the 4x4 symbol is determined, 4x
One can actually determine the starting position in the digit sequence of the row contained in the Y code in the 4 symbol. In the Y code, the most significant digit is determined by making it the only number with a value within the specified range. In this example, one of the four rows starts at position 0-1 in the digit string and then the other three rows to indicate that the row is associated with the least significant digit of the Y coordinate. Rows start at positions 2-6 in the digit column. Therefore, in the Y direction, (2 to 6) (2 to 6) (2 to 6) (0 to 1) (2 to 6) (2 to 6) (2 to 6) (0 to 1) (2 to 6) ) ... there is a sequence of numbers. Therefore, each Y coordinate is encoded using three numbers between 2 and 6, followed by a number between 0 and 1.

By subtracting 1 from the small number and subtracting 2 from the large number, the position in the Y direction can be obtained in the same manner as in the X direction, and the Y coordinate of the mixed radix can be directly determined from this position.

The above method can be used to encode 4 × 4 × 2 = 32 positions in the X direction. Each such position corresponds to 3 differences, 3 × 3
2 = 96 positions are given. Furthermore, 5 × 5 × 5 × 2 = 250 positions can be encoded in the Y direction. Each such position corresponds to 4 rows, 4 × 250 = 10
Give 00 positions. Thus, a total of 96000 positions can be encoded. However, since the encoding of X is based on the difference, it is possible to choose the position where the first digit sequence starts. Considering that this first sequence of digits can start at 7 different positions, 7 × 96000 = 672000 positions can be encoded. When the X coordinate is determined, the starting position of the first number sequence in the first column can be calculated. First
The above-mentioned seven different starting positions of the number sequence make it possible to code the writing surface on different papers or products.

Further, in order to further illustrate the function of the position-coding pattern of this embodiment, a specific example based on the above-described position-code embodiment will be shown below.

FIG. 6 shows an example of an image containing 4 × 4 symbols read by a position determining device. These 4x4 symbols have the following values: 4 4 4 2 3 2 3 4 4 4 4 2 4 1 3 2 4

These values represent the following binary X code and Y code. X code: Y code: 0 0 0 0 0 0 0 0 1 1 1 0 1 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 1 0 0 1 0 1 1 0

The vertical X array encodes positions 2 0 4 6 in the number sequence. The difference between the columns is -42, and the remainder modulo 7 is 542. This is a mixed radix of (5-3) × 8 + (4-3) × 2 + (2-1) = 16.
The position of + 2 + 1 = 19 is encoded. Since the position of the first coded X is position 0, it is in the range of 1-2 and this difference seen in a 4x4 symbol is 20
Will be the second difference. Furthermore, since there is a sum of three columns for each such difference and there is a starting column, the rightmost vertical alignment of the 4x4 X code is the 61st (3x3) X code. 20 + 1 = 61), and the leftmost vertical array belongs to the 58th column.

The horizontal Y sequence encodes positions 0 4 1 3 in the digit string. These numbers start at the 58th column, so the starting position for the row is the remainder of these numbers minus 57 modulo 7 with a starting position of 6 30
It becomes 2. Converted to mixed radix numbers, this is 6-2, 3-2, 0-0, 2-2
= 4 1 0 0. Here, the third digit is the lowest digit in the number of interest. Then, the fourth digit is the most significant digit in the next digit. In this case it must be the same as in the number of interest (an exception is raised if the number of interest consists of the largest possible number of positions). In that case, it can be seen that the start of the next number is one greater than the start of the number of interest). Next, the position of the 4-digit number is 0x50 + 4x10 + 1x2 + 0x in the mixed radix.
1 = 42. Therefore, the third line of the Y code becomes the 43rd line, which has a starting position of 0 or 1, and there are a total of 4 lines for each such line, so the 3rd line is 43 × 4 = 172. Therefore, in this example, the position of the upper left corner for the 4 × 4 symbol group is (58,170).

Since the X alignment of the 4 × 4 group begins on row 170, the X columns of all patterns are at the position of the digit sequence ((2 0 4 6) -169) mod 7 = 1 6
It starts from 35. Numbers 0 to 1 between the last start position (5) and the first start position
9 is encoded in mixed radix and by summing the representation of the numbers 0 to 19 in mixed radix we get the total difference between these columns. A simple algorithm to do this would generate these 20 numbers and sum those numbers directly. The sum of the results is called s. Therefore, the paper or writing surface is obtained by (5-s) mod 7.

In the above example, one embodiment is described, in which each position is 4 ×
It was coded using 4 symbols and a 7-bit digit sequence was used. Of course, this is just one example. Positions can be encoded using a greater or lesser number of symbols. The number of symbols need not be the same in both directions. The number strings may have different lengths, other than binary numbers, or may be based on other bases. Different digit sequences can be used for X-direction encoding and Y-direction encoding. Symbols can have numbers with different values. As is clear from the above,
Coding with 6x6 symbols is currently most preferred, and each symbol can assume four values. Those skilled in the art can easily generalize the above example to such encoding.

In the above example, the markings are dots, but of course other shapes are possible. For example, the markings may consist of dashes or other symbols that start at a virtual raster point and extend into place. As yet another example, the markings can be rectangular, square, triangular, or any other easily detectable shape. The markings may be filled or unfilled.

In the example above, the symbols within the square-shaped partial surface are used for position coding. The partial surfaces can also have another shape, for example a hexagon. The symbols need not be arranged along rows and columns of 90 degrees relative to each other, but may be arranged at other angles, such as 60 degrees, and / or other arrangements. The symbols can also encode positions in polar coordinates or other coordinate systems.

To detect the position code, a virtual raster must be determined. This is done by examining the distance between the different markings. The shortest distance between two markings must be derived from two adjacent symbols with values 1 and 3 (horizontal) or values 2 and 4 (vertical). This places the markings on one raster line between the two raster points. When such a set of markings is detected, the relevant raster points can be determined using knowledge of the distance between the raster points and the amount of marking deviation from the raster points. Once the two raster points have been placed, knowledge of the measured distance to the other marking and the distance between the raster points can be used to determine the next raster point.

The position-coding pattern described above can code a large number of unique positions, to be exact, the absolute coordinates of these positions. It can be said that all the positions or points that can be coded using the position coding pattern together make up a virtual surface. Different parts of the virtual surface can be dedicated to different specific applications. Virtual surface 1
One area can be dedicated to be used, for example, as a writing surface, another area as a character recognition area, and yet another area to be used as various activation icons. Other areas of the virtual surface can be used for other applications. Thus, the corresponding subset of the position-coding pattern can be used, for example, to generate a particular activation icon that can be placed anywhere on the product. The coordinates encoded by this subset of the position-coding pattern are therefore not associated with the location on the product, but always with the location on the virtual surface dedicated to correspond to this activation icon.

In the preferred embodiment, the nominal space between points is 0.3 m.
m. The part of the position-coding pattern that contains 6x6 points, wherever it may be, defines the absolute coordinates of the points on the virtual surface. Each point on the virtual surface is therefore defined by a 1.8 mm x 1.8 mm subset of the position-coding pattern. By determining the position of the 6 × 6 points on the sensor of the device used to read the pattern, the position can be calculated by interpolation on the virtual surface with a resolution of 0.03 mm. Since each position is encoded with 6x6 points, each of which can take one of four values, 2 ⁷² positions can be encoded, which is the nominal space between the points mentioned above. Corresponds to a surface of 4.6 million km ² .

The absolute position coded pattern can be printed on various paper strips and other materials capable of achieving a resolution of approximately 600 dpi. The paper can be of various sizes and shapes to suit the intended use. The pattern can be printed with a standard offset. Common black carbon based printing inks or other printing inks that absorb infrared light are preferably used. This means that other inks, including black inks based on non-carbon, can be used to overlay other printed text on the absolute position coding pattern without disturbing reading from it. .

A surface provided with the above-mentioned pattern printed using a carbon-based black printing ink appears to the human eye as simply a slightly grayish hue surface (concentration 1-3%). It is easy to operate and does not spoil aesthetics.

It will be appreciated that more or less points than described above can be used to define the position on the virtual surface, and the distance between larger or smaller points can be used in the pattern. The examples given here are only used to show how the pattern may be implemented at the present time seems to be preferred.

[Brief description of drawings]

FIG. 1 is a schematic partial cross-sectional view of an embodiment of a handheld device according to the present invention.

FIG. 2 is a plan view of a writing surface provided with a position code that can be used to record handwritten commands.

FIG. 3 is an isometric view of a second embodiment of a handheld device according to the present invention.

FIG. 4 is a schematic of a product provided with an alternative position-coding pattern.

5 is a schematic diagram showing how marks may be designed and arranged in the position code pattern of FIG.

FIG. 6 is a schematic diagram of a code pattern in which 4 × 4 symbols are used to encode a position.

─────────────────────────────────────────────────── ─── Continued front page    (81) Designated countries EP (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, I T, LU, MC, NL, PT, SE, TR), OA (BF , BJ, CF, CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG), AP (GH, G M, KE, LS, MW, MZ, SD, SL, SZ, TZ , UG, ZW), EA (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), AE, AG, AL, AM, AT, AU, AZ, BA, BB, BG, BR, BY, B Z, CA, CH, CN, CR, CU, CZ, DE, DK , DM, DZ, EE, ES, FI, GB, GD, GE, GH, GM, HR, HU, ID, IL, IN, IS, J P, KE, KG, KP, KR, KZ, LC, LK, LR , LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, MX, MZ, NO, NZ, PL, PT, R O, RU, SD, SE, SG, SI, SK, SL, TJ , TM, TR, TT, TZ, UA, UG, US, UZ, VN, YU, ZA, ZW

Claims (24)

[Claims] 1. A handheld electronic device adapted to perform at least one action, a recording device (14) for recording a stroke when the device is moved, said stroke being a command. Interpreting means for determining whether or not
6) and a processing means (16) for executing an operation after judging that the command is the command.
And a device characterized by. 2. The recording device has a writing surface (3) on which the command is written.
Device according to claim 1, adapted to record said command electronically by detecting a position code (4) arranged above. 3. An optical sensor (14), wherein the recording device is adapted to record an image of the writing surface (3), and the position within the image to obtain a digital representation of the command. Device according to claim 2, comprising a signal processing device (16) adapted to use the code (4). 4. Apparatus according to claim 3, wherein said signal processing device (16) comprises a character interpretation function adapted to translate the digital representation of said command into a character encoding form such as ASCII code. . 5. The apparatus of claim 1, wherein the recording device is further adapted to record message information used in the operation in essentially the same manner as recording the command. . 6. A device according to claim 5, wherein the recording device is adapted to record the information by detecting a position code on the writing surface. 7. The device has at least two modes, one of which is a command mode for recording the command and the other of which is an information mode for recording the message information. Or the device according to item 6. 8. The device of claim 7, wherein the device is adapted to initiate use of the command mode when a user writes the predetermined command with the device. 9. The device is adapted to initiate use of the command mode when the device detects that the writing surface (3) has a predetermined design. Item 7. The apparatus according to item 7. 10. The device of claim 1, wherein the device comprises an accelerometer for electronically recording the command. 11. The device in connection with the writing of the command, by determining the relative position between the image and the optical sensor (14) for recording images of partially overlapping content. The apparatus of claim 1, comprising a signal processing device adapted to determine how is moved. 12. A device according to any one of the preceding claims, wherein the device is a mobile phone. 13. A device according to claim 1, wherein the device is a digital pen for electronic recording of information. 14. Only the removable portion of the device is used as a pen for writing commands to perform the operation, the removable portion comprising:
A device according to any one of the preceding claims, which is adapted to communicate with the rest of the device. 15. The device has first and second portions that are divisible and have transceivers for wirelessly communicating with each other, the device comprising: An apparatus according to any one of the preceding claims, which can be controlled by using the first part by writing the command for the start of an operation. 16. A computer-readable software program stored on a storage medium, the computer detecting a command written using a handheld electronic device used as a pen and responding to the command. A software program with instructions that cause it to initiate a predetermined operation. 17. A method for initiating an action in a handheld electronic device, characterized by using the device itself as a pen and writing a symbol of a command to perform the action. Method. 18. A method for controlling a handheld electronic device according to any one of claims 1 to 15, wherein the device is adapted to perform at least one operation, A method characterized in that a stroke is recorded when the device is moved, it is determined whether the stroke includes a command, and, if it is determined that the command, the operation is performed. 19. The method according to claim 18, characterized by electronically recording the command by detecting a position code (4) arranged on a writing surface (3) on which the command is written. Method. 20. An optical sensor (1) for recording an image of the writing surface (3).
20. The method according to claim 19, characterized by 4) and recording by using a signal processor (16) that uses the position code in the image to obtain a digital representation of the command. 21. The method of claim 20, characterized by a character interpretation that translates a digital representation of the command into a character encoding form such as ASCII code. 22. The method of claim 18, characterized by recording message information in essentially the same manner as recording the command. 23. The method of claim 22, characterized by recording the message information by detecting a position code on a writing surface. 24. The method of claim 23, wherein the device is adapted to initiate use of the command mode when a user writes the predetermined command with the device.