JP7739555B2 - Information processing device, control method thereof, and application - Google Patents

Description

The present invention relates to a technique for displaying characters.

Conventionally, a dedicated delete button has often been provided for deleting character strings entered on a touch panel. However, because typical delete buttons can only delete one character at a time, a user who, for example, wanted to shorten an automatically entered filename that was too long had to repeatedly press the delete button to create the desired filename. Furthermore, if a user accidentally entered a character string, they had to move their finger to the delete button and operate it each time. In this regard, Patent Document 1 discloses a technology that, when an incorrect character string is entered using predictive conversion, returns the text to its pre-conversion state with a single operation of the delete button, eliminating the need for repeated button operations. Furthermore, Patent Document 2 discloses a technology that, when an unintended character is accidentally entered through a touch operation, replaces it with the surrounding characters that were originally intended by touching and sliding the same character.

Japanese Patent Application Laid-Open No. 2016-186717 JP 2012-243229 A

The above Patent Documents 1 and 2 did not disclose an application that functions as a control means for controlling a computer to display a UI screen including a first area for displaying an image and a second area for displaying information about the image, and a storage means, wherein the control means, upon receiving an instruction from a user on the image to select a desired area within the image included in the first area, controls the second area to display characters corresponding to the area, the storage means stores information indicating the selected area for use in generating a file name, and, upon receiving an instruction from a user on the image to deselect the area, the control means is capable of controlling the second area to delete the characters corresponding to the selected area.

The present invention was made to address the above issues.

The application of the present disclosure is an application for causing a computer to function as a control means for controlling the computer to display a UI screen including a first area for displaying an image and a second area for displaying information about the image, and a storage means, wherein the control means, upon receiving a user instruction on the image to select a desired area within the image included in the first area, controls the computer to display characters corresponding to the area in the second area, the storage means stores information indicating the selected area for use in generating a file name, and, upon receiving a user instruction on the image to deselect the area, the control means is capable of controlling the computer to delete the characters corresponding to the selected area from the second area , and the information indicating the area is used to generate a file name for a file including another image obtained by scanning after the image .

According to the technology of the present disclosure, when an instruction from a user on an image to deselect an area is received, the characters corresponding to the selected area can be deleted .

A diagram showing the overall configuration of an image processing system Block diagram showing the hardware configuration of an MFP A block diagram showing the hardware configuration of an MFP collaboration server and a storage server. Block diagram showing the software configuration of the image processing system Sequence diagram showing the overall processing flow of the image processing system A diagram showing an example of the main screen An example of a login screen A diagram showing an example of a scan setting screen 1A is a diagram showing an example of a request ID, and FIGS. 1B and 1C are diagrams showing examples of responses to an inquiry about a processing status. A diagram showing an example of a file name setting screen A diagram showing an example of a learning request Flowchart showing the flow of processing in the MFP A diagram showing an example of a scanned image A diagram showing an example of a file name condition setting screen A diagram showing an example of a character recognition result A diagram showing an example of a similar document determination result A diagram showing an example of a scanned image Flowchart showing the flow of file name setting processing in the MFP Flowchart showing the flow of file name setting processing in the MFP A diagram showing the initial display part of the preview area A diagram showing an example of a file name setting screen A diagram showing an example of a soft keyboard FIG. 10 is a diagram showing an example of a file name setting screen on which operation guidance is displayed. FIG. 10 is a diagram showing an example of a file name setting screen when a touch operation is performed on a character area; A diagram showing the initial display area of the preview area when the initial magnification is adjusted

The following describes embodiments of the present invention with reference to the accompanying drawings. Please note that the following embodiments do not limit the scope of the invention as claimed, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[Embodiment 1]
<System Configuration>
1 is a diagram showing the overall configuration of an image processing system according to this embodiment. The image processing system includes an MFP (Multifunction Peripheral) 110 and server devices 120 and 130 that provide cloud services over the Internet. The MFP 110 is connected to the server devices 120 and 130 via the Internet so as to be able to communicate with them.

The MFP 110 is an example of an information processing device with a scanning function. The MFP 110 is a multifunction peripheral that has multiple functions, such as a printing function and a box storage function, in addition to the scanning function. The server devices 120 and 130 are both examples of information processing devices that provide cloud services. In this embodiment, the server device 120 provides a cloud service that performs image analysis on scanned images received from the MFP 110 and forwards requests from the MFP 110 to the server device 130 that provides another service. Hereinafter, the cloud service provided by the server device 120 will be referred to as an "MFP collaboration service." The server device 130 provides a cloud service (hereinafter referred to as a "storage service") that stores files sent via the Internet and provides saved files in response to requests from a web browser on a mobile device (not shown). In this embodiment, the server device 120 that provides the MFP collaboration service will be referred to as an "MFP collaboration server," and the server device 130 that provides the storage service will be referred to as a "storage server."

The configuration of image processing system 100 shown in FIG. 1 is an example and is not limited to this. For example, MFP 110 may also have the functions of MFP collaboration server 120. Furthermore, MFP collaboration server 120 may be connected to MFP 110 via a LAN (Local Area Network) rather than over the Internet. Furthermore, storage server 130 may be replaced with a mail server that provides mail distribution services, and the system may be used in situations where scanned images of documents are attached to mail and sent.

<MFP hardware configuration>
FIG. 2 is a block diagram showing the hardware configuration of the MFP 110. The MFP 110 is composed of a control unit 210, an operation unit 220, a printer unit 221, a scanner unit 222, and a modem 223. The control unit 210 is composed of the following units 211 to 219 and controls the overall operation of the MFP 110. The CPU 211 reads and executes various control programs (programs corresponding to various functions shown in the software configuration diagram described below) stored in the ROM 212. The RAM 213 is used as the CPU 211's main memory, work area, and other temporary storage areas. Note that in this embodiment, one CPU 211 uses one memory (RAM 213 or HDD 214) to execute each process shown in the flowcharts described below, but this is not limiting. For example, multiple CPUs and multiple RAMs or HDDs may cooperate to execute each process. The HDD 214 is a large-capacity storage unit that stores image data and various programs. The operation unit I/F 215 is an interface connecting the operation unit 220 and the control unit 210. The operation unit 220 is equipped with a touch panel, a keyboard, and the like, and accepts operations, inputs, and instructions from a user. Note that touch operations on the touch panel include operations with a person's finger and operations with a touch pen. The printer I/F 216 is an interface connecting the printer unit 221 and the control unit 210. Image data to be printed is transferred from the control unit 210 to the printer unit 221 via the printer I/F 216 and printed on a recording medium such as paper. The scanner I/F 217 is an interface connecting the scanner unit 222 and the control unit 210. The scanner unit 222 optically reads an original document set on a document tray or an ADF (Auto Document Feeder), not shown, to generate scanned image data, which is input to the control unit 210 via the scanner I/F 217. The scanned image data generated by the scanner unit 222 can be printed (copied) by the printer unit 221, saved in the HDD 214, or sent as a file to an external device such as the MFP collaboration server 120 via the LAN. The modem I/F 218 is an interface that connects the modem 223 to the control unit 210. The modem 223 transmits image data via facsimile to a facsimile device (not shown) on the PSTN. The network I/F 219 is an interface that connects the control unit 210 (MFP 110) to the LAN. The MFP 110 uses the network I/F 219 to transmit scanned image data to the MFP collaboration server 120 and receive various data from the MFP collaboration server 120. The hardware configuration of the MFP 110 described above is an example, and other components may be included or some components may be omitted as needed.

<Hardware configuration of server device>
3 is a block diagram showing the hardware configuration of the MFP collaboration server 120/storage server 130. The MFP collaboration server 120 and storage server 130 share a common hardware configuration, including a CPU 311, a ROM 312, a RAM 313, a HDD 314, and a network I/F 315. The CPU 311 controls overall operation by reading control programs stored in the ROM 312 and executing various processes. The RAM 313 is used as a temporary storage area, such as the CPU 311's main memory and work area. The HDD 314 is a large-capacity storage unit that stores image data and various programs. The network I/F 315 is an interface that connects the control unit 310 to the Internet. The MFP collaboration server 120 and storage server 130 receive various processing requests from other devices (such as the MFP 110) via the network I/F 315 and return processing results corresponding to the requests.

<Image processing system software configuration>
4 is a block diagram showing the software configuration of image processing system 100 according to this embodiment. Below, the software configurations corresponding to the respective roles of MFP 110, MFP cooperation server 120, and storage server 130 constituting image processing system 100 will be described in order. Note that, of the various functions possessed by each device, the following description will focus on functions related to the process of scanning a document, creating a file, and saving the file on storage server 130.

<MFP software configuration>
The function modules of the MFP 110 are broadly divided into two: a native function module 410 and an additional function module 420. The native function module 410 is an application that is provided as standard in the MFP 110, whereas the additional function module 420 is an application that is additionally installed in the MFP 110. The additional function module 420 is an application based on Java (registered trademark), and makes it easy to add functions to the MFP 110. Note that other additional applications (not shown) may also be installed in the MFP 110.

The native function module 410 has a scan execution unit 411 and a scan image management unit 412. The additional function module 420 has a display control unit 421, a scan control unit 422, a collaborative service request unit 423, and an image processing unit 424.

The display control unit 421 displays a user interface screen (UI screen) on the touch panel of the operation unit 220 to accept various user operations. Examples of various user operations include inputting login authentication information for accessing the MFP linkage server 120, scan settings, instructions to start scanning, file name settings, and instructions to save files.

In response to a user operation performed on the UI screen (for example, pressing the "Start Scan" button), the scan control unit 422 instructs the scan execution unit 411 to execute a scan process along with scan setting information. In accordance with the scan process execution instruction from the scan control unit 422, the scan execution unit 411 causes the scanner unit 240 to read a document via the scanner I/F 217 and generates scanned image data. The generated scanned image data is saved in the HDD 214 by the scanned image management unit 412. At this time, information about a scanned image identifier that uniquely identifies the saved scanned image data is notified to the scan control unit 422. The scanned image identifier is a number, symbol, alphabet, or the like that uniquely identifies an image scanned by the MFP 110. The scan control unit 422 obtains, for example, scanned image data to be converted into a file from the scanned image management unit 412 using the scanned image identifier. Then, the scan control unit 422 instructs the collaboration service request unit 423 to request the MFP collaboration server 120 for the processing required for the file conversion.

The collaboration service request unit 423 requests various processes from the MFP collaboration server 120 and receives their responses. These processes include, for example, login authentication, analysis of scanned images, and transmission of scanned image data. Communication protocols such as REST and SOAP are used for communication with the MFP collaboration server 120.

The image processing unit 424 performs predetermined image processing on the scanned image data to generate an image to be used on the UI screen displayed by the display control unit 421. Details of the predetermined image processing will be described later.

<Server device software configuration>
First, the software configuration of the MFP collaboration server 120 will be described. The MFP collaboration server 120 includes a request control unit 431, an image processing unit 432, a storage server access unit 433, a data management unit 434, and a display control unit 435. The request control unit 431 waits to receive a request from an external device and instructs the image processing unit 432, the storage server access unit 433, and the data management unit 434 to execute predetermined processes according to the content of the received request. The image processing unit 432 performs image analysis processes, such as character region detection, character recognition, and similar document determination, as well as image processing processes, such as rotation and tilt correction, on scanned image data sent from the MFP 110. The storage server access unit 433 makes processing requests to the storage server 130. Cloud services expose various interfaces for saving files to storage servers and retrieving saved files using protocols such as REST and SOAP. The storage server access unit 433 makes requests to the storage server 130 using the exposed interfaces. The data management unit 434 holds and manages user information, various setting data, and the like managed by the MFP cooperation server 120. The display control unit 435 receives requests from a web browser running on a PC or mobile terminal (neither of which are shown) connected via the Internet, and returns screen configuration information (HTML, CSS, etc.) required for screen display. The user can check registered user information and change scan settings via the screen displayed in the web browser.

Next, the software configuration of the storage server 130 will be described. The storage server 130 has a request control unit 441, a file management unit 442, and a display control unit 443. The request control unit 441 waits in a state where it can receive requests from external devices, and in this embodiment, instructs the file management unit 442 to save received files or read saved files in response to requests from the MFP-linked server 120. The request control unit 441 then returns a response in response to the request to the MFP-linked server 120. The display control unit 443 receives requests from a web browser running on a PC or mobile device (neither of which are shown) connected via the Internet, and returns screen configuration information (HTML, CSS, etc.) required for screen display. The user can view and obtain saved files via the screen displayed in the web browser.

<Processing flow of the entire image processing system>
FIG. 5 is a sequence diagram showing the flow of processing between devices when a document is scanned by the MFP 110, the resulting scanned image is converted into a file, and the file is saved on a storage server. FIG. 6 is a diagram showing an example of a main menu UI screen (hereinafter referred to as the "main screen") displayed when the MFP 110 is started. By scanning a document into a file and installing a dedicated application required for using a cloud storage service on the MFP 110, a "Scan and Save to Cloud Storage" button 601 appears on the main screen 600. When a user presses the "Scan and Save to Cloud Storage" button 601 from the menu buttons displayed on the main screen 600, the series of processes shown in the sequence diagram of FIG. 5 begins. Below, the interactions between devices are explained in chronological order according to the sequence diagram of FIG. 5. In the following explanation, the symbol "S" represents a step.

The MFP 110 displays on the operation unit 220 a UI screen (hereinafter referred to as the "login screen") for entering login authentication information to access the MFP-linked server 120 (S501). An example of the login screen is shown in Figure 7. When a user enters a pre-registered user ID and password in input fields 702 and 703 on the login screen 700 and presses the "Login" button 701, a login authentication request is sent to the MFP-linked server 120 (S502).

Upon receiving the login authentication request, the MFP-linked server 120 performs authentication processing using the user ID and password included in the request (S503). If the authentication processing confirms that the user is a legitimate user, the MFP-linked server 120 returns an access token to the MFP 110. Subsequently, the MFP 110 sends this access token along with various requests to the MFP-linked server 120, thereby identifying the currently logged-in user. In this embodiment, completing login to the MFP-linked server 120 also simultaneously completes login to the storage server 130. To achieve this, the user associates in advance, via a web browser on a PC (not shown) on the Internet, the user ID for using the MFP-linked service with the user ID for using the storage service. As a result, if login authentication to the MFP-linked server 120 is successful, login authentication to the storage server 130 is also completed simultaneously, eliminating the need for the user to log in to the storage server 130. The MFP-linked server 120 can then also handle requests for storage services from users who have logged in to its own device. Login authentication can be performed using commonly known methods (Basic authentication, Digest authentication, authorization using OAuth, etc.).

Once login is complete, the MFP 110 displays a UI screen for scan settings (hereinafter referred to as the "scan settings screen") on the operation unit 220 (S507). Figure 8 shows an example of the scan settings screen. The scan settings screen 800 includes a "Start Scan" button 801, a color settings field 802, and a resolution settings field 803. The "Start Scan" button 801 is used to start scanning a document placed on the platen (assuming a document such as an estimate or invoice in this embodiment). The color settings field 802 sets the color mode used during scanning. For example, full color or monochrome can be selected. The resolution settings field 803 sets the resolution used during scanning. For example, 600 dpi or 1200 dpi can be selected. Note that the color mode and resolution are merely examples of setting items; not all of these settings may be present, and other setting items may also be present. Furthermore, the color mode and resolution options may be limited to only the settings required by the storage service. The logged-in user sets detailed conditions for the scan process via this scan setting screen 800. After completing the scan settings, the logged-in user places the document to be scanned on the platen of the MFP 110 and presses the "Start Scan" button 801 to execute the scan (S505). This generates image data that is a digital version of the paper document. After the scan is complete, the MFP 110 sends the image data obtained by the scan to the MFP collaboration server 120 along with an analysis request for the image data (S506).

In the MFP collaboration server 120, upon receiving the request to analyze the scanned image, the request control unit 431 instructs the image processing unit 432 to perform image analysis (S507). At this time, the request control unit 431 returns a request ID that uniquely identifies the received analysis request to the MFP 110. An example of a request ID is shown in FIG. 9A. Meanwhile, upon receiving the instruction to perform analysis, the image processing unit 432 performs analysis on the scanned image (S508). This image analysis process first detects character regions within the scanned image. Character regions can be detected using known methods, such as extracting rectangular regions presumed to be characters from an image binarized using a certain threshold value. Next, character recognition (OCR: Optical Character Recognition) is performed on each character region found by the character region detection process. Then, a process (similar document determination process) is performed to determine whether the document currently being scanned is similar to documents scanned in the past. This similar document determination uses layout information indicating the location of each character region within the scanned image. Specifically, the layout information for a previously scanned image is compared with the layout information for the current scanned image to determine whether the layout of the character areas is similar. This is based on the fact that if the layout of the character areas is similar, it can be inferred that the forms were created using the same format. The layout information for previously scanned images used to determine similar forms is accumulated by the "form learning process" described below. The results obtained by these image analysis processes are passed to the request control unit 431. Note that in this embodiment, the similarity of forms is determined based solely on the degree of similarity in the layout of the character areas; however, for example, the type of form (quote, invoice, etc.) can be identified from the OCR results, and the obtained type information can also be taken into account to determine similarity.

While the image analysis process is being performed, the MFP 110 periodically (e.g., every several hundred milliseconds to a few milliseconds) queries the MFP-linked server 120 about the processing status using the request ID (S509-S509'). This query is repeated until an image analysis process completion response (S510) is received from the MFP-linked server 120. When the MFP-linked server 120 receives the query about the processing status, it checks the progress of the image analysis process corresponding to the request ID. If the process is not complete, it returns a response indicating "in progress" (see FIG. 9B). If the process is complete, it returns a response indicating "completed" (see FIG. 9C). The "status" in this response contains a string indicating the current processing status; specifically, if the process is currently being performed by the MFP-linked server 120, it contains "processing," and if the process is complete, it contains "completed." Note that a string indicating another status, such as "failed" if the process failed, may also be included. As shown in Figure 9 (c), the response upon completion of processing includes status information as well as information about the analysis results of the scanned image.

After receiving the processing completion response, the MFP 110 uses the URL included in the response indicating the storage location of the image analysis results to request the results of the image analysis processing from the MFP-linked server 120 (S511). The URLs in this case include "ocrResultUrl" and "matchingResultUrl." In response to the request, the request control unit 431 of the MFP-linked server 120 returns information about the results of the image analysis processing.

The MFP 110 then uses the acquired result information to display a UI screen for setting a file name (hereinafter referred to as the "file name setting screen") (S512). Figure 10 shows an example of the file name setting screen. When the user sets a file name and presses the Send button 1007, the MFP 110 first sends a learning request to the MFP collaboration server 120, including information about the input operations performed by the user when setting the file name (setting input information) (S513). Figure 11 shows an example of a learning request. The learning content is specified in "learningContent," which includes a "rectInfoArray" related to the character area used in the file name and a "metadataArray" related to the metadata. "rectInfoArray" contains coordinate information for the character area used when setting the file name. "metadataArray" contains information about the character area corresponding to the character string used in the file name and information about the delimiter (separator) associated with the character string. The example in Figure 11 shows that the file name consists of a string of characters in the character region "fileRegion0" at the beginning, followed by a delimiter, and finally a string of characters in the character region "fileRegion1".

In the MFP collaboration server 120 that received the learning request, the request control unit 431 instructs the image processing unit 432 to execute form learning processing (S514). In response to this instruction, the image processing unit 432 saves the layout information of each character area present in the scanned image and the input information (file name configuration information) when setting the file name included in the learning request received in S513 (S515).

The MFP 110 then sends a save request for the scanned image to the MFP-linked server 120, along with information such as the scanned image data and the file name set when the scanned image was created (S516). In the MFP-linked server 120 that received the save request, the request control unit 431 starts the file creation process and returns a response to the MFP 110 indicating that the save request was successfully received. Upon receiving this response, the MFP 110 terminates the process and returns to the scan settings screen display state (S504). Meanwhile, the MFP-linked server 120 obtains file format information from the pre-registered scan settings and creates a file of the scanned image in accordance with that file format (S517). At this time, the created file is given the file name specified in the save request. The scanned image file created in this way is sent to the storage server 130 and saved (S518).

This concludes the processing flow of the entire image processing system.

<MFP processing details>
Next, the processing when using a storage service in the above-mentioned image processing system will be described in detail, focusing on the operation of the MFP 110. Fig. 12 is a flowchart showing the flow of processing in the MFP 110. This series of processing is realized by the CPU 211 in the control unit 210 executing a control program stored in the HDD 214, and is started in response to pressing the "Scan and save to cloud storage" button 601 on the main screen 600 mentioned above. A detailed description will be given below with reference to the flowchart shown in Fig. 12. Note that the symbol "S" at the beginning of each processing step indicates a step.

In S1201, the display control unit 421 displays the aforementioned login screen 700. When a username and password are entered into the input fields 702 and 703 on the login screen 700, respectively, and pressing of the "Login" button 701 is detected, the process proceeds to S1202.

In S1202, the collaboration service request unit 423 sends a login authentication request to the MFP collaboration server 120. When the login authentication by the MFP collaboration server 120 confirms that the user is a legitimate user, the collaboration service request unit 423 receives an access token from the MFP collaboration server 120.

In S1203, the display control unit 421 of the MFP 110 that received the access token displays the aforementioned scan setting screen 800 on the operation unit 220. When the logged-in user places a document on the platen and detects that the "Start Scan" button 801 has been pressed, the display control unit 421 notifies the scan control unit 422 of this.

In S1204, upon receiving the above notification, the scan control unit 422 instructs the scan execution unit 411 to execute a scan process. In response to this instruction, the scan execution unit 411 scans the document placed on the platen. In explaining this flow, a document (more specifically, an estimate) will be used as an example to be scanned. As a result, a scanned image of the estimate as shown in FIG. 13 is obtained. The scanned image data generated by the scan is saved in the image data saving unit 412, and an identifier that can identify the scanned image is notified to the scan control unit 421. At this time, a message screen (not shown) indicating that scanning is in progress may be displayed on the operation unit 220.

Next, in S1205, the collaboration service request unit 423 obtains the scanned image data via the scan control unit 421 and sends it to the MFP collaboration server 120 along with an analysis request. Based on the analysis request, the MFP collaboration server 120 performs the image analysis process described above on the received scanned image data. At this time, a request ID is returned from the request control unit 431 of the MFP collaboration server 120.

In S1206, the collaboration service request unit 423, which has acquired the request ID, sends a processing status acquisition request together with the request ID to the MFP collaboration server 120. Based on the acquisition request, the MFP collaboration server 120 returns a response to the MFP 110 according to the progress status of the image analysis processing identified by the request ID.

In S1207, if the status in the received response is “processing,” indicating that processing is in progress, the linkage service request unit 423 returns to S1206. Then, after waiting for a predetermined time to elapse, the linkage service request unit 423 again sends a processing status acquisition request to the MFP linkage server 120. If the status in the received response is “completed,” indicating that processing is complete, the process proceeds to S1208. Here, the image analysis result information included in the response upon processing completion in this embodiment will be described with reference to the aforementioned FIG. 9C . The image analysis result information includes three types of information: “ocrResultUrl,” “matchingResultUrl,” and “formKeys.” “ocrResultUrl” is URL information for acquiring the results of the OCR processing, which is part of the image analysis processing. “matchingResultUrl” is URL information for acquiring the results of the similar form determination processing, which is part of the image analysis processing. “formKeys” is information on setting values for setting properties such as file name, metadata, and folder path to the scanned image, and is preset in the MFP linkage server 120. “formKeys” includes the following information:
・key: A value that uniquely indicates the setting value to be set on the scanned image ・keyType: A value that indicates the type of the key setting value ・value: The initial value of the setting value (for example, "scan" to indicate that it is a scanned image)
- type: A value indicating the type of value to be entered in the setting
(For example, "string" for strings, "number" for numbers, "date" for dates, etc.)
displayName: Display name when the setting screen is displayed on the touch panel of the MFP required: Value indicating whether it is necessary to input the setting value multiSelect: Value indicating whether to use a character string from multiple character areas in the file name separator: Contents of the delimiter that connects the character strings when using a character string from multiple character areas autoInput: Value indicating whether to automatically input the initial file name

Figure 9(c) shows the "formKeys" when setting a file name for a scanned image. Using the case of Figure 9(c) as an example, we will explain how the values of "value," "displayName," "required," and "multiSelect" affect the file name setting screen (Figure 10) displayed in the next step S1209. Currently, the content of "value" is empty. In this case, the initial value is not displayed in the file name input field 1001 of the file name setting screen 1000. Also, "displayName" is set to "file name." In this case, the string "file name," which indicates the title or label of the setting, is displayed at the top of the screen (in this example, in white text), as in the file name setting screen 1000 of Figure 10. Also, "required" is set to "true." In this case, if the file name input field 1001 is empty and no string is entered, the send button 1007 cannot be pressed. Also, "multiSelect" is set to "true." In this case, it is possible to select multiple character areas detected from the scanned image, thereby allowing a file name to be set by concatenating character strings corresponding to each character area. The above content included in these "formKeys" is set by the user via a file name condition setting screen displayed by the display control unit 435 of the MFP linkage server 120. FIG. 14 shows an example of the file name condition setting screen. A check box 1401 in the file name condition setting screen 1400 allows the user to select whether to automatically enter a file name using information about the character areas previously used when setting a file name for a scanned image. In setting field 1402, when multiple character strings corresponding to multiple character areas are used in a file name, the user sets the delimiter to be inserted between the character strings from a pull-down menu (e.g., underscore, hyphen, etc.). In this example, an underscore is selected. In setting field 1403, a character string (initial file name) is set to an input state when the file name setting screen ( FIG. 10 ) is initially displayed and presented to the user as a file name candidate. The content set in this file name condition setting screen 1400 is retained by the data management unit 434 and referenced when creating a response upon completion of processing.

In S1208, the collaboration service request unit 423 accesses the URL included in the response upon completion of processing to obtain the image analysis results. In the example of Figure 9 (c), information is stored in "ocrResultUrl" and "matchingResultUrl", so these two URLs are accessed to obtain the image analysis results. Table 1 below shows the coordinates of the character areas detected from the scanned image in Figure 13 and the character recognition results (recognized character strings) for each character area.

Figure 15 shows the character recognition results for the scanned image in Figure 13, obtained by accessing "ocrResultUrl." "imageWidth" indicates the number of pixels in the X direction (horizontal) of the scanned image being analyzed. "imageHeight" indicates the number of pixels in the Y direction (vertical) of the scanned image being analyzed. "regions" contains coordinate information ("rect") of the character area detected in the scanned image and string information ("Text") corresponding to that character area. "rect" contains information identifying the detected character area, where "x" is the X coordinate of the upper left corner of the character area, "y" is the Y coordinate of the upper left corner of the character area, "width" is the number of pixels in the X (width) direction of the character area, and "height" is the number of pixels in the Y (height) direction of the character area. "text" indicates the string information obtained by OCR processing the character area identified by "rect." This "rect" and "text" information is included for all character areas detected in the scanned image being analyzed. Note that some of this information is omitted in Figure 15.

Figure 16 shows the similar form determination results for the scanned image in Figure 13, obtained by accessing "matchingResultUrl." "matched" contains a value of "true" or "false" indicating whether the previously scanned image was found to be similar to the scanned image currently being analyzed through the similar form determination process. A value of "true" indicates that the scanned image currently being scanned is similar to a previously scanned form, while a value of "false" indicates that the scanned image is not similar. If a similar form is found, "formId" contains a value (ID value) that uniquely identifies the scanned image of the previously scanned similar form. On the other hand, if a similar form is not found, a newly created ID value that uniquely identifies the scanned image of the currently scanned form is contained. This newly created ID value is then used to learn the input information (file name configuration information) used when setting the file name for scanned images of forms for which no similar forms previously existed. "matchingScore" contains a value indicating the degree of match if a similar form is found. In this embodiment, similarity is expressed as a real value between 0 and 1 that indicates the degree of match between the layout information of text areas in a previous scanned image and the layout information of text areas in the current scanned image, with larger values indicating more similar forms. "rectInfoArray" contains information indicating the text areas in the scanned image being analyzed currently that correspond to the text areas used when setting the file name for the scanned image of the previous similar form.

In the example of Figure 16, a file name was previously set using the two character strings "Quotation" and "Shinagawa Co., Ltd." for the scanned image of Figure 13 obtained by scanning an estimate, and that input information was learned. After that, a quote created in the same format was scanned to generate the scanned image shown in Figure 17, and the similar form determination process was performed on this scanned image, which was determined to be similar to the scanned image of Figure 13. The example of Figure 16 shows that the information on the character areas used when setting the file name for the scanned image of Figure 13 has been stored as information on the character areas that will be automatically entered. The process leading up to the results of the similar form determination process shown in Figure 16 is explained below.

As a premise, it is assumed that the learning results based on the learning request in Figure 11 are saved in the data management unit 434. Then, in the similar form determination process, the coordinate information of the character areas of "Quotation" and "Shinagawa Co., Ltd." used when setting the previous file name is used to identify character areas in the character area contained in the current scanned image that partially overlap with the character area indicated by that coordinate information. The coordinate information of the character area identified as partially overlapping and the character string of that character area are then stored in "text" of "rectInfoArray." "key" contains a value that uniquely identifies the character area to be used for automatic file name entry. "region" stores the coordinate information of the character area uniquely indicated by "key" and the character string obtained by OCR processing of that character area. "rect" contains information that uniquely identifies the character area uniquely indicated by "key." In this case, "x" is the X coordinate of the upper left corner of the character area, "y" is the Y coordinate of the upper left corner of the character area, "width" is the number of pixels in the X (width) direction of the character area, and "height" is the number of pixels in the Y (height) direction of the character area. "text" contains information about the character string obtained by OCR processing the character area specified by "rect." "metadataArray" stores information indicating the order of the character areas used when automatically entering a file name and where the delimiters will be placed. If property information such as metadata is set in addition to the file name, the necessary information is added to "rectInfoArray" and "metadataArray." "key" stores a value that uniquely indicates the setting value to be set for the scanned image. "keyType" stores a value indicating the type of the "key" setting value. When used for a file name, "key" stores "filename" and "keyType" stores "filename." "value" stores information about the character area and delimiters used for the "key" value. In the example of Figure 16, the character string from the character area with the "key" of "fileRegion0" in "rectInfoArray" is first, the delimiter is second, and the character string from the character area with the "key" of "fileRegion1" is third. These characters are then automatically entered into the file name input field 1001 in this order and displayed as the initial file name.

In S1209, the display control unit 421 displays the file name setting screen 1000 described above on the operation unit 220, allowing the user to set a file name for the scanned image. Details of this file name setting process will be described later.

In S1210, the collaboration service request unit 423 references the response to the processing status request of S1206, determines whether automatic file name input is enabled, and determines the next step. Specifically, it references the value of "autoInput" included in the response at the time of processing completion shown in Figure 9 (c) above, and if the value is "true", which means that automatic input is enabled, proceeds to S1211. On the other hand, if the value is "false", which means that automatic input is not enabled, proceeds to S1214.

In S1211, the linked service request unit 423 determines whether a scanned image similar to the current scanned image existed in the past, based on the image analysis results obtained in S1208. Specifically, it references the value of "matched" included in the results of the similar form determination process shown in Figure 16 above, and if the value is "true", indicating that a similar image exists among the past scanned images, it proceeds to S1212. On the other hand, if the value is "false", indicating that a similar image does not exist, it proceeds to S1214.

In S1212, the collaboration service request unit 423 determines whether the user made any changes to the initial file name that was automatically entered into the file name input field 1001 during the file name setting process in S1209. Changes in this case include, for example, changing the character area used in the current file name to a different character area from that used in the initial file name, or adding or deleting delimiters. This determination is made by comparing the value of "value" of "metadataArray" included in the results of the similar form determination process in FIG. 16 with the information entered into the file name input field 1001 after the file name was set for the current scanned image. If the comparison reveals a difference, it is determined that a change has been made to the initial file name, and the process proceeds to S1213. If the comparison reveals a match, it is determined that no change has been made to the initial file name, and the process proceeds to S1214. It should be noted that even if the user has deleted the initial character string of the file name that was automatically entered using the "value" of "formKeys," the input information may not be considered to have changed, and the process may proceed to S1214.

In S1213, the collaboration service request unit 423 sends to the MFP collaboration server 120 a learning request (see Figure 11 above) for the input information at the time of setting, which indicates the structure of the file name set for the current scanned image. In this case, the "formId" included in the learning request uses the value of "formId" included in the similarity determination process result of the image analysis results obtained in S1208. In the example of Figure 16, the value of "formId" is "aaaaaaaa-ffff-49ab-acf8-55558888eeee", so this is stored in the learning request. Furthermore, "learningContent" stores the content of the input information used when setting the file name. First, "rectInfoArray" stores the coordinate information of the character area to which the character string used in the file name belongs, for the number of character areas. Then, "metadataArray" stores information on the character area and delimiter used to set the file name. Referring to Table 1 above, the user's operations at this time are explained as follows: In preview area 1002 of file name setting screen 1000, the user first touches number 1, "Quotation." Next, the user touches number 8, "Shinagawa Co., Ltd.", and finally presses send button 1007. Upon receiving the learning request, MFP-linked server 120 stores the input information based on these user operations, and will use the stored input information from the next image analysis process onwards. After sending the learning request, if a response is received from MFP-linked server 120 indicating that the learning request was successfully processed, the process proceeds to S1214.

In S1214, the collaboration service request unit 423 associates the scanned image data to be processed with the file name set in S1209 and transmits them to the MFP collaboration server 120. This completion of transmission ends the process. Note that in this embodiment, the scanned image data and its file name are transmitted last, but they may also be transmitted to the MFP collaboration server 120 in parallel with the scanning process being completed and the scanned image data being saved by the scanned image management unit 412.

The above is the processing flow when using a storage service, focusing on the operation of the MFP 110. Note that the above explanation assumes a situation in which a file name is set when a single scanned image obtained by scanning a single estimate is to be filed. However, this embodiment is not limited to this, and can also be applied to a situation in which a file name is set when multiple scanned images obtained by scanning multiple documents are to be filed. In this case, a button for turning pages may be provided within the file name setting screen 1000, allowing file names to be set from the text area of the scanned image on the second or third page.

(File name setting process)
Next, the file name setting process executed by the display control unit 421 in S1209 described above will be described in detail with reference to the flowcharts shown in FIGS.

In S1801, a file name (initial file name) that is initially displayed in the file name input field 1001 of the file name setting screen 1000 described above is generated. The initial file name is generated using the following procedure. First, the response received upon completion of the processing described above is referenced, and if a value is entered in the "value" of "formKeys", that value is set to the beginning of the file name. For example, if the string "scan" is entered as the "value" of "formKeys", "scan" is set to the beginning of the initial file name. Next, a predetermined string is set according to the similar form determination results obtained by accessing "matchingResultUrl". In other words, if the value of "matched" is "true" and the "matchingScore" is a certain number or greater (for example, 0.6 or greater), a string is set according to the file name structure specified in the "value" of "metadataArray". Specifically, the string of the character area specified in "rectInfoArray" is set in the order specified in "value". In the example of the similar forms determination results in Figure 16, the value of "matched" is "true" and the "matchingScore" is "0.74...", which is a numerical value above a certain level. The "value" of "metadataArray" specifies that the first element is "fileRegion0", followed by "separator", and finally by "fileRegion1". Therefore, the "text" value of "fileRegion0" ("quote"), the "separator" value included in the processing completion response ("_ (underscore)"), and the "text" value of "fileRegion1" ("Shimomaruko Co., Ltd.") are concatenated in this order. In this way, "quote_Shimomaruko Co., Ltd." is generated as the initial file name. If the "value" of "formKeys" contains a value (for example, "scan_"), the initial file name will be "scan_quote_Shimomaruko Co., Ltd." with that character prepended. In the similar form determination results, if the value of "matched" is "false" and the "matchingScore" is a numerical value below a certain value, and if the "value" of "formKeys" contains a string, that string will be used as the initial file name. Also, if the "value" of "formKeys" does not contain a string, there will be no initial file name (it will be blank). In this embodiment, the two values of "matched" and "matchingScore" are used as the conditions for generating the initial file name, and the information specified in "rectInfoArray" in accordance with the order of the "value" of "metadataArray". However, this is not limited to this, and only one of the results may be used as the condition. For example, the initial file name may be generated only if the value of "matched" is "true", or the file name may be generated only if the value indicating the degree of similarity is equal to or greater than a threshold.

In S1802, the process determines how much the scanned image should be reduced when initially displayed in the preview area 1002 of the file name setting screen 1000 (hereinafter referred to as the "initial scaling factor"). This initial scaling factor is determined using the character recognition results obtained by accessing "ocrResultUrl." First, the leftmost and rightmost character areas are determined from all character areas detected in the scanned image. In this case, the leftmost character area is the character area with the smallest x-coordinate value among all character areas. The rightmost character area is the character area with the largest sum of the x-coordinate value and the character area width value among all character areas. The initial scaling factor is determined as the scaling factor such that the left side of the leftmost character area thus determined coincides with the left side of the preview area 1002, and the right side of the rightmost character area coincides with the right side of the preview area 1002. Note that if the edges of the determined character area and the preview area are perfectly aligned, it may be difficult for the user to select (press) the character area if the character area is small. Therefore, the initial scaling factor may be determined with a certain margin. For example, the initial scaling factor may be determined so that a 10-pixel margin is secured from the left edge of the character area that will be displayed on the leftmost side of the preview area 1002 and from the right edge of the character area that will be displayed on the rightmost side. Furthermore, if UI components such as various operation buttons are superimposed on the preview area 1002, the initial scaling factor may be determined so that the character area does not overlap the UI components. Specifically, the width of the UI components may be subtracted from the width of the preview area 1002, and the initial scaling factor may be determined so that the leftmost and rightmost character areas fit within the width of the remaining area. In the example of the file name setting screen 1000 in Figure 10 mentioned above, there are three operation buttons (enlarge button 1004, reduce button 1005, and return button 1006) in the preview area 1002, so the initial magnification is determined so that the text area does not overlap these buttons. Also, header information related to the printing process may be printed near the top, bottom, left, and right edges of the document to be scanned, and it is unlikely that the text in the text area corresponding to such header information will be adopted as part of the file name. Therefore, text areas that exist within a certain distance (e.g., 50 pixels) from the top, bottom, left, and right edges of the scanned image may be excluded when determining the leftmost and rightmost text areas.

In S1803, it is determined which portion of the scanned image after scaling using the initial scaling factor determined in S1802 will be initially displayed in the preview area 1002 of the file name setting screen 1000. In the scanned image after scaling using the scaling factor determined in S1802, the width of the leftmost and rightmost character areas approximately matches the width of the preview area 1002. Therefore, first, the portion to be initially displayed (hereinafter referred to as the "initial display portion") is determined so that the left and right character areas exactly fit within the preview area 1002 in the horizontal direction, for example, so that the left edge of the leftmost character area coincides with the left edge of the preview area 1002. Then, the initial display portion in the vertical direction is determined so that the topmost character area is positioned near the top edge of the preview area 1002. If margins were taken into consideration when determining the initial scaling factor, the initial display portion can be determined taking those margins into account as well. For example, character areas within a certain distance (e.g., 50 pixels) from the top of the scanned image may be excluded from the display, and the topmost character area among the remaining character areas may be displayed at the top of the preview area 1002. FIG. 20 is a diagram showing the initial display area determined for the scanned image of FIG. 13. The line frame in FIG. 20 indicates the range of the scanned image initially displayed in the preview area 1002. Note that as a result of determining the initial display area as described above, if a similar form exists and an initial file name is automatically entered in the file name input field 1001, the character area of the character string used in the initial file name may not be displayed in the preview area 1002. In this case, the initial display area may be determined so that the character area of the character string used in the initial file name is displayed in the center of the preview area 1002. At this time, the character string constituting the initial file name automatically entered in the file name input field 1001 and the character area of the character string used in the initial file name may each be highlighted (e.g., by changing the color). FIG. 21 shows an example of the scan settings screen 1000 when the character string "Kawasaki Co., Ltd." is automatically entered as the initial file name for the scanned image of FIG. 13 . Here, the character area corresponding to the character string "Kawasaki Co., Ltd." is positioned in the center of the preview area 1002, and the color of this character area and the character string "Kawasaki Co., Ltd." entered in the file name input field 1001 are displayed in different colors. If the character size of the highlighted character area is small, the initial scaling factor may be adjusted to an appropriate character size. This adjusted initial scaling factor can be obtained, for example, based on the size of the touch panel of the operation unit 220 and the range initially displayed in the preview area 1002. Specifically, the character size is calculated based on the height of the character area of the character string used in the initial file name, and the initial preview scaling factor is adjusted so that the calculated point size is equal to or greater than a threshold (predetermined size).

In S1804, the file name setting screen 1000 described above is displayed on the operation unit 220 using the processing results of S1801 to S1803. The user uses the file name setting screen 1000, an example of which is shown in FIG. 10, to set a file name for the scanned image to be saved on the storage server 130. This is explained in detail below.

1) File Name Input Field When the user touches the file name input field 1001, the display on the touch panel of the operation unit 220 switches to a soft keyboard, allowing the user to input any characters or edit the automatically input character string. For example, when the user touches the character string portion of the file name input field 1001 in which "Kawasaki Co., Ltd." has been automatically input, as in the example of FIG. 21 described above, the soft keyboard 2200 shown in FIG. 22 is displayed. At this time, as shown in FIG. 22, the input area 2201 is displayed with the touched character string entered. This allows the user to operate the soft keyboard 2200 to correct the touched character string "Kawasaki Co., Ltd." Furthermore, when the user touches the file name input field 1001 in which no character string has been entered, as shown in FIG. 10, the soft keyboard 2200 is displayed with the input area 2201 empty, allowing the user to operate the soft keyboard 2200 to enter any character string into the file name input field 1001.

2) Preview Area As described above, the preview area 1002 displays a specific portion of the scanned image after scaling. When a user touches and selects a text area in the preview area 1002, the text in the selected text area is added to the file name input field 1001. To indicate that a text area has been selected, the selected text area may be highlighted by underlining, surrounding it with a frame, adding a color, or other means. The display area may also be resized or the display magnification may be changed so that the selected text area is centered. Furthermore, when multiple text areas are selected, the display mode for each text area may be different, such as by adding a different color to each text area. When the user touches a selected text area again, the selection is canceled (i.e., the lines and colors added to the text area are removed and the text in the file name input field 1002 is restored to its original state), and the text corresponding to the selected text area is also deleted. In this embodiment, when a text area is unselected, a frame or the like indicating each text area is not displayed in the preview area 1002. However, a frame or the like indicating each character area may be displayed in advance so that the user can recognize the touchable portion, and the color of the frame or the like may be changed when the character area is selected. Alternatively, the user may use a button or the like to switch whether or not to display the character area so that it can be recognized. The user can then move the displayed image portion by swiping in the preview area 1002.

3) Various Operation Buttons The button 1003 with an "x" mark is a delete button for deleting the last character string of the character string being input into the file name input field 101. The button 1004 with a "+" mark is a zoom-in button for enlarging the image currently displayed in the preview area 1002 and is used when, for example, wanting to display each character area larger. The button 1005 with a "-" mark is a zoom-out button for reducing the image currently displayed in the preview area 1002 and is used when, for example, wanting to display a wider range. When zooming in or out, the display position is adjusted so that the coordinates of the center of the preview area 1002 remain the same as before zooming in or out. The button 1006 with a "U-turn arrow" mark is a return button for returning the preview area 1002 to its initial display state after moving the display position by a swipe operation or changing the display magnification by pressing the zoom-in button 1004 or the zoom-out button 1005. The "Send" button 1007 is a button for converting the scanned image into a file with the file name set on the file name setting screen 1000 and saving the file in the storage server 130. In response to pressing the "Send" button 1007, the scanned image data is sent to the MFP cooperation server 120 together with information about the set file name.

In S1805, operation guidance is displayed within the file name setting screen 1000 in its initial display state. This operation guidance allows even first-time users to easily understand that a file name can be set by touching and selecting a text area within the preview area 1002. Figure 23 shows a state in which message 2301 serving as operation guidance is superimposed above the preview area 1002 of the file name setting screen 1000 shown in Figure 10. Because such operation guidance is unnecessary for users who already know how to operate the screen, the operation guidance is hidden upon detection of a user operation, such as a touch operation or swipe operation. Furthermore, the top of a typical document often contains characters or numbers that are important to the document, such as the document title, company name, form number, or date. Therefore, to prevent the operation guidance from overlapping with the text area of these important characters, the image portion displayed in the preview area 1002 may be shifted downward, depending on the attributes of the character string. Furthermore, if the operation guidance overlaps one of the character areas or comes within a certain distance of the character area, the display position can be determined so that it does not overlap or get too close. In this way, by flexibly determining the display position of the operation guidance rather than fixing it, it becomes easier for the user to select a character area.

S1806 is a process for monitoring whether or not a user operation has occurred. The presence or absence of a user operation is checked at predetermined intervals, and if any user operation is detected, the process proceeds to S1807. Then, in S1807, as described above, the operation guidance is hidden.

Steps S1808 to S1815 are determination processes that determine the next process based on the content of the detected user operation. Once the process corresponding to the detected user operation is complete, the process returns to step S1806 and waits for the next user operation to be detected.

In S1808, it is determined whether the detected user operation was a touch operation on any of the character areas in the preview area 1002. Specifically, the coordinates of the touched position are first converted to correspond to the coordinates of the scanned image. Then, if the converted coordinates are included in any of the character areas identified by "regions" in the character recognition result (see FIG. 15) described above, it is determined that the operation touched a character area. If the determination shows that the operation touched a character area, the process proceeds to S1816; if the operation did not touch a character area, the process proceeds to S1809. In S1816, if the operation touched a character area, it is determined whether the touched character area has already been selected. If the touched character area was an unselected character area, the process proceeds to S1817; if the touched character area was a selected character area, the process proceeds to S1818.

In S1817, the character string in the touched character area is added to the file name input field 1001, and the character area is highlighted to indicate that it has been selected. At this time, if no other character string has been entered in the file name input field 1001, the character string is added as is. The fact that the character area was used to set the file name is then stored as input information at the time of file name setting. On the other hand, if another character string has already been entered in the file name input field 1001, a delimiter is added, and then the character string in the touched character area is added. In this case, in addition to the fact that the character area was used to set the file name, the insertion of a delimiter between the character strings, such as "first character area," "delimiter," and "second character area," is also stored. Figure 24 shows the file setting screen 1000 when the character area recognized as "quote" is first selected, followed by the character area recognized as "Shimomaruko Co., Ltd." Two character strings corresponding to the order of the touched character areas are entered in the file name input field 1001, with an underscore inserted between the two character strings as a separator.

In S1818, the character string in the touched character area is deleted from the file name input field 1001, the highlighting of the character area indicating that it has been selected is canceled, and the display returns to the normal state indicating that it is not selected. Furthermore, if the character string includes a delimiter, the delimiter is also deleted. For example, assume that the file name structure used to generate the initial file name is ["first character area", "delimiter", "second character area"]. In this case, if the user touches the character area that is the "first character area", the delimiter is deleted along with the character string in the first character area, and only the character string in the "second character area" remains in the file name input field 1001.

If the detected user operation was not a touch on the text area, in S1809, it is determined whether the detected user operation was a swipe operation within the preview area 1002. If a swipe operation was detected, the process proceeds to S1819; if not, the process proceeds to S1810.

In S1819, processing is performed to move the image portion displayed in preview area 1002 in the swipe direction. Specifically, the image portion displayed in preview area 1002 is updated depending on the amount of movement of the user's finger on the touch panel. This processing may be performed only when a touch operation is detected in an area other than the text area in preview area 1002. Furthermore, even if the touch operation is on the text area, it may be treated as a swipe operation rather than a touch on the text area depending on the amount the user subsequently moves their finger.

In S1810, it is determined whether the detected user operation was a press of the Zoom In button 1004 or the Zoom Out button 1005. If the operation was a press of the Zoom In button 1004 or the Zoom Out button 1005, the process proceeds to S1820; if not, the process proceeds to S1811.

In S1820, a process is executed to enlarge or reduce the image portion displayed in the preview area 1002. Specifically, if the enlarge button 1004 is pressed, the magnification of the scanned image is increased by a fixed amount, and if the reduce button 1005 is pressed, the magnification of the scanned image is decreased by a fixed amount. The fixed amount here is arbitrary, but is, for example, approximately 10% based on the current magnification. In this case, if an initial magnification or magnification at which the width/height of the scanned image matches the width/height of the preview area 1002 exists before the fixed amount of enlargement or reduction is performed, the magnification may be kept at that value. Note that the scanned image will not be reduced below a magnification at which the entire scanned image fits within the preview area 1002 (minimum magnification), and is controlled so that it cannot be enlarged above a magnification at which the smallest character area becomes a fixed size (maximum magnification).

In S1811, it is determined whether the detected user operation was a press of the return button 1006. If it was a press of the return button 1006, proceed to S1821; if not, proceed to S1812.

In S1821, processing is performed to return the preview area 1002 to its initial display state. Specifically, the magnification of the scanned image displayed in the preview area 1002 is returned to the initial magnification determined in S1802, and the display area in the preview area 1002 is returned to the initial display area determined in S1803. At this time, if the initial magnification or initial display area has been adjusted, it is desirable to return it to the initial magnification and initial display area after the adjustments.

In S1812, it is determined whether the detected user operation was a touch operation on the character string entered in the file name input field 1001. At this time, if a touch operation on the blank area after the character string entered in the file name input field 1001 is detected, it can be treated as a touch operation on the last character string. If it is a touch operation on the character string entered in the file name input field 1001, proceed to S1822; if not, proceed to S1813.

In S1822, the editing soft keyboard is displayed on the operation unit 220 with the touched character string entered in the input area 2201. Assume now that the file name setting screen 1000 is in the state shown in FIG. 24. In this state, if a touch operation on the character string "Estimate" in the file name input field 1001 is detected, the soft keyboard 2200 is displayed with the character string "Estimate" entered in the input area 2201 of the soft keyboard. Furthermore, if a touch operation on the character string "Shimomaruko Co., Ltd." or the blank area following it is detected, the soft keyboard 2200 is displayed with the final character string, "_Shimomaruko Co., Ltd.", entered in the input area 2201. Note that operation of the soft keyboard 2200 is the same as operation of a general keyboard, and therefore a description thereof will be omitted. The following S1823 is a process for monitoring whether or not a user operation has occurred. The system checks at predetermined intervals whether the OK button 2202 on the soft keyboard 2200 has been pressed, and if pressing of the OK button 2202 is detected, the system proceeds to S1824. If a key other than the OK button is pressed, the character string displayed in the input area 2201 is edited according to the pressed key. For example, the user can use the soft keyboard 2200 to correct character strings that have been incorrectly recognized during character recognition processing.

In S1824, it is determined whether a change to delete a delimiter has been made. In this embodiment, this determination is made only if the character string touched in S1812 is displayed with a delimiter in S1822; character strings that are not displayed with a delimiter are treated as if the delimiter has not been deleted. That is, if the character string is displayed with a delimiter added to the beginning in S1822, when the OK button 2202 is pressed, it is determined whether the character string has been edited to delete the delimiter based on whether a delimiter is present at the beginning of the character string. For example, if the character string "Shimomaruko Co., Ltd." is touched in S1812, the input area 2201 of the soft keyboard 2200 displays "_Shimomaruko Co., Ltd." with the delimiter added at the beginning, as described above. Therefore, by checking whether a delimiter remains at the beginning, it is determined whether the delimiter has been deleted. Furthermore, when the character string "quote" is touched in S1812, "quote" is displayed in the input area 2201 as described above, so in this case, it is treated as if no change to delete the delimiter was made.

In the example described above, if a delimiter character exists before the character string touched in S1812, the character string with the delimiter character added to the beginning is displayed in input area 2201 in S1822. However, this is not limited to this. For example, if a delimiter character exists after the character string touched in S1812 and the touched character string is displayed in input area 2201 with the delimiter character added to the end in S1822, a similar determination can be made in S1824. That is, in the example of Figure 24, a delimiter character is added to the end of input area 2201 of soft keyboard 2200 in response to a touch operation on the character string "quote." In this case, it is possible to determine whether the delimiter character has been deleted by checking whether a delimiter character remains at the end in step S1824.

If it is determined in S1824 that the delimiter has been deleted, the process proceeds to S1825. Then, in S1825, it is stored that the delimiter has been deleted from the file name. For example, suppose the file name structure used to generate the initial file name was ["first character area", "delimiter", "second character area"]. At this time, if the character string corresponding to "second character area" in the file name input field 1001 is touched and the delimiter is deleted using the soft keyboard 2200, the changed file name structure will become ["first character area", "second character area"]. Then, the information about this changed file name structure will be retained.

Next, in S1826, it is determined whether a change has been made to add a delimiter to the file name. If it is determined that a delimiter has been added, the process proceeds to S1827. Then, in S1827, it is stored that a delimiter has been added to the file name. For example, suppose the file name structure used to generate the initial file name was ["first character area", "second character area"]. At this time, if the character string corresponding to "second character area" in the file name input field 1001 is touched and a delimiter is added to the beginning using the soft keyboard 2200, the changed file name structure will be ["first character area", "delimiter", "second character area"]. Then, the information about this changed file name structure is retained.

In step S1828, the display screen of the operation unit 220 is returned to the file name setting screen 1000 from the soft keyboard 2200, and the file name is revised to reflect the changes. At this time, the display portion of the scanned image may be changed so that the character area corresponding to the revised character string is centered in the preview area 1002. Even if part of the character string used in the file name is revised, if the file name structure remains unchanged, no new information about the file name structure is retained. However, if a revision is made by deleting all characters from a certain character string, the file name structure also changes. Therefore, even in this case (although this is omitted from the flowchart), the structure information of the revised file name is retained. For example, suppose the file name structure used to generate the initial file name was ["first character area," "delimiter," "second character area"]. At this time, suppose the character string corresponding to the "second character area" in the file name input field 1001 is touched and all characters are deleted using the soft keyboard 2200. In this case, the unnecessary delimiter is also deleted, and the revised file name structure changes to ["first character area"], so the information about the revised file name structure is retained.

In S1813, it is determined whether the detected user operation was a touch operation on the blank file name input field 1001 where no string has been entered. If it was a touch operation on the blank file name input field 1001, the process proceeds to S1829; if not, the process proceeds to S1814.

In S1829, the soft keyboard for adding a character string is displayed on the operation unit 220 with nothing entered in the input area 2201. The following S1830 is a process for monitoring whether or not a user operation has occurred. The process checks at predetermined intervals whether or not the OK button 2202 on the soft keyboard 2200 has been pressed, and if pressing of the OK button 2202 is detected, the process proceeds to S1831. Then, in S1831, the display screen of the operation unit 220 is returned from the soft keyboard 2200 to the file name setting screen 1000, and the entered character string is added to the file name.

In S1814, it is determined whether the detected user operation was a press of the Delete button 1003. If the Delete button 1003 was pressed, the process proceeds to S1832; if not, the process proceeds to S1815. In S1832, the last character of the string entered in the file name input field 1001 is deleted. For example, if the file name structure used to generate the initial file name is ["First character area", "Delimiter", "Second character area", pressing the Delete button 1003 deletes the character string corresponding to the "Second character area". As a result, the file name structure after deletion becomes ["First character area 1"], and the file name will contain only the character string corresponding to the "First character area". At this time, the character area corresponding to the deleted character string is returned to a state indicating that it is not selected in the preview area 1002. The information about the changed file name structure is then retained.

In S1815, it is determined whether the detected user operation was a press of the Send button 1007. If the Send button 1007 was pressed, this processing ends and the process proceeds to S1210 in the flowchart of Figure 12.

The above is the content of the file name setting process executed in S1209.

<Modification 1>
In the above-described step S1802 (determining the initial scaling factor), if the average height of each character area displayed in preview falls below a predetermined threshold value when the initial scaling factor is determined based on the leftmost and rightmost character areas, the initial scaling factor may be changed so that it is equal to or greater than the predetermined threshold value. In this case, the "average height" may be the average for all character areas or the average for a portion of the character areas. Examples of the average for a portion of the character areas include the average height of the character areas displayed in the preview area 1002 at the determined initial scaling factor, or the average height of the character areas in a specific portion of the scanned image (e.g., the upper half). A specific example will be described. Assume that the distance (width) between the left edge of the leftmost character area and the right edge of the rightmost character area is 2000 px, the width of the preview area 1002 is 800 px, and the predetermined threshold value is 15 px. In this example, if the average height of all character areas is 60 px, the calculated initial scaling factor is 0.4 (= 800 ÷ 2000). Since the average height of all character areas is 24 px (= 60 × 0.4), which is not below the predetermined threshold, 0.4 becomes the initial scaling factor. On the other hand, if the average height of all character areas is 30 px, the average height of all character areas is 12 px (= 30 × 0.4). Since this is below the predetermined threshold, the initial scaling factor is changed to 0.5 (= 0.4 × 15 ÷ 12) so that the average height of all character areas is equal to or greater than the predetermined threshold (here, the same value as the threshold). Figure 25 shows an example of the initial display of the scan setting screen 1000 when the initial scaling factor is adjusted because the average height of the character areas does not satisfy the threshold. In this example, after the initial scaling factor is increased, the display area is determined in step S1803 to match the left edge of the leftmost character area and the top edge of the topmost character area. As a result, the area corresponding to the right side of the form is not included in the display portion, but the visibility of each character area present in the preview area 1002 is ensured.

<Modification 2>
In the above-described step S1803 (determining the initial display area), if there are multiple character areas used in the initial file name, the topmost character area among the used character areas may be displayed in the preview area 1002. Alternatively, the semantic type of the character string in each character area may be determined, and the display area may be adjusted so that the character area most likely to be used in the file name is displayed in the preview area 1002. Alternatively, the display area may be determined so that a pre-specified number of character areas are displayed in the preview area 1002. Furthermore, the display area may be determined so that the midpoint between the topmost and bottommost character areas of the character string used in the initial file name is located in the center of the preview area 1002.

<Modification 3>
In the above-described step S1805 (display of operation guidance), if an initial file name based on the file name structure of a similar form is automatically entered in the file name input field 1001, the operation guidance may be displayed so as not to overlap the character area of the character string used in the initial file name. In this case, if multiple character strings are used in the initial file name, it may be difficult to display the operation guidance without overlapping the character areas of those character strings. In such cases, the character area of the first character string in the initial file name may be given priority, and the operation guidance may be displayed overlapping part or all of the character area of the last character string. Furthermore, when setting document properties other than the initial file name, such as metadata, the character area where the operation guidance should not overlap may be determined based on the type of the document property, and the display position of the operation guidance may be determined accordingly. For example, when setting a value for a date-type document property, the operation guidance is displayed so as not to overlap the character area of the date in the target document. Whether the character string corresponding to each character area is a date is determined by whether the character string extracted from the scanned image matches a specific character sequence. For example, if numbers are entered in the "yy,""mm," and "dd" fields in the yy/mm/dd format, it can be determined to be a date.

<Modification 4>
In the above-described step S1822 (display of the editing soft keyboard), instead of displaying the touched character string in an input state in the input area 2201, the entire file name may be displayed in an input state. In this case, to prevent the user from misidentifying the character string to be edited, the character string may be highlighted (by changing the color, adding a frame, etc.) to distinguish it from other character strings, or the cursor may be displayed at the end of the character string, so that the range of the character string to be edited can be clearly seen. Also, instead of displaying the editing soft keyboard in response to a touch operation on the character string input in the file name input field 1001, the editing soft keyboard may be displayed after waiting for an additional user operation (for example, pressing a dedicated button, not shown).

<Modification 5>
In addition, in the description of this embodiment, when a user's touch operation on a character area is detected, a process (S1817, S1818) is executed to add or delete the character string in the touched character area to the file name. However, the system response is slow, and it may take some time from the detection of the touch operation until the addition or deletion of the target character string is reflected in the file name. In this case, the user may mistakenly believe that the system is not responding to their touch operation and repeatedly touch the same character area. This may result in repeated addition and deletion of character string in the same character area, making it impossible for the user to stop the process as desired. For example, touching the character string to be deleted multiple times may result in the character string being re-entered. Therefore, to prevent repeated touch operations on the same character area during the addition or deletion process, a new touch operation for addition or deletion on the same character area may not be accepted until the addition or deletion process is completed. At this time, a message indicating that processing is in progress may also be displayed. This prevents the user from unintentionally adding or deleting character strings during input operations when setting a file name.

(Other Examples)
The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., an ASIC) that realizes one or more of the functions.

Claims (48)

Computer,
a control means for controlling the display of a UI screen including a first area for displaying an image and a second area for displaying information about the image;
storage means,
An application for functioning as
the control means, when receiving an instruction from a user on the image for selecting a desired area in the image included in the first area, controls to display characters corresponding to the selected area in the second area;
the saving means saves information indicating the selected area for use in generating a file name;
the control means is capable of performing control such that, when receiving an instruction from a user on the image to cancel the selection of the area, the character corresponding to the selected area is deleted from the second area;
The information indicating the area is used to generate a file name for a file containing another image obtained by scanning after the image.
An application characterized by: 2. The application according to claim 1, wherein the saving means saves information indicating the selected area for use in generating a file name for any file containing an image of the same type. The application described in claim 1 or 2, characterized in that when the control means receives an instruction from a user to select an area in the image included in the first area that has not yet been selected while no text is displayed in the second area, the control means controls the display of text corresponding to that area in the second area. the region is a region containing characters,
4. The application of claim 1, wherein the user's instruction on the image to deselect the area is an instruction to touch the selected area in the image included in the first area. the saving means is capable of saving information indicating a plurality of areas selected on the image within the UI screen,
The application according to any one of claims 1 to 4 , wherein the file name is generated using characters corresponding to the plurality of areas. The application according to claim 5 , wherein the plurality of areas are selected by the user through separate operations on the image in the UI screen. The application according to claim 5 or 6 , wherein a different frame is displayed in each of the selected areas on the image in the UI screen. The application according to claim 5 , wherein the information indicating the selected plurality of regions is stored in association with different labels. Computer,
a receiving means for receiving an instruction to execute scanning from a user;
a control means for controlling the display of a UI screen including a first area for displaying a scanned image obtained by executing a scan with the scanner based on the instruction, and a second area for displaying information about the scanned image;
storage means,
An application for functioning as
the control means, upon receiving an instruction from a user on the scanned image for selecting a desired area in the scanned image included in the first area, controls to display characters corresponding to the selected area in the second area;
the storage means stores information indicating the selected area for use in generating a file name;
the control means is capable of performing control such that, when receiving an instruction from a user on the scanned image to deselect the area, the character corresponding to the selected area is deleted from the second area;
An application characterized in that the information indicating the area is used to generate a file name of a file including another scanned image obtained by scanning after the scanned image . The application according to claim 9 , wherein the control means, upon receiving an instruction to execute the scan from a user, displays the scanned image without receiving any additional instructions from the user. 11. The application according to claim 9, wherein the storage means stores information indicating the selected area for use in generating file names for any files containing the same type of scanned image. The application described in any one of claims 9 to 11, characterized in that when the control means receives an instruction from a user to select an area in the scanned image included in the first area that has not yet been selected while no character is displayed in the second area, the control means controls the display of character corresponding to that area in the second area. the region is a region containing characters,
13. The application of claim 9, wherein the user's instruction on the scanned image to deselect the area is an instruction to touch the selected area in the scanned image that is included in the first area. the saving means is capable of saving information indicating a plurality of areas selected on the scanned image within the UI screen;
The application according to any one of claims 9 to 13 , wherein the file name is generated using characters corresponding to the plurality of areas. The application according to claim 14 , wherein the plurality of regions are selected by the user on the scanned image in the UI screen through separate operations. The application according to claim 14 or 15 , wherein a different frame is displayed on the scanned image in the UI screen for each of the selected areas. 17. The application according to claim 14 , wherein information indicating the selected plurality of regions is stored in association with different labels. The application according to claim 1 , wherein the information indicating the area is information indicating a position of the area. The application according to any one of claims 1 to 18, characterized in that the user's instruction for selecting the area is an instruction in which the user specifies a position on the first area, and the area is identified based on that position. The application according to any one of claims 1 to 19 , wherein the character corresponding to the area is a character recognized by character recognition processing for the area. 21. The application of claim 20 , wherein the character recognition process is performed before the region is selected. 22. The application of any one of claims 1 to 21 , wherein the first area and the second area do not overlap. An application according to any one of claims 1 to 22 , characterized in that when the character is deleted from the second area, information indicating the area corresponding to the deleted character is not used in generating the file name. a control means for controlling the display of a UI screen including a first area for displaying an image and a second area for displaying information about the image;
A storage means;
An information processing device having:
the control means, when receiving an instruction from a user on the image for selecting a desired area in the image included in the first area, controls to display characters corresponding to the selected area in the second area;
the storage means stores information indicating the selected area for use in generating a file name;
the control means is capable of performing control such that, when receiving an instruction from a user on the image to cancel the selection of the area, the character corresponding to the selected area is deleted from the second area;
10. An information processing apparatus according to claim 9, wherein the information indicating the area is used to generate a file name of a file including another image obtained by scanning after the image . 25. The information processing apparatus according to claim 24 , wherein said saving means saves information indicating the selected area for use in generating a file name for any file containing an image of the same type. The information processing device described in claim 24 or 25, characterized in that when the control means receives an instruction from a user to select an area in the image included in the first area that has not yet been selected while no characters are displayed in the second area, the control means controls the display of characters corresponding to that area in the second area. the region is a region containing characters,
27. The information processing device according to claim 24, wherein the instruction from the user on the image to deselect the area is an instruction to touch the selected area in the image included in the first area. the saving means is capable of saving information indicating a plurality of areas selected on the image within the UI screen,
28. The information processing apparatus according to claim 24 , wherein the file name is generated using characters corresponding to the plurality of areas. The information processing apparatus according to claim 28 , wherein the plurality of regions are selected by the user on the image in the UI screen through separate operations. 30. The information processing apparatus according to claim 28 , wherein a different frame is displayed in each of the selected areas on the image in the UI screen. 31. The information processing apparatus according to claim 28 , wherein the information indicating the selected plurality of regions is stored in association with different labels. a receiving means for receiving an instruction to execute scanning from a user;
a control means for controlling the display of a UI screen including a first area for displaying a scanned image obtained by executing a scan with the scanner based on the instruction, and a second area for displaying information about the scanned image;
A storage means;
An information processing device having:
the control means, upon receiving an instruction from a user on the scanned image for selecting a desired area in the scanned image included in the first area, controls to display characters corresponding to the selected area in the second area;
the saving means saves information indicating the selected area for use in generating a file name;
the control means is capable of performing control such that, when receiving an instruction from a user on the scanned image to deselect the area, the character corresponding to the selected area is deleted from the second area;
The information processing apparatus according to claim 1, wherein the information indicating the area is used to generate a file name of a file including another scanned image obtained by scanning after the scanned image . 33. The information processing apparatus according to claim 32 , wherein said control means, upon receiving an instruction to execute said scan from a user, displays said scanned image without receiving any additional instructions from the user. 34. The information processing apparatus according to claim 32, wherein the storage means stores information indicating the selected area for use in generating a file name for any file containing the same type of scanned image. The information processing device according to any one of claims 32 to 34, characterized in that when the control means receives an instruction from a user to select an area in the scanned image included in the first area that has not yet been selected while no character is displayed in the second area, the control means controls the display of character corresponding to that area in the second area. the region is a region containing characters,
36. The information processing device of claim 32, wherein the user's instruction on the scanned image to deselect the area is an instruction to touch the selected area in the scanned image that is included in the first area. the saving means is capable of saving information indicating a plurality of areas selected on the scanned image within the UI screen;
37. The information processing apparatus according to claim 32 , wherein the file name is generated using characters corresponding to the plurality of areas. The information processing apparatus according to claim 37 , wherein the plurality of regions are selected by the user on the scanned image in the UI screen through separate operations. 39. The information processing apparatus according to claim 37 , wherein a different frame is displayed in each of the selected areas on the scanned image in the UI screen. 40. The information processing apparatus according to claim 37 , wherein the information indicating the selected plurality of regions is stored in association with different labels. 41. The information processing apparatus according to claim 24 , wherein the information indicating the area is information indicating a position of the area. 42. The information processing device according to claim 24 , wherein the instruction from the user for selecting the area is an instruction by the user to specify a position on the first area, and the area is identified based on the position. 43. The information processing apparatus according to claim 24 , wherein the character corresponding to the area is a character recognized by character recognition processing for the area. 44. The information processing apparatus according to claim 43 , wherein the character recognition process is performed before the region is selected. 45. The information processing apparatus according to claim 24 , wherein the first area and the second area do not overlap. 46. An information processing device according to claim 24, wherein, when the character is deleted from the second area, information indicating the area corresponding to the deleted character is not used in generating the file name. a control step of controlling to display a UI screen including a first area for displaying an image and a second area for displaying information about the image;
The preservation process and
A control method comprising:
In the control step, when an instruction on the image from a user for selecting a desired area in the image included in the first area is received, control is performed so that a character corresponding to the selected area is displayed in the second area;
In the saving step, information indicating the selected area is saved for use in generating a file name;
In the control step, when an instruction on the image from a user to cancel the selection of the area is received, the control can be performed so that the character corresponding to the selected area is deleted from the second area ,
A control method, characterized in that the information indicating the area is used to generate a file name of a file including another image obtained by scanning after the image . a receiving step of receiving an instruction to execute scanning from a user;
a control step of controlling to display a UI screen including a first area for displaying a scanned image obtained by executing a scan with the scanner based on the instruction, and a second area for displaying information about the scanned image;
The preservation process and
A control method comprising:
In the control step, when an instruction is received from a user on the scanned image for selecting a desired area in the scanned image included in the first area, control is performed so that a character corresponding to the selected area is displayed in the second area;
In the saving step, information indicating the selected area is saved for use in generating a file name;
In the control step, when an instruction to cancel the selection of the area is received from a user on the scanned image, the control can be performed so that the character corresponding to the selected area is deleted from the second area ;
A control method , characterized in that the information indicating the area is used to generate a file name of a file including another scanned image obtained by scanning after the scanned image .