JP6794162B2 - Text mining methods, text mining programs, and text mining equipment - Google Patents

Description

The present invention relates to text mining, and more particularly to a text mining method for displaying an analysis result of text data on a screen, a text mining program, and a text mining device.

In recent years, text mining, which analyzes a large amount of freely described text data and seeks useful information from the analysis results, has attracted attention. In text mining, for example, words are extracted from the text data to be analyzed, and information is obtained by analyzing the appearance frequency and appearance tendency of the words.

Hereinafter, a text mining device that performs hierarchical cluster analysis on words extracted from text data and displays the analysis results on the screen will be examined. In hierarchical cluster analysis, clusters containing words with high similarity are created hierarchically based on the similarity between words. Generally, the result of the hierarchical cluster analysis is provided to the user (analyst) using the tree diagram (dendrogram) shown in FIG.

In relation to the present invention, Patent Document 1 has a hierarchical clustering means for constructing a tree diagram, searching the tree diagram, generating an index capable of identifying the upper layer from the lower layer, and storing the index in the storage means. A clustering device is described. Patent Document 2 describes a distance matrix calculation means that calculates the distance between keywords, generates distance matrix data capable of searching the distance between keywords from the keywords, and stores the distance matrix data in the storage means, and a hierarchy of keywords using the distance matrix. A query providing device having a clustering means for storing the constructed dendrogram as a bottom-up index that can be searched from the lower layer to the upper layer in the storage means is described.

Japanese Unexamined Patent Publication No. 2011-216021 Japanese Unexamined Patent Publication No. 2012-150539

The conventional text mining device displays the result of the hierarchical cluster analysis on the screen using a tree diagram. However, such a text mining device has a problem that the user cannot intuitively understand the analysis result. For example, when the number of clusters is set to 4 in the analysis result shown in FIG. 15, the user sets a cutting line on the tree diagram as shown in FIG. However, the user cannot intuitively recognize the words contained in each cluster just by looking at such a tree diagram. In addition, the user cannot intuitively grasp how the words included in each cluster change when the number of clusters is changed when the number of words is large.

Moreover, since the frequency of occurrence of words is not described in the tree diagram, the user cannot know which word is important. Further, when the text data to be analyzed is time-series data having information such as date and time, the user may request to know the temporal change of the analysis result. However, the conventional text mining device cannot meet such a demand of the user.

Therefore, it is an object of the present invention to provide a text mining method, a text mining program, and a text mining device that display the results of hierarchical cluster analysis on a screen so that the user can intuitively understand them.

The first aspect of the present invention is a text mining method for displaying an analysis result of text data on a screen.
A text analysis step that performs a hierarchical cluster analysis on words extracted from the input text data,
A screen generation step that generates screen data based on the analysis result of the text analysis step,
An analysis result display step for displaying a screen based on the screen data is provided.
In the screen generation step, based on the number of groups and the maximum number of data in the group, the cluster of the number of groups is obtained from the analysis result, and the group containing the words included in the cluster is displayed on the screen. It generates the screen data for,
The said group, as the name, frequency of the words included in the cluster corresponding to the group is characterized that you have given the highest word.

The second aspect of the present invention is the first aspect of the present invention.
The words included in the group are selected from the words included in the cluster corresponding to the group in descending order of frequency of occurrence.

The third aspect of the present invention is the second aspect of the present invention.
The group is characterized in that it has a size corresponding to the total frequency of occurrence of words included in the cluster corresponding to the group in the screen.

The fourth aspect of the present invention is the third aspect of the present invention.
The words included in the group are characterized in that they have a size corresponding to the frequency of appearance of the words in the screen.

The fifth aspect of the present invention is the first aspect of the present invention.
It also has an instruction input step for inputting instructions from the user.
One of the text analysis step and the screen generation step is executed based on the instruction input in the instruction input step.

The sixth aspect of the present invention is the fifth aspect of the present invention.
The instruction input step receives the setting instruction of the number of groups and receives the instruction.
The screen generation step is characterized in that the screen data is generated based on the number of groups set in the instruction input step.

The seventh aspect of the present invention is the fifth aspect of the present invention.
The instruction input step receives the setting instruction of the maximum number of data and receives the instruction.
The screen generation step is characterized in that the screen data is generated based on the maximum number of data set in the instruction input step.

The eighth aspect of the present invention is the fifth aspect of the present invention.
The instruction input step receives the setting instruction of the analysis target period and receives the instruction.
The text analysis step is characterized in that the hierarchical cluster analysis is performed on the words included in the text data within the analysis target period set in the instruction input step among the text data.

The ninth aspect of the present invention is the fifth aspect of the present invention.
The instruction input step receives the setting instruction for analysis purpose and receives the setting instruction.
The text analysis step is characterized in that the hierarchical cluster analysis is performed by extracting a type of word according to the analysis purpose set in the instruction input step from the text data.

The tenth aspect of the present invention is the fifth aspect of the present invention.
The instruction input step receives the word exclusion instruction and receives the word exclusion instruction.
The text analysis step is characterized in that the hierarchical cluster analysis is performed by excluding the words specified in the instruction input step.

The eleventh aspect of the present invention is the fifth aspect of the present invention.
The instruction input step receives a synonym registration instruction and receives
The text analysis step is characterized in that a plurality of words instructed in the instruction input step are regarded as the same word and the hierarchical cluster analysis is performed.

The twelfth aspect of the present invention is the fifth aspect of the present invention.
The instruction input step receives the compound word registration instruction and receives the compound word registration instruction.
The text analysis step is characterized in that a plurality of words instructed in the instruction input step are merged into one word to perform the hierarchical cluster analysis.

The thirteenth aspect of the present invention is the first aspect of the present invention.
The screen generation step is characterized in that screen data for displaying an analysis result screen including the group and an analysis setting screen for setting a display mode of the analysis result screen is generated.

The fourteenth aspect of the present invention is a text mining program that displays the analysis result of text data on a screen.
A text analysis step that performs a hierarchical cluster analysis on words extracted from the input text data,
A screen generation step that generates screen data based on the analysis result of the text analysis step,
Based on the screen data, the CPU causes the computer to execute the analysis result display step of displaying the screen using the memory.
In the screen generation step, based on the number of groups and the maximum number of data in the group, the cluster of the number of groups is obtained from the analysis result, and the group containing the words included in the cluster is displayed on the screen. It generates the screen data for,
The said group, as the name, frequency of the words included in the cluster corresponding to the group is characterized that you have given the highest word.

The fifteenth aspect of the present invention is the fourteenth aspect of the present invention.
The words included in the group are selected from the words included in the cluster corresponding to the group in descending order of frequency of occurrence.

The sixteenth aspect of the present invention is the fifteenth aspect of the present invention.
The group is characterized in that it has a size corresponding to the total frequency of occurrence of words included in the cluster corresponding to the group in the screen.

The seventeenth aspect of the present invention is the sixteenth aspect of the present invention.
The words included in the group are characterized in that they have a size corresponding to the frequency of appearance of the words in the screen.

The eighteenth aspect of the present invention is the fourteenth aspect of the present invention.
The computer is further executed with an instruction input step for inputting an instruction from the user.
One of the text analysis step and the screen generation step is executed based on the instruction input in the instruction input step.

The nineteenth aspect of the present invention is the fourteenth aspect of the present invention.
The screen generation step is characterized in that screen data for displaying an analysis result screen including the group and an analysis setting screen for setting a display mode of the analysis result screen is generated.

A twentieth aspect of the present invention is a text mining device that displays an analysis result of text data on a screen.
A text analysis unit that performs hierarchical cluster analysis on words extracted from the input text data,
A screen generation unit that generates screen data based on the analysis results of the text analysis unit,
It is equipped with an analysis result display unit that displays the screen based on the screen data.
The screen generation unit obtains a cluster of the number of groups from the analysis result based on the number of groups and the maximum number of data in the group, and displays a group containing words included in the cluster in the maximum number of data or less on the screen. It generates the screen data for,
The said group, as the name, frequency of the words included in the cluster corresponding to the group is characterized that you have given the highest word.

The 21st aspect of the present invention is the 20th aspect of the present invention.
The words included in the group are selected from the words included in the cluster corresponding to the group in descending order of frequency of occurrence.

The 22nd aspect of the present invention is the 21st aspect of the present invention.
The group is characterized in that it has a size corresponding to the total frequency of occurrence of words included in the cluster corresponding to the group in the screen.

The 23rd aspect of the present invention is the 22nd aspect of the present invention.
The words included in the group are characterized in that they have a size corresponding to the frequency of appearance of the words in the screen.

The twenty-fourth aspect of the present invention is the twentieth aspect of the present invention.
Further equipped with an instruction input unit for inputting instructions from the user,
One of the text analysis unit and the screen generation unit operates based on the instruction input by the instruction input unit.

The 25th aspect of the present invention is the 20th aspect of the present invention.
The screen generation unit is characterized in that screen data for displaying an analysis result screen including the group and an analysis setting screen for setting a display mode of the analysis result screen is generated.

According to the first, 14th, or twentieth aspect of the present invention, a group containing the words contained in the cluster is displayed on the screen based on the result of performing a hierarchical cluster analysis on the words contained in the text data. To. In addition, the number of words included in the group is limited to the maximum number of data or less. Therefore, the user can intuitively understand the result of the hierarchical cluster analysis when looking at the screen.

According to the second, fifteenth or twenty-first aspect of the present invention, words having a high frequency of occurrence among the words included in the cluster are displayed inside the group. Therefore, the user can easily recognize the frequently occurring words included in each cluster.

According to the third, 16th or 22nd aspect of the present invention, the group has a size corresponding to the total frequency of occurrence of words contained in the cluster in the screen. Therefore, the user can easily recognize the cluster in which the total frequency of occurrence of words is large.

According to the fourth, 17th or 23rd aspect of the present invention, the word has a size in the screen according to the frequency of the word. Therefore, the user can easily recognize words that frequently appear.

According to the fifth, eighteenth or twenty-fourth aspect of the present invention, the display mode of the result of the hierarchical cluster analysis can be switched according to the instruction from the user.

According to the sixth aspect of the present invention, the number of groups (number of clusters) displayed on the screen can be switched according to an instruction from the user.

According to the seventh aspect of the present invention, the upper limit of the number of words included in the group can be switched according to the instruction from the user.

According to the eighth aspect of the present invention, the result of performing the hierarchical cluster analysis on the words included in the text data within the analysis target period instructed by the user is displayed on the screen. Therefore, the user can easily recognize the temporal change of the result of the hierarchical cluster analysis.

According to the ninth aspect of the present invention, it is possible to display on the screen the result of performing the hierarchical cluster analysis by switching the type of the word to be analyzed according to the analysis purpose instructed by the user.

According to the tenth aspect of the present invention, the result of performing the hierarchical cluster analysis excluding the word instructed by the user can be displayed on the screen.

According to the eleventh aspect of the present invention, it is possible to display on the screen the result of performing the hierarchical cluster analysis by regarding a plurality of words instructed by the user as the same word.

According to the twelfth aspect of the present invention, it is possible to display on the screen the result of performing a hierarchical cluster analysis by merging a plurality of words instructed by the user into one word.

According to the thirteenth , nineteenth or twenty-fifth aspect of the present invention, the analysis result screen and the analysis setting screen are displayed. Therefore, the user can easily switch the display mode of the result of the hierarchical cluster analysis by using the analysis setting screen.

It is a block diagram which shows the structure of the text mining apparatus which concerns on embodiment of this invention. It is a block diagram which shows the structure of the computer which functions as the text mining apparatus shown in FIG. It is a figure which shows the display screen of the text mining apparatus shown in FIG. It is a flowchart which shows the operation of the text mining apparatus shown in FIG. It is a flowchart of the screen data generation processing of the text mining apparatus shown in FIG. It is a figure which shows the data designation screen of the text mining apparatus shown in FIG. It is a figure which shows the example of the text data input to the text mining apparatus shown in FIG. It is a figure which shows the purpose designation screen of the text mining apparatus shown in FIG. It is a figure which shows the synonym list selection screen of the text mining apparatus shown in FIG. It is a figure which shows the compound word list selection screen of the text mining apparatus shown in FIG. It is a figure which shows the analysis result screen before and after setting the analysis target period in the text mining apparatus shown in FIG. It is a figure which shows the analysis result screen before and after performing word exclusion in the text mining apparatus shown in FIG. It is a figure which shows the analysis result screen before and after the synonym registration in the text mining apparatus shown in FIG. It is a figure which shows the analysis result screen before and after performing compound word registration in the text mining apparatus shown in FIG. It is a figure which shows the example of the tree diagram. It is a figure which shows the state which set the cluster number in the tree diagram shown in FIG.

Hereinafter, the text mining method, the text mining program, and the text mining apparatus according to the embodiment of the present invention will be described with reference to the drawings. The text mining method according to the present embodiment is typically executed using a computer. The text mining program according to the present embodiment is a program for implementing a text mining method using a computer. The text mining device according to the present embodiment is typically configured by using a computer. The computer that executes the text mining program functions as a text mining device.

FIG. 1 is a block diagram showing a configuration of a text mining device according to an embodiment of the present invention. The text mining device 10 shown in FIG. 1 includes an instruction input unit 11, a text analysis unit 12, a screen generation unit 13, and an analysis result display unit 14. The text data 5 to be analyzed is input to the text mining device 10. The text mining device 10 performs a hierarchical cluster analysis on the words extracted from the input text data 5, and displays the analysis result on the screen.

The outline of the operation of the text mining device 10 is as follows. An instruction from the user is input to the instruction input unit 11. The text analysis unit 12 extracts words from the input text data 5, and performs a hierarchical cluster analysis on the extracted words. The screen generation unit 13 generates screen data based on the analysis result by the text analysis unit 12. The analysis result display unit 14 displays a screen based on the screen data generated by the screen generation unit 13.

The instruction from the user input to the instruction input unit 11 includes setting the number of groups, setting the maximum number of data in the group, setting the analysis target period, word exclusion, synonym registration, compound word registration, and the like. When the text data 5 is time-series data having information such as date and time, the text analysis unit 12 within the analysis target period set by the instruction input unit 11 among the input text data 5. Perform a hierarchical cluster analysis on the words contained in the text data of.

The screen generation unit 13 follows the number of groups and the maximum number of data in the group when generating screen data (details will be described later). Further, when the user inputs a new instruction, the screen generation unit 13 generates new screen data and the analysis result display unit 14 displays the new screen after the instructed process is performed. In this way, the text mining device 10 switches between the analysis mode of the text data 5 and the display mode of the analysis result in response to an instruction from the user.

FIG. 2 is a block diagram showing a configuration of a computer functioning as a text mining device 10. The computer 20 shown in FIG. 2 includes a CPU 21, a main memory 22, a storage unit 23, an input unit 24, a display unit 25, a communication unit 26, and a storage medium reading unit 27. For the main memory 22, for example, DRAM is used. For example, a hard disk or a solid state drive is used for the storage unit 23. The input unit 24 includes, for example, a keyboard 28 and a mouse 29. For the display unit 25, for example, a liquid crystal display is used. The communication unit 26 is an interface circuit for wired communication or wireless communication. The storage medium reading unit 27 is an interface circuit of the storage medium 30 that stores programs and the like. As the storage medium 30, for example, a non-transient storage medium such as a CD-ROM, a DVD-ROM, or a USB memory is used.

When the computer 20 executes the text mining program 31, the storage unit 23 stores the text mining program 31 and the text data 5. The text mining program 31 and the text data 5 may be, for example, those received from a server or another computer using the communication unit 26, or those read from the storage medium 30 using the storage medium reading unit 27.

When the text mining program 31 is executed, the text mining program 31 and the text data 5 are copied and transferred to the main memory 22. The CPU 21 uses the main memory 22 as a working memory and executes the text mining program 31 stored in the main memory 22 to process the text data 5 stored in the main memory 22. At this time, the computer 20 functions as a text mining device 10. The configuration of the computer 20 described above is only an example, and the text mining device 10 can be configured by using any computer.

FIG. 3 is a diagram showing a display screen of the text mining device 10. The display screen 40 shown in FIG. 3 includes an analysis result screen 41 and an analysis setting screen 42. The analysis result screen 41 displays the analysis result by the text analysis unit 12. On the analysis setting screen 42, a graphical user interface component for setting the analysis mode in the text analysis unit 12 and the characteristics of the screen data generated by the screen generation unit 13 is displayed.

Setting the number of clusters for the results of a hierarchical cluster analysis determines the words contained in each cluster. When the text mining device 10 displays the result of hierarchical cluster analysis on the words extracted from the text data 5 on the screen, the group corresponding to the cluster is shown in FIG. 3 instead of the tree diagram. Display with.

In the following description, the cluster displayed on the screen is also referred to as a group. The user uses the instruction input unit 11 to specify the number of groups (number of clusters) and the maximum number of data in the group (upper limit of the number of words included in the group). Hereinafter, the former will be m and the latter will be n.

In the text mining device 10, the words included in the text data 5 are classified into m clusters, and each cluster contains one or more words. M groups are displayed on the analysis result screen 41, and words are displayed inside each group. The group is displayed using a cloud shape, and the words contained in the group are displayed inside the elliptical area. The number of words included in each group is limited to n or less. For example, when a cluster contains 10 words when n = 5, 5 words are displayed inside the group on the analysis result screen 41.

On the analysis setting screen 42, the first slider for setting the number of groups m, the two first buttons (with the symbol "+" or "-"), and the maximum number of data n in the group are set. A second slider and two second buttons for the purpose, and four boxes and two third buttons (with a left-pointing arrow or a right-pointing arrow) for setting the analysis target period are displayed.

The user operates the mouse 29 to move the knob of the first slider to the left or right, or presses the first button to instruct the number of groups m. The number of groups m increases when the first button with the symbol "+" is pressed, and decreases when the first button with the symbol "-" is pressed. The initial value of the number of groups m is set to, for example, the square root of the word type included in the analysis result by the text analysis unit 12, or an integer close to this. For example, when the analysis result by the text analysis unit 12 includes 16 kinds of words, the initial value of the number of groups m is set to 4.

The user operates the mouse 29 to move the knob of the second slider to the left or right, or presses the second button to instruct the maximum number of data n in the group. The maximum number of data n in the group increases or decreases when the second button is pressed. The initial value of the maximum number of data n in the group is set to 5, for example.

When the text data 5 is time series data, the user operates the keyboard 28 or the mouse 29 to specify the date and time using the four boxes, or presses the third button. Indicate the analysis period. The analysis target period moves to the past by a predetermined amount (for example, one month) when the third button with a left-pointing arrow is pressed, and moves in the opposite direction by a predetermined amount when the third button with a right-pointing arrow is pressed. Moving. The initial value of the analysis target period is set to, for example, the period from the oldest time to the newest time of the text data 5. If the text data 5 is not time series data, the user cannot specify the analysis target period.

Groups of 1 or more and m or less are displayed on the analysis result screen 41, and 1 or more and n or less words are displayed inside each group. Each group is displayed larger on the screen as the total frequency of occurrence of words contained in the corresponding cluster increases. When the number of words contained in the cluster exceeds n, n words with a high frequency of occurrence are displayed inside the group. The words included in the group and the elliptical area containing them are displayed larger on the screen as the frequency of occurrence of the words increases. Each group is given a name. For the name of the group, the word that appears most frequently among the words included in the cluster is used. The name of the group is underlined inside the group. If the word cannot be displayed inside the elliptical area, the symbol "..." is displayed instead of the word.

On the analysis result screen 41, a third slider for designating the zoom magnification and two fourth buttons (with the symbol “+” or “−”) are displayed. The user operates the mouse 29 to move the knob of the third slider to the left or right, or presses the fourth button to set the zoom magnification. On the analysis result screen 41, a group including a word is displayed in an enlarged or reduced size according to a set zoom magnification. The initial value of the zoom magnification is set to 100%. All groups are displayed on the analysis result screen 41 in the initial state.

When the user changes the number of groups m, the maximum number of data n in the group, or the analysis target period on the analysis setting screen 42, the content of the analysis result screen 41 changes accordingly. When the user instructs word exclusion, synonym registration, or compound word registration on the analysis result screen 41, the content of the analysis result screen 41 changes accordingly.

The text mining device 10 stores an excluded word list that stores words to be excluded, a synonym list that stores words to be processed as synonyms, and a synonym list that stores words to be excluded when performing hierarchical cluster analysis on the words extracted from the text data 5. Refer to the compound word list that stores the words to be processed as compound words. In the synonym list, a plurality of words having the same meaning (or almost the same meaning) are stored in association with one word representing these words. In the compound word list, a plurality of words that are concatenated into one compound word and a compound word in which these words are concatenated are stored in association with each other. In the synonym list, for example, "university student" and "student" and "university student" representing both are stored in association with each other. In the compound word list, for example, "patience" and "strong" and "patience" that connects the two are stored in association with each other. The text mining device 10 may have a plurality of synonym lists and a plurality of compound word lists.

FIG. 4 is a flowchart showing the operation of the text mining device 10. FIG. 5 is a flowchart showing details of the screen data generation process (step S111 shown in FIG. 4) of the text mining device 10. The input unit 24 and the CPU 21 that executes step S113 function as the instruction input unit 11. The CPU 21 that executes steps S109 to S110 functions as the text analysis unit 12. The CPU 21 that executes step S111 functions as a screen generation unit 13. The display unit 25 and the CPU 21 that executes step S112 function as the analysis result display unit 14. Hereinafter, the operation of the text mining device 10 will be described with reference to FIGS. 4 and 5.

First, the CPU 21 causes the display unit 25 to display the data designation screen 51 shown in FIG. 6 (step S101). On the data designation screen 51, a box for specifying a file name and a box for specifying a folder name are displayed. The user specifies the text data 5 to be analyzed by designating the file name or the folder name on the data designation screen 51. The text data 5 may be stored in a storage unit 23 such as a hard disk, or may be stored in a server or another computer connected by using the communication unit 26.

Next, the CPU 21 transfers the text data 5 designated using the data designation screen 51 to the main memory 22. As a result, the text data 5 is input to the text mining device 10 (step S102). FIG. 7 is a diagram showing an example of text data 5. The text data shown in FIG. 7 is report data created by a university student, and is time-series data having date information. The type of text data 5 analyzed by the text mining device 10 is arbitrary.

Next, the CPU 21 causes the display unit 25 to display the purpose designation screen 52 shown in FIG. 8 (step S103). On the purpose designation screen 52, three radio buttons corresponding to the contents, features, and reputation are displayed. By operating the mouse 29 and pressing any radio button, the user selects the analysis purpose from the contents, features, and reputation. Next, the CPU 21 receives the analysis purpose specified by using the purpose designation screen 52. As a result, the analysis purpose is input to the text mining device 10 (step S104).

Next, the CPU 21 causes the display unit 25 to display the synonym list selection screen 53 shown in FIG. 9 (step S105). On the synonym list selection screen 53, the name of the synonym list included in the text mining device 10 and the synonyms registered in each synonym list are displayed. The user specifies a synonym list to be used by operating the mouse 29 to select one of the synonym lists on the synonym list selection screen 53. As a result, the text mining device 10 selects a synonym list (step S106).

Next, the CPU 21 causes the display unit 25 to display the compound word list selection screen 54 shown in FIG. 10 (step S107). On the compound word list selection screen 54, the name of the compound word list included in the text mining device 10 and the compound words registered in each compound word list are displayed. The user specifies a compound word list to be used by operating the mouse 29 to select one of the compound word lists on the compound word list selection screen 54. As a result, the text mining device 10 selects the compound word list (step S108).

Next, the CPU 21 considers the excluded word list, the synonym list, and the compound word list, and analyzes the text data 5 input in step S102 from the text data within the analysis target period specified in step S104. A type of word according to the purpose is extracted (step S109). When the purpose of analysis is "content", the CPU 21 extracts nouns, proper nouns, place names, and personal names from the text data 5. When the purpose of analysis is "feature", the CPU 21 extracts nouns, proper nouns, sa-variant nouns, and verbs from the text data 5. When the purpose of analysis is "reputation", the CPU 21 extracts adjectives, adjective verbs, and interjections from the text data 5. The text mining device 10 may support analysis purposes other than the above three. Further, the CPU 21 may extract a word of a type different from the above according to each analysis purpose.

When the text data 5 is time-series data, the CPU 21 extracts a word from only the text data included in the analysis target period instructed by the user among the text data 5 when executing step S109. .. Further, when the word W1 is stored in the excluded word list, the CPU 21 ignores all the words W1 included in the text data 5 when executing step S109. Further, when the words W2 and W3 and the word W2 representing both of them are stored in association with each other in the selected synonym list, the CPU 21 includes the word W2 and the word W3 in the text data 5 when the step S109 is executed. All the words W3 are processed as words W2. Further, when the words W4 and W5 and the word W6 in which both are connected are stored in association with each other in the selected compound word list, the CPU 21 stores the word W4 and the word W5 in the text data 5 when the step S109 is executed. All the consecutive words W4 and W5 included are processed as words W6.

Next, the CPU 21 performs a hierarchical cluster analysis on the words extracted in step S109 (step S110). In step S110, the CPU 21 obtains the similarity between the two words based on, for example, the distance between the two words in the text data 5 (how far the two words appear). The CPU 21 performs a hierarchical cluster analysis using a predetermined method (for example, the shortest distance method, the longest distance method, the group average method, the decimal method, the Ward method, etc.) based on the similarity between the obtained words. Further, the CPU 21 obtains the frequency of occurrence of each word in step S110.

Next, the CPU 21 generates screen data for displaying the analysis result based on the result of the hierarchical cluster analysis obtained in step S110 (step S111). The CPU 21 performs the process shown in FIG. 5 in step S111.

The CPU 21 has m as the number of groups and n as the maximum number of data in the group (step S201). Next, the CPU 21 sets the number of clusters to m for the result of the hierarchical cluster analysis, and obtains m clusters (step S202). Next, the CPU 21 obtains the total frequency of occurrence of words included in the cluster for each cluster (step S203). Next, the CPU 21 determines the display size of each group based on the total appearance frequency obtained in step S203 (step S204). In step S204, the larger the total frequency of occurrence of words included in the cluster, the larger the display size of the group is determined.

Next, the CPU 21 selects a word to be displayed from the words included in the cluster for each cluster (step S205). In step S205, n or less words are selected from the words included in each cluster in descending order of frequency of occurrence. Next, the CPU 21 determines the display size of the word for each word selected in step S205 based on the frequency of occurrence of the word (step S206). In step S206, the more frequently the word appears, the larger the display size of the word is determined.

Next, the CPU 21 generates screen data for displaying the result of the hierarchical cluster analysis (step S207). The screen data generated in step S207 includes m groups (represented by cloud figures) having the size determined in step S204. Within each group are n or less words having the size determined in step S206. Words are displayed inside the group on the screen. The CPU 21 ends the screen data generation process after executing step S207.

Next, the CPU 21 causes the display unit 25 to display a screen based on the screen data generated in step S111 (step S112). Next, the CPU 21 receives an instruction from the user (step S113). Next, the CPU 21 proceeds to any of steps S115 to S120 according to the type of instruction received in step S113 (step S114).

If the instruction received in step S113 is "setting the number of groups", the CPU 21 proceeds to step S115. In this case, the CPU 21 sets the number of groups m to the value specified by the user (step S115), and proceeds to step S111. After that, screen data is generated based on the set number of groups m, and a new screen is displayed. As a result, the analysis result screen including the specified number of groups is displayed.

If the instruction received in step S113 is "setting the maximum number of data in the group", the CPU 21 proceeds to step S116. In this case, the CPU 21 sets the maximum number of data n in the group to a value specified by the user (step S116), and proceeds to step S111. After that, screen data is generated based on the maximum number of data n in the set group, and a new screen is displayed. As a result, the analysis result screen in which the number of words included in each group is limited to the specified value or less is displayed.

If the instruction received in step S113 is "setting the analysis target period", the CPU 21 proceeds to step S117. In this case, the CPU 21 sets the analysis target period to the period specified by the user (step S117), and proceeds to step S109. After that, hierarchical cluster analysis is performed with reference to the set analysis target period, screen data for displaying new analysis results is generated, and a new screen is displayed. As a result, the result of hierarchical cluster analysis on the words contained in the text data within the specified analysis target period is displayed on the screen.

FIG. 11 is a diagram showing analysis result screens before and after setting the analysis target period. Of the input text data 5, the text data from 0:00 on January 1, 2014 to 24:00 on December 31, 2015 is displayed on the analysis result screen 61 before setting shown in FIG. 11 (a). The results of a hierarchical cluster analysis of the words contained in are displayed. Of the input text data 5, the text data from 0:00 on March 1, 2014 to 24:00 on September 30, 2014 is displayed on the analysis result screen 62 after the setting shown in FIG. 11 (b). The results of a hierarchical cluster analysis of the words contained in are displayed. The display content of the analysis result screen 61 and the display content of the analysis result screen 62 are different. The user can easily recognize the temporal change of the result of the hierarchical cluster analysis by looking at the analysis result screen before and after setting the analysis target period.

If the instruction received in step S113 is "word exclusion", the CPU 21 proceeds to step S118. In this case, the CPU 21 adds the specified word to the excluded word list (step S118), and proceeds to step S109. After that, a hierarchical cluster analysis is performed excluding the specified word, screen data for displaying a new analysis result is generated, and a new screen is displayed. As a result, the result of the hierarchical cluster analysis excluding the specified word is displayed on the screen.

FIG. 12 is a diagram showing analysis result screens before and after word exclusion. The user operates the mouse 29 to select a word to be excluded, and then instructs the word exclusion. On the analysis result screen 63 before word exclusion shown in FIG. 12A, “society” is selected, and “word exclusion” is selected from the menu. After that, the result of hierarchical cluster analysis excluding "society" is displayed on the screen. On the analysis result screen 64 after excluding words shown in FIG. 12B, "advancement to school" is displayed instead of "society". Among the words included in the same cluster as "society", "advancement" has the highest frequency of appearance next to the five words displayed on the analysis result screen 63.

If the instruction received in step S113 is "synonym registration", the CPU 21 proceeds to step S119. In this case, the CPU 21 adds the instructed word to the list of synonyms in use (step S119), and proceeds to step S109. After that, a hierarchical cluster analysis is performed in consideration of the indicated synonyms, screen data for displaying a new analysis result is generated, and a new screen is displayed. As a result, the result of hierarchical cluster analysis using the indicated word as a synonym is displayed on the screen.

FIG. 13 is a diagram showing analysis result screens before and after registration of synonyms. The user operates the mouse 29 to select a plurality of words to be registered as synonyms, and then instructs the synonym registration. On the analysis result screen 65 before registration of synonyms shown in FIG. 13A, "university student" and "student" are selected, and "registration of synonyms" is selected from the menu. After that, the result of hierarchical cluster analysis with "university student" and "student" as synonyms is displayed on the screen. On the analysis result screen 66 after registration of synonyms shown in FIG. 13B, "university student" is displayed in a size larger than that of the analysis result screen 65, and "advancement" is displayed instead of "student". The "university student" is displayed in a size larger than that of the "university student" in the analysis result screen 65 according to the total of the appearance frequency of the "university student" and the appearance frequency of the "student".

If the instruction received in step S113 is "compound word registration", the CPU 21 proceeds to step S120. In this case, the CPU 21 adds the instructed word to the compound word list in use (step S120), and proceeds to step S109. After that, a hierarchical cluster analysis is performed in consideration of the instructed compound word, screen data for displaying a new analysis result is generated, and a new screen is displayed. As a result, the result of hierarchical cluster analysis using the specified word as a compound word is displayed on the screen.

FIG. 14 is a diagram showing analysis result screens before and after compound word registration. The user operates the mouse 29 to select a plurality of words to be registered as compound words, and then instructs "synonym registration". On the analysis result screen 67 before compound word registration shown in FIG. 14A, “patience” and “strong” are selected, and “compound word registration” is selected from the menu. After that, the result of hierarchical cluster analysis using "patience" and "strong" as compound words is displayed on the screen. On the analysis result screen 68 after the compound word registration shown in FIG. 14B, "patience" is displayed in a size smaller than "patience" and "strong" instead of "patience" and "strong".

As described above, in the text mining method according to the present embodiment, screen data is based on a text analysis step in which a hierarchical cluster analysis is performed on words extracted from input text data and an analysis result by the text analysis step. It is provided with a screen generation step for generating a screen and an analysis result display step for displaying a screen based on screen data. In the screen generation step, m clusters are obtained from the analysis result based on the number of groups m and the maximum number of data n in the group, and the screen data for displaying the group containing n or less words contained in the cluster on the screen. To generate. According to the text mining method according to the present embodiment, a group including the words included in the cluster is displayed on the screen based on the result of performing a hierarchical cluster analysis on the words included in the text data. Further, the number of words included in the group is limited to n or less. Therefore, the user can intuitively understand the result of the hierarchical cluster analysis when looking at the screen.

In addition, the words included in the group are selected from the words included in the cluster corresponding to the group in descending order of frequency of occurrence. Therefore, the words that frequently appear among the words included in the cluster are displayed inside the group. Therefore, the user can easily recognize the frequently occurring words included in each cluster. In addition, the group has a size corresponding to the total frequency of occurrence of words included in the cluster corresponding to the group in the screen. Therefore, the user can easily recognize the cluster in which the total frequency of occurrence of words is large. In addition, the words included in the group have a size corresponding to the frequency of appearance of the words on the screen. Therefore, the user can easily recognize words that frequently appear.

Further, the text mining method includes an instruction input step for inputting an instruction from the user, and either the text analysis step or the screen generation step is executed based on the instruction input in the instruction input step. Therefore, the display mode of the result of the hierarchical cluster analysis can be switched according to the instruction from the user. In particular, the instruction input step receives a setting instruction of the number of groups m, and the screen generation step generates screen data based on the number of groups m specified in the instruction input step. As a result, the number of areas displayed on the screen (the number of clusters) can be switched according to the instruction from the user. Further, the instruction input step receives the maximum number of data n in the group, and the screen generation step generates screen data based on the maximum number of data n in the group specified in the instruction input step. As a result, the number of words displayed in the area can be switched according to the instruction from the user.

In addition, the instruction input step receives the instruction of the analysis target period, and the text analysis step performs hierarchical cluster analysis on the words contained in the text data within the analysis target period specified in the instruction input step among the text data. Do. Therefore, the result of hierarchical cluster analysis on the words included in the text data within the analysis target period instructed by the user is displayed on the screen. Therefore, the user can easily recognize the temporal change of the result of the hierarchical cluster analysis. Further, the instruction input step receives a setting instruction for the analysis purpose, and the text analysis step extracts a type of word according to the analysis purpose set in the instruction input step from the text data 5 and performs a hierarchical cluster analysis. As a result, it is possible to switch the type of the word to be analyzed according to the analysis purpose instructed by the user and display the result of the hierarchical cluster analysis on the screen.

In addition, the instruction input step receives the word exclusion instruction, and the text analysis step excludes the words instructed in the instruction input step to perform a hierarchical cluster analysis. As a result, it is possible to display the result of performing the hierarchical cluster analysis by excluding the words instructed by the user. In addition, the instruction input step receives a synonym registration instruction, and the text analysis step considers a plurality of words instructed in the instruction input step as the same word and performs a hierarchical cluster analysis. As a result, it is possible to display the result of the hierarchical cluster analysis on the screen by regarding a plurality of words instructed by the user as the same word. Further, the instruction input step receives the compound word registration instruction, and the text analysis step merges a plurality of words specified in the instruction input step into one word to perform a hierarchical cluster analysis. As a result, it is possible to display on the screen the result of performing the hierarchical cluster analysis by merging a plurality of words instructed by the user into one word.

In addition, the screen generation step generates screen data for displaying the analysis result screen including the group and the analysis setting screen for setting the display mode of the analysis result screen. Therefore, the analysis result screen and the analysis setting screen are displayed. Therefore, the user can easily switch the display mode of the result of the hierarchical cluster analysis by using the analysis setting screen.

The text mining program 31 according to the present embodiment and the text mining device 10 according to the present embodiment have the same configuration as the text mining processing method according to the present embodiment, and have the same effects.

According to the text mining method, the text mining program, and the text mining device according to the present embodiment, the maximum data of the words contained in the cluster is obtained based on the result of performing the hierarchical cluster analysis on the words contained in the text data. Groups containing less than a few are displayed on the screen. Therefore, the user can intuitively understand the result of the hierarchical cluster analysis when looking at the screen.

5 ... Text data 10 ... Text mining device 11 ... Instruction input unit 12 ... Text analysis unit 13 ... Screen generation unit 14 ... Analysis result display unit 20 ... Computer 21 ... CPU
22 ... Main memory 24 ... Input unit 25 ... Display unit 30 ... Storage medium 31 ... Text mining program 40 ... Display screen 41, 61-68 ... Analysis result screen 42 ... Analysis setting screen 51 ... Data specification screen 52 ... Purpose specification screen 53 … Synonym list selection screen 54… Compound word list selection screen

Claims (25)

It is a text mining method that displays the analysis result of text data on the screen.
A text analysis step that performs a hierarchical cluster analysis on words extracted from the input text data,
A screen generation step that generates screen data based on the analysis result of the text analysis step,
An analysis result display step for displaying a screen based on the screen data is provided.
In the screen generation step, based on the number of groups and the maximum number of data in the group, the cluster of the number of groups is obtained from the analysis result, and the group containing the words included in the cluster is displayed on the screen. It generates the screen data for,
The said group, as the name, and wherein the Rukoto frequency among the words included in a cluster corresponding to the group is marked with the highest word, text mining methods. The text mining method according to claim 1, wherein the words included in the group are selected from the words included in the cluster corresponding to the group in descending order of frequency of appearance. The text mining method according to claim 2, wherein the group has a size corresponding to the total frequency of occurrence of words included in the cluster corresponding to the group in the screen. The text mining method according to claim 3, wherein the words included in the group have a size corresponding to the frequency of appearance of the words in the screen. It also has an instruction input step for inputting instructions from the user.
The text mining method according to claim 1, wherein any of the text analysis step and the screen generation step is executed based on the instruction input in the instruction input step. The instruction input step receives the setting instruction of the number of groups and receives the instruction.
The text mining method according to claim 5, wherein the screen generation step generates the screen data based on the number of groups set in the instruction input step. The instruction input step receives the setting instruction of the maximum number of data and receives the instruction.
The text mining method according to claim 5, wherein the screen generation step generates the screen data based on the maximum number of data set in the instruction input step. The instruction input step receives the setting instruction of the analysis target period and receives the instruction.
5. The text analysis step is characterized in that the hierarchical cluster analysis is performed on the words included in the text data within the analysis target period set in the instruction input step among the text data. The text mining method described in. The instruction input step receives the setting instruction for analysis purpose and receives the setting instruction.
The text analysis step according to claim 5, wherein the text analysis step extracts words of a type corresponding to the analysis purpose set in the instruction input step from the text data and performs the hierarchical cluster analysis. Text mining method. The instruction input step receives the word exclusion instruction and receives the word exclusion instruction.
The text mining method according to claim 5, wherein the text analysis step excludes words instructed in the instruction input step and performs the hierarchical cluster analysis. The instruction input step receives a synonym registration instruction and receives
The text mining method according to claim 5, wherein the text analysis step considers a plurality of words instructed in the instruction input step as the same word and performs the hierarchical cluster analysis. The instruction input step receives the compound word registration instruction and receives the compound word registration instruction.
The text mining method according to claim 5, wherein the text analysis step merges a plurality of words instructed in the instruction input step into one word to perform the hierarchical cluster analysis. The screen generation step is characterized in that screen data for displaying an analysis result screen including the group and an analysis setting screen for setting a display mode of the analysis result screen is generated. The text mining method described in. A text mining program that displays the analysis results of text data on the screen.
A text analysis step that performs a hierarchical cluster analysis on words extracted from the input text data,
A screen generation step that generates screen data based on the analysis result of the text analysis step,
Based on the screen data, the CPU causes the computer to execute the analysis result display step of displaying the screen using the memory.
In the screen generation step, a cluster of the number of groups is obtained from the analysis result based on the number of groups and the maximum number of data in the group, and the group containing words included in the cluster is displayed on the screen. It generates the screen data for,
The said group, as the name, and wherein the Rukoto frequency among the words included in a cluster corresponding to the group is marked with the highest word, text mining program. The text mining program according to claim 14, wherein the words included in the group are selected from the words included in the cluster corresponding to the group in descending order of frequency of occurrence. The text mining program according to claim 15, wherein the group has a size corresponding to the total frequency of occurrence of words included in the cluster corresponding to the group in the screen. The text mining program according to claim 16, wherein the words included in the group have a size corresponding to the frequency of appearance of the words in the screen. The computer is further executed with an instruction input step for inputting an instruction from the user.
The text mining program according to claim 14, wherein any of the text analysis step and the screen generation step is executed based on the instruction input in the instruction input step. 14. The screen generation step is characterized in that screen data for displaying an analysis result screen including the group and an analysis setting screen for setting a display mode of the analysis result screen is generated. The text mining program described in. A text mining device that displays the analysis results of text data on the screen.
A text analysis unit that performs hierarchical cluster analysis on words extracted from the input text data,
A screen generation unit that generates screen data based on the analysis results of the text analysis unit,
It is equipped with an analysis result display unit that displays the screen based on the screen data.
The screen generation unit obtains a cluster of the number of groups from the analysis result based on the number of groups and the maximum number of data in the group, and displays a group containing words included in the cluster in the maximum number of data or less on the screen. It generates the screen data for,
The said group, as the name, and wherein the Rukoto frequency among the words included in a cluster corresponding to the group is marked with the highest word, text mining device.
The text mining apparatus according to claim 20, wherein the words included in the group are selected from the words included in the cluster corresponding to the group in descending order of frequency of appearance. The text mining apparatus according to claim 21, wherein the group has a size corresponding to the total frequency of occurrence of words included in the cluster corresponding to the group in the screen. The text mining apparatus according to claim 22, wherein the words included in the group have a size corresponding to the frequency of appearance of the words in the screen. Further equipped with an instruction input unit for inputting instructions from the user,
The text mining apparatus according to claim 20, wherein either the text analysis unit or the screen generation unit operates based on an instruction input by the instruction input unit. The screen generation unit is characterized in that it generates screen data for displaying an analysis result screen including the group and an analysis setting screen for setting a display mode of the analysis result screen. The text mining device described in.

Images