CN107368506A - Unstructured data analysis system and method - Google Patents

Abstract

An unstructured data analysis system comprising: an unstructured data analysis algorithm residing on a server and accessible via a browser, the unstructured data analysis algorithm operable to: receiving unstructured data from one or more remote sources, applying one or more analysis tools to the unstructured data, and displaying summary information to one or more users; wherein the summary information is displayed to the one or more users in a presentation layer, an exploration layer, and an annotation layer. The unstructured-data analysis algorithm is also operable to receive external data from one or more remote sources. The presentation layer displays one or more of: unstructured data, summaries of unstructured data, and summary information. The exploration layer allows one or more users to modify the granularity of the summary information, thereby modifying the granularity of the presentation layer. One or more users may simultaneously interact with the unstructured data analysis system via the annotation layer.

Description

Unstructured data analysis system and method

Cross References to Related Applications

This patent application/patent claims co-pending U.S. Provisional Patent Application No. 62/159,662, filed May 11, 2015, entitled "UNSTRUCTUREDDATA ANALYTICS SYSTEMS AND METHODS INCLUDING A VISUALIZATION INTERFACE," and filed on May 11, 2015 Priority to U.S. Provisional Patent Application No. 62/159,683, filed entitled "UNSTRUCTURED DATAANALYTICS SYSTEMS AND METHODS INCLUDING NATURAL LANGUAGE PROCESSING AND STATISTICS FUNCTIONS," both of which are incorporated herein by reference in their entirety.

technical field

The present invention generally relates to methods and systems for analyzing large text corpora and unstructured data. More specifically, the present invention relates to methods and systems for analyzing large text corpora and unstructured data using visual analysis and topic modeling, visual interfaces, and natural language processing and statistical capabilities.

Background technique

The management of large and growing collections of textual information and unstructured data is a challenging problem. Data repositories of knowledge-rich textual information have become ubiquitous, resulting in massive amounts of data to be collated, mined, and analyzed. As the number of documents increases, learning the meaning of a text corpus becomes cognitively costly and time-consuming.

The challenge of automatic summarization of large text corpora has become a major concern for researchers in the field of natural language processing (NLP). To summarize text corpora, researchers have developed techniques such as Latent Semantic Analysis (LSA) for extracting and representing the contextual meaning of words. LSA produces a concept space that can be used for document classification and clustering. Recently, probabilistic topic models have emerged as an advantageous new technique for finding semantically meaningful topics in unstructured text collections. To further provide visual summaries of text corpora, researchers from the knowledge discovery and visualization community have developed tools to support the visualization and exploration of large text corpora based on both LSA and probabilistic topic models. ) tools and techniques.

Although probabilistic topic models have demonstrated their strengths in terms of interpretation and semantic association, few interactive visualization systems have exploited such models to support the exploration and analysis of text corpora. Example-based visualization and probabilistic implicit semantic visualization methods have projected documents onto semantic two-dimensional (2D) graphs while estimating the topics of text corpora. Although document clusters obey selected labels well, there is little opportunity for interactive exploration and analysis of document clusters. One exception is the time-based visualization system TIARA, which applies the ThemeRiver metaphor to visually summarize text collections based on topical content. Through the analysis of the TIARA system, users can answer questions such as: What are the main topics in the document corpus? And how has the topic evolved over time?

However, when analyzing large text corpora, there are many other real-world questions that are difficult to answer with current text analysis visualization systems. Specifically, questions about the relationship between topics and documents are difficult to answer with existing tools. Such questions include: What are the document characteristics of the document-based topic distribution? And what document includes multiple topics at once (and what are these multiple topics)? In the domain of scientific policy, for example, documents with multiple topics may indicate interdisciplinary (ie, covering more than one body of knowledge) publications. Similarly, in the context of social media analysis, documents with multiple topics can represent unique news articles related to different hot topics.

In order to overcome the shortcomings associated with existing methods and systems, and to help users understand large text corpora more effectively, the present invention provides a novel visual analysis system that integrates the latest probabilistic topic models, latent Dirichlet distribution ( LDA) integrated with interactive visualization. In order to describe the document corpus, the method and system of the present invention first use LDA to extract a set of semantically meaningful topics. Unlike most traditional clustering techniques that assign documents to specific clusters, LDA models consider different topical aspects of each individual document. This allows efficient comprehensive text analysis of larger documents that can contain multiple topics. To highlight this property of the model, the method and system of the present invention exploit the parallel coordinates metaphor to represent probability distributions across topic documents. This presentation allows users to discover single-topic versus multi-topic documents, as well as the relative importance of each topic to the document of interest. Furthermore, since most text corpora are inherently temporal, the systems and methods of the present invention also show topic evolution over time.

Furthermore, the present invention enables companies including analysts, marketers, business unit leaders, information technology staff, and C-type employees to gain actionable insights from any type of textual data. The technology allows people to enhance their decision-making process on a data-driven basis. The technology ingests text data and uses deep computing and statistical algorithms to identify themes, topics and emerging issues within each data set. Display results in an interactive visual format, enabling anyone in the company to analyze data holistically or granularly. All types of text data can be analyzed - internal data such as emails, chats, surveys, call centers and focus groups, or external data such as social media, review sites, forums and news sites. The technology can handle a large number of languages, ensuring feedback loops from all over the world can be analyzed. However, highly customizable features are selected that allow tuning of the analysis effects. Most companies are sitting on a treasure trove of unstructured text data, but have little ability to mine it for intelligence.

Contents of the invention

Again, in various example embodiments, the methods and systems of the present invention tightly integrate interaction visualization with state-of-the-art probabilistic topic models. Specifically, to address the issues posed above in this paper, the method and system of the present invention exploit the Parallel Coordinates (PC) metaphor to represent the probability distribution of cross-topic documents. This curated presentation not only shows how many topics the document is related to, but also how important each topic is to the document. Furthermore, the method and system of the present invention provide a rich set of interactions that can help users automatically partition document collections based on the number of topics in the documents. In addition to showing the relationships between topics and documents, the method and system of the present invention also support other tasks necessary for understanding a collection of documents, such as summarizing the main topics of a collection of documents and showing how topics evolve over time.

The problem set that the method and system of the present invention can effectively solve when analyzing a large text corpus includes: What is the main topic of the captured document collection? What are the document characteristics for document-based topic distribution? What document covers multiple topics at once? And how have topics of interest evolved over time? In order to help users answer these questions, the method and system of the present invention first use the LDA model to extract a set of semantically meaningful topics. To support topic model-based visual exploration of document collections, the method and system of the present invention employ multiple coordinated views to highlight both topical and temporal features of a document corpus. A novel contribution of the method and system of the present invention lies in the depiction of the probability distribution of documents by topic, and supports interactive identification and more detailed inspection of single-topic and multi-topic documents.

In an example embodiment, the present invention provides a computerized method for textual data analysis, comprising: receiving, at one or more processors, textual data to be analyzed from one or more memories; using the one or more A further plurality of processors formats the text data for subsequent analysis; using the one or more processors, a probabilistic topic model is applied to the text data to extract a set of semantically meaningful topics, the set of semantically meaningful Meaningful topics collectively describe all or a portion of the text data; using a keyword weighting module executing on the one or more processors, a topic cloud view representing topics as tag clouds is generated, where each tag cloud is associated with multiple keywords; use the topic ranking module executed on the one or more processors to generate a document distribution view representing the distribution of all or part of the text data on multiple topics; use the one or more processors to generate a document distribution view A document entropy calculation module executed on one or more processors generates a document scatter diagram view representing how many topics can be attributed to all or part of the data in this article; using a temporary topic trend calculation module executed on the one or more processors, Generate a time view representing the occurrence of topics over time with respect to all or a portion of the text data; and in the analysis of all or a portion of the text data, display a topic cloud view, a document distribution view, a document scatter diagram view to the user and one or more of the time views. The text data includes one or more of: text data exported from documents, text data exported from files, text data exported from one or more data repositories, and text exported from the Internet data. Probabilistic topic models generate a set of latent topics and represent each topic as a multinomial distribution over multiple keywords. Text data is described as a probabilistic mixture of topics. Optionally, keywords are ranked to indicate their importance to a given topic and relationship to each other. Optionally, keywords are highlighted to indicate their importance to multiple topics. Topics are ordered to represent their relationships. This article also provides various other sample functions.

In another example embodiment, the present invention provides a computerized method for textual data analysis, comprising: one or more memories and one or more processors, the memories operable to store data to be analyzed text data, the processor is operable to receive the text data for analysis; algorithms executing on the one or more processors are operable to: format the text data for subsequent analysis; an algorithm executing on one or more processors operable to: apply a probabilistic topic model to text data to extract a set of semantically meaningful topics that together describe the text data all or a portion of; a keyword weighting module executing on the one or more processors, operable to: generate a topic cloud view representing topics as tag clouds, where each tag cloud is associated with a plurality of keywords a topic ranking module executed on the one or more processors, operable to: generate a document distribution view representing distribution of all or a portion of the text data over a plurality of topics; among the one or more a document entropy calculation module executing on a processor operable to: generate a document scattergram view showing how many topics are attributable to all or a portion of the data herein; temporal topic trends executing on the one or more processors a computing module operable to: generate a temporal view representing changes in topics over time with respect to all or a portion of the text data; and a display operable to: contribute to the analysis of all or a portion of the text data The user displays one or more of a topic cloud view, a document distribution view, a document scatter diagram view, and a time view. The text data includes one or more of: text data exported from documents, text data exported from files, text data exported from one or more data repositories, and text exported from the Internet data. Probabilistic topic models generate a set of latent topics and represent each topic as a multinomial distribution over multiple keywords. Text data is described as a probabilistic mixture of topics. Optionally, keywords are ranked to indicate their importance to a given topic and relationship to each other. Optionally, keywords are highlighted to indicate their importance to multiple topics. Topics are ordered to represent their relationships. This article also provides various other sample functions.

Again, the present invention enables companies including analysts, marketers, business unit leaders, IT staff, and C-type employees to gain actionable insights from any type of textual data. The technology allows people to enhance their decision-making process on a data-driven basis. The technology ingests text data and uses deep computing and statistical algorithms to identify themes, topics and emerging issues within each data set. Display results in an interactive visual format, allowing anyone in the company to analyze data holistically or granularly. All types of text data can be analyzed - internal data such as emails, chats, surveys, call centers and focus groups, or external data such as social media, review sites, forums and news sites. The technology can handle a large number of languages, ensuring that feedback loops from all over the world can be analyzed. However, highly customizable features are selected that allow tuning of the analysis effects. Most companies are sitting on a treasure trove of unstructured text data, but have little ability to mine it for intelligence.

In additional example embodiments, the present invention provides an unstructured data analysis system comprising: an unstructured data analysis algorithm resident on a server and accessible via a browser, the unstructured data analysis algorithm Operable to receive unstructured data from one or more remote sources, apply one or more analytical tools to the unstructured data, and display summary information to one or more users; wherein in presentation Summary information is displayed to one or more users in one or more of an exploration layer, an exploration layer, and an annotation layer. Unstructured data includes one or more of: customer experience data, telecommunications data, email data, and social media data. The unstructured data analysis algorithm is further operable to: receive external data from one or more remote sources. External data includes one or more of: Internet data, government data, and commercial data. The one or more analytical tools applied to the unstructured data include one or more of: statistical algorithms, machine learning, natural language processing, and text mining. The presentation layer displays one or more of: unstructured data, a summary of the unstructured data, and summarized information. The exploration layer allows one or more users to modify the granularity of the summary information, thereby modifying the granularity of the presentation layer. One or more users can simultaneously interact with the unstructured data analysis system via the annotation layer. Summary information is also displayed to one or more users in the composite layer.

In another additional example embodiment, the present invention provides a method of analyzing unstructured data comprising: providing an algorithm for analyzing unstructured data, resident on a server and accessible via a browser, the unstructured data The data analysis algorithm is operable to receive unstructured data from one or more remote sources, apply one or more analytical tools to the unstructured data, and display summary information to one or more users; wherein in presenting The summary information is displayed to one or more users in one or more of layer, exploration layer, and annotation layer. Unstructured data includes one or more of: customer experience data, telecommunications data, email data, and social media data. The unstructured data analysis algorithm is further operable to: receive external data from one or more remote sources. External data includes one or more of: Internet data, government data, and commercial data. The one or more analytical tools applied to the unstructured data include one or more of: statistical algorithms, machine learning, natural language processing, and text mining. The presentation layer displays one or more of: one or more of unstructured data, a summary of the unstructured data, and summary information. The exploration layer allows one or more users to modify the granularity of the summary information, thereby modifying the granularity of the presentation layer. One or more users can simultaneously interact with the unstructured data analysis system via the annotation layer. Summary information is also displayed to one or more users in the composite layer.

Description of drawings

The present invention is illustrated and described herein with reference to the various drawings in which like reference numerals are used to identify like method steps/system components where appropriate, and in which:

Fig. 1 is a schematic diagram showing an example embodiment of the visual text corpus analysis tool of the present invention;

Figure 2 is an example display showing a topic cloud view of the visual text corpus analysis tool of the present invention;

Figure 3 is an example display showing a document distribution view of the visual text corpus analysis tool of the present invention;

Figure 4 is a series of graphs showing the distribution of documents on one topic, two topics and more than two topics according to the method and system of the present invention;

Figure 5 is an example display showing a topic cloud view of the visual text corpus analysis tool of the present invention;

Figure 6 is an example display showing a temporal view of the visual text corpus analysis tool of the present invention; and

Figure 7 is a schematic diagram illustrating an example embodiment of an unstructured data analysis system according to the present invention;

Figure 8 is a schematic diagram illustrating another example embodiment of the unstructured data analysis system of the present invention;

Figure 9 is a schematic diagram illustrating an additional example embodiment of the unstructured data analysis system of the present invention;

Figure 10 is a schematic diagram illustrating another example embodiment of the unstructured data analysis system of the present invention;

Figure 11 is a schematic diagram illustrating an example embodiment of the presentation layer of the unstructured data analysis system of the present invention;

Figure 12 is a schematic diagram illustrating an example embodiment of the exploration layer of the unstructured data analysis system of the present invention; and

Figure 13 is a schematic diagram illustrating an example embodiment of the annotation layer of the unstructured data analysis system of the present invention.

detailed description

The two-line work, namely the text analysis model and the text visualization technique is the main inspiration for the preliminary design of the present invention. These concepts are then refined and expanded upon, as described in more detail below.

The first major advance in text processing was the Vector Space Model (VSM). In this model, text is represented as vectors in a high-dimensional space, where each dimension is associated with a unique term within the document. A well-known example of a VSM is TF-IDF, which evaluates how important words are to documents in a corpus. Although VSM has empirically shown its effectiveness, it suffers from numerous inherent shortcomings in capturing statistical structure between and within documents.

To overcome the shortcomings of VSMs, researchers have introduced LSA, a factor analysis that reduces the term-document matrix to a much lower dimensional subspace that captures most of the variables in the corpus. Although LSA overcomes some of the shortcomings of VSM, it also has its limitations. The new feature space is difficult to interpret because each dimension is a linear combination of a set of words from the original space.

Realizing the limitations of LSA, researchers have proposed generative probabilistic models for document modeling. For example, researchers have introduced generative models that represent the content of words and documents with probabilistic topics, rather than purely spatial representations. A unique advantage of this representation is that each topic is independently interpretable, providing a probability distribution based on words for picking coherent clusters of associated terms. The LDA model assumes an implicit structure consisting of a set of topics; each document is generated by choosing a topic-based distribution, and then randomly generating each word based on the topics selected using that distribution. For example, the extracted topics capture meaningful structure in otherwise unstructured data, as shown by analyzing scientific abstracts and newspaper archives. At the cognitive level, LDA models perform well in predicting word associations and the effects of semantic associations and ambiguities on a variety of language processing and storage tasks.

Due to the various advantages of the LDA model, the method and system of the present invention first use the model to extract a set of semantically meaningful topics for a given text corpus. The method and system of the present invention then presents the probabilistic results in an intuitive manner so that complex models can be easily consumed by users when analyzing large text corpora.

In addition to the advantages in automatic text processing technology, artificial intelligence still plays a key role in analyzing text corpora. Therefore, a large number of visualization systems and techniques based on text processing methods have been developed to keep users in the process.

For example, using VSM, tools have been introduced to visualize email content with the aim of delineating relationships in terms of conversation history. The keywords in the visualization are generated based on the TF-IDF algorithm.

Other tools enable users to visually explore text corpora through social network metaphors based on latent semantic analysis results. Other visualization systems have used multidimensional projection methods such as principal component analysis (PCA) and/or multidimensional scaling (MDS) to visualize text corpora. These projection techniques are similar in spirit to LSA in that they represent text as vectors with term frequencies as their features, and then identify a lower dimensional projection space. The visualization system is thus based on these projection techniques including IN-SPIRE. More recently, others have proposed secondary frameworks for topology-based projection and visualization tools for visualizing large collections of classified documents. However, unlike most traditional clustering techniques that assign documents to specific clusters, the method and system of the present invention consider different topical aspects of each individual document.

Since the debut of topic models, visualization systems have used these models due to their advantages over previous text processing techniques. Example-based visualization and probabilistic latent semantic visualization tools have projected documents onto static 2D graphs while estimating the topics of text corpora. Although visual clustering results are better than those obtained from multidimensional projection methods, it has several limitations. First, as the number of extracted topics grows, document clusters in 2D projections are no longer separable based on topics. Furthermore, there is little room for interactive mining and analysis of document clusters in these visualization tools. Recently, TIARA, a time-based interactive visualization system that presents topics extracted from a given text corpus in a time-sensitive manner, has been introduced. TIARA provides a good overview of topics as they evolve over time. However, the relationship between documents and topics is less clear.

Thus, the method and system of the present invention present the probability distribution of documents across the extracted topics in addition to describing topic evolution over time. Thus, the method and system of the present invention provide an overview of document characteristics based on their topic distribution and enable users to identify documents that address multiple topics at once.

The method and system of the present invention support exploration of document collections at multiple levels. At an overview level, the system assists the user in answering the following questions: What are the main topics of the document collection? And what are the characteristics of the documents in that collection? At the facet level, the system supports activities such as identifying temporal trends for a particular topic, and identifying documents related to multiple topics of interest. At the detail level, the system allows access to the detailed content of each individual document as needed. Based on a state-of-the-art topic model, the system employs multiple coordination views, each addressing one of the above-mentioned problems.

Now specifically referring to FIG. 1 , in an exemplary embodiment, the overall structure of a visual text corpus analysis tool 10 of the present invention includes: an offline text preprocessing 12 and a topic modeling module 14 . The text pre-processing module 12 is operable to condition the text of the relevant document 16 for subsequent processing, exploration and analysis. Such textual preprocessing may include, but is not limited to, textual processing from social media (e.g., Twitter posts and Facebook profiles), books (e.g., documents from the Gutenberg online book project), and other documents (e.g., emails, Word documents, etc.) text preprocessing.

As mentioned above, topic models have several advantages over traditional text processing techniques. Accordingly, the visual text corpus analysis tool 10 of the present invention utilizes the probabilistic topic model in the topic modeling module 14 to summarize relevant documents 16 . More specifically, LDA is used to first extract a set of semantically meaningful topics. LDA produces a set of latent topics, each represented as a multinomial distribution over keywords, and assumes that each document can be described as a probabilistic mixture of these topics. P(z) is the distribution over topics z in a particular document. Assume that the text collection 16 includes D documents and T topics. Determining topics is an iterative process using the visual text corpus analysis tool 10 . The tool 10 enables a user to interactively specify a number of topics to be considered necessary in their domain of analysis. Users are allowed to modify the topic modeling module 14 based on findings from their visualization interactions and investigations, enabling them to modify the number of topics and/or the number of iterations of the definition process. The visual text corpus analysis tool 10 also enables users to add, remove, and merge topics to the topic modeling module 14 .

Therefore, the document collection 16 is first preprocessed to remove stop words and the like. Then, the Stanford Topic Modeling Toolbox (STMT) or the like is used to extract the topic set from the document collection 16 . The extracted topic and probabilistic document distributions serve as input for additional visualizations.

The visual design of the tool 10 of the present invention includes four coordination summaries, which may be displayed and manipulated individually or in combination on a suitable graphical user interface (GUI): (1) a document distribution view showing the probability distribution of documents across topics 18; (2) a topic cloud 20 presenting the content of extracted topics; (3) a temporal view 22 highlighting the temporal evolution of topics; and (4) a document scattergram view facilitating interactive selection of single-topic versus multi-topic documents twenty four. Each of the four overviews serves a different purpose, and they are coordinated through a rich set of user interactions. Additionally, when any document is selected, the detail view renders the textual content of that document as needed.

In order to help the user quickly grasp the key points of the document collection, the main topics are presented as tag clouds in the topic cloud view 20 . In the topic cloud view 20, each row displays a topic, which includes, for example, a plurality of keywords related to that topic. Since each topic is modeled as a multinomial distribution based on keywords, the weight of each keyword indicates its importance to the topic. To encapsulate this information in the tag cloud, keywords are aligned from left to right, with the most important keywords placed at the beginning. In addition, since a keyword can appear in multiple topics, the display size or weight of each keyword reflects its occurrence in all topics. However, it will be apparent to those skilled in the art that other configurations can be used. An example of a topic cloud view 20 is provided in FIG. 2 . To assist users in understanding the main topics in the document collection 16, topics are presented in a sequence such that semantically similar topics are close together so that there is continuity when browsing topics in turn. Since the LDA model does not model the relationship between topics, topics are reordered by defining a similarity measure. The visual text corpus analysis tool 10 utilizes the Hellinger distance function to characterize a similarity measure that represents the closeness of topics. The visual text corpus analysis tool 10 visualizes similarity measures to provide users with an understanding of the semantic level of topic distribution and help reduce their cognitive overload by clustering topic spaces.

The topic cloud view 20 also provides a set of interactions for the user to help the user quickly understand the topic. For example, hovering over a particular keyword will cause all other occurrences of that keyword in the tag cloud to be highlighted. Users can also search for specific keywords of interest. Furthermore, the topic cloud view 20 works closely with all other views to quickly provide information on specific topics as needed.

The topic cloud view 20 is produced in part by an online keyword weighting module 26 operable to aggregate the results of the topic modeling modules. It classifies words in a given topic based on their probability in that topic, and more likely words will be placed at the top of the classification queue. This probability value is labeled with the value calculated by the topic modeling module 14 . For example, the word's frequency in the entire text corpus is used to determine the size of the word in the topic cloud view, and normalized based on the maximum word frequency. For example, the higher the frequency, the larger the word. For example, tool 10 defaults to representing the most likely 50 words for each topic. The number of words can be modified interactively by the user.

In order to provide an overview of documents as a mix of topics, the tool 10 of the present invention highlights the distribution of each document across all extracted topics. The chosen representation transforms the document probability distribution into a signal-like style representing each document. More specifically, a parallel coordinates metaphor is employed, where each axis represents a topic and each line represents a document in the collection 16 . This is illustrated in FIG. 3 . In this chosen configuration, all variables (ie topics) are evenly spaced and each variable shares the same range of values from 0 to 1 . Thus, when viewing the document distribution view 18, it is not necessary to understand a document based on its value on each individual axis, but rather it can be understood based on its pattern on all axes as a whole. However, it will be apparent to those skilled in the art that other configurations can be used.

One limitation of LDA is that it does not directly model cross-correlations between topic occurrences, which is naturally expected in most text corpora. The tool 10 of the present invention overcomes this limitation using visualization by making the cross-correlations between topics more prominent. Coincidentally, a feature of parallel coordinate visualization is that it is easier to spot associations between adjacent axes. Thus, topics can be ordered in such a way that semantically similar topics are adjacent to each other, so that associations between similar topics become visually prominent. The topic similarity is defined in terms of the Euclidean distance between two topics in all documents16:

where d _k is one of the D documents in the entire set 16, and P(d _k ) is the probability distribution of the kth document over all topics. Therefore, P(d _k |z=i) represents the probability of topic i when document k is generated. When plotting topics as an axis in the selected interface, start with the topic with the highest concentration of probability and then find the topic most similar to the current topic based on the distance between topics. Figure 3 illustrates document visualization across topics after topic reordering. The relationship (ie, on the adjacent axis) between any two most similar topics becomes visually identifiable.

The document distribution view 18 is generated in part by an online topic ranking module 28 operable to perform the functions described above as well as signal representations of individual documents. This signal is an indication of the different nature of the document. View 18 shows that documents with a significant distribution on a single topic are very focused on a particular topic, whereas documents with a distribution on 2 or 3 topics indicate a variable focus.

When exploring the distribution of documents over topics, it is easy to find that given documents exhibit different characteristics based on the number of topics they have. Figure 4 shows documents 30 focusing on only one topic, documents 32 on two topics, and documents 34 on more than two topics. Different amounts of topics within documents can be interpreted as different characteristics within the context of a given collection of documents 16 . For example, in a collection of scientific publications, documents with a topic represent publications related to a particular field of scientific research. Documents with two or more topics are more likely to represent interdisciplinary research articles, which typically integrate two or more specialized bodies of knowledge.

In addition, the document distribution view 18 provides a rich set of interactions, such as swiping, highlighting, and the like. Brushing a range of percentages on a topic allows the user to select documents that have a certain probability for that particular topic. By combining information and document characteristics related to the main topic from both the topic cloud view 20 and the document distribution view 18 , the user can efficiently develop an overview of the document collection 16 .

The document distribution view 18 enables the user to identify documents of interest to a particular topic by swiping the upper range on the topic. However, it is not so easy to identify documents related to two or more topics in a large corpus because they are masked by single topic documents with high probability values. To alleviate this problem, all documents are separated in such a way that single-topic documents and multi-topic documents can be easily separated. This is document scatterplot view 24.

As can be seen in the document distribution view 18, each document is converted into a signal-like probability distribution pattern. In this representation, documents with multiple topics appear noisier than those explicitly focused on one topic. In information theory, Shannon entropy is a measure of the amount of uncertainty associated with a random variable. Assuming topics are random variables for each document in our context, Shannon entropy can be used to distinguish clean signals from noisy ones. Therefore, the tool 10 of the present invention applies Shannon entropy to differentiate documents based on the number of topics they have. The entropy of each document based on its probability distribution across topics is computed as:

where P(d _k ) is the probability distribution of the kth document on all topics. Each document can then be plotted in the document scatterplot view 24 based on its entropy and its maximum probability value on topic (normalized to [0,1]) (see FIG. 5 ). In this presentation, for example, single-topic (with higher maximum value and lower entropy) documents are in the upper left corner of the scatterplot, while the lower right corner captures documents with a higher number of topics (with lower maximum value and higher entropy) document. When selected, a pie chart is shown to describe the topic distribution of a particular document. In Figure 5, each pie chart represents selected documents, where each color represents a topic. As shown, documents with smaller entropy values appear as pie charts with solid circles; whereas documents with larger entropy values appear as multicolored, indicating that the entropy values correspond to the number of topics in the input documents.

In summary, the document scattergram view 24 enables the user to interactively identify subgroups of documents having a desired number of topics through selection of documents in different areas. The document scattergram view 24 is generated in part by a document entropy calculation module 36 operable to perform the functions described above as well as grouping documents in any given text corpus. The document scatterplot view 24 intentionally groups documents based on their entropy and visually illustrates the focus on that given corpus, implying whether that corpus focuses on a single topic or variable topics.

Since most collections of documents 16 accumulate over time, presenting this temporal information helps to assist users in understanding how the topics of the corpus evolve. Referring now specifically to FIG. 6, the time view 22 is created as an interactive ThemeRiver, with each band representing a topic. In a text corpus, each document is associated with a timestamp, so the height of each band over time can be determined by summing the distribution of documents on that topic within each time frame. The units of the time frame depend on the corpus, for example a year might be an appropriate time unit for scientific publications, while a month or even a day would be more appropriate for a news corpus. After the time unit has been selected, the documents are divided into corresponding time frames based on the timestamps. However, for each time frame, the height of each topic is calculated by summing the distribution of topics from the document over that time frame.

For example, the order of topics (top to bottom) is the same in both the topic cloud view 20 and the document distribution view 18 . Topics are assigned a color or style by interpolating the color or style spectrum using the normalized distance between all adjacent topics. As a result, a more similar pair of topics is assigned a more similar color or style.

In summary, time view 22 provides a visual summary of how the topics of document collection 16 evolve over time. In addition to this representation, various interactions are supported within the time view 22 . Selection of a time frame (one vertical unit of time) results in filtering of all documents published within the selected time frame. Similarly, for example, clicking on the intersection of a topic band and a time frame in time view 22 results in the selection of documents published during that time frame that have a greater than 30% probability on the selected topic. Thus, it is possible to identify what documents contributed to the generation of the topic during a certain time period. Time view 22 adds richness by revealing hidden time information in document collection 16 and allowing users to perform filtering based on time and topic.

The temporal view 22 is generated in part by a temporal topic trend calculation module 38 operable to perform the functions described above as well as inspection of detailed documents. The time view 22 enables the user to directly select, for example, documents within a certain time range and retrieve corresponding data. The temporal view 22 plays a key role in showing the user the basis for identified visual patterns and trends by revealing document details associated with such depictions.

Upon selection of any document, the tool 10 of the present invention provides details of the actual textual content of the document in question. Since any topic model is far from perfect, the function of the detailed view is twofold: first, it provides the user with context to develop a deep understanding of the topic and the keywords associated with the topic; second, it helps the user verify the patterns shown in the visualization .

Since understanding a large text corpus16 involves the utilization of all four views, the coordination between all views needs to be carefully considered. At the topic level, hovering over a topic in any view that involves topic representation will highlight the same topic in other views. For example, if the user hovers over one axis in the document distribution view 18, the same topic is highlighted in both the topic cloud view 20 and the time view 22. Therefore, users are able to quickly synthesize information about keywords, document distribution, and temporal trends of a specific topic. Additionally, views are coordinated by color or style, where each thread has the same color or style across all views.

At the document level, selecting any collection of documents in a view that includes individual documents will highlight the same collection of documents in other views. For example, swiping in the document scatter view 20 is immediately reflected in the document distribution view 18, and vice versa. When a user selects several documents in the document scattergram view 24 that have two salient topics (ie, middle ranges), viewing the distribution of these documents helps the user understand the topic combinations of the documents.

Regarding time, it supports filtering of documents written/published within a specific time period. For example, clicking on a time frame (ie, one vertical time unit) in the time view 22 results in a filter of all documents published within the selected time span. Similarly, clicking on the intersection of a topic band and a time frame in the time view 22 results in a selection of documents published during that time period that have topics that contribute significantly to those documents (e.g., greater than 30% probability). This selection is shown in both the document distribution view 18 and the document scatter diagram view 24 . This feature allows users to filter documents based on time and topic of interest, and then examine documents published within a selected time frame.

The tool 10 of the present invention allows a user to explore and query a large document corpus 16 from multiple viewpoints. Beginning with the topic cloud view 20, the user can view a summary of the corpus 16 and identify topics or even keywords of interest. According to the document distribution view 18, the user can locate a topic of interest and select a document of interest to the topic by performing a swiping operation on the vertical axis. The user can then visually identify which other topics the selected collection of documents are related to by viewing the distributions in the document distribution view 18 and the document scatter diagram view 24 . Furthermore, the user can always check the details of the document based on the selection. For example, if a user wants to identify interdisciplinary/multidisciplinary publications in the corpus 16 , he/she is equipped to do so in the document scatter diagram view 24 by selecting the middle to bottom right document. Furthermore, if the user is interested in querying the corpus 16 by the time factor, he/she can perform selections in the time view 22 by clicking on a time frame or on the intersection of a specific time frame and topic. In summary, the tool 10 of the present invention employs multiple coordinated views to support interactive exploration of the text corpus 16 . Each of the views is designed to address one of four important questions.

In order to evaluate the efficiency of the tool 10 of the present invention in answering the four target questions, the tool 10 was applied to explore and analyze two text corpora including the National Natural Science Foundation of China (NSF) awarded from 2006 to 2010 Scientific proposals and publications in IEEE VAST Proceedings.

Case study 1. Analyzing Science Proposals. In this case study, we first describe the data we collected. We then characterize the target domain and present a set of tasks based on our conversations with NSF program managers. Finally, we show how tools can assist expert users in solving these tasks.

Data collection and preparation. To examine whether this tool can assist program managers in making funding decisions and managing awarding investment structures, we first collected information and intelligent systems (IIS) departments that are part of the Computer and Information Sciences and Engineering (CISE) Board of Directors from 2000 to 2010 Awarded proposal. The collection consists of close to 4000 grants with structured data on grant number, board, department, program, program manager, principal investigator, and grant date; and proposal summaries in the form of unstructured text. We process all collected abstracts, where each abstract constitutes a single document in the corpus. We remove the list of standard stop words. This gives us a vocabulary of 334,447 words. We then use the LDA model to extract 30 topics from the corpus.

domain characterization. A core part of NSF's mission is to keep the United States at the forefront of discovery by funding research in traditional academic fields, including identifying broader implications, and by funding research that is transformative and interdisciplinary. To achieve the former, NSF program managers need to identify appropriate reviewers and team members to ensure the best possible peer review. To effectively execute the latter, program managers need to identify emerging areas and research topics in order to fund interdisciplinary and variable research. In addition to making investment decisions, project managers also need to manage their awarded investment structures. Although program managers have done well in the past, they need new ways to help them because of the naturally fast-changing nature of science and the dramatic increase in the number of proposals submitted. To map high-level tasks to executables, we design three tasks related to the decision-making and awarding of investment structures. Task 1 focuses on grouping new proposal submissions based on their topics. The task requires understanding the main topics of a text corpus and filtering a collection of sub-documents based on their characteristics relative to topics. Task 2 is to identify suitable reviewers for a proposal submission, which also involves knowing whether the submission is related to multiple topics to gather the correct experts. Finally, Task 3 focuses on the temporal aspects of award funding structures involved in discovering topical trends over time.

Expert assessment. Because NSF program managers are particularly busy, we invited former NSF program managers to conduct our expert assessments. Participants have two years of work experience as a program manager at NSF. At the beginning of this assessment, we spend 30 minutes demonstrating the system design and functionality of each visualization. We then asked participants to use the tool to perform the following three tasks.

Task 1. Group 200 newly submitted proposals based on topics. Starting from the topic cloud view, participants quickly browse through the extracted topics to get an overview of newly submitted proposals. Since the participants were responsible for proposals in the fields of robotics and computer vision, she quickly focused her attention on these two topics. When selecting proposals focused on a topic about robotics, participants quickly glanced at the titles in the detailed view to verify their relevance. Although the participant ensured that each selected proposal was relevant, she also noticed that proposals were scattered in their positions in the scatterplot view of the document. Since proposals in the bottom right position are more likely to include two or more topics, participants were interested to know what other topics these proposals covered. By further filtering in the document scatterplot view for those proposals that appeared to be more interdisciplinary, participants found that they related to other fields such as neuroscience and social communication. When the relevant document is selected in the document distribution view, the detailed view is invoked so that the project manager can view previously awarded PIs.

Task 2. Identify suitable reviewers. To identify reviewers, participants first wanted to roughly group proposals. Based on the initial exploration, participants concluded that there are broadly two sets of proposals: one focusing on the core of the field of robotics, and the other using body of knowledge from other fields such as neuroscience and social communication. To identify reviewers for two sets of proposals, participants want to find PIs from previously awarded proposals. By examining historical data, project managers locate robotics-related topics in the document distribution view. She then swipes across the top range of the axis to select proposals related to that topic. Finally, participants turn to the detailed view to view previously awarded PIs in the field of robotics. For interdisciplinary proposals in group 2, participants go through a similar process to identify other experts from other related fields (e.g. neuroscience) to serve on the review map, ensuring the best possible peer review.

Task 3. Analyze time trends in vested investment structures. At the investment structure level, the former project manager is interested in looking at the time trends of her area of responsibility in recent years. Exploring the time view, participants found a steady trend in awarded proposals in the field of robotics, although the overall number of awarded proposals increased between 2006 and 2009. Unlike the steady trend in robotics, the number of proposals awarded on the topic of "Using Technology to Help People with Disabilities" has grown every year. A former project manager commented that this view was valuable to her because it allowed her to see investment trends on different topics that were otherwise difficult to spot.

Overall, participants felt that each view in the tool was well designed with a clear purpose. She commented that the tool can be a facilitator in the workflow of project managers. In particular, she likes the fact that our tool can automatically suggest proposals that are more interdisciplinary, because this is difficult to judge in traditional ways. She also likes the collaboration between views, which helps her quickly synthesize information from different aspects of the same corpus.

case study 2. Analysis of VAST Conference Proceedings. As the field of visual analytics matures, it is instructive to review how the field has evolved. One way to address this issue is to analyze publications that have been accepted by the foremost venues in visual analytics. In this case study, we recruited four researchers to explore papers published at VAST conferences/symposiums since the field began in 2006. As all users are familiar with the field of visual analytics, we want to encourage free exploration, as opposed to the well-structured tasks below. Following the assessment, we grouped participants' findings into two groups: discovering causal relationships between temporal evolution of topics and funding sources, and learning interesting subfields within the field of visual analytics.

Data collection and preparation. We first collect all papers published in VAST conferences/symposiums from 2006 to 2010. A total of 123 publications were collected. We then parse each publication into fields including title, author, year of publication, abstract, body, and references. We perform topic modeling (from introduction to conclusion) on the entire body of each article, where each article constitutes a document in the corpus. Removing the standard stop words leaves us with a vocabulary of 317,315 words. Based on our recordings for different tracks of each VAST conference, we extracted 19 topics from the corpus.

user evaluation. Of the four researchers we recruited, two are senior researchers in the field of visual analytics, while the other two are PhD students who have visual analytics as their main research interest. In this evaluation, we provide high-level tasks for all participants and encourage freer mining. After introducing the tool, we asked each participant to identify core topics in the field and how the field has evolved over the past 5 years. We roughly categorize usage styles into two groups: identifying rising/decreasing topics, and using the system as an educational tool.

Identify rising/declining topics. After scanning all topics in the topic cloud view, a senior researcher commented that the topic fits well with the paper track from the VAST conference. When looking at the time trends for each topic, participants noticed several clear rising and falling patterns. For example, the topic of video news analysis initially attracted a lot of attention, but the attention quickly dwindled year by year. He also noticed a similar trend on the topic of network traffic monitoring and analysis. Relating this pattern to his knowledge, the participant explained the trend because when the field started, the Department of Homeland Security (DHS), which was the main source of funding at that time, led the field of concern. Next, participants turned to rising patterns, which indicated concern among those topics that had arisen in recent years. Specifically, both topic trend and uncertainty analysis and topic dimension analysis and reduction have attracted more attention since 2008. Also relating the style to his own knowledge, the participant commented that this was likely the result of a Foundation for Data and Visual Analysis (FODAVA) project jointly introduced by NSF and DHS.

Learn about the field of visual analytics. Another senior researcher (who was teaching a visual analytics course at the time) commented that he could see the tool being useful for his course. Students can explore the full range of VAST publications and identify papers related to topics of interest for use in course presentations. Similarly, another participant wanted to see what had been done in the field of visual analytics for text analytics. He starts by locating the topic, and then selects publications that rank highly on that topic in the document distribution view. He quickly scans the paper titles in the detailed view and verifies that all selected papers meet his interests. He also noticed that some of the papers in this selection seemed to be related to other topics such as entity extraction and database querying. After the study, he requests a screen capture of the detailed view, enabling him to look up the papers he identified during the study study.

Overall, participants found the tool helpful in helping them explore evolutions in the field of visual analytics and identify publications based on their own interests for further investigation.

Those skilled in the art will understand that various modules and processes of the present invention are implemented using processing devices such as computers. Such processing devices, such as computers, may include one or more general or special purpose processors, such as microprocessors, digital signal processors, custom processors, and field programmable gate arrays (FPGAs), and uniquely stored program instructions (including Both software and firmware) that control one or more processors, in combination with specific non-processor circuitry, carry out some, most, or all of the functions of the methods and systems of the present invention. Alternatively, some or all of the functions may be implemented by a state machine without stored program instructions or in one or more application-specific integrated circuits (ASICs) in which each function or some combination of functions is implemented as a custom logic. Of course, combinations of the above methods can be used. Furthermore, some example embodiments may be implemented via a non-transitory computer readable storage medium having computer readable code stored thereon for programming a computer, server, appliance, device, etc. Each of these may include a processor to perform the functions described and claimed herein. Examples of such computer-readable storage media include, but are not limited to: hard disks, optical storage devices, magnetic storage devices, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM) ), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory, etc. When stored in a non-transitory computer-readable medium, the software may include instructions executable by a processor which, in response to such execution, cause the processor and/or any other circuitry to perform a set of operations, steps, methods, procedures, algorithms, etc.

Again, the present invention enables companies including analysts, marketers, business unit leaders, IT staff, and C-type employees to gain actionable insights from any type of textual data. The technology allows people to enhance their decision-making process on a data-driven basis. The technology ingests text data and, through deep computing and statistical algorithms, identifies themes, topics and emerging issues within each data set. Display results in an interactive visual format, allowing anyone in the company to analyze data holistically or granularly. All types of text data can be analyzed - internal data such as emails, chats, surveys, call centers and focus groups, or external data such as social media, review sites, forums and news sites. The technology can handle a large number of languages, ensuring feedback loops from all over the world can be analyzed. However, highly customizable features are selected that allow tuning of the analysis effects. Most companies are sitting on a treasure trove of unstructured text data, but have little ability to mine it for intelligence.

In general, the software of the present invention delivers deep learning-based data analytics in a sophisticated visualization platform that uncovers, analyzes and infers actionable strategies across a wide range of business decision domains. It connects call center audio, email, news, social media, chat, transactional data, customer feedback and analytics in an advantageous way to discover connections within data that impacts sales, customer service, operations and risk analysis stakeholders. Structured data is also leveraged, including retail transactions, survey data, personal profiles, etc., as well as national and international industry, government, and product-specific data sources. The software is accessible from any browser device and integrates predictive modeling, artificial intelligence, and statistical NLP to analyze unstructured data of any type. Visualizations are provided holistically and/or granularly. The overall system 40 is shown schematically in FIG. 7 . The system 40 uses a high-throughput multilingual API for information tagging using complex term extraction, entity indicator extraction, geospatial indicator extraction, temporal indicator extraction, and opinion sentiment analysis. The system 40 also uses data-driven semantic machine learning and clustering, using automatic term association, statistical topic summarization, influencer interference, context-aware content ranking, content network association, and product-centric analysis.

Referring now specifically to Figures 8 and 9, in one example embodiment, the present invention provides an enhanced intelligence platform 45 that helps companies find the shortest path from data to revenue. It brings together fragmented data silos, creates a unified visual analytics layer on top, and enables users from multiple business functions to efficiently and collaboratively extract valuable insights. Platform 45 sits securely on top of an organization's data lake and is compatible with multiple levels of the data infrastructure. It automatically ingests unstructured data (e.g., emails, call logs) as well as structured data (e.g., sales, budget, finance) through deep computational and statistical algorithms. It processes tens of millions of feedback and data points in real time and identifies themes, topics, and emerging issues within the organization. It helps dynamically correlate customer experience trends with all corporate data. Platform 45 is fully interactive and easy to use. Anyone in the organization, from front-line employees, analysts, and salespeople to business unit leaders and C-type employees, can interact with data holistically or granularly, customize their own dashboards, and share findings with others. In addition to the data analysis background engine, platform 45 is also supported with a fully enhanced user's UI experience. The present invention provides users with pixel-perfect dashboards with customizable visualizations. This makes the analytics job of presenting the user much easier and more controllable. Rich interactions in the exploration layer allow users to quickly start analyzing details and keep contextual information around it. The present invention ensures, and the flexible data analysis environment guarantees that the user never loses connection with the data at a general level while diving into the details. This goes beyond just a few visualizations; extending the user experience to a variety of useful data analysis and visualizations. Annotating and collaborating on analysis results has never been easier. The present invention completely changes the way people can find, share and collaborate on analytical tasks. Users are able to annotate and share their findings with colleagues, enabling collaboration within and outside each data analysis group. In summary, the present invention enhances decision-making by providing a realistic environment for data analysis.

FIG. 10 is a schematic diagram illustrating another example embodiment of an unstructured data analysis system 50 of the present invention. Typically, customer experience data 52, telecommunications data 54, email data 56, social media data 58, and other data 60, such as those closely related to a business enterprise, are aggregated in data repositories 62, and such as Internet data, government data, etc. The external data source 64 is pulled into an unstructured data analysis algorithm 66 that resides, for example, on a web server and can be accessed via a browser. As described in detail herein above, unstructured data analysis algorithms 66 apply predictive modeling, artificial intelligence, and statistical NLP to the data to uncover, analyze, infer, and visualize actionable information. Advantageously, the executable information can be viewed by various business units 68 , stakeholders, or other users, all of whom can add or otherwise modify visualizations and share the results via a common interactive user interface 70 .

FIG. 11 is a schematic diagram illustrating an example embodiment of a presentation layer 80 of the unstructured data analysis system 50 ( FIG. 8 ) of the present invention; generally, the presentation layer 80 allows display of information about unstructured data and/or results. Various summary information. For example, presentation layer 80 is shown displaying customer experience data 82 , telecommunications data 84 , and sales data 86 .

FIG. 12 is a schematic diagram illustrating an example embodiment of the exploration layer 90 of the unstructured data analysis system 50 ( FIG. 8 ) of the present invention. In general, the exploration layer 90 allows display of various summary information about the unstructured data and/or results. Exploration layer 90 also allows temporal granularity to be selected and displayed with further detail. This "drill down" also updates other visualizations, including the presentation layer 80, accordingly. For example, snapshot 94 is shown selected from customer experience data 92 .

Figure 13 is a schematic diagram illustrating an example embodiment of the annotation layer 100 of the unstructured data analysis system 50 (Figure 8) of the present invention. Annotation layer 100 is configured to display various results, as well as customer experience data 102, telecommunications data 104, email 106, social media data 108, other data 110, etc., and to receive user annotations 112, which may be shared via user Interface 114 is accessed by all users or selected users.

Although the invention has been illustrated and described herein with reference to preferred embodiments and specific examples thereof, those skilled in the art will readily appreciate that other embodiments and examples can perform similar functions and/or achieve similar results. It is therefore to be understood that all such equivalent embodiments and examples are within the spirit and scope of the invention and are intended to be covered by the appended claims.

Claims (18)

1. a kind of unstructured data analysis system, including：

It is resident on the server and can be via the unstructured data parser of browser access, the unstructured data Parser, which can operate, to be used for：Unstructured data is received from one or more remote sources, to unstructured data application One or more analysis tools, and shown to one or more users and summarize information；

Wherein presentation layer, explore layer and annotate layer in it is one or more it is middle to one or more users show it is described always Tie information.

2. system according to claim 1, wherein the unstructured data include it is following in it is one or more：Visitor Family experience data, teledata, e-mail data, social media data and transaction data.

3. system according to claim 1, wherein the unstructured data parser can also be operated and is used for：From one Individual or more remote source receives external data.

4. system according to claim 3, wherein the external data include it is following in it is one or more：Internet Data, government data and business data.

5. system according to claim 1, wherein one or more analysis tool bags applied to unstructured data Include it is following in it is one or more：Statistic algorithm, machine learning, natural language processing and text mining.

6. system according to claim 1, wherein the presentation layer show it is following in it is one or more：It is unstructured Data, the summary of unstructured data and the summary information.

7. system according to claim 1, wherein the exploration layer allows one or more users to change the summary The granularity of information, thus change the granularity of presentation layer.

8. system according to claim 1, one of them or more user can via annotation layer simultaneously with it is described non- Structured data analysis system interacts.

9. system according to claim 1, wherein showing the summary to one or more users also in combination layer Information.

10. a kind of unstructured data analysis method, including：

There is provided it is resident on the server and can be described unstructured via the unstructured data parser of browser access Data analysis algorithm, which can operate, to be used for：Unstructured data is received from one or more remote sources, to unstructured data Using one or more analysis tools, and shown to one or more users and summarize information；

Wherein presentation layer, explore layer and annotate layer in it is one or more it is middle to one or more users show it is described always Tie information.

11. according to the method for claim 10, wherein the unstructured data include it is following in it is one or more： Customer experience data, teledata, e-mail data, social media data and transaction data.

12. according to the method for claim 10, wherein the unstructured data parser can also be operated and is used for：From One or more remote sources receive external data.

13. according to the method for claim 12, wherein the external data include it is following in it is one or more：Interconnection Network data, government data and business data.

14. according to the method for claim 10, wherein one or more analysis tools applied to unstructured data Including one or more in following：Statistic algorithm, machine learning, natural language processing and text mining.

15. according to the method for claim 10, wherein the presentation layer show it is following in it is one or more：It is non-structural Change data, the summary of unstructured data and the summary information.

16. according to the method for claim 10, wherein the exploration layer allows one or more users' modifications described total The granularity of information is tied, thus changes the granularity of presentation layer.

17. according to the method for claim 10, one of them or more user can via annotation layer simultaneously with non-knot Structure data analysis system interacts.

18. according to the method for claim 10, wherein also in combination layer to one or more users show it is described always Tie information.