JP2006527886A - Content collection method and apparatus for online purchasing system - Google Patents

Abstract

A method comprising processing a plurality of product information records from product information sources 42, 46. The shopping server 20 generates one or more web patrons for collecting product information from Internet accessible information such as web pages 42, 46 held in the merchant server 40 and / or manufacturer server 44. To do. The shopping server 20 processes the product information to form a product information record collected in the group, and the product information is assigned to each group corresponding to the same product. In order to identify the product, a unique product ID corresponds to the product associated with each group. Specify each identified product category in the classification. Attributes in the product information record corresponding to the group for each product are found. Product characteristics are created based on the attributes for each product, and each product characteristic is stored in a category with a specified product in the category to create a catalog 26.

Description

Related technology

  This application claims to benefit from US patent application Ser. No. 10 / 657,740 dated September 11, 2003 and US Patent Application Ser. No. 60 / 479,153 dated Jun. 18, 2003, the disclosure of which is incorporated herein by reference. Incorporated herein by reference.

  The present invention relates to a system for creating a catalog of goods and services on a communication network. In particular, the present invention relates to a method and system for collecting content for an online catalog system.

  The Internet is a worldwide network of computers linked by various hardware communication links that operate all of a standard set of protocols known as TCP / IP (Transmission Control Protocol / Internet Protocol). The recent growth of the Internet is significant, mostly due to the widespread use of browsing software known as browsers and HTTP (Hypertext Conversion Protocol) that allows a simple GUI (Graphical User Interface) used to communicate over the Internet. Is supported in the field. Browsers are typically provided on computers that are used to access content on the Internet, i.e. client computers. HTTP is a component of TCP / IP, HTML (Hypertext Markup Language), and recently XHTML (Extensible Hypertext Markup Language), which is an improvement to XML (Extensible Markup Language) and HTML. Provides user access to various format files using a standard page description language known as. A collection of servers on the Internet that use HTTP has become known as the “World Wide Web” or simply “Web”.

  As is well known and appreciated in the past, there are currently a large number of web pages with various content. Tools have been developed to allow users to search for these web pages to obtain different web pages with different content of interest. One way to locate the desired web page is to use a “search engine” that searches for web pages with a particular keyword or keywords. A search engine typically has three components: a crawler (robot, bot, automated site searcher, etc.), an index and a software program that presents the search results to the user. It is. In order to gather other information such as URLs and text of pages that can use the search engine in searches for keywords, the patrolman automatically goes from the web server to the web server, and to the site where it gathers. Patrol. When the information collected by the patrolman is complete, the information related to the web page is stored and indexed in a search engine database. When a user searches for information from a web type in a keyword in a search engine search field, the search engine software program uses algorithm functions and criteria to find a keyword that matches the information stored in the database. Some programs retrieve all of the text on each page, while other programs simply retrieve the URLs and / or titles of the page. The software program then categorizes the search results and provides results that prioritize the user based on web page associations. Different search engine software programs use different methods for determining web page associations. For example, the software may include a counter to look up the “meta tag” of the page and count the number of keywords present on the text of the page, and / or to have the web page webmaster displayed as a search result Web page reputation and other factors such as special organization may be considered.

  One basic application of the web is shopping, i.e. the purchase of goods or services, i.e. products. In fact, each major commercial “block mortar” merchant has established a product showcase and sales website. Many more manufacturers sell their products directly on the web. And excessive online retailers are present in the block mortar world, although they haven't existed before. As a result, each product is actually valid for purchase on the web from multiple retailers. This situation increases the efficiency of the market by allowing shoppers to easily compare products and sales periods from multiple retailers without requiring physical movement to the retailer's location. Yes.

  However, in order to compare the products and time periods of different retailers, the various retailer websites must be “visited” individually. First, this requires knowledge of URLs for each merchant website or the use of troublesome and inaccurate search engines. It is possible to open different sites in different browser windows for a better comparison. However, the various formats of each retailer website are bored with direct comparison of products and time periods. When making a purchase decision, the purchase or purchases must be made through individual retailer websites. In addition, buyers are usually required to log in to their respective merchant websites before entering a purchase, for example by entering a username or password, and proceed to the next site. For example, if a buyer decides to purchase three items from three different retailers, three login procedures and three purchase procedures, i.e., a procedure for generating a purchase on the retailer website, will be Each must be done manually through the merchant website and their own interface.

  It is well known to integrate multiple websites into one environment known as a “shopping portal”. Shopping portals typically include a web server that represents an integrated interface that displays multiple products from various retailers. Thus, conventional shopping portals facilitate comparative shopping, thus increasing market efficiency. It is known to prepare catalogs of proposed products from various retailers organized into product category classifications to provide an integrated shopping experience. However, because various retailers and other parties with product information records store all the information in various data formats and layouts, collecting information for product catalogs is redundant and lengthy requiring a large amount of manual manipulation. It is an intensive task.

  In one aspect of the present invention, product information is collected from a plurality of product information sources having completely different formats for product information, and the information is stored in a classification device to create a product catalog stored on a computer-readable medium. Is the method. The method includes processing a plurality of product information records from the product information source into one or more groups based on the fact that product information records are appropriate for the same product; and identifying the product To correlate a unique product ID corresponding to the product associated with each group, and to identify each identified product to determine a category for the identified product in the classifier. A step of comparing with a category of the classification device, a step of determining an attribute for each categorized product based on the product information record corresponding to each group, and creating a product characteristic based on the determined attribute And storing the product characteristic in the correspondingly determined category of the classification device.

  The present invention will be described through preferred embodiments and drawings.

  A preferred embodiment computer configuration for providing an integrated online shopping experience and product catalog generation is shown in FIG. The commerce system 10 includes a client computer 12 that executes a browser application 14 that supports the HTTP protocol. The client computer 12 is connected to the Internet 100 that operates as a communication channel, typically through an ISP (Internet Service Provider). For example, the client computer 12 is connected to the ISP through a conventional dial-up connection using a modem or a broadband connection such as an ISDN (Integrated Services Digital Network), cable modem, DSL (Digital Subscriber Line) connection. The shopping server 20 is also connected to the Internet 100 by a known method. The shopping server 20 is known as an HTTP server application and executes a web server control application 22 stored in a memory device. For example, public domain web server software applications from NCSA or APACHE can be used. The shopping server 20 also executes an agent server control application 24 that uses a secure connection for privacy (its function will be described in detail below).

  The plurality of merchant servers 40 provide online shopping using conventional commerce server control applications, ie, software that performs some of the main online storefront functions such as product display, online ordering, inventory management and the like. Retailer server 40 and commerce server software are well known and will therefore not be described in detail here. Further, the retailer server 40 can store a product information record including information about products to be sold.

  In the preferred embodiment, each of the client computer 12, shopping server 20, and merchant server 40 can communicate using a secure connection protocol such as SSL or S-HTTP. The non-secure connection 30 and the secure connection 32 are clearly shown separately. Typically, however, these connections occur over the same physical connection or communication channel, such as the Internet 100. In addition, the shopping server 20 and the retailer server 40 have a number of web pages stored in a memory device as files in HTML format and / or other formats. The shopping server 20 also includes a product catalog 26 and a buyer database 28 stored in the memory device as will be described in detail below.

  By issuing a URL request to the shopping server 20 through the Internet 100, the client computer 12 can request display of a web page stored in the shopping server 20. For example, a user of the client computer 12, that is, a buyer, navigates a web page that is stored in the shopping server 20 and configured with product information from the product catalog 26, thereby selecting one or more products for purchase. You can choose. The product catalog 26 may be in the form of a database, from a merchant server 40 using a plurality of retail product descriptions, prices and other product information and automated web pavers as described in detail below. The selected product information may be included. Product information in the product catalog 26 is periodically updated in accordance with the latest product information on the retailer server 40. However, as will become apparent below, product information need not be updated in real time.

  Search for product information from the product catalog 26 by product type, identification number, price, keyword or product feature in a desired manner using the interface of the shopping server 20 as represented by the buyer in the browser application 14 of the client computer 12 Can be displayed. Products associated with products from multiple merchant servers 40 in the browser window of client computer 12 to allow buyers to compare and select products from one or more merchant servers 40 based on the product information. Product information in the catalog 26 can be displayed side by side. For example, the user may search for all instances of a particular item by product name or identification number and may select the instance for purchase from a retailer with the lowest price. In login to the shopping server 20, for example, a user is specified by inputting a user name and a user ID as specific data, and therefore, the user avoids having to re-enter previously registered data and preferences. Can do.

  A preferred embodiment communication channel connection in an automated purchase procedure is shown in FIG. For clarity, the following description generally relates to only one merchant server 40. However, the procedure described below can also be performed for multiple products from multiple retailer servers 40. For example, when the user selects a product for purchase by clicking a “Purchase” button, a secure connection 32 is established between the client computer 12 and the agent server application 24 of the shopping server 20. The agent server application 24 then opens a secure connection 32 in parallel with the commerce server application of the retailer server 40. The secure connection 32 is shown as a direct connection between computers for clarity. However, it should be understood that secure connection 32 is an SSL connection over the Internet 100 and other types of communication channels.

  As shown in FIG. 2, the first “cookie” 29 (that is, a file including information such as specific information to be used by the server) is established in the shopping server 20, and the second cookie 18 is sent to the client computer 12. Established. The first cookie 29 allows the merchant server 40 to track the status of the order acceptance process, and the second cookie 18 allows the shopping server 20 to track the status of the order place process. . Referring to FIG. 3, the first cookie 29 includes information identifying an order acceptance session between the shopping server 20 and the merchant server 40, that is, a merchant session ID. The information to be identified can be any character string or code, which allows the merchant server 40 to identify the order acceptance session. Similarly, the second cookie 18 includes a “nonce” (ie, a single random sequence) and other information that identifies an order location session between the client computer 12 and the shopping server 20. The shopping server 20 maintains a record 55, which may be, for example, a database that associates the nonce of the second cookie 18 with the corresponding transaction record 54 by showing the transaction record 54 (see the description of FIG. 4 below). An index table or the like. This is because changes in the transaction reported from the merchant server 40 are recorded in the transaction record 54, but the two are synchronized by the effect of the nonce instructions on the transaction record 54. Transaction record 54 also includes a corresponding merchant session ID. Thus, when the buyer resumes an idle session, such as by confirming an order through the client computer 12, the shopping server 20 examines the second cookie 18 to identify the corresponding order location session and situation, thus The session can be resumed in a secure manner. The shopping server 20 can also locate the corresponding first cookie 29 and indicate it to the merchant server 40 to resume the corresponding order acceptance session.

  Note that typically there are multiple order locations and corresponding order acceptance sessions occur simultaneously. The cookie handling procedure described above allows to be accurately correlated to all sessions, thus allowing a seamless shopping experience. In order to perform buyer assistance during execution of the purchase procedure of the retailer server 40, the shopping server 20 uses information stored in the buyer database 28. The buyer database 28 is suitable for registered buyers such as their name, address, shopping preferences, credit card number, merchant account information (such as user name and user id for buyers at each particular merchant), etc. Information may be included. Of course, all data in the buyer database 28 may be collected during the registration procedure and encrypted in a known manner for safety.

  FIG. 4 shows the purchase procedure of the shopping server 20, ie the function of the agent server 24 in great detail. Remember that the purchase procedure usually starts after the buyer logs in to the shopping server 20 or otherwise uniquely identifies themselves. In the first stage of the purchase procedure, a buyer searches for products in catalog 26, browses products in various ways, and selects one or more products for purchase from one or more merchant servers 40. After login, the main process 50 of the agent server control application 24 generates a purchase form for display to the buyer. If the buyer wishes to change the information in the purchase form 52 for the current purchase procedure, the buyer can simply edit the purchase form 52. For example, the buyer may want to change the shipping address or shipping method. Of course, the buyer profile may be edited to change the default buyer information in the buyer database 28. If such information exists as a buyer profile in the buyer database 28, the purchase form 52 is automatically filed in advance with default buyer information corresponding to the buyer. If no such information exists for the buyer, the buyer is prompted to enter the information and that information is used in the current purchase procedure. For the current transaction procedure, the main process 52 is responsible for all transaction information including transaction status (credit card information from the buyer database 28, billing address, other information and retail of the selected product from the product catalog 26). A transaction record 54 is also created that keeps track of the merchant SKUs, shipping options, and other information.

  The main process 50 also triggers a purchase process 56 and directs the purchase process 56 to the corresponding transaction record. As the buyer selects various products and options using the browser interface of the client computer 12, the transaction record 54 is updated. Note that at a given time, there are multiple purchase procedures for multiple buyers, each with individual purchase processes 56 and corresponding transaction records 54. The purchase process 56 continues to be executed in parallel with the main process 50 until the purchase procedure is completed. The purchase process 56 constantly updates the transaction record 54 based on the buyer's selection. In the meantime, the main process 50 examines the transaction record 54 for update status. Thus, the main process 50 is updated in the situation of each purchase procedure.

  When the first stage of the purchase procedure is complete, i.e., the buyer has selected all desired products and options from all desired retailers, the main process 50 is the browser of the client computer system 12 for authenticating the order by the buyer. A confirmation page 58 is shown to the buyer through the interface. A confirmation page 58 is generated in communication between the shopping server computer system 20 and the appropriate merchant server 40 using the secure connection 32 between the shopping server computer system 20 and the appropriate merchant server 40. In particular, the retailer server 40 authenticates the price information, shipping information and other detailed information of the desired purchase by automatically navigating to each retailer's inspection page or other information page and searching for updated information. To do so, the shopping server computer system 20 uses the information in the transaction record 54. In order to allow the purchase process 56 to navigate the retailer server 40 automatically, the purchase procedure of the retailer server 40 is integrated into the purchase process of the shopping server 20. A back-end test script or the like can be used to determine specific purchase procedure steps for the merchant server 40.

  If specific retail merchant account information exists in the buyer database 28 for the buyer, that account information is used when the retailer server 40 executes the purchase procedure. If not, a new account is created for the buyer at the merchant and the account information is stored in the buyer database 28 for later use. Since the shopping server 20 uses the merchant account information corresponding to the buyer, the buyer can retain preferred buyer points and other benefits and discounts as if shopping directly at the merchant server 40. .

  In the preferred embodiment, until the confirmation page 58 is generated between the client computer 12 and the shopping server 20 using information from the product catalog 26 that may not be completely updated due to the unstable state of online commerce. Remember that communication can continue. Accordingly, the confirmation page 58 includes real-time price information and shipping information obtained from the merchant server 40 for each selected product in the transaction record 54. Upon receipt of the purchaser confirmation of the order summarized on the confirmation page 58, all transaction information in the transaction record 54 is saved and the second cookie 18 is saved to allow a procedure to be resumed later in the same user session. Is done. The purchase process 56 remains idle while the buyer waits to confirm the order by selecting a button on the confirmation page 58 or taking other actions.

  The purchase transaction is completed in the second stage of the purchase procedure. In particular, the second cookie 18 is used to resume a previous user session at the merchant server 40. Subsequently, a purchase procedure is executed at the merchant server 40 and an order is executed at the merchant server 40 that uses the information in the transaction record 54 to perform the purchase process. Then, the order is confirmed by the shopping server 20, the transaction record 54 is updated, and a receipt page 60 showing the transaction information, confirmation number, etc. from the retailer server 40 is generated. Once again, products can be selected from multiple merchant servers 40, in which case multiple purchase procedures on each merchant server 40 that automatically uses the appropriate buyer account information for each merchant server 40. It is important to note that is executed and verified.

  In the purchase procedure described above, it is conceivable to allow the shopping server 20 to operate as an agent on behalf of the buyer due to the mutual relationship with the retailer server 40. However, some retailers do not feel comfortable with buyers using the agent website. In particular, many retailers rely on advertising in their business models, partnership programs, and so on, so that buyers can visit their websites and purchase pages, especially directly, during shopping. Their business objectives are simply achieved. Thus, the proxy server mode of shopping server 20 can be used to propose many of the advantages of the integrated purchase procedure described above, while at the same time hoping buyers to visit their website purchase page. Satisfy vendors.

  Internet proxy servers are known. The phrase “proxy server” generally relates to a server connected between a client application, such as a web browser, and a web server for intercepting requests. The proxy server then supplies enough of the same content so that the request is directed to the web server, while simultaneously filtering data, monitoring data, and cached copies of web pages. Auxiliary functions such as supply In the preferred embodiment, this general concept is extended and applied to an integrated shopping environment. As shown in FIG. 2, in the proxy server mode, a parallel secure connection 32 is set between the client computer and the shopping server 20 and between the shopping server 20 and the retailer server 40. The first stage of the purchase procedure is performed in the same way as described above for the standard mode. In order to use the proxy server mode, the merchant database 28 preferably includes a form map for placing fields in the order form of the merchant server 40 into fields in the buyer database 26. External editing tools are provided for developing form maps in a known manner. For example, the editing tool is configured to read an HTML form and parse a data field. These data fields are then correlated to corresponding fields in the customer database 28.

  When the client computer 12 requests a web page from the merchant server 40 during the second stage of the purchase procedure, the shopping server 20 processes the request to direct the shopping server 20 on behalf of the merchant server 40. Then, HREFs, that is, references to HTML documents, and POSTs, that is, proposals for HTML forms are screened. When the merchant server 40 sends a blank form to the client computer 12, the form is intercepted by the shopping server 20 and the form is filled with information from the buyer database 26. When the client computer 12 returns a form to the merchant server 40, the shopping server 20 regenerates a reply that keeps all changes in the form content made by the buyer complete. The first cookie 29 and the second cookie 18 track the session as described above.

  In the proxy server mode, when the buyer selects a purchase button from the confirmation page 58 or other page displayed on the client computer 12, the shopping cart page of the retailer server 40 or another purchase page is displayed. The buyer manually executes the purchase procedure of the merchant server 40 using the interface of the merchant server 40 as displayed on the client computer 12. This manual procedure is performed for each merchant server 40 that requires proxy mode. Accordingly, in the proxy server mode, the buyer sees all purchase pages on the retailer server 40 and executes the purchase procedure. However, the shopping server 20 provides assistance by filling in the form. In another aspect of the purchase procedure that uses the proxy server mode, the purchase procedure that does not use the proxy server mode described above can be used.

  The present invention facilitates online commerce by allowing an integrated purchasing experience from multiple retailers. The present invention may be implemented on any type of communication channel using the Internet, a local area network (LAN), a wide area network (WAN), a direct computer connection, or any other type of communication hardware and protocol. . Any type of hardware or combination of hardware may be used as the various clients and servers. Accordingly, the term “computer” as used herein refers to computer devices and data such as personal computers, portable computers, dumb terminals, thin clients, portable devices, wireless telephones, combinations of such devices, and the like. Relevant to any type of terminal. The various clients and servers may be a single computer at a single location, or multiple computers at a single location or multiple locations. For example, the server may be composed of a plurality of excess computers that are installed in common at various locations for downsizing. Appropriate server and client software may be used and any communication protocol may be used. It can communicate over wireless, electrical, fiber optic, and other cables using radio frequency, infrared, and other technologies. Any interface may be used to select a product for purchase. The various information may be stored in any format, so the term “database” as used herein relates to database files, index tables, and any other collection.

  As indicated above, the product catalog 26 may include multiple retail commercial product descriptions, prices, delivery dates, and other product information. Such product information is picked up from product information records from various sources using the automated patroller described below and corresponds to the current product valid at the merchant server 40. May be updated regularly. The term “traveler” as used herein is equivalent to any software that performs content searches on the network, including “bots”, “robots”, “automated site searchers” and others. Good. As shown in FIG. 1, the commerce system 10 includes a client computer 12 that executes a browser application 14 and a shopping server 20 that executes an agent server control application 24, and the client computer 12 and the shopping server 20 provide the communication channel. 100. Further, in the preferred embodiment, a plurality of manufacturer's servers 44 are also connected to the Internet 100 via non-secure connections 30. In this regard, the shopping server 20 is used to collect product information from a plurality of information sources connected to the Internet 100 related to products in the product category and store the collected information in the product catalog 26 in a classification form. May be. However, the collection of product information may be achieved using a computer other than the computer that executes the agent server central application 24, and the resulting information is valid for the computer that executes the agent server central application 24. It should be noted that can be done.

  FIG. 7 shows an example of the classification 700 of the product catalog 26. 7 is the US patent application number? ? (Insert classification application number), the disclosure of which is incorporated herein by reference. The classification 700 includes a first hierarchy category 714, a second hierarchy category 716, a third hierarchy category 720, product specifications, ie characteristics 718 and 722. Note that the classification 700 is defined in a tree-like structure, where the category includes attributes that define specifications for products in the category. Product specifications 718 and 722 inherit the attributes of the parent category and may include numeric values for the attributes and applicable measurement units.

  The plurality of information sources may include a plurality of merchant information sources and manufacturer product characteristic information sources, which are located on the merchant server 40 and the manufacturer server 44, respectively. In the preferred embodiment, it is appropriate that the plurality of merchant information sources are the merchant web pages 42 and the plurality of manufacturer product characteristic information sources are the manufacturer product characteristic web pages 46. Furthermore, the information source of the product information record can be various product advertisement information sources, and the product advertisement information source reports and provides additional information related to one product or a plurality of products in the product category. It may be a web page. In practice, the manufacturer's product characteristic information source may be considered only a subset of the product advertisement information source. Also, when a manufacturer sells a product directly on the Internet 100, the manufacturer's server 44 is one of the same as the retailer's server 40.

  In a preferred embodiment, the shopping server 20 is operated to provide at least one visitor for visiting a plurality of information sources, which are related to various products in the product category. In order to collect product information from a plurality of information sources, for example, it is collected at a merchant server 40 and a manufacturer server 44. The patroller may include a product advertisement patroller 72 that collects product phrase information from the manufacturer's product characteristics web pages collected at the manufacturer's server 44. The patrol person may also include a product proposal patrol person 74 who collects product phrase information and price information of each product in the product category from the merchant web pages collected at the merchant server 40. Of course, it should be noted that the product advertiser 72 may visit the retailer's web page, and the product suggester 74 may visit the manufacturer's product characteristics webpage. In addition, a single traveler may be provided to perform the functions of both product advertisement traveler 72 and product proposal traveler 74. Any type of number of patrolmen may be used.

  In a preferred embodiment, product advertisement circulator 72 and / or product proposal circulator 74 may gather information from product catalog 26, which uses manufacturer identification, product model, and computer language. This relates to product phrase information that preferably includes one phrase from each of a plurality of information sources and at least one characteristic of the phrase. The term “phrase” collected by the patrolmen 72, 74 may appear in sources such as the manufacturer's product characteristics web page 46, the retailer web page 42 and / or the product advertisement web page (not shown). Associated with one or more columns of several characters. The term “property” corresponds to several attributes of a sequence of alphanumeric characters on the web page. For example, a phrase characteristic may be the frequency, location, font size, font style, font case, font effect and font color of the phrase on the web page, as well as the frequency of the word order (the phrase immediately following each other) and It is good also as simultaneous appearance (phrase in the word mutually predetermined beforehand) of a phrase. Furthermore, the term “computer language” is used herein in connection with the interdisciplinary field of language modeling using computer analysis to process linguistic data, such as any of the phrase characteristics described above. In a preferred embodiment, the patrolmen 72, 74 and the computer language used by them are software programs designed to perform the functions described above. For this reason, by using a computer language, the product advertisement circulator 72 and / or the product proposal circulator 74 collects product phrase information that can be processed and used in the manner described below.

  As described above, FIG. 5 illustrates the method of the preferred embodiment of the present invention, where the product ad circulator 72 uses the computer language module 75 to obtain product phrase information. The acquired product phrase information (ie, the phrase and its characteristics) is stored in the product phrase database 76 for further analysis. The commerce system 10 of FIG. 1 further analyzes the product phrase information using the statistical analysis module 78, either through the shopping server 20 or in another way, such as other computers / servers, thereby providing any Can be used to provide rankings of product phrases in different product categories. These ranked product phrases typically represent commonly found product characteristics for a given product or product category and are also stored in the product phrase database 76.

  Further, as described in detail below, the preferred embodiment of the present invention analyzes stored product phrase information to determine whether each product phrase information in the product phrase database 76 is actually a product characteristic. The characteristic definition tool 80 is used. The terms “product characteristics” and “product characteristics” should be understood to be words, numbers, phrases or combinations thereof that descriptively characterize a product or product category. The property definition tool 80 of the preferred embodiment is a software algorithm executed on the shopping server 20 or other device, as shown in the steps of FIG.

  Thus, for example, product advertiser 72 may obtain a server 44 of a manufacturer such as a computer manufacturer (or other server having a product advertising information source) to obtain product phrase information provided on a web page. ) Web page content, which includes phrases and phrase characteristics, as well as information provided in the web page URL address and other meta tags. This product phrase information is stored in the product phrase database 76 for further analysis. In this example, the computer manufacturer's home web page probably has a meta tag containing the phrase “computer” and the phrase “computer” throughout its home web page, and the home web page is on the web page. It may or may not have special characteristics such as formatting and positioning that distinguish phrases from the rest of the text. Due to the use of the computer language module 75, the product advertiser 72 recognizes and acquires such information about the phrase “computer” such as its presence in the home web page's meta tag, frequency in the home web page and other special characteristics. Statistically distinguishing that the home web page is related to the computer and that the web page linked to the home web page is also probably related to the computer, so that this phrase It can be determined that the product category in particular.

  In order to ensure an accurate determination, product advertiser 72 preferably patrols through a significant portion of the linked web page before determining whether the phrase is a product characteristic. For example, product ad circulator 72 may patrol through multiple web pages linked to its home web page to collect product phrase information, in this example, perhaps the presence and given of the phrase “computer” Other phrases known to be associated with the product category may further be identified, for example, the phrase “Mb RAM”, etc. arranged with the phrases “MHz”, “Floppy” and / or “Gb”. Based on the analysis of the frequency and characteristics of such phrases by statistical analysis module 78, the phrase “computer” can be determined to be a product characteristic that defines a product category. In this regard, another category database 79 may be provided with various product categories and most likely associated keyword phrases to ensure accuracy of product category determination. Further, in addition to or as an alternative to the category database 79, a human confirmation process may be provided to further ensure the accuracy of product category determination.

  In addition, the plurality of web pages linked to the home web page may typically include an alphanumeric string or data string that identifies other important characteristics of the product or product category. It should also be noted that such strings are often distinguished from the rest of the text on the web page by location on the web page, formatting, and the characteristics of the string. For example, the character string may be located near the top or left hand side of the web page and have a larger font size than the majority of other character strings on the web page. In this regard, such important strings may have a noticeable font size, font style (bold, etc.), font effects (italicized and / or underlined, etc.), etc. Good. With respect to the computer of this example, important attributes / characteristics of the computer such as speed and capacity may be indicated by character strings with phrases such as “Mb RAM”, “MHz”, and “Gb”. Due to the use of the computer language module 75, the product ad circulator 72 determines the phrase and its characteristics such as frequency, location, font size, font style, font case, font effect, font color, word order and co-occurrence as product phrases. It can be recognized and acquired as information. Such information is stored in the product phrase database 76 and the statistical analysis module 78 is used to provide a phrase in a given product category and a ranking of the phrases stored in the product phrase database 76. . In particular, statistical analysis module 78 recognizes that a particular phrase has been emphasized by the presence of one or more characteristics, and as a result is higher than other phrases that do not have the characteristics associated with that phrase. Provide a ranking for that particular phrase.

  In a preferred embodiment of the present invention, a ranked list of phrases in the product phrase database 76 to determine whether the phrases in the product phrase information are actually relevant product characteristics that descriptively characterize the product or product category. May be further analyzed by the property definition tool 80. It should be noted that the property definition tool 80 is preferably run by a human editor, and thus, in connection with ranked phrases generated using a computer and statistical analysis module 78. A judgment of social convention may be made. However, a computer or other logic device may be used to automate / semi-automate the function of the property definition tool 80.

  As shown in FIG. 5, the determination of whether the phrase in the product phrase information is a product characteristic is preferably performed at step 81 by a human editor. If it is determined that the phrase is not related to product characteristics, the phrase and product phrase information is discarded and the next phrase is analyzed. If the phrase is determined to be a product characteristic (ie, the phrase descriptively characterizes the product or product category), as if it were a numeric string, as shown in step 82 A decision is made. For example, in the computer of this example, a good numerical phrase is a numerical value that defines RAM, hard drive capacity, processor speed, and the like. If the phrase is numeric, it is entered at step 84 whether the range and interval are appropriate and recorded for the corresponding numeric phrase. For example, for RAM, the computer model may be valid with 32, 64, 96 or 128 Mb RAM. In this case, the range may be 32 to 128 Mb and the interval may be 32 Mb.

  Then, as shown in step 86, a determination is made as to whether the numeric phrase is evaluable, i.e., whether the numeric phrase indicates product quality and / or satisfies the desirability of the product. . Since the amount of RAM directly meets the capacity and desirability of the computer so that it can be easily determined, the numerical value associated with the RAM can be evaluated. If the numerical phrase can be evaluated, as shown in step 88, the direction of evaluation, that is, whether the higher numerical value is better or the lower numerical value is better is determined. Also, for this example, a computer with a high-capacity RAM is more desirable than a computer with a low-capacity RAM, so a higher number is better and the direction of evaluation is better when the value of the numeric phrase increases. ing. The direction can be determined manually by a human editor or automatically through a computer or other device, for example in the context of price. Of course, for specific product characteristics that are numerical and can be evaluated, lower numbers may be better. For example, a low numerical value is preferable to a high numerical value in the weight of a notebook computer. Similarly, the direction of evaluation facilitates ranking of each phrase as shown in step 89. Therefore, the numerical phrase “32” is ranked lower than “64” or the like at least for the product characteristics of the RAM. If the numeric phrase cannot be evaluated, at step 89, a human editor may enter a ranking for the numeric phrase based on his product knowledge and experience, and the next phrase is analyzed. Of course, because such numeric phrases are not evaluable, the ranking for that phrase will probably be lower compared to evaluable numeric phrases.

  For step 82 described in detail, if the phrase is not numeric, it is determined in step 83 whether the phrase is discontinuous and in step 85 whether the numeric name counted for the discontinuous phrase has been entered. . For example, in this example, the phrases “CD-ROM”, “CD-R”, and “CD-RW” are all discontinuous phrases that are also characteristics of a computer. Then, at step 86, it is determined whether the discontinuous phrase can be evaluated. When evaluated, each of these CD memory devices provides a different level of functionality to the computer, as known in computer technology, so that they can indicate product quality and / or These discontinuous phrases can be evaluated in meeting the desirability of the product. In this respect, the property definition tool 80 inputs the direction of evaluation, as shown in step 88, and inputs the ranking of discontinuous phrases, as shown in step 89, simultaneously ranking the products. In this example, the phrases “CD-ROM”, “CD-R” and “CD-RW” are in order of desirability in a computer, as evaluated in computer technology. Once again, the order and ranking may be determined manually or automatically. Further, if the discontinuous phrase cannot be evaluated, in step 89, the ranking for the discontinuous phrase is input and the next phrase is analyzed. Of course, since such a discontinuous phrase is not evaluable, the ranking for that phrase is probably lower compared to an evaluable discontinuous phrase.

  If the phrase is not discontinuous and numerical, the phrase is set as free-form text as shown in step 87, the ranking is input and the next phrase is analyzed as shown in step 89. Such free-form text is relatively rare because the phrase is considered a product characteristic. However, for certain products, such free-form text may preferably add a numerical value to the product, and conversely, it may be undesirable to reduce the numerical value of the product. For example, autographing and inscription on a product may be considered as free-form text that adds a numeric value to the product.

  In the preferred embodiment of the present invention shown in FIG. 5, once the analysis is completed with product phrase information stored in the product phrase database 76, as shown in the property definition tool 80, the resulting product characteristics are (ie, The product or product category is descriptively characterized) and stored in the product characteristic database 90. Furthermore, by executing the characteristic definition tool 80, these product characteristics are stored in the product characteristic database 90, and ranked in the order of importance in the products and product categories. For this reason, in the computer of this example described above, the product characteristic database 90 is relevant for evaluating product characteristics such as MHz, Mb RAM, Gb, CD-ROM, CD-R, CD-RW, and the computer. You may have a record of many other product characteristics and attributes. In addition, through the execution of the property definition tool 80, these product properties are similarly ranked. As described in more detail below, a catalog 26 is created on each individual retailer web page that is used to evaluate products available from various retailers, which allows multiple online retailers to be seamlessly connected. Provides a single shopping interface that integrates into the database, and thus may search and use these stored product characteristics in the product characteristics database to facilitate comparative shopping in an online environment.

  Of course, in the above-described aspect of the present invention, examples applied to computers have been described. However, the present invention is not limited to these, and how the product advertisement patrol person 72 and the characteristic definition tool 80 are used. It should be noted that only a computer has been selected as an example. In this regard, it should be appreciated that the present invention is applicable to all different products and services that are effectively traded on a networked environment such as the Internet 100. For this reason, the present invention can be applied to all products and many services including insurance, financial services, rental, housing, transportation, vacation packs, and the like in the same manner as described above.

  FIG. 6 shows a block diagram of a method for validating an online retailer product proposal and creating a new product record based on product characteristics, in accordance with a preferred embodiment of the present invention. As can be seen, the lower portion of FIG. 6 is based on a method for determining product characteristics as collected by the product ad circulator 72 from the manufacturer's product characteristics web page 46 and other sources. FIG. 5 provides a schematic illustration of the above-described FIG. In a similar manner, product phrase information may be gathered from a merchant information source, such as the merchant web page 42, by a product suggester 74 generated at the shopping server 20. For this reason, as with product advertisement circulator 72, product proposal circulator 74 receives product phrase information from retailer web page 42 that includes a phrase that uses a computer language module (not shown) and at least one characteristic of the phrase. Collect. The characteristics of the phrase are phrase frequency, location, font size, font style, font case, font effect and font color, word order frequency and phrase appearance on the web page.

  In addition, because each retailer is likely to propose various models of one or more manufacturers, the product proposal circulator 74 also obtains information regarding product models and manufacturer characteristics. Also, because each product is purchased through that particular retailer at a special price set by a particular retailer, retailer specific information such as the retailer's URL and suggested price information for each product is also available. Information about the product, the price, and the identification of the merchant who proposes the product at a given price are all related to each other, gathered by the product proposal trooper 74. All of the collected information described above is stored in the product proposal database 92. The details of how the manufacturer identification, product model, product phrase information, retailer identification information, and price information are all collected are basically the same as the method described above for the product ad circulator 72 in FIG. Therefore, it is omitted here to avoid repetition. However, based on the teachings described above, how such information is gathered using the product suggester 74, how it is saved in the product proposal database 92, and how it is used to create the product catalog 26. It will be clear to those skilled in the art.

  Thus, in connection with the computer example described above, the merchant specific information such as all of the requested information about each computer make and model proposed on the computer merchant web page 42 and the URL associated with the merchant. In order to collect information, the product suggester 74 can visit various web pages of computer retailers. This information is stored in the product proposal database 92 in the same format for further processing.

  As shown in FIG. 6, according to a preferred embodiment of the present invention, if the information described above is collected and stored in the product proposal database 92, various merchant proposals can be obtained from the product catalog 26 of the shopping server 20 shown in FIG. It is confirmed using the confirmation tool 93 that it matches the product information stored in. As described above, the product catalog 26 of the shopping server 20 stores product information such as product descriptions, prices and other product information for a plurality of retail businesses, and the product proposals for which the plurality of retailers are automated. The patrol person 74 may be used to select from the merchant server 40. The shopping server 20 accesses the client server 12 to represent product information stored in the product catalog 26, thereby providing a single shopping interface with multiple online retailers that are seamlessly integrated thereby providing online Facilitates comparative shopping in the environment.

  For each product proposal in the product proposal database 92, the product catalog 26 is searched to determine if a match corresponding to the product catalog 26 exists, as shown in step 94. If such a match exists, it is determined that the confirmation has been completed for a particular product proposal, and another product proposal in the product proposal catalog 92 is selected for confirmation through the confirmation tool 93. It should be noted that an editor of a person accessing the product catalog 26 through a computer or other means may execute the verification tool 93. However, the confirmation tool 93 may be executed on a computer or other automated device.

  However, if no match corresponding to product catalog 26 is found, product record creation tool 95 is executed to update product catalog 26 with a new product proposal found in product proposal database 92. In this regard, the product characteristics database 90 described above is accessed to provide the product record creation tool 95 with the product characteristics ranked by product or product category. Then, a record is created in the product catalog 26 for the new product proposal found in the product proposal database 92, and in order to guarantee the acquisition of important product characteristics for the new product proposal by the record, the product advertisement patrol person 72 Such ranked product characteristics obtained through are used. It should be noted that the product record creation tool 95 may be executed by a human editor. However, the product record creation tool 95 may be executed on a computer or other automated device.

  Similarly, for the computer example, the ranked product characteristics processed through the product definition tool 80 and stored in the product characteristics database 90 are provided to the product record creation tool 95 so that a new record is created. In addition, all of the important product characteristics for a new product proposal are acquired and stored in the product catalog 26. These product characteristics include MHz, Mb RAM, Gb, CD-ROM, CD-R, CD-RW, etc. and many other product characteristics stored in the product characteristic database 90 related to computer evaluation and Attributes may be included. Of course, the application to computer sales is only selected as an example, and the present invention is applicable to all different products and services that are effectively traded in a network-connected environment such as the Internet 100. It should be noted that there is.

  For this reason, this method allows valid new product proposals in the market to be easily integrated into the product catalog 26, thereby assuring the client computer 12 an accurate proposal for the latest products and prices. Therefore, the product catalog 26 is regularly updated. However, as evaluated above, since the update is performed through the product proposal patrol person 74, the product information is not only updated in real time, but is also periodically updated in the background of the shopping server 20. As previously described, products on the product catalog 26 are displayed in order to display valid products for purchase by product type, identification number, price, keyword and product characteristics in a desirable manner using the shopping server 20 interface. Information can be used. Product information can be displayed side by side in the browser window of the client computer 12 to allow the buyer to compare shops and select products from one or more retailer servers 40 based on the product information. Once the buyer has selected all desired products and options from all desired retailers, as described above, the buyer can authenticate the purchase order to confirm the price information, shipping information and other details of the desired purchase. Completes shopping and "checkout".

  In the above-described embodiment, the property definition tool 80, the confirmation tool 93, and the product record creation tool 95 are preferably performed by an editor of a person who uses a computer or other device. Therefore, the execution thereof is semi-automatic. It is noted that there is. Since the present invention provides these different tools with different functions, each editor can be used by a human editor with different capabilities and knowledge. For example, a relatively low level of capability and knowledge is required to execute the confirmation tool 93 and product record creation tool 95, but a higher level of capability and knowledge is required to execute the property definition tool 80. . Therefore, since it is preferred that the tools and their individual functions be distinguished, the present embodiment allows for the management of competent human resources and is an experienced human editor for executing the property definition tool 80. You can appoint an inexperienced editor to perform other tools. Of course, in alternative embodiments, the tools described above may be combined in part or in whole, and may be automatically performed through a computer or other device, which can be used by a human editor. Is no longer necessary.

  Another embodiment of the present invention is shown in FIG. The embodiment of FIG. 8 includes a property definition tool that is incorporated to gather product information from multiple sources, such as merchant server 40 and manufacturer server 44, and generate product catalog 26 in a highly automated manner. It is out. The property definition tool 800 can be in the form of software executed on a general-purpose computer such as the shopping server 20 in the embodiment of FIG. A property definition tool 800 includes a clusterer module 802, a categorizer module 804, a name selector / cleaner module 806, an image selector module 808, a property scraper module 810, an auxiliary keyword generator module 812, and a property aggregator module. 814, a description generator module 816, a similar product discovery module 818, and a post-processing module 820.

  A high level functional flow chart of the preferred embodiment property definition tool 800 is shown in FIG. Each step of FIG. 9 is described in further detail below. In step 900, clusterer module 802 analyzes retailer proposals from a plurality of product information records, such as records stored on retailer server 40, and collects them according to what is most likely to be the same product, ie, group. And specify or create a UPID (Universal Product ID) for each. Of course, product information can be gathered from product information records using patrolmen such as the ad patroler 72 described above. In the present embodiment, the patrol person can extract the product information record as it is for automatic processing by the characteristic definition tool 800.

  In step 902, based on a comparative analysis of the product with other products already classified in the category of classification, the categorizer module 804 positions each UPID in the category in the product catalog classification. The determination of product catalog classification is generally well known. In step 904, the name selector / cleanser module 806 selects the best name from a variety of different names for the products found in the group of product records and connects the extra inserted text commonly found in the product name field. Remove the name of the given text. We then assemble an optional long name that has a consistent style and has only the most relevant content for the particular category in question.

  In step 906, based on size, type, quality, harmony, source reliability, and other factors, the image selector module 808 selects the most preferred product image from all the various product information records in the group. In step 908, the property scraper module 810 scrapes the attribute values standardized for each product from each valid product information record and provides a confidence rating for each act of scraping that performs it. In step 910, the auxiliary keyword generator module 812 repeats the same processing as in step 908 except that it relates to keywords that can be expanded for each group rather than for standardized attributes.

  In step 912, the characteristic aggregator module uses a weighted decision engine to combine many scrapings and determine a final number for each attribute of each product. Process the output of the characteristic scraper module 810. Also, at step 810, auxiliary keywords are standardized based on one of various algorithms.

  In step 914, generator 816 constructs a short text describing each product. The text is not evaluable and is based on a template that uses the attribute values achieved in steps 810 and 812. For example, this can be done by randomly selecting one of many descriptive templates that can be illustrated with references to the most important characteristics of the product and performing unimportant word or phrase changes to avoid monotony. Can be achieved. The final step is to ensure proper sentence formatting, punctuation and capitalization.

  In step 916, the similar product module 818 recognizes related similar products such as accessories, parts, connectors, etc., and the bundles or kits that make up the set of similar products. Any item below the confidence limit can be specially handled in various ways, either manually or automatically. The similar product module 818 is described in detail below.

  Clustering performed by the clusterer module 802 is a process of grouping together various retailer proposals that do not have a pre-assigned UPID and creating a new UPID for each group. This process results in a one product to multi-price relationship. Many features are extracted from retailer proposal records, manufacturer records, distributor records, and other records (collectively referred to here as “product information records”) to enable clustering. Is done.

  Clustering may be performed by the manufacturer, that is, the “primary (first) key” of the process is the manufacturer name. Thus, the first problem to be solved is to determine that the different forms of reference text for the same manufacturer are really the same. The risk of accidentally clustering products from different manufacturers is small, but when it happens it is a very serious error and therefore it is important for the system to avoid this error. Discovering patterns in manufacturer identification numbers and model identifiers has made the system very large. This is a proof that the manufacturer has already been separated from the product name-it is unusual for a product information record to include the manufacturer or brand name as part of the product name field. Thus, a first pass may be required to configure a manufacturer name field for use in clustering.

  As long as the manufacturer name (or brand name) is obtained separately, the clusterer module 802 assigns this name as the data string used for the sub-sequence search to select all product information records, and the product In the information record, the manufacturer name starts with this sub-array. For example, clusterer module 802 can be started on a product proposal space having a manufacturer name that begins with “bell”. This defines the data set for processing when selecting a product proposal with a manufacturer column such as “Bell Atlantic”, “Bell Industries, Inc.”, “Bell Micro Products”. These names are standardized (excluding unnecessary parts such as “Incorporated” and “Corporation”), and more words are found until the smallest manufacturer name that matches one manufacturer name from the catalog is found in the catalog. Removed from the end. As an example, all data providers labeled with “Sony” today have several different names, and therefore, one of several “Sony Like” manufacturer names is designated as “Standard”. Further work is required to determine what should be used. The system may use manufacturer aliases from various data sources, and the catalog is “bootstrapped” either manually or through a UPC match (which does not require the retailer to supply the manufacturer name). The alias may be quoted when the product is created and the retailer proposal is placed on a new product.

After defining a data set for a particular path, the clusterer module 804 begins the clustering step. There are many known standard AI “clustering” algorithms, any one of which can satisfy a baseline clustering. However, sufficient clustering of the product will often increase baseline clustering with many elements and adjustments as described below. A simple overview of the baseline clustering procedure is as follows:
a. If none of the clusters, i.e. the product group already exists, the first product becomes the first cluster b. Another way to search through clusters for close matching c. Combine close fits d. If there is no close match, it becomes a single cluster

  There are several approaches to determine “close fit”. The following section describes an example of that approach. The first approach to determining if products are the same, i.e., the same group, is to examine manufacturer identification numbers, possibly UPCs and model IDs, and normalize surface changes in their names. In the meantime, the clusterer module 802 may create several paths and combine together what was originally another cluster together to limit that possibility. This limitation is often the potency of the clusterer module 802, which says that the letter-number pattern is inductive and appears to be in various product families around the industry. Is to use this knowledge to filter out bad data. For all data sources, bad data is common to how it works in an ideal world. The clusterer module 802 really works differently in many ways due to the fact that it is.

  In one example, one manufacturer may give all of the video camcorders an identification number that begins with the letters “VCM” followed by one or more letters, and may give three numbers. However, some merchants insert their internal SKUs in the manufacturer identification number field (actually a very common problem). Since the retailer's internal SKU does not conform to the number-character pattern described above and is actually very different, the system programmed to retrieve the identification number in the expected format is an array with the internal SKU. Will continue to inspect other parts of the record, and the record will find some other data to allow clustering of problem proposals (for example, a bad manufacturer identification number) Regardless, the product name is complete and perfectly matches many other proposals in the database).

  The clusterer module 802 can also adapt many methods to retailers that are changing the UPC of a product (most of which are not intact). Some remove the first zero, add one or two digits with their own internal meaning to the end, and exclude the checksum digits. Through automated trial and error, clusterer module 802 attempts to deconfigure and reconfigure specific merchant patterns that pull UPCs. In order to see that the transformation rules fully reconstruct the final UPC from the retailer's unique UPC, the constant complete UPC has at least one known source of information, and many known practical This can be done by applying technology (for example, this can be done using a Hidden Markov model or a standard logic program). The Hidden Markov Model (HMM) is a known variant of a restricted state machine that has a set of states Q, output alphabet O, conversion probability A, output probability B, and initial state probability Pi. . The current state cannot be detected. Instead, each state produces an output with a certain probability B. Normally, the state Q and the output O are considered, and three HMMs (A, B, Pi) are indicated. HMMs are known for use in speech recognition and other applications.

  The clusterer module 802 has many other functions described below. After clustering is completed in step 900 of FIG. 8, additional merchant proposals that appear beyond the week and month periods are typically matched using the same method as the original clustering in step 900. Automatic verification is performed.

When present, model IDs and product names are often the best clues to determine if items in different product description records are the same, and therefore clusterer module 802 should be clustered together in the same group. It is. However, there are many ways this can fail and should be considered a failure. The first is only punctuation and capitalization, with the following differences:
"$ 69.49 Sony MD-74 Mini-Disc player"
"$ 75.99 Sony MD74"
"$ 68.00 Sony MD 74 Mini Disc Personal Player"

Another one has the following extra term attached to the model ID:
"$ 59.99 Sony Black MD 74"
"$ 68.99 Sony Silver MD-74 BB"

  Here, “BB” means “bass boost”, which is a simple feature of all such models and does not really indicate that it is a different product. In contrast, the difference between Black and Silver shows a clear difference. Many people who want to buy a silver model do not want to have a black model, and in some cases a retailer may be responsible for one more color than the other in the same device.

Only basic logic is required to handle and resolve differences in punctuation, but you can gain special handling for items such as product and kit colors, sizes and superior features, as shown below: it can.
"$ 99.50 Special! Sony MD-74 Premium Kit with Mini Speakers and Leather Case" ($ 99.50 Special, with Sony MD-74, Premium Kit, Mini Speaker and Leather Case)
In this case, the price differences; the presence of “kits”; the description of the product database items (speakers, cases) that indicate what should be in their own or another category of their own products, all clues, It conveys that this proposal is likely to be a “set” of products, including MD-74, not just MD-74 itself.

In many cases, ambiguous sequence matching is important in providing some clue as if the proposals are the same and paying attention to the spelling of the source data. Spellera is, for example, as follows:
"$ 74.50 Sony Mini-Disc Player MD-74"
"$ 69.49 Sony Mini-Discc [sic] Player MD74"

As a first pass, an ambiguous match on the entire array can easily complete the first candidate group for clustering. For example, the following list is chosen from a number of suggestions that are countless as the first cluster, with a single ambiguous match:
"Sony MD-80 Mini-Disc Player" (Sony MD-80 Mini Disc Player)
"Mini-Disc Player MD-74 from Sony" (Mini Disc Player MD-74 made by Sony)
"Sony New MD-74 music disc player"(Sony's new MD-74 music disc player)
"M-740 Symphony Synth from Moog" (M-740 Symphony Synthetic Mook)
"Sony MD-74 Personal Music Device" (Sony MD-74 personal music device)
"Sony 8-inch Mini-TV-80" (Sony 8-inch mini-TV-80)

  Note that all these have clearly overlapping portions of text content. However, “Symphony” and “Sony” have a 67% ambiguous match—8 of the 12 characters that connect both words are the same and in the same order. This shows how some ambiguous candidate matches still cannot be avoided. Nevertheless, the fact that the clusterer module 802 has only 6 products to process and not 600,000 is tremendously limited. The remaining is to specify that the difference between MD-74 and MD-80 is important in order to separate the first item in the above list from the remaining items in the list. ”(Or“ Moog ”) and“ TV ”to determine to invalidate the fourth and sixth items, respectively.

  Clusterer module 802 also ignores certain words. Note that in the example above, the words “New” and “Player” really do not add anything. “New” is an example of an exceptional word that is very often used in all types of product markets and is handled separately from the way random words are handled. In particular, it should be ignored for all purposes except to distinguish new products from second-hand and restored products. On the other hand, “Player” is useful in the initial path to create the initial cluster, however, the system has such great importance that the presence vs. absence of this word is counted as a different product. Not shown. This is based on the fact that “Player” is a “common noun” associated with the entire category of products. Other examples are “TV”, “CD”, “video” and others. In particular, at a stage after the clustering step 900, common nouns are ignored because it is merely an option for the trader to include or not include it in the product name. This does not remove the fact that they are an important clue in creating the first pass to be included in the initial cluster.

Numbers such as “74” and “80” in “MD-74” and “MD-80” are clearly decisive to distinguish the two models of the product from each other. As a general rule, different numbers in model names and product names should be used to indicate different products. However, there should be exceptions as well. For example:
"$ 12.99 Hasbro Wayne Gretzky # 13 ActionFigure"
"$ 13.99 Hasbro Wayne Gretzky 6-inch Action Figure Mighty Ducks"

  The difference here between “13” and “6” makes the clusterer module 802 expect that they are different products when they are actually the same. The best way to solve this is to recognize as a system that attributes for a particular category may be represented in numbers and may be connected as part of the product name. In this case, the sports action figure has a jersey number (“# 13”) as a possible attribute and a height in inches (“6-inch”). Having system checks for these parameters can avoid dramatically reducing the probability of matching due to the mere appearance of different numbers in the product name array. Without finding such a parameter, whenever there is a different number in the product name, the system reduces the probability of a matching product estimate.

Despite this, additional numbers in names such as the difference between “3” and “4” below exclude clustering:
"$ 13.99 Hasbro Wayne Gretzky # 13 ActionFigure Series 3"
"$ 14.99 Hasbro Wayne Gretzky 6-inch Action Figure Mighty Ducks Series 4"

To broadly cover specialized retailers and many categories specialized for a particular category to determine the names that are considered important to divide the cluster, i.e. form a group of items corresponding to a particular product Similarly, a difference frequency analysis is performed between proposals already cataloged in a particular category and the entire catalog. The result of this analysis is, for each category, a list of terms that are generally more frequently in that category than in the entire source. This is useful for categorizing new items because a very common word (or phrase) in one category is usually a word that the system considers not showing any differences between products in that category. (Described later) and is also effective for clustering. Take the word "saber" as an example. This is a rare word in the whole document, but it is mentioned very often in action figures and is given to the superiority of Star Wars action figures that "appear with light sabers". Now assume that the system has found two proposals:
"$ 5.99 Obi Wan Kenobi 6" Nabo garb withlight saber "
"$ 5.99 Obi Wan Kenobi 6" Naboo garb "

  Usually, the presence of a "saber" on one side but not on the other places more emphasis on counting items as different, however, a system of how common is the "saber" to be listed in this category Recognition increases the possibility that the word is only an optional descriptive phrase and that one action figure is not special to the other. This is also probable and only one of many factors that can be evaluated.

  Another practical check that assists the system in the clustering step 900 is to check previous probabilities, for example how much emergency, as evidenced by valid product information records and other information content. Is to check if there is a similar product on the market. The higher the number, the more suspicious even the insignificant differences in product names in the system. However, if the number is low, the system can tolerate minor differences. For example, if there are only one or two “Abraham Lincoln Action Figure” products in the database, the probability of a proposal to place additional products is relatively poor. In contrast, having more than 100 different “Luke Skywalker Action Figure” products in the database suggests that a new Luke Skywalker proposal with an insignificant difference in its name is a very different new model. is doing. In other words, the chances of a Luke Skywalker action figure being clustered incorrectly are initially very large—the chance of belonging to the group to which it is given is 1 out of 100. On the other hand, if there are two Abraham Lincoln action figures, the chances of a proposal belonging to that cluster or another cluster will be immediately 50-50. This is a factor in the reliability of clustering prediction.

  In determining the number of groups, the retailer's scope is considered (1) so that the maximum selection of similar items proposed by a single retailer is provided as the minimum number of groups for that item's family. (2) It may be considered that differences in the product range of different retailers may be inferred to provide further clues to the appropriate number of groups. For example, if the system addresses hundreds of proposals as seen in the “Luke Skywalker Action Figure” and assumes one retailer out of 37 different “Luke Skywalker” proposals, the system ( It can be estimated that at least 37 clusters are needed for this product family (believing that one particular retailer does not replicate too many proposals in the dataset). Furthermore, if there are about 1.5 unique proposals for every 10 proposals already in UPID products (for example, KB Kids has 23 such action figures) This means that 3 out of 23 are unique among those proposed by KB Kids only, and this kind of proportion is found in all retailers where Luke Skywalker action figures are already UPID. (Meaning that it is the average of the percentages), if we get the same percentage of historical differences, to estimate how many presumed new unique products exist (UPIDized) Beyond the remaining set of merchants who do not yet have product proposals), the system can use this information. All these measurements are effective for “practical” or “practical” measurements that can be performed to weight the confidence levels of the various temporary clustering joins-in good agreement with the measurements described above Couplings (ie close to the estimated figure) have a higher confidence level than those that deviate significantly from the measurement.

  In many cases, the name, description, and specification make the clusterer module 802 difficult to determine if two products are the same, its own price is an important and possibly obvious factor. For example, if one proposal is $ 7.99 ($) and another proposal is $ 59.99, and superficially the description is unlikely to be the same product. However, there are some cautions. First, note that the clusterer analyzes and analyzes exceptional situations such as part-time disposal, inventory clean-up sales, repaired items, and re-certified items (such as returned products in unsealed boxes). Should. In some cases, these items are further reduced in cost.

  Another difference that has been brokered is the typical difference in retailer prices. Many first tier retailers typically charge up to 30% or more than some discount retailers. Another consideration is price competitiveness and consistency in the catalog. In one category, the system can determine from an already cataloged item that price fluctuations between retailers are typically greater than that of other categories. After considering all these factors, if the price difference is still large, the likelihood of a proposal in the same product decreases accordingly.

  Without considering retailer pricing, most retailers list the MSRP of a product to typically boast obvious “savings” derived as the difference between the retailer price and its MSRP. Since retailers typically incorporate the same MSRP from manufacturers and distributors, retailers tend to be the same in what they interpret as MSRP at the actual retailer price itself. When this information is valid, the clusterer module 802 is weighted, and actually is heavier than the retailer pricing. Many other factors as well, because the data is sometimes incomplete (ie, through a typographical error in a retailer or, rarely, another retailer reflects more up-to-date MSRP). Through retailers with older MSRPs, weighting should be done as well as Boolean checks.

  The dilemma found in these parameters is that clustering is ideally performed before categorization. This requires a trial guess like product categorization, even though categorization does not finish until clustering is finished. Thus, a dialectical or iterative process flow between the algorithms of the clusterer module 802 and the categorizer module 804 is sometimes desirable and even necessary. The clusterer module 802 modifies the cluster membership in terms of experimental categorization, but this is followed by a re-check of the categorization, in which case the categorizer module 804 changes its category “guess”. Correct as a result of the cluster. This iterative process continues until the results of both clustering step 900 and categorization step 902 stabilize and both exceed the required confidence limits. The combination of both results with the highest minimum confidence score between both the clusterer module 802 and the categorizer module 804 is dominant. In other words, if clusterer module 802 has clusters C1 and C2 and categorizer module 804 outputs categories A and B, the outcome matrix shown in Table 1 below results. In Table 1, * indicates clustering reliability / categorization reliability.

  Assuming a confidence limit of 0.70 for both clusterer module 802 and categorizer module 804, the system proceeds in cluster C1 and category B, and the minimum confidence in that scenario is 0.71-superior to other scenarios Both exceed the aforementioned limits. Of course, other algorithms that correlate the results of the clusterer module 802 and the categorizer module 804 may be used.

In general, the system speculates that the more different words in two product names, the less likely they are the same product. However, the system needs to be able to interpret synonyms, narrower terms, and broader terms in an intelligent manner. For example, consider the following:
"Sony MD-74 mini-disc player" (Sony MD-74 Mini Disc Player)
"Sony Inc., MD-74 muic listening device" (Sony Inc., MD-74 Music Listening Device)

  On the surface, different words appear to be more than the same word. However, “player” is a narrower term for “device” (in other words, “device” is a broader term for “player”). On the other hand, “Sony” and “Sony Incorporated” are treated as synonyms. These words show partial confidence in the fit. In a join with model number matching, these facts appear to be sufficient for the clusterer module 802 to confidently cluster these proposals.

Also, one product often has two or more variants. For example, a toaster oven may have black and white, whose model ID is one of the following:
“PG-400-B” (where B represents black)
"PG-400-W" (W indicates white)
“PG-400” (where the text description follows black, white or both)

Other examples include right / left-handed golf clubs. These are shown as follows:
Titleist Pro 700 Driver R
Titleist Pro 700 Driver L

  In general, these variants are superficial in some respects, but still have a separate individual identification number from the manufacturer and are not less important to a particular buyer. They are therefore given unique product IDs. However, they may be part of a single product line or a variation of a basic model configuration (eg, if you think of a right-handed golf club as a basic model, left-handed is a variant and is based on a black toaster oven. Think of it as a model, almond color is a variant, etc.).

The intention to collect products is partly a matter of hand. However, clustering generally corresponds to assigning a single model, ie, product, to various price proposals by grouping together with product information records corresponding to the product. However, there are meaningful composition variants, which are counted as technically different models, but are often "basically" the same model and slightly modified styles by customers and retailers, And so on. Similarly, a product model is part of a series, and multiple product series are considered part of a product family. Thus, the clusterer module 802 can provide multiple levels of super (upper) clustering and / or sub (lower) clustering. One of the charts labeled with many possible meanings for these levels is as follows:
(1) Product line (2) Product family (3) Product series (4) Model (*)
(5) Model configuration

  The asterisk (*) indicates the baseline clustering performed by the clusterer module 802 at the retailer's suggestion level and occurs according to the procedure before super-clustering (level 1-3) and sub-clustering (level 5). An example of all five levels is the Fuji Lifebook P-2040 with 384 MB of RAM. The Fujitsu brand has the “Lifebook” product line, which has the “P” family of notebook computers (as opposed to the “S” family), and the family has the “2000” series. (For the “1000” series), which has a “2040” model (for the 2080 and 2100 models), and can optionally rebrand products with 384MB of RAM (For those with 256 or 512MB RAM).

  The same basic method of clustering is used at any level, simply having features that either consider or do not consider to be important for different sets of differences in naming and problem levels.

  Categorization step 902 includes processing for assigning each UPID to the appropriate category in classification 700. This is usually done in two processes. First, the attributes and attribute value sets defined for each category are examined in a known manner along their aliases, synonyms, broader terms, and the like. Secondly, the actual product information records already classified in their respective categories are examined. Several AI machine-learning algorithms can be used for classification including case-based reasoning, genetic algorithms, neural networks, etc. without limitation. What is imported is feature extraction that precedes the invocation of the machine-learning module, not so much that a special type of machine-learning module is used.

  In the feature extraction process of the categorization step 902, even if the attribute name, numerical value, unit of measurement, brand name, keyword and phrase found in the product description, each matching item found in the product information record is included in the category. Count to benefit existing products. Conversely, products that are found to be competitive are primarily deducted from the probability score. Eventually, the final score reaches each UPID for each category. It is desirable for the confidence limit to exceed one, only one of the leaf-node categories. In the few cases where this result is not obtained, manual (or other external) authentication may be used, i.e. the categorization is different.

Usually, for market reasons, resellers give very long and very “unpleasant” names for their products. Resellers try to please search engine patrons just as they are people--that is, they want to include each possible fragment of relevant text in the product name This is especially true in the world. On the other hand, they want Google and Yahoo! I am afraid that I will not be able to obtain search results for ranking. The result is
"Sony MD-74 Mini-Disc Player"
The ideal product name is often written in the product record as follows:
“New Sony MD-74 (MD74, MD74) Mini-Disc Player Personal Music Listening Device with Rebate (Free Discount) and Free Leather Case (For Free Leather Case) Now For Grads D

  Fortunately, not all names are actually long and unrelated. However, the name removal step 904 is still required in many cases. The first obvious step in determining an attractive name screen to be used for UPID product records is to delete very long ones and prefer one that is not long.

Another type of unfavorable name is trying to overfill all its name fields, like a mini product specification table, for example:
"Sony MD-74 23-hr battery (23-hour battery), 6 watt output (6 watt output), headphones (headphones), 8 oz (8 ounces)"
“Think about this with:
“Sony Corp. MD-74 Mini-Disc Personal Music Listening Device”
Here, the length of the name alone is not helpful because they are almost the same length, but the latter is preferable to the former for the purpose of cataloging. Since the former does not include many attractive names and attribute value columns from the product record, the name is designated as “disadvantage”, ie, selected as the product name in the name selector / cleanser module 806 Let's not have.

  In addition, vendors have a convention of overloading the name field of the product information record to bring in many other elements of information apart from the name of the product. In practice, this is not a limitation in vocabulary. However, the role of a particular language is semantically related to the market, and that market is more commonly included in the market through manual lists, or in the “fluff” of markets that disrupt product names. Is one of the markets through the statistical account processing of Word. The statistical approach is attractive to be further automated. This procedure requires some sample data with a comparing data set showing the same product name that does not have a market language in order to tag it to have “market language” in its name. Differential analysis with word / phrase frequency, word proximity and other standard statistical NLP methods can be applied to determine a good probabilistic profile that constitutes the market language for each category of product. .

  It would be desirable to generate any longer standard names if the name selector / cleanser 806 limits the possible names to simple, concise and complete product names. The standard name means that it follows a consistent form throughout the category: (a) the most important variant components (color, right hand etc.) and (b) the most important attributes (digital camera resolution etc.) Only mentions. Having a reliable and consistent style of long names allows those using that catalog to experience maximum flexibility in making short names or long names surface as the best fit for their application.

  In step 906, based on size, type, quality, harmony, source reliability, etc., the image selector module 808 selects the most preferred image from all the various sources. Various rules and limits can be used to select the most preferred image. For example, the image may have a certain minimum resolution and size. Alternatively, the image selector module 808 may be programmed to use an image from a product information record from a list of preferred information sources (eg, merchant server 40) in order of effectiveness.

  In step 908, the property scraper module 810 extracts a web page or PDF that forms a source of information for retailers, manufacturers, distributors, critics, etc. to extract product specification information from the source in a standardized form. Analyze and analyze product information records such as documents. The property scraper module 810 discards or leaves the text of the product information record in question.

  The property gathering step 908 is performed as follows. First, in order to separate the main product specifications of the page from cross-sell and up-sell products, and linked accessories, DOM (document object modeling) Execute. Next, different sentences, phrases and table structures are analyzed in order to place individual specifications one at a time on the page. What does not exist and other functions are handled separately, so that the characteristic scraper module 810 is not mistakenly configured to incorporate them into the product. Parsing synonyms and aliases to standardize the special terms used in various product information records for both attribute names and numerical values. Bonus (additional) keywords and specifications are drawn that do not fit the pre-defined specifications for the product category in question, including collecting new specifications that will “fall between rifts”. Based on whether both the attribute name and numeric value are found, or just a numeric string, or some irrelevant word is found between the attribute name and numeric value, or a line break or adjacency in the table A level of confidence is assigned to each act of collection based on whether there is a selected cell (introducing the risk that all of these items are present, the specification has been changed, limited and abandoned anyway) Therefore, the specifications that should be may not appear correctly). Scrapers may also pick up conflicting information on the page (eg, explaining that they sell a single size but later are valid for many sizes), which reduces trust. , And text that appears to be correct (ie, text that is more closely placed with other specifications on the page) is given greater confidence. The result of the weighted combination of all these methods is a confidence score for each individual specification from each product information record collected.

  Handling both synonyms, eg “lbs” for “lb”, and conversions, eg “2.2 lb” for “1 kg”, converting and recognizing units of measure to standards for each specification Often important to the system. In other words, the fact that the attribute name is not found can sometimes be certain that the unit is the correct value (eg, for a 3.1 megapixel camera, the uniqueness of the “megapixel” unit within that category). Because of the nature, the word “resolution” is not necessary to infer that “3.1 megapixel” is the resolution). Numbers may be parsed in all differently represented styles, including styles with fractions, Roman numerals, and commas. Numeric ranges need to be recognized both as possibilities in attribute setup, i.e. they have a composite attribute composed of "minimum" and "maximum" primitive attributes. Different types of phrases may indicate ranges such as a comma-separated list of numbers (eg, 1, 2 or 3 players), a minimum-maximum range connected by a hyphen.

  Body type attributes may have different rules. The specification requires an attribute name for which a Boolean type attribute (no / yes) should be present, and does not require the word “yes”. Determining the correct “no” value usually requires the system to search for other negative words (eg, “selective” or “not included”) other than the simple “no”, so somewhat It is difficult to handle.

  In other cases, especially if you have an attribute that allows multiple numeric values, or if the language of the numeric value itself is obvious, limit it by including the attribute in a limited description in the product name field or product text. For example, “This HDTV television is ...” clearly indicates to the reader that “HDTV” corresponds to the attribute “compatibility”.

  The source document from a particular product information record is often a tag-laden document type in HTML, XML, PDF or other tags. This is both a benefit and a loss to the property scraper module 810, and these tags can indicate or hide the required specifications. Therefore, multiple paths (using different methods of handling tags) may be used, as shown below.

  One way is to simply ignore the tag by analyzing the tag. Although this method is simple, many specifications can be obtained in practice. For example, if you can read “Height: 6 inches” from the action figure product page on the website, read “Height: <TC> <Font: Helvetica> </ B> <I> 6inches” in the HTML information source. Can do. In other words, there are tags for the purpose of determining the position of information in the table, changing the font from one column to the next, and the like. By simply confusing the tags in one row of the table (while keeping the information in one row), the scraper finds "Height: 6" and gets very close to the specification being gathered.

However, in many other cases, tagging is analyzed and analyzed rather than simply abandoned to obtain the desired result. Looking at the same example above, you can see the following words at the top of the column,
"Phantom Series 4" (Phantom Series 4)
At the bottom of the column, you can see the following words:
“Vehicle included” (including vehicles)
By analyzing and maintaining the tag of the column description, the characteristic scraper module 810 has “Phantom Series 4” in one row, “Height: 6 inches” in one row, and “Including vehicle” in another row. Recognize If all tags are ignored, the property scraper module 810 loses this column boundary and recognizes the following consecutive columns:
“Phantom Series 4 Height: 6 vehicle included”
This makes it difficult to analyze and analyze and probably has some risks. The risk is that the system thinks that the action figure is 4 inches high and is with a 6 inch vehicle. For this reason, it is important to actually analyze the tag and thereby maintain the document structure.

  There are countless other ways in which tagging is effective. In another case, the system tries to determine that the list of specifications ends with a text block. For example, product information records often change fonts, text styles, and paragraph indentations when the specification ends and when the list of cross-sold products begins. It is important that this change be noted, and cross-sold products will not be unexpectedly configured as a characteristic of the main product itself (for example, to configure a memory card that is an optional accessory for a digital camera) Is configured with the camera).

  Many product information records produce tables that unfortunately do not place attribute names and values that are closely related together. An example is shown in Table 2 below. In Table 2, "P-2000 series configuration suitable for you" is taken up.

  In this example, many variant models are listed together in one table, and the header column is parsed and collected one column at a time to correctly collect specifications for each model.

  Some tables found in product information records are more complex and represent multi-dimensional join specifications. A very common example is the waist size and crotch size combination of pants, usually some are valid instead of all possible combinations. An example is shown in Table 3 below. In Table 3, * indicates that odd and even sizes are valid, and ** indicates that even numbers are valid.

  Note that not only is the table parsed to actually collect the “standard” tables shown in Table 4 below for the scraper, but also the annotations are understood.

  The system forces some attribute values to default to “no” or “non” when there are multiple sources that do not mention the attribute and when there is no source that describes it. To do. This is required because vendors tend not to describe when their products are missing features, but only when the products have features. For example, only a small number of high-end digital cameras have interchangeable lenses. If there is no description of this feature on the product page, it is actually guaranteed that the camera does not have it. However, merchants do not miss the opportunity to brag, even if they do not have their camera with this type of lens. Therefore, when the system finds multiple reliable sources that fail to describe features for a particular camera, it detects this pattern and sets the default “No” for the attribute “Interchangeable Lens”. You can start.

  The auxiliary keyword generator 812 analyzes each product information record with a reference to the open keyword for each category rather than for standardized specifications. These continue from the DOM analysis, where the system recognizes the columns and evidence that the product information record is supposed to be placed in front of the specification and does not properly conform to the pre-defined specifications in the system. This capture is a very special one-off specification, but on the other hand falls into the rift. For example, in 50 different stroller seats there are only one or two products called "one hand harness release", but this characteristic is not a recognized and standardized specification in the category in which the attributes are listed. Nonetheless, being represented in a set of product information records that are correctly combined with other specifications for the same car seat adds a phrase as an “Appendix” to the standardized specifications. Make that system possible. (1) Categories with many secret and unique features that are not well standardized, or (2) System management that stays on top of new combinations of specifications that appear when manufacturers add new features to their products This is a very powerful feature to help the administrator (the administrator has re-examined the case where a warning was issued and a number of overlapping auxiliary keyword specifications were added for a particular category, and the The purpose is to find out if the case constitutes a new specification to be added in a standardized way).

  The preferred embodiment characteristic aggregator module 814 is a decision engine that assigns a weight to one product information record that is greater than the others, and specifications that are collected from all product information records for a particular product in the final set. Try to combine. This eliminates the very common contradiction found in many sources. A characteristic aggregator specifies great importance in finding multiple testimonies for a specification-that specification contains data from multiple sources that have different formats and match in the basic content of the specification in question. Defined to have.

  The weight may be set automatically or manually. An automatic set will result in how often the product information record is canceled in time with the final result. The weighting includes the entire product information record, just one category, just one attribute in one category, just one numeric value of one attribute in that category, just one manufacturer of the product in that category Either can be set. Also, for image reliability, overall weighting can be performed for each category, for each manufacturer, and for each manufacturer in the category.

The main components of a similar product module of the preferred embodiment are as follows:
1) Product-related tools for manually defining accessory relationships between products and categories with constraints, and for gazing or manually ignoring specific products designated as similar products created automatically, 2) A similar product engine that includes logic and algorithms to search for raw source data from where the relevant product relationships have been “discovered” automatically.

  All of the following relationships can be defined:

  Category-category relationship: Requested on condition that the product of category A is similar to the product of category B. Characteristic constraints are arbitrary. For example, CompactFlash® cards are similar to the PDA category, and have the limitation that the memory module type for PDAs is “compact flash” in order to obtain for that relationship.

  Category-product relationship: Requests that the product category is similar to a specific product. As an example, the “XBOX cartridge” adopted as a complete category is similar to the specific product “XBOX game console”.

  Product-category relationship: Require that a particular product is similar to a complete product category, with any constraints. For example, a particular leather case is similar to the complete category of “digital camera” and has the condition that the characteristic of “form factor” is shown as “compact”.

  Product-product relationship: requires a specific product to be related to another specific product, for example, a specific model printer cartridge is similar to a specific model photo printer.

  Software tools that allow all of the above-mentioned relationships defined in the manual may be provided with or without constraints and are selectively marked as “potential”. A software tool may be defined to allow the user to show similar products as one of “accessories”, “parts”, “supply”, and the like. Each type also has a “highlighted” type, which commands the knowledge of a human editor to be of special interest. In the absence of relationships highlighted in the manual for each product, the discovery engine removes products that are closely similar (the comparison of categories and ambiguous names) to give variation to the top three highlighted relationships. Automatically emphasize the relationship of the highest score (eg, the same compact flash card size variation has the highest score, but it may be desired to simply emphasize that size variation). A software tool may be provided to allow a potential relationship to be negated, i.e., to show the user that the category should not be considered similar to other categories. This is to help the discovery engine avoid erroneous wasteful processing. This is accomplished by creating a potential category-category relationship with a manual score of zero.

  Similar product module 818 includes an algorithm for verifying that the components of the various product information sources are “similar product tables” on that web page or other information source. For example, it is certainly found that the HTML template portion of the merchant web page is in its accessory list. This is done by visiting the merchant's page, searching the current high score and manual relationships, and identifying and recording the area of the page where these links were found. Thus, on subsequent visits to the merchant site, this information can be used to adjust the score, depending on the links to possibly related products found on the page. Similar product module 818 checks for products in a known category according to links found in various similar product tables. When the scanner finds n repeated examples of products in the same category for products in the currently scanning category, an appropriate category-category relationship marked as “potential” is automatically created, e-mail or Appropriate category managers are notified through other communication channels.

Similar product module 818 typically operates in accordance with the following algorithm to create a similar product at step 916.
1. For each product in category a. As each retailer, get proposals from all retailers (this part is connected to multiple)
i. Load and parse a retailer's web page (and cache the page)
ii. Search for links to “related products” on the page iii. If there is a link on that page that leads to the auxiliary accessory page, follow that link: a. i. Return to iv. Reversely search for product reference sentences, return to the catalog, associate them, calculate product-related scores, and record their placement and scores. Calculation of Final Product Relationship Score Each of these algorithm steps will now be described in detail. The main challenge of the similar product module 818 is that the merchant can recognize when it is associated with a product on its web page. Because of the variability in the representation of product names, only reliable methods that can clearly recognize a product can go through “retailers SKUs” or unique product identifiers that retailers use to associate products. In order to be able to recognize that the link on the merchant page is actually already associated with the product in the catalog, it is necessary to perform the “Retail Merchant SKU Discovery” step. Note that the first time you search for a merchant URL, check to see if you have performed this step before, otherwise the program will load all of the URLs from the merchant present in catalog 26. The URLs are compared with each other and the variable part is determined as SKU, which is placed back to the product IDs in the catalog 26 and a symbol that helps free the merchant SKU from the URL Recorded as each retailer, along with defining the scope of the URL encountered on the retailer page The URL is analyzed using a delimiter and each sub-sequence of the URL is searched in a list of previously recorded SKUs from the merchant. 818 recognizes that the URL is associated with a product in the catalog.

  Often, when a similar product module 818 processes a merchant's page or other source of product information to retrieve relevant product links, the merchant may place a list of that product on other pages. decide. The page is, for example, http: // www. buydig. com / shop. php? prod_id = CNPSA70 & adv = cnet. It is. In that case, the similar product module 818 analyzes the language to find product relationships at and following such links. A similar product module 818 can use a mini lexicon, and that this link is actually related to the accessory for the given product for each resulting product-related final score. Trust can be included. Sometimes the links to the accessories to list may actually be small images. OCR can be used to obtain text using the image in a known manner.

A number of factors are used to calculate the product relevance score for relevance found at each retailer. These include the following:
Provide a higher score based on whether the related product was manufactured by the same company.
-If the link contains a language such as "... for ...", check how well the rest of the text fits and increase or decrease the score accordingly. This is incorporated into the consideration of a reference sentence such as “For Palm 500 Series”, where the word “Series” indicates that 515 is part of that series, so Palm 515 decides to get a bonus. In contrast, if the link is “Palm505 only”, the presence of the word “alone” indicates that Palm 515 associated with this product gets a lower score. This type of analysis requires confirmation of model IDs, different types of recognition of language inclusion / exclusion, series characteristics and conformance of model IDs.
If the link contains a general reference to a complete category or product, for example, when the PDA category is scanned, “Viking MMC32M 32MB MultiMedia Card (multimedia card) for a MP3 player (MP3 player) , PDA or digital camera (digital camera) "or the like, a bonus is given depending on whether or not the wording of the category matches. This requires a good vocabulary range of synonyms for category names.
When a link is found by analyzing an HTML document from a retailer, the “group text concept” that occurs before the link is searched. For example, there are “Add-ons” and “XXX accessories” in front of a set of related product links. This search is difficult because there are many ways for retailers to put such headings in HTML. Consider all placement, text features, and language when searching for relevant product links. When a group text concept is found, the score is increased if the language indicates that the list of links constitutes the associated product. Other types of associations such as “Other people who purchase this product also purchase these ...” reduce the score. These “headers” are actually sometimes small images, and OCR to obtain text using the images may be used in these cases.
• Abandon associations for products in the same category.
• Consider the price of related products when accessories of major products such as digital cameras and notebook computers are generally lower cost than major products.

If all of the relevance is collected for the product from all of the retailers, the entire relevance score is calculated. The factors in the calculation are:
Collect all of the merchant references and give higher scores to related products linked by one or more merchants. If not all retailers yielding the product to be scanned, this is not entirely certain, for example it may be just one retailer in our list of retailers for a given product.
• Retailers who evaluate how well characterized the related product (set manually by the catalog editor) increases or decreases the contribution to the overall score for a given relationship.
A potential category-category relationship takes into account and contributes to changing its score. The relationship depends either positively or negatively on the previously discovered potential relationship score.

When all categories of a product have been scanned, consider the following to determine whether the entire category itself is similar to a product or some other category:
Category counting-count the total number of related products in each category. It is more likely that a category with further related products is certainly related to the overall similar category of products, and thus its score is adjusted based on the category count. For example, in the “portable device case” category, almost all products in this category already have relationships (or many relationships) with other products. This fact indicates that the complete category itself, i.e., "portable device case", has more opportunities to be surely associated with some products as a similar category of products.
• Giving score penalties to related categories of "miscellaneous" types-such categories have several products that have a solid association, but usually in miscellaneous categories, Nothing related to a specific product.
• Given the length of the category tree and such relationships are usually undesirable, give a penalty for scores to relationships from sibling categories (eg, desktops associated with notebooks).
Catalog editors mark specific categories as they improve or worsen the likelihood of having an associated product, and the system uses that information to adjust the score (these are potential category-category relationships) is there?).
• If the scores exceed the scanning limits, they are saved in a database where the score is manually invalidated if necessary. There are actually two limits: a “scanning limit” and a “publish limit”. When the “scanning limit” is met, the relationship is saved, but not at the published limit. The idea here is that a good relationship gets a low score for several reasons, but the user may ignore the score manually if he thinks the relationship is appropriate for publication.

  The final path for each product is to “emphasize” the top three (or top n) products. The highlighted related products are composed of the highest-scoring products and products that are not at all similar (to give the user a good change when the related product page is first displayed). The similar product module 818 takes the association with the highest score and moves to the next page to check the category that has the product name and is similar to the product name. If the product exceeds the similar limit, the second product is not highlighted and the system moves to the third highest score product, etc. The catalog editor highlights the products in the manual and they are executed.

  Many relationships are sometimes found to be very similar, making browsing of those products tedious. Similar product module 818 provides clustering information that allows application. The highest score relationship in the cluster is shown arbitrarily, the others are not shown, and a “more like this” link is shown instead. Clusters are created using the opposite logic of relationship scores in emphasis phrases and their similarities, where the relationship score is a category, manufacturer, ambiguous product name match (one attribute is different Including differences in focus on different types of languages including manual), manually created cluster patterns, prices, etc.

  When a similar product list is created and saved, a product information record is retrieved from the retailer for products in the accessory category marked for that category. Product links and languages that indicate similar products / models are located by searching the product database and using known sophisticated analysis techniques to assign confidence levels to the results. Results from all retailers are collected, an election mechanism is applied, and another list is created. Both of these lists of relationships may be used to derive a score collected. If the score is greater than a predetermined limit, a product-product relationship is created in the table. This includes inferences for other product relationships because the main product is in the product line, for example, one case fits all of the Palm M series.

  In the second-pass scanning, only “potential” relationships are considered; the engine invalidates links that are not found to follow these potential relationships; To summarize. When such an item reaches a specific amount (or when a specific time elapses), the first-pass scanning is repeated.

  In the procedures described above for clusterer module 802, categorizer module 804, characteristic scraper module 810, characteristic aggregator module 814, description generator module 816, and similar product module 818, numerical values of products and proposals (or relationships and sets) are required. It is possible and will be below the confidence limit. Post processing module 820 handles such products. These modules can then be deferred and saved until further data becomes available for an automated system to operate, or they can be incomplete, just a “UPIDized” form on the surface Is held in either. Alternatively, these modules may be moved to a tool for a human editor to repair them as much as possible. For example, a product may not have enough information to classify a TV with sufficient reliability between a CRT TV and a portable TV, an appropriate error condition may be set, Attach the attention of the problem to the attribute in question, so that it is satisfied.

  Previous testing of the overall procedure outlined in this document has resulted in at least 80% automation and in some categories on the order of 96% (ie, compared to “violent” methods of manual data entry). On the one hand, it maintains comparable accuracy and indeed good standardization for manual methods; this ranges from action figures to stroller seat heart monitors Results from testing over 20 different product categories.

  Furthermore, it should be noted that in one embodiment of the present invention, it is described above that the Internet is a networked computer environment and the patroller is a web patroller. Further, in the above-described embodiment, the manufacturer's product feature web page is considered as the information source of the manufacturer's product feature, and the retailer's web page is considered as the information source of the retailer's information. However, the invention is not so limited and may be applied to other types of networked computer environments and other information sources as well. The present invention uses any type of communication hardware and protocol over any type of Internet, local area network (LAN), wide area network (WAN), direct computer connection, or other communication channel. You may implement. Any type of hardware or combination of hardware may be used for various clients and servers. Thus, the term “computer” described above corresponds to any type of computer device or data terminal, such as a personal computer, portable computer, dumb terminal, thin client, portable device, wireless telephone, any of these devices. Combinations can also be mentioned. The various clients and servers may be a single computer at a single location, or multiple computers at a single or multiple locations. For example, the server may be configured by a plurality of excess computers that are installed in common at various places for downsizing (facility scalability). Any suitable server or client software may be used and any communication protocol may be used. Communication may be performed on electrical cables, fiber optic cables, other cables, or by radio methods using radio frequencies, infrared, or other techniques. Any interface may be used to select a product for purchase. Various information may be stored in any format, so the term “database” described above corresponds to the collection of any information such as database files, index tables, and the like.

  Thus, the methods and apparatus described above according to embodiments of the present invention provide a highly effective system and method for gathering desired product information. As well appreciated herein, the present invention facilitates online commerce by allowing the buyer to prepare important product information, thereby facilitating the buyer's informed purchase decision. The present invention also provides a novel method for effectively collecting such product information from a networked computer environment, providing buyers with updated product information, thereby facilitating buyer purchase decisions. Provide a new way to

  The invention has been described through preferred embodiments. However, various modifications may be made without departing from the scope of the present invention as defined in the appended claims and their equivalents.

  <Glossary>

  Similar products: products that are functionally integrated with other products-eg accessories (envelope feeder for printers), parts (replacement screens for PDAs), supplies (paper and printer cartridges), maintenance supplies (tape head cleaners) ). The difference between them is important in relation to the wearability of the item, typically whether it is purchased once or purchased many times.

  Related Products: A broad term that encompasses all similarities, variants, families and assembled products.

  Attribute value set: All possible values recorded (or recordable) in the system for a particular attribute.

  Attributes: Product function, association, quality, quantity, purpose, material, format, structure, manufactured effect. See also characteristics.

  Categorization: The process of assigning products to categories in the most appropriate classification.

  Category: A group of products that share the same basic property definition with each other and occupy each other with the same node in the classification.

  Clustering: The process of collecting proposals from different retailers of the same product (or the same combination of products) into a group.

  Confidence score: A measure of confidence that the data is correct or reliable-for example, product names and references are parsed and correctly identified as submitted by data sources as similar products to other products, How certain is the system saying on the web page that Product A is an accessory for Product B?

  DOM: (Document target modeling): To divide into various areas such as header, footer, product specification table, product text description, recommended accessories, cross-sell / up-sell product list, navigation bar, ad block, etc. Analyzing HTML web pages. This is a preliminary step for setting up feature collection, item clustering, accessory discovery, etc.

  Minimum manufacturer name: The column of results after the manufacturer name column is stripped of various unrelated and common suffixes such as “Incorporated”, “Corporation”.

  Standardization: Determining attribute names and / or attribute values that have the same meaning but are expressed in different superficial nomenclatures and place them in a single consistent representation.

  Standardized attribute value set: A fully standardized attribute value that includes applicable units of measurement. The attribute name itself may or may not be standardized.

  Standardized attribute: A standardized attribute is a standardized name of each attribute. The numeric set for that attribute may or may not be standardized.

  Standardized specification: A set of data in which all elements are composed of standardized attribute names / numerical values / unit information.

  Potential relationship: Establishing a relationship that, by definition, is suitable but follows specific product suitability tests. For example, styluses are usually “potentially” linked to PDAs, but follow suitability.

  Earlier probability: The difference in random guess that is certain from all the possibilities that exist conceptually. For example, if there are two products of the given type in the catalog in clustering and a new unknown proposal is to be analyzed, the clustering has the previous probability of 0.33 (two products in the database One of the three chances, either the same as one of the three or the third new one). One new proposal compared against 99 of the cataloged products is an earlier probability of clustering of 0.01. Earlier probabilities affect confidence estimates at various stages of clustering. The prior probabilities play a role in various other aspects in automatic catalog generation, with the necessary changes.

  Product suite: The main product combined with several accessories, parts and supplies.

  Characteristic: An attribute that is essentially or otherwise derived from the product's perception or use of its essential characteristics. Some product attributes, such as brand, price, and distributor SKU, are not characteristic because they are derived from other external effects apart from product use.

  Semi-UPID: A set of data representing product proposals is UPID in most examples, but a few product proposals are not UPID (see UPID).

  Sibling product: An entirely different product that is a member of the same product line, series, family, but the same, for example, Palm Vx for Palm V.

  Classification: Hierarchical tree or other grouping of product categories

  UPID: “Universal Product ID” An identifier of one particular product in the variously named variously named proposals. For valid locations, the manufacturer identification number, model ID, catalog number, and ISBN number are useful as UPIDs. In many cases, such UPIDs do not exist in external data sources and are created and assigned by the system.

  Variant product: A version of a product that is basically the same product, but with different features from the manufacturer or merchant. For example, a notebook with 128 MB of RAM and the same notebook but with 256 MB of RAM.

2 is a block diagram illustrating a computer configuration including a plurality of manufacturer's servers according to a preferred embodiment of the present invention. FIG. 2 is a block diagram schematically illustrating communication channel connections for an automated purchase procedure and illustrating a portion of the configuration of FIG. 2 is a block diagram illustrating a cookie handling procedure of a preferred embodiment. It is typical explanatory drawing which shows the automated purchase procedure inside the shopping server of preferable embodiment. 2 is a block diagram illustrating a method of processing product characteristic information collected from a plurality of manufacturer's servers according to one embodiment of the present invention. FIG. 6 is a block diagram illustrating a method of identifying a product for sale by an online retailer and creating a new product record based on the determined product characteristics according to an embodiment of the present invention. It is typical explanatory drawing which shows the catalog classification | category of preferable embodiment. It is typical explanatory drawing which shows the characteristic definition tool of another preferable embodiment. It is a flowchart which shows operation of the characteristic definition tool of FIG.

Claims (72)

A method for creating a product catalog stored on a computer readable medium by collecting product information from a plurality of product information sources having completely different formats for product information and storing the information in a classification device,
Processing a plurality of product information records from the product information source into one or more groups based on the fact that product information records are appropriate for the same product;
Correlating a unique product ID corresponding to the product associated with each group to identify the product;
Comparing each identified product with a category of the classifier to determine a category for the identified product in the classifier;
Determining an attribute for each categorized product based on the product information record corresponding to each group;
Creating product characteristics based on the determined attributes;
Storing the product characteristic in the correspondingly determined category of the classifier;
Having a method.   The step of processing determines that it is appropriate that the corresponding products in the product information record are the same by comparing data sequences associated with the products, and determines a common data sequence The method of claim 1 including the step of:   The method of claim 2, wherein the data string includes at least one of a manufacturer identification number, a model identifier, and a single product code.   The method of claim 1, wherein the step of processing comprises comparing product names in the product information record and grouping together all products having substantially similar product names.   The method of claim 4, wherein the product names are compared regardless of capitalization and punctuation differences.   The method of claim 4, further comprising a second processing step of analyzing and ignoring common nouns associated with the product in the step of determining the group.   The method of claim 1, wherein the step of processing comprises analyzing the selected adjectives and ignoring the selected adjectives in the step of determining the group.   The method of claim 1, wherein the step of processing comprises considering a price in the product information record associated with the product.   The method of claim 1, wherein the step of processing comprises considering a relationship between synonyms, broader terms, and narrower terms in the description of the product in the product information record.   The method of claim 1, wherein the step of processing comprises considering a merchant scope indicated in the product information record.   The method of claim 1, wherein the step of processing includes grouping the products into subgroups and / or supergroups. The determining step includes:
Collecting attribute values from multiple product information records in the group and assigning a credit rating to each collected attribute value;
Combining the attribute value with a set of product characteristic attributes based on the credit rating;
The method of claim 1, comprising:   The method of claim 1, further comprising determining a product name for each identified product. Determining the product name comprises:
Selecting the best name of multiple variant products from the product information records in the group;
Removing the excess connected text best name and formatting the removed best name into a predetermined style product name;
The method of claim 13, comprising:   The method of claim 1, further comprising determining a product image for each identified product by selecting a most preferred product image from the product information record.   The method of claim 1, further comprising generating a description of each identified product from at least a portion of the determined attributes.   The processing step includes examining the specific code associated with each product information record by analyzing the specific code present in the product information record and determining the commonness between the codes. The method of claim 1, comprising the steps of:   18. The method of claim 17, wherein each product information record is examined one or more times to determine a common specific code associated with each product.   The method of claim 1, further comprising the step of repeating the processing step after the comparing step and executing the comparing step again.   Further comprising determining when one or more results of the processing, correlating, comparing and determining steps are below a predetermined confidence level and alerting the results for further processing. The method of claim 1, wherein:   21. The method of claim 20, wherein the alerted result is deferred, saved, and reprocessed when additional product information sources become available.   21. The method of claim 20, wherein the alerted result is moved to a processing tool for manual operation. A method for creating a product catalog stored on a computer readable medium by collecting product information from a plurality of product information sources having completely different formats for product information,
Processing a plurality of product information records from the product information source into one or more groups based on product information records being appropriate for the same product;
Correlating unique product IDs corresponding to the products identified for each group;
Comparing each identified product with a category in the classifier to determine a category for the identified product in the classifier;
Repeating the processing step and the correlating step after execution of the comparing step to modify a group to which the plurality of product information records belong;
Determining an attribute for each categorized product based on the product information record corresponding to each group;
Creating product characteristics based on the determined attributes;
Storing the product characteristic in the correspondingly determined category of the classifier;
Having a method.   The step of processing determines that it is appropriate that the corresponding products in the product information record are the same by comparing data sequences associated with the products, and determines a common data sequence 24. The method of claim 23, comprising the step of:   The method of claim 24, wherein the data string includes at least one of a manufacturer identification number, a model identifier, and a single product code.   The method of claim 24, wherein the step of processing comprises comparing product names in the product information record and grouping together all products having substantially similar product names.   Assigning a collective credit score to the grouping of information generated in the processing step, categorizing a credit score into the category generated in the comparing step, and repeating the repeating step until the credit score is stabilized 24. The method of claim 23, further comprising:   Further warning the result of one or both of the processing and categorizing steps when the credit score associated with one or both of the processing and categorizing steps is less than or equal to a predetermined minimum value. 28. The method of claim 27, comprising:   29. The method of claim 28, wherein the alerted result is deferred, saved, and reprocessed when a further product information source becomes available.   29. The method of claim 28, wherein the alerted result is moved to a processing tool for manual operation.   The comparing step includes a step of inspecting the attribute and attribute value set in each category and a step of inspecting an actual product information record already classified in each category. the method of.   The method of claim 23, wherein the correlating assigns different product IDs to the same product of different colors.   The method of claim 1, wherein the step of processing comprises generating a patrol person from a server for the product information source. A method for collecting product information from a plurality of product information sources in a networked computer environment comprising:
Generating a patrol person from a server interconnected to the network computing environment to visit the plurality of information sources;
Collecting product phrase information and features of the product phrase information from each of the plurality of information sources through the patrolperson;
Creating a product catalog based on the product phrase information and features of the product phrase information;
Having a method.   The at least one feature of the phrase includes at least one of the frequency, position, font size, font style, font case, font effect, font color, arrangement, and co-occurrence of the phrase in each of the plurality of information sources. 35. The method of claim 34.   36. The method of claim 35, wherein the plurality of information sources includes at least one of a manufacturer product property information source, a product advertisement information source, and a retailer information source.   31. The method of claim 30, wherein the patroller includes a product advertisement patroller gathering product phrase information from at least one of the manufacturer's product property information source and the product advertisement information source.   35. The method of claim 34, further comprising comparing the product phrase information to a category of a classifier to determine a product category of a product described by the product phrase information.   The method of claim 1, further comprising determining a similar product for at least one of the products. Determining the similar product comprises:
Analyzing at least one product information record corresponding to the product;
Tracking the link to the associated product information record if a link exists in the product information record for the associated product;
Reversely searching for a reference to the product in the associated product information record;
Associating the associated product in the associated product information record with the product in the catalog;
40. The method of claim 39, comprising:   The method of claim 23, further comprising determining a similar product for at least one of the products. Determining the similar product comprises:
Analyzing at least one product information record corresponding to the product;
Tracking the link to the associated product information record if a link exists in the product information record for the associated product;
Reversely searching for a reference to the product in the associated product information record;
Associating the associated product in the associated product information record with the product in the catalog;
24. The method of claim 23, comprising:   35. The method of claim 34, further comprising determining a similar product for at least one of the products. Determining the similar product comprises:
Analyzing at least one product information record corresponding to the product;
Tracking the link to the associated product information record if a link exists in the product information record for the associated product;
Reversely searching for a reference to the product in the associated product information record;
Associating the associated product in the associated product information record with the product in the catalog;
35. The method of claim 34, comprising: In a system for creating a product catalog by collecting product information from a plurality of product information sources having completely different formats for product information and storing the information in a classification device,
Means for processing a plurality of product information records from the product information source into one or more groups on the basis that a product information record is appropriate for the same product;
Means for correlating a unique product ID corresponding to the product associated with each group to identify the product;
Means for comparing each identified product with a category of the classifier to determine a category for the identified product in the classifier;
Means for determining an attribute for each categorized product based on the product information record corresponding to each group;
Means for creating product characteristics based on the determined attributes;
Means for storing the product characteristics in the correspondingly determined categories of the classifier;
Having a system.   The means for processing includes means for determining that it is appropriate that the corresponding products in the product information record are the same by comparing data strings associated with the products, and a common The system of claim 45, comprising means for determining a data string.   The system of claim 45, wherein the data string includes at least one of a manufacturer identification number, a model identifier, and a single product code.   The means for processing comprises means for comparing product names in the product information record and means for grouping together all products having substantially similar product names. 46. The system of claim 45.   49. The system of claim 48, wherein the product names are compared regardless of capitalization and punctuation differences.   49. The system of claim 48, further comprising means for analyzing the common noun associated with the product and excluding it from processing the common noun with the means for determining.   The system of claim 45, further comprising means for analyzing the selected adjective from the determining means and excluding it from processing the selected adjective.   The system of claim 45, wherein the means for processing comprises means for considering a price in the product information record associated with the product.   46. The system of claim 45, wherein the means for processing comprises means for considering synonyms, broader terms, and narrower terms associated with the product description in the product information record.   46. The system of claim 45, wherein the means for processing comprises means for considering a merchant scope indicated in the product information record.   46. The system of claim 45, wherein the means for processing comprises means for grouping the products into subgroups and / or supergroups. The means for determining is
Means for collecting attribute values from multiple product information records in the group and assigning a credit rating to each attribute value collected;
Means for combining the attribute value into a set of product characteristic attributes based on the credit rating;
46. The system of claim 45, comprising:   45. The system of claim 44, further comprising means for determining a product name for each identified product. Means for determining the product name are:
Means for selecting the best name of multiple variant names from product information records in the group;
Means for excluding the best name of excess connected text and formatting the excluded name into a predetermined style product name;
58. The system of claim 57, comprising:   The system of claim 45, further comprising means for determining a product image for each product identified by selecting a most preferred product image from the product information record.   46. The system of claim 45, further comprising means for generating a description of each product identified from at least a portion of the determined attributes.   The means for processing examines the specific code associated with each product information record by analyzing the specific code present in the product information record and is analyzed to determine the commonality between the codes 46. The system of claim 45, comprising means for comparing specific codes.   62. The system of claim 61, wherein the means for inspecting inspects each product information record one or more times to determine a common specific code associated with each product.   Means for determining when one or more results of the processing, correlating, comparing and determining steps are below a predetermined confidence level and alerting the results for further processing; 46. The system of claim 45, comprising:   64. The system of claim 63, wherein the alerted result is deferred, saved, and reprocessed when a further product information source becomes available.   64. The system of claim 63, wherein the alerted result is moved to a processing tool for manual operation. In a system for collecting product information from multiple product information sources in a networked computer environment,
Means for generating a patrol person from a server interconnected to the network computing environment to visit the plurality of information sources;
Means for gathering product phrase information and features of the product phrase information from each of the plurality of information sources through the patroller;
Means for creating a catalog of products based on the product phrase information and features of the product phrase information;
Having a system.   The at least one characteristic of the phrase includes at least one of the frequency, position, font size, font style, font case, font effect, font color, arrangement, and co-occurrence of the phrase in each of the plurality of information sources. 68. The system of claim 66.   68. The system of claim 67, wherein the plurality of information sources includes at least one of a manufacturer product property information source, a product advertisement information source, and a retailer information source.   67. The system of claim 66, wherein the patroller includes a product advertisement patroller gathering product phrase information from at least one of the manufacturer's product property information source and the product advertisement information source.   The system of claim 66, further comprising means for comparing the product phrase information with a category of a classifier to determine a product category of a product described by the product phrase information.   The system of claim 45, further comprising means for determining a similar product for at least one of the products. The means for determining the similar product is:
Means for analyzing at least one product information record corresponding to a product and tracking the link to the associated product information record if a link exists in the product information record for the associated product;
Means for reversely retrieving a reference to the product in the associated product information record;
Means for associating the associated product in the associated product information record with the product in the catalog;
72. The system of claim 71, comprising:

Images