CN114860721A - Index data construction method and device - Google Patents
Index data construction method and device Download PDFInfo
- Publication number
- CN114860721A CN114860721A CN202210445890.1A CN202210445890A CN114860721A CN 114860721 A CN114860721 A CN 114860721A CN 202210445890 A CN202210445890 A CN 202210445890A CN 114860721 A CN114860721 A CN 114860721A
- Authority
- CN
- China
- Prior art keywords
- data
- service
- change
- changed
- message
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2272—Management thereof
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2358—Change logging, detection, and notification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for constructing index data, and relates to the technical field of computers. One embodiment of the method comprises: receiving the change data of the service data in the source system through a uniform data acquisition interface, wherein the change data comprises a service identifier and a service change field of the service data; writing the changed data into a first database according to a preset structure, and generating a change message after the changed data is successfully written, wherein the change message comprises a service identifier of the service data; and responding to the generated change message, inquiring fields for generating indexes from the first database according to the service identification in the change message, and assembling the inquired fields to obtain index data. The implementation mode can realize the unified control of data writing, solves the problem that new data is covered by old data, improves the accuracy and efficiency of data writing, and reduces the performance requirement on the system.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for constructing index data.
Background
The function of data retrieval in current systems is to store data via the ES (ELASTICSEARCH, a LUCENE-based search server) and provide the system with data searching capabilities. Each data change point needs to assemble data separately and call the API (application program interface) of the ES, and directly call the API provided by the ES at the position where the data change is generated to complete the writing of the data. Different fields in a database table in the ES come from different systems, and the change fields from different systems need to be uniformly aggregated. The data of each system has respective data version, and the data version of the fields from different systems when the ES is written cannot be checked.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:
the method can not realize the uniform control of data writing, is easy to cause the problem that new data is covered by old data, has low accuracy and efficiency of data writing and has higher requirement on the performance of the system.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for constructing index data, which can implement unified control of data writing, solve the problem that new data is covered by old data, improve accuracy and efficiency of data writing, and reduce performance requirements on a system.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a method for constructing index data.
A method for constructing index data comprises the following steps: receiving changed data of service data in a source system through a uniform data acquisition interface, wherein the changed data comprises a service identifier and a service change field of the service data; writing the changed data into a first database according to a preset structure, and generating a change message after the changed data is successfully written, wherein the change message comprises a service identifier of the service data; and responding to the generation of the change message, inquiring a field for generating an index from the first database according to the service identification in the change message, and assembling the inquired field to obtain index data.
Optionally, the change data further includes a data version number of the service data; the receiving of the change data of the service data in the source system through the unified data acquisition interface includes: and receiving the changed data of the service data in the source system by taking the service identifier, the data version number and the service change field as interface parameters of the unified data acquisition interface.
Optionally, after receiving the changed data of the service data in the source system through the unified data acquisition interface, the method includes: and performing field verification on the received changed data of the service data in the source system to confirm that the received changed data of the service data in the source system comprises each field in the interface parameters.
Optionally, after the change data of the service data in the source system, which is received by the confirmation, includes each field in the interface parameter, the method includes: assembling the received change data of the service data in the source system into a first message, and issuing the first message to a first message middleware; the writing the changed data into the first database according to the preset structure comprises: and in response to the first message issued to the first message middleware, writing the changed data to a first database according to a preset structure.
Optionally, the writing the changed data into the first database according to a preset structure includes: carrying out data validity verification on the changed data and confirming that the data validity verification is passed, wherein the data validity verification comprises verifying a service identifier, a data version number and a service change field in the changed data; and converting the changed data according to the data storage structure of the first database and writing the converted data into the first database.
Optionally, the converting the changed data according to the data storage structure of the first database and writing the changed data into the first database includes: converting the changed data according to a data storage structure of the first database to obtain the converted changed data; acquiring a distributed lock, and after confirming that the distributed lock is successfully acquired, carrying out version judgment on the changed data to confirm that the changed data is the latest version; writing the converted changed data of the latest version into the first database according to a corresponding source system identifier, wherein the source system identifier is the identifier of the source system to which the service data belongs, which is determined according to the service change field in the changed data.
Optionally, the performing version determination on the changed data to confirm that the changed data is the latest version includes: acquiring the latest stored version number of the changed data from a second database; and comparing the data version number in the changed data with the stored version number, and confirming that the changed data is the latest version when the data version number in the changed data is greater than or equal to the stored version number.
Optionally, after writing the latest version of the converted change data into the first database according to the corresponding source system identifier, the method includes: releasing the distributed lock, and updating the stored version number in the second database to the data version number in the changed data.
Optionally, after the generating the change message, the method includes: publishing the change message to second message middleware; the querying, in response to generating the change message and according to the service identifier in the change message, a field for generating an index from the first database includes: responding to the change message issued to the second message middleware, and inquiring fields for generating indexes from the first database according to service identifications in the change message, wherein the fields for generating indexes are one or more fields of the service data.
Optionally, the assembling of the queried fields to obtain index data includes: and assembling the inquired fields according to an index structure of a storage cluster to obtain the index data, and storing the index data into the storage cluster by calling an application program interface of the storage cluster, wherein the index data is used for inquiring a corresponding service identifier by the storage cluster so as to obtain the service data from a source system of the service data according to the corresponding service identifier.
According to another aspect of the embodiments of the present invention, an apparatus for constructing index data is provided.
An index data construction apparatus comprising: the system comprises a change data acquisition module, a data processing module and a data processing module, wherein the change data acquisition module is used for receiving change data of service data in a source system through a uniform data acquisition interface, and the change data comprises a service identifier and a service change field of the service data; the change data writing module is used for writing the change data into a first database according to a preset structure and generating a change message after the change data is successfully written, wherein the change message comprises the service identifier of the service data; and the index data construction module is used for responding to the generation of the change message, inquiring a field for generating an index from the first database according to the service identifier in the change message, and assembling the field by the inquired field to obtain index data.
Optionally, the change data further includes a data version number of the service data; the change data acquisition module is further configured to: and receiving the changed data of the service data in the source system by taking the service identifier, the data version number and the service change field as interface parameters of the unified data acquisition interface.
Optionally, a field verification module is further included, configured to: and performing field verification on the received changed data of the service data in the source system to confirm that the received changed data of the service data in the source system comprises each field in the interface parameters.
Optionally, the field verification module is further configured to: assembling the received change data of the service data in the source system into a first message, and issuing the first message to a first message middleware; the change data writing module is further configured to: and in response to the first message issued to the first message middleware, writing the changed data to a first database according to a preset structure.
Optionally, the change data writing module is further configured to: carrying out data validity verification on the changed data and confirming that the data validity verification is passed, wherein the data validity verification comprises verifying a service identifier, a data version number and a service change field in the changed data; and converting the changed data according to the data storage structure of the first database and writing the converted data into the first database.
Optionally, the change data writing module is further configured to: converting the changed data according to a data storage structure of the first database to obtain the converted changed data; acquiring a distributed lock, and after confirming that the distributed lock is successfully acquired, carrying out version judgment on the changed data to confirm that the changed data is the latest version; and writing the changed data after the conversion of the latest version into the first database according to a corresponding source system identifier, wherein the source system identifier is the identifier of the source system to which the business data belongs, which is determined according to the business change field in the changed data.
Optionally, the changed data writing module is further configured to: acquiring the latest stored version number of the changed data from a second database; and comparing the data version number in the changed data with the stored version number, and confirming that the changed data is the latest version when the data version number in the changed data is greater than or equal to the stored version number.
Optionally, the change data writing module is further configured to: releasing the distributed lock, and updating the stored version number in the second database to the data version number in the changed data.
Optionally, the change data writing module is further configured to: publishing the change message to second message middleware; the index data construction module is further configured to: responding to the change message issued to the second message middleware, and inquiring fields for generating indexes from the first database according to service identifications in the change message, wherein the fields for generating indexes are one or more fields of the service data.
Optionally, the index data construction module is further configured to: and assembling the inquired fields according to an index structure of a storage cluster to obtain the index data, and storing the index data into the storage cluster by calling an application program interface of the storage cluster, wherein the index data is used for inquiring a corresponding service identifier by the storage cluster so as to obtain the service data from a source system of the service data according to the corresponding service identifier.
According to yet another aspect of an embodiment of the present invention, an electronic device is provided.
An electronic device, comprising: one or more processors; a memory for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the method for constructing index data provided by the embodiments of the present invention.
According to yet another aspect of an embodiment of the present invention, a computer-readable medium is provided.
A computer-readable medium, on which a computer program is stored, which, when executed by a processor, implements the method for constructing index data provided by an embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: receiving the change data of the service data in the source system through a uniform data acquisition interface, wherein the change data comprises a service identifier and a service change field of the service data; writing the changed data into a first database according to a preset structure, and generating a change message after the changed data is successfully written, wherein the change message comprises a service identifier of the service data; and responding to the generated change message, inquiring fields for generating indexes from the first database according to the service identification in the change message, and assembling the inquired fields to obtain index data. The method can realize the unified control of data writing, solve the problem that new data is covered by old data, improve the accuracy and efficiency of data writing, and reduce the performance requirements on the system.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a diagram illustrating the main steps of a method for constructing index data according to an embodiment of the present invention;
FIG. 2 is a flow diagram of construction of index data according to one embodiment of the invention;
FIG. 3 is one of the flow diagrams for collecting change data according to one embodiment of the invention;
FIG. 4 is a second schematic flow chart illustrating the collection of change data according to an embodiment of the present invention;
FIG. 5 is a flow diagram of writing altered data according to one embodiment of the invention;
FIG. 6 is a schematic flow diagram of the use of a distributed lock according to one embodiment of the invention;
FIG. 7 is a schematic flow diagram for building an index according to one embodiment of the invention;
FIG. 8 is a schematic diagram of the main blocks of an index data construction apparatus according to an embodiment of the present invention;
FIG. 9 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 10 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of main steps of a method for constructing index data according to an embodiment of the present invention.
As shown in fig. 1, the method for constructing index data according to an embodiment of the present invention mainly includes the following steps S101 to S103.
Step S101: and receiving the changed data of the service data in the source system through a uniform data acquisition interface, wherein the changed data comprises the service identification and the service change field of the service data.
The change data may also include a data version number of the service data.
Receiving, by a unified data acquisition interface, change data of service data in a source system, where the change data may include: and receiving the changed data of the service data in the source system by taking the service identifier, the data version number and the service change field as the interface parameters of the unified data acquisition interface.
After receiving the changed data of the service data in the source system through the unified data acquisition interface, the method may include: and carrying out field verification on the received changed data of the service data in the source system so as to confirm that the received changed data of the service data in the source system comprises each field in the interface parameters. Each field in the interface parameter comprises a service identifier, a data version number and a service change field.
After confirming that the received change data of the service data in the source system includes each field in the interface parameter, the method may include: and assembling the received change data of the service data in the source system into a first message, and issuing the first message to first message middleware.
Step S102: and writing the changed data into the first database according to a preset structure, and generating a change message after the changed data is successfully written, wherein the change message comprises the service identification of the service data.
Writing the changed data into the first database according to the preset structure may include: and in response to the first message issued to the first message middleware, writing the changed data to the first database according to a preset structure. The first database may be, for example, HBASE (open source non-relational distributed database) or MONGODB (distributed file storage based database).
The preset structure may be a data storage structure of the first database. Writing the changed data into the first database according to the preset structure may include: carrying out data validity verification on the changed data and confirming that the data validity verification passes, wherein the data validity verification comprises the verification of a service identifier, a data version number and a service change field in the changed data; and converting the changed data according to the data storage structure of the first database and writing the converted data into the first database. The data validity verification may include verifying whether the data version number, the change field, and the service identifier are null, and if none of the data version number, the change field, and the service identifier are null, the data is verified to be passed, and if any of the data version number, the change field, and the service identifier is null, the data is abnormal data, and the data is not verified to be passed.
Converting the changed data according to the data storage structure of the first database and writing the converted data into the first database, which may include: converting the changed data according to a data storage structure of a first database to obtain converted changed data; acquiring a distributed lock, and after confirming that the distributed lock is successfully acquired, carrying out version judgment on the changed data to confirm that the changed data is the latest version; and writing the converted changed data of the latest version into the first database according to the corresponding source system identification, wherein the source system identification is the identification of the source system to which the service data belongs, which is determined according to the service change field in the changed data. The distributed lock may be implemented by a SETNX command of redis (centralized caching middleware), a database, or a ZK (ZooKeeper, distributed application coordination service).
The determining the version of the changed data to confirm that the changed data is the latest version may include: acquiring the latest stored version number of the changed data from a second database; the data version number in the changed data is compared with the stored version number, and when the data version number in the changed data is greater than or equal to the stored version number, the changed data is confirmed to be the latest version.
After writing the latest version of the converted change data into the first database according to the corresponding source system identifier, the method may include: the distributed lock is released and the stored version number in the second database is updated to the data version number in the changed data.
After generating the change message, the method may include: the change message is published to second message middleware. The second message middleware may be the same message middleware as the first message middleware.
Step S103: and responding to the generated change message, inquiring fields for generating indexes from the first database according to the service identification in the change message, and assembling the inquired fields to obtain index data.
In response to generating the change message, querying the first database for a field for generating an index according to the service identifier in the change message may include: and responding to the change message issued to the second message middleware, and inquiring a field for generating an index from the first database according to the service identification in the change message, wherein the field for generating the index is one or more fields of the service data, the fields can include service change fields contained in the change data, and can also include fields of the service data which are already stored in the first database.
Assembling the index data from the queried fields may include: and assembling the inquired fields according to the index structure of the storage cluster to obtain index data, storing the index data into the storage cluster by calling an application program interface of the storage cluster, wherein the index data is used for storing a service identifier corresponding to the cluster inquiry so as to obtain the service data from a source system of the service data according to the corresponding service identifier. And storing a cluster such as an ES, assembling the inquired fields to obtain index data, performing data assembly according to a mapping structure of the ES, and writing the assembled data into the ES to complete the construction of the index data.
FIG. 2 is a flow diagram of construction of index data according to one embodiment of the invention.
As shown in FIG. 2, at the system collection side, grooming each system involves searching for change points that build required fields, and calling a unified data collection API (i.e., a unified data collection interface) at the change points to collect change data for each change point. When data change occurs in each system (e.g., system a, system B, and system C in fig. 2), the unified data collection API is called, and data is pushed to the data receiving server in a remote interface calling manner (step 201). After receiving the changed data, the data receiving server assembles the changed data according to a preset structure, and sends a message including the changed data (the changed data is, for example a, example B, and example C in fig. 2, and respectively corresponds to the system a, the system B, and the system C) to the data aggregation server through the message middleware (step 202). After receiving the changed data through the message middleware (step 203), the data aggregation server writes the changed data into HBASE (namely a first database) in a distributed lock mode (step 204) so as to prevent the latest data from being covered by old data due to concurrency or message disorder, and after successfully writing the changed data into HBASE, the data aggregation server sends a data change message containing service identification of the service data to the index construction server through the message middleware (step 205). After receiving the data change message (step 206), the index construction server queries the latest data from the HBASE through the service identifier of the service data (step 207), assembles the data according to the mapping structure of the ES, and writes the assembled data into the ES (step 208), so as to complete construction of the index data. Wherein the message middleware is a middleware technology for processing push and receive of messages, such as RABBITMQ; ES is a LUCENE-based search server, providing a distributed multi-user capability full-text search engine; distributed locks are a way to control the synchronous access to shared resources between distributed systems; HBASE is an open-source, non-relational, distributed database.
In one embodiment, a client system (e.g., client system a, client system B, client system C in fig. 2) sends a request including a search condition to the ES, retrieves a service identifier of service data satisfying the search condition from the ES (step 209), and obtains specific data corresponding to the service identifier from the system according to the retrieved service identifier (step 210) to complete a search service.
FIG. 3 is one of the flow diagrams for collecting change data according to one embodiment of the invention.
As shown in fig. 3, in an embodiment, the service identifier, the data version number, and the service change field are used as interface parameters of the unified data acquisition interface to receive the change data of the service data in the source system. Specifically, the data collection process mainly includes adding a call to the unified data collection API to the functional points related to data change, which are distributed in each module or each source system (e.g., system a, system B, and system C in fig. 2), and collecting change data of each system. The mandatory field of the unified data acquisition API comprises a service identifier, a data version number and a service change field, wherein the service identifier is used for uniquely determining a piece of change data, if the change data is an order, the service identifier can be an order number, if the change data is a commodity, the service identifier can be a commodity number, the data version number is a current version of the service data after being changed for many times, the data version number is used for uniquely marking each change of each piece of service data and is related to a source system of the service data, and the service field is a JSON data structure and is used for transmitting field values related to changes. After the data acquisition end completes parameter assembly, a specific service provided by a data receiving server (namely a server) is called in an RPC (Remote Procedure Call) mode to complete changed data pushing. The examples of the mandatory fields of the service data are shown in table 1.
Table 1 examples of bye fields for traffic data
In one embodiment, field verification is performed on the received changed data of the service data in the source system to confirm that the received changed data of the service data in the source system includes fields in the interface parameters. Specifically, after the data acquisition end completes parameter assembly, each field to be transmitted can be checked, whether the service identifier is empty or not is judged, if the service identifier is empty, processing is performed according to abnormal logic, if the service identifier is not empty, whether the data version number is empty or not is judged, if the data version number is empty, processing is performed according to abnormal logic, if the data version number is not empty, the service change field is assembled, and a unified data acquisition API is called to perform data pushing.
FIG. 4 is a second flowchart illustrating a process of collecting change data according to an embodiment of the invention.
As shown in fig. 4, in one embodiment, the unified data collection API is called by the data receiving server, the received change data of the service data in the source system is assembled into a first message, and the first message is published to the first message middleware. Specifically, after the unified data acquisition API of the data receiving server receives data acquired by the data acquisition end, the pushed data must be transmitted, the pushed data must be checked, if the pushed data must be transmitted, the changed data is assembled into a message (namely, a first message) and is sent to the data aggregation server through a message middleware (namely, the first middleware). The checking may include checking whether there is a field missing, for example, checking whether a data version number, a changed field, and a service identifier are null, if none of them are null, the checking is passed, and if any one of them is null, the checking is abnormal data, and the checking is not passed.
FIG. 5 is a flow diagram of writing altered data according to one embodiment of the invention.
As shown in FIG. 5, in one embodiment, in response to a first message posted to the first message middleware, change data is written to a first database in a preset structure. And carrying out data validity verification on the changed data and confirming that the data validity verification passes, wherein the data validity verification comprises the verification of a service identifier, a data version number and a service change field in the changed data. Specifically, the data aggregation server receives a first message sent by the data receiving server, and performs data validity verification on the changed data to determine that the data version number, the changed field and the service identifier are all not null.
In one embodiment, the changed data is converted according to the data storage structure of the first database, and the converted changed data is obtained. Specifically, the alteration data is converted in the data format of the storage structure in HBASE (i.e., the first database). The data field structure stored in HBASE is similar to the data field structure in ES search service, so that the index construction server can perform the least data conversion amount in the subsequent index data construction. The structure of the HBASE table is partitioned and stored by column families according to the source system to which the field belongs, taking a commodity search service as an example, the function satisfied by the search index is commodity identification number list search, and the search condition includes fields of a commodity number, a commodity name, whether to pre-sell a commodity, whether to kill a commodity in seconds, and the like, and the design of the HBASE column family storage structure can be as shown in table 2, wherein each column family corresponds to one source system.
TABLE 2 HBASE column family storage architecture example
FIG. 6 is a schematic flow diagram of using a distributed lock, according to one embodiment of the invention.
As shown in fig. 5 and fig. 6, in an embodiment, the distributed lock is acquired, and after the distributed lock is successfully acquired, the version of the changed data is determined to confirm that the changed data is the latest version; writing the converted changed data of the latest version into a first database according to the corresponding source system identifier; and releasing the distributed lock, updating the stored version number in the second database into the data version number in the changed data, and issuing the changed message to the second message middleware. Specifically, after receiving the change data (step 601), obtaining the distributed lock through an SETNX command of a redis (i.e., a second database) (step 602), where a KEY naming rule of the SETNX command may be update _ service identifier _ data source identifier, and if the SETNX command is successfully executed, the lock is successfully preempted. After the lock is successfully robbed, acquiring the latest stored version number of the changed data from the redis according to the service identifier and the source system identifier of the changed data (step 603), comparing the data version number in the changed data with the stored version number (step 604), confirming that the changed data is the latest version (step 605) when the data version number in the changed data is greater than or equal to the stored version number, continuing the process, confirming that the changed data is not the latest version and discarding the changed data when the data version number in the changed data is less than the stored version number. After the changed data is confirmed to be the latest version, the changed data is updated to HBASE according to the column family corresponding to the service identification and the source system identification of the changed data (step 606), the version number of the current data is updated to redis through a SET command (step 607) after the writing is successful (step 608), the distributed lock is released through a DEL command (step 609) (step 610), and a change message including the service identification is issued to the second message middleware. And the identifier of the column source system to which the service data belongs is determined according to the service change field in the change data.
In one embodiment, the validity time of the distributed lock may be set when setting the SETNX command, if the distributed lock expires or if an exception occurs during execution, the distributed lock may be released.
FIG. 7 is a flow diagram of building an index, according to one embodiment of the invention.
As shown in fig. 7, in an embodiment, in response to a change message issued to the second message middleware, a field used for generating an index is queried from the first database according to a service identifier in the change message, the queried field is assembled according to an index structure of the storage cluster to obtain index data, and the index data is stored in the storage cluster by calling an application program interface of the storage cluster. Specifically, after receiving the change message, the index construction server parses the service identifier from the change message, queries field information for constructing index data from HBASE by using the service identifier as a query condition, constructs the index data according to the mapping structure of the ES after obtaining the field information, and calls the API of the ES to write the data into the ES after completing the construction of the index data. Based on the ES index, a search service is built that provides a search API to an external system (e.g., a client system) that can complete the retrieval of data by calling the search API. Mapping is similar to a table structure definition schema in a database, and mapping may define names of fields in an index, define data types of the fields, such as character strings, numbers, boolean, and the like, and set a relevant configuration of an inverted index, such as setting a field to be not indexed.
In one embodiment, fields used for generating indexes are inquired from a first database in a distributed lock mode, the inquired fields are assembled to obtain index data, and the index data are stored in a storage cluster. In particular, the distributed lock is acquired by a SETNX command before the fields used to generate the index are queried from the first database. After the index data is saved to the storage cluster, the distributed lock is released.
In one embodiment, the index data is used for storing a service identifier corresponding to the cluster query, so as to obtain the service data from a source system of the service data according to the corresponding service identifier. Specifically, according to the search condition from the client system, the service identifier of the data meeting the condition is obtained from the ES, the detailed data is obtained from the data source system through the service identifier, and finally the data is assembled and returned to the client system. The ES only stores fields related to search conditions, the service identification is stored as a search result, and other query conditions are stored as index fields, so that the storage capacity of the ES is ensured to be small enough based on the two points, and the performance of ES data retrieval is improved; meanwhile, the search service constructed based on the ES only queries the service identifier when the ES is queried, and the data source system obtains the detailed data through the service identifier to assemble the detailed data, so that the load pressure is dispersed, and the load pressure of the ES is reduced.
In one embodiment, related applications involved in the construction process of the search service are deployed in a cluster manner, and all construction efficiency of the search can be improved by increasing and decreasing the number of running instances of each cluster; meanwhile, the scheme introduces a message middleware to release the coupling among the process nodes, and each process can independently adjust the message consumption rate to achieve the highest efficiency of the index construction process.
Fig. 8 is a schematic diagram of main blocks of an index data construction apparatus according to an embodiment of the present invention.
As shown in fig. 8, the apparatus 800 for constructing index data according to an embodiment of the present invention mainly includes: a changed data acquisition module 801, a changed data writing module 802 and an index data construction module 803.
And the change data acquisition module 801 is configured to receive change data of the service data in the source system through a unified data acquisition interface, where the change data includes a service identifier and a service change field of the service data.
And a changed data writing module 802, configured to write the changed data into the first database according to a preset structure, and generate a change message after the changed data is successfully written, where the change message includes a service identifier of the service data.
And the index data constructing module 803 is configured to, in response to the change message being generated, query, according to the service identifier in the change message, a field for generating an index from the first database, and assemble the queried field to obtain index data.
In one embodiment, the change data further includes a data version number of the business data; the change data acquisition module 801 is specifically configured to: and receiving the changed data of the service data in the source system by taking the service identifier, the data version number and the service change field as the interface parameters of the unified data acquisition interface.
In one embodiment, a field verification module may be further included to: and carrying out field verification on the received changed data of the service data in the source system so as to confirm that the received changed data of the service data in the source system comprises each field in the interface parameters.
In one embodiment, the field verification module is specifically configured to: assembling the received change data of the service data in the source system into a first message, and issuing the first message to a first message middleware; the changed data write module 802 is further configured to: and in response to the first message issued to the first message middleware, writing the changed data to the first database according to a preset structure.
In one embodiment, the changed data writing module 802 is specifically configured to: carrying out data validity verification on the changed data and confirming that the data validity verification passes, wherein the data validity verification comprises the verification of a service identifier, a data version number and a service change field in the changed data; and converting the changed data according to the data storage structure of the first database and writing the converted data into the first database.
In one embodiment, the changed data writing module 802 is specifically configured to: converting the changed data according to a data storage structure of a first database to obtain converted changed data; acquiring a distributed lock, and after confirming that the distributed lock is successfully acquired, carrying out version judgment on the changed data to confirm that the changed data is the latest version; and writing the converted changed data of the latest version into the first database according to the corresponding source system identification, wherein the source system identification is the identification of the source system to which the service data belongs, which is determined according to the service change field in the changed data.
In one embodiment, the changed data writing module 802 is specifically configured to: acquiring the latest stored version number of the changed data from a second database; the data version number in the changed data is compared with the stored version number, and when the data version number in the changed data is greater than or equal to the stored version number, the changed data is confirmed to be the latest version.
In one embodiment, the changed data writing module 802 is specifically configured to: the distributed lock is released and the stored version number in the second database is updated to the data version number in the changed data.
In one embodiment, the changed data writing module 802 is specifically configured to: issuing the change message to second message middleware; the index data building module 803 is specifically configured to: and responding to the change message issued to the second message middleware, and inquiring a field for generating the index from the first database according to the service identification in the change message, wherein the field for generating the index is one or more fields of the service data.
In one embodiment, the index data building module 803 is specifically configured to: and assembling the inquired fields according to the index structure of the storage cluster to obtain index data, storing the index data into the storage cluster by calling an application program interface of the storage cluster, wherein the index data is used for storing a service identifier corresponding to the cluster inquiry, and acquiring the service data from a source system of the service data according to the corresponding service identifier.
In addition, the detailed implementation of the index data constructing apparatus in the embodiment of the present invention has been described in detail in the above index data constructing method, and therefore, the repeated description is not repeated here.
Fig. 9 shows an exemplary system architecture 900 of a method or apparatus for constructing index data to which an embodiment of the present invention can be applied.
As shown in fig. 9, the system architecture 900 may include end devices 901, 902, 903, a network 904, and a server 905. Network 904 is the medium used to provide communication links between terminal devices 901, 902, 903 and server 905. Network 904 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 901, 902, 903 to interact with a server 905 over a network 904 to receive or send messages and the like. The terminal devices 901, 902, 903 may have various messaging client applications installed thereon, such as shopping applications, web browser applications, search applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 901, 902, 903 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 905 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 901, 902, 903. The backend management server may analyze and perform other processing on the received data such as the product information query request, and feed back a processing result (for example, target push information, product information — just an example) to the terminal device.
It should be noted that the index data constructing method provided by the embodiment of the present invention is generally executed by the server 905, and accordingly, the index data constructing apparatus is generally disposed in the server 905.
It should be understood that the number of terminal devices, networks, and servers in fig. 9 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 10, shown is a block diagram of a computer system 1000 suitable for use with a terminal device or server implementing embodiments of the present invention. The terminal device or the server shown in fig. 10 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 10, the computer system 1000 includes a Central Processing Unit (CPU)1001 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)1002 or a program loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. In the RAM 1003, various programs and data necessary for the operation of the system 1000 are also stored. The CPU 1001, ROM 1002, and RAM 1003 are connected to each other via a bus 1004. An input/output (I/O) interface 1005 is also connected to bus 1004.
The following components are connected to the I/O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 1008 including a hard disk and the like; and a communication section 1009 including a network interface card such as a LAN card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The driver 1010 is also connected to the I/O interface 1005 as necessary. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1010 as necessary, so that a computer program read out therefrom is mounted into the storage section 1008 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication part 1009 and/or installed from the removable medium 1011. The computer program executes the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 1001.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor comprises a change data acquisition module, a change data writing module and an index data construction module. The names of these modules do not in some cases form a limitation on the modules themselves, and for example, the change data collection module may also be described as a "module for receiving change data of business data in a source system through a unified data collection interface".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: receiving the change data of the service data in the source system through a uniform data acquisition interface, wherein the change data comprises a service identifier and a service change field of the service data; writing the changed data into a first database according to a preset structure, and generating a change message after the changed data is successfully written, wherein the change message comprises a service identifier of the service data; and responding to the generated change message, inquiring fields for generating indexes from the first database according to the service identification in the change message, and assembling the inquired fields to obtain index data.
According to the technical scheme of the embodiment of the invention, the changed data of the business data in the source system is received through a uniform data acquisition interface, and the changed data comprises the business identification and the business changed field of the business data; writing the changed data into a first database according to a preset structure, and generating a change message after the changed data is successfully written, wherein the change message comprises a service identifier of the service data; and responding to the generated change message, inquiring fields for generating indexes from the first database according to the service identification in the change message, and assembling the inquired fields to obtain index data. The method can realize the unified control of data writing, solve the problem that new data is covered by old data, improve the accuracy and efficiency of data writing, and reduce the performance requirements on the system.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (22)
1. A method for constructing index data is characterized by comprising the following steps:
receiving changed data of service data in a source system through a uniform data acquisition interface, wherein the changed data comprises a service identifier and a service change field of the service data;
writing the changed data into a first database according to a preset structure, and generating a change message after the changed data is successfully written, wherein the change message comprises a service identifier of the service data;
and responding to the generation of the change message, inquiring a field for generating an index from the first database according to the service identification in the change message, and assembling the inquired field to obtain index data.
2. The method of claim 1, wherein the change data further comprises a data version number of the business data;
the receiving of the change data of the service data in the source system through the unified data acquisition interface includes:
and receiving the changed data of the service data in the source system by taking the service identifier, the data version number and the service change field as interface parameters of the unified data acquisition interface.
3. The method of claim 2, wherein after receiving the changed data of the service data in the source system through the unified data collection interface, the method comprises:
and performing field verification on the received changed data of the service data in the source system to confirm that the received changed data of the service data in the source system comprises each field in the interface parameters.
4. The method according to claim 3, wherein after the step of confirming that the received change data of the service data in the source system includes the fields in the interface parameters, the method comprises: assembling the received change data of the service data in the source system into a first message, and issuing the first message to a first message middleware;
the writing the changed data into the first database according to the preset structure comprises: and in response to the first message issued to the first message middleware, writing the changed data to a first database according to a preset structure.
5. The method of claim 2 or 4, wherein writing the changed data to the first database according to a preset structure comprises:
carrying out data validity verification on the changed data and confirming that the data validity verification is passed, wherein the data validity verification comprises verifying a service identifier, a data version number and a service change field in the changed data;
and converting the changed data according to the data storage structure of the first database and writing the converted data into the first database.
6. The method of claim 5, wherein converting and writing the changed data to the first database according to the data storage structure of the first database comprises:
converting the changed data according to a data storage structure of the first database to obtain the converted changed data;
acquiring a distributed lock, and after confirming that the distributed lock is successfully acquired, performing version judgment on the changed data to confirm that the changed data is the latest version;
writing the converted changed data of the latest version into the first database according to a corresponding source system identifier, wherein the source system identifier is the identifier of the source system to which the service data belongs, which is determined according to the service change field in the changed data.
7. The method of claim 6, wherein said making a version determination of the change data to confirm that the change data is a latest version comprises:
acquiring the latest stored version number of the changed data from a second database;
and comparing the data version number in the changed data with the stored version number, and confirming that the changed data is the latest version when the data version number in the changed data is greater than or equal to the stored version number.
8. The method of claim 7, wherein after writing the latest version of the transformed change data to the first database according to the corresponding source system identification, comprising:
releasing the distributed lock, and updating the stored version number in the second database to the data version number in the changed data.
9. The method of claim 1, wherein after generating the change message, the method comprises: publishing the change message to second message middleware;
the querying, in response to generating the change message and according to the service identifier in the change message, a field for generating an index from the first database includes: responding to the change message issued to the second message middleware, and inquiring fields for generating indexes from the first database according to service identifications in the change message, wherein the fields for generating indexes are one or more fields of the service data.
10. The method of claim 9, wherein the assembling of the index data from the queried fields comprises:
and assembling the inquired fields according to an index structure of a storage cluster to obtain the index data, and storing the index data into the storage cluster by calling an application program interface of the storage cluster, wherein the index data is used for inquiring a corresponding service identifier by the storage cluster so as to obtain the service data from a source system of the service data according to the corresponding service identifier.
11. An index data construction device, comprising:
the system comprises a change data acquisition module, a data processing module and a data processing module, wherein the change data acquisition module is used for receiving change data of service data in a source system through a uniform data acquisition interface, and the change data comprises a service identifier and a service change field of the service data;
the change data writing module is used for writing the change data into a first database according to a preset structure and generating a change message after the change data is successfully written, wherein the change message comprises the service identifier of the service data;
and the index data construction module is used for responding to the generation of the change message, inquiring a field for generating an index from the first database according to the service identifier in the change message, and assembling the field by the inquired field to obtain index data.
12. The apparatus of claim 11, wherein the change data further comprises a data version number of the service data;
the change data collection module is further configured to:
and receiving the changed data of the service data in the source system by taking the service identifier, the data version number and the service change field as interface parameters of the unified data acquisition interface.
13. The apparatus of claim 12, further comprising a field validation module to:
and performing field verification on the received changed data of the service data in the source system to confirm that the received changed data of the service data in the source system comprises each field in the interface parameters.
14. The apparatus of claim 13, wherein the field validation module is further configured to: assembling the received change data of the service data in the source system into a first message, and issuing the first message to a first message middleware;
the change data writing module is further configured to: and in response to the first message issued to the first message middleware, writing the changed data to a first database according to a preset structure.
15. The apparatus of claim 12 or 14, wherein the change data writing module is further configured to:
carrying out data validity verification on the changed data and confirming that the data validity verification is passed, wherein the data validity verification comprises verifying a service identifier, a data version number and a service change field in the changed data;
and converting the changed data according to the data storage structure of the first database and writing the converted data into the first database.
16. The apparatus of claim 15, wherein the change data writing module is further configured to:
converting the changed data according to a data storage structure of the first database to obtain the converted changed data;
acquiring a distributed lock, and after confirming that the distributed lock is successfully acquired, carrying out version judgment on the changed data to confirm that the changed data is the latest version;
writing the converted changed data of the latest version into the first database according to a corresponding source system identifier, wherein the source system identifier is the identifier of the source system to which the service data belongs, which is determined according to the service change field in the changed data.
17. The apparatus of claim 16, wherein the change data writing module is further configured to:
acquiring the latest stored version number of the changed data from a second database;
and comparing the data version number in the changed data with the stored version number, and confirming that the changed data is the latest version when the data version number in the changed data is greater than or equal to the stored version number.
18. The apparatus of claim 17, wherein the change data writing module is further configured to:
releasing the distributed lock, and updating the stored version number in the second database to the data version number in the changed data.
19. The apparatus of claim 11, wherein the change data writing module is further configured to: publishing the change message to second message middleware;
the index data construction module is further configured to: responding to the change message issued to the second message middleware, and inquiring fields for generating indexes from the first database according to service identifications in the change message, wherein the fields for generating indexes are one or more fields of the service data.
20. The apparatus of claim 19, wherein the index data construction module is further configured to:
and assembling the inquired fields according to an index structure of a storage cluster to obtain the index data, and storing the index data into the storage cluster by calling an application program interface of the storage cluster, wherein the index data is used for inquiring a corresponding service identifier by the storage cluster so as to obtain the service data from a source system of the service data according to the corresponding service identifier.
21. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-10.
22. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210445890.1A CN114860721A (en) | 2022-04-26 | 2022-04-26 | Index data construction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210445890.1A CN114860721A (en) | 2022-04-26 | 2022-04-26 | Index data construction method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114860721A true CN114860721A (en) | 2022-08-05 |
Family
ID=82632995
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210445890.1A Pending CN114860721A (en) | 2022-04-26 | 2022-04-26 | Index data construction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114860721A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116431673A (en) * | 2023-04-19 | 2023-07-14 | 中国人民财产保险股份有限公司 | Data information query method, device, equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107783975A (en) * | 2016-08-24 | 2018-03-09 | 北京京东尚科信息技术有限公司 | The method and apparatus of distributed data base synchronization process |
| WO2021218143A1 (en) * | 2020-04-29 | 2021-11-04 | 平安科技（深圳）有限公司 | Data synchronization method and apparatus, and server and storage medium |
| CN113760320A (en) * | 2021-01-04 | 2021-12-07 | 北京京东振世信息技术有限公司 | Metadata deployment method and device |
| CN113961580A (en) * | 2021-12-22 | 2022-01-21 | 联通智网科技股份有限公司 | Data query method, service system and electronic equipment |
| CN113961643A (en) * | 2021-10-20 | 2022-01-21 | 广州华多网络科技有限公司 | Search engine updating method and device, equipment, medium and product thereof |
| CN114116737A (en) * | 2021-10-22 | 2022-03-01 | 北京旷视科技有限公司 | Object updating method, apparatus and electronic device |
-
2022
- 2022-04-26 CN CN202210445890.1A patent/CN114860721A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107783975A (en) * | 2016-08-24 | 2018-03-09 | 北京京东尚科信息技术有限公司 | The method and apparatus of distributed data base synchronization process |
| WO2021218143A1 (en) * | 2020-04-29 | 2021-11-04 | 平安科技（深圳）有限公司 | Data synchronization method and apparatus, and server and storage medium |
| CN113760320A (en) * | 2021-01-04 | 2021-12-07 | 北京京东振世信息技术有限公司 | Metadata deployment method and device |
| CN113961643A (en) * | 2021-10-20 | 2022-01-21 | 广州华多网络科技有限公司 | Search engine updating method and device, equipment, medium and product thereof |
| CN114116737A (en) * | 2021-10-22 | 2022-03-01 | 北京旷视科技有限公司 | Object updating method, apparatus and electronic device |
| CN113961580A (en) * | 2021-12-22 | 2022-01-21 | 联通智网科技股份有限公司 | Data query method, service system and electronic equipment |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116431673A (en) * | 2023-04-19 | 2023-07-14 | 中国人民财产保险股份有限公司 | Data information query method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110019350B (en) | Data query method and device based on configuration information | |
| CN113760948B (en) | Data query method and device | |
| US20210248143A1 (en) | Automatically executing graphql queries on databases | |
| CN112860451A (en) | Multi-tenant data processing method and device based on SaaS | |
| CN112445626B (en) | Data processing method and device based on message middleware | |
| CN112948138B (en) | A method and device for processing messages | |
| CN113190517B (en) | Data integration method and device, electronic equipment and computer readable medium | |
| US12306983B2 (en) | Systems and methods for using secure, encrypted communications across distributed computer networks to provide variable resiliency when indexing blockchain states for performing blockchain operations in decentralized applications using cryptography-based digital repositories | |
| US12393665B2 (en) | Method of processing cross-domain authorization and method of processing cross-domain call | |
| CN113641706B (en) | Data query method and device | |
| CN113297222B (en) | Report data acquisition method and device, electronic equipment and storage medium | |
| CN111460129A (en) | Method and device for generating identification, electronic equipment and storage medium | |
| CN112214500A (en) | Data comparison method and device, electronic equipment and storage medium | |
| CN110909022A (en) | A data query method and device | |
| CN113077201B (en) | Method, device and system for analyzing service parameters | |
| CN114860721A (en) | Index data construction method and device | |
| CN110109912B (en) | Identifier generation method and device | |
| CN110705935B (en) | Logistics document processing method and device | |
| CN113760343A (en) | Method and device for processing service request | |
| CN118260329A (en) | Method, apparatus, device and computer readable medium for processing business data table | |
| CN113535768A (en) | Production monitoring method and device | |
| CN110362599B (en) | Data storage method, device, electronic equipment and computer readable medium | |
| CN113421163B (en) | A reconciliation method, reconciliation application cluster and related client | |
| CN113553206B (en) | Data event execution method and device, electronic equipment and computer readable medium | |
| CN117194463A (en) | Method and device for inquiring report data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |