CN106294353A - Information processing method and device - Google Patents

Abstract

The present invention discloses an information processing method and device. The method includes: storing configuration information to form a configuration matrix; wherein, the serial number of each element in the configuration matrix has a mapping relationship with a query index; the storage content of the element It is the query content of the query based on the query index; the query index is obtained from the input data; the sequence number of the element to be queried is determined according to the query index; and the query content is queried according to the sequence number.

Description

Information processing method and device

technical field

The present invention relates to the field of information processing, in particular to an information processing method and device.

Background technique

In the existing information processing process, there is usually a lot of information content stored, such as using a matrix to store information content, and then matching the query index with the stored information content to obtain the final query content.

This information query and matching method, when there are a lot of information content, needs to be matched one by one and other traversal methods to query information, which will obviously lead to problems such as slow information query speed and more resources consumed by query equipment.

Contents of the invention

In view of this, the embodiments of the present invention expect to provide an information processing method and device, which can speed up the efficiency of data query and reduce the resources consumed by information query.

In order to achieve the above object, technical solution of the present invention is achieved in that way:

The first aspect of the embodiments of the present invention provides an information processing method, the method including:

Storing configuration information to form a configuration matrix; wherein, the sequence number of each element in the configuration matrix has a mapping relationship with the query index; the storage content of the element is the query content of the query based on the query index;

obtaining said query index from input data;

Determine the serial number of the element to be queried according to the query index;

The query content is queried according to the serial number.

Preferably, the configuration matrix includes zero elements and non-zero elements;

The non-zero element is: formed according to the information record in the configuration information that has determined the corresponding relationship between the query index and the query content; the query content is the content of the non-zero element;

The zero element is an element that has no mapping relationship between the sequence number of the element and the query index in the configuration information; the storage content of the non-zero element is empty.

Preferably, the query index is the terminal's mobile equipment international identity code IMEI;

The serial number of the element is the last N digits of the IMEI;

The N is a positive integer not greater than 15;

The query content includes a communication identification corresponding to the IMEI.

Preferably, the receiving input data and obtaining the query index include:

determining whether the communication identification is confirmed in the input data;

If the communication identification is missing, extracting the IMEI from the input data;

The determining the serial number of the element to be queried according to the query index includes:

Extract the last N digits of the IMEI;

The querying of the query content based on the serial number includes:

The communication identifier is obtained from the corresponding storage location according to the last N of the IMEI.

Preferably, the method also includes:

Supplementing the communication identification corresponding to the IMEI to the input data to form output data.

The second aspect of the embodiment of the present invention provides an information processing device, the device comprising:

The storage unit is used to store configuration information and form a configuration matrix; wherein, the serial number of each element in the configuration matrix has a mapping relationship with the query index; the storage content of the element is the query content of the query based on the query index;

an acquisition unit, configured to acquire the query index from input data;

a determining unit, configured to determine the serial number of the element to be queried according to the query index;

A query unit, configured to obtain the query content according to the serial number.

Preferably, the configuration matrix includes zero elements and non-zero elements;

The non-zero element is: formed according to the information record in the configuration information that has determined the corresponding relationship between the query index and the query content; the query content is the content of the non-zero element;

The zero element is an element that has no mapping relationship between the sequence number of the element and the query index in the configuration information; the storage content of the non-zero element is empty.

Preferably, the query index is the terminal's mobile equipment international identity code IMEI;

The serial number of the element is the last N digits of the IMEI;

The N is a positive integer not greater than 15;

The query content includes a communication identification corresponding to the IMEI.

Preferably, the obtaining unit is specifically configured to determine whether the communication identification is indeed contained in the input data; and if the communication identification is missing, extract the IMEI from the input data;

The determining unit is specifically configured to extract the last N digits of the IMEI;

The query unit is specifically configured to obtain the communication identification from the corresponding storage location according to the last N of the IMEI.

Preferably, the device also includes:

A forming unit, configured to add the communication identification corresponding to the IMEI to the input data to form output data.

In the information processing method and device of the embodiment of the present invention, the configuration matrix formed by the configuration information has a mapping relationship with the query index, and when performing information query based on the query index, the query index is converted into the serial number of the element according to the mapping relationship, and the The element to be queried can be determined, and the content stored in the element is extracted as the query content to be queried. Obviously, this avoids matching the query index with the query index in the stored content one by one as in the prior art , thereby improving the query efficiency and reducing the hardware and software resources consumed by the query.

Description of drawings

FIG. 1 is one of the schematic flow diagrams of the information processing method described in the embodiment of the present invention;

Fig. 2 is the second schematic flow diagram of the information processing method described in the embodiment of the present invention;

Fig. 3 is the third schematic flow diagram of the information processing method described in the embodiment of the present invention;

FIG. 4 is a schematic structural diagram of an information processing device according to an embodiment of the present invention.

detailed description

The technical solutions of the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

Method example:

As shown in Figure 1, this embodiment provides an information processing method, the method comprising:

Step S110: storing configuration information to form a configuration matrix; wherein, the sequence number of each element in the configuration matrix has a mapping relationship with the query index; the stored content of the element is the query content for query based on the query index;

Step S120: Obtain the query index from the input data;

Step S130: Determine the serial number of the element to be queried according to the query index;

Step S140: Query the query content according to the serial number.

The sequence number can refer to the order number of each element in the matrix, such as a 10*10 matrix, including 100 elements, when sorting the elements in the configuration matrix, the count can be arranged, then the first row The serial number of the first element is 1, the serial number of the second element in the first row is 2; it can also be counted by column, then the serial number of the first element in the first row is 1, and the first element in the second row The serial number is 2.

Since the query index has a mapping relationship with the serial number of the element, the serial number of the element storing the query content can be determined according to the query index, so that the query content can be quickly queried by means of matrix element query , so as to avoid obtaining the query content by matching the query index and stored information as in the prior art, which obviously greatly improves the efficiency of information query and reduces the system overhead of information query, and in this application , the stored query content itself does not include the query index, which also reduces the amount of data storage. In a specific implementation, the query index may also be stored in the elements of the matrix, so as to be checked later and provide more accurate query results.

The information processing method described in this embodiment can be applied to a flow computing system, and the flow computing is mainly applied to functions such as real-time data processing and statistical learning. With the explosion of Internet big data, stream computing also adopts more advanced distributed computing methods to improve processing speed, which is called distributed stream computing system. The most representative distributed stream computing systems are ibm stream and strom. Applications written based on distributed stream computing systems can run on large clusters composed of hundreds or thousands of commercial machines, and use a Parallel processing of T-level data sets in a reliable and fault-tolerant manner. The application program divides the tasks that need to be calculated into many small pieces, and makes them pass through different processing nodes in a stream-like manner to complete real-time data analysis and event processing. The information query method described in this embodiment can be applied to information query in a stream computing system.

The query index described in this embodiment is preferably information such as numbers and characters that can be used for sorting by numbers, letters, etc.; and usually has unique identification information; such as the unique IMEI or communication identification or various card distribution. Serial number and other information. Specifically, for example, when applied to the processing of student achievement query information, the query index can be the student number or the admission ticket number, and the query content can include test scores; using the method described in this embodiment, the test scores of each student can be generated into a configuration matrix , and the student number or admission ticket number will be used as the serial number of each element in the matrix, so that after the student enters the student number or admission ticket number, the query device can immediately find the corresponding element and its internal memory by jumping pointers. Content.

The configuration matrix includes zero elements and non-zero elements;

The non-zero element is: formed according to the information record in the configuration information that has determined the corresponding relationship between the query index and the query content; the query content is the content of the non-zero element;

The zero element is an element that has no mapping relationship between the sequence number of the element and the query index in the configuration information; the storage content of the non-zero element is empty.

When there are many zero elements, a sparse matrix is formed; the zero elements in the sparse matrix do not occupy storage space, and obviously, when the query index is not stored, the stored data content is reduced.

The query index is the terminal's mobile equipment international identity code IMEI;

The serial number of the element is the last N digits of the IMEI;

The N is a positive integer not greater than 15;

The query content includes a communication identification corresponding to the IMEI.

The communication identification may be information such as a mobile phone number, a landline phone number, etc. used as an identification in the communication process. In a specific implementation, the query index may also be the communication identifier, and the query content may be the IMEI or other content.

The following table is a specific example:

Mobile phone IMEI number mobile phone number 460000806513440 13904075196 460000954645033 13976318088 460004070113304 13835739538 460005007585583 13837314528 460005041200278 13879979487 460005051417846 13590952542

Table 1

If said N=10, adopt the method described in this embodiment, form configuration matrix then as follows:

Mobile phone IMEI number serial number storage content 000000000000000 Null Null ... Null Null 460000806513440 0806513440 13904075196 ... Null Null 460000954645033 0954645033 13976318088 ... Null Null 460004070113304 4070113304 13835739538 ... Null Null 460005007585583 15007585583 13837314528 ... Null Null

Table 2

It can be seen from Table 2 that the serial number in the storage structure is the last 10 digits of the IMEI number (currently the first 5 digits of the IMEI number remain unchanged), such a storage format means that the intercepted 10 digits of the IMEI number in the input data can be directly located To the location of the mobile phone number in the storage structure, if a piece of data is input externally, the mobile phone number field of this data is empty, and the mobile phone number needs to be supplemented. In addition, in the storage structure of the sparse matrix, the non-zero elements are far smaller than the total number of elements, and zero elements do not take up space in physical storage, which can not only improve query efficiency, but also save storage space.

Further, the step S120 may include: determining whether the communication identification is indeed contained in the input data; if the communication identification is missing, extracting the IMEI from the input data; the step S130 may include: extracting The last N digits of the IMEI. The step S140 may include: obtaining the communication identification according to the last N of the IMEI to the corresponding storage location. When the input external data includes the communication identification, the step S120 to step S140 may not be performed, and the input data may be processed according to the flow to be processed in advance, such as transferring to the next processing node for data processing .

As a further supplement to the above embodiment, the method further includes: adding the communication identification corresponding to the IMEI to the input data to form output data.

Several specific examples are provided below in conjunction with the method described in this embodiment:

Example one:

Scenarios used in this example:

The streaming computing cluster needs to collect user log data, but there is a problem with the data quality in the source system. The problem is that some records in the source data lack mobile phone numbers, which need to be supplemented during the ETL (Extract-Load-Transform) process, otherwise the mobile phone number is missing The records will not be able to use the mobile phone number for business applications.

If the mobile phone number is missing in the source data record provided by the source system, the method described in this embodiment is used to supplement the mobile phone number. For example, another field IMEI (International Mobile Equipment Identity) in the source data record is associated with the user data table to obtain the mobile phone number in the user data table and added to the mobile phone number field of the source data record. The source data record is the input data.

As shown in Figure 2, this example specifically includes:

Step 1: Input external data to the stream computing mobile phone number supplement node, the input data can be as shown in Table 3:

table 3

Step 2: Determine whether the mobile phone number is missing, and confirm that the mobile phone number field of the input data is empty;

Step 3: Find the missing mobile phone number, and obtain the last 10 digits of the IMEI number from the external input data. For example, the IMEI number is: [460005007585583], and the intercepted value is: [5007585583];

Step 4: Obtain the mobile phone number data corresponding to the serial number in the configuration data storage through the last 10 digits of IMEI: value[5007585583]=13837314528;

Step 5: Configure data storage to return the mobile phone number data to the stream computing mobile phone number replenishment node;

Step 6: The stream computing mobile phone number supplement node will supplement the returned data to the input data. At this moment, the content of the input data is:

Step 7: The supplemented input data is sent to the next processing node for subsequent real-time processing.

So far, the quick matching function based on streaming computing products has come to an end.

As shown in Figure 3,

Streams uses XSD (XML Schema Definition) to describe the JAVA operator model, covering: syntax expressions, required parameters, input port definitions, output port definitions, and dependent class libraries. When the JAVA operator is created, the configuration file information will be generated accordingly. The stream calls JAVA according to the XML configuration generated by JAVA. When calling the stream computing language, it only needs to call the file name of JAVA. For example, the code "HdfsSink" is the name of the custom JAVA operator created, and it only needs to be called as the operator name when calling.

Example two:

As shown in Figure 3, this example provides an information processing method including:

input configuration data into the configuration generation tool;

The configuration generation tool forms the configuration data into a sparse matrix storage;

Receive a piece of user data; the user data is the above-mentioned input data; the user data includes the query index;

Use index matching to quickly locate; the positioning result formed by positioning can be used for query content acquisition;

Query content acquisition;

After the query content is added to the user data, an updated user data is formed.

The method described in the embodiment of the present application is applicable to various information processing that requires information query matching processing, and is especially suitable for streaming information processing.

It can be applied to the field of business analysis system. The sparse matrix-based storage structure stores the configuration information in the memory, and by embedding the fast matching algorithm implemented by JAVA into the streaming node, it can quickly locate and configure during the real-time input of massive data. Data location, fetch the desired data. It not only improves processing efficiency, meets real-time requirements, but also avoids resource consumption (CPU, processing time) caused by searching data in common storage structures.

Device example:

As shown in Figure 4, this embodiment provides an information processing device, wherein the device includes:

The storage unit 110 is used to store configuration information and form a configuration matrix; wherein, the sequence number of each element in the configuration matrix has a mapping relationship with the query index; the storage content of the element is the query content of the query based on the query index;

An obtaining unit 120, configured to obtain the query index from input data;

A determining unit 130, configured to determine the serial number of the element to be queried according to the query index;

The query unit 140 is configured to obtain the query content according to the serial number.

The storage unit 110 in this embodiment may include a storage medium to store the configuration information in a matrix. Specifically, when the sequence numbers corresponding to the query indexes in the configuration information forming the configuration matrix are very sparse, usually the configuration matrix forms a sparse matrix. The elements in the configuration matrix that have a mapping relationship with the query index in the configuration information are usually non-zero elements, while the elements that have no mapping relationship with the configuration information are usually non-zero elements. The non-zero elements in the sparse matrix are formed according to the information records in the configuration information that have determined the corresponding relationship between the query index and the query content.

The storage medium may be various storage media such as RAM and flash, and may be a non-transient storage medium.

The acquisition unit 120 may include a communication interface, receive the input data from an external device or directly receive the query index, and the acquisition unit 120 may also include a parser or an information processor with an information analysis function, which can be used for The input data is parsed to obtain a query index.

The specific structures of the determining unit 130 and the querying unit 140 may include various types of processors, and the processors determine the functions of the determining unit 130 and the querying unit 140 by executing executable codes. The processor may include electronic devices with information processing functions such as a central processing unit CPU, a microprocessor MCU, a digital signal processor DSP, an application processor AP, or a programmable processor PLC. The determination unit 130 and the query unit 140 may respectively correspond to different processors, or may be integrated and correspond to the same processor.

Based on the above, this embodiment provides an information processing device capable of implementing the information processing method in the above method embodiment, which obviously has the advantages of high efficiency and less consumption of system resources when performing information query matching. At the same time, if the query index is not stored in each element, the amount of information storage will be reduced, thereby reducing the occupation of storage space. The system resources here may include CPU processing resources, thread occupation time, memory occupation, and the like.

The query index is the terminal's mobile equipment international identity code IMEI; the serial number of the element is the last N digits of the IMEI; the N is a positive integer not greater than 15; the N can be specifically 10, etc. The query content includes a communication identification corresponding to the IMEI. The communication identification may be a mobile phone number or the like. The query content may also include user name, communication package, user's current prepaid fee and other information.

Apparently, the query index and query content described in this embodiment are only a specific example of the present application, and are not limited to this specific example during specific implementation.

Further, the obtaining unit 120 is specifically configured to determine whether the communication identification is indeed in the input data; and if the communication identification is missing, extract the IMEI from the input data;

The determining unit 130 is specifically configured to extract the last N digits of the IMEI;

The query unit 140 is specifically configured to obtain the communication identifier from the corresponding storage location according to the last N of the IMEI.

In this embodiment, the acquisition unit 120 will query the query content according to the query only when it is determined that the communication identifier is indeed in the input data, that is, the input data lacks part of the data and needs to be supplemented by extracting information from the configuration information, so that Complete the missing information in the input data to form updated data content and input it to subsequent processing nodes. In conjunction with the query of the communication identification, in this embodiment, the acquisition unit judges whether the input data includes the communication identification, and when the communication identification is confirmed, extracts the IMEI; and facilitates the subsequent determination unit to extract the N bits from the IMEI as Determine the sequence number of the element to be looked up.

The physical structures of the acquisition unit 120, the determination unit 130 and the query unit 140 in this embodiment can be referred to the foregoing part, and will not be repeated here. In short, the acquisition unit 120, the determination unit 130 and the query unit 140 can be Inquiry on communication identification.

The device further includes: a forming unit, configured to add the communication identification corresponding to the IMEI to the input data to form output data. Supplement the query content to the input data to form the supplemented data, which is convenient for subsequent node functions.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or May be integrated into another system, or some features may be ignored, or not implemented. In addition, the coupling, or direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms of.

The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units; Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention can be fully integrated into one processing module, or each unit can be used as a single unit, or two or more units can be integrated into one unit; the above-mentioned integration The unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

Those of ordinary skill in the art can understand that all or part of the steps for realizing the above-mentioned method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the Including the steps of the foregoing method embodiment; and the aforementioned storage medium includes: various storage devices, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk A medium on which program code can be stored.

The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present invention. Should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (10)

1. an information processing method, it is characterised in that described method includes:

Storage configuration information, forms configuring matrix；Wherein, in described configuring matrix the serial number of each element with Search index has mapping relations；The storage content of described element is to carry out inquiring about based on described search index Inquiry content；

Described search index is obtained from input data；

The serial number of element to be checked is determined according to described search index；

Described inquiry content is inquired about according to described serial number.

Method the most according to claim 1, it is characterised in that

Described configuring matrix includes neutral element and nonzero element；

Described nonzero element is: close according to having determined that in described configuration information that search index is corresponding with inquiry content The information record of system is formed；Described inquiry content is the content of described nonzero element；

Described neutral element is that the serial number according to described element does not map with search index in described configuration information The element of relation；The storage content of described nonzero element is empty.

Method the most according to claim 1, it is characterised in that

Described search index is the mobile device international identity code IMEI of terminal；

The rear N position of IMEI described in the Serial No. of described element；

Described N is the positive integer of no more than 15；

Described inquiry content includes the communication identifier corresponding to described IMEI.

Method the most according to claim 3, it is characterised in that

Described reception inputs data, obtains described search index, including:

Determine the most certain described communication identifier in described input data；

If lacking described communication identifier, then from IMEI described in described input extracting data；

The described serial number determining element to be checked according to described search index, including:

Extract the rear N position of described IMEI；

Described according to described serial number inquire about described inquiry content, including:

According to the rear N of described IMEI to communication identifier described in corresponding storage position acquisition.

Method the most according to claim 4, it is characterised in that

Described method also includes:

The communication identifier corresponding with described IMEI is added in described input data, form output data.

6. an information processor, it is characterised in that described device includes:

Memory element, is used for storing configuration information, forms configuring matrix；Wherein, each element in configuring matrix Serial number and search index there are mapping relations；The storage content of described element is based on described search index Carry out the inquiry content inquired about；

Acquiring unit, for from search index described in input data acquisition；

Determine unit, for determining the serial number of element to be checked according to described search index；

Query unit, for obtaining described inquiry content according to described serial number.

Device the most according to claim 6, it is characterised in that

Described configuring matrix includes neutral element and nonzero element；

Described nonzero element is: close according to having determined that in described configuration information that search index is corresponding with inquiry content The information record of system is formed；Described inquiry content is the content of described nonzero element；

Described neutral element is that the serial number according to described element does not map with search index in described configuration information The element of relation；The storage content of described nonzero element is empty.

Device the most according to claim 6, it is characterised in that

Described search index is the mobile device international identity code IMEI of terminal；

The rear N position of IMEI described in the Serial No. of described element；

Described N is the positive integer of no more than 15；

Described inquiry content includes the communication identifier corresponding to described IMEI.

Device the most according to claim 8, it is characterised in that

Described acquiring unit, specifically for determining the most certain described communication identifier in described input data；And If lacking described communication identifier, then from IMEI described in described input extracting data；

Described determine unit, specifically for extracting the rear N position of described IMEI；

Described query unit, specifically for the rear N according to described IMEI to corresponding storage position acquisition institute State communication identifier.

Device the most according to claim 9, it is characterised in that

Described device also includes:

Form unit, for the communication identifier corresponding with described IMEI is added in described input data, Form output data.