WO2018196650A1

WO2018196650A1 - User feature data acquisition method and device, server, and medium

Info

Publication number: WO2018196650A1
Application number: PCT/CN2018/083298
Authority: WO
Inventors: 王海平
Original assignee: 平安科技（深圳）有限公司
Priority date: 2017-04-26
Filing date: 2018-04-17
Publication date: 2018-11-01
Also published as: CN107798037A

Abstract

A user feature data acquisition method and device, a server, and a medium are provided, applicable to the technical field of the Internet. The method comprises: synchronizing user basic data in each service database to a distributed file system to build a user attributes library; using a computing engine to analyze and summarize the synchronized user basic data to obtain feature data corresponding to each user; based on an interface of the user attribute library, obtaining a data access request sent by a service system; and verifying validity of the data access request, and if the data access request is valid, returning, to the service system, user feature data matching the data access request. When multiple service databases are separated, the solution can rapidly obtain feature data corresponding to each user. The user feature data is evaluated comprehensively, and operations of the service system are not affected. Accurate data support is provided for functions such as personalized service customization and service recommendation.

Description

Method, device, server and medium for acquiring user characteristic data

This application is submitted to the China Patent Office on April 26, 2017, and the application number is 1 201710282779.4. The title of the invention is the priority of the Chinese Patent Application for "Using the User Feature Data Acquisition Method and Server", the entire contents of which are hereby incorporated by reference.

Technical field

The present application belongs to the field of Internet technologies, and in particular, to a method, an apparatus, a server, and a medium for acquiring user feature data.

Background technique

With the development of enterprise information, most companies have their own business systems. For example, financial and insurance companies have various types of business systems, such as life insurance insurance systems, property insurance insurance systems, and customer personal information systems. Each business system has its own independent database, which stores the operational data of the enterprise in this business.

However, if the report data of the business system is only queried from a separate database, it is difficult to comprehensively evaluate the user's feature information, and it is unable to provide data support for the user's personalized service customization and service recommendation functions. In the case of a large number of business databases and a large amount of business data, the business system cannot quickly analyze multi-dimensional user feature analysis results, which easily affects the stable operation of the business system.

In summary, in the prior art, in the case of multi-service database distribution, there is a problem that it is difficult to quickly acquire multi-dimensional user feature data.

technical problem

In view of this, the embodiment of the present application provides a method, an apparatus, a server, and a medium for acquiring user feature data, so as to solve the problem that in the prior art, when multi-service database is distributed, it is difficult to quickly acquire multi-dimensional user feature data. problem.

Technical solution

A first aspect of the embodiment of the present application provides a method for acquiring user feature data, including:

Synchronize user-based data in each business database to a distributed file system to build a user attribute library;

The calculation engine is used to analyze and summarize the synchronized user basic data, and the feature data corresponding to each user is obtained;

Obtaining a data access request sent by the service system based on the interface of the user attribute library;

Verifying the validity of the data access request, and if the data access request is legal, returning the feature data of the user that matches the data access request to the service system.

A second aspect of the embodiments of the present application provides a device for acquiring user feature data, where the device for acquiring user feature data includes a module for performing a method for acquiring user feature data according to the above first aspect.

A third aspect of an embodiment of the present application provides a server including a memory and a processor, the memory storing computer readable instructions executable on the processor, the processor executing the computer readable The steps of the method for acquiring user feature data as described in the first aspect are implemented when instructed.

A fourth aspect of the embodiments of the present application provides a computer readable storage medium storing computer readable instructions, the computer readable instructions being executed by a processor to implement the first aspect as described in the first aspect The steps of the method of obtaining user characteristic data.

Beneficial effect

In the embodiment of the present application, in the case that the multi-service database is separated, by constructing a user attribute library that summarizes the basic data of each user, the feature data corresponding to each user can be quickly parsed based on the calculation engine, and the user feature data is realized. Comprehensive assessment. Moreover, since the analysis and aggregation process is performed in the user attribute library, it does not affect the operation of the existing business system. In addition, by receiving the data access request by using the interface, each service system can conveniently obtain the feature data of the user, and can provide accurate functions for personalized service customization and service recommendation according to different feature data corresponding to different users. data support. At the same time, since the business system needs to perform conversion processing on the user basic data in other service databases by itself, the efficiency of the user system for obtaining the user characteristic data is improved.

DRAWINGS

1 is a flowchart of an implementation of a method for acquiring user feature data provided by an embodiment of the present application;

2 is a flowchart of an implementation of a method for acquiring user feature data according to another embodiment of the present application;

FIG. 3 is a specific implementation flowchart of a method S4 for acquiring user feature data according to an embodiment of the present application;

FIG. 4 is a flowchart of another specific implementation of the method S4 for acquiring user feature data provided by the embodiment of the present application;

FIG. 5 is a schematic diagram of an application scenario applicable to a method for acquiring user feature data according to an embodiment of the present disclosure;

FIG. 6 is a structural block diagram of an apparatus for acquiring user feature data according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a server provided by an embodiment of the present application.

Embodiments of the invention

In order to explain the technical solutions described in the present application, the following description will be made by way of specific embodiments.

The method for acquiring user feature data according to the embodiment of the present application may be executed on a server on which a distributed file system is installed. Among them, the distributed file system can be a Hadoop distributed file system (HDFS file system), which can be adapted to run on common hardware (commodity) Hardware) provides high throughput data access. FIG. 1 is a flowchart showing an implementation process of a method for acquiring user feature data provided by an embodiment of the present application, which is described in detail as follows:

S1: Synchronize user-based data in each business database to a distributed file system to build a user attribute library.

In the embodiment of the present application, the service database is used to manage and store data resources generated by the service system, and the data resources include user basic data. Among them, the user refers to the customer directly facing the business, that is, the user basic data is the business data or basic information related to the customer. For example, the user-based data generated by the life insurance system is the personal basic data of the policy customer. User-based data generated by different business systems can be stored in different business databases.

After the user-based data stored in each service database is synchronized to the HDFS file system by using the Hadoop technology, since the user-based data includes the personal data and the consumption information of the user when handling different services, the user-specific data system is generated substantially. The HDFS file system of the user-based data constitutes a user attribute library, which is also a database.

In the synchronization process of the above-mentioned user basic data, although there is a transmission time difference, the real-time synchronization between the service database and the user attribute database cannot be maintained, but since the user attribute library has low requirements on the real-time performance of the resource data, the above-mentioned user basic data is not required. The synchronization method can meet the construction requirements of the user attribute library.

S2: The calculation engine is used to analyze and summarize the synchronized user basic data, and the feature data corresponding to each user is obtained.

In the embodiment of the present application, SPARK refers to Apache Spark, which is a fast and universal computing engine designed for large-scale data processing. The SPARK Compute Engine enables memory-distributed datasets to optimize iterative workloads in addition to providing interactive queries.

The user attribute library uses the SPARK calculation engine and the ETL (Extraction, Transformation, Loading) tool to perform conversion and filtering analysis on the collected large amount of user basic data, and finally loads the processing result into the data warehouse according to the predefined data warehouse model. In the middle, the processing result is the summarized user characteristic data. The user characteristic data stored in the user attribute database is summary information for the usage, usage habits, and attention service of the user in each business system, and is summary statistics for each user.

Since the user attribute library is a Hadoop distributed file system, in the actual system structure, a Hadoop distributed file system can be composed of multiple servers. When the user base data is synchronized to the HDFS file system, the user base data from different service databases can also be synchronized to different servers, and each server uses the SPARK calculation engine and the ETL tool to synchronize the users on the server. The basic data is analyzed and summarized, and then the analysis results of each server are summarized.

In the embodiment of the present application, since the user attribute library uses multiple servers in parallel to implement analysis and summary of user basic data, the data processing efficiency of the system is improved.

S3: Obtain a data access request sent by the service system based on the interface of the user attribute library.

The interface is a shared boundary between the user attribute library and other external components for information exchange. It is developed by using the java servlet, and is composed of a preset expression program structure and data provided by a preset SQL language. The user's feature data is published externally by the interface of the user attribute library. If other business systems within the enterprise need to obtain or invoke the feature data of a certain user, the interface provided by the user attribute library is invoked by the post method to issue a data access request. The user attribute library interacts with the business system through the http protocol.

As another embodiment of the present application, as shown in FIG. 2, the foregoing step S1 specifically includes:

S11: Based on the JOB mode, the user basic data in each service database is synchronized to the Hadoop distributed file system at a preset time interval to construct a user attribute library.

After the foregoing S2, before S3, the method for acquiring the user feature data further includes:

S21: Save the feature data corresponding to each user to a temporary table of the user attribute library, and delete the temporary table in the next time before the user basic data in each service database is synchronized to the user attribute database. The feature data that has been stored.

In the embodiment of the present application, the JOB is a function provided by the database to periodically execute a certain stored procedure or a package body. In order to enable the HDFS file system to trigger the synchronization operation, the data synchronization is implemented by the SQL JOB timing job at the database level. The implementation principle includes: creating a data synchronization task by using the JOB method, and in the execution condition corresponding to the data synchronization task, Set the task execution time and the task execution interval. When the current time reaches the preset task execution time, or when the difference between the current time and the task execution time is a multiple of the task execution interval, the HDFS file system is created to establish a database connection for each service. The database is connected. Thereafter, the SQL statement preset to the HDFS file system is executed, and according to the syntax analysis result of the SQL statement, if the table parameter corresponding to the SQL statement exists in the service database, the HDFS file system is preset from the service database by the identity authority. The user base data is read and the read user base data update is inserted into the HDFS file system.

In particular, the HDFS file system creates a database connection that includes creating a dblink for the remote database. Based on the dblink method, the HDFS file system can quickly acquire the user base data of the remote service database as fast as accessing the local database, thereby improving synchronization efficiency.

If the data to be synchronized preset in the data synchronization task of the JOB mode is the user basic data generated by the service system within a preset time period, when the data synchronization task is executed, the HDFS file system only puts the service system in the pre-predetermined time. Set the user base data generated during the time period to be synchronized to the local. Preferably, the preset time period is from the first day of the month to the day before. That is, the HDFS file system only synchronizes the user-based data generated by the business system from the 1st of the month to the day.

On the day of the data synchronization task execution, the user base data in the service database is still in the real-time update state because the day has not passed. Therefore, in order to prevent the HDFS file system from being synchronized, the user base data in the service database is updated again. As a result, the HDFS file system and the service database are not substantially synchronized. When the HDFS file system performs the data synchronization task, only the user basic data generated by the business system before the day is synchronized. In addition, in actual operation, since the effective time of the user's business is calculated on a monthly basis, and the 31st of each month is the expiration date, for example, the user is insured for 5 on October 2 The monthly life insurance policy, the March 31 of the second year is the expiration date of the policy. Therefore, most of the user-based data in the service database before the 1st of each month is historical data. In order to improve the synchronization effectiveness of the HDFS file system for user-based data, reduce the amount of synchronization data, and improve synchronization efficiency, the HDFS file system only synchronizes monthly. User-based data generated by the business system from the 1st to the day.

In particular, in the user attribute library, the user basic data is divided into a common level and an important level according to the user level. The user property library is pre-configured with two JOB tasks for setting the analysis summary action execution time of the user base data of the common level and the important level.

For example, the user attribute library performs an analysis and summary operation on the user-level data of the synchronized important level according to the rules preset by the JOB task, and performs the analysis summary operation on the weekly basis for the user-level data of the common level.

Preferably, the user attribute library performs an analysis summary operation on the user base data only during the non-access busy period. For example, according to the statistical result, the business system usually concentrates on accessing the user characteristic data in the user attribute database during the time period from 09:00 to 18:00, and the JOB task analyzes and summarizes the user basic data that has been synchronized to the local. The task execution time can be set to 0:00 in the morning.

In the embodiment of the present application, the execution time of the analysis and summary action of the user attribute database is limited, so that when the service system issues a data access request, the user attribute database can respond in time, and does not occupy too much due to the analysis and summary operation. System resources improve the response efficiency of the user property library. By storing the feature data in the temporary table, the consumption of the memory resources of the user attribute library can be reduced, and the feature data in the temporary table can be cleared by timing to ensure that the amount of data in the temporary table is not excessive, so that the user attribute library has more Good performance.

S4: Verify the validity of the data access request, and if the data access request is legal, return the feature data of the user that matches the data access request to the service system.

When the user attribute database receives the data access request based on the user feature data sent by any service system from the interface, it does not immediately return the user's feature data to the service system, but first determines whether the data access request is legal, including: Whether the source of the data access request satisfies the preset data acquisition authority and determines whether the data content of the data access request is abnormal. Only when the data access request is legal, the user property library queries and returns the user feature data requested by the business system. The user feature data requested by the service system is the feature data that matches the user number carried in the data access request. The user attribute library performs a query from the stored data according to the user number, and converts the feature data obtained by the query into data in an xml format and returns.

As an embodiment of the present application, FIG. 3 shows a specific implementation process of the foregoing step S4, which is described in detail as follows:

S411: Determine whether the IP address carried by the data access request and the access key are both present in a configuration file of the user attribute database.

The user attribute library parses the received data access request, and reads the first attribute value corresponding to the source IP field and the second attribute value corresponding to the access key field from the data report of the data access request, then the first An attribute value is an IP address carried by the data access request, and represents an IP address currently used by the service system that issues the data access request, and the second attribute value is an access key, which is input by the operator of the service system. Account key.

During the run, the user property library loads the settings of the environment and the collection of files that it needs, which is the configuration file. In order to verify the validity of the data access request, the source IP address and the access key that are allowed to acquire the user feature data are pre-stored in the configuration file. The legal IP address saved in the configuration file includes the exact IP address, IP address segment, or IP address in the form of a wildcard. As long as the user attribute database detects that the IP address carried by the data access request is the same as the IP address in the configuration file or belongs to any IP address segment in the configuration file, it may be determined that the IP address carried by the data access request exists. In the configuration file of the user property library.

S412: If the IP address and the access key carried in the data access request are both in the configuration file of the user attribute database, acquire attribute information in the data access request, where the attribute information includes a data request parameter. And an encrypted visitor signature.

In the embodiment of the present application, the visitor signature is key data attached to the data access request or a transformation of the key data. The user attribute library decrypts the received visitor signature by using a pre-agreed decryption key, thereby determining whether the source of the data access request is a real authorized user, and confirming the message integrity of the data access request, thereby receiving the illegality When the numerator falsifies the data access request, the user attribute library can also recognize and respond to the data access request, thereby improving the transmission security of the user characteristic data.

In particular, if the IP address or access key carried by the data access request does not exist in the configuration file of the user attribute library, the user attribute library does not respond to the data access request.

S413: Perform decryption processing on the visitor signature, determine whether the decrypted signature of the visitor is the same as the signature in the configuration file, and determine whether the data request parameter satisfies a preset condition.

S414: If the decrypted signature of the visitor is the same as the signature in the configuration file, and the data request parameter meets a preset condition, determining that the data access request is legal and matching the data access request The feature data is returned to the business system.

The preset condition is the grammatical structure rule, including the character type attribute and the range of the value range. Only when each data request parameter in the data access request satisfies a preset grammatical structure rule, the user attribute database determines that the data access request is legal and responds to the data access request, that is, according to the characteristic data of the required user. After querying the feature data corresponding to the user from the user attribute library, return to the business system that issued the data access request.

If the decrypted visitor signature is different from the signature in the configuration file, and any data request parameter in the data access request does not satisfy the preset syntax structure rule, the user attribute database generates an error prompt information, and based on the error prompt information. Respond to the data access request.

For example, if the IP address and the access key carried in the data access request exist in the configuration file of the user attribute database and the signature of the visitor is legal, if the content of the data access request message is "select" Name from persons", according to the grammatical structure rules, the parameter type after "from" should be the table name, and the user attribute library does not have the "persons" table, therefore, the user attribute library determines that the data request parameter is not satisfied. The default syntax structure rules, and the generated error message packet is returned through the interface to the business system that issues the data access request.

Preferably, when receiving the data access request, the user attribute library first determines whether the number of data access requests received in the first time period reaches a preset threshold. The first time period is a time interval from when the data access request is received to when the data access request is received, and the length of the first time period is a preset duration. If the number of data access requests received in the first time period has reached a preset threshold, the data access request received at the current time does not respond, and the above steps S411 to S414 are not performed.

In the embodiment of the present application, the user attribute library determines the number of data access requests within the preset duration, and can not make any response when the number exceeds the threshold, thereby preventing the user attribute library from continuously processing a large number of data access requests, and controlling The processing frequency of the data access request ensures the normal operation of the user attribute library.

In particular, when the user attribute database provides querying user feature data externally, the view is performed in a view manner. The service system directly invokes the view provided by the user attribute library through the foregoing interface, and can directly request the user feature data corresponding to the view. Therefore, when the storage data is adjusted within the user attribute library, since the view and the interface are not changed, the service system that invokes the user attribute data of the user attribute library through the interface and the view will not be affected, and the adjustment process is performed on the service. The system is invisible.

For example, when the user attribute database transfers historical user feature data or performs sub-table operations on a large number of user feature data, although the storage structure of the user feature data changes, in the view of the business system, the interface can still pass through the interface. The user view data obtained by the embodiment of the present application ensures the normal operation of the service system and improves the reliability of the service system.

As an embodiment of the present application, FIG. 4 shows another specific implementation flow of the foregoing step S4, which is described in detail as follows:

S421: Acquire an IP address of the data access request.

S422: Acquire a total number of historical data access requests based on the IP address within a preset duration.

In the embodiment of the present application, according to the access log file, the user attribute database determines that the source IP of the data access request received in the preset time period is the same as the IP address in S421. The preset time period is a time interval from a moment before the data access request is received to when the data access request is received, and the length of the preset time period is a preset duration.

The detailed message of the above log file includes information such as the source time of each data access request, the request parameter, the source IP, the processing duration of the user attribute database for each data access request, and the processing result.

S423: If the total number of historical data access requests based on the IP address is greater than a preset threshold within a preset duration, the data access request is cached, and the cached data access request is cached after a preset time delay. Legality is verified.

And if the total number of data access requests based on the IP address in the preset duration is greater than a first preset threshold and less than a second preset threshold, the data access request is cached. If the duration of the data access request cached in the pre-created cache area has reached the preset duration, the user attribute database performs legality verification on the data access request.

If the total number of data access requests based on the IP address within the preset duration is greater than a second predetermined threshold, the data access request is not responded.

S424: If the data access request is legal, return the feature data that matches the data access request to the service system.

The principle of the steps not mentioned in the embodiment of the present application is consistent with the principle of the steps in the foregoing embodiments, and therefore will not be further described.

In the embodiment of the present application, by delaying or not responding to the data access request of the same source IP, the resource occupancy of the malicious data request to the user attribute database can be alleviated in the case of a malicious request, and the user attribute is guaranteed. The normal operation of the library; by performing log records on the received data access requests, it is possible to provide maintenance personnel with troubleshooting clues and improve maintenance efficiency after a problem occurs in the user attribute library after the maintenance process.

Further, in addition to providing the foregoing interface for the external service system to acquire the feature data of the user, the user attribute library also provides a monitoring interface for the external monitoring server for monitoring the interface service of the user attribute library. The monitoring server continuously detects whether the monitoring interface returns a normal response packet for 24 hours. When the normal response packet is not received, it indicates that the interface service is abnormal. At this time, the monitoring server will issue an alarm prompt, so that the operation and maintenance personnel can quickly perform troubleshooting and restore the interface service as soon as possible, so that the user interface can be improved based on the monitoring interface. Reliability to avoid disruption of business systems.

As a specific implementation example of the embodiment of the present application, in an actual application scenario shown in FIG. 5, the application database is interconnected with the life insurance system database, the property insurance system database, and the financial fund database through a network. The application database is used to synchronize the basic information of each user in the life insurance system database, the property insurance system database and the wealth management fund database to the local, so that the application database is constructed as the above user attribute library. Thereafter, the application database utilizes the SPARK calculation engine to perform comprehensive analysis and processing on the basic information of each user that has been synchronized locally, and for any user, based on the life insurance insurance status of the user, the property insurance insurance status, and the purchase of the financial fund, etc., The database will fully evaluate the user's consumption behavior characteristics data. On the other hand, the short MMS push system inside the enterprise requests and acquires the consumption behavior characteristic data of the user A in the application database via the user unified view query interface provided by the application database, and the short multimedia message pushing system returns the characteristics according to the application database. Data, knowing that user A is accustomed to purchasing stock funds and hybrid funds, so the short-letter push system is selected to send push information about stock funds and hybrid funds, and the push information is sent to user A's mobile phone through SMS. . Therefore, based on the interface provided by the application database and the user characteristic data summarized and analyzed based on the application database, the business system such as the short multimedia messaging system realizes personalized customization of the user and service recommendation.

Corresponding to the method for acquiring the user feature data described in the foregoing embodiment, FIG. 6 is a structural block diagram of the device for acquiring user feature data provided by the embodiment of the present application. For the convenience of description, only the embodiment of the present application is shown. Related parts.

As shown in FIG. 6, in this embodiment, a device for acquiring user feature data includes:

The synchronization module 61 is configured to synchronize user basic data in each service database to a distributed file system to construct a user attribute library.

The summary module 62 is configured to analyze and summarize the synchronized user basic data by using a calculation engine to obtain feature data corresponding to each user.

The obtaining module 63 is configured to obtain a data access request sent by the service system based on the interface of the user attribute library.

The returning module 64 is configured to verify the validity of the data access request, and if the data access request is legal, return the feature data that matches the data access request to the service system.

Optionally, the returning module 74 includes:

The determining submodule is configured to determine whether the IP address carried by the data access request and the access key are both present in a configuration file of the user attribute database.

a first obtaining submodule, configured to acquire attribute information in the data access request, if the IP address and the access key carried by the data access request are both in a configuration file of the user attribute database, The attribute information includes the visitor signature and the data request parameters.

a first returning submodule, configured to: if the decrypted signature of the visitor is the same as the signature in the configuration file, and the data request parameter satisfies a preset condition, the data access request is legal, and the The feature data matched by the data access request is returned to the business system.

Optionally, the synchronization module 61 includes:

The first synchronization sub-module is configured to synchronize the user-based data in each service database to the distributed file system at a preset time interval based on the JOB mode to construct a user attribute library.

The device for acquiring user feature data further includes:

a storage module, configured to save feature data corresponding to each user to a temporary table of the user attribute library, and delete the user basic data in each service database to the user attribute database next time The feature data already stored in the temporary table.

Optionally, the returning module 64 includes:

The second obtaining submodule is configured to obtain an IP address of the data access request.

The third obtaining submodule is configured to obtain a total number of historical data access requests based on the IP address within a preset duration.

a cache submodule, configured to cache the data access request if the total number of historical data access requests based on the IP address is greater than a preset threshold within a preset duration, and to cache the The validity of the data access request is verified.

And a second returning submodule, configured to return the feature data that matches the data access request to the service system if the data access request is legal.

Optionally, the synchronization module 61 includes:

The second synchronization sub-module is configured to synchronize user basic data in a preset time period to a distributed file system in each service database to construct a user attribute library.

The preset time period is the first day of the current month to the day before the current time.

FIG. 7 is a schematic diagram of a server according to an embodiment of the present application. As shown in FIG. 7, the server 7 of this embodiment includes a processor 70 and a memory 71 in which computer readable instructions 72, such as user feature data, are executable to run on the processor 70. program. When the processor 70 executes the computer readable instructions 72, the steps in the foregoing method for acquiring the respective user feature data are implemented, for example, steps 101 to 104 shown in FIG. Alternatively, the processor 70, when executing the computer readable instructions 72, implements the functions of the various modules/units in the various apparatus embodiments described above, such as the functions of the modules 61-64 shown in FIG.

Illustratively, the computer readable instructions 72 may be partitioned into one or more modules/units that are stored in the memory 71 and executed by the processor 70, To complete this application. The one or more modules/units may be a series of computer readable instruction segments capable of performing a particular function for describing the execution of the computer readable instructions 72 in the server 7.

The server 7 can be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The server may include, but is not limited to, processor 70 and memory 71. It will be understood by those skilled in the art that FIG. 7 is only an example of the server 7, and does not constitute a limitation on the server 7, and may include more or less components than those illustrated, or combine some components, or different components, such as The server may also include an input and output device, a network access device, a bus, and the like.

The so-called processor 70 can be a central processing unit (Central Processing Unit, CPU), can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor or any conventional processor or the like.

The memory 71 may be an internal storage unit of the server 7, such as a hard disk or a memory of the server 7. The memory 71 may also be an external storage device of the server 7, such as a plug-in hard disk equipped with the server 7, a smart memory card (SMC), and a Secure Digital (SD) card. Flash card, etc. Further, the memory 71 may also include both an internal storage unit of the server 7 and an external storage device. The memory 71 is configured to store the computer readable instructions and other programs and data required by the server. The memory 71 can also be used to temporarily store data that has been output or is about to be output.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application, in essence or the contribution to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

The above embodiments are only used to explain the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still The technical solutions described in the embodiments are modified, or the equivalents of the technical features are replaced by the equivalents. The modifications and substitutions of the embodiments do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

A method for acquiring user feature data, comprising:

Synchronize user-based data in each business database to a distributed file system to build a user attribute library;

The calculation engine is used to analyze and summarize the synchronized user basic data, and the feature data corresponding to each user is obtained;

Obtaining a data access request sent by the service system based on the interface of the user attribute library;

Verifying the validity of the data access request, and if the data access request is legal, returning the feature data of the user that matches the data access request to the service system.
The method for acquiring user feature data according to claim 1, wherein the verifying the legality of the data access request, and if the access request is legal, the user matching the data access request The feature data is returned to the business system, including:

Determining whether the IP address carried by the data access request and the access key are present in a configuration file of the user attribute database;

And if the IP address and the access key carried in the data access request are both in the configuration file of the user attribute database, acquiring attribute information in the data access request, where the attribute information includes a data request parameter and Encrypted visitor signature;

Decrypting the visitor signature, determining whether the decrypted signature of the visitor is the same as the signature in the configuration file, and determining whether the data request parameter satisfies a preset condition;

If the decrypted signature of the visitor is the same as the signature in the configuration file, and the data request parameter satisfies a preset condition, determining that the data access request is legal and matching the data access request The feature data is returned to the business system.
The method for acquiring user feature data according to claim 1, wherein the synchronizing user basic data in each service database to a distributed file system to construct a user attribute database comprises:

Based on the JOB method, the user basic data in each service database is synchronized to the distributed file system at a preset time interval to construct a user attribute library;

After the step of analyzing and summarizing the synchronized user basic data by using the calculation engine to obtain the feature data corresponding to each user, the method further includes:

Saving feature data corresponding to each user to a temporary table of the user attribute library, and deleting the stored in the temporary table before synchronizing the user basic data in each service database to the user attribute library next time The feature data.
The method for acquiring user feature data according to claim 1, wherein the verifying the legality of the data access request, and if the data access request is legal, matching the data access request Returning the feature data to the business system, including:

Obtaining an IP address of the data access request;

Obtaining a total number of historical data access requests based on the IP address within a preset duration;

And if the total number of historical data access requests based on the IP address in the preset duration is greater than a preset threshold, the data access request is cached, and the validity of the cached data access request is determined after a preset delay authenticating;

If the data access request is legal, the feature data matching the data access request is returned to the service system.
The method for acquiring user feature data according to claim 1, wherein the synchronizing user basic data in each service database to a distributed file system to construct a user attribute database comprises:

Synchronizing user basic data in a preset time period to a distributed file system in each business database to construct a user attribute database;

The preset time period is the first day of the current month to the day before the current time.
A device for acquiring user feature data, comprising:

a synchronization module, configured to synchronize user basic data in each business database to a distributed file system to construct a user attribute library;

a summary module, configured to analyze and summarize the synchronized user basic data by using a calculation engine, to obtain feature data corresponding to each user;

An obtaining module, configured to acquire, according to an interface of the user attribute library, a data access request sent by a service system;

And returning a module, configured to verify validity of the data access request, and if the data access request is legal, return the feature data that matches the data access request to the service system.
The device for acquiring user feature data according to claim 6, wherein the returning module comprises:

a determining sub-module, configured to determine whether an IP address carried by the data access request and an access key are present in a configuration file of the user attribute database;

a first obtaining submodule, configured to acquire attribute information in the data access request, if the IP address and the access key carried by the data access request are both in a configuration file of the user attribute database, The attribute information includes a visitor signature and a data request parameter;

a first returning submodule, configured to: if the decrypted signature of the visitor is the same as the signature in the configuration file, and the data request parameter satisfies a preset condition, the data access request is legal, and the The feature data matched by the data access request is returned to the business system.
The device for acquiring user feature data according to claim 6, wherein the synchronization module comprises:

a first synchronization submodule, configured to synchronize user basic data in each service database to a distributed file system at a preset time interval based on a JOB manner to construct a user attribute database;

The device for acquiring user feature data further includes:

a storage module, configured to save feature data corresponding to each user to a temporary table of the user attribute library, and delete the user basic data in each service database to the user attribute database next time The feature data already stored in the temporary table.
The device for acquiring user feature data according to claim 6, wherein the returning module comprises:

a second obtaining submodule, configured to obtain an IP address of the data access request;

a third obtaining submodule, configured to obtain a total number of historical data access requests based on the IP address within a preset duration;

a cache submodule, configured to cache the data access request if the total number of historical data access requests based on the IP address is greater than a preset threshold within a preset duration, and to cache the Verify the legality of the data access request;

And a second returning submodule, configured to return the feature data that matches the data access request to the service system if the data access request is legal.
The device for acquiring user feature data according to any one of claims 6 to 3, wherein the synchronization module comprises:

The second synchronization sub-module is configured to synchronize user basic data in a preset time period to a distributed file system in each service database to construct a user attribute library.

The preset time period is the first day of the current month to the day before the current time.
A server, comprising: a memory and a processor, wherein the memory stores computer readable instructions executable on the processor, the processor executing the computer readable instructions to:

Synchronize user-based data in each business database to a distributed file system to build a user attribute library;

The calculation engine is used to analyze and summarize the synchronized user basic data, and the feature data corresponding to each user is obtained;

Obtaining a data access request sent by the service system based on the interface of the user attribute library;

Verifying the validity of the data access request, and if the data access request is legal, returning the feature data of the user that matches the data access request to the service system.
The server according to claim 11, wherein said verifying validity of said data access request, and if said access request is legal, returning feature data of a user matching said data access request to The business system includes:

Determining whether the IP address carried by the data access request and the access key are present in a configuration file of the user attribute database;

And if the IP address and the access key carried in the data access request are both in the configuration file of the user attribute database, acquiring attribute information in the data access request, where the attribute information includes a data request parameter and Encrypted visitor signature;

Decrypting the visitor signature, determining whether the decrypted signature of the visitor is the same as the signature in the configuration file, and determining whether the data request parameter satisfies a preset condition;

If the decrypted signature of the visitor is the same as the signature in the configuration file, and the data request parameter satisfies a preset condition, determining that the data access request is legal and matching the data access request The feature data is returned to the business system.
The server according to claim 11, wherein the synchronizing user basic data in each service database to a distributed file system to construct a user attribute database comprises:

Based on the JOB method, the user basic data in each service database is synchronized to the distributed file system at a preset time interval to construct a user attribute library;

After the step of analyzing and summarizing the synchronized user basic data by using the calculation engine to obtain the feature data corresponding to each user, the method further includes:

Saving feature data corresponding to each user to a temporary table of the user attribute library, and deleting the stored in the temporary table before synchronizing the user basic data in each service database to the user attribute library next time The feature data.
The server according to claim 11, wherein said verifying validity of said data access request, and returning said feature data matching said data access request if said data access request is legal To the business system, including:

Obtaining an IP address of the data access request;

Obtaining a total number of historical data access requests based on the IP address within a preset duration;

And if the total number of historical data access requests based on the IP address in the preset duration is greater than a preset threshold, the data access request is cached, and the validity of the cached data access request is determined after a preset delay authenticating;

If the data access request is legal, the feature data matching the data access request is returned to the service system.
The server according to claim 11, wherein the synchronizing user basic data in each service database to a distributed file system to construct a user attribute database comprises:

Synchronizing user basic data in a preset time period to a distributed file system in each business database to construct a user attribute database;

The preset time period is the first day of the current month to the day before the current time.
A computer readable storage medium storing computer readable instructions, wherein the computer readable instructions, when executed by at least one processor, implement the following steps:

Synchronize user-based data in each business database to a distributed file system to build a user attribute library;

The calculation engine is used to analyze and summarize the synchronized user basic data, and the feature data corresponding to each user is obtained;

Obtaining a data access request sent by the service system based on the interface of the user attribute library;

Verifying the validity of the data access request, and if the data access request is legal, returning the feature data of the user that matches the data access request to the service system.
The computer readable storage medium according to claim 16, wherein said verifying validity of said data access request, and if said access request is legitimate, matching said user of said data access request Feature data is returned to the business system, including:

Determining whether the IP address carried by the data access request and the access key are present in a configuration file of the user attribute database;

And if the IP address and the access key carried in the data access request are both in the configuration file of the user attribute database, acquiring attribute information in the data access request, where the attribute information includes a data request parameter and Encrypted visitor signature;

Decrypting the visitor signature, determining whether the decrypted signature of the visitor is the same as the signature in the configuration file, and determining whether the data request parameter satisfies a preset condition;

If the decrypted signature of the visitor is the same as the signature in the configuration file, and the data request parameter satisfies a preset condition, determining that the data access request is legal and matching the data access request The feature data is returned to the business system.
The computer readable storage medium according to claim 16, wherein the synchronizing user basic data in each service database to a distributed file system to construct a user attribute library comprises:

Based on the JOB method, the user basic data in each service database is synchronized to the distributed file system at a preset time interval to construct a user attribute library;

After the step of analyzing and summarizing the synchronized user basic data by using the calculation engine to obtain the feature data corresponding to each user, the method further includes:

Saving feature data corresponding to each user to a temporary table of the user attribute library, and deleting the stored in the temporary table before synchronizing the user basic data in each service database to the user attribute library next time The feature data.
The computer readable storage medium according to claim 16, wherein said verifying the legality of said data access request, and if said data access request is legitimate, matching said data access request Returning the feature data to the business system, including:

Obtaining an IP address of the data access request;

Obtaining a total number of historical data access requests based on the IP address within a preset duration;

And if the total number of historical data access requests based on the IP address in the preset duration is greater than a preset threshold, the data access request is cached, and the validity of the cached data access request is determined after a preset delay authenticating;

If the data access request is legal, the feature data matching the data access request is returned to the service system.
The computer readable storage medium according to claim 16, wherein the synchronizing user basic data in each service database to a distributed file system to construct a user attribute library comprises:

Synchronizing user basic data in a preset time period to a distributed file system in each business database to construct a user attribute database;

The preset time period is the first day of the current month to the day before the current time.