CN113127509B - Adaptation method and device for SQL execution engine in PaaS platform - Google Patents

Abstract

The invention discloses a method and a device for adapting SQL execution engines in a PaaS platform, wherein the method comprises the following steps: receiving SQL sentences input by a user; analyzing the SQL sentence to obtain SQL metadata information; extracting key information in SQL metadata information from a Hadoop platform to form metadata; inputting SQL metadata information and table metadata into an SQL selector for processing to obtain an SQL execution engine selection result; and providing the SQL sentence to the SQL execution engine to select the SQL execution engine corresponding to the result for execution. By the method, the automatic adaptation of the SQL execution engine is realized, and when a user accesses the PaaS platform data by using SQL, the optimal SQL engine is automatically identified and adapted according to the SQL submitted by the user, so that uncertainty of artificial experience is avoided, and the fact that the most suitable SQL execution engine is used for executing corresponding SQL sentences in the PaaS platform can be ensured.

Description

Adaptation method and device for SQL execution engine in PaaS platform

technical field

The present invention relates to the field of computer technology, in particular to an adaptation method and device for an SQL execution engine in a PaaS platform.

Background technique

The bottom layer of open data services of PaaS (Platform as a Service) platform is mostly based on Hadoop, and its general execution process is that users access Hadoop data through PaaS platform data services, and execute calculations and store data in Hadoop. completed. There are many SQL processing engines on Hadoop, and the mainstream ones include hive, sparkSQL, flink, impala and so on. Among them, each SQL processing engine is suitable for different scenarios, and the optimal effect can only be exerted in the scenarios in which it is good at, and the operating efficiency and stability will be greatly reduced or even fail in unsuitable scenarios. If an inappropriate SQL engine is used to run programs on the Hadoop platform, it is prone to instability or failure in operating efficiency, which increases the difficulty of business operation and maintenance. Moreover, the Hadoop platform generally has a lot of programs running every day, and a large number of programs with low operating efficiency will cause an obvious waste of computing resources on the entire platform.

In the prior art, when developing and submitting SQL jobs, technicians mostly rely on the experience of the technicians or determine the SQL execution engine based on the results of repeated test runs.

However, on the one hand, judging and selecting an execution engine based on manual experience relies heavily on the personal ability of technicians, and there is no guarantee that the selected SQL engine is the most suitable for the business scenario. On the other hand, it is inefficient to select the optimal engine by selecting different SQL engines for efficiency testing, and repeatedly running the same program on each SQL engine to test the operating efficiency will cause a waste of platform computing resources. Therefore, there is an urgent need for an efficient SQL execution engine adaptation method.

Contents of the invention

In view of the above problems, the present invention is proposed to provide a method and device for adapting a SQL execution engine in a PaaS platform that overcomes the above problems or at least partially solves the above problems.

According to one aspect of the present invention, a method for adapting an SQL execution engine in a PaaS platform is provided, including:

Receive the SQL statement entered by the user;

Analyze the SQL statement to obtain SQL metadata information;

Extract the key information in the SQL metadata information from the Hadoop platform to form table metadata;

Input the SQL metadata information and table metadata into the SQL selector for processing, and obtain the selection result of the SQL execution engine;

Provide the SQL statement to the SQL execution engine corresponding to the SQL execution engine selection result for execution.

Optionally, the method further includes:

Monitor the execution process of the SQL execution engine corresponding to the selection result of the SQL execution engine to obtain operating performance indicators;

Store running performance indicators, SQL metadata information, and table metadata in the historical indicator library.

Optionally, the SQL selector comprises at least one decision unit with a predictive model;

Then, the SQL metadata information and table metadata are input into the SQL selector for processing, and the selection results of the SQL execution engine are obtained, including:

Integrating SQL metadata information and table metadata into a decision request, providing the decision request to at least one decision-making unit, so that at least one decision-making unit can make a decision and output the decision result of the SQL execution engine;

According to the way of voting, the SQL execution engine selection result is selected from at least one SQL execution engine decision result.

Optionally, a self-learning method is used to train the predictive model of the SQL selector according to the data in the historical index database, or to train the predictive model of the decision-making unit included in the SQL selector according to the data in the historical index database.

Optionally, the SQL metadata information includes at least one of the following information: table, field type, field length, result field, sorting information, association information, operator type, and the number of times inserted into the temporary table;

The key information in the SQL metadata information includes at least one of the following information: table information in the SQL metadata information, field information in the SQL metadata information, and storage information in the SQL metadata information. .

According to another aspect of the present invention, an adaptation device for an SQL execution engine in a PaaS platform is provided, including:

The SQL receiver is used to receive the SQL statement input by the user;

The SQL parser is used to parse the SQL statement to obtain SQL metadata information;

The table metadata acquirer is used to extract the key information in the SQL metadata information from the Hadoop platform to form table metadata;

SQL selector, which receives and processes SQL metadata information and table metadata, and outputs the selection result of SQL execution engine;

The SQL executor provides the SQL statement to the SQL execution engine corresponding to the selection result of the SQL execution engine for execution.

Optionally, the device further includes:

The detection module is used to monitor the execution process of the SQL execution engine corresponding to the SQL execution engine selection result, and obtain the operation performance index;

The storage module is used to store running performance indicators, SQL metadata information, and table metadata in the historical indicator library.

Optionally, the SQL selector includes at least one decision-making unit with a predictive model, a data receiving unit, and a decision-making output device;

The data receiving unit is used to receive SQL metadata and table metadata, integrate SQL metadata information and table metadata into a decision request, and provide the decision request to at least one decision unit for at least one decision unit to make a decision and output SQL Execution engine decision results;

The decision output device is configured to select a SQL execution engine selection result from at least one SQL execution engine decision result in a voting manner.

Optionally, the device further includes:

The self-learning module is used to train the predictive model of the SQL selector according to the data in the historical indicator database by self-learning, or to train the predictive model of the decision-making unit contained in the SQL selector according to the data in the historical indicator database train.

Optionally, the SQL metadata information includes at least one of the following information: table, field type, field length, result field, sorting information, association information, operator type, and the number of times inserted into the temporary table;

The key information in the SQL metadata information includes at least one of the following information: table information in the SQL metadata information, field information in the SQL metadata information, and storage information in the SQL metadata information. .

According to yet another aspect of the present invention, a computing device is provided, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete mutual communication through the communication bus communication;

The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the adaptation method of the SQL execution engine in the PaaS platform.

According to another aspect of the present invention, a computer storage medium is provided, and at least one executable instruction is stored in the storage medium, and the executable instruction causes the processor to execute the adaptation method of the SQL execution engine in the PaaS platform as described above. corresponding operation.

According to the adaptation method and device of the SQL execution engine in the PaaS platform of the present invention, the method includes: receiving the SQL statement input by the user; parsing the SQL statement to obtain SQL metadata information; converting the key information in the SQL metadata information from Extracted by the Hadoop platform to form table metadata; input the SQL metadata information and table metadata into the SQL selector for processing, and obtain the selection result of the SQL execution engine; provide the SQL statement to the SQL execution engine corresponding to the selection result of the SQL execution engine to execute. Through the above method, the present invention realizes the automatic adaptation of the SQL execution engine. When the user uses SQL to access the PaaS platform data, the optimal SQL engine is automatically identified and adapted according to the SQL submitted by the user, avoiding the uncertainty of human experience, and being able to Ensure that the most appropriate SQL execution engine is used to execute the corresponding SQL statement in the PaaS platform.

The above description is only an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable , the specific embodiments of the present invention are enumerated below.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiment. The drawings are only for the purpose of illustrating a preferred embodiment and are not to be considered as limiting the invention. Also throughout the drawings, the same reference numerals are used to designate the same components. In the attached picture:

Fig. 1 shows the flowchart of the adaptation method of the SQL execution engine in the PaaS platform provided by the embodiment of the present invention;

FIG. 2 shows a flowchart of a method for adapting an SQL execution engine in a PaaS platform provided by another embodiment of the present invention;

FIG. 3 shows a schematic structural diagram of an adaptation device for an SQL execution engine in a PaaS platform provided by an embodiment of the present invention;

Fig. 4 shows the schematic flow diagram in the SQL selector in the embodiment of the present invention;

FIG. 5 shows a schematic structural diagram of an adaptation device for an SQL execution engine in a PaaS platform in an embodiment of the present invention;

Fig. 6 shows a schematic structural diagram of a computing device provided by an embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

Fig. 1 shows the flowchart of the adaptation method of the SQL execution engine in the PaaS platform provided by the embodiment of the present invention, in the present embodiment, the PaaS platform is implemented based on the Hadoop platform, as shown in Fig. 1, the method includes the following steps:

Step S101, receiving an SQL statement input by a user.

When platform users use SQL to access PaaS platform data, they need to submit SQL statements. In practical applications, platform users may include program development tasks, business analysts, data modelers, and so on.

Step S102, analyzing the SQL statement to obtain SQL metadata information.

After receiving the SQL statement input by the user, the SQL statement is parsed, and the SQL metadata is parsed out.

Step S103, extract key information in the SQL metadata information from the Hadoop platform to form table metadata.

Then, the SQL metadata information is matched with each table metadata of the Hadoop platform, and the key information in the SQL metadata information is extracted to form the Hadoop table metadata.

In step S104, the SQL metadata information and the table metadata are input into the SQL selector for processing, and the selection result of the SQL execution engine is obtained.

Input the SQL metadata information and table metadata into the SQL selector, and the SQL selector makes a selection judgment based on these data, selects a SQL execution engine that is most suitable for processing SQL statements, and obtains the selection result of the SQL execution engine.

For example, the SQL selector is a predictive model, which predicts and judges SQL metadata information and table metadata through the predictive model, and outputs the selection result of the SQL execution engine.

Step S105, providing the SQL statement to the SQL execution engine corresponding to the SQL execution engine selection result for execution.

Finally, the SQL statement is provided to the selected most appropriate SQL execution engine for execution.

According to the adaptation method of the SQL execution engine in the PaaS platform provided by the embodiment of the present invention, by analyzing the SQL statement input by the user, the corresponding SQL metadata and Hadoop table metadata are extracted, and the selection is made according to these data, and the The most suitable SQL execution engine. This method provides a method of automatically adapting the SQL execution engine. When the user uses SQL to access the PaaS platform data, the optimal SQL engine is automatically identified and adapted according to the SQL submitted by the user, avoiding the uncertainty of manual experience. Ensure that the most appropriate SQL execution engine is used in the PaaS platform to execute the corresponding SQL statement, thereby avoiding the problem of unstable operation efficiency or operation failure, reducing the difficulty of business operation and maintenance, and avoiding the waste of computing resources.

Fig. 2 shows the flowchart of the adaptation method of the SQL execution engine in the PaaS platform provided by another embodiment of the present invention, in the present embodiment, the PaaS platform is implemented based on the Hadoop platform, as shown in Fig. 2, the method includes the following steps:

Step S201, receiving an SQL statement input by a user.

When platform users use SQL to access PaaS platform data, they need to submit SQL statements. In practical applications, platform users may include program development tasks, business analysts, data modelers, and so on.

Step S202, analyzing the SQL statement to obtain SQL metadata information.

Parse the SQL statement, and parse out the SQL metadata information. Optionally, the SQL metadata information includes at least one of the following information: table, field type, field length, result field, sorting information, association information, operator type , Insert the number of temporary tables.

An example of the detailed metadata information list that needs to be obtained for SQL statement parsing is as follows, where Table 1 is an example of the metadata information of the input table entered into SQL. Table 2 is an example of associated metadata information entered into SQL. Table 3 is an example of key operator metadata information entered into SQL.

Table I

Table II

Table three

In step S203, the key information in the SQL metadata information is extracted from the Hadoop platform to form table metadata.

The SQL metadata information is matched with each table metadata of the Hadoop platform, and the key information in the SQL metadata information is extracted to form the Hadoop table metadata.

Optionally, the key information in the SQL metadata information includes at least one of the following information: table information in the SQL metadata information, field information in the SQL metadata information, and storage information in the SQL metadata information. The formed table metadata includes at least one of the following: table size, storage format, field type, and length.

Table 4 is an example of the field metadata information in the hadoop platform of the input table for inputting SQL, and Table 5 is an example of the storage metadata information of the input table for inputting SQL in the hadoop platform.

Table four

Table five

Step S204, integrating SQL metadata information and table metadata into a decision request, and providing the decision request to at least one decision unit, so that at least one decision unit can make a decision and output a decision result of the SQL execution engine.

In the embodiment of the present invention, the SQL selector includes at least one decision-making unit with a predictive model, the total number of decision-making units is an odd number, and each decision-making unit is independent of each other. Integrate SQL metadata information and table metadata into decision-making requests, and provide decision-making requests to each decision-making unit. Each decision-making unit makes a decision and judgment on the received decision-making request through its own prediction model, and each decision-making unit will output a SQL execution Engine decision request.

Among them, each decision-making unit conducts the training of the prediction model according to the data in the historical index library through self-learning. The historical indicator library includes historical running SQL metadata information, historical table metadata, and historical SQL running performance indicators. Specifically, the prediction model of the decision-making unit is trained according to a preset period through an independent learning model.

Step S205, selecting a SQL execution engine selection result from at least one SQL execution engine decision result in a voting manner.

Then, according to the voting method, select the final result from the SQL execution engine decision results output by each decision-making unit, that is, the SQL execution engine selection result. The SQL execution engine selection result is based on the SQL metadata and Hadoop table The most suitable execution engine for running SQL statements is selected by the metadata.

Step S206, providing the SQL statement to the SQL execution engine corresponding to the SQL execution engine selection result for execution.

Provide the SQL statement to the selected most appropriate SQL execution for execution.

Step S207, monitor the execution process of the SQL execution engine corresponding to the SQL execution engine selection result, and obtain the operation performance index.

Monitor the running process of SQL statements on the selected optimal SQL execution engine to obtain running performance indicators, which specifically include running time, CPU consumption, memory consumption, etc.

Step S208, storing the operation performance index, SQL metadata information, and table metadata in the historical index database.

Merge and record running performance indicators, SQL metadata information, and table metadata in the historical indicator library. According to the above content, the data in the historical indicator library can be used for training the prediction model of the decision-making unit.

Table 6 is an example of performance indicators running on the SQL execution engine.

Table six

According to the adaptation method of the SQL execution engine in the PaaS platform provided by the embodiment of the present invention, by analyzing the SQL statement input by the user, the corresponding SQL metadata and Hadoop table metadata are extracted, and these data are processed by multiple decision-making units. Make a decision, and select the most suitable SQL execution engine based on the SQL decision result. This method provides a method of automatically adapting the SQL execution engine. When the user uses SQL to access the data of the PaaS platform, the optimal SQL engine is automatically identified and adapted according to the SQL submitted by the user, avoiding the uncertainty of manual experience. Ensure that the most appropriate SQL execution engine is used to execute the corresponding SQL statements in the PaaS platform, thereby avoiding the problem of unstable operation efficiency or failure, reducing the difficulty of business operation and maintenance, and avoiding the waste of platform computing resources. At the same time, the decision-making unit has the ability of self-learning and self-optimization, and performs self-optimization according to the data in the historical index database, so as to improve the accuracy of selection and avoid the workload of model tuning.

FIG. 3 shows a schematic structural diagram of an adaptation device for an SQL execution engine in a PaaS platform provided by an embodiment of the present invention. As shown in FIG. 3 , the device includes:

SQL receiver 31, for receiving the SQL statement input by the user;

The SQL parser 32 is configured to parse the SQL statement to obtain SQL metadata information.

The device first receives the SQL statement submitted by the user, and submits it to the SQL parser for parsing the SQL. The process of parsing SQL is to extract the required metadata information from the SQL statement submitted by the user, including the table, field, associated type, associated field, sorting type, sorting field, number of times inserted into the temporary table, etc. to be queried, and obtained as table metadata Input data for selectors and SQL selectors.

The table metadata acquirer 33 is configured to extract key information in the SQL metadata information from the Hadoop platform to form table metadata.

The table metadata acquirer receives SQL and extracts the key information in the SQL metadata information from the Hadoop platform to form table metadata, which is used as the input data of the SQL selector.

The SQL selector 34 is configured to receive and process SQL metadata information and table metadata, and output a SQL execution engine selection result.

The metadata parsed by the SQL parser and the table metadata acquired by the table metadata acquirer are used as input data and submitted to the SQL engine selector to select the optimal SQL engine.

The SQL executor 35 provides the SQL statement to the SQL execution engine corresponding to the SQL execution engine selection result for execution.

In an optional manner, the SQL selector 34 includes a decision maker, a data receiving unit, and a decision outputter; the decision maker includes at least one decision unit with a predictive model

The data receiving unit is used to receive SQL metadata and table metadata, integrate SQL metadata information and table metadata into a decision request, and provide the decision request to at least one decision unit for at least one decision unit to make a decision and output SQL Execution engine decision result.

The decision output device is configured to select a SQL execution engine selection result from at least one SQL execution engine decision result in a voting manner.

The data receiving unit is responsible for receiving the submitted SQL metadata and table metadata of the hadoop platform, and integrating these two data into a complete decision request and submitting it to the decision maker. The decision maker is composed of a group of odd number of independent decision-making units, which respectively make decisions and judgments on the decision requests submitted by the data receiving unit, and output the optimal SQL execution engine to the decision output unit. After receiving the output results of each decision-making unit, the decision-making output device selects the final result by voting, that is, the optimal SQL execution engine.

Fig. 4 shows the schematic flow chart in the SQL selector in the embodiment of the present invention, as shown in Fig. 4, first receive SQL metadata and Hadoop table metadata, then the SQL metadata that receives and Hadoop table metadata are integrated into The decision-making request is sent to decision-making unit 1, decision-making unit 2, decision-making unit 3, decision-making unit 4..., the output results of each decision-making unit are provided to the decision output device, and the decision output device outputs the final selection result of the optimal SQL execution engine .

In an optional manner, the above-mentioned device also includes: a detection module, configured to monitor the execution process of the SQL execution engine corresponding to the SQL execution engine selection result, to obtain an operating performance index; a storage module, configured to store the operating performance index , SQL metadata information, and table metadata are stored in the historical indicator database.

Fig. 5 shows a schematic structural diagram of an adaptation device of an SQL execution engine in a PaaS platform in an embodiment of the present invention. As shown in Fig. 5, the device includes: a SQL receiver, a SQL parser, a Hadoop table metadata acquirer, a SQL Execution engine selector, SQL submission executor, SQL selection self-learning device, historical SQL operation indicator library.

Among them, the SQL receiver receives the SQL statement submitted by the user, submits it to the SQL parser to parse the SQL, and extracts the SQL metadata information as the input data of the Hadoop table metadata acquirer and the SQL execution engine selector. The Hadoop table metadata acquirer matches the SQL metadata information with each table metadata in the Hadoop platform, and obtains the key information in the SQL metadata to form the Hadoop table metadata, which is also used as the input data of the SQL execution engine selector.

Then, the metadata parsed by the SQL parser and the table metadata acquired by the Hadoop platform table metadata acquirer are used as input data, and submitted to the SQL execution engine selector to select the optimal SQL engine. The SQL execution engine selector further includes a data receiving unit, a decision maker, and a decision outputter. The specific implementation process is as follows:

The data receiving unit is responsible for receiving the submitted SQL metadata and table metadata of the hadoop platform, and integrating these two data into a complete decision request and submitting it to the decision maker.

The decision maker is composed of a group of odd number of independent decision-making units, which respectively make decision-making judgments on the decision-making requests submitted by the data receiving unit, and output the decision-making results of the optimal SQL execution engine to the decision-making output unit.

After receiving the output results of each decision-making unit, the decision-making output device selects the final result by voting, that is, the optimal SQL execution engine.

Among them, the SQL engine selection self-learner provides decision-making support for each decision-making unit in the SQL execution engine selector, and is the brain of the decision-making unit. Each decision-making unit has an independent set of algorithm models, which automatically learn every day to continuously accumulate and improve decision-making accuracy.

Decision-making unit self-learning specifically refers to obtaining the SQL metadata, table metadata, and historical SQL operation performance indicators of each historical operation from the historical SQL operation index database, and performing daily decision-making calculation training through an independent learning model.

Finally, the SQL submission executor submits the SQL statement to the selected SQL execution engine, such as hive, spark, flink, impala, and so on.

An embodiment of the present invention provides a non-volatile computer storage medium, the computer storage medium stores at least one executable instruction, and the computer executable instruction can execute the SQL execution engine in the PaaS platform in any of the above method embodiments adaptation method.

Specifically, the executable instruction can be used to make the processor perform the following operations:

Receive the SQL statement entered by the user;

Analyze the SQL statement to obtain SQL metadata information;

Extract the key information in the SQL metadata information from the Hadoop platform to form table metadata;

Input the SQL metadata information and table metadata into the SQL selector for processing, and obtain the selection result of the SQL execution engine;

Provide the SQL statement to the SQL execution engine corresponding to the SQL execution engine selection result for execution.

In an optional manner, the executable instructions cause the processor to perform the following operations:

Monitor the execution process of the SQL execution engine corresponding to the selection result of the SQL execution engine to obtain operating performance indicators;

Store running performance indicators, SQL metadata information, and table metadata in the historical indicator library.

In an optional manner, the SQL selector includes at least one decision-making unit with a predictive model, and the executable instructions cause the processor to perform the following operations:

Integrating SQL metadata information and table metadata into a decision request, providing the decision request to at least one decision-making unit, so that at least one decision-making unit can make a decision and output the decision result of the SQL execution engine;

According to the way of voting, the SQL execution engine selection result is selected from at least one SQL execution engine decision result.

In an optional manner, the executable instructions cause the processor to perform the following operations:

In the way of self-learning, the prediction model of the SQL selector is trained according to the data in the historical index database, or the prediction model of the decision-making unit contained in the SQL selector is trained according to the data in the historical index database.

In an optional manner, the SQL metadata information includes at least one of the following information: table, field type, field length, result field, sorting information, association information, operator type, and the number of times inserted into the temporary table;

The key information in the SQL metadata information includes at least one of the following information: table information in the SQL metadata information, field information in the SQL metadata information, and storage information in the SQL metadata information.

FIG. 6 shows a schematic structural diagram of an embodiment of a computing device according to the present invention, and the specific embodiment of the present invention does not limit the specific implementation of the computing device.

As shown in FIG. 6 , the computing device may include: a processor (processor) 602 , a communication interface (Communications Interface) 604 , a memory (memory) 606 , and a communication bus 608 .

Wherein: the processor 602 , the communication interface 604 , and the memory 606 communicate with each other through the communication bus 608 . The communication interface 604 is used to communicate with network elements of other devices such as clients or other servers. The processor 602 is configured to execute the program 610, specifically, may execute the relevant steps in the embodiment of the method for adapting the SQL execution engine in the PaaS platform of the computing device.

Specifically, the program 610 may include program codes including computer operation instructions.

The processor 602 may be a central processing unit CPU, or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computing device may be of the same type, such as one or more CPUs, or may be different types of processors, such as one or more CPUs and one or more ASICs.

The memory 606 is used for storing the program 610 . The memory 606 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

The program 610 can specifically be used to make the processor 602 perform the following operations:

Receive the SQL statement entered by the user;

Analyze the SQL statement to obtain SQL metadata information;

Extract the key information in the SQL metadata information from the Hadoop platform to form table metadata;

Input the SQL metadata information and table metadata into the SQL selector for processing, and obtain the selection result of the SQL execution engine;

Provide the SQL statement to the SQL execution engine corresponding to the SQL execution engine selection result for execution.

In an optional manner, the program 610 enables the processor 602 to perform the following operations:

Monitor the execution process of the SQL execution engine corresponding to the selection result of the SQL execution engine to obtain operating performance indicators;

Store running performance indicators, SQL metadata information, and table metadata in the historical indicator library.

In an optional manner, the SQL selector includes at least one decision-making unit with a predictive model; the program 610 enables the processor 602 to perform the following operations:

Integrating SQL metadata information and table metadata into a decision request, providing the decision request to at least one decision-making unit, so that at least one decision-making unit can make a decision and output the decision result of the SQL execution engine;

According to the way of voting, the SQL execution engine selection result is selected from at least one SQL execution engine decision result.

In an optional manner, the program 610 enables the processor 602 to perform the following operations: use self-learning to train the predictive model of the SQL selector according to the data in the historical index library, or use the historical index The data in the database trains the predictive model of the decision-making unit contained in the SQL selector.

In an optional manner, the SQL metadata information includes at least one of the following information: table, field type, field length, result field, sorting information, association information, operator type, and the number of times inserted into the temporary table;

The key information in the SQL metadata information includes at least one of the following information: table information in the SQL metadata information, field information in the SQL metadata information, and storage information in the SQL metadata information.

The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other device. Various generic systems can also be used with the teachings based on this. The structure required to construct such a system is apparent from the above description. Furthermore, embodiments of the present invention are not directed to any particular programming language. It should be understood that various programming languages can be used to implement the content of the present invention described herein, and the above description of specific languages is for disclosing the best mode of the present invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, in order to streamline the present disclosure and to facilitate an understanding of one or more of the various inventive aspects, various features of the embodiments of the invention are sometimes grouped together into a single implementation examples, figures, or descriptions thereof. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art can understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. Modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore may be divided into a plurality of sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method or method so disclosed may be used in any combination, except that at least some of such features and/or processes or units are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that although some embodiments herein include some features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention. And form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components according to the embodiments of the present invention. The present invention can also be implemented as an apparatus or an apparatus program (for example, a computer program and a computer program product) for performing a part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet site, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the execution order.

Claims (10)

1. A method for adapting an SQL execution engine in a PaaS platform, comprising: Receive the SQL statement entered by the user; Analyzing the SQL statement to obtain SQL metadata information; The SQL metadata information is matched with each table metadata of the Hadoop platform, and the key information in the SQL metadata information is extracted to form table metadata, and the table metadata includes at least one of the following: the size of the table, the storage format, field type, length; The SQL metadata information and the table metadata are input into the SQL selector for processing, and the selection result of the SQL execution engine is obtained; Wherein, the predictive model of the SQL selector is obtained according to the data training in the historical index library, the historical index library includes operating performance indicators, SQL metadata information and table metadata, and the operating performance indicators include at least one of the following : Running time, CPU consumption, memory consumption; The SQL statement is provided to the SQL execution engine to select the SQL execution engine corresponding to the result for execution. 2. The method of claim 1, wherein the SQL selector comprises at least one decision unit with a predictive model; Then, the SQL metadata information and table metadata are input into the SQL selector for processing, and the selection result of the SQL execution engine is obtained, which specifically includes: Integrating the SQL metadata information and table metadata into a decision request, and providing the decision request to the at least one decision-making unit for the at least one decision-making unit to make a decision and output a SQL execution engine decision result; In a voting manner, a SQL execution engine selection result is selected from the at least one SQL execution engine decision result. 3. The method of claim 1, wherein the method further comprises: Monitoring the execution process of the SQL execution engine corresponding to the SQL execution engine selection result to obtain an operating performance index; The operation performance index, the SQL metadata information, and the table metadata are stored in a historical index database. 4. The method according to any one of claims 1-3, wherein, adopting a self-learning method, the prediction model of the SQL selector is trained according to the data in the historical index library, or according to the data in the historical index library The data of the SQL selector is used to train the predictive model of at least one decision-making unit included in the SQL selector. 5. The method according to claim 1, wherein the SQL metadata information includes at least one of the following information: table, field type, field length, result field, sorting information, association information, operator type, insert The number of temporary tables; The key information in the SQL metadata information includes at least one of the following information: table information in the SQL metadata information, field information in the SQL metadata information, and storage information in the SQL metadata information. 6. An adaptation device for an SQL execution engine in a PaaS platform, comprising: The SQL receiver is used to receive the SQL statement input by the user; An SQL parser, configured to parse the SQL statement to obtain SQL metadata information; The table metadata acquirer is used to match the SQL metadata information to each table metadata of the Hadoop platform, extract key information in the SQL metadata information, and form table metadata, and the table metadata includes at least one of the following : table size, storage format, field type, length; The SQL selector receives and processes the SQL metadata information and table metadata, and outputs the selection result of the SQL execution engine; wherein, the prediction model of the SQL selector is obtained according to data training in the historical index database, and the historical index The library contains running performance indicators, SQL metadata information and table metadata, and the running performance indicators include at least one of the following: running time, CPU consumption, and memory consumption; The SQL executor provides the SQL statement to the SQL execution engine corresponding to the SQL execution engine selection result for execution. 7. The device according to claim 6, wherein the SQL selector comprises at least one decision unit with a predictive model, a data receiving unit, and a decision output device; The data receiving unit is configured to receive SQL metadata and table metadata, integrate the SQL metadata information and table metadata into a decision request, and provide the decision request to the at least one decision unit for all The at least one decision-making unit makes a decision and outputs a SQL execution engine decision result; The decision output unit is configured to select a SQL execution engine selection result from the at least one SQL execution engine decision result in a voting manner. 8. The apparatus of claim 6, wherein the apparatus further comprises: A detection module, configured to monitor the execution process of the SQL execution engine corresponding to the selection result of the SQL execution engine, and obtain an operation performance index; A storage module, configured to store the operating performance indicators, the SQL metadata information, and table metadata in a historical indicator library. 9. A computing device, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete mutual communication through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the adaptation method of the SQL execution engine in the PaaS platform according to any one of claims 1-5. 10. A computer storage medium, at least one executable instruction is stored in the storage medium, and the executable instruction causes the processor to execute the SQL execution engine in the PaaS platform according to any one of claims 1-5. The operation corresponding to the adaptation method.