CN114547083A - Data processing method and device and electronic equipment - Google Patents

Abstract

The present application discloses a data processing method, device and electronic device, which are applied to a code development platform. The method includes: obtaining database operation information, where the database operation information at least includes a first database operation statement using a first language specification; determining the database operation The target database engine to be called; based on the second language specification supported by the target database engine configured in the database connection program interface, the first database operation statement is converted into a second database operation statement conforming to the second language specification; the second database operation statement The action statement is transmitted to the target database engine. The solution of the present application can greatly reduce the complexity of database engine invocation, and can also reduce abnormal situations such as database engine invocation errors caused by incorrect language specifications used in database operation statements.

Description

Data processing method, device and electronic device

technical field

The present application relates to the technical field of data processing, and more particularly, to a data processing method, apparatus and electronic device.

Background technique

The code development platform often involves calling different database engines in the process of developing applications.

The language specifications used by different database engines are different, which requires the code development platform to use operation statements with different language specifications when calling different database engines, which is more complex and prone to errors due to language specifications. This leads to abnormal situations such as database engine call errors.

SUMMARY OF THE INVENTION

The present application provides a data processing method, apparatus and electronic device.

Wherein, a data processing method is applied to a code development platform, and the method includes:

obtaining database operation information, where the database operation information at least includes a first database operation statement using a first language specification;

Determine the target database engine to be called for database operations;

Based on the second language specification supported by the target database engine configured in the database connection program interface, converting the first database operation statement into a second database operation statement conforming to the second language specification;

The second database operation statement is transmitted to the target database engine.

In a possible implementation manner, the database operation information further includes: the application type of the target application in the code development platform that needs to perform the database operation;

Described determining the target database engine that the database operation needs to call, including:

Based on the application type of the target application, a target database engine suitable for the target application is determined.

In yet another possible implementation manner, determining a target database engine suitable for the target application based on the application type of the target application includes:

Based on the application type of the target application, an engine classification model is used to determine a target database engine suitable for the target application. The engine classification model is obtained by training with multiple application data samples marked with suitable database engines. The application data The sample includes at least the application type of the application.

In another possible implementation manner, the application data sample further includes: log data of a database engine used by the application, and the log data includes: engine information of the database engine invoked by the application and the application invocation logs generated by the database engine;

The engine classification model is obtained by training as follows:

Obtain historical data of database engines used by multiple applications, where the historical data includes application types of the applications and log data of the database engines used by the applications;

Determine the database engine suitable for the application based on the historical data of the database engine used by the application;

According to the database engine suitable for the application, use the application type of the application and the log data of the database engine used by the application to train the support vector machine classifier, and determine the trained support vector machine classifier as the engine classification model.

In another possible implementation manner, the first database operation statement is converted into a second language specification that conforms to the second language specification based on the second language specification supported by the target database engine configured in the database connection program interface. The second database operation statement includes:

Determine the target decoding module corresponding to the target database engine based on the decoding modules corresponding to different database engines configured in the database connection program interface;

Based on the conversion rules of the target database engine in the target decoding module, the first database operation statement is converted into a second database operation statement that conforms to the second language specification corresponding to the target data engine library.

In another possible implementation manner, the first database operation statement is converted into a second language specification that conforms to the second language specification based on the second language specification supported by the target database engine configured in the database connection program interface. The second database operation statement includes:

Determine the logical relationship between the constituent objects in the first database operation statement based on the database connection program interface;

In combination with the logical relationship and the semantic specification library corresponding to the target database engine configured in the database connection program interface, semantic expansion and relationship interpretation are performed on each constituent object in the first database operation statement, and the first database operation statement is obtained. 1. Semantic relationship information between the constituent objects in the database operation statement;

The first database operation statement is converted into a second database operation statement conforming to the second language specification corresponding to the target database engine in combination with the semantic relationship information of each constituent object in the first database operation statement.

In yet another possible implementation, determining the logical relationship between the constituent objects in the first database operation statement based on the database connection program interface includes:

constructing a logical syntax tree for representing the first database operation statement based on the database connection program interface;

In combination with the logical relationship and the semantic specification library corresponding to the target database engine configured in the database connection program interface, semantic expansion and relationship interpretation are performed on each constituent object in the first database operation statement, and the result is obtained. Describe the semantic relationship information between the constituent objects in the first database operation statement, including:

In combination with the semantic specification library corresponding to the configured target database engine, semantic expansion and relational interpretation are performed on each constituent object in the logical syntax tree to obtain a relational syntax tree;

Converting the first database operation statement into a second database operation statement conforming to the second language specification corresponding to the target database engine includes:

converting the relational syntax tree into at least one converted relational syntax tree that conforms to the second language specification corresponding to the target database engine;

The second database operation statement is converted based on the converted relational syntax tree.

In yet another possible implementation, before the second database operation statement is converted based on the converted relational syntax tree, the method further includes:

The at least one converted relational syntax tree is optimized and screened based on the cost optimizer to obtain the filtered converted relational syntax tree;

The conversion of the second database operation statement based on the converted relational syntax tree includes:

The second database operation statement is converted based on the filtered relational syntax tree after conversion.

Among them, a data processing device, including:

an information obtaining unit, configured to obtain database operation information, where the database operation information at least includes a first database operation statement using a first language specification;

The engine determination unit is used to determine the target database engine to be called for the database operation;

a statement conversion unit, configured to convert the first database operation statement into a second database operation statement conforming to the second language specification based on the second language specification supported by the target database engine configured in the database connection program interface;

An engine invoking unit, configured to transmit the second database operation statement to the target database engine.

Wherein, an electronic device includes at least a memory and a processor;

Wherein, the processor is used to execute the data processing method described in any one of the above;

The memory is used to store programs required by the processor to perform operations.

It can be seen from the above solutions that the code development platform of the present application can determine the target database engine to be called for the database operation after obtaining the first database operation statement using the first language specification, and based on the target database configured in the database connection program interface The second language specification supported by the engine can convert the first database operation statement into a second database operation statement that conforms to the second language specification, so that users do not need to use different language specifications for different database engines to write database operation statements, which greatly reduces the It reduces the complexity of database engine invocation; it can also reduce abnormal situations such as database engine invocation errors due to incorrect language specifications used in database operation statements.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

1 is a schematic flowchart of a data processing method provided by an embodiment of the present application;

2 is a schematic flowchart of a training engine classification model in an embodiment of the application;

3 is another schematic flowchart of a data processing method provided by an embodiment of the present application;

FIG. 4 is a schematic flowchart of a data processing method provided by an embodiment of the present application in an application scenario;

FIG. 5 is a schematic diagram of a composition structure of a data processing apparatus provided by an embodiment of the present application;

FIG. 6 is a schematic structural diagram of a composition of an electronic device provided by an embodiment of the present application.

The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims and the above-mentioned drawings are used to distinguish similar parts and not necessarily to describe a particular order or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances so that the embodiments of the application described herein can be practiced in sequences other than those illustrated herein.

Detailed ways

The solution of the present application is suitable for the scenario where the code development platform calls the database engine, which can improve the convenience for the code development platform to call the database engine, and reduce calling exceptions and the like.

In this application, the code development platform can be used for the code development of various applications. During the application development process and in the testing after application development, the applications developed by the code development platform often need to obtain various data resources from the database. Here On the basis, the code development platform needs to call the database engine to obtain the data resources required for application or application development.

There are many possibilities for the code development platform. For example, the code development platform can be a low-code platform (also called a low-code development platform). In the process of developing an application on a low-code platform, it is often necessary to call a certain database for the application. Database engine to obtain data resources required for developing applications or applications, etc.

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present application.

As shown in FIG. 1 , it shows a schematic flowchart of a data processing method provided by an embodiment of the present application, and the method of this embodiment can be applied to a code development platform.

The method of this embodiment may include the following steps:

S101, obtain database operation information.

Wherein, the database operation information includes at least a first database operation statement using a first language specification.

The first language specification may be a standard language specification set by the code development platform.

Wherein, each computer language may have multiple language specifications, and if the computer language supported by the code development platform is determined, the first language specification belongs to one of multiple language specifications of the computer language.

For example, if the code development platform uses Structured Query Language (SQL) to write database operation statements, a language specification can be selected from multiple language specifications applicable to the SQL language as the first language specification. For example, the language specification ISO/IEC 9075 in the SQL standard may be set as the standard language specification, and the first database statement is an SQL statement using the standard language specification.

It can be understood that the database operation information can be input by the user in the interactive interface provided by the code development platform. In the interactive interface, in addition to the first database operation statement, the user can also input other information about the database operation. In addition to the first database operation statement, the database operation information may also include application information of the application that needs to call the database, information of the operating user, etc., which is not limited.

S102: Determine a target database engine to be called for the database operation.

The database engine can realize the interpretation of the database operation statement, and realize the data acquisition of the database based on the interpretation of the database operation statement. It can be understood that when the data requester needs to access the database, the database operation statement needs to be interpreted by the database engine, and the required data is obtained from the database based on the database operation statement and returned to the data requester.

The database engine to be called for the database operation is the database engine corresponding to the database to be operated based on the first database operation statement. For the convenience of distinction, the database engine to be called for the database operation in this application is called the target database engine.

S103 , based on the second language specification supported by the target database engine configured in the database connection program interface, convert the first database operation statement into a second database operation statement conforming to the second language specification.

The database connection program interface is an application program interface used to standardize how the platform or client program accesses the database, and it provides methods such as querying and updating data in the database. For example, the database connection program interface may be a Java Database Connectivity (Java Database Connectivity, JDBC).

In this application, the code development platform is provided with a unified database connection program interface, instead of setting different database connection program interfaces for different database engines. Language specifications supported by different database engines can be configured in the database connection program interface set in the code development platform. On this basis, after determining the target database engine, the database connection program interface can determine the language specification supported by the target database engine.

In order to distinguish it from the language specification adopted by the first database operation statement, the language specification supported by the target database is referred to as the second language specification. In order to enable the database connection program interface to call the second database operation statement, the first database operation statement will be converted into a second database operation statement conforming to the second language specification based on the database connection program interface, so that the target database engine can understand and interpret the The second database operation statement.

In a possible implementation manner, the database connection program interface may be preset with decoding modules corresponding to different database engines, and on this basis, the database connection program interface of the present application can determine the target decoding module corresponding to the target database engine. Correspondingly, based on the conversion rules of the target database engine in the target decoding module, the first database operation statement can be converted into a second database operation statement that conforms to the second language specification corresponding to the target data engine library.

Since the database connection program interface can configure the decoding modules of different database engines, according to the different business requirements in the code development platform, the decoding modules corresponding to the database engines can be added, deleted or modified in the database connection program interface.

For example, the decoding module can be added to the database connection program interface (such as JDBC) in the form of a plug-in. If the code development platform needs to add a new call requirement to a certain database engine, this kind of database connection program interface can be inserted into the database connection program interface. The decoding module of the database engine enables the code development platform to also support the conversion of database operation statements using the first language specification into database operation statements adapted to the database engine.

S104: Transmit the second database operation statement to the target database engine.

It can be understood that after the second database operation statement is transmitted to the target database engine, after the target database engine acquires data from the database based on the second database operation statement, the acquired data can also be returned to the code development platform.

It can be seen from the above that the code development platform of the present application can determine the target database engine to be called by the database operation after obtaining the first database operation statement using the first language specification, based on the target database configured in the database connection program interface. The second language specification supported by the engine can convert the first database operation statement into a second database operation statement that conforms to the second language specification, so that users do not need to use different language specifications for different database engines to write database operation statements, which greatly reduces the It reduces the complexity of database engine invocation; it can also reduce abnormal situations such as database engine invocation errors due to incorrect language specifications used in database operation statements.

It can be understood that, in the present application, there can be many ways to determine the target database engine to be called for the database operation.

In a possible implementation manner, the user can specify the database engine to be called, and on this basis, the target database engine to be called for the database operation set by the user can be obtained.

However, in practical applications, different applications may be suitable for calling data through different database engines. For example, when the same application uses different database engines, there will be differences in performance such as the response time of accessing the database. It is difficult for users to know the database engine suitable for different applications, which may affect the performance of data acquisition or bring more burden to the server where the database is located.

Based on this, in another possible implementation manner, the database operation information in the present application may further include: the application type of the target application that needs to perform database operations in the code development platform. Correspondingly, the code development platform can also determine a target database engine suitable for the target application based on the application type of the target application.

For example, database engines suitable for different types of applications can be pre-configured, and on this basis, the target database engines corresponding to the target application can be determined according to the database engines corresponding to the different types of applications and the application type of the target application.

In an optional manner, the present application may pre-train an engine classification model, and the engine classification model may be obtained by training using multiple application data samples marked with a suitable database engine. Correspondingly, based on the application type of the target application, the engine classification model can be used to determine a target database engine suitable for the target application.

Wherein, the application data sample includes at least the application type of the application. The application sample data may further include: log data of the database engine used by the application. The log data may include: engine information of the database engine called by the application, and logs generated by the application calling the database engine.

For example, the engine information of the database engine called by the application may include the name and type of the database engine.

The log generated by the application calling the database engine may be data such as the response time of the data obtained by the application through the database engine and the amount of data requested, and may also include: performance information such as memory and CPU of the server where the database corresponding to the database engine is located. There is no limit to this.

Among them, any supervised training method may be adopted for training the classification model of the engine, and the specific training process may not be limited.

For ease of understanding, the following describes the process of training the engine classification model by taking the engine classification model as a trained Support Vector Machines (Support Vector Machines) classifier as an example.

As shown in FIG. 2 , which shows a schematic flowchart of a training engine classification model provided by an embodiment of the present application, the training method of the present embodiment may include:

S201: Obtain historical data of database engines used by multiple applications.

The historical data includes the application type of the application and the log data of the database engine used by the application. For the log data, please refer to the previous related introduction, which will not be repeated here.

It can be understood that if any application developed in the code development platform uses the code development platform to call the database engine, the code development platform can obtain data such as the type of the application and the log of the application calling (or using) the database engine. On this basis, the code development platform can store the relevant data of the application using the database engine as historical data.

S202: Determine a database engine suitable for the application based on historical data of the database engine used by the application.

The code development platform can use the historical data of the database engine in combination with the application, and can determine the database engine with the best database operation performance as the database engine suitable for the application.

For example, the code development platform can use the database engine that takes the shortest time to respond to the database operation corresponding to the application as the database engine suitable for the application.

For another example, the code development platform can also comprehensively determine a suitable database engine for the application on the basis of the time-consuming response and in combination with data such as the performance of the server where the database corresponding to the database engine is located.

S203 , according to a database engine suitable for the application, using the application type of the application and the log data of the database engine used by the application, train a support vector machine classifier, and determine the trained support vector machine classifier as an engine classification model.

Among them, the database engine suitable for the application can be used as the annotation data of the application, and the application type of the application and the log data using the database engine can be used as training data samples. On this basis, the support vector machine classifier can be trained by supervised training.

For example, when the function value of the loss function of the trained SVM classifier converges or the number of repetitions of the training reaches the set number of times, it can be considered that the training is completed, so that the trained SVM classifier can be used as the engine classification model. Of course, there may be various possibilities for determining the specific judgment manner of the completion of the training of the support vector machine classifier, which is not limited in this application.

It can be understood that FIG. 2 is only for the convenience of understanding, and a method of training the engine classification model is used as an example to illustrate. In practical applications, the specific method of training the engine classification model may also have other possibilities, and this application also does not add to this. limit.

It can be understood that, in the present application, there may also be various possibilities for the specific implementation of converting the first database operation statement into the second database operation statement.

In order to more accurately convert the second database operation statement, the present application provides an implementation manner of converting the second database operation statement. The solution of the present application is described below by taking this implementation manner as an example.

As shown in FIG. 3 , it shows a schematic flowchart of a data processing method provided by an embodiment of the present application, and the method of this embodiment can be applied to a code development platform.

The method of this embodiment may include the following steps:

S301, obtain database operation information.

Wherein, the database operation information includes at least a first database operation statement using a first language specification.

S302: Determine the target database engine to be called for the database operation.

For the above steps S301 and S302, reference may be made to the relevant introduction of the previous embodiment, and details are not repeated here.

S303: Determine the logical relationship between the constituent objects in the first database operation statement based on the database connection program interface.

Wherein, the database operation statement may form objects such as functions, operators, variables, operators, and some parameters, and the logical relationship determined here refers to the association relationship between the constituent objects in the first database operation statement.

For example, if the first database operation statement contains a conditional statement, and the conditional statement may contain a conditional judgment between two objects, it can be determined that the logical relationship between the two objects is a corresponding conditional judgment.

In a possible implementation manner, the present application may also construct a logical syntax tree for representing the first database operation statement through a database connection program interface. The logical syntax tree is constructed by performing syntax analysis on the database operation statement according to the syntax rules, and logically expresses the same tree-like syntax structure as the database operation statement.

Through the logical syntax tree, the grammatical logical relationship of each constituent object in the database operation statement can be more intuitively represented, which is beneficial to the subsequent more accurate and effective semantic conversion.

It can be understood that, the process of converting the corresponding logical syntax tree based on the first database operation statement may have multiple implementation possibilities, which are not limited in this application.

S304, combining the logical relationship and the semantic specification library corresponding to the target database engine configured in the database connection program interface, perform semantic expansion and relationship interpretation on each component object in the first database operation statement, and obtain each component object in the first database operation statement Semantic relationship information between them.

Among them, the database connection program interface can be configured with semantic specification libraries of different database engines. The semantic specification library of the database engine can record the database corresponding to the database engine, and the detailed description of the tables and fields in the database.

For example, the database storage type, table type, table structure, primary key and index fields, column type, and column type restrictions and other related information of the database corresponding to the database engine can also include aliases for some query operations and column expressions.

It can be seen from this that the semantic specification library of the target database engine may contain detailed descriptions of the relevant contents and operations in the target database engine and its corresponding database. Based on this, combined with the semantic specification library of the target database engine, in-depth semantic interpretation of each constituent object in the first database operation statement can be performed, so that the semantic information of each constituent object in the first database operation statement can be expanded and further clarified.

For example, suppose the first database operation statement is "Select...from table A", but table A may exist in many databases, and the table A in the database corresponding to the target database engine will have an alias or In this way, the full name of table A will be defined in the semantic specification library of the target database engine. If it is Jdb.A, then the semantics of this tableA needs to be modified or added as Jdb.A.

For another example, for a subquery defined using WITH syntax, based on the semantic specification library, the subquery can be associated with a FROM/JOIN node that references the subquery.

By interpreting the semantic relationship of each constituent object in the first database operation statement, not only some special semantic information of the constituent objects in the database corresponding to the target database engine can be clarified, but also the semantic relationship between different constituent objects can be clarified, so as to provide information for subsequent conversion. The database operation statement provides the basis.

In a possible implementation manner, if the logical relationship corresponding to the first database operation statement is represented by the logical syntax tree in the previous step S303, the database connection program interface can combine the semantic specification library corresponding to the configured target database engine to perform the logical syntax tree. Each constituent object in the above is used for semantic expansion and relational interpretation, and a relational syntax tree is obtained.

The relational syntax tree can represent the logical syntax tree and the syntax relation information between the constituent objects in the logical syntax tree.

It can be understood that by traversing the logical syntax tree and according to the syntax specification library of the target database engine, each node in the logical syntax tree can be semantically interpreted, and extended semantic information can be added to the nodes in the logical syntax tree. Adding semantic relational explanations between different nodes in the syntax tree can turn the logical syntax tree into a relational syntax tree with richer semantic relationships.

The source of all identifiers, functions, parameters and other objects in the first database operation statement and one or more kinds of semantic information such as types can be determined through the relational syntax tree. Through the relational syntax tree, not only can the first database operation be learned The semantics of the statement can also provide necessary semantic information for the conversion operation in converting the first database operation statement into the second database operation statement.

S305: Combine the semantic relationship information of each constituent object in the first database operation statement, and convert the first database operation statement into a second database operation statement that conforms to the second language specification corresponding to the target database engine.

There may be many possibilities for the conversion algorithm used to convert one database operation statement to another database operation statement, which is not limited in this application. However, only converting statements based on the conversion algorithm may result in conversion errors, or some conversion operations cannot be performed.

In the present application, by determining the semantic relationship information between the constituent objects in the first database operation statement, an information basis can be provided for the statement conversion, so as to provide an information basis for some conversion operations during the execution of the conversion algorithm, so as to avoid the Missing information that prevents some statement transformations from being performed can also reduce statement transformation errors.

For example, two constituent objects need to be combined in statement transformation. If the semantic relationship between these two constituent objects is not clear, there may be a transformation error when converting to a statement transformation of another language specification, and the semantic relationship is combined. information, the relationship between the two can be clarified, and then the corresponding conversion processing can be completed.

For another example, assuming that according to the semantic relationship information, it can be determined that there is no arbitrary identifier in a table in the statement that is derived from this table, and it can be determined that this table is a useless table, then in the process of statement conversion, this table can be converted. remove.

In a possible implementation manner, on the premise of converting the relational syntax tree, the relational syntax tree may be converted into at least one converted relational syntax tree that conforms to the second language specification corresponding to the target database engine. That is to say, the relational syntax tree is traversed according to the second language specification corresponding to the target database engine, and the sentences expressed by the relational syntax tree are equivalently converted to obtain the expressed database operation behavior and the relational syntax tree. The expressed database operation behavior is the same, but the language used to specify different syntax trees.

Correspondingly, the second database operation statement may be converted based on the converted relational syntax tree.

It is understandable that, considering the same identifier and objects such as functions, or the same set of object relationships, etc., there may be multiple expressions with the same meaning but different forms in the second language specification. Therefore, the converted relationship There may be more than one type of syntax tree.

The present application can convert a database operation statement based on any converted relational syntax tree to obtain a second database operation statement that conforms to the second language specification.

It is understandable that although different relational syntax trees after conversion express the same meaning of statements, there will be differences in the execution performance required for the converted statements corresponding to different relational syntax trees after conversion. In order to improve the subsequent calls to the database engine to query data performance, the present application can also select from a plurality of converted relational syntax trees a post-conversion relational syntax tree with the best query performance for performing a query operation, and determine a second database operation based on the converted relational syntax tree statement.

In a possible implementation manner, in the present application, at least one post-conversion relational syntax tree may be optimized and screened based on a cost optimizer to obtain a filtered post-conversion relational syntax tree. Correspondingly, the second database operation statement can be converted based on the filtered relational syntax tree after conversion.

Among them, the Cost Based Optimizer (CBO) may also be referred to as a cost-based optimizer. In the process of optimizing and screening at least one converted relational syntax tree based on CBO, the consumption of memory, input and output, etc. required to execute each converted relational syntax tree will be compared, and then a resource consumption with the least resource consumption or execution will be determined. The best performing transformed relational syntax tree.

S306: Transmit the second database operation statement to the target database engine.

For this step, reference may be made to the relevant introduction of the foregoing embodiments, and details are not described herein again.

In order to facilitate understanding of the benefits of the solution of the present application, the solution of the present application will be described below with reference to an application scenario.

Taking the scenario where the code development platform adopts the SQL language as an example, the solution of the present application will be described in combination with a possible implementation manner.

As shown in FIG. 4 , it shows a schematic flowchart of the data processing method provided by the embodiment of the present application in an application scenario, and the method of this embodiment is applied to a code development platform. This embodiment may include:

S401, obtain database operation information.

The database operation information includes at least a first SQL statement using the first SQL language specification and an application type of a target application that needs to perform database operations.

For example, if the code development platform sets the SQL language specification as the ISO/IEC 9075 standard, the first SQL statement is the SQL statement using the ISO/IEC 9075 specification.

S402, based on the application type of the target application through the JDBC interface, and using the engine classification model, determine a target database engine suitable for the target application.

For example, by inputting the application type of the target application into the engine classification model, the suitable probability of each database engine output by the engine classification model can be obtained, and the database engine with the largest suitable probability is determined as the target database engine.

Of course, this is just an example, and there are other possibilities for determining the category of the database engine suitable for the target application in combination with the engine classification model, which is not limited.

S403, based on the decoding modules corresponding to different database engines configured in the JDBC interface, determine a target decoding module corresponding to the target database engine.

S404, the first SQL statement is converted into a logical syntax tree by the target decoding module.

There are many possibilities for converting from an SQL statement to a logical syntax tree, and there is no restriction on the specific conversion method.

S405 , by combining the target decoding module with the semantic specification library corresponding to the target database engine, perform semantic expansion and relational interpretation on each constituent object in the logical syntax tree to obtain a relational syntax tree.

The relational syntax tree can represent the logical syntax tree and the syntax relation information between the constituent objects in the logical syntax tree.

S406: Convert the relational syntax tree into at least one converted relational syntax tree that conforms to the second SQL language specification corresponding to the target database engine.

For example, if the SQL language specification supported by the target database engine is SparkSQL, the relational syntax tree needs to be converted based on the SparkSQL language specification.

S407: Perform optimization screening on at least one converted relational syntax tree based on the CBO, to obtain a filtered converted relational syntax tree.

S408 , convert the second SQL statement based on the filtered relational syntax tree after conversion.

S409, transmit the second SQL statement to the target database engine.

It can be understood that, in the embodiment of FIG. 4 , after the code development platform sets the first SQL language specification adopted by the code development platform, the user can uniformly use the first SQL language specification to write SQL statements, and only input required SQL statements. The application type of the database operation, and the user does not need to care about the database engine that needs to be called, nor the language specification that the database engine needs to support. The code development platform calls the engine classification model configured in the unified JDBC interface, not only can determine the appropriate database engine, but also can convert the first SQL statement into a suitable database engine based on the decoding module corresponding to the database engine in JDBC SQL statements greatly improve the convenience of database operations.

In addition, since the decoding modules of different database engines can be inserted into the same JDBC interface in the form of plug-ins, when the business scenario of the code development platform changes, which causes the type of database engine to be called by the code development platform to change, you can Flexibly develop the decoding module of the corresponding database engine and set it in the JDBC interface in the form of a plug-in, and delete the decoding module corresponding to the database engine that is not required in JDBC, and the flexibility is high.

Corresponding to a data processing method of the present application, the present application also provides a data processing apparatus.

As shown in FIG. 5, it shows a schematic diagram of the composition and structure of the data processing apparatus provided by the present application. The apparatus in this embodiment may be applied to a code development platform, and this embodiment may include:

an information obtaining unit 501, configured to obtain database operation information, where the database operation information at least includes a first database operation statement using a first language specification;

An engine determination unit 502, configured to determine a target database engine to be called by the database operation;

A statement conversion unit 503, configured to convert the first database operation statement into a second database operation statement conforming to the second language specification based on the second language specification supported by the target database engine configured in the database connection program interface;

The engine invoking unit 504 is configured to transmit the second database operation statement to the target database engine.

In a possible implementation manner, the database operation information obtained by the information obtaining unit further includes: the application type of the target application in the code development platform that needs to perform the database operation.

The engine determination unit can include:

The engine determination subunit is configured to determine a target database engine suitable for the target application based on the application type of the target application.

In a possible implementation manner, the engine determination subunit is specifically configured to use an engine classification model to determine a target database engine suitable for the target application based on the application type of the target application, and the engine classification model is: It is obtained by training with a plurality of application data samples marked with a suitable database engine, and the application data samples at least include the application type of the application.

In a possible implementation manner, the application data sample further includes: log data of a database engine used by the application, and the log data includes: engine information of the database engine invoked by the application and information about the database engine invoked by the application. generated logs;

The device also includes a model training model, and the model training unit is used to train the engine classification model in the following manner:

Obtain historical data of database engines used by multiple applications, where the historical data includes application types of the applications and log data of the database engines used by the applications;

Determine the database engine suitable for the application based on the historical data of the database engine used by the application;

According to the database engine suitable for the application, use the application type of the application and the log data of the database engine used by the application to train the support vector machine classifier, and determine the trained support vector machine classifier as the engine classification model.

In yet another possible implementation, the statement conversion unit includes:

The module determines the subunit for determining the target decoding module corresponding to the target database engine based on the decoding modules corresponding to different database engines configured in the database connection program interface;

The statement conversion subunit is used to convert the first database operation statement into a second language specification that conforms to the second language specification corresponding to the target data engine library based on the conversion rules of the target database engine in the target decoding module. Database operation statement.

In yet another possible implementation, the statement conversion unit includes:

a relationship determination unit, configured to determine the logical relationship between the constituent objects in the first database operation statement based on the database connection program interface;

A semantic interpretation unit, configured to perform semantic expansion and relationship on each constituent object in the first database operation statement in combination with the logical relationship and the semantic specification library corresponding to the target database engine configured in the database connection program interface Interpretation to obtain semantic relationship information between the constituent objects in the first database operation statement;

A conversion processing unit, configured to convert the first database operation statement into a second database operation that conforms to the second language specification corresponding to the target database engine in combination with the semantic relationship information of each constituent object in the first database operation statement statement.

In another possible implementation manner, the relationship determination unit includes: a relationship tree construction unit, configured to construct a logical syntax tree for representing the first database operation statement based on the database connection program interface;

Semantic Interpretation Units, including:

A relational tree conversion unit, configured to perform semantic expansion and relational interpretation on each constituent object in the logical syntax tree in combination with the semantic specification library corresponding to the configured target database engine, to obtain a relational syntax tree;

Conversion processing units, including:

A tree specification unit, configured to convert the relational syntax tree into at least one converted relational syntax tree that complies with the second language specification corresponding to the target database engine;

A tree-to-statement unit is configured to convert a second database operation statement based on the converted relational syntax tree.

In yet another possible implementation, the device further includes:

A screening unit, configured to optimize and screen the at least one post-conversion relational syntax tree based on the cost optimizer before the tree-to-statement unit converts the second database operation statement based on the post-conversion relational syntax tree, to obtain a screened The converted relational syntax tree obtained;

The tree-to-statement unit is specifically configured to convert the second database operation statement based on the filtered relational syntax tree after conversion.

In another aspect, the present application also provides an electronic device, as shown in FIG. 6 , which shows a schematic structural diagram of the electronic device, the electronic device can be any type of electronic device, and the electronic device at least includes memory 601 and processor 602;

The processor 601 is configured to execute the data processing method in any one of the above embodiments.

The memory 602 is used to store programs required by the processor to perform operations.

It can be understood that the electronic device may further include a display unit 603 and an input unit 604 .

Of course, the electronic device may also have more or less components than in FIG. 6 , which is not limited.

In another aspect, the present application also provides a computer-readable storage medium, where at least one instruction, at least one program, code set or instruction set is stored in the computer-readable storage medium, the at least one instruction, the at least one A piece of program, the code set or the instruction set is loaded and executed by the processor to implement the data processing method described in any one of the above embodiments.

The present application also proposes a computer program comprising computer instructions stored in a computer-readable storage medium. When the computer program runs on the electronic device, it is used to execute the data processing method described in any one of the above embodiments.

It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. For the same and similar parts among the various embodiments, refer to each other Can. Meanwhile, the features described in each embodiment in this specification can be replaced or combined with each other, so that those skilled in the art can realize or use the present application. As for the apparatus-type embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for related parts, please refer to the partial descriptions of the method embodiments.

Finally, it should also be noted that in this document, relational terms such as first and second are used only to distinguish one entity or operation from another, and do not necessarily require or imply these entities or that there is any such actual relationship or sequence between operations. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article, or device that includes the element.

The above description of the disclosed embodiments enables any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, this application is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. a data processing method, applied to a code development platform, the method comprising: obtaining database operation information, where the database operation information at least includes a first database operation statement using a first language specification; Determine the target database engine to be called for database operations; Based on the second language specification supported by the target database engine configured in the database connection program interface, converting the first database operation statement into a second database operation statement conforming to the second language specification; The second database operation statement is transmitted to the target database engine. 2. The method according to claim 1, wherein the database operation information further comprises: the application type of the target application that needs to perform the database operation in the code development platform; Described determining the target database engine that the database operation needs to call, including: Based on the application type of the target application, a target database engine suitable for the target application is determined. 3. The method according to claim 2, wherein determining a target database engine suitable for the target application based on the application type of the target application, comprising: Based on the application type of the target application, an engine classification model is used to determine a target database engine suitable for the target application. The engine classification model is obtained by training with multiple application data samples marked with suitable database engines. The application data The sample includes at least the application type of the application. 4. The method according to claim 3, wherein the application data sample further comprises: log data of a database engine used by the application, the log data comprising: engine information of a database engine called by the application and the application The log generated by calling the database engine; The engine classification model is obtained by training as follows: Obtain historical data of database engines used by multiple applications, where the historical data includes application types of the applications and log data of the database engines used by the applications; Determine the database engine suitable for the application based on the historical data of the database engine used by the application; According to the database engine suitable for the application, use the application type of the application and the log data of the database engine used by the application to train the support vector machine classifier, and determine the trained support vector machine classifier as the engine classification model. 5 . The method according to claim 1 , wherein the first database operation statement is converted to conform to the second language specification based on the second language specification supported by the target database engine configured in the database connection program interface. 6 . The second database operation statement includes: Determine the target decoding module corresponding to the target database engine based on the decoding modules corresponding to different database engines configured in the database connection program interface; Based on the conversion rules of the target database engine in the target decoding module, the first database operation statement is converted into a second database operation statement that conforms to the second language specification corresponding to the target data engine library. 6 . The method according to claim 1 , wherein the first database operation statement is converted to conform to the second language specification based on the second language specification supported by the target database engine configured in the database connection program interface. 7 . The second database operation statement includes: Determine the logical relationship between the constituent objects in the first database operation statement based on the database connection program interface; In combination with the logical relationship and the semantic specification library corresponding to the target database engine configured in the database connection program interface, semantic expansion and relationship interpretation are performed on each constituent object in the first database operation statement, and the first database operation statement is obtained. 1. Semantic relationship information between the constituent objects in the database operation statement; The first database operation statement is converted into a second database operation statement conforming to the second language specification corresponding to the target database engine in combination with the semantic relationship information of each constituent object in the first database operation statement. 7. The method according to claim 6, wherein the logical relationship between each constituent object in the first database operation statement is determined based on the database connection program interface, comprising: constructing a logical syntax tree for representing the first database operation statement based on the database connection program interface; In combination with the logical relationship and the semantic specification library corresponding to the target database engine configured in the database connection program interface, semantic expansion and relationship interpretation are performed on each constituent object in the first database operation statement, and the result is obtained. Describe the semantic relationship information between the constituent objects in the first database operation statement, including: In combination with the semantic specification library corresponding to the configured target database engine, semantic expansion and relational interpretation are performed on each constituent object in the logical syntax tree to obtain a relational syntax tree; Converting the first database operation statement into a second database operation statement conforming to the second language specification corresponding to the target database engine includes: converting the relational syntax tree into at least one converted relational syntax tree conforming to the second language specification corresponding to the target database engine; The second database operation statement is converted based on the converted relational syntax tree. 8. The method according to claim 7, wherein before the second database operation statement is converted based on the converted relational syntax tree, the method further comprises: The at least one converted relational syntax tree is optimized and screened based on the cost optimizer to obtain the filtered converted relational syntax tree; The conversion of the second database operation statement based on the converted relational syntax tree includes: The second database operation statement is converted based on the filtered relational syntax tree after conversion. 9. A data processing device, comprising: an information obtaining unit, configured to obtain database operation information, where the database operation information at least includes a first database operation statement using a first language specification; The engine determination unit is used to determine the target database engine to be called for the database operation; a statement conversion unit, configured to convert the first database operation statement into a second database operation statement conforming to the second language specification based on the second language specification supported by the target database engine configured in the database connection program interface; An engine invoking unit, configured to transmit the second database operation statement to the target database engine. 10. An electronic device, comprising at least a memory and a processor; Wherein, the processor is configured to execute the data processing method according to any one of claims 1 to 8; The memory is used to store programs required by the processor to perform operations.

Images