CN117251498A

CN117251498A - Development method, device, equipment and medium for data warehouse access interface

Info

Publication number: CN117251498A
Application number: CN202311313376.3A
Authority: CN
Inventors: 郭忆; 李卓豪; 余方正; 陈志辉; 云娜; 余利华
Original assignee: Hangzhou Netease Shuzhifan Technology Co ltd
Current assignee: Hangzhou Netease Shuzhifan Technology Co ltd
Priority date: 2023-10-10
Filing date: 2023-10-10
Publication date: 2023-12-19

Abstract

The embodiment of the application particularly provides a development method, a device, equipment and a medium for a data warehouse access interface. The development method for the data warehouse access interface comprises the steps of responding to configuration operation of a developer for each parameter configuration item in the access interface configuration interface, and obtaining parameter values of each parameter; generating a data warehouse access interface according to the corresponding relation between the access module and the parameters and the parameter values of the parameters; the corresponding relation is generated by disassembling the data warehouse access interface codes according to SQL grammar rules; the data warehouse access interface is used for being called by a calling party after being released. According to the method, the data warehouse access interface is generated through visual configuration operation aiming at the data warehouse access interface, so that complicated operation of data warehouse access interface development is simplified, and the data warehouse access interface development efficiency is improved.

Description

Development method, device, equipment and medium for data warehouse access interface

Technical Field

The present application relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a medium for developing a data warehouse access interface.

Background

This section is intended to provide a background or context to the embodiments of the application recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.

In a data query scenario for a data warehouse, to facilitate invoicing of the data warehouse by a caller, it is often necessary to develop a data warehouse fetch interface for fetching for the data warehouse.

In the prior art, a mode of manually writing codes is generally adopted to develop a data warehouse access interface.

However, since the development operation of the data warehouse access interface is complicated in this way, the efficiency of the data warehouse access interface development is low.

Disclosure of Invention

Because the efficiency of data warehouse access interface development is low in the prior art, an improved data warehouse access interface development method is highly needed to improve the efficiency of data warehouse access interface development.

In this context, it is desirable for embodiments of the present application to provide a method, apparatus, device, and medium for developing a data warehouse access interface.

In one aspect, an embodiment of the present application provides a method for developing an access interface for a data warehouse, which is applied to an interface developing device, and includes:

Responding to configuration operation of a developer for each parameter configuration item in the access interface configuration interface, and acquiring parameter values of each parameter;

generating a data warehouse access interface according to the corresponding relation between the access module and the parameters and the parameter values of the parameters; the corresponding relation is generated by disassembling the data warehouse access interface codes according to SQL grammar rules; the data warehouse access interface is used for being called by a calling party after being released.

In one embodiment, the data warehouse access interface is generated according to the corresponding relation between the access module and the parameters and the parameter values of the parameters, and the data warehouse access interface comprises:

generating a structured object corresponding to the access module according to the parameter value of the parameter corresponding to the access module;

based on the structured objects, a data warehouse access interface is generated.

In one embodiment, the structured object comprises at least one of the following:

a result set object, a library table object, a condition screening object, an aggregation object, a paging object and a sorting object;

the parameters corresponding to the result set object comprise a returned result parameter list; the parameters corresponding to the library table object comprise at least one of the following: a database, a data table, and a database schema; the parameters corresponding to the condition screening object comprise at least one of the following: inquiring condition entry parameters, class attribute parameters and calculation parameters; the parameters corresponding to the aggregation object comprise aggregation parameters; the parameters corresponding to the paging object include at least one of: limiting the query parameters and offsetting the query parameters; the parameters corresponding to the ordering object include at least one of: a sort type parameter and an aggregate sort parameter.

In one embodiment, the method further comprises:

in the interface operating environment, a data warehouse access interface is published.

In one embodiment, the interface runtime environment includes a sandbox environment and an online environment;

in an interface operating environment, a publishing data warehouse access interface comprising:

in a sandbox environment, a data warehouse access interface is issued;

and if the data warehouse access interface is determined to be successfully tested, the data warehouse access interface is released in an online environment.

In one embodiment, the interface development device includes a data gateway;

the data gateway is used for receiving the access request containing the access interface domain name sent by other equipment, and selecting between the sandbox environment or the online environment according to the environment parameters in the access request so as to call the data warehouse access interface under the selected interface operation environment;

the access interface domain name is obtained after the data gateway performs instance registration.

In one embodiment, the interface development device further comprises an SQL adapter;

the SQL adapter is used for converting the structured object in the data warehouse access interface into an SQL statement according to input parameters in access requests sent by other devices.

In one embodiment, the interface development device further comprises a driver routing table;

the driver route table is registered with the drivers corresponding to the databases respectively;

the driver program is used for establishing data source connection between the interface development equipment and the database corresponding to the access request;

the data source connection is used for the data warehouse access interface to access the corresponding database.

In one embodiment, the interface development device further comprises a storage management plug-in library; at least one plug-in containing a driver is stored in the storage management plug-in library;

the storage path of the plug-in is generated according to the corresponding database type name and the data source version number.

In one embodiment, the drivers corresponding to the databases are loaded by a class loader; each driver is in one-to-one correspondence with each loader; the class loader is generated by inheriting the class loader base class and rewriting the loadClass method.

In one embodiment, the interface development device further comprises HikariCP;

HikariCP is used to manage the data source connections established for the databases.

In one embodiment, the interface development device further comprises a database management center;

The database management center is used for registering and managing each database.

In one embodiment, the method further comprises:

monitoring configuration update information of a data warehouse access interface; the configuration updating information is information for updating parameters in the data warehouse access interface;

and in response to the presence of the configuration update information, updating the data warehouse access interface based on the configuration update information.

In one embodiment, updating the data warehouse access interface based on the configuration update information includes:

if the storage root path of the configuration update information is determined to represent the sandbox environment, updating a data warehouse access interface in the sandbox environment;

and if the storage root path of the configuration update information is determined to characterize the online environment, updating the data warehouse access interface in the online environment.

In one aspect, an embodiment of the present application provides a development device for a data warehouse access interface, which is applied to an interface development device, and includes:

the acquisition unit is used for responding to the configuration operation of a developer for each parameter configuration item in the access interface configuration interface and acquiring the parameter value of each parameter;

the generation unit is used for generating a data warehouse access interface according to the corresponding relation between the access module and the parameters and the parameter values of the parameters; the corresponding relation is generated by disassembling the data warehouse access interface codes according to SQL grammar rules; the data warehouse access interface is used for being called by a calling party after being released.

In one embodiment, the generating unit is configured to:

In one embodiment, the generating unit is further configured to:

The generating unit is further configured to:

in a sandbox environment, a data warehouse access interface is issued;

In one embodiment, the interface development device includes a data gateway;

In one embodiment, the interface development device further comprises HikariCP;

In one embodiment, the generating unit is further configured to:

In one aspect, an embodiment of the present application provides an electronic device, including:

a processor; and

a memory storing computer instructions for causing a processor to perform the steps of the method provided in developing various alternative implementations of any of the above described access interfaces for data warehouses.

In one aspect, a storage medium is provided in an embodiment of the present application, where computer instructions are stored, where the computer instructions are configured to cause a computer to perform the steps of a method as provided in any of the various alternative implementations of the development of a data warehouse access interface described above.

The development method for the data warehouse access interface comprises the steps of responding to configuration operation of a developer for each parameter configuration item in the access interface configuration interface, and obtaining parameter values of each parameter; generating a data warehouse access interface according to the corresponding relation between the access module and the parameters and the parameter values of the parameters; the corresponding relation is generated by disassembling the data warehouse access interface codes according to SQL grammar rules; the data warehouse access interface is used for being called by a calling party after being released. According to the method, the data warehouse access interface is generated through visual configuration operation aiming at the data warehouse access interface, so that complicated operation of data warehouse access interface development is simplified, and the data warehouse access interface development efficiency is improved.

Drawings

The above, as well as additional purposes, features, and advantages of exemplary embodiments of the present application will become readily apparent from the following detailed description when read in conjunction with the accompanying drawings. Several embodiments of the present application are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings, in which:

FIG. 1 schematically illustrates an example diagram of a data warehouse access interface development and publishing process in an embodiment of the present application;

FIG. 2 schematically illustrates a flow chart of a method of developing a data warehouse access interface in an embodiment of the present application;

FIG. 3 schematically illustrates an example diagram of an access interface configuration interface in an embodiment of the present application;

FIG. 4 schematically illustrates an example diagram of a data warehouse access interface code in an embodiment of the present application;

FIG. 5 schematically illustrates an example diagram of a structured object in an embodiment of the present application;

FIG. 6 schematically illustrates an exemplary diagram of a method for constructing a library table object according to an embodiment of the present application;

FIG. 7 schematically illustrates an example diagram of a JSON string in an embodiment of the present application;

FIG. 8 schematically illustrates an example diagram of data warehouse access interface development in an embodiment of the present application;

FIG. 9 schematically illustrates a flow chart of a method for data warehouse access interface publishing in an embodiment of the application;

FIG. 10 schematically illustrates an architecture example diagram of a data warehouse access interface distribution system in an embodiment of the present application;

FIG. 11 schematically illustrates a flow chart of a method for updating a data warehouse access interface in an embodiment of the present application;

FIG. 12 schematically illustrates an example diagram of a database or data warehouse query process in an embodiment of the present application;

FIG. 13 schematically illustrates an architecture example diagram of a development system for a data warehouse access interface in an embodiment of the present application;

FIG. 14 schematically shows a structural example diagram of a storage medium in an embodiment of the present application;

FIG. 15 schematically illustrates a block diagram of a development device for a data warehouse access interface in an embodiment of the present application;

fig. 16 schematically shows a structural example diagram of an electronic device in the embodiment of the present application.

In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

Detailed Description

The principles and spirit of the present application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are presented merely to enable one skilled in the art to better understand and practice the present application and are not intended to limit the scope of the present application in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

Those skilled in the art will appreciate that the embodiments of the present application may be implemented as a system, apparatus, device, method, or computer program product. Thus, the present application may be embodied in the form of: complete hardware, complete software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.

According to the embodiment of the application, a development method, a device, equipment and a medium for a data warehouse access interface are provided.

In this document, it should be understood that any number of elements in the drawings is for illustration and not limitation, and that any naming is used only for distinction and not for any limitation.

Some of the terms referred to in the embodiments of the present application will be described first to facilitate understanding by those skilled in the art.

Terminal equipment: the mobile terminal, stationary terminal or portable terminal may be, for example, a mobile handset, a site, a unit, a device, a multimedia computer, a multimedia tablet, an internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communications system device, a personal navigation device, a personal digital assistant, an audio/video player, a digital camera/camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a game device, or any combination thereof, including the accessories and peripherals of these devices, or any combination thereof. It is also contemplated that the terminal device can support any type of interface (e.g., wearable device) for the user, etc.

And (3) a server: the system can be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and can also be a cloud server for providing basic services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, big data, artificial intelligent platforms and the like.

Application programming interface (Application Programming Interface, API): is a set of definitions and protocols for building and integrating application software.

Data warehouse: a data warehouse is a data set that supports management decisions, and in this embodiment of the present application, for ease of description, the data warehouse may also be referred to as a silo. Data is subject-oriented, integrated, not easily lost, and time-varying. A data warehouse is a snapshot collection of all operating environments and external data sources. It does not need to be very accurate because it must be extracted from the operating environment on a specific time basis.

Data application layer (Application Data Store, ADS): the system is used for storing personalized statistical index data, report data and the like of the data products, and is also used for providing data for the data products and data analysis so as to provide subsequent service inquiry, analysis, data distribution and the like.

Ad Hoc query (Ad Hoc): the user flexibly selects the query condition according to the own requirement, and then the system can generate a corresponding statistical report according to the selection of the user.

Development of a chimney: each business line is independently built by different development teams, the technical stacks of the different business lines are different, and the different development teams are not connected with each other, so that repeated development is easy to cause.

Low Code (Low Code): is a visual application development method, and uses less codes to deliver application programs at a faster speed.

Structured query language (Structured Query Language, SQL): is a programming language for storing and processing information in a relational database.

Sustained integration (Continuous Integration, CI)/sustained deployment (Continuous Deployment, CD): is continuously integrated, continuously delivered, and continuously deployed.

Maven: is a software (especially Java software) project management and automatic construction tool provided by Apache (a web server) software foundation.

Data source: a database or database server for use by a database application.

Parental delegation mode: is an operating rule for multiple class loaders in Java, if a class loader needs to load a class, it will first delegate this class request to the parent class loader of that loader to complete, and so on for each layer, recursively up to the top layer.

Data warehouse access interface: is an API for fetching data from a data warehouse.

Hard coding: is a software development practice that embeds data directly into the source code of a program or other executable object, as opposed to obtaining the data from outside or generating the data at runtime. Hard-coded data can typically only be modified by editing the source code and recompiling the executable.

The principles and spirit of the present application are explained in detail below with reference to several representative embodiments thereof.

Summary of The Invention

With the development of internet technology, mass data analysis gradually becomes a necessary link for fine operation and product decision-making. In some data analysis scenarios, a caller typically needs to retrieve data from a data warehouse through a data warehouse access interface. Wherein the data warehouse access interface is an API for accessing data from a data warehouse.

However, due to the diversity of data analysis scenarios, the need for different invoker fetches is often different. To meet the requirements of different callers, developers often need to convert different access requirements into different data warehouse access interfaces to provide access services meeting the corresponding requirements for the callers through development of APIs.

In the traditional mode, research and development personnel usually analyze the access requirement first, then manually write a database connection code and an access logic code according to an analysis result, and deploy and test the generated access interface of the data warehouse in a production environment so as to finish delivery.

However, in this way, the development of the data warehouse access interface generally has the problems of complicated operation steps, low development efficiency and difficult multiplexing. Because ADS tables of a data warehouse are usually of different storage types, SQL syntax adopted in research and development is usually different, so in order to ensure logic correctness and better performance of an access interface of the data warehouse, a developer is usually required to repeatedly adjust codes, and deploy and test a server. The data processing code required in many different scenarios is typically consistent, such as post-processing and computing the query result set. However, since data developers of different service lines are generally different, multiplexing of data processing related code is difficult to achieve, resulting in repeated development.

Furthermore, in this way, there is often a problem that the delivery flow of the data warehouse access interface is long and the cost is high. This is due to the conventional data warehouse fetch interface delivery flow, which requires execution of server code development, offline online debugging and online publishing flows for the fetch requirements of each iteration version. The time to develop and deliver a data warehouse access interface typically takes days to schedule, even weeks, which typically severely impacts the landing of business. In addition, data warehouse access interfaces for different service lines are developed, and chimney type construction usually exists. If the data source on which a data warehouse access interface depends changes, and the access logic changes, the code of the data warehouse access interface must be modified at the server, and downlink debugging test and online release must be performed again, so that the change of the data source can result in a longer time window and obviously affect the use of a business party.

Further, in this manner, there is often a problem in that multiple types and versions of data sources are difficult to adapt. Since most tables of the data warehouse ADS layer are used for the impromptu query of the service system, different data source types are usually selected based on different scenes, for example, an operator has more use requirement on a wide table with large data volume (namely a database table with more fields) in a report analysis scene, so that the scene query involves large data volume but has lower query frequency, and is suitable for adopting a column database management system (Clickhouse) data source; for another example, some C-terminal applications need to recommend real-time according to user image data, and the scene data is usually queried according to Identification (ID) and stored in a key-value (key-value) form, so that performance requirements of high concurrency and low latency are required to be met, and data sources such as Hbase and dis are usually selected. However, different drivers are required to implement connectors of different data source types, and it is often difficult to be compatible with drivers of different versions of a unified data source, and development of a chimney may result in that data source connections cannot be uniformly managed, resulting in connection abuse.

In consideration of the fact that the data warehouse access interface codes can be disassembled to generate a plurality of access modules, access interface configuration interfaces are generated based on the access modules, and then developers can visually configure the data warehouse access interfaces through the access interface configuration interfaces to generate the data warehouse access interfaces, so that the problems that operation steps are complicated, research and development efficiency is low and multiplexing is difficult in developing the data warehouse access interfaces in a traditional mode are solved; and a deployment framework of a sandbox environment and an on-line environment multi-cluster can be built, the data warehouse access interfaces are subjected to centralized management and marketing, and access requests aiming at the data warehouse access interfaces are subjected to unified management through a data gateway, so that the problems of longer delivery flow and higher cost existing in the traditional mode of data warehouse access interface development are solved; the application embodiment provides a technical scheme for developing the data warehouse access interface, namely, an API developer can complete the access logic development of the API through a configuration service and a configuration and development mode, and because the data gateway has the capability of processing data API requests, the API developer can issue the developed data API into an API market through the function of the data gateway by using the API CI/CD to provide service system call and can also perform centralized management on all the data APIs through the data gateway. Furthermore, the data gateway is used as an access flow inlet of the data warehouse, which is unified to the outside, can process the call request and provide the service of API call for the upstream service, and has the basic functions of flow limiting fusing, authentication and audit. The API request parameters are analyzed and converted by the service logic layer, and finally become executable SQL, and the adapted data source connection drive is obtained through the plug-in routing table of the plug-in layer, so that SQL query is executed.

Having described the basic principles of the present application, various non-limiting embodiments of the present application are specifically described below.

Application scene overview

The embodiment of the application provides a method for developing a data warehouse access interface, which is applied to interface development equipment. The interface development device may be any electronic device of a device type suitable for implementation, for example, may be a terminal device, a server, etc., which will not be described in detail in this application.

Referring to fig. 1, an exemplary diagram of a data warehouse access interface development and publishing process provided in an embodiment of the present application is shown. The development flow of the data warehouse access interface is described below with reference to fig. 1.

Specifically, a developer firstly carries out API configuration, generates a data warehouse access interface, and obtains the data warehouse access interface in a state to be distributed. Then, through the unit test node, the internal unit test is carried out on the data warehouse access interface. And after the test of the unit is passed, the data warehouse access interface is issued in the sandbox environment through the sandbox environment node, and the data warehouse access interface issued in the sandbox environment is tested through the test node. And after the test is passed, formally releasing the data warehouse access interface in an online environment through the online release node.

Further, the update information can be configured, and the data warehouse access interface is updated, so that a new version of the data warehouse access interface is obtained.

Further, after the data warehouse access interface is released, API offline information can be configured, so that the data warehouse access interface is controlled to be offline.

Exemplary method

A method of developing a data warehouse access interface according to an exemplary embodiment of the present application is described below with reference to fig. 2 in conjunction with fig. 1. It should be noted that the above application scenario is only shown for the convenience of understanding the spirit and principles of the present application, and the embodiments of the present application are not limited in this respect. Rather, embodiments of the present application may be applied to any scenario where applicable.

Referring to fig. 2, which is a flowchart of a method for developing a data warehouse access interface, the method for developing a data warehouse access interface provided in the embodiment of the application includes the following steps:

step 101: and responding to the configuration operation of the developer for each parameter configuration item in the access interface configuration interface, and acquiring the parameter value of each parameter.

Alternatively, the configuration operation may include a parameter value input operation or a parameter value selection operation.

In one embodiment, a developer may visually configure a data warehouse access interface through an access interface configuration interface to generate the data warehouse access interface. The number-taking interface configuration interface displays a plurality of parameter configuration items, and a developer can execute parameter value input operation or parameter value selection operation and the like aiming at different parameter configuration items, so that parameter values of all parameters can be input. Referring to fig. 3, an exemplary diagram of an access interface configuration interface is shown, and a visual configuration of the access interface configuration interface is illustrated in fig. 3. The fetch interface configuration interface of fig. 3 includes a plurality of parameters, and a user may input parameter values corresponding to the parameters to generate the data warehouse fetch interface. Specifically, a database name and a data table name can be input in the access interface configuration interface to configure the from module, and a plurality of parameter names, corresponding binding fields and parameter types can be input in the access interface configuration interface to configure the select module, so that after a submit button is clicked, the data warehouse access interface can be generated.

In this way, the developer can configure the parameter values of various parameters of the data warehouse access interface to be developed in a visual configuration mode, and the data warehouse access interface is not required to be developed in a code writing mode.

Step 102: and generating a data warehouse access interface according to the corresponding relation between the access module and the parameters and the parameter values of the parameters.

The corresponding relation is generated by disassembling the data warehouse access interface codes according to the grammar rules of the structured query language (Structured Query Language, SQL). The data warehouse access interface is used for being called by a calling party after being released.

In one embodiment, the implementation of step 102 may include the steps of:

s1021: and generating a structured object corresponding to the access module according to the parameter value of the parameter corresponding to the access module.

In one embodiment, after the developer performs the visual configuration on the front end of the data warehouse access interface configuration, the front end of the data warehouse access interface configuration generates lightweight data exchange format (JavaScript Object Notation, JSON) data based on the access module and the parameter values of the corresponding parameters thereof, and sends the JSON data to the back end (e.g., the server). The data warehouse access interface configuration front end may be a world wide Web (Web) front end. And the rear end converts the JSON data to obtain a structured object corresponding to the number taking module.

Wherein the fetch module may include at least one of: a query (Select) module, a source (From) module, a screening (white) module, a Group by module, a paging module (Limit offset), and a sort by module.

Referring to FIG. 4, an exemplary diagram of a data warehouse access interface code is shown. FIG. 4 illustrates an example piece of data warehouse access interface code. The data warehouse access interface code of fig. 4 is disassembled, so that a plurality of access modules can be obtained. Specifically, select_expr [ … … ] in fig. 4 can be disassembled into SELECT modules, FROM table_references [ … … ] in fig. 4 can be disassembled into FROM modules, where_condition in fig. 4 can be disassembled into WHERE modules, GROUP BY { … … } in fig. 4 can be disassembled into GROUP BY modules, ORDER BY { … … } can be disassembled into ORDER BY modules, LIMIT { [ offset, ] … … } can be disassembled into LIMIT offset modules.

Wherein the structured object may comprise at least one of: results set objects, library table objects, condition screening objects, aggregation objects, paging objects, and ordering objects.

The system comprises a query module, a source module, a screening module, an aggregation module, a paging module and a sequencing module, wherein the corresponding structured objects are as follows in sequence: results set objects, library table objects, condition screening objects, aggregation objects, paging objects, and ordering objects.

The query module is used for querying the result set. The source module is used for configuring database information. The screening module is used for configuring the screening conditions of the query. The aggregation module is used for aggregation calculation. The paging module is used for paging inquiry. The ordering module is used for ordering the data based on the ordering field.

In this way, the data warehouse access interface code can be disassembled into a plurality of access modules so as to enable a developer to visually configure the data warehouse access interface code, and a plurality of structured objects are generated based on parameters respectively configured by the developer for the access modules.

Referring to FIG. 5, an exemplary diagram of a structured object is shown. The structured object and its associated parameters are described below in conjunction with fig. 5.

In fig. 5, the parameters corresponding to the result set object include a returned result parameter list; the parameters corresponding to the library table object comprise at least one of the following: database (db), data table, and database schema (schema); the parameters corresponding to the condition screening object comprise at least one of the following: inquiring a condition input parameter (input), a class attribute parameter (field) and a calculation parameter (cutoff); the parameters corresponding to the aggregation object comprise an aggregation parameter (group Key); the parameters corresponding to the paging object include at least one of: a limit query parameter (limit) and an offset query parameter (offset); the parameters corresponding to the ordering object include at least one of: an order Type parameter (order Type) and an aggregate order parameter (order By Key).

In one embodiment, the parameters may be classified according to the application mode of the parameters, and specifically, the types of the parameters may include request for entering a parameter, outputting a result parameter, and embedding a parameter. Request entry is used for data input, output result parameters are used for result output, and built-in parameters are used for system configuration.

The parameters of the library table object, the condition screening object and the aggregation object are built-in parameters, and the parameters of the paging object and the ordering object are paging entries, wherein the query condition entries and the paging entries are one of request entries.

S1022: based on the structured objects, a data warehouse access interface is generated.

The disassembly of the data warehouse access interface code is described below.

Since SQL is the core logic of the data warehouse access interface. The execution process of the data warehouse access interface can be generalized into request analysis, SQL (structured query language) assembly, query and result return, and the parameters of the data warehouse access interface can be divided into request input parameters, output result parameters and built-in parameters. Thus, the implementation of step 102 may further include:

according to SQL grammar rules, disassembling the data warehouse access interface codes to obtain a plurality of access modules, generating a corresponding relation between the access modules and parameters, and defining structured objects respectively corresponding to the access modules according to the corresponding relation.

The result set object corresponds to a SELECT clause in the SQL, which contains a query return field, and an API developer can generate the result set object only by configuring the naming and type of the return parameters of the data warehouse access interface.

Wherein, the library table object corresponds to a FROM clause of SQL. The metadata center can perform unified registration and management on each library table for determining the data source, and then an API developer can configure and generate library table objects by selecting the library tables from the metadata center.

Referring to FIG. 6, an exemplary diagram of a method of library table object construction is shown. In FIG. 6, a developer first registers a data source with a metadata center. And then, the developer configures the front end through the access interface of the data warehouse, opens the access interface configuration interface, and selects the candidate library table provided in the access interface configuration interface. The data warehouse access interface configuration front end generates corresponding JSON data based on a library table selected by a developer, and sends the JSON data to the server. The server builds a library table object based on the JSON data.

Therefore, the database table is more convenient to manage by registering and managing the database tables through the metadata center, and when the data warehouse access interface is configured, the database table is selected from the metadata center, so that a developer can intuitively inquire and trace the reference relation between the data warehouse access interface and the data source, and the problems of poor flexibility and difficult management in the traditional mode are avoided.

The condition screening objects comprise calculation types, target fields and joint operators, and developers configure the target fields, calculation types and request parameters of each condition expression object and the joint operators among the condition expression objects to form a parallel or nested structure. Then, the data warehouse access interface configuration front end converts the structure into a section of JSON character string and transmits the section of JSON character string to the server. Referring to fig. 7, an exemplary JSON string is shown. The JSON string of the condition screening object is illustrated in fig. 7. The JSON is a recursive nested structure. The subtjList in FIG. 7 is a sub-conditional JSON list, and may include a conditional expression object structure or a sub-conditional structure formed by associating multiple sets of conditional expressions. Through the recursion nested structure, the server can convert the JSON string into a conditional expression: WHERE ("A" = $ { param1} AND "B" LIKE "% $ { param2 }") OR ("C" IN% { param3} AND "D" <% { param4 }). Wherein,% { } is SQL placeholder, the content in { } is request entry, and the placeholder is replaced by request entry, so that the complete conditional expression can be obtained.

The aggregate object forms an aggregate query clause of the GROUP BY key1 and key2 BY configuring a key list of the GROUP.

The paging object needs to define an API paging entry, set a paging page number, a paging size, and a sorting field, bind the paging entry and the sorting field, and construct a limit.

Referring to FIG. 8, an exemplary diagram of data warehouse access interface development is shown. In fig. 8, for the output result parameters corresponding to the query module, performing the configuration of the return parameters, and constructing the result set object corresponding to the query module through the output result parameters corresponding to the query module; configuring a data source according to the built-in parameters corresponding to the source module, and constructing a library table object corresponding to the source module through the built-in parameters corresponding to the source module; configuring screening conditions aiming at built-in parameters corresponding to the screening module, and constructing a condition screening object through the built-in parameters corresponding to the screening module; configuring the aggregation parameters according to the built-in parameters corresponding to the aggregation modules, and constructing an aggregation object through the built-in parameters corresponding to the aggregation modules; and carrying out paging configuration on the paging entry corresponding to the paging module, and passing through the paging entry corresponding to the paging module. Constructing a paging object; and performing sequencing configuration on the paging entries corresponding to the sequencing module, and constructing sequencing objects through the paging entries corresponding to the sequencing module. And generating a data warehouse access interface based on each structured object. Further, when the data warehouse access interface is applied, code conversion can be carried out on the data warehouse access interface according to the access parameters in the data warehouse access interface request of the user, and the SQL statement corresponding to the data warehouse access interface is obtained.

In the embodiment of the present application, the arrangement order of the access modules in fig. 8 is merely taken as an example for illustration, and in practical application, the arrangement order of the access modules may be any order, which is not limited herein.

In the embodiment of the application, based on the structured object, the development flow of the configured data warehouse access interface is realized, the transition from the hard-coded development mode to the low-code configuration mode is realized, and the development efficiency of a data team is improved.

Further, after the data warehouse access interface is generated, the data warehouse access interface is published.

The whole delivery flow of the traditional data warehouse access interface is considered to be subjected to a plurality of links such as code writing, construction of a deployment server package, integrated release to a specific running environment and the like. The whole process is CI/CD facing to the code engineering of the service end, which is complex, time-consuming and low in fault tolerance. In the embodiment of the application, a life cycle management mode of an API (application program interface) based on meta-information issue is provided to realize the CI/CD (customer interface/compact disc) process of a data warehouse access interface, and the problems of long and inadequately agile delivery process of the traditional data warehouse access interface are solved.

The following describes a method for issuing a data warehouse access interface in an embodiment of the present application.

In one embodiment, in an interface runtime environment, a data warehouse access interface is published. Alternatively, the interface runtime environment may include a sandbox environment as well as an online environment.

Referring to FIG. 9, a flow chart of a method for publishing a data warehouse access interface is shown. The following describes a method for issuing a data warehouse access interface with reference to fig. 9, and the specific implementation flow of the method includes:

step 801: in a sandbox environment, a data warehouse access interface is published.

In one embodiment, after the data warehouse access interface configuration of the developer is used for generating the data warehouse access interface, the data warehouse access interface in a state to be distributed is obtained, the data warehouse access interface is distributed into a sandbox environment, and the data warehouse access interface in the sandbox environment is tested.

Furthermore, before the data warehouse access interface is tested, the data warehouse access interface can be tested by a unit, and after the unit test is determined to pass, the data warehouse access interface is tested.

Step 802: and if the data warehouse access interface is determined to be successfully tested, the data warehouse access interface is released in an online environment.

In one embodiment, if it is determined that the sandboxed environment test passes, a formal release may be applied. After approval by the API administrator, the data warehouse access interface is published to the online environment.

After the data warehouse access interface configuration is completed, the data warehouse access interface configuration can be released to an online environment after being tested by a service party, and in order to ensure the isolation of testing and online formal calling links, the interface operation environment of query service deployment based on a data gateway is divided into a sandbox environment and an online environment.

Wherein the sandboxed environment may include at least one sandbox cluster, and the online environment may include at least one online cluster. The query service may deploy multiple sets of clusters as Java virtual machine (Java Virtual Machine, JVM) processes. The query service can be deployed in any virtual machine environment such as a k8s cloud native environment and a virtual machine. The number of clusters and the number of instances of each cluster can be expanded and contracted according to the actual service requirement, which is not limited herein.

Further, in order to make the caller have no sense on the difference between the sandbox environment and the online environment as much as possible, so as to reduce the adapting cost of the caller, in the embodiment of the present application, a request routing scheme is provided, that is, by means of a data gateway, a data warehouse access interface in the sandbox environment or a data warehouse access interface in the online environment is selected to be operated.

In one embodiment, an interface development device includes a data gateway. As one example, the data gateway may be a hypertext transfer protocol (Hyper Text Transport Protocol, HTTP) gateway.

The data gateway is used for receiving the access request containing the access interface domain name sent by other equipment, and selecting between the sandbox environment or the online environment according to the environment parameters in the access request so as to call the data warehouse access interface under the selected interface operation environment; the access interface domain name is obtained after the data gateway performs instance registration.

Referring to FIG. 10, an exemplary architecture diagram of a data warehouse access interface distribution system is shown.

In fig. 10, the system includes a data gateway, a sandbox environment, and an online environment. The sandboxed environment includes sandbox cluster 1 and sandbox cluster 2. The online environment includes online cluster 1 and online cluster 2.

The cluster information in the sandbox environment and the online environment is a black box for the calling party. The data gateway is the only entry of the fetch request (i.e. API request) of the calling party, namely, all sandboxes and online clusters acquire the corresponding fetch request through a unified data gateway route.

In one embodiment, the data gateway performs instance registration to obtain the access interface domain name for calling by the caller. The calling party calls a fetch request to the data gateway based on the fetch interface domain name. After receiving the access request sent by other devices, the data gateway selects whether to call the data warehouse access interface in the sandbox environment or the online environment according to the environment parameters in the access request.

In one embodiment, the data Gateway's technology choice selects Kong Wangguan (Kong Gateway). Kong is an open source API Gateway project that is highly available and easily extensible based on openResity, and can configure the routing policy for downstream service calls. OpenResity is a high-performance web platform based on Nginx and Lua

As one example, the environment parameter is a Header, and if the value of the Header parameter is "sadbox" and the incoming route identifier "target", the fetch request is forwarded to the sandbox environment. If the Header parameter value is "online", the fetch request is forwarded to the online environment.

Thus, the routing policy may be configured based on Kong Gateway so that the data Gateway may determine the Header parameter of the fetch request (e.g., HTTP request) to determine the selected environment. The caller does not need to modify the request domain name (namely the fetch interface domain name) of the API, and can realize the switching between the sandbox environment test and the off-line environment formal call only by adjusting the transferred parameter value in the Header.

In the embodiment of the application, after the data warehouse access interface is developed and released, the data warehouse access interface can be presented in the data gateway API bazaar. And a caller of each service line can judge whether a data warehouse access interface meeting the current requirement exists or not by browsing the API bazaar. If the caller determines that the data warehouse access interface meeting the requirement exists, the application authority of the data warehouse access interface and the corresponding request secret key can be applied to the API administrator.

In this way, based on the API CI/CD flow, the mode transition from continuous integration of the server-side code to continuous integration of the API is realized. The deployment architecture of the sandbox environment and the online environment multi-cluster is built, centralized management and marketing are carried out on the data warehouse access interfaces, and access requests aiming at the data warehouse access interfaces are uniformly managed through the data gateway, so that the adaptation cost of an API calling party is reduced, and joint debugging and testing efficiency and demand delivery efficiency are improved. Through the data gateway, the data warehouse access interface is intensively developed and managed, and can be developed for one time and multiple times, so that the aim of multiplexing the API and the data source is fulfilled, and the chimney type development is avoided. And the labor is saved, and the data storage cost is reduced.

Further, after the data warehouse access interface is published, the data warehouse access interface may also be updated.

Referring to fig. 11, a flowchart of a method for updating a data warehouse access interface is shown, and a specific implementation flow of the method includes:

step 1000: and monitoring configuration update information of the data warehouse access interface.

The configuration update information is information for updating parameters in the data warehouse access interface. The configuration update information includes at least one of: parameters of a data warehouse access interface, SQL configuration information and generated JSON.

Step 1001: and in response to the presence of the configuration update information, updating the data warehouse access interface based on the configuration update information.

In one embodiment, when step 1001 is performed, the following steps may be employed:

s10011: if the storage root path of the configuration update information is determined to represent the sandbox environment, updating a data warehouse access interface in the sandbox environment;

optionally, the storage root path in the update information is configured, so that the storage root path can be named according to the interface running environment for subsequent distinction, and the updated data warehouse access interface can be selected through the storage root path in the update information.

S10012: and if the storage root path of the configuration update information is determined to characterize the online environment, updating the data warehouse access interface in the online environment.

Specifically, the interface development device may further include a management service and a coordination service. In connection with fig. 10, the api publishing system further includes a management service (Zookeeper).

When the data warehouse access interface is updated, configuration updating information of the data warehouse access interface in each interface operation environment is uniformly issued through the management service.

In one embodiment, the Zookeeper snoop is first registered through the query service. Then, when the data warehouse access interface is updated (i.e. updated), the developer can issue the meta information (i.e. configuration update information) of the data warehouse access interface to the meta information node in the Zookeeper through the management service. The interface running environment communicates with the Zookeeper through the query service, and monitors the meta-information change event through a Zookeeper Watch mechanism.

As one example, the storage root path of the sandboxed environment is the/dataservice/sadbox path and the storage root path of the online environment is the/dataservice/prd path. API nodes under the sandboxed environment listening/dataservice/sadbox path. An API node under the on-line ambience listening/dataservice/prd path. When the meta information is changed, each query service instance re-pulls the meta information of a specific API node under the monitoring path and updates the meta information to the memory, so that the upgrading of the data warehouse access interface can be realized, and further, a new API request can be processed.

Further, the data warehouse access interface can be controlled to be offline through API offline information.

In one embodiment, monitoring API offline information of a data warehouse access interface; the API offline information is information for indicating the offline of the data warehouse access interface; and controlling the data warehouse access interface to be offline based on the API offline information in response to the existence of the API offline information.

The distribution of the data warehouse access interface is also realized through the management service and the coordination service.

In one embodiment, monitoring API release information of a data warehouse access interface; the API release information is an access interface for releasing the data warehouse; and responding to the existence of the API release information, and releasing the data warehouse access interface based on the API release information.

In this embodiment of the present application, based on the same principle as updating the data warehouse access interface through configuration update information, the data warehouse access interface is controlled to be offline through API offline information, and the data warehouse access interface is published through API publishing information, which is not described herein.

In the embodiment of the application, if the API of the online environment needs to be changed, online configuration upgrading can be performed. The entire CI/CD cycle of the API flows only involving changes to the meta-information that originate from changes in the configuration. The developer does not need to pack and deploy codes of the whole server side in a corresponding physical environment, only needs to deploy a sandbox environment and an online environment at the same time, configures a page at a Web side, operates the release of a data warehouse access interface in the sandbox environment and a formal environment, and issues meta information to the corresponding environment, so that the upgrading of the data warehouse access interface can be realized. The data warehouse access interface requests sent to the data warehouse access interfaces of different interface operation environments are managed in a unified mode through the data gateway, configuration update information of the data warehouse access interfaces of different interface operation environments is managed in a unified mode through the management service, management of the whole life cycle of the data warehouse access interfaces is achieved, and the problems that the data warehouse access interface joint debugging test flow is long and the update cost is high are solved.

Further, to meet the needs of the ad hoc queries in different scenarios, several bins of data are usually imported into different data sources by data warehouse technology (ETL) to provide services for business side queries. Thus, the data gateway needs to have the capability of multi-type and multi-version data source queries. The traditional development mode is to introduce various driver Jar packages through Maven in open source framework (e.g. Springboot) engineering, and finally construct an engineering master Jar package. In querying the data sources, a data source connection needs to be created by the driver of each data source. However, this approach has a problem: because the data sources are various and have multiple versions, after various data source JAR packages are introduced, the engineering is bulky and huge, and the problem that the same data source JAR introduces different versions to cause conflicts easily occurs. When multiple versions are dependent against a conflict, only one of them can be selected, which can result in partial data sources being disconnected. For this reason, in the embodiments of the present application, the following improvements are also made to the data source configuration and connection.

In one embodiment, the interface development device further comprises an SQL adapter; the SQL adapter is used for converting the structured object in the data warehouse access interface into an SQL statement according to input parameters in access requests sent by other devices.

This is because there is a small difference in the SQL syntax of the various data sources, so the plug-in for each data source is correspondingly provided with an SQL adapter. The SQL adapters corresponding to the different plug-ins may be the same or different. Specifically, when the API executes the query logic, the structured object is converted into an SQL statement corresponding to the data source through the SQL adapter.

In one embodiment, the interface development device further comprises a driver routing table; the driver route table is registered with the drivers corresponding to the databases respectively; the driver program is used for establishing data source connection between the interface development equipment and the database corresponding to the access request; the data source connection is used for the data warehouse access interface to access the corresponding database.

The plug-in of the data source is a Jar package integrating a driver, an SQL customizing module and a connection management module.

In one embodiment, after loading the plugin, the driver of the plugin registers the driver routing table in the memory, and stores the driver routing table in a local cache manner (e.g., a Guava cache). As one example, the corresponding key of the driver in the driver routing table may be a driver type+version number, with value being the driver object.

The Driver class (i.e., class loader) can be used for loading a Driver of a data source of the SQL query, and the data source connection between the interface development device and the data source is established through the Driver according to Java database connection (Java DataBase Connectivity, JDBC) standard.

It should be noted that, in the conventional manner, the server side of the data gateway is developed based on JAVA language, and when loading JAVA class, a parent delegation mode is generally adopted for loading. Dependencies introduced in Maven are typically loaded by an application class loader (AppClassLoader). Because the class names of the same name are only loaded once, if the same driver package of different versions is introduced, the driver class of the same name is only loaded once, and coexistence of the multiple versions of the driver class cannot be realized, thereby causing class 'conflict'.

In order to break the limitation of class loading, in the embodiment of the application, the class loader is customized, and the class loader is generated by inheriting the class loader base class and rewriting the loadClass method, so that the parent delegation model is broken, and the driver is prevented from being loaded by the AppClassLoader. Therefore, the plug-in of each data source is correspondingly provided with a class loader, so that classes in plug-ins Jar of different versions can be loaded into the application, and the conflict is solved.

In one embodiment, the interface development device further comprises a storage management plug-in library; at least one plug-in containing a driver is stored in the storage management plug-in library; the storage path of the plug-in is generated according to the corresponding database type name and the data source version number.

Alternatively, the storage management plug-in library may employ a distributed file system (Hadoop Distribute File System, HDFS).

As one example, the storage root path for each plug-in is unified, beginning with/data-plug in, and the sub-path naming convention for each plug-in is./ { data source type name }/{ data source version number }. For example, mySQL 8.0 plugin, storage path is/data-plug in/MySQL/8.0/.

Therefore, the plug-ins can be uniformly stored and managed through the storage management plug-in library, and further the data gateway of each cluster can use the plug-ins.

In one embodiment, the interface development device further comprises HikariCP; hikariCP is used to manage the data source connections established for the databases.

In consideration of the creation time of the data source connection and the cost in performance, the data source connection established for each data source through the driver is uniformly managed through the connection pool. In the embodiment of the application, the HikariCP is taken as a connection pool in consideration of high performance and reliable stability of the HikariCP, and the created connection of each data source is initialized through the HikariCP and the connection of each data source is managed uniformly. Further, periodic security detection (i.e., health check) may also be performed on each data source by HikariCP. Therefore, the aim of multiplexing the data source connection can be achieved, and the cost in performance is reduced.

In the embodiment of the present application, hikariCP is only used as an example of a connection pool, and any other connection pool software may be used in practical application, which is not limited herein.

In one embodiment, the interface development device further comprises a database management center; the database management center is used for registering and managing each database.

Reference is now made to FIG. 11, which is an exemplary diagram of a database or data warehouse query process. The process of driver loading is described below in conjunction with fig. 12. And uniformly managing a plurality of plugins through a storage management plugin library, wherein each plugin comprises a driver. And respectively loading and storing the drivers in the corresponding plugins in the plugin library through a plurality of class loaders, and registering the drivers of the plugins in the memory into a driver routing table. When the data warehouse access interface is called, the type and the version number of the data source of the access request are determined, and a driver object (namely a driver corresponding to the data source) is obtained from a driver routing table according to the type and the version number of the data source, so that the data source connection is established with the requested database through the driver object. The database includes a data warehouse (Hive) and MySQL, among other things.

In the embodiment of the application, the data source is connected with the driver program to be inserted, the driver program routing table is established, and a plurality of versions and types of data sources can be supported, so that the types of the data sources selectable in various business scenes can be richer and more flexible.

Furthermore, if the adaptation of a new data source needs to be supported, only plug-in development is needed, the cost is low, the delivery is quick, after the plug-in test is passed, the plug-in can be uniformly managed through a plug-in library, and the purposes of primary development and global sharing are achieved.

Referring to FIG. 13, an exemplary architecture diagram of a development system for a data warehouse access interface is shown. In fig. 13, the system includes a traffic entry layer, a business logic layer, an execution engine layer, plug-in routing, connection data sources, API configuration services, and a relational database (Relational database management system, RDBMS).

The flow entry layer comprises a current limiting fusing component, gateway authentication and call auditing. The business logic layer comprises request entry resolution, mybatis and meta-information Cache (Cache). The execution engine layer comprises a connection pool cache, a plug-in adapter and a User-defined function (UDF) module. The plug-in layer includes plug-ins for a plurality of databases. The data source layer includes a plurality of databases. The API configuration service comprises API information management, API life cycle management, authority management and Zookeeper.

In one embodiment, the flow entry layer receives an access request sent by an upstream system, and sends the access request passing authentication to the service logic layer after authenticating the access request through gateway authentication. The business logic layer carries out parameter entering analysis on the access request through the request parameter entering analysis module, acquires a data warehouse access interface through the meta information cache, and sends the analyzed access request parameters and the data warehouse access interface to the execution engine layer. The execution engine layer carries out code conversion on the access request parameters and the access interfaces of the data warehouse through the plug-in adapter to obtain the access interface codes of the data warehouse, and judges whether the data source connection of the corresponding database exists in the cache of the connection pool or not based on the access request parameters. If yes, the established data source connection is directly obtained from the connection pool cache, otherwise, the data source connection between the data source connection and the database of the access request is established through the corresponding plug-in the plug-in layer, and the data warehouse access interface code is executed to access the data from the database of the access request through the data source connection.

Further, if it is determined that the data warehouse access interface is updated, the API developer issues configuration update information to the Zookeeper through API information management. The meta information cache acquires configuration updating information through the Zookeeper, and updates the data warehouse access interface based on the configuration updating information.

Optionally, the data warehouse access interface can be applied to a plurality of scenes such as data products, report analysis, client call and the like, and has the advantages of smaller response time and higher call success rate. The unified flow inlet of the data warehouse access interface access bin greatly improves the convenience and the access efficiency of the service system access.

In the embodiment of the application, based on the SQL structured object, the development flow of the configured data warehouse access interface is realized, and the problems of low development efficiency and poor reusability of the data warehouse access interface are solved based on the data warehouse access interface mart. Furthermore, the problem that the data API delivery flow is long and the updating cost is high is solved based on the API CI/CD flow issued by the meta-information. Furthermore, the data source connection driver is plugin, a driver route table is established, and the problems that the cost is high, the multiple data source versions cannot be compatible and the data source connection management is disordered when the data warehouse access interface is adapted to multiple data sources are solved.

Exemplary Medium

Having described the development method for the data warehouse access interface of the exemplary embodiment of the present application, next, the storage medium of the exemplary embodiment of the present application is described with reference to fig. 14.

The computer readable storage medium 1400 shown in fig. 14 is only one example and should not be construed as limiting the functionality and scope of application of the embodiments herein.

In an exemplary embodiment of the present application, the computer-readable storage medium 1400 may be a readable signal medium or a readable storage medium. On which a program product is stored which enables the implementation of the method described above. In some possible embodiments, the various aspects of the present application may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the present application as described in the above "exemplary method" section of the present application, when the program product is run on the terminal device.

More specific examples of the computer-readable storage medium 1400 in the present application may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

In this application, computer-readable storage medium 1400 may include a data signal propagated in baseband or as part of a carrier wave, with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Alternatively, program code embodied on computer readable storage medium 1400 may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

In some application scenarios, program code for performing the operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user electronic device, partly on the user device, partly on the remote electronic device, or entirely on the remote electronic device or server. In the case of remote electronic devices, the remote electronic device may be connected to the consumer electronic device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external electronic device (e.g., connected through the internet using an internet service provider).

Exemplary apparatus

Based on the same inventive concept, the embodiment of the application also provides a development device for the data warehouse access interface, and because the principle of solving the problem by the device and the equipment is similar to that of a development method for the data warehouse access interface, the implementation of the device can be referred to the implementation of the method, and the repetition is omitted.

Referring to fig. 15, a block diagram of a development device for a data warehouse access interface in an embodiment of the present application is shown. In some embodiments, a development device for a data warehouse access interface of the present application includes:

an obtaining unit 1501, configured to obtain parameter values of each parameter in response to a configuration operation of a developer for each parameter configuration item in the access interface configuration interface;

a generating unit 1502, configured to generate a data warehouse access interface according to a correspondence between the access module and the parameters and parameter values of the parameters; the corresponding relation is generated by disassembling the data warehouse access interface codes according to SQL grammar rules; the data warehouse access interface is used for being called by a calling party after being released.

In one embodiment, the generating unit 1502 is configured to:

In one embodiment, the generating unit 1502 is further configured to:

The generating unit 1502 is further configured to:

in a sandbox environment, a data warehouse access interface is issued;

In one embodiment, the interface development device includes a data gateway;

In one embodiment, the interface development device further comprises HikariCP;

In one embodiment, the generating unit 1502 is further configured to:

The development method for the data warehouse access interface comprises the steps of responding to configuration operation of a developer for each parameter configuration item in the access interface configuration interface, and obtaining parameter values of each parameter; generating a data warehouse access interface according to the corresponding relation between the access module and the parameters and the parameter values of the parameters; the corresponding relation is generated by disassembling the data warehouse access interface codes according to the SQL grammar rule of the structured query language; the data warehouse access interface is used for being called by a calling party after being released. Thus, complicated operation of data warehouse access interface development is simplified, and efficiency of data warehouse access interface development is improved.

Exemplary electronic device

Having described the development method, medium, and apparatus for a data warehouse access interface of an exemplary embodiment of the present application, an electronic device of an exemplary embodiment of the present application is described next with reference to fig. 16.

The electronic device 1600 shown in fig. 16 is merely an example, and should not be construed as limiting the functionality and scope of application of the embodiments herein.

As shown in fig. 16, the electronic device 1600 is embodied in the form of a general-purpose electronic device. The components of the electronic device 1600 may include, but are not limited to: at least one processing unit 1610, at least one memory unit 1620, a bus 1630 connecting the different system components (including the memory unit 1620 and the processing unit 1610).

Wherein the storage unit stores program code that is executable by the processing unit 1610 such that the processing unit 1610 performs steps according to various exemplary embodiments of the present application described in the above section of the "exemplary method" of the present application.

In some embodiments, the processing unit 1610 may perform the above-described embodiments.

The memory unit 1620 may include readable media in the form of volatile memory units, such as Random Access Memory (RAM) 16201 and/or cache memory 16202, and may further include Read Only Memory (ROM) 16203.

The storage unit 1620 may also include a program/utility 16204 having a set (at least one) of program modules 16205, such program modules 16205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.

Bus 1630 may include a data bus, an address bus, and a control bus.

The electronic device 1600 may also communicate with one or more external devices 1640 (e.g., keyboard, pointing device, bluetooth device, etc.) via input/output (I/O) interface 1650. Also, electronic device 1600 can communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet, through network adapter 1660. As shown in fig. 16, network adapter 1660 communicates with other modules of electronic device 1600 over bus 1630. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 1600, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.

From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a usb disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause an electronic device (may be a personal computer, a server, a terminal device, or a network device, etc.) to perform the method according to the embodiments of the present application.

It should be noted that while in the above detailed description, several units/modules or sub-units/modules of a development apparatus for a data warehouse access interface are mentioned, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more units/modules described above may be embodied in one unit/module in accordance with embodiments of the present application. Conversely, the features and functions of one unit/module described above may be further divided into ones that are embodied by a plurality of units/modules.

Furthermore, although the operations of the methods of the present application are depicted in the drawings in a particular order, this is not required to or suggested that these operations must be performed in this particular order or that all of the illustrated operations must be performed in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.

While the spirit and principles of this application have been described with reference to several particular embodiments, it is to be understood that this application is not limited to the disclosed particular embodiments nor does it imply that features in the various aspects are not useful in combination, nor are they intended to be in any way useful for the convenience of the description. The application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A development method for a data warehouse access interface, applied to an interface development device, comprising the following steps:

generating a data warehouse access interface according to the corresponding relation between the access module and the parameters and the parameter values of the parameters; the corresponding relation is generated by disassembling the data warehouse access interface codes according to the SQL grammar rule of the structured query language; the data warehouse access interface is used for being called by a calling party after being released.

2. The method of claim 1, wherein generating the data warehouse access interface based on the correspondence between the access module and the parameters and the parameter values of the parameters comprises:

generating a structured object corresponding to the number taking module according to the parameter value of the parameter corresponding to the number taking module;

and generating the data warehouse access interface based on the structured object.

3. The method of claim 2, wherein the structured objects comprise at least one of:

the parameters corresponding to the result set object comprise a returned result parameter list; the parameters corresponding to the library table object comprise at least one of the following: a database, a data table, and a database schema; the parameters corresponding to the condition screening object comprise at least one of the following: inquiring condition entry parameters, class attribute parameters and calculation parameters; the parameters corresponding to the aggregation object comprise aggregation parameters; the parameters corresponding to the paging object include at least one of the following: limiting the query parameters and offsetting the query parameters; the parameters corresponding to the ordering objects comprise at least one of the following: a sort type parameter and an aggregate sort parameter.

4. A method according to any one of claims 1-3, wherein the method further comprises:

and in an interface running environment, issuing the data warehouse access interface.

5. The method of claim 4, wherein the interface runtime environment comprises a sandbox environment and an online environment;

the issuing the data warehouse access interface in the interface operation environment comprises the following steps:

in the sandbox environment, issuing the data warehouse access interface;

and if the data warehouse access interface is determined to be successfully tested, the data warehouse access interface is released in the online environment.

6. The method of claim 5, wherein the interface development device comprises a data gateway;

the data gateway is used for receiving a fetch request containing a fetch interface domain name sent by other equipment, and selecting between the sandbox environment or the online environment according to the environment parameters in the fetch request so as to call the data warehouse fetch interface under the selected interface operation environment;

7. The method of claim 2, wherein the interface development device further comprises an SQL adapter;

The SQL adapter is used for converting the structured object in the data warehouse access interface into an SQL sentence according to input parameters in access requests sent by other devices.

8. A development apparatus for a data warehouse access interface, applied to an interface development device, the apparatus comprising:

the generation unit is used for generating a data warehouse access interface according to the corresponding relation between the access module and the parameter 5 and the parameter value of each parameter; the corresponding relation is generated by disassembling the data warehouse access interface codes according to the SQL grammar rule of the structured query language; the data warehouse access interface is used for being called by a calling party after being released.

9. An electronic device, comprising:

a processor; and

memory storing computer instructions for causing the processor to perform the method according to any one of claims 1 to 7.

10. A computer readable storage medium, characterized in that computer instructions for causing a computer to perform the method according to any one of claims 1 to 7 are stored.