CN119003629A - Method, apparatus, device and storage medium for managing data - Google Patents

Abstract

Embodiments of the present disclosure provide methods, devices, equipment, and storage media for data management. The method includes: in response to a data creation request for a target application, determining configuration information of a source data set imported from a first data source, the configuration information including access information of the first data source and a mapping relationship between fields in the source data set and fields used in the target application; based on the access information, obtaining at least a portion of the data in the source data set from the first data source; and creating a target data set used in the target application based on the mapping relationship and the obtained data. In this way, the application creator can easily import data from an existing data source into the data source of the target application, and the target application can provide better data services for end users.

Description

Method, apparatus, device and storage medium for managing data

Technical Field

Example embodiments of the present disclosure relate generally to the field of data processing and, more particularly, relate to a method, apparatus, device, and computer-readable storage medium for managing data.

Background

With the rapid development of informatization, terminal devices are deployed with more and more applications that can provide services to users. The application may be an application that provides communication services, an application that provides corporate attendance management services, and so on. The terminal equipment presents corresponding content through a user interface of the application and realizes interaction with the user so as to meet various service requirements of the user.

Implementation of some service functions in an application requires some data sources external to the data, i.e. third party data sources to support. The data are accessed to an application management platform, and are used for business statistics analysis after being processed to a certain extent, so that the data play a role in enabling businesses. Therefore, how to access the data of these third party data sources to the application management platform is of great importance.

Disclosure of Invention

In a first aspect of the present disclosure, a method of managing data is provided. The method comprises the following steps: determining configuration information of a source data set imported from a first data source in response to a data creation request for a target application, the configuration information including access information of the first data source and a mapping relationship between fields in the source data set and fields used in the target application; acquiring at least a portion of the data in the source data set from the first data source based on the access information; and creating a target data set for use in the target application based on the mapping relationship and the acquired data.

In a second aspect of the present disclosure, an apparatus for managing data is provided. The device comprises: a configuration information determining module configured to determine configuration information of a source data set imported from a first data source in response to a data creation request for a target application, the configuration information including access information of the first data source and a mapping relationship between fields in the source data set and fields used in the target application; a request receiving module configured to receive an access request of the target application to the shared data object; a request response module configured to access the at least a portion of the first data set from the first data source in response to the access request to perform an operation corresponding to the access request; and a response module configured to provide a result of the operation as a response of the shared data object to the access request.

In a third aspect of the present disclosure, an electronic device is provided. The apparatus comprises at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. The instructions, when executed by at least one processing unit, cause the electronic device to perform the method of the first aspect.

In a fourth aspect of the present disclosure, a computer-readable storage medium is provided. A medium has stored thereon a computer program which, when executed by a processor, implements the method of the first aspect.

It should be understood that what is described in this section is not intended to limit the key features or essential features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The above and other features, advantages and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. In the drawings, wherein like or similar reference numerals denote like or similar elements, in which:

FIG. 1 illustrates a schematic diagram of an example environment 100 in which embodiments of the present disclosure may be implemented;

FIG. 2 illustrates a data management system architecture schematic diagram in accordance with some embodiments of the present disclosure;

FIG. 3 illustrates a flowchart of an example process 300 of managing data, according to some embodiments of the present disclosure;

FIG. 4 illustrates a field mapping relationship page schematic provided by some embodiments of the present disclosure;

FIG. 5 illustrates a data creation request page schematic diagram according to some embodiments of the present disclosure;

FIG. 6 illustrates a data source selection page schematic diagram in accordance with some embodiments of the present disclosure;

FIG. 7 illustrates a data set information fill-in page schematic diagram in accordance with some embodiments of the present disclosure;

FIG. 8 illustrates an account authorization page schematic diagram in accordance with some embodiments of the present disclosure;

FIG. 9 illustrates a new account page schematic diagram according to some embodiments of the present disclosure;

FIG. 10 illustrates a new account page schematic diagram according to some embodiments of the present disclosure;

FIG. 11 illustrates a data set setup page diagram according to some embodiments of the present disclosure;

FIG. 12 illustrates a synchronous configuration page diagram for a source data set and a target data set in accordance with some embodiments of the present disclosure;

FIG. 13 illustrates an information viewing page schematic of a target dataset according to some embodiments of the present disclosure;

FIG. 14 illustrates an information viewing page schematic of a data synchronization task in accordance with some embodiments of the present disclosure;

FIG. 15 illustrates a history details page schematic diagram according to some embodiments of the present disclosure;

FIG. 16 illustrates a schematic diagram of a data management system architecture, according to some embodiments of the present disclosure;

FIG. 17 illustrates a detailed page schematic of various data sets in accordance with some embodiments of the present disclosure;

FIG. 18 illustrates a schematic block diagram of an apparatus for managing data according to some embodiments of the present disclosure;

Fig. 19 illustrates a block diagram of an electronic device in which one or more embodiments of the disclosure may be implemented.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure have been illustrated in the accompanying drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather, these embodiments are provided so that this disclosure will be more thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.

In describing embodiments of the present disclosure, the term "comprising" and its like should be taken to be open-ended, i.e., including, but not limited to. The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions are also possible below.

In this context, unless explicitly stated otherwise, performing a step "in response to a" does not mean that the step is performed immediately after "a", but may include one or more intermediate steps.

It will be appreciated that the data (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) involved in the present technical solution should comply with the corresponding legal regulations and the requirements of the relevant regulations.

It will be appreciated that prior to using the technical solutions disclosed in the embodiments of the present disclosure, the relevant users, which may include any type of rights subjects, such as individuals, enterprises, groups, etc., should be informed and authorized by appropriate means of the types of information, usage ranges, usage scenarios, etc. involved in the present disclosure according to relevant legal regulations.

For example, in response to receiving an active request from a user, prompt information is sent to the relevant user to explicitly prompt the relevant user that the operation requested to be performed will need to obtain and use information to the relevant user, so that the relevant user may autonomously select whether to provide information to software or hardware such as an electronic device, an application program, a server, or a storage medium that performs the operation of the technical solution of the present disclosure according to the prompt information.

As an alternative but non-limiting implementation manner, in response to receiving an active request from a relevant user, the prompt information may be sent to the relevant user, for example, in a popup window, where the prompt information may be presented in a text manner. In addition, a selection control for the user to select to provide information to the electronic device in a 'consent' or 'disagreement' manner can be carried in the popup window.

It will be appreciated that the above-described notification and user authorization process is merely illustrative and not limiting of the implementations of the present disclosure, and that other ways of satisfying relevant legal regulations may be applied to the implementations of the present disclosure.

As used herein, the term "model" may learn the association between the respective inputs and outputs from training data so that, for a given input, a corresponding output may be generated after training is completed. The generation of the model may be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs through the use of multiple layers of processing units. The neural network model is one example of a deep learning-based model. The "model" may also be referred to herein as a "machine learning model," "machine learning network," or "learning network," which terms are used interchangeably herein.

FIG. 1 illustrates a schematic diagram of an example environment 100 in which embodiments of the present disclosure may be implemented. The environment 100 relates to an application management platform 110 that may support the creation of applications and/or the running of applications. In some embodiments, the portion of the application management platform 110 that is used to support application creation may also be referred to as an application creation portion. In some embodiments, the portion of the application management platform 110 that is used to support application execution may also be referred to as an application execution portion.

As shown in fig. 1, the application creation section may provide the user 105 with a creation and release environment for the application. The user 105 may be referred to as an application creation user, creator. In some embodiments, the application creation portion may be a low code platform that provides a collection of tools for application creation. The application creation part can support visual development of various applications, so that a developer can skip the process of manual coding, and the development period and cost of the applications are quickened. The application creation portion may support any suitable platform for a user to develop one or more types of applications, and may include, for example, an application platform-based, i.e., service (aPaaS) platform. The platform can support the user to develop the application with high efficiency, and realize the operations of application creation, application function adjustment and the like.

The application creation portion may be deployed locally to the terminal device of the user 105 and/or may be supported by a server device. For example, a terminal device of user 105 may run a client with an application creation portion that may support user interactions with the application creation portion provided by the server. In the case where the application creation section runs locally at the user's terminal device, the user 105 can interact directly with the local application creation section using the terminal device. In the case where the application creation section operates on the server side device, the server side device may realize service provision to the client side operating in the terminal device based on the communication connection with the terminal device. The application creation section may present the corresponding page 130 to the user 105 based on the operation of the user 105 to output to the user 105 and/or receive information related to application creation from the user 105.

In some embodiments, the application creation section may be associated with a corresponding database in which data or information required for the application creation process supported by the application creation section is stored. For example, the database may store codes and description information and the like corresponding to the respective functional modules constituting the application. The application creation section may also perform operations such as calling, adding, deleting, updating, etc. on the function modules in the database. The database may also store operations that are performed on different functional blocks. For example, in a scenario in which an application is to be created, the application creation section may call corresponding function blocks from the database to build the application.

In an embodiment of the present disclosure, the user 105 may create the target application 120 on the application creation section as needed and issue the target application 120. The target application 120 may be published to any suitable application running portion as long as the application running portion is capable of supporting the running of the target application 120. After release, the target application 120 may be maintained by one or more users 165. User 165 may be referred to as an application operator user, operator of target application 120. The application execution portion may present the corresponding page 160 to the user 165 based on the operation of the user 165 to output to the user 165 and/or receive information related to application maintenance from the user 165.

After release, the target application 120 may be available for operation by one or more users 145. User 145 may be referred to as an end user of target application 120. In some embodiments, the target application 120 may include or be implemented as a digital assistant 122. The digital assistant 122 may be configured to have an intelligent dialog. In the example shown in FIG. 1, digital assistant 122 may be integrated within target application 120 as part of target application 120 to assist in performing task processing within target application 120. In other examples, digital assistant 122 may be configured as a stand-alone application, such as a web application or other type of application. In such an example, the digital assistant 122 may be considered the same application as the target application 120. The digital assistant 122 is provided to assist the user in various task processing needs for different applications, scenarios. During interaction with the digital assistant 122, the user enters an interaction message, and the digital assistant 122 provides a reply message in response to the user input. In general, the digital assistant 122 is capable of supporting a user to input questions in natural language and perform tasks and provide replies based on understanding and logical reasoning capabilities of the natural language input.

In some embodiments, digital assistant 122 may interact with user 145 as a contact. For example, digital assistant 122 may be implemented in an Instant Messaging (IM) application. Digital assistant 122 may interact with user 145 in a single chat session with user 145. In some embodiments, digital assistant 122 may interact with multiple users in a group chat session that includes multiple users.

For each user 145, the client of the application running portion may present an interactive window 142 of the target application 120 or digital assistant 122, such as a session window with the digital assistant 122, in the client interface. User 145 may enter a session message in the session window and target application 120 may determine a reply message to digital assistant 122 based on the created configuration information and present to the user in interactive window 142. In some embodiments, depending on the configuration of the target application 120, the interaction message with the target application 120 may include a multimodal form of message, such as a text message (e.g., natural language text), a voice message, an image message, a video message, and so forth.

Similar to the application creation section, the application execution section may be deployed locally at the terminal device of each user 145 and/or may be supported by a server device. For example, a terminal device of user 145 may run a client having an application running portion that may support user interaction with the application running portion provided by the server. In the case where the application run section is running locally at the user's terminal device, the user 145 may interact directly with the local application run section using the terminal device. In the case where the application running section runs on the server device, the server device may realize service provision to the client running in the terminal device based on the communication connection with the terminal device. The application execution portion may present the corresponding application page to the user 145 based on the operation of the user 145 to output to the user 145 and/or receive information related to the application usage from the user 145.

In some embodiments, the implementation of at least a portion of the functionality of the target application 120, and/or the implementation of at least a portion of the functionality of the digital assistant 122 in the target application 120, may be model-based. One or more models 155, such as the capabilities of the models 155, may be invoked during creation or execution of the target application 120. In target application 120, digital assistant 122 may utilize model 155 to understand user input and provide a reply to the user based on the output of model 155.

In the creation process, testing of the target application 120 by the application management platform 110 requires the use of the model 155 to determine that the results of the operation of the target application 120 are expected. During execution, the application execution portion may need to utilize model 155 to determine a response result to a user in response to different operation requests by the user of target application 120.

Although shown as being independent of the application management platform 110, one or more models 155 may run on the application management platform 110, or other remote servers. In some embodiments, the model 155 may be a machine learning model, a deep learning model, a neural network, or the like. In some embodiments, the model may be based on a Language Model (LM). The language model can have question-answering capability by learning from a large number of corpora. Model 155 may also be based on other suitable models.

The application management platform 110 may run on an appropriate electronic device. The electronic device herein may be any type of device having computing capabilities, including a terminal device or a server device. The terminal device may be any type of mobile terminal, fixed terminal, or portable terminal, including a mobile handset, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, media computer, multimedia tablet, personal Communication System (PCS) device, personal navigation device, personal Digital Assistant (PDA), audio/video player, digital camera/camcorder, positioning device, television receiver, radio broadcast receiver, electronic book device, game device, or any combination of the preceding, including accessories and peripherals for these devices, or any combination thereof. The server devices may include, for example, computing systems/servers, such as mainframes, edge computing nodes, computing devices in a cloud environment, and so forth. In some embodiments, the management platform 110 may be implemented based on cloud services.

It should be understood that the structure and function of environment 100 are described for illustrative purposes only and are not meant to suggest any limitation as to the scope of the disclosure. For example, while FIG. 1 shows a single user interacting with the application creation section and a single user interacting with the application execution section, in practice multiple users may access the application management platform 110 to each create a digital assistant, and each digital assistant may be used to interact with multiple users.

When creating a new application, there may already be some data sources available for the application. If the new application created is able to utilize the data in these data sources, the data sources may be enriched on the one hand and the data in these existing data sources may be made more valuable on the other hand.

According to an embodiment of the present disclosure, an improved management data scheme is provided. In this scheme, configuration information of a source data set imported from a first data source is determined in response to a data creation request for a target application. The configuration information includes access information of the first data source and a mapping relationship between fields in the source data set and fields used in the target application. At least a portion of the data in the source data set is obtained from the first data source based on the access information. Based on the mapping relationship and the acquired data, a target data set for use in the target application is created. In an embodiment of the present disclosure, a dataset used in a target application is created from an existing data source based on access information and a mapping relationship. In this way, the subsequent target application may provide various suitable data services to the end user based on the target data set. This way of creating a data set can enrich the data sources on the one hand and make the data in these existing data sources of greater value on the other hand. In this way, the application creator can conveniently import data from an existing data source into the data source of the target application, while the target application can provide better data services to the end user.

Some example embodiments of the present disclosure will be described in detail below with reference to examples of the accompanying drawings. It should be understood that the pages shown in the drawings are merely examples, and that various page designs may actually exist. Individual graphical elements in a page may have different arrangements and different visual representations, one or more of which may be omitted or replaced, and one or more other elements may also be present. Embodiments of the disclosure are not limited in this respect.

Fig. 2 illustrates a schematic diagram of a data management system architecture, according to some embodiments of the present disclosure. An exemplary overall scheme of data management in some embodiments of the present disclosure is now described with reference to fig. 2.

The application management platform 110 is used to build a target application 120. The target application 120 needs to support data independent of the data source (also referred to as a third party data source or an external data source) of the application management platform 110 in order to implement the corresponding function. In this example, the third party data sources include data source A210, data source B220, and data source C230.

The application management platform 110 is deployed with a data integration component, which mainly realizes functions of data integration, credential management, connector ecology, and the like. The data integration component has the data integration link capacity, can solve the complex configuration, data conversion, increment synchronization and other capacities of large batch of data in the synchronization process, and shields the data reading mode difference and complex logic of different service applications.

The architecture mainly comprises two scenes of a construction state and an operation state of the target application, wherein the construction state is a construction state of the target application, and the operation state is used for releasing the constructed target application.

In the build state, the application management platform 110 first selects an external data source, i.e., one or more data sources from among the third party data sources. And then carrying out account authorization to determine whether the selected data source is authorized to be accessed. If the authority is available, selecting a data object, and setting a screening condition to select a part of data from the data set of the selected data source according to the screening condition. And according to the mapping relation between the fields of the data set and the fields of the target data set, storing a part of data selected from the data set in the target data set according to a preset synchronous task requirement.

In the run state, the application management platform 110 may store data selected by the third party data source in the target data set in two ways, one is one-time import and the other is timing delta synchronization.

For a clearer description of the present disclosure, the following embodiments are further described with respect to the various processes depicted in fig. 2.

Fig. 3 illustrates a flowchart of an example process 300 of managing data, according to some embodiments of the present disclosure. For ease of discussion, the process 300 will be described with reference to FIG. 1. The process 300 may be implemented, for example, at the application management platform 110.

In response to the data creation request for the target application 120, the application management platform 110 determines configuration information for the source data set imported from the first data source at block 310. The configuration information includes access information of the first data source and a mapping relationship between fields in the source data set and fields used in the target application 120. The first data source may be selected by the user 105 from a third party data source.

In response to receiving the target application 120 creation request, the application management platform 110 determines configuration information for the source data set imported from the first data source. The first data source is a third party data source. The configuration information includes access information of the first data source and a mapping relationship between fields in the source data set and fields used in the target application 120.

The access information in the configuration information is used to access the first data source. In general, the third party data source generally includes a plurality of data sources, and in order to determine which data source needs to obtain data from, the access information needs to be configured with the data source where the data to be obtained is located, that is, the identification information of the first data source. The identification information may include an access account number for the first data source, or an access path for the first data source, and so on.

The data used by the first data source and the target application 120 may have inapplicable problems, e.g., may be identified by different fields for the same content. Then the mapping relationship between the fields in the source dataset and the fields used in the target application 120 is also included in the configuration information for adaptation. The mapping is used to save fields in the source dataset in a manner more appropriate for the target application 120.

In some embodiments, the mapping relationship may be a predetermined mapping between a first field in the source dataset and a second field used in the target application 120. For example, the application management platform 110 may use a predefined or default field mapping relationship.

To increase mapping flexibility, in some embodiments, the mapping relationship may also be a user-specified mapping between a third field in the source dataset and a fourth field used in the target application 120. That is, the user may modify the mapping between the third field in the source dataset and the fourth field used in the target application 120 as desired.

It should be noted that, since the fields in the target data set are preset (for example, are common in the target application), the fields in the target data set are generally not modifiable by default, and each field is mapped to which field in the third party data source is modifiable.

Fig. 4 illustrates a field mapping relationship page schematic provided by some embodiments of the present disclosure, and is now described with respect to fig. 4. The field mapping relationship page 400 shows the mapping relationship between the fields in the source data set and the fields used in the target application 120. Specifically, field A410 in the source data set corresponds to field AA in the target data set, and field B420 in the source data set corresponds to field B in the target data set. In some embodiments, fields AA and BB may not be modified, but the fields to which they are mapped may be modified.

To obtain configuration information, in some embodiments, the application management platform 110 may provide a data source selection page in response to a data creation request; receiving, via the data source selection page, a selection of a first data source of the plurality of data sources; providing an access information input page for a first data source; receiving an indication of access information for a first data source via an access information input page; providing a data set setting page for a first data source; and receiving, via the data set settings page, a data selection of at least a portion of the source data set in the first data source.

An example is described with reference to fig. 5 to 10. As shown in fig. 5, a control that triggers a data creation request, such as a "import from outside" control 510, is provided on the data creation request page 500. The user may click on the control 510 to trigger a data creation request, at which point the application management platform 110 receives the data creation request.

Upon receiving the data creation request, the application management platform 110 may provide a data source selection page 600. All selectable data source options are provided in the data source selection page 600. As shown in fig. 6, alternative data sources include: data source A210, data source B220, and data source C230. The user may select a first data source from the plurality of data sources by clicking a control on the data source selection page for the data source to be selected. For example, data source a is selected as the first data source.

It should be noted that, after the first data source is selected, the page is filled in based on the data set information, and the basic information of the target data set is set. As shown in fig. 7, the user needs to input corresponding setting information in the input box 701 of the name corresponding to the data set, the input box 702 of the API name, and the input box 703 corresponding to the description, respectively, and click the "next" control 710, before the target data set that can be used for storing data can be constructed.

For security, after the first data source is selected, account authorization operation is required before the content contained in the first data source is displayed on the page. Specifically, the application management platform 110 provides an account authorization page, and the first data source is taken as the data source a 210 for illustration. If the account number of the data source a 210 has not been added before, an account number authorization page 800 shown in fig. 8 is provided, and an "add account number" control 810 is provided on the account number authorization page 800. After the user clicks the "Add Account" control 810 on the Account authorization page 800 and clicks the "Next" control, the application management platform 110 will provide the new Account page 900 as shown in FIG. 9. The newly-built account page 900 displays setting reminding information such as an account, a password, a host, a computer port, a data source and the like. And the user can input information through the corresponding setting boxes under the setting reminding, and finally click a 'save and connect' control to realize account authorization. The account number and password entered by the user 105 may be used as access account information in the access information, and the entered "host" and "computer port" and "data source" may be used as access path information in the access information.

If the account number of data source A210 was previously added, the application management platform 110 will provide the account number authorization page 1000 shown in FIG. 10. The account authorization page 1000 is provided with all authorized account information. For example, the authorized accounts in fig. 10 include account a 1010, account B, account C, account D, and account E. If the user wants to select the account a 1010 to log in, the user can click on the control corresponding to the account a 1010.

After the first data source is selected and account authorization is completed, the application management platform 110 may provide a data set setup page for the first data source. In the data set setup page, the user may select data for at least a portion of the source data set in data source A210 (which is the first data source selected at this time) by clicking or other triggering operation. At this point, the application management platform 110 receives a data selection of at least a portion of the source data set in data source A210. All data in the source data set can be selected, and non-all data in the source data set can be selected based on a preset selection rule, and it is noted that the selected non-all data is not null.

To determine which data in the source data set to select, the application management platform 110 may present at least one field in the source data set in the data set settings page 1100. As shown in FIG. 11, the selectable fields under the source data set in data source A210 include field A410, field B420, field C, and field D, among others. The user 105 may click on the controls of the various fields on the dataset settings page 1100. At this time, the application management platform 110 may obtain one or more field selections of the at least one field as the field to be imported. For example, if the page 1100 is set based on the data set shown in fig. 11, and the user clicks on the control of the field a410 and the control of the field B420, the imported fields are the field a410 and the field B420.

Based on the access information, the application management platform 110 obtains at least a portion of the data in the source data set from the first data source at block 320. At block 330, the application management platform 110 creates a target data set for use in the target application 120 based on the mapping relationship and the acquired data.

After at least a portion of the data in the source data set and the mapping relationship are obtained from the first data source, a target data set for use in the target application 120 may be created based on the mapping relationship and the obtained data. The target data set contains the obtained data, and for each field contained in the obtained data, the mapping field corresponding to the field determined based on the mapping relation is stored in the target data set, and the field value under the field is stored under the mapping field.

In some embodiments, to determine when to synchronize data for a source data set with a target data set for data management, the application management platform 110 may also provide a synchronization configuration page for the source data set and the target data set. The synchronization configuration page is used for configuring data synchronization tasks between the source data set and the target data set. Then, via the synchronization configuration page, the application management platform 110 may receive synchronization information for the data synchronization task as part of the configuration information.

In some embodiments, the synchronization information includes at least one of: the frequency at which data in the target data set is periodically synchronized with data in the source data set, or the time range at which data in the target data set is first synchronized with data in the source data set. In addition, the synchronization information may also include execution date, execution time, and the like.

An example is described with reference to fig. 12. Fig. 12 shows a synchronization configuration page 1200 for a source data set and a target data set. In the synchronization configuration page 1200, a plurality of setting options such as "execution frequency of synchronization task", "monthly execution date", "execution time", and "time range of first synchronization" are presented, as shown in fig. 12. For the setting option of "execution frequency of synchronous task", the user can click on any one of the control corresponding to "daily", the control corresponding to "weekly" and the control corresponding to "monthly" to realize the selection of the execution frequency of synchronous task. The user can also input the corresponding date in the input box under the "monthly execution date" to realize the selection of the execution date. For example, "1 day" is input under the input box to realize the setting that No. 1 of each month realizes the execution of the data synchronization task. The selection of the execution time of the execution day can also be realized by inputting the corresponding time in the input box under the execution time. For example, "12:00" is entered under the input box to enable setting of 12:00 for each day of execution to enable execution of the data synchronization task. In addition, because the synchronization meaning of the data far from the current time is not too great, in order to reduce the workload and improve the synchronization efficiency, the corresponding first synchronization time can be input in the input box under the 'first synchronization time range'. For example, "2022-12-29" is entered under the input box to achieve synchronization for data only for 2022-12-29 to the present time of day.

The target application 120 will be published after the build is completed. In some embodiments, after the target application 120 is published, the application management platform 110 provides launch controls for data synchronization tasks between the source data set and the target data set in pages associated with the target data set. And in response to the start control being triggered, performing the data synchronization task based on the synchronization information of the data synchronization task.

For a target data set created by importing data based on a third-party data source, the data synchronization is not automatically started by default after the online data is released, and related personnel are required to manually start a synchronization task. The relevant person may be a user 165, e.g., an application operator user, operator of the target application 120.

To initiate a data sync task, the user may click on a launch control for the data sync task between the source data set and the target data set in a page associated with the target data set, at which point the application management platform 110 performs the data sync task based on the synchronization information of the data sync task.

In some embodiments, a page associated with the target data set may be a informational view page of the target data set. An example is described with reference to fig. 13. As shown in FIG. 13, the information view page 1300 of the target data set has displayed thereon the name "XXX data" of the target data set, which is at least a portion selected from the source data set of data source A. The current startup state is displayed in the information view page 1300 of the target data set. If the active state is an inactive state, then an "active" control 1320 is provided on the information viewing page 1300 for the target dataset. When the user 165 clicks on the "start" control 1320 in the information view page of the target dataset, the application management platform 110 performs the data synchronization task based on the synchronization information (e.g., task information configured in the build state) of the data synchronization task.

In some embodiments, pages associated with the target data set may be information-viewing pages for the data-synchronization task. As shown in fig. 14, in this example, the current startup state is displayed in the information view page 1400 of the data synchronization task. If the startup state is the un-startup state, the information viewing page 1400 of the data synchronization task is provided with a "startup" control 1410. When the user clicks the "start" control 1410, the application management platform 110 performs a data synchronization task based on the synchronization information of the data synchronization task.

It should be noted that, if the current start state is the start state, the "start" control 1410 in the information view page 1400 of the data synchronization task may be replaced by the "stop" control. When the user clicks the "stop" control, the application management platform 110 stops performing the data synchronization task.

In some embodiments, to facilitate prompting an associated party to synchronize task execution, the application management platform 110 may also send a prompting message associated with execution of the data synchronization task to an associated party of the target application 120. In some embodiments, if a preset operation of the prompt message by the associated party is received, an information viewing page of the target data set is provided. The party associated with the target application 120 may include a creator, of the target application 120. The correspondents of the target application 120 may also include application-dimension users, dimension operators, of the target application 120.

Specifically, when the target application 120 starts synchronization for the first time, the application management platform 110 sends a message for starting the first synchronization to the associated party. The message for starting the first synchronization comprises the type of the message, the identification of the data set of the data synchronization, the identification of the initiator, the identification of the third party data source of the data synchronization and the like.

If the first synchronization is started, the application management platform 110 sends a message for starting the first synchronization to the associated party. The message for starting to perform the first synchronization includes the type of the message, the identification of the data set of the data synchronization, the identification of the third party data source of the data synchronization, the start time, and the like.

If the first synchronization is completed, the application management platform 110 sends a message for completing the first synchronization to the associated party. The message for completing the first synchronization comprises the type of the message, the identification of the data set of the data synchronization, the identification of the third party data source of the data synchronization, the starting time, the ending time and the like.

Specifically, the application management platform 110 may send a prompt message associated with the execution of the data synchronization task to a terminal device corresponding to the associated party. The alert message may be provided and presented in the form of a message card. After the related party views the prompt message related to the execution of the data synchronization task displayed by the terminal device, the related party can perform preset operation on the prompt message on a display page of the terminal device. The preset operation may be to click on the alert message or other operation. The associated device may be a device independent of the device in which the application management platform 110 is located, or may be a device in which the application management platform 110 is located. The application management platform 110 provides an information view page of the target data set, such as the information view page 1300 shown in fig. 13, in response to a preset operation of the prompt message by the associated party.

To further facilitate the viewing of data synchronization by the correspondents, in some embodiments, the application management platform 110 may also provide a data synchronization history details page. Specifically, the user may click on the "view more" control 1350 at the top right corner of the information view page 1300 of the target dataset, which may display a drop down menu. The drop down menu contains a "data synchronization history" control 1360. The user 165 opens the data history details page after clicking on the "data synchronization history" control 1360. For example, a history details page 1500 shown in fig. 15, a record in which data of total synchronization is recorded in the history details page 1500, the number of synchronization failures, and the like. In addition, the user 165 may click on an "operation" control in the history details page 1500 to view a detailed error log of the data sync. The detailed error log includes an error log recording time point, failed recording data (JSON format), a failure cause, and the like. In order to reduce the storage pressure of the application management platform 110, the application management platform 110 is set to only store a recent preset number of detail error logs or detail error logs of preset days. The preset number may be 800, 1000, etc., and the preset number of days may be 30, 60, etc. Specifically, the preset number and the preset number of days can be set according to the requirements.

Furthermore, a "re-import" control 1340 is provided on the information view page 1300 for the target data set created from the import of the third party data set before the target application 120 is published, that is, while the target application 120 is in the build state. Clicking on the "re-import" control 1340 by the user may trigger re-importation of test data from a first data source set at the time of creation of the target data set, which may overwrite existing data of the target data set.

To facilitate the associator to view the data synchronization over a period of time for data statistics and analysis, in some embodiments, the application management platform 110 may also perform incremental data synchronization message pushing at a preset period. By way of example, message pushing may be performed on a day level, message pushing may be performed on a month level, and so forth. For example, at 10 am each day, the data delta synchronization record for the day preceding the pushing of the target data set. The data increment synchronization record records the total amount of data increment of the previous day, whether the data is abnormal or not, and the like.

In some embodiments, after the target application 120 is published, the application management platform 110 may also receive an update to the mapping relationship between the fields in the source data set and the fields used in the target application 120 and update the created target data set based on the updated mapping relationship.

Fig. 16 is a schematic diagram of a data management system according to some embodiments of the present disclosure, and the description of fig. 16 will be now provided. The application management platform 110 may build and manage the target application 120. The target application 120 may include one or more functions such as Business Intelligence (BI) functions related to data analysis, add-drop-Change (CRUD) functions on data, question-answer (QnA) functions, and others.

In some embodiments, for a data set in data source 1630, application management platform 110 can create part or all of the data set as a shared data object for use by target application 120. For example, CRUD functionality in target application 120 may need to access the shared data object for read and write operations. An access operation initiated with respect to a shared data object is applied to the data set represented by the shared data object to read and write the data set stored in the first data source.

In some embodiments, the application management platform 110 may import data in the third party data source 1610. Specifically, portions of the data in the third party data source 1610 are data integrated and then synchronized into the data set 1620 of the application management platform 110 so that applications supporting BI functionality read the data in the data set 1620 to serve users. For example, the dataset 1620 is constructed and maintained as described above with reference to fig. 2-15.

In some embodiments, the application management platform 110 may further vectorize and upload uploaded cloud documents (Docs), PDF, PPT, and other types of data, various types of data objects, existing data sets, and the like, to a knowledge base. Data for other service desks is stored in the knowledge base of the application management platform 110 directly based on API calls to support QnA functional application service subscribers.

In some embodiments, because the data sources for different data sets are different, the data sources for each data set may be displayed in each data set list page 1700. As shown in fig. 17, for the data set 1, the corresponding data source is the object source, specifically, the data source is the object 1. For data set 2, the corresponding data source is the data stream source, specifically, the data source is data stream 1. For the data set 3, the corresponding data source is an external source, specifically, the data source is the data source 3.

In some embodiments, in the creation of target application 120, user 105 may configure target application 120 to invoke a machine learning model, such as model 155. For example, one or more functions that the target application 120 has may invoke the machine learning model. In this way, after the target application 120 is published, the machine learning model may be utilized to provide results for the user query. In particular, if a user query is received, information indicative of at least a portion of the target data set may be provided to the machine learning model based on the user query. Results associated with the provided portion for the user query may then be obtained from the machine learning model. As an example, if a user queries for partial data indicating a target data set of an object to analyze, information indicating the portion to be analyzed may be provided to a machine learning model. Accordingly, the machine learning model may output the analysis of the portion of data.

In some embodiments, the application management platform 110 may generate a data object associated with the target application based on at least a portion of the target data set. If an access request for a data object is received by a target application, the application management platform 110 may perform an operation with the access request object on the data object.

The application management platform 110 may use all data of the target data set as data objects associated with the target application, or may use non-all data of the target data set as data associated with the target application, but it is necessary to ensure that the non-all data is not null. The data objects associated with the target application are the data objects required for the target application to implement some functional service.

After receiving the access request of the target application to the data object, the application management platform 110 performs an operation corresponding to the access request on the data object. The access request may be a request to create a data item, a request to read a data item, a request to update a data item, a request to delete a data item, and so forth.

Specifically, if the access request is a request for creating a data item, the data item to be created is created in the data object based on the data item to be created carried in the access request. If the access request is a request for reading the data item, reading a data item value corresponding to the data item to be read in the data object based on the data item to be read carried in the access request. If the access request is a request for updating the data item, updating the data item before updating into the data item to be updated in the data object based on the data item to be updated carried in the access request. If the access request is a request for deleting the data item, deleting the data item to be read in the data object based on the data item to be deleted carried in the access request.

Fig. 18 illustrates a schematic block diagram of an apparatus 1800 for managing data according to some embodiments of the present disclosure. The apparatus 1800 may be implemented in or included within the application execution platform 140, for example. The various modules/components in the apparatus 1800 may be implemented by hardware, software, firmware, or any combination thereof.

As shown, the apparatus 1800 includes a configuration information determination module 1810 configured to determine configuration information of a source data set imported from a first data source in response to a data creation request for a target application, the configuration information including access information of the first data source and a mapping relationship between fields in the source data set and fields used in the target application. The apparatus 1000 further comprises a data acquisition module 1820 configured to acquire at least a portion of the data in the source data set from the first data source based on the access information. The apparatus 1000 further comprises a data set creation module 1830 configured to create a target data set for use in the target application based on the mapping relation and the acquired data.

In some embodiments, the configuration information determination module 1810 is configured to provide a data source selection page in response to a data creation request; receiving, via the data source selection page, a selection of a first data source of the plurality of data sources; providing an access information input page for a first data source; receiving an indication of access information for a first data source via an access information input page; providing a data set setting page for a first data source; and receiving, via the data set settings page, a data selection of at least a portion of the source data set in the first data source.

In some embodiments, the configuration information determination module 1810 is configured to: receiving a selection of a source dataset via a dataset selection control in a dataset setting page; presenting at least one field in the source dataset in a dataset settings page; and receiving, via the dataset settings page, one or more field selections from the at least one field as fields to import.

In some embodiments, the access information indicates at least one of: an access account number for the first data source, or an access path for the first data source.

In some embodiments, the apparatus 1800 further comprises: the page providing module is configured to provide a synchronous configuration page aiming at the source data set and the target data set, wherein the synchronous configuration page is used for configuring a data synchronous task between the source data set and the target data set; the apparatus 1800 further comprises: and the synchronization information receiving module is configured to receive the synchronization information for the data synchronization task as a part of the configuration information through the synchronization configuration page.

In some embodiments, wherein the synchronization information includes at least one of: the frequency at which data in the target data set is periodically synchronized with data in the source data set, or the time range at which data in the target data set is first synchronized with data in the source data set.

In some embodiments, wherein the mapping relationship comprises at least one of: a predetermined mapping between a first field in the source data set and a second field used in the target application, or a user-specified mapping between a third field in the source data set and a fourth field used in the target application.

In some embodiments, the apparatus 1800 further comprises: a launch module configured to provide launch controls for data synchronization tasks between the source data set and the target data set in pages associated with the target data set after the target application is published; the apparatus 1800 further comprises: and the task execution module is configured to execute the data synchronization task based on the synchronization information of the data synchronization task in response to the start control being triggered.

In some embodiments, wherein the page associated with the target dataset includes at least one of: information viewing pages of the target data set or information viewing pages of the data synchronization task.

In some embodiments, the apparatus 1800 further comprises: the information prompt module is configured to send a prompt message associated with the execution of the data synchronization task to an associated party of the target application; and responding to the preset operation of the association party on the prompt message, and providing an information viewing page of the target data set.

In some embodiments, the apparatus 1800 further comprises: the updating module is configured to receive the updating of the mapping relation after the target application is released; and updating the created target data set based on the updated mapping relationship.

In some embodiments, the apparatus 1800 further comprises: a data object generation module configured to generate a data object associated with the target application based on at least a portion of the target data set; a task execution module configured to receive an access request of the target application to the data object; and performing an operation corresponding to the access request on the data object.

In some embodiments, the apparatus 1800 further comprises: the query receiving module is configured to receive a user query aiming at the target application after the target application is issued; a data providing module configured to provide at least a portion of the target data set to a machine learning model based on the user query; and a result acquisition module configured to acquire results for the user query associated with the at least a portion from the machine learning model.

Fig. 19 illustrates a block diagram of an electronic device 1900 in which one or more embodiments of the disclosure may be implemented. It should be understood that the electronic device 1900 illustrated in fig. 19 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 1900 shown in fig. 19 may include or be implemented as the application management platform 110, a terminal device, a server device of fig. 1, or the apparatus 1800 of fig. 18.

As shown in fig. 19, the electronic device 1900 is in the form of a general-purpose electronic device. Components of electronic device 1900 may include, but are not limited to, one or more processors or processing units 1910, memory 1920, storage devices 1930, one or more communication units 1940, one or more input devices 1950, and one or more output devices 1960. The processing unit 1910 may be a real or virtual processor and is capable of performing various processes according to programs stored in the memory 1920. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of electronic device 1900.

Electronic device 1900 typically includes multiple computer storage media. Such a medium may be any available media that is accessible by electronic device 1900, including, but not limited to, volatile and non-volatile media, removable and non-removable media. The memory 1920 may be volatile memory (e.g., registers, cache, random Access Memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 1130 may be a removable or non-removable medium and may include a machine readable medium such as a flash drive, a magnetic disk, or any other medium which may be used to store information and/or data and which may be accessed within the electronic device 1900.

Electronic device 1900 may further include additional removable/non-removable, volatile/nonvolatile storage media. Although not shown in fig. 19, a magnetic disk drive for reading from or writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data medium interfaces. The memory 1920 may include a computer program product 1925 having one or more program modules configured to perform the various methods or acts of the various embodiments of the present disclosure.

The communication unit 1940 enables communication with other electronic devices through a communication medium. Additionally, the functionality of the components of the electronic device 1900 may be implemented in a single computing cluster or in multiple computing machines capable of communicating over a communication connection. Thus, electronic device 1900 may operate in a networked environment using logical connections to one or more other servers, a network Personal Computer (PC), or another network node.

The input device 1950 may be one or more input devices such as a mouse, keyboard, trackball, etc. The output device 1960 may be one or more output devices such as a display, speakers, printer, etc. The electronic device 1900 may also communicate with one or more external devices (not shown), such as storage devices, display devices, etc., with one or more devices that enable a user to interact with the electronic device 1900, or with any device (e.g., network card, modem, etc.) that enables the electronic device 1900 to communicate with one or more other electronic devices, as desired, via the communication unit 1940. Such communication may be performed via an input/output (I/O) interface (not shown).

According to an exemplary implementation of the present disclosure, a computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method described above is provided. According to an exemplary implementation of the present disclosure, there is also provided a computer program product tangibly stored on a non-transitory computer-readable medium and comprising computer-executable instructions that are executed by a processor to implement the method described above.

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus, devices, and computer program products implemented according to the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer readable program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The foregoing description of implementations of the present disclosure has been provided for illustrative purposes, is not exhaustive, and is not limited to the implementations disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various implementations described. The terminology used herein was chosen in order to best explain the principles of each implementation, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand each implementation disclosed herein.

Claims (16)

1. A method for managing data, comprising: In response to a data creation request for a target application, determining configuration information of a source data set imported from a first data source, the configuration information including access information of the first data source and a mapping relationship between fields in the source data set and fields used in the target application; Based on the access information, obtaining at least a portion of the data in the source data set from the first data source; and Based on the mapping relationship and the acquired data, a target data set used in the target application is created. 2. The method of claim 1 , wherein receiving the configuration information comprises: In response to the data creation request, providing a data source selection page; receiving, via the data source selection page, a selection of the first data source from among a plurality of data sources; providing an access information input page for the first data source; receiving, via the access information input page, an indication of access information for the first data source; providing a data set setting page for the first data source; and A data selection of the at least a portion of the source data set in the first data source is received via the data set setting page. 3. The method of claim 2, wherein receiving a selection of at least a portion of the source data set in the first data source comprises: Receiving a selection of the source dataset via a dataset selection control in the dataset setting page; presenting at least one field in the source dataset in the dataset settings page; and A selection of one or more fields of the at least one field is received via the data set settings page as fields to be imported. 4. The method of claim 1, wherein the access information indicates at least one of the following: an access account for the first data source, or An access path for the first data source. 5. The method according to claim 1, further comprising: Providing a synchronization configuration page for the source data set and the target data set, wherein the synchronization configuration page is used to configure a data synchronization task between the source data set and the target data set; and The synchronization information for the data synchronization task is received via the synchronization configuration page as a part of the configuration information. 6. The method according to claim 5, wherein the synchronization information comprises at least one of the following: the frequency of periodically synchronizing the data in the target dataset with the data in the source dataset, or The time range for first synchronizing the data in the target dataset with the data in the source dataset. 7. The method according to claim 1, wherein the mapping relationship comprises at least one of the following: a predetermined mapping between a first field in the source data set and a second field used in the target application, or A user-specified mapping between a third field in the source data set and a fourth field used in the target application. 8. The method according to claim 1, further comprising: After the target application is published, providing, in a page associated with the target data set, a start control for a data synchronization task between the source data set and the target data set; and In response to the start control being triggered, the data synchronization task is executed based on the synchronization information of the data synchronization task. 9. The method according to claim 8, wherein the page associated with the target data set comprises at least one of the following: The information viewing page of the target dataset, or The information viewing page of the data synchronization task. 10. The method according to claim 8, further comprising: Sending a prompt message associated with the execution of the data synchronization task to an associated party of the target application; and In response to a preset operation of the associated party on the prompt message, an information viewing page of the target data set is provided. 11. The method according to claim 1, further comprising: After the target application is released, receiving an update to the mapping relationship; and Based on the updated mapping relationship, the created target data set is updated. 12. The method according to claim 1, further comprising: generating a data object associated with the target application based on at least a portion of the target data set; receiving a request from the target application to access the data object; and An operation corresponding to the access request is performed on the data object. 13. The method according to claim 1, further comprising: After the target application is published, receiving a user query for the target application; Based on the user query, providing information indicating at least a portion of the target data set to a machine learning model; and Obtaining results for the user query associated with the at least one portion from the machine learning model. 14. A device for managing data, comprising: a configuration information determination module, configured to determine, in response to a data creation request for a target application, configuration information of a source data set imported from a first data source, the configuration information including access information of the first data source and a mapping relationship between fields in the source data set and fields used in the target application; a data acquisition module, configured to acquire at least a portion of the data in the source data set from the first data source based on the access information; The data set creation module is configured to create a target data set used in the target application based on the mapping relationship and the acquired data. 15. An electronic device, comprising: at least one processing unit; and At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 13 when executed by the at least one processing unit. 16. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 13.