CN117527785A

CN117527785A - Method and system for supporting space engineering file data uploading and full link management

Info

Publication number: CN117527785A
Application number: CN202311496431.7A
Authority: CN
Inventors: 李盛阳; 刘云飞; 杜晨
Original assignee: Technology and Engineering Center for Space Utilization of CAS
Current assignee: Technology and Engineering Center for Space Utilization of CAS
Priority date: 2023-11-10
Filing date: 2023-11-10
Publication date: 2024-02-06
Anticipated expiration: 2043-11-10
Also published as: CN117527785B

Abstract

The invention relates to a method and system that supports aerospace engineering file data upload and full-link management. The method includes: obtaining the business unit and file type of the file to be uploaded; and determining the file to be uploaded based on the business unit and the file type. File configuration rules corresponding to the file; according to the file configuration rules, perform normative verification on the file name of the file to be uploaded; perform repeatability verification on the files to be uploaded that pass the normative verification; Upload the files to be uploaded with the above mentioned repeatability verification. Through the method of the present invention, the problems in the prior art of dispersed storage of multi-service system data, diverse data types, and inconsistent data processing logic leading to inability to be managed and applied in a unified manner are solved, and unified and centralized upload processing is provided for file data, thereby improving the efficiency of uploading file data. Efficiency and accuracy ensure standardized data access and full life cycle compliance management.

Description

Method and system for supporting space engineering file data uploading and full link management

Technical Field

The invention relates to the technical field of manned space engineering data processing, in particular to a method and a system for supporting space engineering file data uploading and full link management.

Background

In the field of manned space flight engineering, the data distributed management generated by a plurality of business units before is related, when a user needs to work for data acquisition and data analysis of each unit, coordination of multiple business departments is needed, data collection is difficult, efficiency is low, and user experience is poor.

In order to promote centralized unified and efficient management of relevant data of the manned aerospace task, a proprietary cloud platform is built, and a storage management and service foundation base is provided for task data generated in the implementation process of the manned aerospace engineering task. The integrated data management and service platform for storage of the manned space projects is constructed based on a proprietary cloud platform, and is used for managing engineering tasks of each business unit corresponding to the service manned space projects and providing services such as data exchange, storage management, sharing distribution and the like.

Therefore, it is necessary to provide a method for uploading and archiving data supporting multiple business units and multiple file types, and according to the characteristics of the relevant file data of the engineering task of each business unit, unified and centralized uploading processing is provided for the file data, so that the high efficiency and accuracy of file uploading are ensured, and the standardized access of the data and the compliance management of the whole life cycle are ensured.

Disclosure of Invention

The invention aims to solve the technical problems described in the background art by providing a method and a system for supporting space engineering file data uploading and full link management.

In a first aspect, the present invention solves the above technical problems by providing the following technical solutions: a method for supporting space engineering file data uploading and full link management comprises the following steps:

acquiring a service unit and a file type of a file to be uploaded;

determining a file configuration rule corresponding to a file to be uploaded according to the service unit and the file type;

according to the file configuration rule, carrying out standardability check on the file name of the file to be uploaded;

repeatedly checking the file to be uploaded, which passes through the normative checking;

uploading the file to be uploaded which passes the repeatability verification.

The beneficial effects of the invention are as follows: based on service units and file types, file configuration rules are pre-configured, so that unified management of diversity of file data of different service units and file types can be realized, files to be uploaded can be more unified through respectively carrying out standardization check and repeatability check on the files to be uploaded, thereby improving the high efficiency and accuracy of file data uploading, ensuring standardized access of data and compliance management of full life cycle, and in addition, unified, high efficiency and accurate uploading of files can be realized for a large number of files.

On the basis of the technical scheme, the invention can be improved as follows.

Further, the performing normalization verification on the file name of the file to be uploaded according to the file configuration rule includes:

s11, splitting the file name of the file to be uploaded according to the underline according to the file configuration rule to obtain i meta-information nodes;

s12, judging whether data exists under each meta-information node, if so, executing S13, and if so, judging that the normalization check fails;

s13, judging whether the field length corresponding to each meta-information node meets the preset field length requirement or not for each meta-information node, if so, executing S14, and if not, judging that the normalization check fails;

s14, judging whether the field range corresponding to each meta-information node meets the preset field range requirement or not, if so, executing S15, and if not, judging that the normalization check fails;

s15, judging whether the field type corresponding to each meta-information node meets the preset field type requirement or not, if so, judging that the normalization check is successful, and if not, judging that the normalization check is failed;

The above-mentioned carrying out repeatability check on the file to be uploaded passing the normative check includes:

s21, determining a file name set corresponding to the file to be uploaded from a database according to the service units and the file types of the file to be uploaded, judging whether an intersection exists between the file name of the file to be uploaded and the file name set, if so, executing S22, and if not, judging that the repeatability check passes, wherein the database stores uploaded files corresponding to different service units and uploaded files corresponding to different file types;

s22, judging whether the file size of the first file is equal to the file size of the file to be uploaded, if not, executing S23, and if so, judging that the repeatability check fails, wherein the first file is a file corresponding to a file name with an intersection with the file name of the file to be uploaded in a file name set;

s23, judging whether the first file and the file to be uploaded are files uploaded by the same user, if so, judging that the repeatability check passes, and if not, judging that the repeatability check fails, and if the files to be uploaded are breakpoint continuous.

The technical scheme has the advantages that for normative verification, whether the file name of the file to be uploaded has data, field length, field range and field type is comprehensively verified, normative of the file name of the file to be uploaded can be guaranteed, and for repeatability verification, whether the file to be uploaded has intersection with a corresponding file name set, file size and whether a user uploading the file is the same user or not is comprehensively verified, the file name of the file to be uploaded can be comprehensively verified, and accurate repeatability verification can be performed on the file to be uploaded.

Further, after uploading the file to be uploaded passing the repeatability check, the method further comprises: and carrying out integrity check on the file to be uploaded.

The further scheme has the beneficial effects that after the file to be uploaded is uploaded, the file to be uploaded is subjected to integrity check, so that the uploaded file is ensured to be complete.

Further, after uploading the file to be uploaded passing the repeatability check, the method further comprises: extracting meta information of a file to be uploaded; storing meta information of the file to be uploaded to a distributed database and a Kafka message queue; and synchronously writing the meta-information of the file to be uploaded into an ES database based on the meta-information of the file to be uploaded stored in the Kafka message queue.

The adoption of the further scheme has the beneficial effects that for the uploaded file, the meta information of the file can be respectively stored in the distributed database, the ES database and the Kafka message queue to realize the storage of different requirements, wherein the storage in the ES database can also support the full text retrieval of the file information.

Further, after synchronously writing the meta information of the file to be uploaded into the ES database, the method further includes:

consistency verification is carried out on the meta information of the file to be uploaded written into the ES database, the meta information of the file to be uploaded stored in the distributed database and the meta information of the file to be uploaded stored in the Kafka message queue; and if the consistency check is not passed, carrying out data supplementing processing on the meta information of the file to be uploaded in the ES database.

The adoption of the further scheme has the advantages that after the meta-information of the file to be uploaded is synchronously written into the ES database, consistency verification is carried out on the meta-information of the file to be uploaded, which is written into the ES database, the meta-information of the file to be uploaded and stored in the distributed database, and the meta-information of the file to be uploaded, which is stored in the Kafka message queue, so that the consistency of the data written into the ES database, the meta-information stored in the Kafka message queue and the data in the meta-information database can be ensured, and in case of inconsistency, the supplementary data processing can be carried out on the ES data.

Further, the method comprises the following steps: in the process of uploading the file to be uploaded, monitoring the whole uploading link process, acquiring file uploading related information, and visually displaying the file uploading related information, wherein the file uploading related information comprises file uploading progress, file uploading state and file archiving state information.

The adoption of the further scheme has the beneficial effects that the user can more intuitively know the file uploading condition by visually displaying the file uploading related information, and the user experience is improved.

Further, uploading the file to be uploaded passing the repeatability verification includes: determining a target transmission mode of the file to be uploaded according to a transmission scene of the file to be uploaded, wherein the target transmission mode is an API mode or a Web mode; and uploading the file to be uploaded which passes the repeatability verification according to the target transmission mode.

The adoption of the further scheme has the beneficial effects that based on different transmission scenes, different file transmission modes are adopted so as to meet the uploading requirements of different transmission scenes.

In a second aspect, the present invention further provides a system for supporting space engineering file data uploading and full link management, for solving the above technical problem, where the system includes:

The acquisition module is used for acquiring a business unit and a file type of a file to be uploaded;

the rule determining module is used for determining a file configuration rule corresponding to a file to be uploaded according to the service unit and the file type;

the normalization verification module is used for performing normalization verification on the file name of the file to be uploaded according to the file configuration rule;

the repeatability verification module is used for carrying out repeatability verification on the file to be uploaded which passes the standard verification;

and the uploading module is used for uploading the file to be uploaded which passes the repeatability verification.

In a third aspect, the present invention further provides an electronic device for solving the above technical problem, where the electronic device includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor, and when the processor executes the computer program, the method for supporting uploading of data and full link management of an aerospace engineering file according to the present invention is implemented.

In a fourth aspect, the present invention further provides a computer readable storage medium, where a computer program is stored, where the computer program, when executed by a processor, implements the method for supporting space engineering file data uploading and full link management according to the present invention.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that are required to be used in the description of the embodiments of the present invention will be briefly described below.

FIG. 1 is a flow chart of a method for supporting space engineering file data uploading and full link management according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a system supporting space project file data uploading and full link management according to one embodiment of the present invention;

FIG. 3 is a logic flow diagram of archive configuration management provided by one embodiment of the present invention;

FIG. 4 is a schematic flow chart of a verification method before file transmission according to an embodiment of the present invention;

fig. 5 is a schematic flow chart of a method for detecting normalization of file data according to an embodiment of the present invention;

FIG. 6 is a flowchart of a method for detecting file data repeatability according to an embodiment of the present invention;

fig. 7 is a flow chart of a data uploading method according to an embodiment of the present invention;

fig. 8 is a schematic diagram of an uploading stage flow of a data uploading manner according to an embodiment of the present invention;

FIG. 9 is a flowchart of a file integrity verification algorithm according to an embodiment of the present invention;

fig. 10 is a flowchart illustrating a method for extracting meta information of a file according to an embodiment of the present invention;

FIG. 11 is a flowchart illustrating a method for verifying and maintaining consistency of metadata of a file according to an embodiment of the present invention;

FIG. 12 is a logic flow diagram of a method for monitoring data of an uploaded file according to an embodiment of the present invention;

FIG. 13 is a schematic structural diagram of a system supporting space project file data uploading and full link management according to an embodiment of the present invention;

fig. 14 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The principles and features of the present invention are described below with examples given for the purpose of illustration only and are not intended to limit the scope of the invention.

The following describes the technical scheme of the present invention and how the technical scheme of the present invention solves the above technical problems in detail with specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.

The scheme provided by the embodiment of the invention can be applied to any transmission scene needing file uploading. The embodiment of the invention provides a possible implementation manner, as shown in fig. 1, and provides a flowchart of a method for supporting space engineering file data uploading and full link management, where the method can be executed by any electronic device, for example, a terminal device, or a terminal device and a server together. For convenience of description, a method provided by an embodiment of the present invention will be described below by taking a terminal device as an execution body, and the method may include the following steps as shown in a flowchart in fig. 1:

s110, acquiring a service unit and a file type of a file to be uploaded;

s120, determining a file configuration rule corresponding to the file to be uploaded according to the service unit and the file type;

s130, according to the file configuration rule, carrying out normalization check on the file name of the file to be uploaded;

s140, repeatedly checking the file to be uploaded, which passes through the normalization check;

and S150, uploading the file to be uploaded, which passes the repeatability verification.

According to the method, based on the business units and the file types, the file configuration rules are pre-configured, so that unified management of the diversity of file data of different business units and different file types can be realized, and through respectively carrying out standardization check and repeatability check on the files to be uploaded, the files to be uploaded can be more unified, so that the efficiency and the accuracy of uploading the file data can be improved, the standardized access of the data and the compliance management of the full life cycle can be ensured, and in addition, unified, efficient and accurate uploading of the files can be realized for a large number of files.

The solution of the present invention will be further described in connection with the following specific embodiments, in which the solution may be implemented based on the data processing system shown in fig. 2, as shown in fig. 2, which comprises an archive configuration module, a data detection module, a data upload module, a data archive module and an archive visualization monitoring module, wherein,

the archiving configuration module is configured to configure naming detection rules (i.e. preset field length requirements, preset field range requirements, and preset field type requirements) of the file, where the rules are used for performing normalization verification on the file name of the file.

The data detection module is used for carrying out normalization check, repeatability check and integrity check on the file to be uploaded.

And the data uploading module is used for managing file uploading modes, including an API mode and a Web mode.

The file visual monitoring module is used for monitoring and displaying file uploading related information, wherein the file uploading related information comprises file uploading progress, file uploading state and file archiving state information, the file uploading progress can be displayed in a percentage mode, and specifically, the file uploading state and the file archiving state information are divided into: the state of uploading corresponding to the uploading, the state of uploading success corresponding to the uploading completed, the state of archiving corresponding to the archiving in the archiving, the state of archiving success or archiving failure corresponding to the archiving completed, the state of rejecting archiving, etc., the state of deleting the file corresponding to the successful file, etc.

And the data archiving module is used for storing the uploaded file into the distributed file system, storing the extracted file meta-information, the file information and uploading task information (task related information appearing later) into the distributed database, writing the file meta-information into the ES, storing the file meta-information into the Kafka message queue, and performing consistency verification.

The system also supports configuration of file uploading rule information, namely information of rules to be followed when the file is configured to be uploaded, can also establish archive configuration information rules according to business units of the file to be uploaded, file types of the file to be uploaded, meta information extraction rules (used for extracting meta information of the file) and the like, can also input meta information detection rules (such as file names and the like) of corresponding files in an archive rule configuration module, generates and stores a corresponding archive rule configuration file based on the rules, and defines and generates a file meta information table containing field information corresponding to the file configuration information rules, wherein the file meta information table is used for storing meta information of the files.

Based on the above system, the method for supporting space engineering file data uploading and full link management provided in this embodiment may include the following steps:

S110, acquiring a service unit and a file type of a file to be uploaded;

the service unit refers to a unit to which a file to be uploaded belongs. The file types may include:

data general: the general class of file data, which is determined by the file usage status, may be referred to as a secondary classification, such as archiving data.

Data subclass: the sub-class of file data, which is determined by the file service attributes, may also be referred to as a three-level classification, e.g., raw telemetry data.

in the scheme of the invention, different file configuration rules are correspondingly configured for different business units and different file types, and the file configuration rules corresponding to the file to be uploaded can be determined based on the business units and the file types of the file to be uploaded. The file configuration rule is a configuration rule corresponding to a file name, and as an example, one file configuration rule may be described as: the file name consists of three elements A, B and C.

In particular, referring to a logic flow diagram for archive configuration management shown in FIG. 3; the file configuration rule can be configured by determining file classification firstly, namely, after determining service units and file types of the files firstly, starting configuration, then dividing file names of the files to obtain a plurality of nodes i (modes), and configuring meta-information corresponding to the nodes for each node i, namely, configuring which types of meta-information the meta-information corresponding to the node i consists of to obtain the file configuration rule corresponding to each node; then, the length (length) of the node detection may be further configured, for example, the field length requirement that needs to be met by the configuration node i may be further configured, and then, the node precision range may be further configured, for example, the precision range requirement that needs to be met by the configuration node i, and the node ambiguity range, i.e., the ambiguity range requirement that needs to be met by the configuration node i, may be collectively referred to as a node range requirement (range). At this time, the configuration is completed, and the configuration process ends. Through the configuration, file configuration rules corresponding to different file types and corresponding preset requirements can be obtained for different service units.

In this application, after the file name corresponding to each file is divided into i nodes according to the underline, the element correspondence of each node i corresponding to the file name of the file may be stored in the array corresponding to the file name (one file name may correspond to one array), then each node i corresponds to one identifier, each identifier has one subscript, for each node i, the subscript value corresponding to the node i is used as the key corresponding to the node i, the file configuration rule corresponding to the node i is configured, the file configuration rule corresponding to the node i is used as the value corresponding to the node i, and the file configuration rule corresponding to each node i may be the same or different, so that for one file name, each node corresponding to the file name and the file configuration rule corresponding to each node may be stored in k-v.

As one example, the file name of a file is: A_B_C_D_E_F_G, the configuration rule corresponding to the file is stored through a Map type data structure, the elements are k-v type data, 7 elements are obtained, key values of the elements are respectively 0 to 6, value values are rules of meta-information nodes separated from the file, the file name is stored in an array a after being separated according to underlining, the corresponding subscript of A in the array a is 0, the corresponding subscript of B in the array a is 1, the corresponding subscript of C in the array a is 2, the corresponding subscript of D in the array a is 3, and the file name can be stored in the array a by analogy. For element A, the subscript 0 corresponding to element A is the key corresponding to element A, and the file configuration rule corresponding to element A is the value corresponding to the element with the key value of 0 in the configuration rule. For example, if the value is empty, the element corresponding to the value only needs to have a value, that is, only needs to have the element, and the file configuration rule corresponding to the subscript 3 and 3 is: if the length is required to be 4, the length is required to be 1, and the rule is obviously not satisfied, the standardability check of the file name is terminated, and the check is judged to be failed.

In addition, in the scheme of the application, the file names of the files to be transmitted are named in an underlined mode, and then the file names can be divided according to the underlined mode, and one configuration condition corresponding to the split file names is as follows: the number of the split nodes is set, if the number of the split nodes is not equal to the set number, the split file names can be judged to be file names which do not accord with the normative check, if the number of the split nodes is equal to the set number, the split file names can be judged to accord with the normative check, and then the normative check can be carried out on the split file names.

As an example, for example, the configuration condition is that 7 valued nodes are required among the nodes corresponding to one file name, and if the file name is a_b_c or a_b_c_d_e_f_g_h, it means that the two file names are not opposite, because the number of nodes corresponding to the two file names is 3 and 8, respectively, and neither is 7.

Referring to fig. 4, before S130, the method further includes:

the user invokes the uploading starting interface to acquire login information of the user, performs authority verification on the uploading user according to the login information of the user (the login information can be matched with the login information stored in advance generally), if the login information is matched with the login information, the operation authority is indicated, if the login information is not matched with the login information, the user is judged whether the user has the operation authority, if the login information is matched with the login information, the managing uploading interface is called, uploading files (files to be uploaded) and file configuration rules are acquired, then the files are subjected to normalization and repeatability verification according to the file configuration rules corresponding to the files to be uploaded (processing procedures corresponding to S130 and S140 below), if the login information is not matched with the login information stored in advance, uploading is not allowed, after the normalization and repeatability verification is passed, the files are transmitted, if the normalization and the repeatability verification is not passed, the user can be prompted to verify reasons of failure, and uploading is finished. After the user authority verification, the following scheme corresponding to S130 is performed. In addition, in the scheme of the application, file list related information can be obtained, the file list related information stores related information of each file waiting to be uploaded, and based on the file list related information, the files waiting to be uploaded can be known.

the normalization check refers to checking whether the file name accords with a preset naming rule.

Optionally, referring to fig. 5 specifically, according to the file configuration rule, performing normalization verification on the file name of the file to be uploaded includes:

s11, splitting the file name of the file to be uploaded according to underlines according to a file configuration rule to obtain i meta-information nodes (also simply called nodes); this step corresponds to the splitting of the file name shown in fig. 5 under-lined;

before the step S11 is executed, a step corresponding to the file name, the service unit, and the type (the data major class and the data minor class) shown in fig. 5 is executed, that is, a specified file name rule (file configuration rule) is obtained according to the service unit, the data major class, and the data minor class reading configuration file, and then S11 is executed;

s12, judging whether data exists under the meta-information nodes or not, if so, executing S13, and if so, judging that the normalization check fails;

Referring to fig. 5, for each node, a file configuration rule corresponding to the node is determined, and then, normalization verification of S12 to S15 is performed on the node based on the file configuration rule. One implementation manner of determining the file configuration rule corresponding to each node is as follows: for each node, determining a key corresponding to the node (i.e. a key in the spliced configuration file shown in fig. 5) from an array corresponding to a file name of the file to be uploaded, and determining a value corresponding to the key (corresponding to the data of the node acquired according to the key in fig. 5) according to the key, wherein the file configuration rule corresponding to the value is the file configuration rule corresponding to the node.

S13, judging whether the field length corresponding to each meta-information node meets the preset field length requirement, if so, executing S14, and if not, judging that the normalization check fails;

s14, judging whether the field range corresponding to each meta-information node meets the preset field range requirement, if so, executing S15, and if not, judging that the normalization check fails; wherein the field range requirements include a specific range requirement of a file name and a fuzzy range requirement of the file name.

S15, judging whether the field type corresponding to each meta-information node meets the preset field type requirement, if so, judging that the normalization check is successful, and if not, judging that the normalization check is failed;

it should be noted that, for i bytes corresponding to the file name of the file to be uploaded, each node performs the normalization check according to the steps from S12 to S15, and if one byte in the i bytes fails the check in any step from S12 to S15, it indicates that the normalization check of the file name of the file to be uploaded fails. When the normalization check fails, a file name that fails the check may also be returned.

S140, repeatedly checking the file to be uploaded, which passes through the normalization check; the repeatability check refers to checking whether the file name and the file are repeated or not. The following describes in detail the process of the repeatability verification with reference to fig. 6, where the above-mentioned performing the repeatability verification on the file to be uploaded that passes the standard verification includes:

s21, determining a file name set (corresponding to a file name set under the same service unit, a data major class and a data minor class (file types) in a database file table shown in FIG. 6) corresponding to the file to be uploaded through the standardization verification from a database according to the service unit and the file type of the file to be uploaded, judging whether an intersection exists between the file name of the file to be uploaded and the file name set, if so, executing S22, and if not, judging that the repeatability verification is passed, wherein the database stores uploaded files corresponding to different service units and uploaded files corresponding to different file types;

The file names in the file name set determined from the database may include a file name corresponding to a service unit identical to a service unit of the file to be uploaded and a file name corresponding to a file type identical to a file type of the file to be uploaded.

Optionally, if the file to be uploaded includes at least two files, before S21, the method further includes:

s20, checking whether repeated file names exist in the file names corresponding to the file to be uploaded, if not, executing S21, if so, judging that the repeated verification fails, and returning the file names failing to be checked, and prompting that the file names are repeated; the step S22 corresponds to the checking of whether the file name set itself is repeated in fig. 6, and the file name set appearing in fig. 6 is the set where at least two file names corresponding to the file to be uploaded are located.

S22, judging whether the file size of the first file is equal to the file size of the file to be uploaded, if not, executing S23, if so, judging that the repeatability check fails, and further returning the file name of which the check fails and prompting that the file is uploaded. The first file is a file corresponding to a file name with an intersection set of file names of the files to be uploaded in the file name set; this step corresponds to the detection of whether the original file (file to be uploaded) and the uploaded file (first file) shown in fig. 6 are equal in size;

S23, judging whether the first file and the file to be uploaded are files uploaded by the same user, if so, judging that the repeatability check passes, and carrying out breakpoint continuous uploading on the file to be uploaded, and if not, judging that the repeatability check fails. This step corresponds to the detection of whether the user upload is unified or not as shown in fig. 6. If the repeatability verification is not passed, in the scheme of the application, the file name failing to verify can be returned, and the file is prompted to be uploaded.

Optionally, multiple uploading modes are supported in the scheme, such as slice parallelization, breakpoint continuous transmission and the like.

Optionally, the uploading manners corresponding to the files with different data amounts are different, and S140 uploads the file to be uploaded that passes the repeatability verification may specifically include:

determining a target transmission mode of the file to be uploaded according to the transmission scene of the file to be uploaded, wherein the target transmission mode is an API mode or a Web mode; and uploading the file to be uploaded passing the repeatability verification according to the target transmission mode.

Specifically, files with different data amounts can correspond to different transmission scenes, that is, the transmission scenes in the scheme of the application can be divided according to the data amount of the files, for example, for a large number of files, the target transmission mode can be an API mode, the API bottom layer is an interface for uploading files based on the FTP protocol, for a small number of files, the target transmission mode can be a Web mode, and the Web bottom layer is an uploading interface for transmitting the files and related information thereof based on the HTTP protocol and the FTP protocol. The related information of the FTP protocol file includes the name of the service unit, the threshold size of the file (i.e. the threshold for judging whether the file is a large number of files or a small number of files), the file uploading mode, the uploading state, the uploading time, the uploading user, etc.

As an example, referring to the steps of the API uploading method shown in fig. 7, the method specifically includes:

s1401, after the preparation before the transmission of the uploaded file is completed, a path for uploading the FTP directory is obtained, wherein the FTP directory path is an absolute path newly created according to the information of the unit (service unit), the data type (file type), the date and the like of the current user, and the uploading directory is newly created at the same time.

And S1402, when uploading is started, modifying the uploading state of the file into uploading, judging whether the file is subjected to breakpoint continuous transmission according to the verification result in step S140, and carrying out corresponding transmission, wherein the method specifically corresponds to the judgment of whether the FTP has a file part uploading result shown in fig. 7, if yes, the file is subjected to breakpoint continuous transmission, and then executing step S1403, if no, executing step S1403.

S1403, in the uploading process, the large file (file to be uploaded) is split into a plurality of small files to be uploaded in a fragmented mode, and meanwhile the size of the uploaded file is updated to a Redis cache.

And S1404, if the fragment uploading is not in error, after the normal completion, carrying out integrity check on the file to be uploaded, updating the file state to be completed if the check is passed, warehousing the meta information of the file to be uploaded, completing the uploading task, and prompting the user to upload detailed results.

S1405, if the fragment uploading is wrong or the file integrity check fails, the uploading of the file to be uploaded fails, and the uploaded remote file is deleted.

If the file to be uploaded adopts the Web mode, referring to fig. 8, the specific flow is as follows:

traversing the file queue to see whether the file is not uploaded, if yes, starting uploading the file to be uploaded, and when the file starts uploading, calling a transmission management server manager to update the file uploading state into uploading. Whether the current file is uploaded and recorded in the Redis is judged, if yes, an uploading record file path is obtained from the Redis, the uploading progress is analyzed, and if not, the uploading record is regenerated. In the file uploading process, judging whether the file needs to be transmitted in a slicing mode according to the size of the file, if so, carrying out slicing transmission, otherwise, directly uploading, finding a file ID through an uploading path and a file name after the transmission is completed, and updating the size of the file into a Redis cache. And when the file uploading is finished, calling a manager, and updating the file uploading state to be 'uploaded'. Traversing the file queue, if no file is not uploaded, finishing the uploading task, checking the uploading states of all files in the uploading task, returning success if all the files succeed, returning the file name of the uploading failure if the file is transmitted failure, and prompting the user to upload the result by a popup window.

Optionally, after uploading the file to be uploaded passing the repeatability check, the method further comprises:

s160, carrying out integrity check on the file to be uploaded.

Referring to fig. 9, one implementation manner of the integrity check of the file to be uploaded is: before uploading, calculating a first MD5 value (namely, the hash value of the original file corresponding to FIG. 9) corresponding to the file to be uploaded through an MD5 algorithm, selecting a file uploading mode, completing file transmission, after uploading, calculating a second MD5 value (namely, the hash value of the uploaded file shown in FIG. 9) corresponding to the file to be uploaded through the MD5 algorithm, comparing the first MD5 value with the second MD5 value, if the first MD5 value and the second MD5 value are equal, indicating that the uploaded file to be uploaded is complete, otherwise, indicating that the file to be uploaded may be damaged or modified in the transmission process.

Optionally, referring to the flowchart shown in fig. 10, after uploading the file to be uploaded that passes the repeatability check, the method further includes:

extracting meta information of the file to be uploaded; the meta information table and the file name configuration file (file configuration rule) of the file can be obtained according to the service unit, the data major class and the data minor class corresponding to the uploading file (file to be uploaded); splitting the file name according to the underline, analyzing the file name according to the file name configuration file, and converting the file name into meta-information of the file; storing the meta information of the file to be uploaded to a distributed database and a Kafka message queue; and synchronously writing the meta-information of the file to be uploaded into an ES database based on the meta-information of the file to be uploaded stored in the Kafka message queue.

In the process of synchronously writing the meta information of the file to be uploaded into the ES database, the specific implementation process is as follows: and monitoring the Kafka message queue through the ES, reading the meta-information of the file to be uploaded from the Kafka message queue when the Kafka message queue has data, and writing the read meta-information into the ES database.

Optionally, the task related information corresponding to the file to be uploaded may be stored in an upload task information table of the distributed database, where the upload task information table is used to store the task related information.

Optionally, after synchronously writing the meta information of the file to be uploaded into the ES database, the method further includes: consistency verification is carried out on the meta information of the file to be uploaded written into the ES database, the meta information of the file to be uploaded stored in the distributed database and the meta information of the file to be uploaded stored in the Kafka message queue;

if the consistency check is not passed, the ES missing data is indicated, and the meta-information of the file to be uploaded in the ES database can be subjected to data supplementing processing.

Alternatively, if the consistency check is passed, the uploading task record and file list information may also be generated and saved.

Optionally, the above consistency check may be specifically referred to fig. 11, where a specific implementation process of the consistency check is:

after the uploading of the files (files to be uploaded) of each business unit is completed, the meta information of the files to be uploaded is written into a distributed meta information database and a Kafka message queue respectively; the method comprises the steps of monitoring a Kafka message queue through an ES, reading and writing meta-information of a file to be uploaded into an ES database; the meta information written into the ES database corresponds to the ES index. And comparing whether the meta information of the file to be uploaded in the meta information database, the meta information in the Kafka message queue and the meta information in the ES database are consistent (corresponding to the comparison of the file meta information, the Kafka message and the ES data in FIG. 11), and if so, checking the consistency of data transmission, and generating that the file archiving is successful. If the meta information data in the ES database is missing, an ES insertion statement can be generated and written into the verification report, and the supplementary data of the ES can be completed by executing the verification report.

Optionally, the method further comprises: and in the process of uploading the file to be uploaded, monitoring the whole uploading link process, acquiring the file uploading related information, and visually displaying the file uploading related information, wherein the file uploading related information comprises file uploading progress, file uploading state and file archiving state information.

And monitoring the whole uploading link process of uploading the file to be uploaded, collecting relevant log information such as file uploading progress, file uploading state, file archiving state information and the like in real time, and carrying out visual presentation on the whole uploading link according to the log information so as to efficiently find out and retransmit the file which fails to be uploaded.

Optionally, referring to fig. 12, the step of monitoring the entire uploading archiving process of the file to be uploaded further includes: starting to upload data monitoring, namely starting to monitor the uploaded file to be uploaded, opening an uploading data monitoring interface (corresponding to the opening uploading monitoring interface shown in fig. 12), inputting monitoring screening conditions of uploading file data (corresponding to the inputting uploading monitoring screening conditions shown in fig. 12), wherein the screening conditions comprise a business unit, an uploading mode, a data size threshold, an uploading state, uploading starting time, uploading users and the like, and generating an uploading data monitoring list according to the screening conditions.

According to the result of judging whether the selected uploading data fails to be uploaded or not, displaying different options on the monitoring list, displaying uploading log options if the uploading is successful, clicking the uploading log by a user, expanding an uploading log interface, displaying the uploading link of the file in a visual mode, and checking uploading log information by the user to finish uploading data monitoring.

If the file fails to upload, two options (corresponding to the click button shown in fig. 12) are displayed for the user to retransmit and upload the log respectively, and if the user clicks on the retransmission option (corresponding to the click retransmission button in fig. 12), the file is restarted to upload (corresponding to the start of data retransmission in fig. 12), and the uploading data monitoring flow is ended, namely corresponding to the completion of the remittance data monitoring in fig. 12;

if the user clicks the upload log option (corresponding to clicking the summary log shown in fig. 12), the upload log interface is expanded to display the upload link of the file in a visual form, and the user can look up the upload log information, and the upload data monitoring flow is ended.

For a better description and understanding of the principles of the method provided by the present invention, the following description of the present invention is provided in connection with an alternative embodiment. It should be noted that, the specific implementation manner of each step in this specific embodiment should not be construed as limiting the solution of the present invention, and other implementation manners that can be considered by those skilled in the art based on the principle of the solution provided by the present invention should also be considered as being within the protection scope of the present invention.

In this embodiment, the uploading and archiving of the file data in the unmanned aerial vehicle engineering multitasking unit is a specific embodiment, and in this embodiment, the following steps are implemented:

S1, firstly, configuring various rules and generating a meta-information table, wherein the various rules comprise, but are not limited to, file configuration rules, meta-information extraction rules, archiving configuration information rules and the like;

describing an aerospace engineering as a specific embodiment, wherein the file to be uploaded is classified into three levels according to the attribute of the file, and the three levels are classified into the first level according to different business units to which the file belongs, wherein the first level comprises the steps of;

the files to be uploaded are further classified according to the difference between the data major class and the data minor class, wherein the data major class comprises archiving data, remixing data, data results, result reports and the like; the subclasses of data include simulation models, proprietary data, complete telemetry data, etc.;

FIG. 3 is a logic flow diagram for archive configuration management provided in an embodiment of the present application, as shown in FIG. 3, in some embodiments, the archive configuration management includes the steps of:

after confirming the business units to which the files to be uploaded belong and classifying the files to be uploaded according to the data types, inputting naming rules of the files in the corresponding archiving configuration module to generate archiving configuration files, and carrying out file normalization verification according to the naming rules of the files when the files to be uploaded are uploaded, wherein the archiving configuration files support hot loading.

In the file meta-information table configuration item, the meta-information table attribute of the file is configured, and a meta-information table corresponding to the file to be uploaded is generated, wherein the meta-information table is used for storing the meta-information of the file to be uploaded into the meta-information table after the uploading of the file to be uploaded is completed.

The data elements to be input of the configuration file in the archiving configuration module can be shown in the following table:

and generating an archiving configuration file according to the input elements in the table.

S2, supporting different uploading modes according to different transmission scenes, wherein the uploading mode comprises an uploading mode supporting an FTP protocol and an uploading mode supporting an HTTP protocol. The user selects a file uploading mode according to the actual transmission requirement, and formally starts a file uploading flow.

According to the uploading method, two uploading modes are provided according to different uploading scenes (transmission scenes), wherein the two uploading modes comprise an API uploading mode and a Web uploading mode: the bottom layer of the API uploading mode is uploaded through an FTP protocol interface, the bottom layer of the Web uploading mode is used for uploading files to a server through an HTTP protocol interface, and then the FTP interface is called to upload the files to a distributed file system.

Optionally, in the actual scene, the situation that the files need to be uploaded in batches at fixed time exists, at this time, an API mode can be selected, a resource pool is polled in a mode of a fixed time task, the files are read in batches, the files are uploaded through an FTP interface, and the uploading process supports breakpoint continuous transmission and fragment transmission.

Optionally, when a small amount of files exist in the actual scene, the files can be uploaded by selecting local files on the Web interface in a Web mode, and the uploading process supports breakpoint continuous transmission and fragmented transmission.

And S3, before file transmission, carrying out authority verification on the uploading user according to the user information, acquiring a file configuration rule of the file to be uploaded according to the service unit and the file type of the transmission file, carrying out normalization and repeatability verification on the file name of the file to be uploaded according to the file configuration rule, and calculating a local file MD5 value (corresponding to the first MD5 value).

In order to complete the verification work at the beginning of the file uploading, the embodiment of the application provides a flow chart of a verification method before the formal transmission of the file, and the verification process of the operation authority is described in the foregoing in connection with fig. 4, which is not repeated here.

In order to ensure accuracy and effectiveness of file transfer, fig. 5 is a flowchart of a method for detecting normalization of file data according to an embodiment of the present application, as shown in fig. 5, in some embodiments, the method for detecting normalization includes the following steps:

s311, reading the configuration file according to the business unit, the data major class and the data minor class (corresponding to the types in fig. 5), and obtaining a specified file name rule (file configuration rule), wherein the file name rule can know how the keys corresponding to the file names in the configuration file are formed.

S312, splitting the file name of the file to be uploaded according to underlines, and splitting the file name into i meta-information nodes (which can be simply called nodes);

s313, acquiring configuration metadata of the node according to the key of the configuration file, namely, the metadata corresponding to the key of the file name corresponding to the configuration file, and comparing the metadata with the metadata node data with the split file name.

S314, comparing whether data exists in each node corresponding to the file name, namely, the length of the node field, the range of the node field and the type of the node field, and determining whether each node meets the requirement of configuring node meta-information (the preset field length requirement, the preset field range requirement and the preset field type requirement described above).

S315, traversing all nodes of the file name splitting, comparing with the configuration meta information of the configuration file, and returning the file name failed to the user if the file name normalization check passes if all nodes meet the requirement of the configuration meta information, otherwise, checking fails.

In the step S314, the comparison is performed on the node field range, that is, it is determined whether the field of the node corresponding to the file name of the file to be uploaded meets the preset field range requirement, specifically, it is determined whether the specific node data corresponding to the node with the split file name is a combination formed by specifically designated numbers, letters or certain types of character arrangements, and exemplary, the field range requirement corresponding to the 3 rd node is specified by the configuration file: and if the value is TH, MT and WT, the value of the 3 rd node of the file name of the file to be uploaded is any value of the three values, otherwise, the verification is not passed.

In the step S314, the comparison is performed on the node field types, that is, whether the field of the node corresponding to the file name of the file to be uploaded meets the preset field type requirement is determined, specifically, whether the specific node data corresponding to the node with split file name meets the field type requirement is determined. Illustratively, the configuration file specifies that the field type requirements corresponding to node 3 are: the value must be the combination of the capital letter plus the number, then the 3 rd node value of the file name of the file to be uploaded must meet this condition, otherwise the verification is not passed.

In order to ensure the accuracy and the effectiveness of file transmission, the file to be uploaded which passes through the standardization verification is subjected to the repeatability verification, and the specific verification process can refer to fig. 6, after the standardization and the repeatability verification of the file name of the file to be uploaded are completed, the local MD5 value (the first MD5 value described above) of the file is calculated and stored, and after the file transmission is completed, the integrity verification algorithm is performed.

And S4, uploading the file to be uploaded to a distributed file system after the repeatability verification is passed, and supporting fragment parallelization, breakpoint continuous transmission and the like.

The method for uploading the file to be uploaded can adopt a method for carrying out file slicing and multithreading parallel transmission, so that the file uploading efficiency is improved, the high concurrency supporting characteristic of a system is ensured, and the high efficiency of transmission is ensured; in some embodiments, aiming at uploading interruption caused by network fluctuation or other factors, the accuracy of transmission is ensured by a breakpoint continuous transmission method; and for different uploading scenes, the file transmission is completed in a corresponding uploading mode.

Optionally, for the transmission of a large number of files, an API mode is generally called for transmission, and the API bottom layer is an interface for uploading files based on the FTP protocol. For the transmission of a small amount of files, web mode transmission is generally called, and a Web bottom layer is an uploading interface for transmitting the files and related information thereof based on an HTTP protocol and an FTP protocol. Uploading the file to be uploaded in a Web mode, selecting a file uploading list before uploading the file, enabling the local file to enter a file queue, performing S3 verification and recording on an uploading user and the file, and enabling verification to enter an uploading stage.

And S5, after the file uploading is finished, calculating a file MD5 (the second MD5 value described above) at the server, and carrying out integrity check on the file to be uploaded through an integrity check algorithm. MD5 is a widely used cryptographic hash function, and is mainly used to confirm the integrity of a file during transmission.

S6, after the integrity check is finished, uploading task records (task related information corresponding to the files to be uploaded) and file list information are generated and stored, the files passing through the check are extracted according to corresponding rules (meta information extraction rules), the meta information of the files to be uploaded is stored in a distributed database, meanwhile, the meta information of the files to be uploaded is synchronously written in an Elastic Search (abbreviated ES) database to support full text retrieval of the file information, the meta information of the files to be uploaded is also stored in a Kafka message queue, and when the ES is written, consistency check can be carried out on write information (the meta information of the files to be uploaded stored in the ES database, the meta information of the files to be uploaded stored in the distributed database and the meta information of the files to be uploaded stored in the Kafka message queue), and if the verification is not passed, the ES is deleted, data supplementing operation is carried out on the ES.

When the file integrity check is passed, it indicates that the specified file (file to be uploaded) has been successfully uploaded, and the file list information and the uploading task information need to be recorded and put in storage.

And (3) checking the passed file, splitting the file name according to the underline, extracting the meta-information of the file, and storing the meta-information of the file into a file meta-information table created in the step S1. Fig. 10 is a flow chart of a file meta-information extraction method provided in an embodiment of the present application, describing a related flow of file meta-information extraction, as shown in fig. 10, in some embodiments, the meta-information extraction process includes the following steps:

s601, meta-information extraction is started, and a meta-information table and a file name configuration file of a file are obtained according to a service unit, a data major class and a data minor class to which an uploading file (a file to be uploaded) belongs.

S602, splitting the file name according to the underline, analyzing the file name according to the file name configuration file, and converting the file name into meta-information of the file.

And S603, storing the file meta-information into a database meta-information table corresponding to the business unit to which the file belongs, namely storing the meta-information of the file to be uploaded into the meta-information table in the meta-information database, and synchronously writing the meta-information of the file to be uploaded into the ES database.

In this embodiment of the present application, the meta information database is a distributed database, and different service units may correspond to different databases. Because the number of files is too large, in order to improve the efficiency of file retrieval, after the file meta-information data is accessed into the database, the file meta-information data is written into the ES database through the Kafka message queue for full-text retrieval, and the consistency of data transmission is ensured in the process of data streaming.

And S7, monitoring the whole uploading link process of uploading the file to be uploaded, collecting relevant log information such as file uploading progress, file uploading state, file archiving state information and the like in real time, visually presenting the whole uploading link according to the log information, and efficiently finding out and retransmitting the file which fails to be uploaded.

The embodiment of the application provides a method for monitoring uploading file data, which supports each business unit administrator to check the uploading file data in each authority range, including real-time data and historical data, and can perform multidimensional query on the uploading file, and comprises the following steps: the name of the service unit, the size of the file threshold, the uploading mode, the uploading state, the uploading time and the uploading user, and returning to the uploading file list under the relevant conditions.

The method supports that each business unit administrator can monitor each uploading file within the authority range, click the name of the uploading file, pop up a monitoring window, display the uploading link of the uploading file in a visual mode, and support that the uploading is completed and the uploading link or uploading progress of the uploading file in the uploading process is displayed.

Optionally, the following table is available as input elements for querying the uploaded file:

optionally, after the user inputs the query condition, the relevant information of the uploaded data meeting the query condition is provided as follows:

sequence number	Element name	Description of the invention
			Uploading data names	Naming when uploading data
	Uploading mode	WEB/API
			Uploading time	Start time of upload
	Uploading state	All/upload in/fail/complete
			Uploading initiating center	Each business unit
	Uploading user	Uploading data initiator
			Detail view button	Displaying the log of the whole uploading process and displaying the current call data state
	Data retransmission button	Data retransmission button for displaying failure of data uploading

By the scheme of the invention, the problem that unified management and application cannot be realized due to scattered storage of the data, multiple data types and inconsistent data processing logic of the multi-service system in the prior art is solved, unified and centralized uploading processing is provided for file data, the high efficiency and accuracy of file data uploading are improved, and standardized access of the data and compliance management of a full life cycle are ensured.

Based on the same principle as the method shown in fig. 1, the embodiment of the present invention further provides a system 20 supporting space engineering file data uploading and full link management, as shown in fig. 13, the system 20 may include an acquisition module 210, a rule determination module 220, a normative checking module 230, a repeatability checking module 240, and an uploading module 250, wherein:

an obtaining module 210, configured to obtain a service unit and a file type of a file to be uploaded;

the rule determining module 220 is configured to determine a file configuration rule corresponding to a file to be uploaded according to the service unit and the file type;

the normalization verification module 230 is configured to perform normalization verification on a file name of a file to be uploaded according to a file configuration rule;

the repeatability checking module 240 is configured to perform repeatability checking on the file to be uploaded that passes the normalization checking;

and the uploading module 250 is used for uploading the file to be uploaded which passes the repeatability check.

Optionally, the normalization verification module 230 is specifically configured to perform the following steps when performing normalization verification on a file name of a file to be uploaded according to a file configuration rule:

the repeatability verification module 240 is specifically configured to perform the following steps when performing the repeatability verification on the file to be uploaded that passes the normative verification:

S21, determining a file name set corresponding to the file to be uploaded, which passes through the standardization verification, from a database according to the service units and the file types of the file to be uploaded, judging whether an intersection exists between the file name of the file to be uploaded and the file name set, if so, executing S22, and if not, judging that the repeatability verification passes, wherein the database stores the uploaded files corresponding to different service units and the uploaded files corresponding to different file types;

s22, judging whether the file size of the first file is equal to the file size of the file to be uploaded, if not, executing S23, and if so, judging that the repeatability check fails, wherein the first file is a file corresponding to a file name with an intersection with the file name of the file to be uploaded in the file name set;

s23, judging whether the first file and the file to be uploaded are files uploaded by the same user, if so, judging that the repeatability check passes, and carrying out breakpoint continuous uploading on the file to be uploaded, and if not, judging that the repeatability check fails.

Optionally, after uploading the file to be uploaded that passes the repeatability check, the apparatus further includes: and the integrity checking module is used for checking the integrity of the file to be uploaded.

Optionally, after uploading the file to be uploaded that passes the repeatability check, the apparatus further includes: the storage module is used for extracting meta information of the file to be uploaded; storing the meta information of the file to be uploaded to a distributed database and a Kafka message queue; and synchronously writing the meta-information of the file to be uploaded into an ES database based on the meta-information of the file to be uploaded stored in the Kafka message queue.

Optionally, after synchronously writing the meta information of the file to be uploaded into the ES database, the apparatus further includes:

the consistency verification module is used for carrying out consistency verification on the meta information of the file to be uploaded written into the ES database, the meta information of the file to be uploaded stored in the distributed database and the meta information of the file to be uploaded stored in the Kafka message queue; and if the consistency check is not passed, carrying out data supplementing processing on the meta information of the file to be uploaded in the ES database.

Optionally, the apparatus further comprises:

the display module is used for monitoring the whole uploading link process in the uploading process of the file to be uploaded, acquiring the file uploading related information and visually displaying the file uploading related information, wherein the file uploading related information comprises file uploading progress, file uploading state and file archiving state information.

Optionally, when uploading the file to be uploaded that passes the repeatability verification, the uploading module 240 is specifically configured to: determining a target transmission mode of the file to be uploaded according to a transmission scene of the file to be uploaded, wherein the target transmission mode is an API mode or a Web mode; and uploading the file to be uploaded which passes the repeatability verification according to the target transmission mode.

The system for supporting the uploading of the space engineering file data and the full link management according to the embodiment of the present invention may execute the method for supporting the uploading of the space engineering file data and the full link management according to the embodiment of the present invention, and its implementation principle is similar, and actions executed by each module and unit in the system for supporting the uploading of the space engineering file data and the full link management according to each embodiment of the present invention correspond to steps in the method for supporting the uploading of the space engineering file data and the full link management according to each embodiment of the present invention, and detailed functional descriptions of each module in the system for supporting the uploading of the space engineering file data and the full link management may be referred to in the corresponding method for supporting the uploading of the space engineering file data and the full link management, which are not repeated herein.

The system supporting the space engineering file data uploading and the full link management may be a computer program (including program code) running in a computer device, for example, the system supporting the space engineering file data uploading and the full link management is an application software; the system may be used to perform the corresponding steps in the method provided by the embodiments of the present invention.

In some embodiments, the system for supporting the uploading of the data of the space engineering file and the full link management provided by the embodiments of the present invention may be implemented by combining software and hardware, and as an example, the system for supporting the uploading of the data of the space engineering file and the full link management provided by the embodiments of the present invention may be a processor in the form of a hardware decoding processor, which is programmed to perform the method for supporting the uploading of the data of the space engineering file and the full link management provided by the embodiments of the present invention, for example, the processor in the form of a hardware decoding processor may employ one or more application specific integrated circuits (ASIC, application Specific Integrated Circuit), DSPs, programmable logic devices (PLD, programmable Logic Device), complex programmable logic devices (CPLD, complex Programmable Logic Device), field programmable gate arrays (FPGA, field-Programmable Gate Array) or other electronic components.

In other embodiments, the method for supporting uploading and full link management of the space engineering file data provided by the embodiments of the present invention may be implemented in a software manner, and fig. 13 shows a system for supporting uploading and full link management of the space engineering file data stored in a memory, which may be software in the form of a program, a plug-in, and the like, and includes a series of modules including an acquisition module 210, a rule determination module 220, a verification module 230, and an uploading module 240, which are used to implement the method for supporting uploading and full link management of the space engineering file data provided by the embodiments of the present invention.

Based on the same principles as the methods shown in the embodiments of the present invention, there is also provided in the embodiments of the present invention an electronic device, which may include, but is not limited to: a processor and a memory; a memory for storing a computer program; a processor for executing the method according to any of the embodiments of the invention by invoking a computer program.

In an alternative embodiment, there is provided an electronic device, as shown in fig. 14, the electronic device 4000 shown in fig. 14 includes: a processor 4001 and a memory 4003. Wherein the processor 4001 is coupled to the memory 4003, such as via a bus 4002. Optionally, the electronic device 4000 may further comprise a transceiver 4004, the transceiver 4004 may be used for data interaction between the electronic device and other electronic devices, such as transmission of data and/or reception of data, etc. It should be noted that, in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 is not limited to the embodiment of the present invention.

The processor 4001 may be a CPU (Central Processing Unit ), general purpose processor, DSP (Digital Signal Processor, data signal processor), ASIC (Application Specific Integrated Circuit ), FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic device, transistor logic device, hardware components, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules and circuits described in connection with this disclosure. The processor 4001 may also be a combination that implements computing functionality, e.g., comprising one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

Bus 4002 may include a path to transfer information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture ) bus, or the like. The bus 4002 can be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 14, but not only one bus or one type of bus.

Memory 4003 may be, but is not limited to, ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, RAM (Random Access Memory ) or other type of dynamic storage device that can store information and instructions, EEPROM (Electrically Erasable Programmable Read Only Memory ), CD-ROM (Compact Disc Read Only Memory, compact disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

The memory 4003 is used for storing application program codes (computer programs) for executing the present invention and is controlled to be executed by the processor 4001. The processor 4001 is configured to execute application program codes stored in the memory 4003 to realize what is shown in the foregoing method embodiment. The electronic device shown in fig. 14 is only an example, and should not impose any limitation on the functions and application scope of the embodiment of the present invention.

Embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which when run on a computer, causes the computer to perform the corresponding method embodiments described above. According to another aspect of the present invention, there is also provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the methods provided in the implementation of the various embodiments described above.

Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

It should be appreciated that the flow charts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The computer-readable storage medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above-described embodiments.

The above description is only illustrative of the preferred embodiments of the present invention and of the principles of the technology employed. It will be appreciated by persons skilled in the art that the scope of the disclosure referred to in the present invention is not limited to the specific combinations of technical features described above, but also covers other technical features formed by any combination of the technical features described above or their equivalents without departing from the spirit of the disclosure. Such as the above-mentioned features and the technical features disclosed in the present invention (but not limited to) having similar functions are replaced with each other.

Claims

1. A method to support aerospace engineering document data upload and full-link management, which is characterized by including:

Get the business unit and file type of the file to be uploaded;

Determine the file configuration rules corresponding to the file to be uploaded according to the business unit and the file type;

According to the file configuration rules, perform normative verification on the file name of the file to be uploaded;

Perform repetitive verification on files to be uploaded that pass the normative verification;

Upload the files to be uploaded that pass the repeatability check.

2. The method according to claim 1, characterized in that, according to the file configuration rules, normatively verifying the file name of the file to be uploaded includes:

S11, according to the file configuration rules, split the file name of the file to be uploaded according to underscores to obtain i meta-information nodes;

S12, for each meta-information node, determine whether there is data under the meta-information node. If it is determined that there is data under the meta-information node, execute S13. If it is determined that there is no data under the meta-information node, it is determined that the normative verification failed. ;

S13. For each meta-information node, determine whether the field length corresponding to the meta-information node meets the preset field length requirement. If it meets the field length requirement, execute S14. If it does not meet the field length requirement, It is determined that the normative verification failed;

S14. For each meta-information node, determine whether the field range corresponding to the meta-information node meets the preset field range requirements. If it meets the field range requirements, execute S15. If it does not meet the field range requirements, It is determined that the normative verification failed;

S15. For each meta-information node, determine whether the field type corresponding to the meta-information node meets the preset field type requirements. If it meets the field type requirements, it is determined that the normative verification is successful. If it does not meet the preset field type requirements, it is determined that the normative verification is successful. If the field type requirements are met, the normative verification fails;

The repetitive verification of files to be uploaded that pass the normative verification includes:

S21. According to the business unit and file type of the file to be uploaded, determine the file name set corresponding to the file to be uploaded that has passed the normative verification from the database, and determine the file name of the file to be uploaded and the file name. Whether there is an intersection between the sets. If there is an intersection, perform S22. If there is no intersection, it is determined that the repeatability check has passed. The database stores uploaded files corresponding to different business units and uploaded files corresponding to different file types. upload files;

S22, determine whether the file size of the first file and the file size of the file to be uploaded are equal. If they are not equal, execute S23. If they are equal, it is determined that the repeatability check has failed, wherein the first file is the A file corresponding to a file name in the file name set that intersects with the file name of the file to be uploaded;

S23. Determine whether the first file and the file to be uploaded are files uploaded by the same user. If they are files uploaded by the same user, determine whether the repeatability check is passed, and resume the upload of the file to be uploaded. , if the file is not uploaded by the same user, it is determined that the repeatability check has failed.

3. The method according to claim 1, characterized in that, after uploading the file to be uploaded that passes the repeatability check, the method further includes:

Perform integrity verification on the files to be uploaded.

4. The method according to any one of claims 1 to 3, characterized in that, after uploading the file to be uploaded that passes the repeatability check, the method further includes:

Extract the meta-information of the file to be uploaded;

Store the meta-information of the file to be uploaded in a distributed database and Kafka message queue;

Based on the meta-information of the file to be uploaded stored in the Kafka message queue, the meta-information of the file to be uploaded is synchronously written into the ES database.

5. The method according to claim 4, characterized in that, after synchronously writing the meta information of the file to be uploaded into the ES database, the method further includes:

The meta information of the file to be uploaded written in the ES database, the meta information of the file to be uploaded stored in the distributed database, and the meta information of the file to be uploaded stored in the Kafka message queue Information is checked for consistency;

If the consistency check fails, the meta-information of the file to be uploaded in the ES database is supplemented with data.

6. The method according to any one of claims 1 to 3, characterized in that the method further comprises:

In the process of uploading the file to be uploaded, monitor the entire upload link process, obtain file upload-related information, and visually display the file upload-related information. The file upload-related information includes file upload progress, file upload status, and file upload status. Archive status information.

7. The method according to any one of claims 1 to 3, characterized in that uploading files to be uploaded that pass the repeatability check includes:

According to the transmission scenario of the file to be uploaded, determine the target transmission method of the file to be uploaded, and the target transmission method is API method or Web method;

According to the target transmission method, the files to be uploaded that pass the repeatability check are uploaded.

8. A system that supports aerospace engineering document data upload and full-link management, which is characterized by:

The acquisition module is used to obtain the business unit and file type of the file to be uploaded;

A rule determination module, configured to determine the file configuration rules corresponding to the files to be uploaded based on the business unit and the file type;

A normative verification module, used to normatively verify the file name of the file to be uploaded according to the file configuration rules;

A repeatability verification module, used to perform repeatability verification on files to be uploaded that pass the normative verification;

The upload module is used to upload files to be uploaded that pass the repeatability check.

9. An electronic device, characterized in that it includes a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any one of claims 1-7 is realized. The method described in one item.

10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method of any one of claims 1-7 is implemented. .