CN115760366B - Batch business processing method and system - Google Patents
Batch business processing method and systemInfo
- Publication number
- CN115760366B CN115760366B CN202211474323.5A CN202211474323A CN115760366B CN 115760366 B CN115760366 B CN 115760366B CN 202211474323 A CN202211474323 A CN 202211474323A CN 115760366 B CN115760366 B CN 115760366B
- Authority
- CN
- China
- Prior art keywords
- processing
- task
- service processing
- data
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application provides a batch business processing method and a batch business processing system, which can be used in the financial field, wherein the method comprises the steps of extracting business processing data of batch business to be processed through key fields to obtain a plurality of discrete data of different processing nodes; setting task identification for each discrete data, obtaining discrete tasks according to each discrete data and the corresponding task identification, repeatedly executing steps of respectively sending each discrete task to the corresponding service processing thread according to the task identification, performing service processing through the service processing thread, and updating the task identification until all the discrete task service processing is completed after the service processing is completed. The application can improve the efficiency of batch business processing.
Description
Technical Field
The application relates to the technical field of batch business processing, in particular to the financial field, and particularly relates to a batch business processing method and system.
Background
Proxy bulk business is one of the key modes of centralized business processing for financial institutions such as banks. The existing proxy batch service generally adopts online compressed messages or signatures and encrypted files to send and return the disc, so that the centralized interaction between the bank of the proxy batch service and the entrusted client is realized. The business data is reasonably arranged and processed in the bank through a timing and quasi-real-time mode, so that the centralized and unified business processing is realized. In order to improve service processing timeliness, the system generally adopts a mode of splitting data according to a file and record group, and improves the overall timeliness of the batch system by increasing the parallelism of service processing.
Under the condition that service data are uniformly distributed, the system processing performance can be effectively improved, but if a certain group of conditions (such as social security generation and the like) of large quantity exist in interactive application, service processing timeliness can be greatly reduced, and the concurrent thread numbers cannot solve the service processing timeliness, so that service processing experience difference of generation is large. In addition, the sequence of business processing is controlled by the dependence of the scene and the front and back items, the flexibility of the business processing is insufficient, the situation that the whole business processing is blocked because a certain link is slow is easily caused, and the business experience of proxy batch processing is greatly influenced.
Disclosure of Invention
The application aims to provide a batch business processing method which improves the efficiency of batch business processing. It is another object of the present application to provide a batch business processing system. It is a further object of the application to provide a computer device. It is a further object of the application to provide a readable medium.
In order to achieve the above object, an aspect of the present application discloses a batch service processing method, including:
Extracting service processing data of batch service to be processed through key fields to obtain a plurality of discrete data of different processing nodes;
Setting a task identifier for each piece of discrete data, and obtaining a discrete task according to each piece of discrete data and the corresponding task identifier;
and repeatedly executing the steps of respectively sending each discrete task to a corresponding service processing thread according to the task identifier, performing service processing through the service processing thread, and updating the task identifier until all the discrete task service processing is completed after the service processing is completed.
Preferably, the extracting, by the key field, the service processing data of the batch service to be processed to obtain the plurality of discrete data of different processing nodes specifically includes:
Extracting the business processing data according to the first key field to obtain business processing data;
extracting the service processing data according to a second key field to obtain data to be checked;
Extracting the business processing data according to a third key field to obtain a summary record;
and extracting the business processing data according to the fourth key field to obtain a name record.
Preferably, the step of sending each discrete task to a corresponding service processing thread according to the task identifier, performing service processing by the service processing thread, and updating the task identifier after the service processing is completed specifically includes:
Determining a next processing node for processing the discrete task according to the task identifier, and taking the next processing node as a current node;
The discrete task is sent to a target service processing thread corresponding to the current node, and service processing is carried out through the target service processing thread;
And after the business processing is completed, updating the task identifier into the task identifier processed by the current node.
Preferably, the processing nodes comprise a service acceptance node, a data inspection node, a summary borrowing processing node, a detail party processing node, a summary lending processing node and a data disc returning node.
Preferably, when the task identifier is a to-be-serviced task acceptance identifier, the sending each discrete task to a corresponding service processing thread according to the task identifier, performing service processing by the service processing thread, and updating the task identifier after the service processing is completed specifically includes:
And sending the business acceptance discrete task formed according to the business handling data and the business acceptance identification to a business processing thread corresponding to a business acceptance node for business processing, and updating the business acceptance identification to be the business accepted identification after the business processing is completed.
Preferably, when the task identifier is a service accepted identifier, the sending each discrete task to a corresponding service processing thread according to the task identifier, performing service processing by the service processing thread, and updating the task identifier after the service processing is completed specifically includes:
and sending a data checking task formed according to the data to be checked and the service accepted identification to a service processing thread corresponding to a data checking node for service processing, and updating the service accepted identification into a data checking passing identification after the service processing is completed.
Preferably, when the task identifier is a data inspection passing identifier, the discrete tasks are respectively sent to corresponding service processing threads according to the task identifier, service processing is performed by the service processing threads, and updating the task identifier after the service processing is completed specifically includes:
and sending the summarizing and borrowing processing task formed according to the summarizing record and the data checking pass identifier to a business processing thread corresponding to the summarizing and borrowing processing node for business processing, and updating the data checking pass identifier to be the summarizing and borrowing processing completion identifier after the business processing is completed.
Preferably, when the task identifier is a data inspection passing identifier, the discrete tasks are respectively sent to corresponding service processing threads according to the task identifier, service processing is performed by the service processing threads, and updating the task identifier after the service processing is completed specifically includes:
And sending a detail processing task formed by the identification according to the detail record and the data inspection to a business processing thread corresponding to a detail party processing node for business processing, and updating the data inspection through identification to be the detail party processing completion identification after the business processing is completed.
Preferably, when the task identifier of the summary borrowing processing task is a summary borrowing processing completion identifier, the task identifier of the detail processing task is a detail party processing completion identifier, each discrete task is respectively sent to a corresponding service processing thread according to the task identifier, service processing is performed through the service processing thread, and updating the task identifier specifically includes:
And sending the summary credit processing task formed according to the summary record, the name record and the corresponding identifier to a business processing thread corresponding to a summary credit processing node for business processing, and updating the summary credit processing completion identifier and the detail party processing completion identifier to be summary credit processing completion identifiers after the business processing is completed.
Preferably, when the task identifier of the summary credit processing task is a summary credit processing completion identifier, each discrete task is sent to a corresponding service processing thread according to the task identifier, service processing is performed by the service processing thread, and updating the task identifier after the service processing is completed specifically includes:
and sending the data disc-returning task formed according to the summary record, the name record and the summary credit processing completion identification to a service processing thread corresponding to a data disc-returning node for service processing, and updating the summary credit processing completion identification to be a service processing completion identification after the service processing is completed, wherein all discrete task service processing is completed.
The application also discloses a batch business processing system, which comprises:
The data discrete module is used for extracting service processing data of batch service to be processed through key fields to obtain a plurality of discrete data of different processing nodes;
The discrete task module is used for setting task identifiers for each piece of discrete data and obtaining discrete tasks according to each piece of discrete data and the corresponding task identifiers;
and the task processing module is used for repeatedly executing the steps of respectively sending each discrete task to the corresponding service processing thread according to the task identifier, performing service processing through the service processing thread, and updating the task identifier until all the discrete task service processing is completed after the service processing is completed.
The application also discloses a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor,
The processor, when executing the program, implements the method as described above.
The application also discloses a computer readable medium, on which a computer program is stored,
The program, when executed by a processor, implements the method as described above.
The batch business processing method of the application extracts a plurality of discrete data of different processing nodes through key fields, sets a task identifier for each discrete data, obtains discrete tasks according to each discrete data and the corresponding task identifier, repeatedly executes the steps of respectively sending each discrete task to the corresponding business processing thread according to the task identifier, and carries out business processing through the business processing thread, and after the business processing is completed, the step of updating the task identifier is carried out until all the discrete task business processing is completed. Therefore, the application integrates and splits the business links of the batch agent business to obtain a plurality of processing nodes, determines the discrete data corresponding to different processing nodes, respectively carries out business processing on different discrete data through business processing threads of different processing nodes, sets task identifiers for the discrete data to form discrete tasks, realizes the control of the processing sequence of the discrete tasks through the task identifiers, and ensures the correct processing of batch businesses. Therefore, the large batch service can be divided into discrete tasks and distributed to different service threads for processing, the problem that the batch service is slow to process due to the limitation of the performance of a processor when single thread processing occurs is prevented, the peak clipping and valley filling of the batch service processing are realized, the system pressure for processing the batch service is reduced, and the processing efficiency of the batch service to be processed is improved.
Drawings
In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an embodiment of a bulk business processing method of the present application;
FIG. 2 is a flowchart of an embodiment S100 of a bulk business processing method of the present application;
FIG. 3 is a flow chart comparing a specific embodiment of the batch business processing method of the present application with the prior art;
FIG. 4 is a schematic diagram showing a comparison of an embodiment of a batch service processing method of the present application with a prior art incoming queue;
FIG. 5 is a flowchart of an embodiment S300 of a bulk business processing method of the present application;
FIG. 6 is a block diagram illustrating a particular embodiment of a bulk business processing system of the present application;
Fig. 7 shows a schematic diagram of a computer device suitable for use in implementing embodiments of the application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
It should be noted that the batch business processing method and system disclosed by the application can be used in the financial field and can also be used in any field except the financial field, and the application field of the batch business processing method and system disclosed by the application is not limited.
It should be noted that, in one or more embodiments of the present application, the proxy lot refers to a background lot mode for handling related services according to a proxy principal, where online or lot file is generally used to accept services, and service processing data of the lot typically includes summary records and detail records. The summary record refers to the summary statistical record sent by the principal when the principal initiates proxy processing, such as a proxy service, where the summary record is a fund source of a proxy client and is a borrower. The summary record may contain agent agreement information, a summary total and a summary amount for the total point check. The detail record refers to the detail information, such as the issuing service, sent by the client when the client initiates the proxy processing, and the detail record refers to the fund of the issuing client and is the lender. The detail record may include elements such as a serial number of the generation, a generation account, and an amount.
In order to facilitate understanding of the technical scheme provided by the application, the following description will explain relevant contents of the technical scheme of the application. The batch service processing method provided by the embodiment of the application integrates and splits the service links of batch proxy service to obtain a plurality of processing nodes, determines discrete data corresponding to different processing nodes, respectively carries out service processing on different discrete data through service processing threads of different processing nodes, sets task identifiers for the discrete data to form discrete tasks, realizes control of the processing sequence of the discrete tasks through the task identifiers, and ensures correct processing of batch service. Therefore, the large batch service can be divided into discrete tasks and distributed to different service threads for processing, the problem that the batch service is slow to process due to the limitation of the performance of a processor when single thread processing occurs is prevented, the peak clipping and valley filling of the batch service processing are realized, the system pressure for processing the batch service is reduced, and the processing efficiency of the batch service to be processed is improved.
The implementation process of the batch service processing method provided by the embodiment of the application is described below by taking a batch service processing system as an execution main body as an example. It can be understood that the execution body of the batch service processing method provided by the embodiment of the application includes, but is not limited to, the batch service processing system.
According to one aspect of the application, the present embodiment discloses a batch business processing method. As shown in fig. 1, in this embodiment, the method includes:
S100, extracting service processing data of batch service to be processed through key fields to obtain a plurality of discrete data of different processing nodes.
And S200, setting task identifiers for each piece of discrete data, and obtaining discrete tasks according to each piece of discrete data and the corresponding task identifiers.
And S300, repeatedly executing the steps of respectively sending each discrete task to a corresponding service processing thread according to the task identifier, performing service processing through the service processing thread, and updating the task identifier until all the discrete task service processing is completed after the service processing is completed.
The batch business processing method of the application extracts a plurality of discrete data of different processing nodes through key fields, sets a task identifier for each discrete data, obtains discrete tasks according to each discrete data and the corresponding task identifier, repeatedly executes the steps of respectively sending each discrete task to the corresponding business processing thread according to the task identifier, and carries out business processing through the business processing thread, and after the business processing is completed, the step of updating the task identifier is carried out until all the discrete task business processing is completed. Therefore, the application integrates and splits the business links of the batch agent business to obtain a plurality of processing nodes, determines the discrete data corresponding to different processing nodes, respectively carries out business processing on different discrete data through business processing threads of different processing nodes, sets task identifiers for the discrete data to form discrete tasks, realizes the control of the processing sequence of the discrete tasks through the task identifiers, and ensures the correct processing of batch businesses. Therefore, the large batch service can be divided into discrete tasks and distributed to different service threads for processing, the problem that the batch service is slow to process due to the limitation of the performance of a processor when single thread processing occurs is prevented, the peak clipping and valley filling of the batch service processing are realized, the system pressure for processing the batch service is reduced, and the processing efficiency of the batch service to be processed is improved.
In a preferred embodiment, as shown in fig. 2, the step S100 of extracting, by using a key field, the service processing data of the batch service to be processed to obtain a plurality of discrete data of different processing nodes specifically includes:
s110, extracting the business processing data according to the first key field to obtain business processing data.
And S120, extracting the service processing data according to the second key field to obtain data to be checked.
And S130, extracting the business processing data according to the third key field to obtain a summary record.
And S140, extracting the business processing data according to the fourth key field to obtain a name record.
In particular, it will be appreciated that business process data for bulk business typically contains summary records and detail records. The summary record refers to the summary statistical record sent by the principal when the principal initiates proxy processing, such as a proxy service, where the summary record is a fund source of a proxy client and is a borrower. The summary record may contain agent agreement information, a summary total and a summary amount for the total point check. The detail record refers to the detail information, such as the issuing service, sent by the client when the client initiates the proxy processing, and the detail record refers to the fund of the issuing client and is the lender. The detail record may include elements such as a serial number of the generation, a generation account, and an amount.
In the batch business processing process, the business processing flow generally comprises business acceptance, data inspection, data hashing, business inspection, summarizing and borrowing processing, detail party processing, summarizing and lending processing (back flushing), business disc returning and other processing links. Preferably, according to the characteristics of batch service processing, a plurality of processing nodes including a service acceptance node, a data inspection node, a summary borrowing processing node, a detail party processing node, a summary lending processing node and a data disc returning node can be arranged to break up the group processing flow of the old flow, a serial flow formed by the front-and-back item dependency control of the operation is adjusted to be a concurrent control model of a thread assembly line, the complexity of a single flow is simplified, and the concurrency and timeliness of service processing are improved. For example, in the service acceptance step, the data packet is processed by scanning the service, if the data packet is not processed normally, the acceptance operation of the service data packet is performed, and after the acceptance is completed, the mobile data enters the area to be processed by the service, and waits for the processing of the subsequent data inspection and processing thread, as shown in fig. 3.
Therefore, in order to obtain the discrete service data required by different processing links, in the preferred embodiment, the preset keywords of different processing links are used for extracting the service processing data to obtain a plurality of discrete data, the service processing data are scattered, and the service processing data do not sequentially pass through the plurality of processing links in a serial control mode. Specifically, KAFKA message queues can be introduced, and the data to be processed is processed uniformly and discretely through the formulated key fields. Each service processing layer is added with KAFKA acceptance threads, accepts and lands the service data, and the subsequent service processing is carried out, as shown in fig. 4.
In the preferred embodiment, the business processing data is extracted through the first keyword, the second keyword, the third keyword and the fourth keyword to obtain business processing data, data to be checked, summary records and name records. In the prior art, data are split and discrete according to groups, business processing is performed in series according to groups, and when one group of data is particularly large, the bottleneck of data processing is easily caused. In the application, the system divides the data distribution of the summarizing party and the detail party through the hash of the key words (for example, the medium number) to obtain the summarizing record and the detail record, and forms a summarizing record hash queue and a detail record hash queue respectively, wherein the summarizing record is uniformly placed in the summarizing record hash queue for processing, and the detail record is uniformly placed in the detail record hash queue for processing. The summary borrowing is inserted into a summary master control queue after the processing is finished and is used for the subsequent business processing of controlling the detail party, the summary party processing only pays attention to the front of the implementation of the summary party, and the detail party processing only pays attention to the front of the implementation of the detail party, so that the high-level discrete, tight and orderly high-efficiency flow processing of business data is realized. Wherein the first keyword, the second keyword, the third keyword and the fourth keyword may be determined according to a specific batch service, which is not limited in the present application.
In a preferred embodiment, as shown in fig. 5, the step S300 of sending each discrete task to a corresponding service processing thread according to the task identifier, performing service processing by the service processing thread, and updating the task identifier after the service processing is completed specifically includes:
and S301, determining a next processing node for processing the discrete task according to the task identification, and taking the next processing node as a current node.
S302, the discrete task is sent to a target service processing thread corresponding to the current node, and service processing is carried out through the target service processing thread.
And S303, after the business processing is completed, updating the task identifier into the task identifier processed by the current node.
Specifically, it can be understood that a task identifier can be set for discrete data, and in the service processing process, after the service processing of the current thread is completed, the task identifier can be updated to determine the next service processing thread through the content of the task identifier, so that the ordered control of the processing processes of a plurality of discrete data is realized, and the correct processing of batch services is ensured.
In an embodiment, the processing nodes include a service acceptance node, a data inspection node, a summary borrowing processing node, a detail party processing node, a summary lending processing node and a data rollback node. When the task identifier is a to-be-processed task acceptance identifier, the step S300 of sending each discrete task to a corresponding service processing thread according to the task identifier, performing service processing by the service processing thread, and updating the task identifier after the service processing is completed specifically includes:
and S310, sending the business acceptance discrete task formed according to the business handling data and the business acceptance identification to a business processing thread corresponding to a business acceptance node for business processing, and updating the business acceptance identification to be the business accepted identification after the business processing is completed.
Specifically, it can be understood that before the batch service processing, whether the service can be processed needs to be confirmed, that is, the service acceptance processing needs to be performed, and the specific service acceptance processing process is a conventional technical means in the art and is not described herein. The business processing data can be extracted according to a preset first key field to obtain business processing data, a task identifier of a to-be-processed business acceptance identifier is initially set for the business processing data, then a business acceptance discrete task is formed according to the to-be-processed business acceptance identifier and the business processing data, and the business acceptance discrete task is determined to be required to be sent to a business processing thread corresponding to a business acceptance node for business processing according to the to-be-processed business acceptance identifier. And updating the identification to be accepted by the service as the identification accepted by the service when the service processing is finished, namely, the acceptance can be determined according to the service processing data. Therefore, the next processing node is determined according to the service accepted identification and the sequence of the plurality of processing nodes, and then the discrete task corresponding to the next processing node can be sent to the next processing node for service processing.
Further, optionally, when the task identifier is a service accepted identifier, the step S300 of sending each discrete task to a corresponding service processing thread according to the task identifier, performing service processing by the service processing thread, and updating the task identifier after the service processing is completed may further include:
And S320, sending a data checking task formed according to the data to be checked and the service accepted identification to a service processing thread corresponding to a data checking node for service processing, and updating the service accepted identification to be a data checking passing identification after the service processing is completed.
Specifically, it can be understood that before the batch service processing, the accepted batch service needs to be subjected to data inspection, and the specific data inspection process is a conventional technical means in the field, which is not described herein. And extracting the service processing data according to a preset second key field to obtain data to be checked, wherein the task identifier of the data to be checked of the batch of the service to be processed is the service accepted identifier. Forming a data checking task according to the service accepted identification and the data to be checked, and determining that the data checking task needs to be sent to a service processing thread corresponding to the data checking node for service processing according to the service accepted identification. When the service processing is completed, namely the data inspection passes, the update service has accepted the identification as the data inspection passing identification. Therefore, the next processing node is determined according to the data check passing identifier and the sequence of the plurality of processing nodes, and then the discrete task corresponding to the next processing node can be sent to the next processing node for business processing.
Further, optionally, when the task identifier is a data inspection passing identifier, the S300 sends each discrete task to a corresponding service processing thread according to the task identifier, and performs service processing through the service processing thread, and after the service processing is completed, updating the task identifier may further include:
And S330, sending the summarization borrowing processing task formed by the summarization record and the data inspection passing identifier to a business processing thread corresponding to the summarization borrowing processing node for business processing, and updating the data inspection passing identifier to be the summarization borrowing processing completion identifier after the business processing is completed.
Specifically, it can be understood that the summary record can be obtained by extracting the service processing data according to the preset third key field, and at this time, the task identifier of the summary record of the batch of service to be processed is the data inspection passing identifier. And forming a summary borrowing processing task according to the data inspection passing identifier and the summary record, and determining that the summary borrowing processing task needs to be sent to a business processing thread corresponding to the summary borrowing processing node for business processing according to the data inspection passing identifier. When the business processing is completed, namely the deduction business processing of the borrower is completed, the updated data inspection passes through the mark to be the summary borrowing processing completion mark. Therefore, the next processing node is determined according to the summarizing and borrowing processing completion identification and the sequence of the plurality of processing nodes, and then the discrete task corresponding to the next processing node can be sent to the next processing node for business processing.
Further, optionally, when the task identifier is a data inspection passing identifier, the S300 sends each discrete task to a corresponding service processing thread according to the task identifier, and performs service processing through the service processing thread, and after the service processing is completed, updating the task identifier may further include:
and S340, sending the detail processing task formed by the identification according to the name record and the data inspection to a business processing thread corresponding to a detail processing node for business processing, and updating the data inspection through identification to be the detail processing completion identification after the business processing is completed.
Specifically, it can be understood that the business processing data can be extracted according to the preset fourth key field to obtain a name record, and at this time, the task identifier of the summary record of the batch of business to be processed is a data inspection passing identifier. And forming a detail processing task according to the data inspection through the identification and the name record, and determining that the detail processing task needs to be sent to a business processing thread corresponding to a detail party processing node for business processing according to the data inspection through the identification. When the business process is completed, that is, the transfer process of the detail party is completed, the update data check passes the identification for the completion of the process of the detail party. Therefore, the next processing node is determined according to the detail party processing completion identification and the sequence of the plurality of processing nodes, and then the discrete task corresponding to the next processing node can be sent to the next processing node for business processing.
Further, optionally, when the task identifier of the summary borrowing processing task is a summary borrowing processing completion identifier, the task identifier of the detail processing task is a detail party processing completion identifier, and the step S300 of sending each discrete task to a corresponding service processing thread according to the task identifier, and performing service processing by the service processing thread, after the service processing is completed, updating the task identifier may further include:
And S350, sending the summary credit processing task formed according to the summary record, the name record and the corresponding identifier to a business processing thread corresponding to a summary credit processing node for business processing, and updating the summary credit processing completion identifier and the detail party processing completion identifier to be summary credit processing completion identifiers after business processing is completed.
Specifically, it can be understood that after the processing in S330 and S340 is completed, that is, when the task identifiers of the summary borrowing processing task and the detail processing task are the summary borrowing processing completion identifier and the detail processing completion identifier, summary lending (back flushing) processing is required according to the service processing results of the summary borrowing processing task and the detail processing task, that is, the service failing to transfer in the detail processing process returns to the borrower, and back flushing is completed, so as to prevent inconsistent lending. And forming an summarization credit processing task according to the summarization record, the name record and the corresponding mark, sending the summarization credit processing task to a business processing thread corresponding to a summarization credit processing node for business processing, and updating the summarization credit processing completion mark and the detail party processing completion mark to be summarization credit processing completion marks after business processing is completed. Therefore, the next processing node is determined according to the summary credit processing completion identification and the sequence of the plurality of processing nodes, and then the discrete task corresponding to the next processing node can be sent to the next processing node for business processing.
Further, optionally, when the task identifier of the summary credit processing task is a summary credit processing completion identifier, the S300 sends each discrete task to a corresponding service processing thread according to the task identifier, and performs service processing through the service processing thread, and updating the task identifier after the service processing is completed specifically includes:
And S360, transmitting the data disc-returning task formed according to the summary record, the name record and the summary credit processing completion identification to a service processing thread corresponding to the data disc-returning node for service processing, and updating the summary credit processing completion identification to be a service processing completion identification after the service processing is completed, wherein all discrete task service processing is completed.
Specifically, it can be understood that after the summary credit processing is completed, the batch service needs to be rewound, that is, the processing result of the batch service needs to be determined, and the processing result is fed back to the client. When all processing nodes before the data disc returning node process batch business, namely the task mark is summary credit processing completion mark, the disc returning operation can be triggered, and the small files of the processing results of the summary borrowing processing node, the detail party processing node and the summary credit processing node are combined to obtain large files in a multi-library parallel mode, so that the rapid disc returning of the data files is realized, on one hand, the efficiency is improved, and on the other hand, the dependent coupling of different data files is reduced.
The application can introduce a message engine through decoupling each link of proxy batch service processing, uniformly and discretely process proxy batch service data, and realize ordered service processing by means of refined instruction signals, and has the main advantages that 1) the service processing links are decoupled, the abnormal influence of the service processing links is reduced, and the running safety of the system is improved. 2) The service processing data is discretized, so that peak clipping and valley filling of service processing are effectively realized, the cost loss of the system is reduced, and the running stability of the system is improved. 3) And a message engine is introduced to construct an instruction signal of a service processing step, so that the ordered processing of the service is effectively realized, and the flexibility of a batch proxy service system is improved. 4) The centralized unified processing is adjusted to a streaming parallel service processing mode, so that the interaction among interactive applications is reduced.
Based on the same principle, the embodiment also discloses a batch business processing system. As shown in fig. 6, in the present embodiment, the system includes a data discrete module 11, a discrete task module 12, and a task processing module 13.
The data discrete module 11 is configured to extract, through a key field, service processing data of a batch of services to be processed to obtain a plurality of discrete data of different processing nodes.
The discrete task module 12 is configured to set a task identifier for each discrete data, and obtain a discrete task according to each discrete data and the corresponding task identifier.
The task processing module 13 is configured to repeatedly execute the step of sending each discrete task to a corresponding service processing thread according to the task identifier, performing service processing by using the service processing thread, and updating the task identifier after the service processing is completed until all the discrete task service processing is completed.
Since the principle of solving the problem of the system is similar to that of the above method, the implementation of the system can be referred to the implementation of the method, and will not be repeated here.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer device, which may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
In a typical example, the computer apparatus includes a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing a method performed by a client as described above when executing the program or implementing a method performed by a server as described above when executing the program.
Referring now to FIG. 7, there is illustrated a schematic diagram of a computer device 600 suitable for use in implementing embodiments of the present application.
As shown in fig. 7, the computer apparatus 600 includes a Central Processing Unit (CPU) 601, which can perform various appropriate works and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the system 600 are also stored. The CPU601, ROM602, and RAM603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Connected to the I/O interface 605 are an input section 606 including a keyboard, a mouse, and the like, an output section 607 including a Cathode Ray Tube (CRT), a liquid crystal feedback device (LCD), and the like, and a speaker, and the like, a storage section 608 including a hard disk, and the like, and a communication section 609 including a network interface card such as a LAN card, a modem, and the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on drive 610 as needed, so that a computer program read therefrom is mounted as needed as storage section 608.
In particular, according to embodiments of the present application, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in the same piece or pieces of software and/or hardware when implementing the present application.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.
Claims (11)
1. A batch business processing method, comprising:
Extracting service processing data of batch service to be processed through key fields to obtain a plurality of discrete data of different processing nodes;
Setting a task identifier for each piece of discrete data, and obtaining a discrete task according to each piece of discrete data and the corresponding task identifier;
Repeatedly executing the steps of respectively sending each discrete task to a corresponding service processing thread according to the task identifier, performing service processing through the service processing thread, and updating the task identifier until all the discrete task service processing is completed after the service processing is completed;
The method for extracting the service processing data of the batch service to be processed through the key field to obtain a plurality of discrete data of different processing nodes specifically comprises the following steps:
Extracting the business processing data according to the first key field to obtain business processing data;
extracting the service processing data according to a second key field to obtain data to be checked;
Extracting the business processing data according to a third key field to obtain a summary record;
Extracting the service processing data according to a fourth key field to obtain a detail record;
The processing nodes comprise a service acceptance node, a data checking node, a summary borrowing processing node, a detail party processing node, a summary lending processing node and a data disc returning node.
2. The batch service processing method according to claim 1, wherein the sending each discrete task to a corresponding service processing thread according to the task identifier, performing service processing by the service processing thread, and updating the task identifier after the service processing is completed specifically includes:
Determining a next processing node for processing the discrete task according to the task identifier, and taking the next processing node as a current node;
The discrete task is sent to a target service processing thread corresponding to the current node, and service processing is carried out through the target service processing thread;
And after the business processing is completed, updating the task identifier into the task identifier processed by the current node.
3. The batch service processing method according to claim 1, wherein when the task identifier is a to-be-serviced task acceptance identifier, the method sends each discrete task to a corresponding service processing thread according to the task identifier, and performs service processing by the service processing thread, and updating the task identifier specifically includes:
And sending the business acceptance discrete task formed according to the business handling data and the business acceptance identification to a business processing thread corresponding to a business acceptance node for business processing, and updating the business acceptance identification to be the business accepted identification after the business processing is completed.
4. The batch service processing method according to claim 3, wherein when the task identifier is a service accepted identifier, the discrete tasks are sent to corresponding service processing threads according to the task identifier, the service processing is performed by the service processing threads, and updating the task identifier after the service processing is completed specifically includes:
and sending a data checking task formed according to the data to be checked and the service accepted identification to a service processing thread corresponding to a data checking node for service processing, and updating the service accepted identification into a data checking passing identification after the service processing is completed.
5. The batch service processing method according to claim 4, wherein when the task identifier is a data inspection passing identifier, the discrete tasks are respectively sent to corresponding service processing threads according to the task identifier, service processing is performed by the service processing threads, and updating the task identifier after the service processing is completed specifically includes:
and sending the summarizing and borrowing processing task formed according to the summarizing record and the data checking pass identifier to a business processing thread corresponding to the summarizing and borrowing processing node for business processing, and updating the data checking pass identifier to be the summarizing and borrowing processing completion identifier after the business processing is completed.
6. The batch service processing method according to claim 5, wherein when the task identifier is a data inspection passing identifier, the discrete tasks are respectively sent to corresponding service processing threads according to the task identifier, service processing is performed by the service processing threads, and updating the task identifier after the service processing is completed specifically includes:
And sending a detail processing task formed by the identification according to the detail record and the data inspection to a business processing thread corresponding to a detail party processing node for business processing, and updating the data inspection through identification to be the detail party processing completion identification after business processing is completed.
7. The batch service processing method according to claim 6, wherein when the task identifier of the summary processing task is a summary processing completion identifier, the task identifier of the detail processing task is a detail party processing completion identifier, the discrete tasks are respectively sent to the corresponding service processing threads according to the task identifier, the service processing is performed by the service processing threads, and updating the task identifier after the service processing is completed specifically includes:
And sending the summary credit processing task formed according to the summary record, the detail record and the corresponding identifier to a business processing thread corresponding to a summary credit processing node for business processing, and updating the summary credit processing completion identifier and the detail party processing completion identifier to be summary credit processing completion identifiers after the business processing is completed.
8. The batch service processing method according to claim 7, wherein when the task identifier of the summary credit processing task is a summary credit processing completion identifier, the discrete tasks are sent to corresponding service processing threads according to the task identifier, and service processing is performed by the service processing threads, and updating the task identifier after service processing is completed specifically includes:
And sending the data disc-returning task formed according to the summary record, the detail record and the summary credit processing completion identification to a service processing thread corresponding to a data disc-returning node for service processing, and updating the summary credit processing completion identification to be a service processing completion identification after the service processing is completed, wherein all discrete task service processing is completed.
9. A batch business processing system, comprising:
The data discrete module is used for extracting service processing data of batch service to be processed through key fields to obtain a plurality of discrete data of different processing nodes;
The discrete task module is used for setting task identifiers for each piece of discrete data and obtaining discrete tasks according to each piece of discrete data and the corresponding task identifiers;
The task processing module is used for repeatedly executing the steps of respectively sending each discrete task to a corresponding service processing thread according to the task identifier, performing service processing through the service processing thread, and updating the task identifier until all the discrete task service processing is completed after the service processing is completed;
the data discrete module is specifically configured to extract the service processing data according to a first key field to obtain service processing data;
extracting the service processing data according to a second key field to obtain data to be checked;
Extracting the business processing data according to a third key field to obtain a summary record;
Extracting the service processing data according to a fourth key field to obtain a detail record;
The processing nodes comprise a service acceptance node, a data checking node, a summary borrowing processing node, a detail party processing node, a summary lending processing node and a data disc returning node.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that,
The processor implementing the method according to any of claims 1-8 when executing the program.
11. A computer readable medium having a computer program stored thereon, characterized in that,
The program, when executed by a processor, implements the method of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211474323.5A CN115760366B (en) | 2022-11-23 | 2022-11-23 | Batch business processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211474323.5A CN115760366B (en) | 2022-11-23 | 2022-11-23 | Batch business processing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115760366A CN115760366A (en) | 2023-03-07 |
| CN115760366B true CN115760366B (en) | 2025-09-23 |
Family
ID=85335981
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211474323.5A Active CN115760366B (en) | 2022-11-23 | 2022-11-23 | Batch business processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115760366B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109558237A (en) * | 2017-09-27 | 2019-04-02 | 北京国双科技有限公司 | A kind of task status management method and device |
| CN110515718A (en) * | 2019-08-30 | 2019-11-29 | 深圳前海微众银行股份有限公司 | Method, device, equipment and medium for batch task breakpoint continuation |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8990820B2 (en) * | 2008-12-19 | 2015-03-24 | Microsoft Corporation | Runtime task with inherited dependencies for batch processing |
| CN111813805A (en) * | 2019-04-12 | 2020-10-23 | 中国移动通信集团河南有限公司 | A data processing method and device |
-
2022
- 2022-11-23 CN CN202211474323.5A patent/CN115760366B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109558237A (en) * | 2017-09-27 | 2019-04-02 | 北京国双科技有限公司 | A kind of task status management method and device |
| CN110515718A (en) * | 2019-08-30 | 2019-11-29 | 深圳前海微众银行股份有限公司 | Method, device, equipment and medium for batch task breakpoint continuation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115760366A (en) | 2023-03-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11074245B2 (en) | Method and device for writing service data in block chain system | |
| TWI706278B (en) | Block chain-based transaction processing method and device, and electronic equipment | |
| KR20200120863A (en) | Parallel execution of transactions in a distributed ledger system | |
| CN112189199B (en) | Privacy preserving data collection and analysis | |
| CN111226206A (en) | Take snapshots of blockchain data | |
| US20230052935A1 (en) | Asynchronous accounting method and apparatus for blockchain, medium and electronic device | |
| Garcia Bringas et al. | BlockChain platforms in financial services: current perspective | |
| CN114138834B (en) | Message processing method, device, equipment, storage medium and program product | |
| CN111316256A (en) | Taking snapshots of blockchain data | |
| CN116542668A (en) | Block chain-based data processing method, equipment and readable storage medium | |
| CN113704222A (en) | Method and device for processing service request | |
| CN111597264A (en) | Block chain accounting method and device | |
| CN117957528A (en) | Method and system for parallel processing of smart contracts in a permissioned blockchain | |
| CN114722398B (en) | Smart contract anomaly detection method, blockchain network, device and storage medium | |
| CN115760366B (en) | Batch business processing method and system | |
| CN114218240B (en) | Data change method, device, equipment, medium and program product | |
| CN114650287A (en) | Breakpoint continuous making method, device and system of data file | |
| CN112581277A (en) | Data processing method and device | |
| CN112291241A (en) | Firewall wall opening method, firewall wall opening device and terminal equipment | |
| CN116775772B (en) | Data processing method, device, equipment and readable storage medium | |
| US20240089128A1 (en) | Blockchain monitoring platform | |
| CN114268540B (en) | Rule engine optimization method, device and equipment | |
| CN110889040B (en) | Method and device for pushing information | |
| CN117235122A (en) | Data processing method, apparatus, device, readable storage medium, and program product | |
| CN110430256B (en) | Method, device and computer system for pushing transaction message |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |