[go: up one dir, main page]

CN103489061A - Batch data processing device and batch data processing method - Google Patents

Batch data processing device and batch data processing method Download PDF

Info

Publication number
CN103489061A
CN103489061A CN201310397318.3A CN201310397318A CN103489061A CN 103489061 A CN103489061 A CN 103489061A CN 201310397318 A CN201310397318 A CN 201310397318A CN 103489061 A CN103489061 A CN 103489061A
Authority
CN
China
Prior art keywords
data
document
interim
physics
batch
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201310397318.3A
Other languages
Chinese (zh)
Inventor
李亮
宋亚娟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
You Pu Information Technology Co., Ltd of UFSOFT
Original Assignee
Yonyou Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yonyou Software Co Ltd filed Critical Yonyou Software Co Ltd
Priority to CN201310397318.3A priority Critical patent/CN103489061A/en
Publication of CN103489061A publication Critical patent/CN103489061A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a batch data processing device applied to an EPR system. The batch data processing device comprises an acquiring unit used for acquiring a plurality of upstream receipts, a caching unit used for setting up a corresponding temporary physical table for each upstream receipt in the upstream receipts and caching data in each upstream receipt to the corresponding temporary physical table so as to obtain a plurality of temporary physical tables, an upgrading unit used for upgrading the temporary physical tables according to preset service requirements, and a receipt generating unit used for respectively inserting data in the upgraded temporary physical tables into corresponding service tables of downstream receipts according to the service order so as to generate a plurality of downstream receipts. Correspondingly, the invention further provides a batch data processing method. Through the technical scheme of the batch data processing device and the batch data processing method, one-time large-volume operation can be realized in its true sense, and the deadlock phenomenon can be prevented from occurring at the same time.

Description

Batch data treating apparatus and batch data disposal route
Technical field
The present invention relates to technical field of data processing, in particular to a kind of batch data treating apparatus and batch data disposal route.
Background technology
Existing enterprise is being used ERP(Enterprise Resource Planning, enterprise resource planning) during software systems, along with passing service time and business scale constantly extends expansion, the business datum amount rapid expansion of enterprise, and there are many cover Information software management systems and the slack phenomenon of data transmission while depositing, while causing original ERP management software system middle and upper reaches business paper Mass production downstream traffic document, data transmission performance constantly descends, and even occurs that whole system does not respond the situation of (seemingly-dead machine).If good solution is not paid much attention to and be can not get to these problems, will soon badly influence so the normal work flow of enterprise, even whole enterprise can yield and use the ERP management software, thereby get back to former manual mode, like that the informationalized construction of entire society and propelling are undoubtedly to a kind of great obstruction.Therefore the lifting existing enterprise how suited measures to local conditions is the data transmission performance of ERP management software system in use, and guarantees enterprise's existing business normal operation and need not take the extensive fund budget again, has become problem demanding prompt solution.
The ERP information handling system has for the data conversion treatment mode between cross-system at present: EAI (Enterprise Application Integration, enterprise's application integration instrument), OBA (Office Business Application, Office, the business application instrument), structural system (the Service-Oriented Architecture of object-oriented service, SOA, also be Service-Oriented Architecture Based) etc., all substantially realized the normal circulation of business datum between system.Take Pagination Display in reciprocal process, the lazy pattern loaded, reduce the consumption of PC hardware internal memory, also promoted to a certain extent the operational efficiency of whole system, operating the performance advantage of as far as possible bringing into play SQLServer for SQL database simultaneously.
But existing most of ERP system realizes that giving birth in batches single mechanism has the reason of following several aspects to cause the performance issue under big data quantity:
1, existing in batches living single operation is the cumulative of document one by one, does not integrate and re-assemblies the in enormous quantities disposable operation reached truly.
Can't guarantee the consistance of data write-back order when 2, the concurrent batch of many people is given birth to list, so likely cause affairs and wait for mutually for a long time or deadlock.
3, in existing in batches living single process, each document independently causes data access and write-back frequent alternately, and the I/O expense is large.
Therefore, need a kind of new technical scheme, can realize single job in enormous quantities truly, can avoid the generation of concurrent deadlock situation simultaneously.
Summary of the invention
The present invention just is being based on the problems referred to above, has proposed a kind of new technical scheme, can realize single job in enormous quantities truly, can avoid the generation of concurrent deadlock situation simultaneously.
In view of this, the present invention proposes a kind of batch data treating apparatus, be applied to the EPR system, comprising: acquiring unit, for obtaining a plurality of upstreams document; Buffer unit, for each the upstream document in the document of described a plurality of upstreams creates corresponding interim physics table, and arrive the data buffer storage in described each upstream document in the interim physics table of described correspondence, to obtain a plurality of interim physics tables; Updating block, upgraded described a plurality of interim physics tables according to default business need; The document generation unit is inserted into the data in the described a plurality of interim physics table after upgrading respectively in the traffic table of corresponding downstream document according to work flow, to generate a plurality of downstreams document in affairs.
In this technical scheme, the upstream document is integrated to disposable generation downstream document in batches, promoted like this performance of system, according to work flow, data are inserted to the traffic table of downstream document simultaneously in affairs, mutual wait in the time of can avoiding a plurality of client concurrent operations database by orderly batch data inserting like this between business datum causes the situation of deadlock, has also guaranteed the integrality of data simultaneously.
In technique scheme, preferably, also comprise: setting unit, the order that arranges for according to receiving, arrange the data constraint condition and/or the data template that obtain described a plurality of upstreams document; And described acquiring unit also for, obtain described a plurality of upstream data according to described data constraint condition, and/or obtain described a plurality of upstreams document according to described data template.
In this technical scheme, the user can select the bills data of oneself wanting according to individual demand, thus in the data that make the user obtain, can not comprise oneself use less than the data of redundancy, promoted user's experience.
In technique scheme, preferably, also comprise: verification unit, the data of the described a plurality of interim physics tables for the verification rule according to default after to described renewal are carried out verification.
In this technical scheme, verification unit is set the data in interim physics table are carried out to verification, can guarantee like this accuracy of data in the physics table.
In technique scheme, preferably, described document generation unit also for, in affairs, according to work flow, the data that meet described default verification rule in the described a plurality of interim physics table after upgrading are inserted into respectively in the traffic table of corresponding downstream document, to generate a plurality of downstreams document.
In this technical scheme, the temporary table that verification is passed through just allows to be inserted into the downstream document, guarantees like this accuracy and the validity of the downstream document of generation.
In technique scheme, preferably, also comprise: the data output unit, carry out output display to described a plurality of downstreams document.
In this technical scheme, can generate various forms of data results to the downstream document by the data output unit, thereby for the user with check.
According to a further aspect in the invention, also provide a kind of batch data disposal route, having comprised: step 202, obtain a plurality of upstreams document; Step 204, for each the upstream document in the document of described a plurality of upstreams creates corresponding interim physics table, and arrive the data buffer storage in described each upstream document in the interim physics table of described correspondence, to obtain a plurality of interim physics tables; Step 206, upgraded described a plurality of interim physics tables according to default business need; Step 208 is inserted into the data in the described a plurality of interim physics table after upgrading respectively in the traffic table of corresponding downstream document according to work flow, to generate a plurality of downstreams document in affairs.
In this technical scheme, the upstream document is integrated to disposable generation downstream document in batches, promoted like this performance of system, according to work flow, data are inserted to the traffic table of downstream document simultaneously in affairs, mutual wait in the time of can avoiding a plurality of client concurrent operations database by orderly batch data inserting like this between business datum causes the situation of deadlock, has also guaranteed the integrality of data simultaneously.
In technique scheme, preferably, also comprise: according to the order that arranges received, the data constraint condition and/or the data template that obtain described a plurality of upstreams document are set; And obtain described a plurality of upstream data according to described data constraint condition, and/or obtain described a plurality of upstreams document according to described data template.
In this technical scheme, the user can select the bills data of oneself wanting according to individual demand, thus in the data that make the user obtain, can not comprise oneself use less than the data of redundancy, promoted user's experience.
In technique scheme, preferably, also comprise: the data in the described a plurality of interim physics table according to default verification rule after to described renewal are carried out verification.
In this technical scheme, verification unit is set the data in interim physics table are carried out to verification, can guarantee like this accuracy of data in the physics table.
In technique scheme, preferably, also comprise: in affairs, according to work flow, the data that meet described default verification rule in the described a plurality of interim physics table after upgrading are inserted into respectively in the traffic table of corresponding downstream document, to generate a plurality of downstreams document.
In this technical scheme, the temporary table that verification is passed through just allows to be inserted into the downstream document, guarantees like this accuracy and the validity of the downstream document of generation.
In technique scheme, preferably, also comprise: described a plurality of downstreams document is carried out to output display.
In this technical scheme, can generate various forms of data results to the downstream document by the data output unit, thereby for the user with check.
By above technical scheme, can realize single job in enormous quantities truly, can avoid the generation of concurrent deadlock situation simultaneously.
The accompanying drawing explanation
Fig. 1 shows the block diagram of batch data treating apparatus according to an embodiment of the invention;
Fig. 2 shows the process flow diagram of batch data disposal route according to an embodiment of the invention;
Fig. 3 shows the particular flow sheet of batch data disposal route according to an embodiment of the invention;
Fig. 4 shows the screenshot capture of batch data processing procedure according to an embodiment of the invention;
Fig. 5 shows the position view of batch data treating apparatus according to an embodiment of the invention;
Fig. 6 shows the logical organization schematic diagram of batch data treating apparatus according to an embodiment of the invention;
Fig. 7 shows the configuration procedure schematic diagram of batch data treating apparatus according to an embodiment of the invention;
Fig. 8 shows according to an embodiment of the invention database entity-concern schematic diagram.
Embodiment
In order more clearly to understand above-mentioned purpose of the present invention, feature and advantage, below in conjunction with the drawings and specific embodiments, the present invention is further described in detail.It should be noted that, in the situation that do not conflict, the application's embodiment and the feature in embodiment can combine mutually.
A lot of details have been set forth in the following description so that fully understand the present invention; but; the present invention can also adopt other to be different from other modes described here and implement, and therefore, protection scope of the present invention is not subject to the restriction of following public specific embodiment.
Fig. 1 shows the block diagram of batch data treating apparatus according to an embodiment of the invention.
As shown in Figure 1, the batch data treating apparatus 100 according to an embodiment of the invention, comprising: acquiring unit 102, for obtaining a plurality of upstreams document; Buffer unit 104, for each the upstream document in the document of described a plurality of upstreams creates corresponding interim physics table, and arrive the data buffer storage in described each upstream document in the interim physics table of described correspondence, to obtain a plurality of interim physics tables; Updating block 106, upgraded described a plurality of interim physics tables according to default business need; Document generation unit 108 is inserted into the data in the described a plurality of interim physics table after upgrading respectively in the traffic table of corresponding downstream document according to work flow, to generate a plurality of downstreams document in affairs.
In this technical scheme, the upstream document is integrated to disposable generation downstream document in batches, promoted like this performance of system, according to work flow, data are inserted to the traffic table of downstream document simultaneously in affairs, mutual wait in the time of can avoiding a plurality of client concurrent operations database by orderly batch data inserting like this between business datum causes the situation of deadlock, has also guaranteed the integrality of data simultaneously.
In technique scheme, preferably, also comprise: setting unit 110, the order that arranges for according to receiving, arrange the data constraint condition and/or the data template that obtain described a plurality of upstreams document; And described acquiring unit also for, obtain described a plurality of upstream data according to described data constraint condition, and/or obtain described a plurality of upstreams document according to described data template.
In this technical scheme, the user can select the bills data of oneself wanting according to individual demand, thus in the data that make the user obtain, can not comprise oneself use less than the data of redundancy, promoted user's experience.
In technique scheme, preferably, also comprise: verification unit 112, the data of the described a plurality of interim physics tables for the verification rule according to default after to described renewal are carried out verification.
In this technical scheme, verification unit is set the data in interim physics table are carried out to verification, can guarantee like this accuracy of data in the physics table.
In technique scheme, preferably, described document generation unit 108 also for, in affairs, according to work flow, the data that meet described default verification rule in the described a plurality of interim physics table after upgrading are inserted into respectively in the traffic table of corresponding downstream document, to generate a plurality of downstreams document.
In this technical scheme, the temporary table that verification is passed through just allows to be inserted into the downstream document, guarantees like this accuracy and the validity of the downstream document of generation.
In technique scheme, preferably, also comprise: data output unit 114, carry out output display to described a plurality of downstreams document.
In this technical scheme, can generate various forms of data results to the downstream document by the data output unit, thereby for the user with check.
Fig. 2 shows the process flow diagram of batch data disposal route according to an embodiment of the invention.
As shown in Figure 2, batch data disposal route according to an embodiment of the invention comprises: step 202, obtain a plurality of upstreams document; Step 204, for each the upstream document in the document of described a plurality of upstreams creates corresponding interim physics table, and arrive the data buffer storage in described each upstream document in the interim physics table of described correspondence, to obtain a plurality of interim physics tables; Step 206, upgraded described a plurality of interim physics tables according to default business need; Step 208 is inserted into the data in the described a plurality of interim physics table after upgrading respectively in the traffic table of corresponding downstream document according to work flow, to generate a plurality of downstreams document in affairs.
In this technical scheme, the upstream document is integrated to disposable generation downstream document in batches, promoted like this performance of system, according to work flow, data are inserted to the traffic table of downstream document simultaneously in affairs, mutual wait in the time of can avoiding a plurality of client concurrent operations database by orderly batch data inserting like this between business datum causes the situation of deadlock, has also guaranteed the integrality of data simultaneously.
In technique scheme, preferably, also comprise: according to the order that arranges received, the data constraint condition and/or the data template that obtain described a plurality of upstreams document are set; And obtain described a plurality of upstream data according to described data constraint condition, and/or obtain described a plurality of upstreams document according to described data template.
In this technical scheme, the user can select the bills data of oneself wanting according to individual demand, thus in the data that make the user obtain, can not comprise oneself use less than the data of redundancy, promoted user's experience.
In technique scheme, preferably, also comprise: the data in the described a plurality of interim physics table according to default verification rule after to described renewal are carried out verification.
In this technical scheme, verification unit is set the data in interim physics table are carried out to verification, can guarantee like this accuracy of data in the physics table.
In technique scheme, preferably, also comprise: in affairs, according to work flow, the data that meet described default verification rule in the described a plurality of interim physics table after upgrading are inserted into respectively in the traffic table of corresponding downstream document, to generate a plurality of downstreams document.
In this technical scheme, the temporary table that verification is passed through just allows to be inserted into the downstream document, guarantees like this accuracy and the validity of the downstream document of generation.
In technique scheme, preferably, also comprise: described a plurality of downstreams document is carried out to output display.
In this technical scheme, can generate various forms of data results to the downstream document by the data output unit, thereby for the user with check.
Fig. 3 shows the particular flow sheet of batch data disposal route according to an embodiment of the invention.
As shown in Figure 3, the idiographic flow of batch data disposal route is as follows according to an embodiment of the invention:
Step 302, obtain a plurality of upstreams document according to data constraint condition and/or the data template of user's input.Wherein, the mode of obtaining upstream data can be: 1,, according to concrete single succinct User Interface efficiently that requires to design targetedly of giving birth to, then in conjunction with the data constraint condition of user's input, from upstream bills data set, obtain living forms data; 2, pre-set data template, directly according to data template, receive the data of other system input.Like this, the data of acquisition are all the data that the user needs.
Step 304, organize and create interim physical data buffer memory in batches.Carry out the ephemeral data table of autotelic inquiry generation various uses by existing condition, the mass simultaneous that creates temporary table inserts the record that meets service logic.
Concrete caching process process:
1, data are prepared: no longer repeat to search or obtain data while being intended to follow-up use.
Variable-definition; Importing parameter into processes; Obtain the temporary table title; With the data acquisition of traffic aided (as: long-range number, document template, create linked data object, initialization), once these global informations obtain whole system run duration obtaining of weight more not.Reduce the frequent mutual of data, optimize the business realizing logic.
2, insert in batches the row (table body) that meets service logic.
Establishment table body temporary table is used SELECT INTO to insert in batches the row that meets service logic simultaneously.This process does not need the data transactions of opening an account, so can not impact original system.Simultaneously autotelic efficiently and accurately filters the business datum disposable loading of upstream document to middle temporary table, the degree of coupling between disengaging and original system data.Need to consider the situation of supporting SQL statement to repeat while creating temporary table, first judge whether temporary table exists, if existed again created again after deleting; Also need in addition to increase by one and be used for follow-up index from the row that increase type (Identity), promote the performance of temporary table; The spelling of conditional statement need to be treated to the situation of sky (IsNull).
Step 306, batch updating legitimacy detect the data of buffer memory.On the various temporary tables basis produced at the inquiry gained, the legitimacy Data Update of organizing the relation between each temporary table and showing individually column according to this secondary single business need of reality, and optimize temporary table performance (as building business index etc.).
Concrete renewal process: 1, the disposable AutoID that obtains all table bodies records, and batch updating it; 2, other and the corresponding thresholding of business of batch updating table body temporary table, as: coefficient of conversion, planning price 3, carry out index and insert in batches gauge outfit temporary table data participating in a minute single field in table body temporary table; 4, obtain in batches interim gauge outfit document master meter ID and document number and upgrade it; 5, the ID of the interim sublist of batch updating; 6, all fields that participate in grouping calculating standing crop in table body temporary table are carried out to index and inserted in batches standing crop temporary table data.Simultaneity factor is also supported dynamic self-defining data calibration device, dynamically arranges business rule by friendly interactive interfacing and changes check logic frequently.Amended verification true form directly deposits corresponding the arranging in table of native system in the mode of VBScript, and system directly reads and dissection process when operation.Can do on the original basis the corresponding adjustment of revising after entering system next time simultaneously.
Step 308, judge whether by checking, enters step 310 while being judgment result is that.
Step 310 is inserted into the downstream document set that traffic table generates to the batch data of buffer memory by work flow in affairs, or generates various forms of data results by data output device and represented.Unified according to after the sequence of business major key from temporary table each temporary table data by the mode of carrying out in batches SQL, the SQL statement that disposable execution is organized by service logic in advance, the temporary table data are generated in corresponding traffic table to the situation of the mutual wait deadlock in the time of can avoiding a plurality of client concurrent operations database by orderly batch data inserting between business datum.And only when the disposable execution SQL statement of batch, carry out the control of db transaction, guaranteed the integrality of the insertion of business datum.
Fig. 4 shows the screenshot capture of batch data processing procedure according to an embodiment of the invention.
As shown in Figure 4, the user can select to produce order, target document template, outbound classification, warehouse etc. from interface, after the user selects, as long as click, gives birth to list, just can generate corresponding document.The processing procedure of concrete batch data treating apparatus is as follows:
1), data are prepared: no longer repeat to search or obtain data while being intended to follow-up use.
Variable-definition; Importing parameter into processes; Obtain the temporary table title; With the data acquisition of traffic aided (as: long-range number, document template, create linked data object, initialization), once these global informations obtain whole system run duration obtaining of weight more not.Reduce the frequent mutual of data, optimize the business realizing logic.
2), insert in batches the row (table body) that meets service logic.
Establishment table body temporary table is used SELECT INTO to insert in batches the row that meets service logic simultaneously.This process does not need the data transactions of opening an account, so can not impact original system.Simultaneously autotelic efficiently and accurately filters the business datum disposable loading of upstream document to middle temporary table, the degree of coupling between disengaging and original system data.Need to consider the situation of supporting SQL statement to repeat while creating temporary table, first judge whether temporary table exists, if existed again created again after deleting; Also need in addition to increase by one and be used for follow-up index from the row that increase type (Identity), promote the performance of temporary table; The spelling of conditional statement need to be treated to the situation of sky (IsNull).
3), the disposable AutoID that obtains all table bodies record, and batch updating.
Obtain in batches Autoid by Data View and storing process, given full play to the Optimization Mechanism (Optimizing Queries scheme) of database, need to perform fault-tolerant processing simultaneously.Wherein, AutoID obtain and more fresh code be:
If?Not?GetNewIDbak(oLogin,"rd",iCount,lMainId,lSubId)Then
VBA.Err.Raise vbObjectError+10002,, " application ID a failure! " End If
sSql="update"&strBTblName&"set?autoid=tmpid+"&lSubId&vbCrLf
oConn.Execute?sSql
4), criticize and upgrade business datum.
The keyword segment value of batch updating table body temporary table other and traffic aided, as: coefficient of conversion, planning price etc.Resolve original business check logic when upgrading, directly correct data one step is upgraded and put in place, follow-uply omitted relevant business checking treatment.
5), for table body temporary table, create index and insert the gauge outfit business datum.
Carry out index and insert in batches gauge outfit temporary table data participating in a minute single field in table body temporary table, previous create certainly increase the major key of type-word section as temporary table.This process is that the Business Information of insertion table body temporary table after before having screened by rigid condition is inserted in the gauge outfit temporary table in batches according to certain minute set condition, simultaneously also to consider whether the gauge outfit temporary table exists, if existed after deletion again create and the grouping of insertion table body after Business Information.The field contents that other gauge outfits except grouping (Group) must be wanted is got the maximal value (Max) of table volume data or is directly replaced with constant, but the attributes such as field type that must be clearly concrete and length.
6), upgrade ID and document number.
Obtain in batches interim gauge outfit document master meter ID, document model ID and document number by calling existing storing process, then disposablely be updated to corresponding temporary table.Annotate: the document now got number is all genuine, even the single failure of generate subsequent also can take the document section.
7), the ID of the associated master meter of the interim sublist of batch updating.
Divide single associated main and sub temporary table of foundation by previous, and upgrade external key (master meter) the ID value of sub-temporary table with the main temporary table ID generated.So just truly set up the incidence relation between main and sub temporary table.For follow-up peek and performance optimization, get ready like this.
8), index batch renewal standing crop.
The all fields that participate in grouping calculating standing crop in table body temporary table are carried out to index and inserted in batches standing crop temporary table data.At first the dimension of adding up according to existing scale creates index for the gauge outfit temporary table, then creates the standing crop temporary table.When creating the standing crop temporary table, also need to judge whether to deposit, if existed again created after deleting, the field that need to add up to for those gathers in rear insertion standing crop temporary table by the Sum function.Finally from each temporary table, off-the-shelf business datum is inserted to the entity traffic table in batches, now carry out the integrality that data are guaranteed in transaction management.
9), order is carried out SQL statement in batches.
SQL by business datum sequential organization batch Insert service table; Open affairs; Carry out SQL string in batches; Submit affairs (the rollback affairs if make mistakes) to.Sequential organising data is to realize by Order by mode, the intersection deadlock in the time of effectively avoiding the concurrent execution bulk data transfer of many people like this.Wherein, SQL order run time version is:
oConn.BeginTrans
oConn.Execute?sSql1+sSql2+sSql3
oConn.CommitTrans
ERR:oConn.RollbackTrans
Batch data treating apparatus of the present invention can realize by the mode of plug-in unit, and its concrete placement location as shown in Figure 5.
Below the noun occurred in Fig. 5 is made an explanation:
Plug-in component operation stream, a plug-in component operation stream refers to the flow process that independently comprises plug-in component operation at.A plug-in component operation stream can have a plurality of extension points.
Extension point (Extension Point), for placing the interface of insert receptacle, an extension point can only be placed a container.
Insert receptacle (Plug Container), for placing plug-in unit, an insert receptacle can be put a plurality of plug-in units.Insert receptacle is responsible for moving plug-in unit, processes plug-in unit abnormal, and the interface of adaptive extension point.
Function declaration:
Be responsible for carrying out in order plug-in unit; Be responsible for recording the abnormal of plug-in unit; Interruption for the treatment of plug-in unit; Interface for adaptive extension point.
Plug-in unit is carried out engine, and plug-in unit is carried out engine and is responsible for carrying out the work of calling of any one plug-in unit in plugin library, so need to carry out interface adaptation, execution plug-in unit, the recording exceptional daily record of plug-in unit.
Interface object is the interface of container and extension point.The Systems Programmer provides the parameter of its disclosed interface of wanting when the definition extension point to container by interface object.Interface object can be supported the description of polytype interface.Can be simple types, can be also object type.
Plug-in unit (Plugin), refer to the independently process that meets certain interface norm.Plug-in unit comprises two types, and a kind of is to monopolize the type plug-in unit, and this kind of plug-in unit do not allow other plug-in units and his a shared container.One type is shared.Shared plug-in unit allows other plug-in units and its shared container.
Plug-in unit is shown soon, is the function for the treatment of the automatic acquisition plug-in unit, and when system is enabled, system will be set up plug-in unit and show soon, and the effect of the fast table of plug-in unit is the header that automatically obtains the plug-in unit that this container can move when container starts.The pattern of its use is single routine pattern.
Plug set (Pluginset), refer to the set of plug-in unit.Mode for the assembly of realizing the miscellaneous service logic by dynamic-configuration fits together and realizes corresponding function.
As shown in Figure 5, the batch data treating apparatus can be used as a plug-in unit (as plug-in unit 1 ... plug-in unit N) be placed in the plug-in unit set, certainly, can also place other plug-in unit in the plug-in unit set, while using this plug-in unit, access plug-in unit by insertion point and carry out server, plug-in server finds this plug-in unit from insert receptacle, and is applied.
Fig. 6 shows the logical organization schematic diagram of batch data treating apparatus according to an embodiment of the invention;
Fig. 6 shows the logical organization of each object of batch data treating apparatus and the interbehavior in system, and the relation between class and interface.Concrete, between class, call relation is: the method (GetBizAPI) by calling the interface adaptation services object (APIAdapterSrv) created read configuration file (BizAPIRepository.Config.xml) and obtain business interface service object (STAPI.STInOutVoucherAPI) at first, obtain suction parameter object corresponding to business interface service object (BizAPIAdapter.PrimitiveParameter and ADORecordSetDOMParameter) simultaneously, then give the business interface service object's obtain the value corresponding according to the actual service logic assignment to number attribute, finally by the method (ExEcute) of calling this entity business object, carry out corresponding interface method and realize current operation.
Fig. 7 shows the configuration procedure schematic diagram of batch data treating apparatus according to an embodiment of the invention.
As shown in Figure 7, this device is by calling the method GetBizAPI of APIAdapterService class, then call and be specifically designed to the class APIRepositoryConfig that resolves this configuration file as suction parameter with the configuration information of configuration file BizAPIRepository.Config.xml, finally return results class IBizAPI.Information memory cell class APIRepUnit and configuration subclass APIConfig have been used while resolving the dynamic-configuration fileinfo, by the subclass mapping method, BuildMap creates respectively mapping class APIAdapterMap or APIParameterHandlerMap, and the method GetAPI that finally calls subclass APIConfig realizes the instruction of UI.
Fig. 8 shows according to an embodiment of the invention database entity-concern schematic diagram;
Fig. 8 has described the relation between primary entity in the system, its core is four tables that label is 801,802,803,804, be respectively to deposit and produce order database Basic Information Table (mom_order), production order sublist (mom_orderdetail), production order component materials tables (mom_moallocate), condition filter database table (Rpt_FltDEF_Base), and other two tables enter and the demonstrating data application service for system.
Below, by technical scheme of the present invention and correlation technique contrast, in conjunction with the batch data processing procedure, describe advantage of the present invention in detail:
1, based on original-ERP system, be that plug-in unit is set up corresponding function menu specially.
Analysis-ERP system original function menu setting scheme, make corresponding function menu data script for this plug-in unit preset, realizes the inlet function menu of plug-in unit, follow-up normal call.
2, the succinct data filtering interactive interfacing configuration of current system and cross-system data input device.
At first to explain a concept: succinct interface setting targetedly, refer to according to the actual list of giving birth in batches of user and need to arrange targetedly and the layout interactive interface, remove in those standardized product functions as considering the various fair remaining elements that may arrange, realization succinct data efficiently displaying and typing.
The interactive interfacing configuration refers to that creating one based on exploitation on standard ERP product has more data interaction window targetedly targetedly, and this interface must provide the interface that meets the standardized product requirement simultaneously.Can be absorbed in by this UI interface the consistance that front and back were inputted and guaranteed to key message, high efficiency filter is collected current system for giving birth to single upstream data.For example in ERP system for industrial enterprise, the proportioning outbound is a kind of special material outbound, the user is as produced or assemble a certain father's item product, system can be deployed into the subitem material according to Bill of Material (BOM) (BOM) by it, and calculates and produce or assemble the subitem material quantity that father's item product needed is received.The material outbound is to get material from warehouse, for the production of or external processing, for producing the material of punching at the finished product warehouse-in or committee is outer while putting in storage that system is carried out the material outbound automatically.In batches in proportioning outbound process, the user need to pay close attention to is that affiliated warehouse, outbound classification and those sources satisfied condition of the material delivered note that will generate produces order and get final product, to having the stock of Bill of Material (BOM) (BOM), the proportioning outbound can be strengthened speed and the accuracy of neck material outbound.
And, in existing related system, its proportioning is given birth to single project of displaying and participation typing that may need can be many, and also there is certain efficiency hidden danger in this interactive interface itself.
3, the present invention also supports the data input device of cross-system, exchanges data template and corresponding data by appointment arrange the data-switching input of directly carrying out large handling capacity between system, according to configured in advance template and corresponding relation directly from other system, extract the livings single operation that key business data is participated in current system directly, thereby efficient the collection gathered the other system business datum and participated in giving birth to list to current system.
And at existing ERP system processing mode (as: EAI (Enterprise Application Integration, enterprise's application integration instrument), OBA (Office Business Application, Office business application instrument)): the A system derives instrument according to data of the extra exploitation of the data layout of B system requirements, and own intrasystem business datum is derived.The data importing that the import tool of B system recycling oneself is derived the A system is in corresponding traffic table, then the B system generates according to the existing business logic upstream data tissue imported the downstream data of B system again, and the complicated cost of development of operating process is high and performance cost is large.
4, organize and create in batches interim physical data buffer memory.
Carry out the ephemeral data table of autotelic inquiry generation various uses by existing condition, the mass simultaneous that creates temporary table inserts the record that meets service logic.Concrete caching process process: 1. data are prepared: no longer repeat to search or obtain data (variable-definition while being intended to follow-up use; Importing parameter into processes; Obtain the temporary table title; Data acquisition (as: long-range number/document template/establishment linked data object/initialization) with traffic aided; 2. establishment table body temporary table is used SELECT INTO to insert in batches the row that meets service logic simultaneously.
And, in existing ERP system, it is single not by interim physical data table that upstream and downstream data service data are given birth to, but between each traffic table by PC(Personal Computer-personal computer) the interim processing of internal memory after directly generation.Like this load of each traffic table is striven to the obvious rising that taken that sharply strengthens the while PC memory, directly affect the operation of whole system.
5, batch updating validity checking are data cached.
On the various temporary tables basis produced at the inquiry gained, the legitimacy Data Update of organizing the relation between each temporary table and showing individually column according to this secondary single business need of reality, and optimize temporary table performance (as building business index etc.).Concrete renewal process: 1. the disposable AutoID that obtains all table bodies records, and batch updating it; 2. other and the corresponding thresholding of business of batch updating table body temporary table, as: coefficient of conversion, planning price 3. carry out index and insert in batches gauge outfit temporary table data participating in a minute single field in table body temporary table; 4. obtain in batches interim gauge outfit document master meter ID and document number and upgrade it; 5. the ID of the interim sublist of batch updating; 6. the ID of the interim sublist of batch updating; 7. all fields that participate in grouping calculating standing crop in table body temporary table are carried out to index and inserted in batches standing crop temporary table data.
The present invention simultaneously also supports dynamic self-defining data calibration device, dynamically arranges business rule by friendly interactive interfacing and changes check logic frequently.Amended verification true form directly deposits corresponding the arranging in table of native system in the mode of VBScript, and system directly reads and dissection process when operation.Can do on the original basis the corresponding adjustment of revising after entering system next time simultaneously.
And existing ERP system generally all is cured to business verification rule in system code, do not support to revise by the dynamic secondary development of interface.Be not easy to so at present this ERP application complicated and changeable.
6, in affairs, data cached batch is inserted into to traffic table by work flow.
By the mode of carrying out in batches SQL, each temporary table data unified SQL statement of in advance organizing by service logic according to the rear disposable execution of business major key sequence from temporary table is generated to the temporary table data in corresponding traffic table to the situation of the mutual wait deadlock in the time of can avoiding a plurality of client concurrent operations database by orderly batch data inserting between business datum.And the control of only when disposable execution SQL statement in batches, carrying out db transaction guarantees the integrality of the insertion of business datum, also only some tables of data resource of the least possible time simultaneously.
And original ERP system is not generally considered that multiple spot is parallel and given birth to the sequencing problem that data are transmitted when single, may cause mutual wait in the time of concurrent operations will being caused like this, thereby single performance is given birth in impact greatly.
To sum up, technical scheme of the present invention has a clear superiority in the outbound of processing big data quantity production material with respect to original system.At first more meet the practical operation custom of enterprise application from applying mutual aspect, design record incoming interface targetedly, it is convenient and swift that being recorded into of cancelling that the user will not pay close attention to reduced operating process.Also embodied greatly the advantage of robotization from data transmission procedure in addition, the user sets correlation parameter and Organization of Data rule in advance, and performance and the accuracy that more promotes the business datum circulation without human intervention all processed in follow-up whole material outbound automatically.
More than be described with reference to the accompanying drawings technical scheme of the present invention, by technical scheme of the present invention, can have realized single job in enormous quantities truly, can avoid the generation of concurrent deadlock situation simultaneously.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for a person skilled in the art, the present invention can have various modifications and variations.Within the spirit and principles in the present invention all, any modification of doing, be equal to replacement, improvement etc., within all should being included in protection scope of the present invention.

Claims (10)

1. a batch data treating apparatus, is characterized in that, comprising:
Acquiring unit, for obtaining a plurality of upstreams document;
Buffer unit, for each the upstream document in the document of described a plurality of upstreams creates corresponding interim physics table, and arrive the data buffer storage in described each upstream document in the interim physics table of described correspondence, to obtain a plurality of interim physics tables;
Updating block, upgraded described a plurality of interim physics tables according to default business need;
The document generation unit is inserted into the data in the described a plurality of interim physics table after upgrading respectively in the traffic table of corresponding downstream document according to work flow, to generate a plurality of downstreams document in affairs.
2. batch data treating apparatus according to claim 1, is characterized in that, also comprises:
Setting unit, the order that arranges for according to receiving, arrange the data constraint condition and/or the data template that obtain described a plurality of upstreams document;
And described acquiring unit also for,
Obtain described a plurality of upstream data according to described data constraint condition, and/or obtain described a plurality of upstreams document according to described data template.
3. batch data treating apparatus according to claim 1, is characterized in that, also comprises:
Verification unit, the data of the described a plurality of interim physics tables for the verification rule according to default after to described renewal are carried out verification.
4. batch data treating apparatus according to claim 3, is characterized in that, described document generation unit also for,
In affairs, according to work flow, the data that meet described default verification rule in the described a plurality of interim physics table after upgrading are inserted into respectively in the traffic table of corresponding downstream document, to generate a plurality of downstreams document.
5. according to the described batch data treating apparatus of any one in claim 1 to 4, it is characterized in that, also comprise:
The data output unit, carry out output display to described a plurality of downstreams document.
6. a batch data disposal route, is characterized in that, comprising:
Step 202, obtain a plurality of upstreams document;
Step 204, for each the upstream document in the document of described a plurality of upstreams creates corresponding interim physics table, and arrive the data buffer storage in described each upstream document in the interim physics table of described correspondence, to obtain a plurality of interim physics tables;
Step 206, upgraded described a plurality of interim physics tables according to default business need;
Step 208 is inserted into the data in the described a plurality of interim physics table after upgrading respectively in the traffic table of corresponding downstream document according to work flow, to generate a plurality of downstreams document in affairs.
7. batch data disposal route according to claim 6, is characterized in that, also comprises:
According to the order that arranges received, the data constraint condition and/or the data template that obtain described a plurality of upstreams document are set;
And obtain described a plurality of upstream data according to described data constraint condition, and/or obtain described a plurality of upstreams document according to described data template.
8. batch data disposal route according to claim 6, is characterized in that, also comprises:
Data in described a plurality of interim physics table according to default verification rule after to described renewal are carried out verification.
9. batch data disposal route according to claim 8, is characterized in that, also comprises:
In affairs, according to work flow, the data that meet described default verification rule in the described a plurality of interim physics table after upgrading are inserted into respectively in the traffic table of corresponding downstream document, to generate a plurality of downstreams document.
10. according to the described batch data disposal route of any one in claim 6 to 9, it is characterized in that, also comprise:
Described a plurality of downstreams document is carried out to output display.
CN201310397318.3A 2013-09-04 2013-09-04 Batch data processing device and batch data processing method Pending CN103489061A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310397318.3A CN103489061A (en) 2013-09-04 2013-09-04 Batch data processing device and batch data processing method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310397318.3A CN103489061A (en) 2013-09-04 2013-09-04 Batch data processing device and batch data processing method

Publications (1)

Publication Number Publication Date
CN103489061A true CN103489061A (en) 2014-01-01

Family

ID=49829264

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310397318.3A Pending CN103489061A (en) 2013-09-04 2013-09-04 Batch data processing device and batch data processing method

Country Status (1)

Country Link
CN (1) CN103489061A (en)

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103888378A (en) * 2014-04-09 2014-06-25 北京京东尚科信息技术有限公司 Data exchange system and method based on cache mechanism
CN105426419A (en) * 2015-11-03 2016-03-23 用友网络科技股份有限公司 System and method for data promotion among heterogeneous systems
CN107040567A (en) * 2016-09-27 2017-08-11 阿里巴巴集团控股有限公司 The management-control method and device of pre-allocation of resources amount
CN107608994A (en) * 2016-07-12 2018-01-19 深圳联友科技有限公司 A kind of method and system of batch processing SQL statement insertion database
CN107818114A (en) * 2016-09-14 2018-03-20 中国移动通信有限公司研究院 A kind of data processing method, device and database
CN107818170A (en) * 2017-11-14 2018-03-20 北京思特奇信息技术股份有限公司 A kind of method and system for accessing data-base recording
CN108399246A (en) * 2018-03-01 2018-08-14 金蝶软件(中国)有限公司 A kind of localization method and relevant apparatus of target data
WO2018166145A1 (en) * 2017-03-14 2018-09-20 平安科技(深圳)有限公司 Method and device for batch offering of repayment data
CN109284331A (en) * 2018-08-16 2019-01-29 中国平安人寿保险股份有限公司 Accreditation information acquisition method, terminal device and medium based on business datum resource
CN110019259A (en) * 2017-09-26 2019-07-16 亿阳信通股份有限公司 Data-updating method, device and the storage medium of distributed index service-Engine
CN110515954A (en) * 2019-08-29 2019-11-29 北京博睿宏远数据科技股份有限公司 A data processing method, device, device and storage medium
CN111209273A (en) * 2019-12-29 2020-05-29 航天信息股份有限公司企业服务分公司 Configurable ERP system data unloading method and device
CN111476661A (en) * 2020-05-04 2020-07-31 武汉众邦银行股份有限公司 General ledger system batch processing method for cyclic check entry
CN111752693A (en) * 2020-06-24 2020-10-09 中国建设银行股份有限公司 A transaction processing method, device, device and storage medium
CN114201297A (en) * 2021-12-14 2022-03-18 中国建设银行股份有限公司 Data processing method and device, electronic equipment and storage medium
CN114489935A (en) * 2021-12-29 2022-05-13 北京致远互联软件股份有限公司 A Visual Simulation System for Complex Data Processing Between Documents
CN115834681A (en) * 2022-12-06 2023-03-21 用友网络科技股份有限公司 Business document pushing method and device, storage medium and electronic equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101661508A (en) * 2009-09-29 2010-03-03 金蝶软件(中国)有限公司 Method for generating memu for multi-technology platform and device thereof
US20110113207A1 (en) * 2009-11-12 2011-05-12 Iron Mountain, Incorporated Data processing system with application-controlled allocation of file storage space
CN102982414A (en) * 2012-11-22 2013-03-20 用友软件股份有限公司 Data generating device and method
US20130073315A1 (en) * 2010-04-19 2013-03-21 Philip Cardamone Data processing and transferring method relating to the placement of surgical implants
CN103246651A (en) * 2012-02-02 2013-08-14 金蝶软件(中国)有限公司 Method and system for data inputting

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101661508A (en) * 2009-09-29 2010-03-03 金蝶软件(中国)有限公司 Method for generating memu for multi-technology platform and device thereof
US20110113207A1 (en) * 2009-11-12 2011-05-12 Iron Mountain, Incorporated Data processing system with application-controlled allocation of file storage space
US20130073315A1 (en) * 2010-04-19 2013-03-21 Philip Cardamone Data processing and transferring method relating to the placement of surgical implants
CN103246651A (en) * 2012-02-02 2013-08-14 金蝶软件(中国)有限公司 Method and system for data inputting
CN102982414A (en) * 2012-11-22 2013-03-20 用友软件股份有限公司 Data generating device and method

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103888378B (en) * 2014-04-09 2017-08-25 北京京东尚科信息技术有限公司 A kind of data exchange system and method based on caching mechanism
CN103888378A (en) * 2014-04-09 2014-06-25 北京京东尚科信息技术有限公司 Data exchange system and method based on cache mechanism
CN105426419B (en) * 2015-11-03 2018-11-20 用友网络科技股份有限公司 Data promotion system and method between heterogeneous system
CN105426419A (en) * 2015-11-03 2016-03-23 用友网络科技股份有限公司 System and method for data promotion among heterogeneous systems
CN107608994A (en) * 2016-07-12 2018-01-19 深圳联友科技有限公司 A kind of method and system of batch processing SQL statement insertion database
CN107818114A (en) * 2016-09-14 2018-03-20 中国移动通信有限公司研究院 A kind of data processing method, device and database
CN107040567A (en) * 2016-09-27 2017-08-11 阿里巴巴集团控股有限公司 The management-control method and device of pre-allocation of resources amount
WO2018166145A1 (en) * 2017-03-14 2018-09-20 平安科技(深圳)有限公司 Method and device for batch offering of repayment data
CN110019259A (en) * 2017-09-26 2019-07-16 亿阳信通股份有限公司 Data-updating method, device and the storage medium of distributed index service-Engine
CN110019259B (en) * 2017-09-26 2023-09-22 亿阳信通股份有限公司 Data updating method, device and storage medium of distributed index service engine
CN107818170A (en) * 2017-11-14 2018-03-20 北京思特奇信息技术股份有限公司 A kind of method and system for accessing data-base recording
CN107818170B (en) * 2017-11-14 2021-09-28 北京思特奇信息技术股份有限公司 Method and system for accessing database records
CN108399246A (en) * 2018-03-01 2018-08-14 金蝶软件(中国)有限公司 A kind of localization method and relevant apparatus of target data
CN109284331A (en) * 2018-08-16 2019-01-29 中国平安人寿保险股份有限公司 Accreditation information acquisition method, terminal device and medium based on business datum resource
CN109284331B (en) * 2018-08-16 2024-04-02 中国平安人寿保险股份有限公司 Certificate making information acquisition method based on service data resources, terminal equipment and medium
CN110515954A (en) * 2019-08-29 2019-11-29 北京博睿宏远数据科技股份有限公司 A data processing method, device, device and storage medium
CN110515954B (en) * 2019-08-29 2023-01-31 北京博睿宏远数据科技股份有限公司 Data processing method, device, equipment and storage medium
CN111209273B (en) * 2019-12-29 2023-11-14 航天信息股份有限公司企业服务分公司 Configurable ERP system data unloading method and device
CN111209273A (en) * 2019-12-29 2020-05-29 航天信息股份有限公司企业服务分公司 Configurable ERP system data unloading method and device
CN111476661B (en) * 2020-05-04 2023-07-11 武汉众邦银行股份有限公司 Batch processing method of general ledger system for cyclic check-in account
CN111476661A (en) * 2020-05-04 2020-07-31 武汉众邦银行股份有限公司 General ledger system batch processing method for cyclic check entry
CN111752693A (en) * 2020-06-24 2020-10-09 中国建设银行股份有限公司 A transaction processing method, device, device and storage medium
CN114201297A (en) * 2021-12-14 2022-03-18 中国建设银行股份有限公司 Data processing method and device, electronic equipment and storage medium
CN114489935A (en) * 2021-12-29 2022-05-13 北京致远互联软件股份有限公司 A Visual Simulation System for Complex Data Processing Between Documents
CN115834681A (en) * 2022-12-06 2023-03-21 用友网络科技股份有限公司 Business document pushing method and device, storage medium and electronic equipment

Similar Documents

Publication Publication Date Title
CN103489061A (en) Batch data processing device and batch data processing method
US20210117437A1 (en) Data model transformation
CN100437587C (en) Service processing method and device based on data base
CN106796578B (en) Autoknowledge system and method and memory
US7574379B2 (en) Method and system of using artifacts to identify elements of a component business model
CN107506442B (en) Modeling method and device of model
CN110543303B (en) Visual service platform
CN102033743A (en) System and method for expanding model driving development capability by applying plug-in
CN114461200A (en) A low-code development application and method based on cloud SaaS platform
CN101359290A (en) Method for customizing service document based on metadata and apparatus
CN101084494A (en) User interfaces for data integration systems
CN110249356B (en) Sharing method and system for user-defined ERP function
US20190147029A1 (en) Method and system for generating conversational user interface
US20120227028A1 (en) Graphical programming object population user interface autogeneration
CN114281797A (en) Method for quickly creating basic level data aggregation warehouse based on agile low-code platform
US20250258838A1 (en) Systems and methods for collecting and distributing digital experience information
CN102567590B (en) CAD (computer aided design) server device and method
CN103107919A (en) Method and system for network resource modeling
CN110209744A (en) Relevant database and its operating method and device based on alliance's chain
CN117667960A (en) Business table update method, device, equipment and storage medium
US11977473B2 (en) Providing a pseudo language for manipulating complex variables of an orchestration flow
Burgueno Concurrent and Distributed Model Transformations based on Linda.
CN109634606A (en) A kind of method and device of defined function menu
CN115543428A (en) Simulated data generation method and device based on strategy template
JP2012164177A (en) Program automatic generation system

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C41 Transfer of patent application or patent right or utility model
TA01 Transfer of patent application right

Effective date of registration: 20160127

Address after: 100094 Beijing City, North Road, Haidian District, No. 68, building 2, floor 2

Applicant after: You Pu Information Technology Co., Ltd of UFSOFT

Address before: 100094 Beijing city Haidian District North Road No. 68, UFIDA Software Park

Applicant before: UFIDA Software Co., Ltd.

RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20140101