CN104965687B - Big data processing method and processing device based on instruction set generation - Google Patents
Big data processing method and processing device based on instruction set generation Download PDFInfo
- Publication number
- CN104965687B CN104965687B CN201510303600.XA CN201510303600A CN104965687B CN 104965687 B CN104965687 B CN 104965687B CN 201510303600 A CN201510303600 A CN 201510303600A CN 104965687 B CN104965687 B CN 104965687B
- Authority
- CN
- China
- Prior art keywords
- instruction
- function
- big data
- time function
- intermediate representation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 61
- 238000003672 processing method Methods 0.000 title claims abstract description 11
- 238000000034 method Methods 0.000 claims abstract description 29
- 230000006870 function Effects 0.000 claims description 106
- 230000008569 process Effects 0.000 claims description 12
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 description 5
- 230000003068 static effect Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 229910052754 neon Inorganic materials 0.000 description 1
- GKAOGPIIYCISHV-UHFFFAOYSA-N neon atom Chemical compound [Ne] GKAOGPIIYCISHV-UHFFFAOYSA-N 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Devices For Executing Special Programs (AREA)
Abstract
Description
Claims (8)
- A kind of 1. big data processing method based on instruction set generation, it is characterised in that including:Generate Just-In-Time function;Handled using the Just-In-Time function pair big data;Wherein, the generation Just-In-Time function includes:Instruction database is established, the instruction database internal memory contains a variety of intermediate representation instructions;Determine that corresponding function performs flow according to user's request;The intermediate representation instruction for performing flow path match in the flow selection instruction database with the function is performed according to the function, Generate intermediate representation instruction set;The intermediate representation instruction set is generated into the Just-In-Time function;It is described to determine that corresponding function performs flow, including:Expression tree is generated according to user's request;The expression tree is traveled through, obtains the attribute information of the expression tree;According to the expression tree and its attribute information, determine that the function performs flow.
- 2. big data processing method according to claim 1, it is characterised in that the intermediate representation instruction is deposited after encapsulating Storage is in the instruction database.
- 3. big data processing method according to claim 1, it is characterised in that the Just-In-Time function is to use bottom The generation of virtual machine Just-In-Time method.
- 4. big data processing method according to claim 1, it is characterised in that utilize the Just-In-Time function described Before handling big data, in addition to automatic vectorization processing, specifically include:Analyze the concurrency of the user's request;Corresponding automatic vectorization process instruction is added in the Just-In-Time function according to the analysis result of the concurrency.
- A kind of 5. big data processing unit based on instruction set generation, it is characterised in that including:Function generation module, for generating Just-In-Time function;Processing module, for being handled using the Just-In-Time function pair big data;Wherein, the Just-In-Time function generation module includes:Instruction database setting up submodule, for establishing instruction database, the instruction database internal memory contains a variety of intermediate representation instructions;Flow determination sub-module, for determining that corresponding function performs flow according to user's request;Instruction set generates submodule, and stream is performed with the function for being performed according to the function in the flow selection instruction database The intermediate representation instruction of journey matching, generates intermediate representation instruction set;Function generates submodule, for the intermediate representation instruction set to be generated into the Just-In-Time function;The flow determination sub-module includes:Generation unit is set, for generating expression tree according to user's request;Attribute information acquiring unit, for traveling through the expression tree, obtain the attribute information of the expression tree;Function performs flow determining unit, for according to the expression tree and its attribute information, determining that function performs stream Journey.
- 6. big data processing unit according to claim 5, it is characterised in that the intermediate representation instruction is deposited after encapsulating Storage is in the instruction database.
- 7. big data processing unit according to claim 5, it is characterised in that the Just-In-Time function is to use bottom The generation of virtual machine Just-In-Time method.
- 8. big data processing unit according to claim 5, it is characterised in that also including vectorization processing module, be used for It is described handled using the Just-In-Time function pair big data before, carry out automatic vectorization processing;The vectorization processing module includes:Concurrency analyzes submodule, for analyzing the concurrency of the user's request;Instruction increase submodule, is added accordingly for the analysis result according to the concurrency in the Just-In-Time function Automatic vectorization process instruction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510303600.XA CN104965687B (en) | 2015-06-04 | 2015-06-04 | Big data processing method and processing device based on instruction set generation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510303600.XA CN104965687B (en) | 2015-06-04 | 2015-06-04 | Big data processing method and processing device based on instruction set generation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104965687A CN104965687A (en) | 2015-10-07 |
| CN104965687B true CN104965687B (en) | 2017-12-08 |
Family
ID=54219723
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510303600.XA Active CN104965687B (en) | 2015-06-04 | 2015-06-04 | Big data processing method and processing device based on instruction set generation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104965687B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109165220B (en) * | 2018-08-09 | 2021-06-22 | 天津威努特信息技术有限公司 | Data matching calculation method |
| CN109753306A (en) * | 2018-12-28 | 2019-05-14 | 北京东方国信科技股份有限公司 | A kind of big data processing method of because precompiled function caching engine |
| CN110597554A (en) * | 2019-08-01 | 2019-12-20 | 浙江大学 | A method for automatic generation and optimization of instruction function of instruction set simulator |
| CN112445483B (en) * | 2019-08-27 | 2023-11-24 | 龙芯中科技术股份有限公司 | Instruction generation method and device, electronic equipment and storage medium |
| CN111460454A (en) * | 2020-03-13 | 2020-07-28 | 中国科学院计算技术研究所 | A smart contract similarity retrieval method and system based on stack instruction sequence |
| CN111679858B (en) * | 2020-05-27 | 2025-08-22 | 中国平安财产保险股份有限公司 | Operation instruction processing method, device, computer equipment and storage medium |
| CN114924745A (en) * | 2022-05-19 | 2022-08-19 | 北京百度网讯科技有限公司 | Operating method, device and electronic device for deep learning compiler |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1309349A (en) * | 2001-03-22 | 2001-08-22 | 北京北大天正科技发展有限公司 | Method for on-line customization of software |
| CN1453699A (en) * | 2002-04-26 | 2003-11-05 | 株式会社东芝 | Generating method for developing environment in development on-chip system and media for storing the same program |
| JP2006091945A (en) * | 2004-09-21 | 2006-04-06 | Fuji Xerox Co Ltd | Software processor |
| CN101876908A (en) * | 2010-06-30 | 2010-11-03 | 中兴通讯股份有限公司 | User customizing method and system |
| US8489543B2 (en) * | 2005-08-12 | 2013-07-16 | Sugarcrm Inc. | Customer relationship management system and method |
| CN103207786A (en) * | 2013-04-28 | 2013-07-17 | 中国人民解放军信息工程大学 | Progressive intelligent backtracking vectorization code tuning method |
| CN103984541A (en) * | 2014-04-14 | 2014-08-13 | 美的集团股份有限公司 | Method and system for generating application procedure based on terminal source codes |
-
2015
- 2015-06-04 CN CN201510303600.XA patent/CN104965687B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1309349A (en) * | 2001-03-22 | 2001-08-22 | 北京北大天正科技发展有限公司 | Method for on-line customization of software |
| CN1453699A (en) * | 2002-04-26 | 2003-11-05 | 株式会社东芝 | Generating method for developing environment in development on-chip system and media for storing the same program |
| JP2006091945A (en) * | 2004-09-21 | 2006-04-06 | Fuji Xerox Co Ltd | Software processor |
| US8489543B2 (en) * | 2005-08-12 | 2013-07-16 | Sugarcrm Inc. | Customer relationship management system and method |
| CN101876908A (en) * | 2010-06-30 | 2010-11-03 | 中兴通讯股份有限公司 | User customizing method and system |
| CN103207786A (en) * | 2013-04-28 | 2013-07-17 | 中国人民解放军信息工程大学 | Progressive intelligent backtracking vectorization code tuning method |
| CN103984541A (en) * | 2014-04-14 | 2014-08-13 | 美的集团股份有限公司 | Method and system for generating application procedure based on terminal source codes |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104965687A (en) | 2015-10-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104965687B (en) | Big data processing method and processing device based on instruction set generation | |
| US11036614B1 (en) | Data control-oriented smart contract static analysis method and system | |
| US6381739B1 (en) | Method and apparatus for hierarchical restructuring of computer code | |
| Lattner et al. | Making context-sensitive points-to analysis with heap cloning practical for the real world | |
| KR101360512B1 (en) | Register allocation with simd architecture using write masks | |
| US6530079B1 (en) | Method for optimizing locks in computer programs | |
| US5146594A (en) | Method of producing object program based on interprocedural dataflow analysis of a source program | |
| EP0810523A2 (en) | Method for sequencing computer instruction execution in a data processing system | |
| EP0229245A2 (en) | Method for optimizing register allocation and assignments | |
| Fluet et al. | Implicitly threaded parallelism in Manticore | |
| US8458671B1 (en) | Method and system for stack back-tracing in computer programs | |
| Leupers et al. | Function inlining under code size constraints for embedded processors | |
| CN109460237A (en) | The Compilation Method and device of code | |
| US7694288B2 (en) | Static single assignment form pattern matcher | |
| US20100250564A1 (en) | Translating a comprehension into code for execution on a single instruction, multiple data (simd) execution | |
| US6360360B1 (en) | Object-oriented compiler mechanism for automatically selecting among multiple implementations of objects | |
| Gómez-Zamalloa et al. | Test case generation for object-oriented imperative languages in CLP | |
| US20020062478A1 (en) | Compiler for compiling source programs in an object-oriented programming language | |
| JP5048949B2 (en) | Software tools including asynchronous program flow modeling | |
| Greiner et al. | A provably time-efficient parallel implementation of full speculation | |
| CN103530471B (en) | A kind of CPA method based on simulator | |
| Lindstrom | Static evaluation of functional programs | |
| CN101882190B (en) | Method for formally verifying bytecode intermediate representation program module by module | |
| Zhu et al. | Locality analysis for parallel C programs | |
| Setzer | Java as a functional programming language |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Big data processing method and apparatus based on instruction set generation Effective date of registration: 20190709 Granted publication date: 20171208 Pledgee: Zhongguancun Beijing technology financing Company limited by guarantee Pledgor: BEIJING BONC TECHNOLOGY CO., LTD. Registration number: 2019990000686 |
|
| PC01 | Cancellation of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Date of cancellation: 20211202 Granted publication date: 20171208 Pledgee: Zhongguancun Beijing technology financing Company limited by guarantee Pledgor: BUSINESS-INTELLIGENCE OF ORIENTAL NATIONS Corp.,Ltd. Registration number: 2019990000686 |