CN114676194B - Distributed database lateral expansion method and device and computer equipment - Google Patents
Distributed database lateral expansion method and device and computer equipmentInfo
- Publication number
- CN114676194B CN114676194B CN202210252936.8A CN202210252936A CN114676194B CN 114676194 B CN114676194 B CN 114676194B CN 202210252936 A CN202210252936 A CN 202210252936A CN 114676194 B CN114676194 B CN 114676194B
- Authority
- CN
- China
- Prior art keywords
- geological
- data
- geological drilling
- database
- drilling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明实施例提供了一种分布式数据库横向扩展方法、装置及计算机设备，涉及数据库领域，该方法包括：获取地质钻探数据库，所述地质钻探数据库包括多个地质钻探数据元素，所述地质钻探数据元素包括多个数据属性；根据所述地质钻探数据元素，得到与多个所述地质钻探数据元素对应的数据元素特征；根据所述地质钻探数据库的查询规律和所述数据元素特征，构建地质数据树；根据所述地质数据树和多个所述地质钻探数据元素，得到约束模型；根据所述约束模型，得到横向扩展模型。通过构建地质数据树和约束模型，能够提高地质钻探数据库横向扩展的效率，减少资源占用率。
The embodiment of the present invention provides a distributed database horizontal expansion method, device and computer equipment, which relates to the field of databases. The method includes: obtaining a geological drilling database, the geological drilling database includes multiple geological drilling data elements, and the geological drilling data elements include multiple data attributes; according to the geological drilling data elements, obtaining data element characteristics corresponding to the multiple geological drilling data elements; according to the query rules of the geological drilling database and the data element characteristics, constructing a geological data tree; according to the geological data tree and the multiple geological drilling data elements, obtaining a constraint model; according to the constraint model, obtaining a horizontal expansion model. By constructing the geological data tree and the constraint model, the efficiency of the horizontal expansion of the geological drilling database can be improved and the resource occupancy rate can be reduced.
Description
Technical Field
The present invention relates to the field of database technologies, and in particular, to a method and apparatus for laterally expanding a distributed database, and a computer device.
Background
Conventional database technology mostly employs a stand-alone mode to provide database services that reside on a single computer. The single-machine database model is simple, the database technology is rapidly developed along with the development of the Internet and the arrival of big data age, and the problems of mass storage and concurrent access of the database are solved by adopting a distributed database. Geological drilling data is an important resource in the national geological industry, has the characteristics of large data volume, rich variety, high value and the like, and has lower transverse expansion efficiency of the traditional distributed database aiming at the geological drilling data.
Disclosure of Invention
The invention aims to solve the problems in the prior art to at least a certain extent, and provides a distributed database lateral expansion method, a distributed database lateral expansion device and a distributed database lateral expansion computer device.
The technical scheme of the embodiment of the invention is as follows:
in a first aspect, the present invention provides a method for laterally expanding a distributed database, where the method includes:
obtaining a geological drilling database, the geological drilling database comprising a plurality of geological drilling data elements, the geological drilling data elements comprising a plurality of data attributes;
Obtaining data element characteristics corresponding to a plurality of geological drilling data elements according to the geological drilling data elements;
constructing a geological data tree according to the query rule of the geological drilling database and the data element characteristics;
obtaining a constraint model according to the geological data tree and a plurality of geological drilling data elements;
and obtaining a transverse expansion model according to the constraint model.
According to some embodiments of the invention, the data element feature corresponding to each of the geological drilling data elements is characterized as one of a geological map, a geological morphology, a geological report, a geological file, and a geological classification;
In the case where the geological data tree includes at least two nodes, said constructing a geological data tree according to the query law of the geological drilling database and the data element characteristics, includes:
constructing the geological data tree and/or with the geological classification as a parent node and at least one of the geological map, the geological morphology, the geological report, the geological file and the geological classification as a child node
And constructing the geological data tree by taking the geological report as a father node and at least one of the geological map, the geological morphology and the geological file as a child node.
According to some embodiments of the invention, the deriving a constraint model from the geological data tree and the plurality of geological drilling data elements comprises:
Acquiring attribute characteristics corresponding to each data attribute;
according to the geological data tree, calculating the similarity among a plurality of geological drilling data elements to obtain a similarity coefficient;
According to the similarity coefficient, acquiring each data attribute corresponding to the geological drilling data element, and calculating the distance between attribute features corresponding to each acquired data attribute to obtain a change parameter of the attribute features;
and classifying each geological drilling data element according to the similarity coefficient and the change parameter of the attribute characteristic to obtain the constraint model.
According to some embodiments of the invention, the obtaining a lateral expansion model according to the constraint model includes:
according to the constraint model and a preset statistical model, calculating to obtain the update speed corresponding to the geological drilling data element;
Calculating to obtain spatial position parameters corresponding to each geological drilling data element according to the updating speed, wherein the spatial position parameters represent the spatial positions of the geological drilling data elements in the geological drilling database;
And calculating the expansion efficiency corresponding to the geological drilling database according to each geological drilling data element, the updating speed and the spatial position parameter to obtain the transverse expansion model.
According to some embodiments of the invention, the calculating the expansion efficiency corresponding to the geological drilling database according to each geological drilling data element, the update speed and the spatial position parameter to obtain the lateral expansion model includes:
calculating the corresponding expansion efficiency of the geological drilling database through an expansion efficiency algorithm to obtain the transverse expansion model, wherein the expansion efficiency algorithm is expressed as:
μ represents the spatial position parameter, e i represents the ith geological drilling data element, h j represents the jth data attribute corresponding to the geological drilling data element, i and j represent positive integers, ω represents the data volume of the geological drilling database, and η represents the update rate.
According to some embodiments of the invention, according to the similarity coefficient, each data attribute corresponding to the geological drilling data element is obtained, and a distance between attribute features corresponding to each obtained data attribute is calculated, so as to obtain a change parameter of the attribute features, including:
Calculating the distance between the attribute features corresponding to the data attributes through a distance algorithm to obtain the change parameters of the attribute features;
Wherein the distance algorithm is expressed as:
e(y,z)=((y-z)2B(y-z)1/2)
e (y, z) represents a variation parameter of the attribute, y and z both represent the attribute, and B represents a non-negative definite matrix of data in the geological drilling database.
According to some embodiments of the invention, after the obtaining a lateral expansion model according to the constraint model, the method further includes:
Acquiring the query range of the geological drilling database;
and obtaining a query optimizer according to the query range, the transverse expansion model and a preset query rule.
In a second aspect, the present invention provides a distributed database lateral expansion apparatus, the apparatus comprising:
A data acquisition module for acquiring a geological drilling database having a plurality of geological drilling data elements including a plurality of data attributes;
the first processing module is used for obtaining data element characteristics corresponding to a plurality of geological drilling data elements according to the geological drilling data elements;
the second processing module is used for constructing a geological data tree according to the query rule of the geological drilling database and the data element characteristics;
the third processing module is used for obtaining a constraint model according to the geological data tree and a plurality of geological drilling data elements;
and the fourth processing module is used for obtaining a transverse expansion model according to the constraint model.
In a third aspect, the present invention provides a computer device comprising a memory and a processor, the memory having stored therein computer readable instructions which, when executed by one or more of the processors, cause the one or more processors to perform the steps of the method as described in any of the first aspects above.
In a fourth aspect, the present invention also provides a computer readable storage medium readable and writable by a processor, the storage medium storing computer instructions which when executed by one or more processors cause the one or more processors to perform the steps of a method as described in any of the first aspects above.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
According to the embodiment of the invention, the geological drilling database comprises a plurality of geological drilling data elements, the geological drilling data elements comprise a plurality of data attributes, then data element characteristics corresponding to the plurality of geological drilling data elements are obtained according to the geological drilling data elements, a geological data tree is constructed according to the query rule and the data element characteristics of the geological drilling database, the association relation between data is obtained through the geological data tree, a constraint model is obtained according to the geological data tree and the plurality of geological drilling data elements, the data category meeting the condition is judged through the constraint model, a transverse expansion model is obtained according to the constraint model, the geological drilling database is transversely expanded through the transverse expansion model, and the transverse expansion efficiency of the distributed database is improved. According to the embodiment of the invention, the geological drilling database transverse expansion efficiency can be improved and the resource occupancy rate can be reduced by constructing the geological data tree and the constraint model.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate and do not limit the invention.
FIG. 1 is a schematic diagram of a distributed database lateral expansion device according to an embodiment of the present invention;
FIG. 2 is a flow diagram of a distributed database lateral expansion method according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart of the substeps of step S130 in FIG. 2;
FIG. 4 is a flow chart illustrating the substep of step S140 in FIG. 2;
FIG. 5 is a schematic flow chart of the substeps of step S150 in FIG. 2;
FIG. 6 is a flow chart of a method for lateral expansion of a distributed database according to another embodiment of the present invention;
FIG. 7 is a schematic diagram of a geological data tree of a distributed database lateral expansion method provided by one embodiment of the present invention;
FIG. 8 is a schematic diagram of a geological data tree of a distributed database lateral expansion method according to another embodiment of the present invention
Fig. 9 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
It should be noted that although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different from that in the flowchart. The terms first, second and the like in the description and in the claims and in the above-described figures, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
The lateral expansion is also called horizontal expansion, and supports a large number of requests with more nodes. Based on the above, the embodiment of the invention provides a distributed database lateral expansion method, a device and computer equipment, wherein the distributed database lateral expansion method is used for acquiring a geological drilling database, the geological drilling database comprises a plurality of geological drilling data elements, and the geological drilling data elements comprise a plurality of data attributes; the method comprises the steps of obtaining data element characteristics corresponding to a plurality of geological drilling data elements according to geological drilling data elements, constructing a geological data tree according to query rules and the data element characteristics of a geological drilling database, obtaining association relations among data by constructing the geological data tree, obtaining a constraint model according to the geological data tree and the plurality of geological drilling data elements, judging data types meeting conditions through the constraint model, obtaining a transverse expansion model according to the constraint model, transversely expanding the geological drilling database through the transverse expansion model, and improving the transverse expansion efficiency of the distributed database. According to the method, through constructing the geological data tree and the constraint model, the efficiency of transverse expansion of the geological drilling database can be improved, and the resource occupancy rate is reduced.
Embodiments of the present invention will be further described below with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 shows a schematic structural diagram of a lateral expansion device for a distributed database according to an embodiment of the present invention. In the example of fig. 1, the device acquires a geological drilling database through a data acquisition module 110, wherein the geological drilling database comprises a plurality of geological drilling data elements, the geological drilling data elements comprise a plurality of data attributes, then a first processing module 120 is utilized to obtain data element characteristics corresponding to the plurality of geological drilling data elements according to the geological drilling data elements, a second processing module 130 is utilized to construct a geological data tree according to the query rule and the data element characteristics of the geological drilling database, an association relation between data is obtained through the geological data tree, a third processing module 140 is utilized to obtain a constraint model according to the geological data tree and the plurality of geological drilling data elements, the constraint model is utilized to judge the data category meeting the conditions, a fourth processing module 150 is utilized to obtain a transverse expansion model according to the constraint model, and the transverse expansion model is utilized to transversely expand the geological drilling database, so that the efficiency of transverse expansion of the geological drilling database can be improved, and the resource occupancy rate is reduced.
It should be noted that, the data acquisition module 110 is respectively connected to the first processing module 120, the second processing module 130, the third processing module 140, and the fourth processing module 150, the first processing module 120 is connected to the second processing module 130, and the third processing module 140 is connected to the fourth processing module 150. The first processing module 120, the second processing module 130, the third processing module 140, and the fourth processing module 150 are all central processing units, and the central processing unit generally comprises a logic operation unit, a control unit, and a storage unit. According to different inputs received by the operation unit, different processes are carried out, and the calculation is carried out through an operator in the computer, so that the calculation efficiency is improved, and a large amount of human resources are saved.
The device and the application scenario described in the embodiments of the present invention are for more clearly describing the technical solution provided in the embodiments of the present invention, and do not constitute a limitation on the technical solution provided in the embodiments of the present invention, and as a person skilled in the art can know that the technical solution provided in the embodiments of the present invention is applicable to similar technical problems with the appearance of a new application scenario.
It will be appreciated by those skilled in the art that the distributed database lateral expansion arrangement shown in fig. 1 is not limiting of embodiments of the present invention and may include more or fewer modules than shown, or may combine certain components, or a different arrangement of components.
According to the above-described distributed database lateral expansion device, various embodiments of the distributed database lateral expansion method of the present invention are described below.
Fig. 2 is a schematic flow chart of a lateral expansion method of a distributed database according to an embodiment of the present invention, where the lateral expansion method of the distributed database is applied to a lateral expansion device of the distributed database. The distributed database lateral expansion method includes, but is not limited to, step S110, step S120, step S130, step S140, and step S150.
In step S110, a geological drilling database is obtained, the geological drilling database comprising a plurality of geological drilling data elements, the geological drilling data elements comprising a plurality of data attributes.
It can be appreciated that defining one data model element in the geological database as a geological data element, the geological drilling data element is also called geological drilling data, the geological drilling data element comprises a plurality of attributes, and illustratively comprises ground water level, drilling diameter, low-layer information, drilling information and the like, and the geological drilling data has the characteristics of large data volume, rich variety, high value and the like, and is beneficial to data processing on the geological drilling data by acquiring the geological drilling database, so that a geological data tree is convenient to build later.
Step S120, according to the geological drilling data elements, data element characteristics corresponding to the geological drilling data elements are obtained.
It will be appreciated that the analysis of geological and mineral resource data, according to the line classification method, geological data elements may be represented in an abstract manner as five basic data of a geological map, a geological morphology, a geological report, a geological file and a file classification, and these five basic data are taken as data element characteristics corresponding to the geological data elements. Wherein the geologic map, geologic file, and geologic morphology are physical entities that contain entity data and attribute data, and the geologic report and geologic classification belong to an abstract concept that contains only attribute data and no entity data. These five data elements are the basic data units of geological and mineral data. For ease of description, these five data are defined as triplets. The specific definition is as follows:
DZdataElement=<Type,MetaData,EntiyData>
Wherein Type is a Type identifier for determining a Type of the geological data unit, and corresponds to one of the five basic data units. For a certain geological data element, only one geological map, geological file, geological state, geological report and geological classification is possible, and there is certainty. MetaData is attribute data of a geological data element, which describes attribute information of the geological data element. ENTIYDATA is the physical data of the geologic material. The geologic map, geologic form and geologic file may contain physical data, and the geologic map, geologic file, geologic state, geologic report and geologic classification have good application value. For geological and mineral data with wide variety and large data volume, the method can be used for extracting 5 basic data element characteristics of the data and can be used for classifying the data and describing the data. These five geological data representations are model elements of the integrated geological data model that can be used to describe the static structure and dynamic operation of the model.
And step S130, constructing a geological data tree according to the query rule and the data element characteristics of the geological drilling database.
In an embodiment, the data element characteristics corresponding to each geological drilling data element are characterized as one of a geological map, a geological morphology, a geological report, a geological file and a geological classification, five expression modes corresponding to the data element characteristics are obtained through step S120, and are called geological data elements, and a geological data tree is constructed through the geological data elements and a query rule of a geological drilling database. Five basic geological data elements are used as nodes of the tree data structure for describing the characteristics of geological mineral resource information data and the mutual constraint relation between the geological mineral resource information data and the geological mineral resource information data. Unlike a common tree, a geological data tree is in fact a deformed tree structure, whose geological data elements are not only related to parent nodes and child nodes, but also in which there are three constraint relationships among geological data elements (1) a geological map, a geological file, and a geological morphology as the entity data of the minimum unit among geological data elements, the geological map being unable to be extracted from other types of geological data elements. The geological data elements are one of a geological map, a geological morphology and a geological file type, and can not be successed, and (2) as two conceptual entities, geological reports and geological classifications can be successes and predecessors. If they have precursors, then they can only be geology classified and not other geological data elements, and (3) if the geological report data elements have successors, then they can only be three types of geological maps, geological files and geological morphologies and not geology classified, then the successors of the geological classification elements can be any five geological data elements. By constructing the geological data tree, not only can the association relation between the data elements be obtained, but also the static representation and the dynamic operation of each geological data element can be represented.
It should be noted that, using the three constraint relationships described above, a formalized definition of the geological data tree may be given, DZDataTree = (D, R). Wherein D represents a geological drilling database containing a plurality of geological drilling data elements, R represents a relation set on the geological drilling data elements in D, and the relation set comprises three cases including but not limited to (1) if the geological data tree is empty, the geological data tree is empty if the geological data tree is not represented in D, (2) if only one geological data element is in D but no definition of relation exists, the relation set R= { H } on the geological drilling data elements exists if the geological drilling data elements are contained in D, wherein H has a binary relation that one geological data element in D serves as a root node of the geological data tree, if the geological data element is a geological map, a geological form or a geological file, no subsequent element exists, and if the geological data element is a geological report element, the subsequent geological map, the geological form or the geological form can only be the geological map, the geological file or the geological form. Exemplaryly,There is a division of D- { root }, D 1,D2,…,Dm, where m is greater than 0, for any one j.noteq.k (1. Ltoreq.j, k. Ltoreq.m), there isFor any i+.k (1+.i+.m), there is a geological data element x i∈Di with < root, x i > ∈H. For the D- { root } partition H- { < root, x 1>,…,<root,xm > }, there is a unique partition H 1,H2,…Hm (m > 0), for any j+.k (1+.j, k+.m), there is a unique partition H 1,H2,…Hm (m > 0)For any i (1≤i≤m), H i is a binary relation on D i, (D i,{Hi) is the geological data tree root according to this definition.
Referring to fig. 3, 7 and 8, in the case where the geological data tree includes at least two nodes, the geological data tree is constructed according to the query rules and data element characteristics of the geological drilling database, including, but not limited to, the steps of:
Step 131, constructing a geological data tree and/or using the geological classification as a parent node and at least one of the geological map, the geological morphology, the geological report, the geological file and the geological classification as child nodes
And 132, constructing a geological data tree by taking the geological report as a father node and at least one of a geological map, a geological morphology and a geological file as child nodes.
It should be noted that, in the case that the geological data tree includes at least two nodes, in a geological data tree, a node without a parent node is a root node, if a node has no parent node, it is called that the node has no precursor, if a node has no child node, it is called that the node has no successor, according to the above three constraint relationships, when the geological data tree is classified as a parent node, the geological map, the geological form, the geological report, the geological file and the geological classification can be used as child nodes thereof to construct the geological data tree, and when the geological report is a parent node, at least one of the geological map, the geological form and the geological file is used as a child node to construct the geological data tree. The obtained geological map, geological morphology and geological file can only be used as leaf nodes of the geological data tree, and the constraint conditions are met without follow-up. By constructing the geological data tree, not only can the association relation between the data elements be obtained, but also the static representation and the dynamic operation of each geological data element can be represented.
Step S140, obtaining a constraint model according to the geological data tree and the geological drilling data elements.
It can be understood that the data attribute related to the known geological drilling data type is extracted from the geological data tree, the geological drilling data element corresponding to the data attribute is obtained, the constraint model is obtained according to the geological data tree and the geological drilling data elements, the data category meeting the condition can be judged by constructing the constraint model, and the subsequent transverse expansion of the database and the query optimization of the expanded database are facilitated.
Referring to FIG. 4, from a geological data tree and a plurality of geological drilling data elements, a constraint model is derived, including, but not limited to, the following steps:
step S141, obtaining attribute characteristics corresponding to each data attribute.
It can be understood that, according to the foregoing steps S110 and S120, the geological drilling data element includes a plurality of data attributes, and attribute features corresponding to the plurality of data attributes are obtained in the manner of step S120, which is similar to the foregoing steps and is not described herein. And the constraint model is convenient to construct later by acquiring the attribute characteristics corresponding to each data attribute.
Step S142, calculating the similarity among a plurality of geological drilling data elements according to the geological data tree to obtain a similarity coefficient.
It will be appreciated that data attributes associated with known geological drilling data types are extracted from a geological data tree, a plurality of geological drilling data elements corresponding to the data attributes are obtained from the data attributes, and similarity between the plurality of geological drilling data elements is calculated by a similarity formula to obtain a similarity coefficient. And by calculating the similarity among the geological drilling data elements and performing data processing, the subsequent construction of the constraint model is facilitated.
Wherein the similarity formula is expressed as:
s (w b,wc) represents the similarity coefficient, w b and w c both represent geological drilling data elements, w bc represents the vector between w b and w c, h g represents the data attribute, η represents the update rate, and m represents the number.
Step S143, according to the similarity coefficient, each data attribute corresponding to the geological drilling data element is obtained, and the distance between the attribute features corresponding to each obtained data attribute is calculated to obtain the change parameters of the attribute features.
In one embodiment, the following formula is used to calculate the variation parameters of the attribute features corresponding to the borehole data under different geology:
Wherein e (y, z) represents a variation parameter of the attribute feature, y and z both represent the attribute feature, and m represents the number.
It will be appreciated that, as available from the above, if the value of s (w b,wc) is changed, the data attribute characteristics vary greatly. Assuming that s (w b,wc) =1, in the above formula for solving the variation parameters of the attribute features, the obtained attribute values of the distributed geological drilling data may be negative. Thus, the calculation requires the use of the following formula: Wherein e (y, z) represents a variation parameter of the attribute feature, y and z both represent the attribute feature, and m represents the number.
In an embodiment, according to the similarity coefficient, each data attribute corresponding to the geological drilling data element is obtained, and the distance between attribute features corresponding to each obtained data attribute is calculated to obtain a change parameter of the attribute features, including:
calculating the distance between the attribute features corresponding to each data attribute through a distance algorithm to obtain the change parameters of the attribute features;
Wherein, the distance algorithm is expressed as:
e(y,z)=((y-z)2B(y-z)1/2)
e (y, z) represents the variation parameters of the attribute, y and z both represent the attribute, and B represents the non-negative definite matrix of data in the geological drilling database.
It should be noted that, assuming that B is an identity matrix, the above formula may be converted into:
it should be noted that, assuming that B is a diagonal matrix, the above formula may be converted into:
By the method, data in the distributed geological drilling database can be processed to obtain data variables, and more than 90% of information characteristics of the distributed geological drilling database are obtained.
And S144, classifying each geological drilling data element according to the similarity coefficient and the change parameter of the attribute characteristic to obtain a constraint model.
It can be appreciated that the differences between the geological drilling data elements are obtained according to the similarity coefficients and the change parameters of the attribute characteristics, so that the geological drilling data elements in the geological drilling database can be classified, and the information in the geological drilling database is divided into several different categories according to the characteristics of the geological drilling data elements, so that an accurate data basis is provided for the expansion of the geological drilling database. And establishing a plurality of constraint models, judging the data types meeting the query conditions, and querying the data according to different data types to realize the expansion and optimization of the geological drilling database.
In one embodiment, a plurality of constraint models are built by the following formula:
Wherein, the Representing the difference between the two vectors, Y represents the data element feature vector,A feature vector representing a classification of geological drilling data elements, T representing the median of a set of data, K representing the number, Y j representing a feature of a certain data element. By building multiple constraint models, it can be used to optimize the lateral expansion of the distributed geological drilling database.
And step S150, obtaining a transverse expansion model according to the constraint model.
Referring to fig. 5, a lateral expansion model is derived from the constraint model, including, but not limited to, the following steps:
Step S151, calculating to obtain the update speed corresponding to the geological drilling data element according to the constraint model and the preset statistical model.
It should be noted that, the data growth is affected by the constraint model, and the preset statistical model includes a timer and a counter, where the timer is used to count the time taken by the actual data volume to increase, and the counter is used to count the number of the actual data volume to increase, so that the update speed corresponding to the geological drilling data element is solved according to the number of the data volume to increase and the time of the data volume to increase, and is expressed as η. And by calculating the update speed, the subsequent calculation of the transverse expansion of the geological drilling database is facilitated.
Step S152, calculating to obtain the spatial position parameters corresponding to the geological drilling data elements according to the updating speed, wherein the spatial position parameters represent the spatial positions of the geological drilling data elements in the geological drilling database.
It should be noted that, according to the step S151, according to the update speed, updated geological drilling data elements corresponding to the update speed are obtained, and sequentially added to the dataset formed by the geological drilling data elements according to the increasing order of the geological drilling data elements, so as to obtain subscripts of positions in the dataset, thereby obtaining spatial position parameters corresponding to each geological drilling data element. And the position parameters in the space are obtained through calculation, so that the subsequent transverse expansion of the distributed geological drilling database is facilitated.
And step 153, calculating the expansion efficiency corresponding to the geological drilling database according to each geological drilling data element, the updating speed and the spatial position parameter to obtain a transverse expansion model.
It can be understood that on the basis of each geological drilling data element, the spatial position parameter corresponding to the geological drilling data element is obtained by calculating the update speed corresponding to the geological drilling data element, so that the spatial position of each geological drilling data element is obtained, and according to the calculation mode, a transverse expansion model is constructed, and the transverse expansion of the distributed geological drilling database can be performed by using the transverse expansion model.
It should be noted that, the data volume of the geological drilling database is set to be w 1, the data attribute number is set to be g, and the geological drilling data element set composed of all geological drilling databases is { e 1,e2,…,ew }, where e i is the ith data in the geological drilling databases. The dataset consisting of the data attributes of all geological drilling data elements is { h 1,h2,…,hg }, where h j is the j-th data attribute in the geological drilling database and the update rate of the geological drilling data is η.
In an embodiment, according to each geological drilling data element, the update speed and the spatial position parameter, the expansion efficiency corresponding to the geological drilling database is calculated to obtain a transverse expansion model, which comprises the following steps:
Calculating the expansion efficiency corresponding to the geological drilling database through an expansion efficiency algorithm to obtain a transverse expansion model;
Wherein, the expansion efficiency algorithm is expressed as:
μ represents a spatial position parameter, e i represents an ith geological drilling data element, h j represents a jth data attribute corresponding to the geological drilling data element, i and j represent positive integers, ω represents an expanded data volume of the geological drilling database, w 1 represents an initial data volume of the geological drilling database, and η represents an update rate. According to the method, the geological drilling database can be laterally expanded, so that data support is provided for different industries.
Referring to fig. 6, after deriving the lateral expansion model from the constraint model, the method further includes, but is not limited to, the steps of:
step S210, obtaining the query scope of the geological drilling database.
It should be noted that the scope of a query includes a collection of alternative execution plans for a query request that can produce the same result, each query plan having an order of execution, and different implementations of various operations, which may result in different performance of the execution plans. An execution plan is typically abstracted into an operation tree, where nodes are operations, and the shape of the tree determines the order in which the operations are executed. These operation trees are generated according to the conversion rules of the query request. These query operation trees are equivalent in that they can generate the same result set, and query needs to be performed on all the query operation trees to obtain the query scope. By acquiring the query range, the subsequent construction of the query optimizer is facilitated.
And step S220, obtaining a query optimizer according to the query range, the transverse expansion model and a preset query rule.
It will be appreciated that the query optimizer of the geological drilling database consists of three parts, the query scope of the geological drilling database, the generation of the laterally expanding model of the geological drilling database, and the query rules of the geological drilling database. And (2) acquiring a query range according to the step S210, obtaining a transverse expansion model according to the query range through a distributed database transverse expansion method, and then obtaining a query optimizer by combining with a preset rule. And realizing the transverse expansion query of the distributed geological drilling database through the obtained query optimizer.
It should be noted that the lateral expansion model includes a cost model of the query optimizer, where the cost model of the query optimizer includes a cost function, data statistics information, an intermediate result set estimation tool, and the like, and the main measure of the execution cost is the execution time of the query. The preset query rule is to use a cost model to detect an execution plan, filter an optimal solution which does not generate the execution plan through dynamic programming or simulated annealing strategy in a search space, check the plan according to the cost model to predict response time, comprehensively consider all execution schemes, and finally obtain the optimal execution scheme.
It should be further noted that the distributed geological drilling database query optimizer may be configured to construct the distributed geological drilling database query optimizer according to the transverse expansion model of the drilling database and the element features corresponding to the geological drilling data elements, analyze the data element features, and distinguish the similarity of the fussy and repeated data in the geological drilling database, so that the feature vector describes the related data with high similarity in the geological drilling database, and implement the transverse expansion query of the distributed geological drilling database. An extended optimization process for a distributed geological drilling database refers to a process that generates a Query Execution Plan (QEP) that should minimize the objective function, i.e., the time required for query execution in a distributed environment. The transverse expansion method provided by the invention can also be applied to the transverse expansion of the distributed geological drilling database, and the query optimizer is also applicable, so that in the search space stage, whether the expansion of the distributed geological drilling database is centralized or distributed does not need to be considered, and the execution plan is generated according to a certain conversion rule, so that the method is applicable in any case, and can provide data support for different industries.
Referring to fig. 9, fig. 9 illustrates a computer device 900 provided by an embodiment of the present invention. The computer device 900 may be a server or a terminal, and the internal structure of the computer device 900 includes, but is not limited to:
a memory 910 for storing a program;
the processor 920 is configured to execute the program stored in the memory 910, and when the processor 920 executes the program stored in the memory 910, the processor 920 is configured to perform the above-described distributed database lateral expansion method.
The processor 920 and the memory 910 may be connected by a bus or other means.
The memory 910, as a non-transitory computer readable storage medium, may be used to store a non-transitory software program and a non-transitory computer executable program, such as the distributed database lateral expansion method described in any embodiment of the invention. The processor 920 implements the distributed database lateral expansion method described above by running non-transitory software programs and instructions stored in the memory 910.
The memory 910 may include a storage program area that may store an operating system, an application program required for at least one function, and a storage data area that may store a distributed database lateral expansion method as described above. In addition, memory 910 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some implementations, the memory 910 may optionally include memory located remotely from the processor 920, which may be connected to the processor 920 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The non-transitory software programs and instructions required to implement the distributed database lateral expansion method described above are stored in memory 910, which when executed by one or more processors 920, perform the distributed database lateral expansion method provided by any embodiment of the present invention.
The embodiment of the invention also provides a computer readable storage medium which stores computer executable instructions for executing the distributed database lateral expansion method.
In one embodiment, the storage medium stores computer-executable instructions that are executed by one or more control processors 920, for example, by one of the processors 920 in the computer device 900, so that the one or more processors 920 perform the distributed database lateral expansion method provided by any embodiment of the present invention.
The embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, i.e. may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any information delivery media.
The preferred embodiments of the present invention have been described in detail, but the present invention is not limited to the above embodiments, and those skilled in the art will appreciate that the present invention may be practiced without departing from the spirit of the present invention. Various equivalent modifications and substitutions may be made in the shared context, and are intended to be included within the scope of the present invention as defined in the following claims.
Claims (7)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210252936.8A CN114676194B (en) | 2022-03-15 | 2022-03-15 | Distributed database lateral expansion method and device and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210252936.8A CN114676194B (en) | 2022-03-15 | 2022-03-15 | Distributed database lateral expansion method and device and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114676194A CN114676194A (en) | 2022-06-28 |
| CN114676194B true CN114676194B (en) | 2025-07-25 |
Family
ID=82074710
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210252936.8A Active CN114676194B (en) | 2022-03-15 | 2022-03-15 | Distributed database lateral expansion method and device and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114676194B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102576369A (en) * | 2009-08-24 | 2012-07-11 | 阿玛得斯两合公司 | Continuous full scan data store table and distributed data store featuring predictable answer time for unpredictable workload |
| CN104008167A (en) * | 2014-05-30 | 2014-08-27 | 中国地质调查局发展研究中心 | Geologic map database establishment model for describing geologic bodies through single feature |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102156726B (en) * | 2011-04-01 | 2013-12-25 | 中国测绘科学研究院 | Geographic element querying and extending method based on semantic similarity |
| CN103034650B (en) * | 2011-09-29 | 2015-10-28 | 北京新媒传信科技有限公司 | A kind of data handling system and method |
| US10691682B2 (en) * | 2017-10-04 | 2020-06-23 | EMC IP Holding Company LLC | Storing and processing JSON documents in a SQL database table |
| CN108305184A (en) * | 2017-12-22 | 2018-07-20 | 安徽海螺集团有限责任公司 | A kind of number mining softwares platform |
| CN111984739A (en) * | 2019-05-24 | 2020-11-24 | 中移(苏州)软件技术有限公司 | A conversion method, terminal and storage medium between XML and relational database |
| CN111125114B (en) * | 2019-12-25 | 2023-06-16 | 中国建设银行股份有限公司 | Method and device for laterally expanding database and accessing data table |
-
2022
- 2022-03-15 CN CN202210252936.8A patent/CN114676194B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102576369A (en) * | 2009-08-24 | 2012-07-11 | 阿玛得斯两合公司 | Continuous full scan data store table and distributed data store featuring predictable answer time for unpredictable workload |
| CN104008167A (en) * | 2014-05-30 | 2014-08-27 | 中国地质调查局发展研究中心 | Geologic map database establishment model for describing geologic bodies through single feature |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114676194A (en) | 2022-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Gülagiz et al. | Comparison of hierarchical and non-hierarchical clustering algorithms | |
| CN106452825B (en) | An alarm correlation analysis method for power distribution communication network based on improved decision tree | |
| JP3209163B2 (en) | Classifier | |
| US10963802B1 (en) | Distributed decision variable tuning system for machine learning | |
| US20220036222A1 (en) | Distributed algorithm to find reliable, significant and relevant patterns in large data sets | |
| CN110737805B (en) | Method and device for processing graph model data and terminal equipment | |
| CN111612069A (en) | Density clustering method and device based on dynamic grid hash index | |
| US20240013061A1 (en) | Architecture search method and apparatus for large-scale graph, and device and storage medium | |
| CN117539920B (en) | Data query method and system based on real estate transaction multidimensional data | |
| US10733156B2 (en) | Parallel discretization of continuous variables in supervised or classified dataset | |
| CN118733006A (en) | Script file generation method, task processing method and electronic device | |
| Tiwari | Developments in KD tree and KNN searches | |
| CN119003533A (en) | Real-time index construction and intelligent optimization method and system based on large model driving | |
| CN112257332B (en) | Method and device for evaluating a simulation model | |
| CN119886890A (en) | Block chain-based industrial park multi-enterprise carbon emission monitoring method and system | |
| CN120217525A (en) | A BIM-based construction project visualization management method and system | |
| CN112990364B (en) | Graph data node classification method and device | |
| Olech et al. | Hierarchical gaussian mixture model with objects attached to terminal and non-terminal dendrogram nodes | |
| CN114676194B (en) | Distributed database lateral expansion method and device and computer equipment | |
| CN117010538A (en) | Method and system for predicting agricultural machinery service resource spare parts | |
| CN118656273A (en) | Hard disk failure prediction and data migration method for low-quality data sets | |
| CN114253938A (en) | Data management method, data management device and storage medium | |
| CN108897820B (en) | A Parallelization Method of DENCLUE Algorithm | |
| CN116862658A (en) | Credit evaluation method, apparatus, electronic device, medium and program product | |
| CN114816979A (en) | A Software Defect Prediction Method Based on Cluster Analysis and Decision Tree Algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |