CN103077219A - Method and device for automatically storing data - Google Patents
Method and device for automatically storing data Download PDFInfo
- Publication number
- CN103077219A CN103077219A CN2012105894919A CN201210589491A CN103077219A CN 103077219 A CN103077219 A CN 103077219A CN 2012105894919 A CN2012105894919 A CN 2012105894919A CN 201210589491 A CN201210589491 A CN 201210589491A CN 103077219 A CN103077219 A CN 103077219A
- Authority
- CN
- China
- Prior art keywords
- data
- particular data
- node
- performance
- identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims description 22
- 238000013508 migration Methods 0.000 claims description 3
- 230000005012 migration Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 abstract description 2
- 238000012545 processing Methods 0.000 abstract description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 230000008569 process Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明适用于互联网通信领域，提供了一种数据自动放置装置，所述装置包括：数据识别模块，用于识别特定数据；节点识别模块，用于识别集群中的高性能节点；匹配模块，用于将所述特定数据放置于所述高性能节点上；调整模块，用于定期检查所述特定数据的位置，当所述特定数据的位置不在所述高性能节点上时，启动匹配模块。本发明使得特定数据总是存放于性能高的节点上，系统在处理特定数据访问请求时的操作使用的是优质存储介质，从而保证特定数据拥有较低的访问延迟。
The present invention is applicable to the field of Internet communication, and provides an automatic data placement device, which includes: a data identification module for identifying specific data; a node identification module for identifying high-performance nodes in a cluster; a matching module for After placing the specific data on the high-performance node; an adjustment module, configured to periodically check the location of the specific data, and start a matching module when the location of the specific data is not on the high-performance node. The invention enables specific data to be always stored on high-performance nodes, and the system uses high-quality storage media when processing specific data access requests, thereby ensuring low access delay for specific data.
Description
Technical field
The invention belongs to field of Internet communication, relate in particular to a kind of data and automatically place method and apparatus.
Background technology
Along with the explosive increase of data, the cluster of storage mass data arises at the historic moment.Because the data volume of storing in the cluster is very large, what have reaches the PB level, and synchronization may have a large amount of data accesses.Therefore, whether the deposit position of data is reasonable, directly has influence on the access delay of data.
Access delay refers to the user and improves data access request and receive the needed time of data to the user.In cluster, data volume is huge, and the server that therefore is used for the storage data may have a lot, may run into the very heavy situation of certain server node load during user's request msg, therefore just needs queuing, can cause larger access delay.Solve the method for access delay, the load balancing method is arranged, soon the load balancing in the cluster guarantees not have the bottleneck that certain node becomes service on each node; The user many copies method arranged, namely when some data access frequency is higher, just makes the copies of several these data more, so that can select load server light, close together to come the transmission of data from a plurality of copies.Some has adopted the method for classification storage, active low volume data collection is deposited on the good node of access performance, so that overall access is optimum.
But many times, the access frequency of some data is not very high, but but very important.Namely often not accessed, but accessed the time, the real-time of requirement is higher.This data if process with hierarchical stor, are difficult to prove effective, because the foundation of hierarchical stor grouped data mainly is according to its active degree and moving costs etc.Simply construct many copies or keep load balancing, can not guarantee that also particular data has lower access delay.
In view of the server node in the cluster may be in the situation that access performance differs under many circumstances, and the access characteristics of data is inconsistent, therefore be necessary to adopt a kind of more rational data placement strategy, so that particular data is always deposited on the high node of performance, what the operation of system when processing the special data access request used is the high-quality storage medium, thereby guarantees that particular data has lower access delay.
Summary of the invention
The embodiment of the invention provides a kind of data automatically to place method and apparatus, be intended to solve current cluster when the access characteristics of node visit performance difference and data is inconsistent, particular data is always deposited on the high node of performance, caused to guarantee that particular data has lower access delay.
For this reason, the embodiment of the invention provides following technical scheme:
The automatic laying method of a kind of data may further comprise the steps:
S101: identification particular data;
S102: the high-performance node in the identification cluster;
S103: described particular data is positioned on the described high-performance node;
S104: make regular check on the position of described particular data, when the position of described particular data is not on described high-performance node, repeating step S103.
The embodiment of the invention also provides a kind of data automatic apparatus for placing, comprising:
Data identification module is used for the identification particular data;
The node recognition module is for the high-performance node of identification cluster;
Matching module is used for described particular data is positioned over described high-performance node;
Adjusting module is used for making regular check on the position of described particular data, when the position of described particular data is not on described high-performance node, starts matching module.
Compared with prior art, embodiments of the invention have following advantage:
The embodiment of the invention is by the identification particular data, and the high-performance node in the identification cluster, then described particular data is positioned on the described high-performance node, and make regular check on the position of described particular data, thereby keep particular data is deposited on the good node of access performance, so that particular data has lower access delay.
Description of drawings
Fig. 1 is the method flow diagram of the automatic laying method of data that provides of first embodiment of the invention;
Fig. 2 is the structural drawing of the automatic apparatus for placing of data that provides of second embodiment of the invention.
Embodiment
In order to make purpose of the present invention, technical scheme and advantage clearer, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that described herein only is a part of embodiment of the present invention, rather than whole embodiment.Based on the embodiment among the present invention, the every other embodiment that those of ordinary skills obtain under the prerequisite of not making creative work belongs to the scope of protection of the invention.
Fig. 1 is the method flow diagram of the automatic laying method of data that provides of first embodiment of the invention, for convenience of explanation, only shows the part relevant with the embodiment of the invention.
As shown in Figure 1, the method may further comprise the steps:
Step S101, the identification particular data.
Concrete, the reading system document according to the feature that writes in advance the described particular data in the described system documentation, is identified described particular data.
Preferably, the feature of described particular data is by manually establishing in advance the writing system document.
Step S102, the high-performance node in the identification cluster.
Concrete, by the host name identification feature, identify described high-performance node.
Step S103 is positioned over described particular data on the described high-performance node.
Concrete, described particular data is positioned on the described high-performance node, when described high-performance node storage space is not enough, the inactive data of the part of moving out.
Step S104 makes regular check on the position of described particular data, when the position of described particular data is not on described high-performance node, and repeating step S103.
Concrete, when in the storage system Data Migration and data backup occuring, check the position of described particular data.
Concrete, in hierarchical stor, because the temperature of particular data is not high, in the time of can moving on the lower node of performance, when selecting migrating objects, particular data is screened.
Concrete, the Data Migration that the factor data backup causes, the node at particular data place may lose efficacy or withdraw from, so that the position of particular data changes, then repeating step S103 is to step S104.
Based on identical design, second embodiment of the invention provides a kind of data automatic apparatus for placing, and as shown in Figure 2, this device comprises:
Data identification module 201 is used for the identification particular data.
Concrete, described data identification module reading system document according to the feature that writes in advance the described particular data in the described system documentation, is identified described particular data.
Node recognition module 202 is for the high-performance node of identification cluster.
Concrete, described node recognition module by the host name identification feature, is identified described high-performance node.
Matching module 203 is used for described particular data is positioned over described high-performance node.
Concrete, described matching module 203 when described high-performance node storage space deficiency, the inactive data of the part of moving out.
Adjusting module 204 is used for making regular check on the position of described particular data, when the position of described particular data is not on described high-performance node, starts matching module 203.
The embodiment of the invention is by the identification particular data, and the high-performance node in the identification cluster, then described particular data is positioned on the described high-performance node, and make regular check on the position of described particular data, thereby keep particular data is deposited on the good node of access performance, so that particular data has lower access delay.
It will be appreciated by those skilled in the art that the module in the device among the embodiment can be distributed in the device of embodiment according to the embodiment description, also can carry out respective change and be arranged in the one or more devices that are different from the present embodiment.The module of above-described embodiment can be merged into a module, also can further split into a plurality of submodules.
Through the above description of the embodiments, those skilled in the art can be well understood to the present invention and can realize by the mode that software adds essential general hardware platform, can certainly pass through hardware, but the former is better embodiment in a lot of situation.Based on such understanding, the part that technical scheme of the present invention contributes to prior art in essence in other words can embody with the form of software product, this computer software product is stored in the storage medium, comprise that some instructions are with so that a station terminal equipment (can be mobile phone, personal computer, server, the perhaps network equipment etc.) carry out the described method of each embodiment of the present invention.
The above only is preferred implementation of the present invention; should be pointed out that for those skilled in the art, under the prerequisite that does not break away from the principle of the invention; can also make some improvements and modifications, these improvements and modifications also should be looked protection scope of the present invention.
Claims (9)
1. a data placement method is characterized in that, said method comprising the steps of:
S101, identification particular data;
High-performance node in S102, the identification cluster;
S103, described particular data is positioned on the described high-performance node;
S104, make regular check on the position of described particular data, when the position of described particular data is not on described high-performance node, repeating step C.
2. data placement method as claimed in claim 1 is characterized in that, described step S101 may further comprise the steps:
The reading system document according to the feature that writes in advance the described particular data in the described system documentation, is identified described particular data.
3. data placement method as claimed in claim 1 or 2 is characterized in that, described step S102 may further comprise the steps: by the host name identification feature, identify described high-performance node.
4. data placement method as claimed in claim 1 or 2 is characterized in that, described step S103 may further comprise the steps: when described high-performance node storage space is not enough, and the inactive data of the part of moving out.
5. data placement method as claimed in claim 1 or 2, it is characterized in that, described step S104 may further comprise the steps: when in the storage system Data Migration and data backup occuring, check the position of described particular data, when the position of described particular data is not on described high-performance node, repeating step S103.
6. a data placement device is characterized in that, described device comprises:
Data identification module is used for the identification particular data;
The node recognition module is for the high-performance node of identification cluster;
Matching module is used for described particular data is positioned over described high-performance node;
Adjusting module is used for making regular check on the position of described particular data, when the position of described particular data is not on described high-performance node, starts matching module.
7. data placement device as claimed in claim 6 is characterized in that, described data identification module reading system document according to the feature that writes in advance the described particular data in the described system documentation, is identified described particular data.
8. such as claim 6 or 7 described data placement devices, it is characterized in that, described node recognition module by the host name identification feature, is identified described high-performance node.
9. such as claim 6 or 7 described data placement devices, it is characterized in that, described matching module when described high-performance node storage space is not enough, the inactive data of the part of moving out.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012105894919A CN103077219A (en) | 2012-12-29 | 2012-12-29 | Method and device for automatically storing data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012105894919A CN103077219A (en) | 2012-12-29 | 2012-12-29 | Method and device for automatically storing data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103077219A true CN103077219A (en) | 2013-05-01 |
Family
ID=48153749
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012105894919A Pending CN103077219A (en) | 2012-12-29 | 2012-12-29 | Method and device for automatically storing data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103077219A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105183386A (en) * | 2015-09-14 | 2015-12-23 | 联想(北京)有限公司 | Information processing method and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101582013A (en) * | 2009-06-10 | 2009-11-18 | 成都市华为赛门铁克科技有限公司 | Method, device and system for processing storage hotspots in distributed storage |
| CN101794199A (en) * | 2010-03-26 | 2010-08-04 | 山东高效能服务器和存储研究院 | Method for accelerating performance of computer disk based on disk cache |
| CN102388374A (en) * | 2011-09-28 | 2012-03-21 | 华为技术有限公司 | Method and device for storing data |
| CN102388358A (en) * | 2011-09-30 | 2012-03-21 | 华为技术有限公司 | Method and system for configurating memory device under hybrid storage environment |
-
2012
- 2012-12-29 CN CN2012105894919A patent/CN103077219A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101582013A (en) * | 2009-06-10 | 2009-11-18 | 成都市华为赛门铁克科技有限公司 | Method, device and system for processing storage hotspots in distributed storage |
| CN101794199A (en) * | 2010-03-26 | 2010-08-04 | 山东高效能服务器和存储研究院 | Method for accelerating performance of computer disk based on disk cache |
| CN102388374A (en) * | 2011-09-28 | 2012-03-21 | 华为技术有限公司 | Method and device for storing data |
| CN102388358A (en) * | 2011-09-30 | 2012-03-21 | 华为技术有限公司 | Method and system for configurating memory device under hybrid storage environment |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105183386A (en) * | 2015-09-14 | 2015-12-23 | 联想(北京)有限公司 | Information processing method and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11734125B2 (en) | Tiered cloud storage for different availability and performance requirements | |
| CN101763437B (en) | Method and device for realizing high-speed buffer storage | |
| US9678678B2 (en) | Storage network data retrieval | |
| CN103067433B (en) | A kind of data migration method of distributed memory system, equipment and system | |
| CN102984280A (en) | Data backup system and method for social cloud storage network application | |
| CN103530388A (en) | Performance improving data processing method in cloud storage system | |
| CN104144202B (en) | Access method, system and the device of Hadoop distributed file system | |
| CN107888666A (en) | A kind of cross-region data-storage system and method for data synchronization and device | |
| CN101471845A (en) | Method for adjusting data block counterpart number and metadata server node | |
| CN106302640A (en) | Data request processing method and device | |
| CN103049574B (en) | Realize key assignments file system and the method for file dynamic copies | |
| CN106302659A (en) | A kind of based on cloud storage system promotes access data quick storage method | |
| CN110727738A (en) | Global routing system based on data fragmentation, electronic equipment and storage medium | |
| CN106161573A (en) | Server buffer processing method, Apparatus and system | |
| CN105516284A (en) | Clustered database distributed storage method and device | |
| CN106453611B (en) | A method and device for load balancing of multiple storage nodes | |
| CN106331160A (en) | Data migration method and system | |
| CN105630683A (en) | Cloud testing architecture | |
| WO2016070655A1 (en) | Information backup method, device, server and terminal | |
| CN101483668A (en) | Network storage and access method, device and system for hot spot data | |
| CN103856535B (en) | A kind of method and apparatus obtaining user data | |
| CN103077127A (en) | Method and device for determining data migration target | |
| CN102185768A (en) | Configuration business deploying method and device | |
| CN103077219A (en) | Method and device for automatically storing data | |
| CN105207993A (en) | Data access and scheduling method in CDN, and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20130501 |