CN110347333A - Improve method, apparatus, computer equipment and the storage medium of clone's mirror image performance - Google Patents
Improve method, apparatus, computer equipment and the storage medium of clone's mirror image performance Download PDFInfo
- Publication number
- CN110347333A CN110347333A CN201910441492.0A CN201910441492A CN110347333A CN 110347333 A CN110347333 A CN 110347333A CN 201910441492 A CN201910441492 A CN 201910441492A CN 110347333 A CN110347333 A CN 110347333A
- Authority
- CN
- China
- Prior art keywords
- image
- read
- source
- metadata
- written
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明适用于区块链领域,提供提高克隆镜像性能的方法、装置、计算机设备及存储介质,其中,所述方法包括:根据获取到的待读写对象,判断与所述待读写对象对应的镜像是否存在预先设置的全局镜像继承关系,所述全局镜像继承关系包括源镜像与克隆镜像的继承关系;若存在所述全局镜像继承关系,则根据所述全局镜像继承关系获取与所述待读写对象对应的源对象的元数据,所述元数据包括用于计算所述待读写对象对应的源对象所在的一致性哈希节点的信息;根据所述元数据计算一致性哈希值,得到所述待读写对象对应的源对象所在的一致性哈希节点;在所述待读写对象对应的源对象所在的一致性哈希节点上进行读写。本申请能够缩短读写路径,提高克隆镜像性能。
The present invention is applicable to the blockchain field, and provides methods, devices, computer equipment, and storage media for improving the performance of cloning images, wherein the method includes: according to the obtained objects to be read and written, judging whether the object corresponds to the object to be read or written Whether there is a preset global image inheritance relationship in the image of the image, and the global image inheritance relationship includes the inheritance relationship between the source image and the clone image; if there is the global image inheritance relationship, then according to the global image inheritance relationship, the The metadata of the source object corresponding to the read-write object, the metadata includes information used to calculate the consistent hash node where the source object corresponding to the object to be read and written is located; calculate the consistent hash value according to the metadata , obtain the consistent hash node where the source object corresponding to the object to be read and written is located; read and write on the consistent hash node where the source object corresponding to the object to be read and written is located. The application can shorten the read-write path and improve the performance of the clone image.
Description
技术领域technical field
本发明属于区块链技术领域,尤其涉及提高克隆镜像性能的方法、装置、计算机设备及存储介质。The invention belongs to the technical field of block chains, and in particular relates to a method, a device, a computer device and a storage medium for improving the performance of a clone image.
背景技术Background technique
目前,在一致性哈希存储系统中,镜像中包括多个大小一致的对象,这些对象按一定规则组成镜像,例如:一个1G的镜像,可以包括256个4M的对象,这256个4M的对象按一定规则组成这个1G的镜像。在对镜像进行克隆存储时,通过计算这256个4M对象的哈希值,将这256个4M对象存储在对应的一致性哈希节点上。在需要对某个对象进行读写时,通过哈希值计算到该个4M对象所在的节点,从而在该节点上找到这个4M对象,读取出这个4M对象出来与写入数据合并,从而达到对镜像进行修改的目的。At present, in the consistent hash storage system, the mirror includes multiple objects of the same size, and these objects form a mirror according to certain rules. For example, a 1G mirror can include 256 4M objects, and these 256 4M objects Form this 1G mirror image according to certain rules. When the image is cloned and stored, the 256 4M objects are stored on the corresponding consistent hash nodes by calculating the hash values of the 256 4M objects. When it is necessary to read and write an object, the node where the 4M object is located is calculated by the hash value, so that the 4M object is found on the node, and the 4M object is read out and merged with the written data, so as to achieve The purpose of modifying the image.
一致性哈希算法是将节点分布在哈希环上,将节点间的数据存储到对应的节点中,从而实现数据分布式存储,当其中一个节点坏掉,则将该节点数据迁移到下一节点中进行存储,从而避免节点变化引起整个系统中所有数据的迁移。而在对克隆镜像进行读写时,如果需要读写的对象是基于源对象修改后的数据,有时会需要将源对象读取到客户端进行修改,然后将读取出来的源对象与修改后的数据进行合并得到新的对象,将新对象存储到新节点。可见,在现有技术一致性哈希存储系统中,存在读写路径长,克隆镜像性能差的问题。The consistent hash algorithm is to distribute the nodes on the hash ring, and store the data between the nodes in the corresponding nodes, so as to realize the distributed storage of data. When one of the nodes is broken, the data of the node will be migrated to the next It is stored in the node, so as to avoid the migration of all data in the entire system caused by node changes. When reading and writing the clone image, if the object to be read and written is based on the modified data of the source object, sometimes it is necessary to read the source object to the client for modification, and then compare the read source object with the modified data. The data is merged to obtain a new object, and the new object is stored in a new node. It can be seen that in the prior art consistent hash storage system, there are problems of long read and write paths and poor clone image performance.
发明内容Contents of the invention
本发明实施例提供一种提高克隆镜像性能的方法、装置、计算机设备及存储介质,旨在解决存在读写路径长,克隆镜像性能差的问题。Embodiments of the present invention provide a method, device, computer equipment, and storage medium for improving the performance of a clone image, aiming at solving the problems of long read and write paths and poor performance of the clone image.
为了解决上述问题,本发明实施例是这样实现的,提供一种提高克隆镜像性能的方法,包括步骤:In order to solve the above problems, the embodiment of the present invention is implemented as follows, providing a method for improving the performance of a clone image, including steps:
根据获取到的待读写对象,判断与所述待读写对象对应的镜像是否存在预先设置的全局镜像继承关系,所述全局镜像继承关系包括源镜像与克隆镜像的继承关系;According to the obtained object to be read and written, it is judged whether there is a preset global image inheritance relationship in the image corresponding to the object to be read and written, and the global image inheritance relationship includes the inheritance relationship between the source image and the clone image;
若存在所述全局镜像继承关系,则根据所述全局镜像继承关系获取与所述待读写对象对应的源对象的元数据,所述元数据包括用于计算所述待读写对象对应的源对象所在的一致性哈希节点的信息;If the global image inheritance relationship exists, then according to the global image inheritance relationship, the metadata of the source object corresponding to the object to be read and written is obtained, and the metadata includes the source object used to calculate the source object corresponding to the object to be read and written. Information about the consistent hash node where the object is located;
根据所述元数据计算一致性哈希值,得到所述待读写对象对应的源对象所在的一致性哈希节点;Calculate a consistent hash value according to the metadata, and obtain a consistent hash node where the source object corresponding to the object to be read and written is located;
在所述待读写对象对应的源对象所在的一致性哈希节点上进行读写。Read and write is performed on the consistent hash node where the source object corresponding to the object to be read and written is located.
本发明还提供一种提高克隆镜像性能的装置,包括:The present invention also provides a device for improving the performance of the clone image, including:
第一判断模块,用于根据获取到的待读写对象,判断与所述待读写对象对应的镜像是否存在预先设置的全局镜像继承关系,所述全局镜像继承关系包括源镜像与克隆镜像的继承关系;The first judging module is used to judge whether the image corresponding to the object to be read and written has a preset global image inheritance relationship according to the acquired object to be read and written, and the global image inheritance relationship includes the source image and the clone image. Inheritance relationship;
第一获取模块,用于若存在所述全局镜像继承关系,则根据所述全局镜像继承关系获取与所述待读写对象对应的源对象的元数据,所述元数据包括用于计算所述待读写对象对应的源对象所在的一致性哈希节点的信息;The first acquiring module is configured to acquire, according to the global image inheritance relationship, the metadata of the source object corresponding to the object to be read and written, if the global image inheritance relationship exists, and the metadata includes the metadata used to calculate the Information about the consistent hash node where the source object corresponding to the object to be read and written is located;
计算模块,用于根据所述元数据计算一致性哈希值,得到所述待读写对象对应的源对象所在的一致性哈希节点;A calculation module, configured to calculate a consistent hash value according to the metadata, and obtain a consistent hash node where the source object corresponding to the object to be read and written is located;
读写模块,用于在所述待读写对象对应的源对象所在的一致性哈希节点上进行读写。A read-write module, configured to read and write on the consistent hash node where the source object corresponding to the object to be read and written is located.
本发明还提供一种计算机设备,包括存储器和处理器,所述存储器中存储有计算机程序,所述处理器执行所述计算机程序时实现如权利要求一至七中任一项所述的提高克隆镜像性能的方法的步骤。The present invention also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the improved clone image according to any one of claims 1 to 7 is realized. The steps of the performance method.
本发明还提供一种计算机可读存储介质,其特征在于,所述计算机可读存储介质上存储有计算机程序,所述计算机程序被处理器执行时实现如权利要求一至七中任一项所述的提高克隆镜像性能的方法的步骤。The present invention also provides a computer-readable storage medium, which is characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program according to any one of claims 1 to 7 is realized. The steps of the method for improving the performance of the clone image.
本发明实现的有益效果:本发明由于通过给所述待读写对象对应的镜像预先设置所述全局镜像继承关系,基于所述全局镜像继承关系去获取所述待读写对象的对应源对象的元数据,把所述元数据作为作为一致性哈希算法的计算数据计算出一致性哈希值,从所述一致性哈希值中可以得到所述待读写对象对应的源对象所在的一致性哈希节点,所述待读写对象会在所述一致性哈希节点上进行读写。所以,基于预设的所述全局镜像继承关系,最终可以找到所述待读写对象的源对象所在的所述一致性哈希节点,并在所述一致性哈希节点进行读写,可以缩短读写路径,提高克隆镜像性能。The beneficial effects achieved by the present invention: the present invention obtains the corresponding source object of the object to be read and written based on the inheritance relationship of the global image by presetting the global image inheritance relationship for the image corresponding to the object to be read and written Metadata, using the metadata as calculation data of a consistent hash algorithm to calculate a consistent hash value, from which the consistency of the source object corresponding to the object to be read and written can be obtained. The consistent hash node, the object to be read and written will be read and written on the consistent hash node. Therefore, based on the preset inheritance relationship of the global image, the consistent hash node where the source object of the object to be read and written is located can be finally found, and read and write is performed on the consistent hash node, which can shorten the Read and write paths to improve the performance of cloning images.
附图说明Description of drawings
图1是本申请可以应用于其中的示例性系统架构图;FIG. 1 is an exemplary system architecture diagram to which the present application can be applied;
图2是本发明实施例提供的提高克隆镜像性能的方法的一个实施例的流程图;Fig. 2 is a flowchart of an embodiment of a method for improving the performance of a clone image provided by an embodiment of the present invention;
图2a是本发明实施例提供的提高克隆镜像性能的方法的哈希环结构示意图;Figure 2a is a schematic diagram of a hash ring structure of a method for improving the performance of a clone image provided by an embodiment of the present invention;
图2b是本发明实施例提供的提高克隆镜像性能的方法的另一种哈希环结构示意图;FIG. 2b is a schematic diagram of another hash ring structure of a method for improving the performance of a clone image provided by an embodiment of the present invention;
图2c是本发明实施例提供的提高克隆镜像性能的方法的另一种哈希环结构示意图;FIG. 2c is a schematic diagram of another hash ring structure of a method for improving the performance of a clone image provided by an embodiment of the present invention;
图3是本发明实施例提供的提高克隆镜像性能的方法的另一个实施例的流程图;FIG. 3 is a flowchart of another embodiment of a method for improving the performance of a clone image provided by an embodiment of the present invention;
图4是本发明实施例提供的提高克隆镜像性能的方法的另一个实施例的流程图;FIG. 4 is a flow chart of another embodiment of a method for improving the performance of a clone image provided by an embodiment of the present invention;
图5是图2中S204的一种具体实施方式的流程图;FIG. 5 is a flowchart of a specific implementation of S204 in FIG. 2;
图6是本发明实施例提供的提高克隆镜像性能的方法的另一个实施例的流程图;FIG. 6 is a flowchart of another embodiment of a method for improving the performance of a clone image provided by an embodiment of the present invention;
图7是本发明实施例提供的提高克隆镜像性能的方法的另一个实施例的流程图;FIG. 7 is a flowchart of another embodiment of a method for improving the performance of a clone image provided by an embodiment of the present invention;
图8是本发明实施例提供的提高克隆镜像性能的方法的另一个实施例的流程图;FIG. 8 is a flowchart of another embodiment of a method for improving the performance of a clone image provided by an embodiment of the present invention;
图9是本申请的提高克隆镜像性能的装置的一个实施例的结构示意图;FIG. 9 is a schematic structural diagram of an embodiment of a device for improving the performance of a clone image according to the present application;
图10是本申请的提高克隆镜像性能的装置的另一个实施例的结构示意图;Fig. 10 is a schematic structural diagram of another embodiment of the device for improving the performance of the clone image of the present application;
图11是本申请的提高克隆镜像性能的装置的另一个实施例的结构示意图;Fig. 11 is a schematic structural diagram of another embodiment of the device for improving the performance of the clone image of the present application;
图12是本申请的提高克隆镜像性能的装置的另一个实施例的结构示意图;Fig. 12 is a schematic structural diagram of another embodiment of the device for improving the performance of the clone image of the present application;
图13是本申请的提高克隆镜像性能的装置的另一个实施例的结构示意图;Fig. 13 is a schematic structural diagram of another embodiment of the device for improving the performance of the clone image of the present application;
图14是本申请的提高克隆镜像性能的装置的另一个实施例的结构示意图;Fig. 14 is a schematic structural diagram of another embodiment of the device for improving the performance of the clone image of the present application;
图15是本申请的计算机设备的一个实施例的结构示意图。Fig. 15 is a schematic structural diagram of an embodiment of a computer device of the present application.
具体实施方式Detailed ways
为了使本发明的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本发明进行进一步详细说明。应当理解,此处所描述的具体实施例仅仅用以解释本发明,并不用于限定本发明。In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.
本发明由于通过给待读写对象对应的镜像预先设置全局镜像继承关系,基于全局镜像继承关系去获取待读写对象的对应源对象的元数据,根据元数据计算一致性哈希值,得到待读写对象对应的源对象所在的一致性哈希节点,并在上述节点上进行读写。所以,可以缩短读写路径,提高克隆镜像性能。The present invention pre-sets the global image inheritance relationship for the image corresponding to the object to be read and written, obtains the metadata of the corresponding source object of the object to be read and written based on the inheritance relationship of the global image, calculates the consistent hash value according to the metadata, and obtains the Read and write the consistent hash node where the source object corresponding to the object is located, and read and write on the above node. Therefore, the read-write path can be shortened and the performance of the clone image can be improved.
如图1所示,系统架构100可以包括服务器105,网络102和终端设备101、102、103。网络104用以在服务器105和终端设备101、102、103之间提供通信链路的介质。网络104可以包括各种连接类型,例如有线、无线通信链路或者光纤电缆等。终端设备101、102、103可以是具有显示屏,可以下载应用软件,可以进行数据读写等的各种电子设备,包括但不限于智能手机、平板电脑、膝上型便携计算机和台式计算机等等,客户可以使用终端设备101、102、103通过网络104与服务器105交互,以接收或获取信息等。As shown in FIG. 1 , the system architecture 100 may include a server 105 , a network 102 and terminal devices 101 , 102 , 103 . The network 104 serves as a medium for providing communication links between the server 105 and the terminal devices 101 , 102 , 103 . Network 104 may include various connection types, such as wires, wireless communication links, or fiber optic cables, among others. The terminal devices 101, 102, and 103 can be various electronic devices that have a display screen, can download application software, and can perform data reading and writing, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc. , the client can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or obtain information and the like.
应该理解,图1中的移动终端、网络和设备的数目仅仅是示意性的,根据实现需要,可以具有任意数目的移动终端、网络和服务器。It should be understood that the numbers of mobile terminals, networks, and devices in FIG. 1 are only illustrative, and there may be any number of mobile terminals, networks, and servers according to implementation requirements.
如图2所示,为根据本申请的提高克隆镜像性能的方法所提供的一个实施例的流程图。上述的提高克隆镜像性能的方法,包括步骤:As shown in FIG. 2 , it is a flowchart of an embodiment provided by the method for improving the performance of a clone image according to the present application. The above-mentioned method for improving the performance of the clone image includes steps:
S201,根据获取到的待读写对象,判断与待读写对象对应的镜像是否存在预先设置的全局镜像继承关系,全局镜像继承关系包括源镜像与克隆镜像的继承关系。S201. According to the obtained object to be read and written, determine whether the image corresponding to the object to be read and written has a preset global image inheritance relationship, where the global image inheritance relationship includes an inheritance relationship between a source image and a clone image.
在本实施例中,提高克隆镜像性能的方法运行于其上的电子设备(例如图1所示的移动终端)。其中,上述的待读写对象可以是通过用户对待读写镜像进行操作时自行获取,例如:用户对一个待读写镜像进行读写操作,根据读写的位置获取到待读写对象在镜像中的位置。上述的待读写对象对应的镜像可以是文档文件、应用文件或者其他类型的文件,上述的待读写对象可以是文档文件、应用文件或者其他类型的文件中的对象。需要理解的是,镜像是一种文件存储形式,并且可以把多个文件做成一个镜像文件,上述的待读写对象为镜像中的一部分,对象为镜像的组成部分,一个镜像由多个大小一致的对象按照一定的规则组成,例如:一个1G的镜像,可以由256个4M对象按规则组成,上述的规则可以是顺序规则,即将256个对象按0-255进行排序关联,当要对镜像的12M556K进行读写时,也即是大于3个4M,小于5个4M,则是对4号对象进行读写。用于获取待读写对象的设备可以是手机、平板、计算机等具备数据读写能力的终端设备,上述的终端设备可以通过网络与镜像所在服务器进行交互。In this embodiment, the method for improving the performance of cloning and mirroring runs on an electronic device (for example, the mobile terminal shown in FIG. 1 ). Among them, the above-mentioned objects to be read and written can be obtained by the user when operating the image to be read and written. s position. The mirror images corresponding to the objects to be read and written may be document files, application files or other types of files, and the objects to be read and written may be objects in document files, application files or other types of files. What needs to be understood is that mirroring is a form of file storage, and multiple files can be made into a mirroring file. The above-mentioned objects to be read and written are part of the mirroring, and the objects are components of the mirroring. A mirroring consists of multiple sizes Consistent objects are composed according to certain rules. For example, a 1G mirror image can be composed of 256 4M objects according to rules. The above rules can be sequence rules, that is, 256 objects are sorted and associated according to 0-255. When the 12M556K reads and writes, that is, it is greater than 3 4Ms and less than 5 4Ms, it is to read and write the No. 4 object. The device used to obtain the object to be read and written can be a terminal device capable of reading and writing data, such as a mobile phone, a tablet, and a computer. The above-mentioned terminal device can interact with the server where the image is located through the network.
具体的,上述的全局镜像继承关系可以理解为全局镜像的父子关系,即一个源对象与多个子对象的关系,上述的继承关系还可以包括有克隆镜像的克隆路径,通过克隆路径可以找到克隆镜像的父镜像(源镜像),例如:镜像A′为源镜像A的克隆镜像,通过全局镜像继承关系,找到克隆镜像A′的源镜像A,全局镜像继承关系用于记录源镜像与克隆镜像的克隆关系。这样,缩短了对待读写对象进行读写的路径。Specifically, the above-mentioned global image inheritance relationship can be understood as the parent-child relationship of the global image, that is, the relationship between a source object and multiple child objects. The above-mentioned inheritance relationship can also include the clone path of the clone image, and the clone image can be found through the clone path The parent image (source image), for example: image A' is the clone image of the source image A, and the source image A of the clone image A' is found through the global image inheritance relationship, and the global image inheritance relationship is used to record the relationship between the source image and the clone image clone relationship. In this way, the path for reading and writing the object to be read and written is shortened.
S202,若存在全局镜像继承关系,则根据全局镜像继承关系获取与待读写对象对应的源对象的元数据,元数据包括用于计算待读写对象对应的源对象所在的一致性哈希节点的信息。S202. If there is a global mirror inheritance relationship, obtain the metadata of the source object corresponding to the object to be read and written according to the global mirror inheritance relationship, and the metadata includes the consistent hash node used to calculate the source object corresponding to the object to be read and written. Information.
在本实施例中,上述元数据可以是源镜像中与待读写对象相对应的对象名称,例如:待读写对象为克隆镜像中的A′0,对应在源镜像中的对象为A0,则待读写对象的元数据可以为A0-A′0的数组。上述元数据可以是包括A0与A′0的数组或键值关系,例如:对A′0进行读写,则会根据全局继承关系像找到A′0的源镜像为A,A′0的位移为0,则源镜像中对应的对象的位移也为0,则源对象为A0。对A0进行哈希计算,得到的是源对象A0的哈希值,通过这个哈希值,可以得到源对象所在的一致性哈希节点,即待读写对象与源对象A0共用一个哈希值,因而是通过元数据找到对应A0做为待读写对象的哈希值进行哈希,而不是待读写对象的A′0进行哈希。所以,能够通过元数据找到源对象所在的一致性哈希,避免用A′0得到其他的节点,造成读写时读写路径变长。In this embodiment, the above metadata may be the object name corresponding to the object to be read and written in the source image, for example: the object to be read and written is A'0 in the clone image, and the object corresponding to the source image is A0, Then the metadata of the object to be read and written can be an array of A0-A′0. The above metadata can be an array or a key-value relationship including A0 and A'0. For example, if you read and write A'0, you will find the source image of A'0 as A and the displacement of A'0 according to the global inheritance relationship. is 0, then the displacement of the corresponding object in the source image is also 0, and the source object is A0. Perform hash calculation on A0 to get the hash value of the source object A0. Through this hash value, the consistent hash node where the source object is located can be obtained, that is, the object to be read and written shares a hash value with the source object A0 , so the hash value corresponding to A0 is found through the metadata as the object to be read and written, rather than A'0 of the object to be read and written is hashed. Therefore, it is possible to find the consistent hash of the source object through the metadata, and avoid using A'0 to obtain other nodes, resulting in longer read and write paths when reading and writing.
具体的,如图2a所示,提供一种哈希环结构示意图,对于一致性哈希,可以理解为由N个点,比如232个点组成的哈希环,N表示组成哈希环的点的个数,则对应的哈希值可以是hash(A0)%N)。在哈希环上确定一个初始点,例如:设置O点为初始点,则在O点右侧第一个点代表1,左侧第一个点代表232-1。如图2b所示,假设有3台用于存储对象的存储服务器,分别为服务器A、服务器B、服务器C,那么,这3台服务器会有自己的IP地址,对它们各自的IP地址进行哈希计算,即hash(服务器的IP地址)%232,根据计算得到的值,对应于哈希环上的值,将会得到服务器A、B、C映射在哈希环上的位置,可以是按顺时针顺序分布。之后可以是将需要存储的对象进行哈希计算,可以用对象的名称做为键值,即hash(对象名称)%232,根据计算得到的值,对应于哈希环上的值,将对象映射至哈希环上,如图2c所示。例如:计算得到的值分别为对象1、2、3、4,则对象1、2、3、4与服务器A、B、C共同映射在哈希环上。若哈希环上的对象1、2位于服务器A与服务器C之间,按照顺时针存储,则对象1、2会被存储在服务器A所在节点上,对象3若位于服务器A与服务器B之间,则会被存储在服务器B所在节点上,对象4若位于服务器B与服务器C之间,会被存储在服务器C所在节点上。这样,通过全局镜像继承关系,对元数据进行一致性哈希计算,缩短了对待读写对象进行读写的路径。Specifically, as shown in Figure 2a, a schematic diagram of a hash ring structure is provided. For consistent hashing, it can be understood as a hash ring composed of N points, such as 2-32 points, and N represents the number of points that make up the hash ring. The number of points, the corresponding hash value can be hash(A0)%N). Determine an initial point on the hash ring, for example: set point O as the initial point, then the first point on the right side of point O represents 1, and the first point on the left represents 2 32 -1. As shown in Figure 2b, suppose there are three storage servers used to store objects, namely server A, server B, and server C. Then, these three servers will have their own IP addresses, and their respective IP addresses will be hashed. Greek calculation, that is, hash (IP address of the server) % 2 32 , according to the calculated value, corresponding to the value on the hash ring, will get the positions of servers A, B, and C mapped on the hash ring, which can be Distributed in clockwise order. Afterwards, hash calculation can be performed on the object to be stored, and the name of the object can be used as the key value, that is, hash (object name) % 2 32 . According to the calculated value, which corresponds to the value on the hash ring, the object Mapped to the hash ring, as shown in Figure 2c. For example: the calculated values are objects 1, 2, 3, and 4 respectively, then objects 1, 2, 3, and 4 are mapped on the hash ring together with servers A, B, and C. If objects 1 and 2 on the hash ring are located between server A and server C and stored clockwise, objects 1 and 2 will be stored on the node where server A is located. If object 3 is located between server A and server B , it will be stored on the node where server B is located, and if object 4 is located between server B and server C, it will be stored on the node where server C is located. In this way, through the global mirror inheritance relationship, consistent hash calculation is performed on metadata, which shortens the path for reading and writing objects to be read and written.
S203,根据元数据计算一致性哈希值,得到待读写对象对应的源对象所在的一致性哈希节点。S203. Calculate the consistent hash value according to the metadata, and obtain the consistent hash node where the source object corresponding to the object to be read and written is located.
其中,一致性哈希值是根据一致性哈希算法得到,一致性哈希算法是一种取模算法,即是对值进行取余数的操作,从而确保哈希值是一个整数,比如:存在3个节点,将对象0、对象1、对象2分别存入这3个节点中,则可以对对象0进行取模得到的余数为0,对对象1进行取模得到的余数为1,对对象2进行取模得到的余数为2,则将对象0存入1号节点,对象1存入2号节点,对象2存入3号节点,若存在对象3,取模得到余数为0,则存入1号节点。Among them, the consistent hash value is obtained according to the consistent hash algorithm. The consistent hash algorithm is a modulo algorithm, that is, the operation of taking the remainder of the value, so as to ensure that the hash value is an integer, for example: there is There are 3 nodes, and object 0, object 1, and object 2 are respectively stored in these 3 nodes, then the remainder obtained by taking the modulus of object 0 is 0, and the remainder obtained by taking the modulo of object 1 is 1, and the remainder of object 1 is 1. 2 If the remainder obtained by taking the modulo is 2, store object 0 in node 1, store object 1 in node 2, and store object 2 in node 3. If there is object 3, the remainder obtained by modulo is 0, then store Enter node 1.
具体的,通过一致性哈希算法对元数据进行计算,由于元数据包括用于计算源对象所在的一致性哈希节点的信息,即通过元数据进行一致性哈希计算所得到的值,即是源对象的所在节点的值,从而得到源对象所在的一致性哈希节点。可以理解的是,待读写对象可以是克隆对象,而克隆对象的元数据可以为源对象的键,源对象的名称可以为值。这样,在对源对象进行存储时,是通过源对象的值进行哈希,而根据待读写对象的元数据进行哈希,则是根据克隆对象的元数据(源对象的键),查询到与源对象的键对应的值,也就是通过源对象的值进行哈希的。对相同的值进行哈希,会得到相同的哈希值,从而得到待读写对象的源对象所在的节点。Specifically, the metadata is calculated through the consistent hash algorithm. Since the metadata includes the information used to calculate the consistent hash node where the source object is located, that is, the value obtained through the consistent hash calculation of the metadata, that is is the value of the node where the source object is located, so as to obtain the consistent hash node where the source object is located. It can be understood that the object to be read and written can be a cloned object, and the metadata of the cloned object can be the key of the source object, and the name of the source object can be the value. In this way, when storing the source object, the value of the source object is used for hashing, and the hashing is performed according to the metadata of the object to be read and written. The value corresponding to the key of the source object, that is, hashed by the value of the source object. Hash the same value to get the same hash value, so as to get the node where the source object of the object to be read and written is located.
S204,在待读写对象对应的源对象所在的一致性哈希节点上进行读写。S204, read and write on the consistent hash node where the source object corresponding to the object to be read and written is located.
在本发明实施例中,源对象的节点是通过一致性哈希算法计算后得到的,通过待读写对象的元数据,使用户访问的读写节点为源对象的存储节点,在该节点上进行读写,可以确定该节点必然存在源对象的数据,避免了将源对象的数据进行迁移才能完成读写。In the embodiment of the present invention, the node of the source object is calculated by a consistent hash algorithm. Through the metadata of the object to be read and written, the read-write node accessed by the user is the storage node of the source object. On this node To read and write, it can be determined that the node must have the data of the source object, and the data of the source object cannot be migrated to complete the read and write.
本发明由于通过给待读写对象对应的镜像预先设置全局镜像继承关系,基于全局镜像继承关系去获取待读写对象的对应源对象的元数据,把元数据作为作为一致性哈希算法的计算数据计算出一致性哈希值,从一致性哈希值中可以得到待读写对象对应的源对象所在的一致性哈希节点,那么,待读写对象便会在一致性哈希节点上进行读写。所以,基于预设的全局镜像继承关系,最终可以找到待读写对象的源对象所在的一致性哈希节点,并在一致性哈希节点进行读写,可以缩短读写路径,提高克隆镜像性能。The present invention pre-sets the global image inheritance relationship for the image corresponding to the object to be read and written, obtains the metadata of the corresponding source object of the object to be read and written based on the inheritance relationship of the global image, and uses the metadata as the calculation of the consistent hash algorithm The consistent hash value is calculated from the data, and the consistent hash node where the source object corresponding to the object to be read and written is located can be obtained from the consistent hash value. Then, the object to be read and written will be processed on the consistent hash node. read and write. Therefore, based on the preset global image inheritance relationship, you can finally find the consistent hash node where the source object of the object to be read and written is located, and read and write on the consistent hash node, which can shorten the read and write path and improve the performance of the clone image .
进一步地,如图3所示,在获取待读写对象之后,上述方法还包括步骤:Further, as shown in FIG. 3, after obtaining the object to be read and written, the above method further includes steps:
S301,对源镜像对应的数据进行设置;S301, setting the data corresponding to the source image;
S302,将设置后的源镜像对应的数据通过克隆生成克隆镜像;S302, generating a clone image by cloning the data corresponding to the set source image;
S303,记录源镜像与克隆镜像的形成路径,基于路径,使源镜像与克隆镜像形成全局镜像继承关系。S303. Record the formation paths of the source image and the clone image, and make the source image and the clone image form a global image inheritance relationship based on the paths.
具体的,设置全局镜像继承关系,具体可以是对源镜像进行快照,使源镜像中的数据不可修改,根据源镜像的数据,克隆出一个或者多个克隆镜像,并记录该克隆镜像与源镜像的克隆路径,即上述克隆镜像克隆来自哪一个源对象,例如:源镜像A的克隆镜像有Aa,Ab,则记录Aa,Ab克隆来自源镜像A,形成源镜像A的全局镜像继承关系。Specifically, set the inheritance relationship of the global image. Specifically, it can take a snapshot of the source image so that the data in the source image cannot be modified. According to the data of the source image, one or more clone images are cloned, and the clone image and the source image are recorded. The cloning path, that is, which source object the above clone image is cloned from. For example, if the clone image of source image A has Aa and Ab, record that Aa and Ab are cloned from source image A, forming the global image inheritance relationship of source image A.
在一种可能的实现的实施例方式中,全局镜像继承关系可以记录在一个表单中,通过上述表单可以查询到源镜像与克隆镜像的继承关系;全局镜像继承关系还可以记录在关系树中,通过关系树可以查询到克隆镜像的根节点。In a possible implementation embodiment, the global image inheritance relationship can be recorded in a form, and the inheritance relationship between the source image and the clone image can be queried through the above form; the global image inheritance relationship can also be recorded in the relationship tree, The root node of the clone image can be queried through the relationship tree.
这样,通过对源镜像进行克隆生成克隆镜像,然后记录生成克隆镜像的路克隆径,有利于源镜像与克隆镜像形成全局镜像关系。In this way, by cloning the source image to generate a clone image, and then recording the clone path for generating the clone image, it is beneficial to form a global image relationship between the source image and the clone image.
进一步地,如图4所示,上述方法还包括步骤:Further, as shown in Figure 4, the above method also includes steps:
S401,基于进行设置后的全局镜像继承关系,设置让克隆镜像中的克隆对象与对应的源镜像的源对象相关联的元数据;S401. Based on the set global image inheritance relationship, set metadata for associating the clone object in the clone image with the source object of the corresponding source image;
S402,通过元数据获取与克隆对象对应的源对象,其中,源对象作为一致性哈希进行计算的数据;S402. Obtain a source object corresponding to the cloned object through metadata, where the source object is calculated as a consistent hash;
S403,将克隆对象存储在与克隆对象对应的源对象所计算出的一致性哈希节点上。S403. Store the clone object on the consistent hash node calculated by the source object corresponding to the clone object.
在本发明实施例中,上述的元数据可以是描述克隆对象与源对象关系的数组或键值,例如:源对象的名称为A001,克隆对象的名称可以是B001,其中,A、B分别可以是源镜像及克隆镜像的名称,001可以是镜像中第一个对象的位移值,A001表示上述镜像中的第一个对象,B001克隆则表示克隆镜像中的第一个克隆对象。可以将克隆对象的名称与源对象的名称进行索引关联,例如:通过克隆对象的名称“A′0”索引到源对象的名称“A0”,在对克隆对象进行查找时,可以索引到“A0”,对“A0”进行一致性哈希,从而找到克隆对象的源对象所在节点。In the embodiment of the present invention, the above-mentioned metadata may be an array or a key value describing the relationship between the cloned object and the source object, for example: the name of the source object is A001, the name of the cloned object may be B001, where A and B may be respectively is the name of the source image and the clone image, 001 can be the displacement value of the first object in the image, A001 indicates the first object in the above image, and B001 clone indicates the first cloned object in the clone image. The name of the cloned object can be indexed with the name of the source object, for example: the name "A'0" of the cloned object is indexed to the name "A0" of the source object, and when the cloned object is searched, it can be indexed to "A0 ", perform a consistent hash on "A0", so as to find the node where the source object of the cloned object is located.
具体的,为了使克隆对象与源对象位于同一个节点,可以通过元数据得到源对象的名称,使用源对象的名称做为一致性哈希计算的名称,即克隆对象的一致性哈希值为hash(源对象的名称)%N。应用一致性哈希对克隆对象进行存储可以理解为,对克隆镜像中的克隆对象的元数据应用一致性哈希算法(hash(克隆对象的名称)%N),计算得到该克隆对象的一致性哈希值,根据一致性哈希值,将克隆对象存储在对应的位置节点上,位置节点可以理解为对应的存储器或服务器,在哈希环上,可以存在多个位置节点,形成存储器集群或服务器集群。Specifically, in order to make the cloned object and the source object on the same node, the name of the source object can be obtained through the metadata, and the name of the source object is used as the name of the consistent hash calculation, that is, the consistent hash value of the cloned object is hash(name of source object) %N. Applying consistent hashing to store cloned objects can be understood as applying a consistent hashing algorithm (hash(name of the cloned object)%N) to the metadata of the cloned object in the cloned image to calculate the consistency of the cloned object Hash value, according to the consistent hash value, store the cloned object on the corresponding location node. The location node can be understood as the corresponding storage or server. On the hash ring, there can be multiple location nodes to form a storage cluster or Server cluster.
需要理解的是,在一致性哈希算法中,哈希同一个键,得到的是相同的值,哈希不同的键,可能得到不同的值,例如:对“A0”进行一致性哈希得到的值,不一定会等于对“A0克隆”、“A1”或者“A0克隆”进行一致性哈希得到的值,N越大,得到相同值的概率越小。What needs to be understood is that in the consistent hashing algorithm, if you hash the same key, you get the same value, and if you hash different keys, you may get different values, for example: "A0" is hashed consistently to get The value of is not necessarily equal to the value obtained by consistent hashing of "A0 clone", "A1" or "A0 clone". The larger N is, the lower the probability of getting the same value.
这样,在查找克隆对象时,通过元数据获取源对象进行一致性哈希计算得到所存储的一致性哈希节点,便可以查找到克隆对象所存储的节点,不需要到另外的节点进行查找,进一步缩短了读写路径,提高了克隆镜像的性能。In this way, when searching for a cloned object, obtain the source object through the metadata and perform consistent hash calculation to obtain the stored consistent hash node, and then you can find the node where the cloned object is stored without searching for another node. The read-write path is further shortened, and the performance of the clone image is improved.
进一步地,如图5所示,上述S204的步骤具体包括:Further, as shown in FIG. 5, the above steps of S204 specifically include:
S501,在待读写对象的源对象对应的一致性哈希节点上写入数据时,采用增量写入的方式进行数据写入,索引待读写对象的源对象对应的数据进行增补生成新对象。S501. When writing data on the consistent hash node corresponding to the source object of the object to be read and written, the data is written in an incremental writing manner, and the data corresponding to the source object of the index object to be read and written is supplemented to generate a new object.
在本发明实施例中,需要对待读写对象进行写入时,在源对象的节点上进行增量写入,即只写入需要写入的数据,不用将源对象中的数据复制出来再写入,例如:源对象的数据为4M,需要写入4K数据,则只需要写入4K到该节点上,而不用复制写入4M到该节点上,其余的数据以索引的形式从源对象的数据中索引到新的对象中与4K数据形成新对象。In the embodiment of the present invention, when it is necessary to write to the object to be read and written, incremental writing is performed on the node of the source object, that is, only the data to be written is written, and there is no need to copy the data in the source object for rewriting For example, if the data of the source object is 4M, if you need to write 4K data, you only need to write 4K to the node without copying and writing 4M to the node, and the rest of the data will be indexed from the source object The data is indexed into a new object to form a new object with the 4K data.
这样,通过直接写入所需要写的数据,而不用将源对象的数据复制写入,可以避免写入放大。In this way, write amplification can be avoided by directly writing the data to be written without copying and writing the data of the source object.
进一步地,如图6所示,上述方法还包括:Further, as shown in Figure 6, the above method also includes:
S601,判断新对象中是否存在基于全局镜像继承关系的元数据;S601, judging whether there is metadata based on the global image inheritance relationship in the new object;
S602,若新对象中存在基于全局镜像继承关系的元数据,则根据全局镜像继承关系的元数据对新对象进行一致性哈希存储。S602. If metadata based on the inheritance relationship of the Global Image exists in the new object, perform consistent hash storage on the new object according to the metadata of the inheritance relationship of the Global Image.
在本发明实施例中,若待读写对象本身存在元数据,则在对待读写对象进行数据写入时,保持待读写对象原来的元数据不变,从而得到一个包括有原来元数据的新对象,则新对象中可以是存在基于全局镜像继承关系的元数据,利用该元数据对这个新对象进行一致性哈希计算,得到所需要存储的一致性哈希节点并进行存储。In the embodiment of the present invention, if there is metadata in the object to be read and written, then when writing data to the object to be read and written, keep the original metadata of the object to be read and written unchanged, so as to obtain a file containing the original metadata. new object, the new object may contain metadata based on the inheritance relationship of the global mirror, and use the metadata to perform consistent hash calculation on the new object to obtain and store the consistent hash nodes that need to be stored.
这样,当新对象中存在基于全局镜像继承关系的待读写对象本身存在的元数据时,通过直接对该元数据进行计算便可以得到新对象所存储的节点,可以缩短读写的路径。In this way, when the metadata of the object to be read and written based on the global mirror inheritance relationship exists in the new object, the node where the new object is stored can be obtained by directly calculating the metadata, which can shorten the path of reading and writing.
进一步地,如图7所示,上述方法还包括:Further, as shown in Figure 7, the above method also includes:
S701,基于全局镜像继承关系,给新对象设置对应的元数据;S701. Based on the global image inheritance relationship, set corresponding metadata for the new object;
S702,基于新对象对应的元数据,通过一致性哈希对新对象进行存储,其中,元数据包括用于计算源对象所在的一致性哈希节点的信息。S702. Based on metadata corresponding to the new object, store the new object through a consistent hash, where the metadata includes information used to calculate the consistent hash node where the source object is located.
在本发明实施例中,可以将上述的新对象存储在与源对象同一节点中,及新对象与源对象之间存在全局镜像继承关系,这样,在对新对象进行读写时,可以通过新对象的元数据对源对象进行一致性哈希计算,得到源对象所存储的节点,并将新对象也存储在该节点上。In the embodiment of the present invention, the above-mentioned new object can be stored in the same node as the source object, and there is a global mirror inheritance relationship between the new object and the source object. The metadata of the object performs consistent hash calculation on the source object, obtains the node where the source object is stored, and stores the new object on this node.
这样,可以避免新对象因为迁移到其他节点上,而又需要到源对象所在节点读取源对象的数据,从而进一步导致读写路径变长。In this way, it can avoid the need to read the data of the source object from the node where the source object is located because the new object is migrated to other nodes, which will further cause the read-write path to become longer.
进一步地,如图8所示,上述方法还包括:Further, as shown in Figure 8, the above method also includes:
S801,在生成新对象后,索引源镜像的对象与新对象形成新克隆镜像;S801. After the new object is generated, the object of the index source image and the new object form a new clone image;
S802,为新克隆镜像中的对象设置对应的元数据,基于新克隆镜像中的对象设置对应的元数据,应用一致性哈希对新克隆镜像中的对象进行存储;S802. Set corresponding metadata for the objects in the new clone image, set the corresponding metadata based on the objects in the new clone image, and store the objects in the new clone image by applying a consistent hash;
S803,将新克隆镜像写入全局镜像继承关系中,其中,新克隆镜像中的对象的元数据包括与元数据所在的一致性哈希节点信息。S803. Write the new clone image into the global image inheritance relationship, wherein the metadata of the object in the new clone image includes consistent hash node information where the metadata is located.
在本发明实施例中,基于新对象,索引源镜像中其他对象与新对象形成新克隆镜像,需要说明的是,新克隆镜像中可以包括写入了数据的新对象与索引对象,写入了数据的新对象可以是包括写入数据、索引数据以及元数据,其中,写入数据可以是用户写入的数据,索引数据可以是索引到源对象数据的数据,元数据指的是描述新对象属性的数据,元数据包括源对象的一致性哈希节点信息,索引对象可以理解为源对象中的其他对象,源对象中的其他对象与写入数据的新对象组成了新克隆对象。In this embodiment of the present invention, based on the new object, other objects in the index source image and the new object form a new clone image. The new object of data can include written data, index data and metadata, where the written data can be the data written by the user, the index data can be the data indexed to the source object data, and the metadata refers to the description of the new object The attribute data and metadata include the consistent hash node information of the source object. The index object can be understood as other objects in the source object. The other objects in the source object and the new object to which the data is written form a new clone object.
这样,根据新对象,索引源镜像中其他对象与新对象形成新克隆镜像,从而可以实现对前端业务与后端的数据的同步,并且通过对上述元数据进行一致性哈希计算,可以实现对新克隆镜像中对象的存储。In this way, according to the new object, other objects in the index source image and the new object form a new clone image, so that the front-end business and the back-end data can be synchronized, and by performing consistent hash calculation on the above metadata, the new Clone storage for objects in mirrors.
通过对一致性哈希存储系统中的镜像设置一致性哈希算法得到元数据,在需要对克隆镜像进修改读写时,通过元数据计算出源镜像中源对象在哈希环上所处的位置直接进行读写。通过元数据将克隆对象与源对象进行关联,可以利用源对象相关的元数据计算出源对象的位置,并将克隆对象存储在源对象的节点上。在写入数据时,也无需从源对象的节点中读出源对象进行写操作,而是直接写入源对象的节点中。这样,可以避免读写路径变长,提高克隆镜像的性能,同时,还能避免数据的迁移以及写入放大的问题。The metadata is obtained by setting the consistent hash algorithm for the image in the consistent hash storage system. When it is necessary to modify and read the clone image, the location of the source object in the source image on the hash ring is calculated through the metadata. The location is read and written directly. By associating the clone object with the source object through metadata, the location of the source object can be calculated by using the metadata related to the source object, and the clone object is stored on the node of the source object. When writing data, there is no need to read the source object from the node of the source object to perform a write operation, but directly write to the node of the source object. In this way, the length of the read-write path can be avoided, and the performance of the clone mirror can be improved. At the same time, the problems of data migration and write amplification can also be avoided.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程,是可以通过计算机程序来指令相关的硬件来完成,该计算机程序可存储于一计算机可读取存储介质中,该程序在执行时,可包括如上述各方法的实施例的流程。其中,前述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory,ROM)等非易失性存储介质,或随机存储记忆体(Random Access Memory,RAM)等。Those of ordinary skill in the art can understand that realizing all or part of the processes in the methods of the above embodiments can be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. During execution, it may include the processes of the embodiments of the above-mentioned methods. Wherein, the aforementioned storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM).
应该理解的是,虽然附图的流程图中的各个步骤按照箭头的指示依次显示,但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明,这些步骤的执行并没有严格的顺序限制,其可以以其他的顺序执行。而且,附图的流程图中的至少一部分步骤可以包括多个子步骤或者多个阶段,这些子步骤或者阶段并不必然是在同一时刻执行完成,而是可以在不同的时刻执行,其执行顺序也不必然是依次进行,而是可以与其他步骤或者其他步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。It should be understood that although the various steps in the flow chart of the accompanying drawings are displayed sequentially according to the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and the order of execution is also It is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.
如图9所示,为本实施例所提供的提高克隆镜像性能的装置的示意图,上述装置900包括:第一判断模块901、获取模块902、计算模块903、读写模块904。其中:As shown in FIG. 9 , which is a schematic diagram of the device for improving the performance of clone image provided by this embodiment, the above-mentioned device 900 includes: a first judgment module 901 , an acquisition module 902 , a calculation module 903 , and a read-write module 904 . in:
第一判断模块901,用于根据获取到的待读写对象,判断与待读写对象对应的镜像是否存在预先设置的全局镜像继承关系,全局镜像继承关系包括源镜像与克隆镜像的继承关系;The first judging module 901 is used to judge whether the image corresponding to the object to be read and written has a preset global image inheritance relationship according to the obtained object to be read and written, and the global image inheritance relationship includes the inheritance relationship between the source image and the clone image;
第一获取模块902,用于若存在全局镜像继承关系,则根据全局镜像继承关系获取与待读写对象对应的源对象的元数据,元数据包括用于计算待读写对象对应的源对象所在的一致性哈希节点的信息;The first obtaining module 902 is used to obtain the metadata of the source object corresponding to the object to be read and written according to the inheritance relationship of the global image if there is a global image inheritance relationship. The information of the consistent hash node;
计算模块903,用于根据元数据计算一致性哈希值,得到待读写对象对应的源对象所在的一致性哈希节点;The calculation module 903 is configured to calculate a consistent hash value according to the metadata, and obtain a consistent hash node where the source object corresponding to the object to be read and written is located;
读写模块904,用于在待读写对象对应的源对象所在的一致性哈希节点上进行读写。The read-write module 904 is configured to read and write on the consistent hash node where the source object corresponding to the object to be read and written is located.
进一步地,如图10所示,上述装置900还包括:第一设置模块905、生成模块906、记录模块907。其中,Further, as shown in FIG. 10 , the above device 900 further includes: a first setting module 905 , a generating module 906 , and a recording module 907 . in,
第一设置模块905,用于对源镜像对应的数据进行设置;The first setting module 905 is configured to set the data corresponding to the source image;
生成模块906,用于将设置后的源镜像对应的数据通过克隆生成克隆镜像;A generation module 906, configured to generate a clone image by cloning the data corresponding to the set source image;
记录模块907,用于记录源镜像与克隆镜像的形成路径,基于路径,使源镜像与克隆镜像形成全局镜像继承关系。The recording module 907 is configured to record the formation paths of the source image and the clone image, and make the source image and the clone image form a global image inheritance relationship based on the paths.
进一步地,如图11所示,上述装置900还包括:第二设置模块908、第二获取模块909、第一存储模块910。其中,Further, as shown in FIG. 11 , the above device 900 further includes: a second setting module 908 , a second obtaining module 909 , and a first storage module 910 . in,
第二设置模块908,用于基于进行设置后的全局镜像继承关系,设置让克隆镜像中的克隆对象与对应的源镜像的源对象相关联的元数据;The second setting module 908 is configured to set metadata that associates the cloned object in the cloned image with the source object of the corresponding source image based on the set global image inheritance relationship;
第二获取模块909,用于通过元数据获取与克隆对象对应的源对象,其中,源对象作为一致性哈希进行计算的数据;The second acquiring module 909 is configured to acquire the source object corresponding to the clone object through the metadata, wherein the source object is calculated as a consistent hash;
第一存储模块910,用于将克隆对象存储在与克隆对象对应的源对象所计算出的一致性哈希节点上。The first storage module 910 is configured to store the clone object on the consistent hash node calculated by the source object corresponding to the clone object.
进一步地,上述读写模块904,还用于在待读写对象的源对象对应的一致性哈希节点上写入数据时,采用增量写入的方式进行数据写入,索引待读写对象的源对象对应的数据进行增补生成新对象。Further, the above-mentioned read-write module 904 is also used for writing data on the consistent hash node corresponding to the source object of the object to be read and written, using incremental writing to write data, indexing the object to be read and written The data corresponding to the source object is supplemented to generate a new object.
进一步地,如图12所示,上述装置900还包括:第二判断模块911、第二存储模块912。其中,Further, as shown in FIG. 12 , the above device 900 further includes: a second judgment module 911 and a second storage module 912 . in,
第二判断模块911,用于判断新对象中是否存在基于全局镜像继承关系的元数据;The second judging module 911 is used to judge whether there is metadata based on the global image inheritance relationship in the new object;
第二存储模块912,用于若新对象中存在基于全局镜像继承关系的元数据,则根据全局镜像继承关系的元数据对新对象进行一致性哈希存储。The second storage module 912 is configured to perform consistent hash storage on the new object according to the metadata of the Global Image inheritance relationship if there is metadata based on the Global Image inheritance relationship in the new object.
进一步地,如图13所示,上述装置900还包括:第三设置模块913、第三存储模块914。其中,Further, as shown in FIG. 13 , the above device 900 further includes: a third setting module 913 and a third storage module 914 . in,
第三设置模块913,用于基于全局镜像继承关系,给新对象设置对应的元数据;The third setting module 913 is configured to set corresponding metadata for the new object based on the global image inheritance relationship;
第三存储模块914,用于基于新对象对应的元数据,通过一致性哈希对新对象进行存储,其中,元数据包括用于计算源对象所在的一致性哈希节点的信息。The third storage module 914 is configured to store the new object through a consistent hash based on metadata corresponding to the new object, where the metadata includes information used to calculate the consistent hash node where the source object is located.
进一步地,如图14所示,上述装置900还包括:索引模块915、第四存储模块916、写入模块917。其中,Further, as shown in FIG. 14 , the above device 900 further includes: an indexing module 915 , a fourth storage module 916 , and a writing module 917 . in,
索引模块915,用于在生成新对象后,索引源镜像的对象与新对象形成新克隆镜像;An indexing module 915, configured to form a new clone mirror image between the object mirrored by the index source and the new object after the new object is generated;
第四存储模块916,用于为新克隆镜像中的对象设置对应的元数据,基于新克隆镜像中的对象设置对应的元数据,应用一致性哈希对新克隆镜像中的对象进行存储;The fourth storage module 916 is configured to set corresponding metadata for the objects in the new clone image, set the corresponding metadata based on the objects in the new clone image, and store the objects in the new clone image by applying a consistent hash;
写入模块917,用于将新克隆镜像写入全局镜像继承关系中,其中,新克隆镜像中的对象的元数据包括与元数据所在的一致性哈希节点信息。The writing module 917 is configured to write the new clone image into the global image inheritance relationship, wherein, the metadata of the object in the new clone image includes consistent hash node information with the metadata.
本申请实施例提供的提高克隆镜像性能的装置能够实现图2至图8的方法实施例中的各个实施方式,以及相应有益效果,为避免重复,这里不再赘述。The device for improving the performance of clone mirroring provided by the embodiment of the present application can realize various implementations in the method embodiments in FIG. 2 to FIG. 8 and corresponding beneficial effects. To avoid repetition, details are not repeated here.
为解决上述技术问题,本申请实施例还提供计算机设备。具体请参阅图15,图15为本实施例计算机设备基本结构框图。In order to solve the above technical problems, the embodiment of the present application further provides computer equipment. Please refer to FIG. 15 for details. FIG. 15 is a block diagram of the basic structure of the computer device in this embodiment.
计算机设备15包括通过系统总线相互通信连接存储器151、处理器152、网络接口153。需要指出的是,图中仅示出了具有组件151-153的计算机设备15,但是应理解的是,并不要求实施所有示出的组件,可以替代的实施更多或者更少的组件。其中,本技术领域技术人员可以理解,这里的计算机设备是一种能够按照事先设定或存储的指令,自动进行数值计算和/或信息处理的设备,其硬件包括但不限于微处理器、专用集成电路(ApplicationSpecific Integrated Circuit,ASIC)、可编程门阵列(Field-Programmable GateArray,FPGA)、数字处理器(Digital Signal Processor,DSP)、嵌入式设备等。The computer device 15 includes a memory 151 , a processor 152 , and a network interface 153 that are connected to each other through a system bus. It should be noted that only computer device 15 is shown having components 151-153, but it should be understood that implementation of all of the illustrated components is not required and that more or fewer components may instead be implemented. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculation and/or information processing according to preset or stored instructions, and its hardware includes but is not limited to microprocessors, dedicated Integrated circuit (Application Specific Integrated Circuit, ASIC), programmable gate array (Field-Programmable GateArray, FPGA), digital processor (Digital Signal Processor, DSP), embedded devices, etc.
计算机设备可以是桌上型计算机、笔记本、掌上电脑及云端服务器等计算设备。计算机设备可以与客户通过键盘、鼠标、遥控器、触摸板或声控设备等方式进行人机交互。The computer equipment may be computing equipment such as desktop computers, notebooks, palmtop computers, and cloud servers. Computer equipment can interact with customers through keyboards, mice, remote controls, touch pads or voice-activated devices.
存储器151至少包括一种类型的可读存储介质,可读存储介质包括闪存、硬盘、多媒体卡、卡型存储器(例如,SD或DX存储器等)、随机访问存储器(RAM)、静态随机访问存储器(SRAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、磁盘、光盘等。在一些实施例中,存储器151可以是计算机设备15的内部存储单元,例如该计算机设备15的硬盘或内存。在另一些实施例中,存储器151也可以是计算机设备15的外部存储设备,例如该计算机设备15上配备的插接式硬盘,智能存储卡(Smart Media Card,SMC),安全数字(Secure Digital,SD)卡,闪存卡(Flash Card)等。当然,存储器151还可以既包括计算机设备15的内部存储单元也包括其外部存储设备。本实施例中,存储器151通常用于存储安装于计算机设备15的操作系统和各类应用软件,例如提高克隆镜像性能的方法的程序代码等。此外,存储器151还可以用于暂时地存储已经输出或者将要输出的各类数据。The memory 151 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory ( SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage 151 may be an internal storage unit of the computer device 15 , such as a hard disk or internal memory of the computer device 15 . In some other embodiments, the memory 151 can also be an external storage device of the computer device 15, such as a plug-in hard disk equipped on the computer device 15, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc. Of course, the storage 151 may also include both the internal storage unit of the computer device 15 and its external storage device. In this embodiment, the memory 151 is generally used to store the operating system and various application software installed on the computer device 15, such as program codes of a method for improving the performance of the clone image, and the like. In addition, the memory 151 can also be used to temporarily store various types of data that have been output or will be output.
处理器152在一些实施例中可以是中央处理器(Central Processing Unit,CPU)、控制器、微控制器、微处理器、或其他数据处理芯片。该处理器152通常用于控制计算机设备15的总体操作。本实施例中,处理器152用于运行存储器151中存储的程序代码或者处理数据,例如运行提高克隆镜像性能的方法的程序代码。The processor 152 may be a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 152 is generally used to control the overall operation of the computer device 15 . In this embodiment, the processor 152 is configured to run program codes stored in the memory 151 or process data, for example, run program codes of a method for improving performance of a clone image.
网络接口153可包括无线网络接口或有线网络接口,该网络接口153通常用于在计算机设备15与其他电子设备之间建立通信连接。The network interface 153 may include a wireless network interface or a wired network interface, and the network interface 153 is generally used to establish a communication connection between the computer device 15 and other electronic devices.
本申请还提供了另一种实施方式,即提供一种计算机可读存储介质,计算机可读存储介质存储有提高克隆镜像性能的程序,上述提高克隆镜像性能的程序可被至少一个处理器执行,以使至少一个处理器执行如上述的提高克隆镜像性能的方法的步骤。The present application also provides another implementation manner, which is to provide a computer-readable storage medium, the computer-readable storage medium stores a program for improving the performance of the clone image, and the above-mentioned program for improving the performance of the clone image can be executed by at least one processor, To make at least one processor execute the steps of the method for improving the performance of the clone image as described above.
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件,但很多情况下前者是更佳的实施方式。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中,包括若干指令用以使得一台终端设备(可以是手机,计算机,服务器,空调器,或者网络设备等)执行本申请各个实施例的提高克隆镜像性能的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, disk, CD) contains several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the method for improving the performance of cloning and mirroring in various embodiments of the present application.
本申请的说明书和权利要求书及上述附图说明中的术语“包括”和“具有”以及它们的任何变形,意图在于覆盖不排他的包含。本申请的说明书和权利要求书或上述附图中的术语“第一”、“第二”等是用于区别不同对象,而不是用于描述特定顺序。在本文中提及“实施例”意味着,结合实施例描述的特定特征、结构或特性可以包含在本申请的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例,也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是,本文所描述的实施例可以与其它实施例相结合。The terms "comprising" and "having" and any variations thereof in the specification and claims of the present application and the above descriptions of the drawings are intended to cover non-exclusive inclusion. The terms "first", "second" and the like in the description and claims of the present application or the above drawings are used to distinguish different objects, rather than to describe a specific order. Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The occurrences of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is understood explicitly and implicitly by those skilled in the art that the embodiments described herein can be combined with other embodiments.
以上实施例仅为本发明的较佳实施例而已,并不用以限制本发明,凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等,均应包含在本发明的保护范围之内。The above embodiments are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910441492.0A CN110347333A (en) | 2019-05-24 | 2019-05-24 | Improve method, apparatus, computer equipment and the storage medium of clone's mirror image performance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910441492.0A CN110347333A (en) | 2019-05-24 | 2019-05-24 | Improve method, apparatus, computer equipment and the storage medium of clone's mirror image performance |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110347333A true CN110347333A (en) | 2019-10-18 |
Family
ID=68174025
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910441492.0A Pending CN110347333A (en) | 2019-05-24 | 2019-05-24 | Improve method, apparatus, computer equipment and the storage medium of clone's mirror image performance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110347333A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114860350A (en) * | 2022-07-07 | 2022-08-05 | 杭州子默网络科技有限公司 | Method and device for data processing based on cloud diskless tree image |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104182184A (en) * | 2014-08-27 | 2014-12-03 | 浪潮电子信息产业股份有限公司 | Distributed block storing and cloning method |
| CN106161120A (en) * | 2016-10-08 | 2016-11-23 | 电子科技大学 | The distributed meta-data management method of dynamic equalization load |
| CN106506587A (en) * | 2016-09-23 | 2017-03-15 | 中国人民解放军国防科学技术大学 | A Docker image download method based on distributed storage |
-
2019
- 2019-05-24 CN CN201910441492.0A patent/CN110347333A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104182184A (en) * | 2014-08-27 | 2014-12-03 | 浪潮电子信息产业股份有限公司 | Distributed block storing and cloning method |
| CN106506587A (en) * | 2016-09-23 | 2017-03-15 | 中国人民解放军国防科学技术大学 | A Docker image download method based on distributed storage |
| CN106161120A (en) * | 2016-10-08 | 2016-11-23 | 电子科技大学 | The distributed meta-data management method of dynamic equalization load |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114860350A (en) * | 2022-07-07 | 2022-08-05 | 杭州子默网络科技有限公司 | Method and device for data processing based on cloud diskless tree image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110162525B (en) | B+ tree-based read-write conflict resolution method, device and storage medium | |
| US20150302111A1 (en) | Method and Apparatus for Constructing File System in Key-Value Storage System, and Electronic Device | |
| US10891074B2 (en) | Key-value storage device supporting snapshot function and operating method thereof | |
| CN111723056B (en) | Small file processing method, device, equipment and storage medium | |
| CN111444196B (en) | Method, device and device for generating hash of global state in blockchain ledger | |
| US10372678B2 (en) | Files having unallocated portions within content addressable storage | |
| CN111177143B (en) | Key-value data storage method, device, storage medium and electronic device | |
| WO2017032170A1 (en) | Method and apparatus for importing mirror image file | |
| CN107832062A (en) | A kind of method for updating program and terminal device | |
| CN107562915A (en) | Read the method, apparatus and equipment and computer-readable recording medium of small documents | |
| US11275835B2 (en) | Method of speeding up a full antivirus scan of files on a mobile device | |
| CN107798063B (en) | Snapshot processing method and snapshot processing device | |
| US20210157513A1 (en) | Data reading method, apparatus, and system, and distributed system | |
| CN112286457B (en) | Object deduplication method and device, electronic equipment and machine-readable storage medium | |
| US20200117722A1 (en) | Efficient file storage and retrieval system, method and apparatus | |
| CN116991897A (en) | A query method and device based on disk storage | |
| CN113934362B (en) | Data storage method, device, computer equipment and storage medium | |
| CN112445764B (en) | File operation method, device, storage medium and electronic equipment | |
| CN110347333A (en) | Improve method, apparatus, computer equipment and the storage medium of clone's mirror image performance | |
| US12117907B2 (en) | Data replication using an extended file replication protocol | |
| CN114647630B (en) | File synchronization, information generation method, device, computer equipment and storage medium | |
| CN116628773A (en) | Data processing method, device, electronic device and storage medium | |
| CN110636042B (en) | Method, device and equipment for updating verified block height of server | |
| CN109299067B (en) | Page access method, device and storage medium | |
| CN107305582B (en) | Method and device for processing metadata |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191018 |
|
| RJ01 | Rejection of invention patent application after publication |