CN117271440A - File information storage method, reading method and related equipment based on freeRTOS - Google Patents
File information storage method, reading method and related equipment based on freeRTOS Download PDFInfo
- Publication number
- CN117271440A CN117271440A CN202311554890.6A CN202311554890A CN117271440A CN 117271440 A CN117271440 A CN 117271440A CN 202311554890 A CN202311554890 A CN 202311554890A CN 117271440 A CN117271440 A CN 117271440A
- Authority
- CN
- China
- Prior art keywords
- file
- sub
- information
- storage area
- offset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/113—Details of archiving
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本申请实施例公开了一种基于freeRTOS文件信息存储方法、读取方法及相关设备,涉及文件信息存储领域,本申请根据子文件集的大小和数量确定目标文件的大小;将目标文件划分为文件头存储区、文件名索引存储区、文件信息存储区和正文存储区,根据各子文件的信息确定正文偏移以及子文件信息存储区的偏移量,使得整个文件结构紧凑,避免了使用LittleFS文件系统出现的空间浪费。进一步地,通过将子文件名的哈希值与子文件的数量求余以确定各子文件的存储顺序,通过将存在碰撞的子文件信息的偏移存储在文件信息存储区中,使得文件结构简洁,通过文件名索引区和文件信息存储区的联动,使得存储和读取更加便捷,提高了存储和读取效率。
The embodiments of this application disclose a file information storage method, reading method and related equipment based on freeRTOS, which relate to the field of file information storage. This application determines the size of the target file based on the size and number of the sub-file set; divides the target file into files Head storage area, file name index storage area, file information storage area and text storage area, determine the text offset and the offset of the sub-file information storage area based on the information of each sub-file, making the entire file structure compact and avoiding the use of LittleFS Wasted space in the file system. Furthermore, the storage order of each sub-file is determined by taking the remainder of the hash value of the sub-file name and the number of sub-files, and by storing the offset of the colliding sub-file information in the file information storage area, so that the file structure is Simple, through the linkage of the file name index area and the file information storage area, storage and reading are more convenient and the storage and reading efficiency is improved.
Description
技术领域Technical field
本申请涉及文件信息存储领域,尤其涉及一种基于freeRTOS文件信息存储方法、读取方法及相关设备。This application relates to the field of file information storage, and in particular to a file information storage method, reading method and related equipment based on freeRTOS.
背景技术Background technique
随着设备的小型化,人们对于操作系统的要求越来越高。在嵌入式领域中,嵌入式实时操作系统正得到越来越广泛的应用。采用嵌入式实时操作系统(RTOS)可以更合理、更有效地利用CPU的资源,简化应用软件的设计,缩短系统开发时间,更好地保证系统的实时性和可靠性。With the miniaturization of devices, people's requirements for operating systems are getting higher and higher. In the embedded field, embedded real-time operating systems are becoming more and more widely used. Using an embedded real-time operating system (RTOS) can make more reasonable and effective use of CPU resources, simplify the design of application software, shorten system development time, and better ensure the real-time and reliability of the system.
然而,在freeRTOS操作系统下默认不带文件系统的,数据直接存储在flash中,为了方便应用开发,可以移植第三方的文件系统,比较常用的就是LittleFS。但是在littleFS下,块大小通常是固定大小,通常为4KB,即使文件内容只有1KB,文件系统也会分配4KB的空间,造成3KB的空间浪费,如果要存储的文件特别多,空间浪费会更大。例如,当存储的文件都是1KB时,其会造成3KB×文件个数的空间浪费,这对于具有较小存储空间的设备而言是致命的。However, the freeRTOS operating system does not have a file system by default, and the data is stored directly in flash. In order to facilitate application development, a third-party file system can be transplanted. The more commonly used one is LittleFS. However, under littleFS, the block size is usually a fixed size, usually 4KB. Even if the file content is only 1KB, the file system will allocate 4KB of space, resulting in a waste of 3KB of space. If there are particularly many files to be stored, the waste of space will be even greater. . For example, when the files stored are all 1KB, it will cause a waste of space of 3KB × the number of files, which is fatal for devices with small storage space.
对于非嵌入式系统或者服务器而言,同样存在合并文件的需求。然而,该些系统中更加注重的是读取速度,并不需要考虑存储空间的问题。通常采用单独建立索引的方式,当需要读取时,先通过索引查找目标文件的存储位置,返回系统,系统根据该存储位置,从数据池中取对应的数据。For non-embedded systems or servers, there is also a need to merge files. However, these systems pay more attention to the reading speed and do not need to consider the issue of storage space. Usually, a separate index is created. When it is necessary to read, the storage location of the target file is first searched through the index and returned to the system. The system fetches the corresponding data from the data pool based on the storage location.
因此,急需一种合并文件结构及对应的文件信息存储方法,以解决现有的littleFS下空间利用率以及读写效率不高的问题。Therefore, there is an urgent need for a method of merging file structures and corresponding file information storage to solve the existing problems of low space utilization and low reading and writing efficiency under littleFS.
发明内容Contents of the invention
本申请实施例提供一种基于freeRTOS文件信息存储方法、读取方法及相关设备,可以将多个文件合并存储在在这个合并文件中,并通过文件名快速准确的定位到文件内容并读取出来。The embodiments of this application provide a file information storage method, reading method and related equipment based on freeRTOS. Multiple files can be merged and stored in this merged file, and the file content can be quickly and accurately located through the file name and read out. .
一方面,本申请实施例提供了一种基于freeRTOS文件信息存储方法,所述方法包括:On the one hand, embodiments of the present application provide a method for storing file information based on freeRTOS. The method includes:
筛选确定多个子文件得到待处理子文件集;Screen and determine multiple sub-files to obtain a set of sub-files to be processed;
根据所述子文件集大小及子文件的数量确定目标文件的大小;Determine the size of the target file based on the size of the sub-file set and the number of sub-files;
将所述目标文件的存储空间划分为文件头存储区、文件名索引存储区、文件信息存储区和正文存储区;Divide the storage space of the target file into a file header storage area, a file name index storage area, a file information storage area and a text storage area;
确定所述文件头存储区、文件名索引存储区的存储区域;Determine the storage areas of the file header storage area and the file name index storage area;
写入所述文件头存储区内的文件头信息;Write the file header information in the file header storage area;
初始化所述文件名索引存储区及文件信息存储区;Initialize the file name index storage area and file information storage area;
依次确定所述子文件集的各子文件的文件信息存储偏移,并更新所述文件名索引存储区信息;Determine the file information storage offset of each sub-file of the sub-file set in turn, and update the file name index storage area information;
写入更新所述文件信息存储区并将各子文件写入正文存储区。Write and update the file information storage area and write each sub-file into the text storage area.
进一步地,所述文件头信息包括目标文件标识符、子文件数量、正文偏移、校验位,所述写入所述文件头存储区内的文件头信息包括:Further, the file header information includes a target file identifier, the number of sub-files, a text offset, and a check digit. The file header information written into the file header storage area includes:
写入目标文件标识符信息、子文件数量信息;Write target file identifier information and sub-file number information;
根据子文件数量和各子文件名的长度计算得到正文偏移;The text offset is calculated based on the number of sub-files and the length of each sub-file name;
写入正文偏移信息;Write text offset information;
写入目标文件校验信息。Write verification information to the target file.
进一步地,依次确定所述子文件集的各子文件的文件信息存储偏移,并更新所述文件名索引存储区信息,包括:Further, determining the file information storage offset of each sub-file of the sub-file set in turn, and updating the file name index storage area information includes:
提取当前子文件的文件名,对所述各子文件的文件名进行哈希计算得到各子文件名哈希值;Extract the file name of the current sub-file, and perform hash calculation on the file name of each sub-file to obtain the hash value of each sub-file name;
将当前子文件名哈希值与所述子文件的数量求余;Find the remainder of the current subfile name hash value and the number of said subfiles;
根据求余结果确定当前子文件名索引区的位置及当前子文件信息存储区的偏移量;Determine the location of the current sub-file name index area and the offset of the current sub-file information storage area based on the remainder result;
将当前子文件信息存储区的偏移量写入更新对应文件名索引区信息。Write the offset of the current sub-file information storage area to update the corresponding file name index area information.
进一步地,写入更新所述文件信息存储区并将各子文件写入正文存储区,包括:Further, writing and updating the file information storage area and writing each sub-file into the text storage area includes:
根据当前子文件求余结果确定得到当前子文件正文偏移;Determine and obtain the text offset of the current subfile based on the remainder result of the current subfile;
根据当前子文件信息存储区的偏移量确定当前子文件信息的存储区域;Determine the storage area of the current sub-file information according to the offset of the current sub-file information storage area;
将子文件名大小、子文件大小、子文件正文偏移、子文件名,写入到当前子文件信息存储区;Write the subfile name size, subfile size, subfile text offset, and subfile name to the current subfile information storage area;
将当前子文件移动到当前子文件正文偏移区,写入当前子文件。Move the current subfile to the current subfile text offset area and write to the current subfile.
进一步地,文件信息存储区还包括哈希碰撞下一个子文件信息的偏移,所述方法还包括:Further, the file information storage area also includes an offset of the next sub-file information in a hash collision, and the method further includes:
判断当前子文件求余结果是否存在碰撞;Determine whether there is a collision in the remainder result of the current sub-file;
若存在,重新确定当前子文件文件信息存储区的偏移的正文偏移;If it exists, redetermine the text offset of the offset of the current subfile file information storage area;
将当前子文件的文件信息偏移更新写入到上一文件的哈希碰撞下一个子文件信息的偏移。Write the file information offset update of the current subfile to the offset of the next subfile information against the hash of the previous file.
进一步地,所述方法包括:Further, the method includes:
提取各子文件的文件名,对所述各子文件的文件名进行哈希计算得到各子文件名哈希值;Extract the file name of each sub-file, and perform hash calculation on the file name of each sub-file to obtain the hash value of each sub-file name;
将各子文件名哈希值与所述子文件的数量求余;Calculate the remainder of the hash value of each sub-file name and the number of said sub-files;
对所述各子文件对应的求余结果进行排序;Sort the remainder results corresponding to each of the sub-files;
根据所述排序结果得到各子文件的存储顺序;Obtain the storage order of each sub-file according to the sorting result;
按照各子文件的存储顺序,逐个更新所述文件名索引存储区信息,并逐个写入更新所述文件信息存储区和正文存储区。According to the storage order of each sub-file, the file name index storage area information is updated one by one, and the file information storage area and text storage area are written and updated one by one.
本申请第二方面提供一种基于freeRTOS文件信息读取方法,所述方法包括:The second aspect of this application provides a method for reading file information based on freeRTOS. The method includes:
获取要读取的子文件名;Get the subfile name to be read;
根据所述子文件名计算哈希值,得到该文件在文件名索引区的位置;Calculate the hash value according to the sub-file name to obtain the position of the file in the file name index area;
根据所述文件名索引区的位置得到对应文件信息区的存储偏移量;Obtain the storage offset of the corresponding file information area according to the position of the file name index area;
根据文件信息区的偏移量获取当前子文件的正文偏移量和文件大小;Get the text offset and file size of the current subfile based on the offset of the file information area;
根据当前子文件的正文偏移量和文件大小,定位到子文件正文位置并按所述文件大小读取所述子文件正文信息。According to the text offset and file size of the current sub-file, locate the sub-file text position and read the sub-file text information according to the file size.
进一步地,所述根据文件信息区的存储偏移量获取当前子文件的正文偏移量和文件大小,包括:Further, obtaining the text offset and file size of the current subfile based on the storage offset of the file information area includes:
根据文件信息区的存储偏移量定位;Positioning based on the storage offset of the file information area;
判断文件名是否相同,若相同,则读取正文偏移量信息和文件大小信息;Determine whether the file names are the same. If they are the same, read the text offset information and file size information;
若不相同,则获取当前文件信息区得到哈希碰撞下一个子文件信息的偏移量信息,循环寻找下一个文件信息区信息,直到找到当前子文件正文偏移量和文件大小为止。If they are not the same, obtain the current file information area to obtain the offset information of the next sub-file information of the hash collision, and loop to find the next file information area information until the current sub-file text offset and file size are found.
本申请第三方面提供一种计算机设备,包括存储器和处理器,所述存储器存储有计算机程序,其特征在于,所述处理器执行所述计算机程序时实现前述方法步骤。A third aspect of the present application provides a computer device, including a memory and a processor. The memory stores a computer program. The feature is that the processor implements the foregoing method steps when executing the computer program.
本申请第四方面提供一种计算机可读存储介质,存储有计算机程序,所述计算机程序被处理器执行时,使得所述处理器执行以实现前述方法步骤。A fourth aspect of the present application provides a computer-readable storage medium that stores a computer program. When the computer program is executed by a processor, it causes the processor to execute to implement the foregoing method steps.
本申请各实施例中,将多个文件合并存储在在这个合并文件中,并通过文件名快速准确的定位到文件内容并读取出来,即减少了存储空间的浪费,又提高了读写的效率。具体地,通过改变文件结构,改变文件信息的存储和读取方法,根据子文件集的大小和数量确定目标文件的大小;并且将目标文件划分为文件头存储区、文件名索引存储区、文件信息存储区和正文存储区,根据各子文件的信息确定正文偏移以及子文件信息存储区的偏移量,使得整个文件结构紧凑,各文件依次排列,避免了使用LittleFS文件系统的情况下出现较多的空间浪费。进一步地,本申请各实施例通过将子文件名的哈希值与子文件的数量求余以确定各子文件的存储顺序,通过将存在碰撞的子文件信息的偏移存储在文件信息存储区中,使得文件结构简洁,不再需要单独保存文件名相关信息与存储位置的索引表,通过文件名索引区和文件信息存储区的联动,使得存储和读取更加便捷。目标文件使用文件名的哈希值计算位置,可以快速定位到子文件的具体位置,减少查找的次数,加快访问子文件的速度。In each embodiment of the present application, multiple files are merged and stored in this merged file, and the file content is quickly and accurately located and read through the file name, which reduces the waste of storage space and improves the efficiency of reading and writing. efficiency. Specifically, by changing the file structure, changing the storage and reading method of file information, the size of the target file is determined according to the size and number of sub-file sets; and the target file is divided into a file header storage area, a file name index storage area, and a file name index storage area. The information storage area and the text storage area determine the text offset and the offset of the sub-file information storage area based on the information of each sub-file, making the entire file structure compact, and each file is arranged in order, avoiding the problem of using the LittleFS file system. More space wasted. Further, each embodiment of the present application determines the storage order of each sub-file by taking the remainder of the hash value of the sub-file name and the number of sub-files, and stores the offset of the colliding sub-file information in the file information storage area , the file structure is concise, and there is no need to separately save the index table of file name-related information and storage location. Through the linkage of the file name index area and the file information storage area, storage and reading are made more convenient. The target file uses the hash value of the file name to calculate the location, which can quickly locate the specific location of the sub-file, reduce the number of searches, and speed up access to the sub-file.
附图说明Description of the drawings
构成本申请的一部分的附图用来提供对本发明的进一步理解,本发明的示意性实施例及其说明用于解释本发明,并不构成对本发明的不当限定。The drawings forming a part of this application are used to provide a further understanding of the present invention. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention.
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without exerting creative efforts.
图1是本申请实施例提供的一种基于freeRTOS文件信息存储方法的流程图;Figure 1 is a flow chart of a freeRTOS file information storage method provided by an embodiment of the present application;
图2是本申请实施例提供的一种基于freeRTOS文件信息读取方法的流程图。Figure 2 is a flow chart of a method for reading file information based on freeRTOS provided by an embodiment of the present application.
具体实施方式Detailed ways
本申请实施例提出了一种基于freeRTOS文件信息存储方法,将多个文件合并存储在在这个合并文件中,并通过文件名快速准确的定位到文件内容并读取出来,即减少了存储空间的浪费,又提高了读写的效率。具体地,通过改变文件结构,改变文件信息的存储和读取方法,根据子文件集的大小和数量确定目标文件的大小;并且将目标文件划分为文件头存储区、文件名索引存储区、文件信息存储区和正文存储区,根据各子文件的信息确定正文偏移以及子文件信息存储区的偏移量,使得整个文件结构紧凑,各文件依次排列,避免了使用LittleFS文件系统的情况下出现较多的空间浪费。The embodiment of this application proposes a method for storing file information based on freeRTOS. Multiple files are merged and stored in this merged file, and the file content is quickly and accurately located and read through the file name, that is, the storage space is reduced. Waste and improve the efficiency of reading and writing. Specifically, by changing the file structure, changing the storage and reading method of file information, the size of the target file is determined according to the size and number of sub-file sets; and the target file is divided into a file header storage area, a file name index storage area, and a file name index storage area. The information storage area and the text storage area determine the text offset and the offset of the sub-file information storage area based on the information of each sub-file, making the entire file structure compact, and each file is arranged in order, avoiding the problem of using the LittleFS file system. More space wasted.
进一步地,本申请各实施例通过将子文件名的哈希值与子文件的数量求余以确定各子文件的存储顺序,通过将存在碰撞的子文件信息的偏移存储在文件信息存储区中,使得文件结构简洁,不再需要单独保存文件名相关信息与存储位置的索引表,通过文件名索引区和文件信息存储区的联动,使得存储和读取更加便捷。Further, each embodiment of the present application determines the storage order of each sub-file by taking the remainder of the hash value of the sub-file name and the number of sub-files, and stores the offset of the colliding sub-file information in the file information storage area , the file structure is concise, and there is no need to separately save the index table of file name-related information and storage location. Through the linkage of the file name index area and the file information storage area, storage and reading are made more convenient.
下面将结合附图,对本申请实施例提供的文件信息存储方法进行介绍。The file information storage method provided by the embodiment of the present application will be introduced below with reference to the accompanying drawings.
请参见图1,图1为本申请实施例提供的一种文件信息存储方法的流程图。如图1所示,该数据处理方法包括以下步骤S1-S8。Please refer to Figure 1. Figure 1 is a flow chart of a file information storage method provided by an embodiment of the present application. As shown in Figure 1, the data processing method includes the following steps S1-S8.
S1、筛选确定多个子文件得到待处理子文件集;S1. Screen and determine multiple sub-files to obtain a set of sub-files to be processed;
FreeRTOS是一个迷你的实时操作系统内核。作为一个轻量级的操作系统,功能包括:任务管理、时间管理、信号量、消息队列、内存管理、记录功能、软件定时器、协程等,可基本满足较小系统的需要。FreeRTOS中,常用的文件系统为LittleFS。虽然LittleFS具有掉电恢复、擦写均衡等优点,但是其也具有明显的缺点。LittleFS的空间利用率受到一定的限制,LittleFS的块大小决定了文件存储的最小单位;当块越大,则越有可能会出现空间浪费的情况。例如32K的块大小,即使文件内容只有1KB,LittleFS也会为其分配32K的空间,造成了31K的空间浪费。即便是目前,LittleFS的块最小为4K,当文件内容只有1KB时,依然存在较大的浪费。FreeRTOS is a mini real-time operating system kernel. As a lightweight operating system, its functions include: task management, time management, semaphores, message queues, memory management, recording functions, software timers, coroutines, etc., which can basically meet the needs of smaller systems. In FreeRTOS, the commonly used file system is LittleFS. Although LittleFS has the advantages of power failure recovery, erase and write balancing, etc., it also has obvious shortcomings. The space utilization of LittleFS is subject to certain limitations. The block size of LittleFS determines the smallest unit of file storage; the larger the block, the more likely it is that space will be wasted. For example, if the block size is 32K, even if the file content is only 1KB, LittleFS will allocate 32K space for it, resulting in a waste of 31K space. Even now, the minimum block size of LittleFS is 4K. When the file content is only 1KB, there is still a lot of waste.
可以理解的是,FreeRTOS中,通常会存储大量的子文件,该些子文件可以为系统子文件,可以为用户存储的子文件,也可以是应用的碎片文件等。而这些子文件的文件大小各不相同,若采用现有的LittleFS文件系统,会造成较大的空间浪费。因此,本实施例中,通过采用本申请的文件存储方法,可以克服上述缺陷。It is understandable that FreeRTOS usually stores a large number of sub-files. These sub-files can be system sub-files, sub-files stored by users, or application fragment files, etc. The file sizes of these sub-files are different. If the existing LittleFS file system is used, it will cause a large waste of space. Therefore, in this embodiment, by adopting the file storage method of the present application, the above defects can be overcome.
本申请通过对多个子文件进行筛选,以得到待处理子文件集。可以理解的是,本申请并非对所有的子文件都采用本申请的文件存储方法。由于LittleFS中规定了块的大小,当数据文件的大小与LittleFS中规定了块的大小相同时,该数据文件可以充分利用存储资源,并不会造成资源浪费。因此,本实施例中,通过对多个子文件进行分析筛选,得到待处理子文件集,该待处理子文件集采用本申请的文件存储方法。This application filters multiple sub-files to obtain a set of sub-files to be processed. It can be understood that this application does not use the file storage method of this application for all sub-files. Since the block size is specified in LittleFS, when the size of the data file is the same as the block size specified in LittleFS, the data file can make full use of storage resources and will not waste resources. Therefore, in this embodiment, a set of sub-files to be processed is obtained by analyzing and filtering multiple sub-files, and the set of sub-files to be processed adopts the file storage method of the present application.
在一个实施例中,对多个子文件的筛选,可以根据其数据大小进行筛选。具体地,提取多个子文件的文件大小,计算文件大小与LittleFS中规定的块大小的余数,余数满足预设条件下,其对应的子文件通过筛选,将其作为待处理子文件。示例性的,LittleFS中规定的块大小可以为4K,预设条件可以为余数不等于0,则在筛选的过程中,文件大小不等于4的整数倍的子文件都会通过筛选,形成待处理子文件集。可以理解的是,预设条件为余数不等于0,仅仅是其中一种实现方式,预设条件也可以为余数小于预设阈值,该阈值可以根据LittleFS中规定的块大小进行设置,例如块大小的一半或者四分之三。该筛选条件是为了筛选出会造成存储空间严重浪费的子文件构成待处理子文件集,将该待处理子文件集采用本申请的文件信息存储方法,以避免其造成存储空间的浪费。In one embodiment, multiple sub-files can be filtered based on their data size. Specifically, the file sizes of multiple sub-files are extracted, and the remainder of the file size and the block size specified in LittleFS is calculated. If the remainder meets the preset conditions, the corresponding sub-files are filtered and used as sub-files to be processed. For example, the block size specified in LittleFS can be 4K, and the preset condition can be that the remainder is not equal to 0. Then during the filtering process, sub-files whose file size is not equal to an integer multiple of 4 will be filtered and form sub-files to be processed. File set. It can be understood that the preset condition is that the remainder is not equal to 0, which is just one of the implementation methods. The preset condition can also be that the remainder is less than the preset threshold. The threshold can be set according to the block size specified in LittleFS, such as the block size. half or three-quarters. This filtering condition is to filter out the sub-files that will cause serious waste of storage space to form a set of sub-files to be processed, and use the file information storage method of this application to avoid the set of sub-files to be processed, which will cause a waste of storage space.
S2、根据所述子文件集大小及子文件的数量确定目标文件的大小;S2. Determine the size of the target file according to the size of the sub-file set and the number of sub-files;
所述目标文件为采用本申请的文件信息存储方法所最终生成的合并文件。本实施例中,目标文件会存储子文件集的相关数据,因此,根据所述子文件集大小及子文件的数量确定目标文件的大小。The target file is a merged file finally generated by using the file information storage method of this application. In this embodiment, the target file stores relevant data of the sub-file set. Therefore, the size of the target file is determined based on the size of the sub-file set and the number of sub-files.
S3、将所述目标文件的存储空间划分为文件头存储区、文件名索引存储区、文件信息存储区和正文存储区。S3. Divide the storage space of the target file into a file header storage area, a file name index storage area, a file information storage area and a text storage area.
由于目标文件为采用本申请的文件信息存储方法所最终生成的合并文件。其需要存储有子文件集的相关数据,并且,目标文件中还必须包含有相应的标识信息以及索引信息,以便系统可以查找到所需要的子文件。Because the target file is a merged file finally generated by using the file information storage method of this application. It needs to store relevant data of the sub-file set, and the target file must also contain corresponding identification information and index information so that the system can find the required sub-files.
因此,本申请一个实施例中,将所述目标文件的存储空间划分为文件头存储区、文件名索引存储区、文件信息存储区和正文存储区。文件头存储区为该目标文件的文件头,其存储有相关的文件头信息。具体地,所述文件头信息包括:4字节的文件标识符、2字节的子文件个数、4字节的正文偏移数据以及1字节的校验字节。文件标识符用于标识该目标文件,使得系统能够快速的识别该目标文件;子文件个数用于表征该目标文件中所存储的子文件个数;正文偏移数据为目标文件中的正文存储区的偏移值;校验字节则用于对目标文件进行校验。可以理解的是,文件头信息中各个部分仅为示例,其可以根据需要以及实际情况进行修改。Therefore, in one embodiment of the present application, the storage space of the target file is divided into a file header storage area, a file name index storage area, a file information storage area, and a text storage area. The file header storage area is the file header of the target file, which stores relevant file header information. Specifically, the file header information includes: a 4-byte file identifier, a 2-byte sub-file number, a 4-byte text offset data, and a 1-byte check byte. The file identifier is used to identify the target file, allowing the system to quickly identify the target file; the number of sub-files is used to represent the number of sub-files stored in the target file; the text offset data is the text stored in the target file The offset value of the area; the check byte is used to verify the target file. It can be understood that each part of the file header information is only an example, and it can be modified according to needs and actual conditions.
本申请一个实施例中,采用文件名索引存储区和文件信息存储区共同完成文件索引功能,实现了粗-精的文件索引功能,既减少了索引区占用的存储空间,又保证了索引效率。文件名索引存储区,用于存储文件信息存储区的偏移,通过文件信息存储区的偏移可快速定位到具体的一个或者几个子文件。文件信息存储区中存储了各个子文件的详细信息,包括:1字节的子文件名长度、4字节的子文件正文偏移、4字节的子文件长度以及子文件名。当需要读取文件时,通过文件信息存储区的偏移可快速定位到具体的一个或者几个子文件,而通过定位到的文件信息存储区内具体的文件信息,可以得到子文件正文偏移,就可以从文件正文存储区中准确地定位到所需的子文件数据。由上可知,本申请中,并非通过简单地提取各个子文件的文件名,将其直接进行存储记录下其存储的位置;如此操作下虽然也可以找到所需的子文件,但是效率低下。本申请中,通过在目标文件中设置了文件名索引存储区和文件信息存储区,使用两个区的信息共同完成文件索引功能,实现了粗-精的文件索引功能,通过文件名的相关信息对文件信息进行精准定位,并且从文件信息存储区中得到准确地文件信息,根据得到的文件信息由文件正文存储区中获得子文件的完整数据;既减少了索引区占用的存储空间,又保证了索引效率。In one embodiment of the present application, the file name index storage area and the file information storage area are used to jointly complete the file indexing function, realizing a coarse-fine file indexing function, which not only reduces the storage space occupied by the index area, but also ensures indexing efficiency. The file name index storage area is used to store the offset of the file information storage area. One or several specific sub-files can be quickly located through the offset of the file information storage area. The file information storage area stores detailed information of each sub-file, including: 1-byte sub-file name length, 4-byte sub-file text offset, 4-byte sub-file length and sub-file name. When a file needs to be read, one or several specific sub-files can be quickly located through the offset in the file information storage area, and the sub-file text offset can be obtained through the specific file information in the located file information storage area. You can accurately locate the required sub-file data from the file text storage area. It can be seen from the above that in this application, the file name of each sub-file is not simply extracted, and the file name is directly stored to record its storage location; although the required sub-file can be found in this way, the efficiency is low. In this application, a file name index storage area and a file information storage area are set up in the target file, and the information in the two areas is used to jointly complete the file index function, thereby realizing a coarse-fine file index function. Through the relevant information of the file name Accurately locate the file information and obtain accurate file information from the file information storage area. Based on the obtained file information, complete data of the sub-file is obtained from the file text storage area; this not only reduces the storage space occupied by the index area, but also ensures improve indexing efficiency.
示例性的,本实施例中的目标文件的文件结构如下表所示:For example, the file structure of the target file in this embodiment is as shown in the following table:
文件头存储区、文件名索引存储区、文件信息存储区和正文存储区分别按照上表预留相应的存储空间,以便后续信息的更新写入。The file header storage area, file name index storage area, file information storage area and text storage area respectively reserve corresponding storage space according to the above table to facilitate subsequent information update and writing.
S4、确定所述文件头存储区、文件名索引存储区的存储区域;S4. Determine the storage areas of the file header storage area and the file name index storage area;
本实施例为一种基于freeRTOS文件信息存储方法,在筛选出待处理子文件集后,根据子文件集大小及子文件数量确定目标文件的大小后,需要对目标文件的每个区域大小进行计算并预留相应的空间,以便后续将文件写入至对应的区域。This embodiment is a file information storage method based on freeRTOS. After filtering out the sub-file set to be processed, and determining the size of the target file based on the size of the sub-file set and the number of sub-files, it is necessary to calculate the size of each area of the target file. And reserve the corresponding space for subsequent writing of files to the corresponding area.
本申请一个实施例中,文件头存储区存储有该目标文件的文件头。具体地,所述文件头信息包括:4字节的文件标识符、2字节的子文件个数、4字节的正文偏移数据以及1字节的校验字节。文件名索引存储区,用于存储文件信息存储区的偏移,其占用的空间大小为子文件个数×4字节。这两部分所需要占用的空间是固定的,在子文件集确定的情况下,文件头存储区和文件名索引存储区的存储区域可以确定并预留。至于两个区域内的偏移值,可以待后续确定后填入。In one embodiment of the present application, the file header storage area stores the file header of the target file. Specifically, the file header information includes: a 4-byte file identifier, a 2-byte sub-file number, a 4-byte text offset data, and a 1-byte check byte. The file name index storage area is used to store the offset of the file information storage area. The space occupied by it is the number of sub-files x 4 bytes. The space required by these two parts is fixed. When the sub-file set is determined, the storage areas of the file header storage area and the file name index storage area can be determined and reserved. As for the offset value within the two areas, it can be filled in after subsequent determination.
S5、写入所述文件头存储区内的文件头信息;S5. Write the file header information in the file header storage area;
在确定了目标文件的文件头存储区和文件名索引存储区的存储区域后,可以对文件头信息进行写入。所述文件头信息包括目标文件标识符、子文件数量、正文偏移、校验位,写入目标文件头信息包括:写入目标文件标识符信息、子文件数量信息;根据子文件数量和各子文件名的长度计算得到正文偏移;写入正文偏移信息;写入目标文件校验信息。文件头信息中的文件标识符、子文件个数以及校验信息,可以在子文件集确定后就进行写入操作。如此,可以在大量文件写入之前,先将一部分预设的数据预先写入,减轻后续写入数据的压力。After determining the storage area of the file header storage area and the file name index storage area of the target file, the file header information can be written. The file header information includes the target file identifier, the number of sub-files, the text offset, and the check digit. The written target file header information includes: the written target file identifier information and the sub-file number information; according to the number of sub-files and each The length of the subfile name is calculated to obtain the text offset; the text offset information is written; and the target file verification information is written. The file identifier, number of sub-files and verification information in the file header information can be written after the sub-file set is determined. In this way, a part of the preset data can be pre-written before a large number of files are written, thereby reducing the pressure of subsequent data writing.
S6、初始化所述文件名索引存储区及文件信息存储区;S6. Initialize the file name index storage area and file information storage area;
在筛选出待处理子文件集后,根据子文件集大小及子文件数量确定目标文件的大小后,需要对目标文件的每个区域大小进行计算并预留相应的空间,以便后续将文件写入至对应的区域。而文件名索引存储区及文件信息存储区所需要占用的空间是固定的,在子文件集确定的情况下,文件头存储区和文件名索引存储区的存储区域可以确定并预留,并且为了防止预留的空间内存储有其他无关的数据,在写入数据前,需要对文件名索引存储区及文件信息存储区进行初始化。After filtering out the sub-file set to be processed, and determining the size of the target file based on the size of the sub-file set and the number of sub-files, it is necessary to calculate the size of each area of the target file and reserve the corresponding space for subsequent writing of the file. to the corresponding area. The space required by the file name index storage area and file information storage area is fixed. When the sub-file set is determined, the storage areas of the file header storage area and file name index storage area can be determined and reserved, and for To prevent other irrelevant data from being stored in the reserved space, the file name index storage area and file information storage area need to be initialized before writing data.
S7、依次确定所述子文件集的各子文件的文件信息存储偏移,并更新所述文件名索引存储区信息;S7. Determine the file information storage offset of each sub-file of the sub-file set in sequence, and update the file name index storage area information;
对子文件集中的各个子文件进行分析,根据其文件数据大小、文件名的内容及其长度可以确定文件信息存储区所需的存储空间、各子文件存储在文件名索引存储区的存储位置以及文件信息存储区的偏移量;将文件信息存储区的偏移量更新子文件名索引存储区的对应位置。Analyze each sub-file in the sub-file set, and determine the storage space required for the file information storage area, the storage location of each sub-file in the file name index storage area, and The offset of the file information storage area; update the offset of the file information storage area to the corresponding position of the sub-file name index storage area.
具体地,提取当前子文件的文件名,对所述各子文件的文件名进行哈希计算得到各子文件名哈希值;将当前子文件名哈希值与所述子文件的数量求余;根据求余结果确定当前子文件名索引区的位置及当前子文件信息存储区的偏移量;将当前子文件信息存储区的偏移量写入更新对应文件名索引区信息。本实施例中,根据子文件的数量对文件名索引存储区的存储空间进行预留,其预留空间为子文件个数×4字节,以确保子文件相关信息能够全部存放至子文件名索引存储区以及文件信息存储区。通过对子文件的文件名进行哈希后与子文件数量求余来确定其在文件名索引存储区以及文件信息存储区存放的位置,以便后续需要查找该子文件时能够迅速的找到该子文件的位置并且定位到其在文件正文存储区的位置。Specifically, extract the file name of the current sub-file, perform hash calculation on the file name of each sub-file to obtain the hash value of each sub-file name; calculate the remainder of the hash value of the current sub-file name and the number of the sub-files. ; Determine the location of the current sub-file name index area and the offset of the current sub-file information storage area based on the remainder result; write the offset of the current sub-file information storage area to update the corresponding file name index area information. In this embodiment, the storage space of the file name index storage area is reserved according to the number of sub-files. The reserved space is the number of sub-files × 4 bytes to ensure that all sub-file related information can be stored in the sub-file name. Index storage area and file information storage area. By hashing the file name of the sub-file and finding the remainder of the number of sub-files, the location where it is stored in the file name index storage area and the file information storage area is determined, so that the sub-file can be quickly found when the sub-file needs to be found later. location and locate its location in the file body storage area.
S8、写入更新所述文件信息存储区并将各子文件写入正文存储区。S8. Write and update the file information storage area and write each sub-file into the text storage area.
根据求余结果确定当前子文件名索引区的位置及当前子文件信息存储区的偏移量后,将子文件的相关信息,例如子文件名长度、子文件正文偏移、子文件长度、子文件名等信息写入文件信息存储区对应位置;并且将子文件数据存放至文件正文存储区。After determining the position of the current sub-file name index area and the offset of the current sub-file information storage area according to the remainder result, the relevant information of the sub-file, such as the length of the sub-file name, the offset of the sub-file body, the length of the sub-file, the Information such as the file name is written to the corresponding location in the file information storage area; and the sub-file data is stored in the file text storage area.
具体地,根据当前子文件求余结果确定得到当前子文件正文偏移;根据当前子文件信息存储区的偏移量确定当前子文件信息的存储区域;将子文件名大小、子文件大小、子文件正文偏移、子文件名,写入到当前子文件信息存储区;将当前子文件移动到当前子文件正文偏移区,写入当前子文件。可以理解的是,子文件求余结果直接决定了其在存放位置,根据其存放的位置将相应的信息写入文件名索引存储区、文件信息存储区以及文件正文存储区;各个区的存储空间已经针对子文件集进行了预留,可以确保每个子文件紧凑的写入,避免了现有技术中固定4K的块大小所造成的空间浪费。Specifically, the offset of the current sub-file text is determined based on the remainder of the current sub-file; the storage area of the current sub-file information is determined based on the offset of the current sub-file information storage area; the sub-file name size, sub-file size, sub-file information storage area are determined. The file text offset and sub-file name are written to the current sub-file information storage area; the current sub-file is moved to the current sub-file text offset area and written to the current sub-file. It can be understood that the remainder result of the sub-file directly determines its storage location. According to its storage location, the corresponding information is written into the file name index storage area, file information storage area and file text storage area; the storage space of each area Reservations have been made for sub-file sets, which ensures compact writing of each sub-file and avoids the waste of space caused by the fixed 4K block size in the existing technology.
由于子文件的文件名各异,其哈希后的哈希值也各不相同,但是由于本申请实施例中是将哈希值与子文件个数求余,那么就会存在有两个或者多个子文件的计算结果是相同的情况。针对上述情况,本申请提出了进一步地解决方案。Since the file names of sub-files are different, the hash values after hashing are also different. However, in the embodiment of this application, the hash value is modulated by the number of sub-files, then there will be two or The calculation result of multiple sub-files is the same. In response to the above situation, this application proposes further solutions.
在本申请一个实施例中,文件信息存储区还包括哈希碰撞下一个子文件信息的偏移,可为该偏移分配4字节空间。本申请的方法在得到子文件求余结果之后,对该结果进行分析,判断当前子文件求余结果是否存在碰撞。碰撞,可以理解为有两个或者多个子文件的计算结果是相同的。若存在碰撞,则重新确定当前子文件文件信息存储区的偏移的正文偏移;将当前子文件的文件信息偏移更新写入到上一文件的哈希碰撞下一个子文件信息的偏移。在写入数据时,通过对碰撞的判断,更加合理的将子文件信息写入目标文件,以便在文件读取的时候,通过由粗到精的定位方式,精准又快速的找到所需的文件数据。可以理解的是,在本实施例中,发生碰撞的子文件,存放在文件信息存储区和文件正文存储区的位置为相邻位置,在文件名索引存储区处共用一个位置的存储区域,用于存储碰撞子文件的第一个子文件的文件信息偏移值。In one embodiment of the present application, the file information storage area also includes an offset of the sub-file information next to the hash collision, and 4 bytes of space can be allocated for this offset. After obtaining the sub-file remainder result, the method of this application analyzes the result to determine whether there is a collision in the current sub-file remainder result. Collision can be understood as having two or more sub-files with the same calculation results. If there is a collision, re-determine the text offset of the offset of the current sub-file file information storage area; write the file information offset update of the current sub-file to the hash of the previous file to collide with the offset of the next sub-file information. . When writing data, by judging the collision, the sub-file information is written to the target file more reasonably, so that when the file is read, the required file can be found accurately and quickly through the rough-to-fine positioning method. data. It can be understood that in this embodiment, the colliding sub-files are stored in adjacent positions in the file information storage area and the file text storage area, and share the same storage area in the file name index storage area. Use The file information offset value of the first subfile that stores the collision subfile.
进一步地,本申请还包括另一个实施方式。本申请在文件写入过程中,可以对筛选出的待处理子文件集的各个子文件进行分析,提取各子文件的文件名,对所述各子文件的文件名进行哈希计算得到各子文件名哈希值;将各子文件名哈希值与所述子文件的数量求余;对所述各子文件对应的求余结果进行排序;根据所述排序结果得到各子文件的存储顺序;按照各子文件的存储顺序,逐个更新所述文件名索引存储区信息,并逐个写入更新所述文件信息存储区和正文存储区。进一步地,在对所述各子文件对应的求余结果进行排序时,若存在多个文件的求余结果相同,则可进一步地获取求余结果相同的子文件的文件优先级或者访问频次等信息,对其进行进一步排序。在本实施例中,通过对子文件集内的各个子文件进行统一分析,提取各个子文件的文件名,并计算器子文件名哈希值与所述子文件的数量的余,可以对各个子文件在目标文件中的存储位置进行排序,根据该排序结果,预留各个区域的存储空间,并且将相应的数据写入目标文件中。并且,本实施方式中,由于目标文件中包括了文件名索引存储区以及文件信息存储区,子文件的余用于读取文件名索引存储区内存放的文件信息的偏移值,进而通过文件信息存储区进一步定位到子文件征文偏移;然而,当存在碰撞时,文件信息的偏移值处对应的文件信息不止一个,若排在前面的文件信息不匹配,则需要通过哈希碰撞下一个子文件信息的偏移定位到下一文件信息,若碰撞文件较多时,会造成读取缓慢。本实施例中,通过获取求余结果相同的子文件的文件优先级或者访问频次等信息,对其进行进一步排序,使得重要的或者高频访问的子文件排列在较前的位置,在文件读取的时候可以优先被读取到,提高了读写效率。Furthermore, this application also includes another embodiment. During the file writing process, this application can analyze each sub-file of the filtered sub-file set to be processed, extract the file name of each sub-file, and perform hash calculation on the file name of each sub-file to obtain each sub-file. File name hash value; calculate the remainder of each sub-file name hash value and the number of the sub-files; sort the remainder results corresponding to each sub-file; obtain the storage order of each sub-file according to the sorting result ; According to the storage order of each sub-file, update the file name index storage area information one by one, and write and update the file information storage area and text storage area one by one. Further, when sorting the remainder results corresponding to each sub-file, if there are multiple files with the same remainder results, the file priority or access frequency of the sub-files with the same remainder results can be further obtained. information to further sort it. In this embodiment, by performing unified analysis on each sub-file in the sub-file set, extracting the file name of each sub-file, and calculating the remainder of the hash value of the sub-file name and the number of the sub-files, each sub-file can be analyzed The sub-files are sorted by their storage locations in the target file, and storage space in each area is reserved based on the sorting results, and corresponding data is written into the target file. Moreover, in this embodiment, since the target file includes a file name index storage area and a file information storage area, the remainder of the sub-file is used to read the offset value of the file information stored in the file name index storage area, and then through the file The information storage area further locates the sub-file essay offset; however, when there is a collision, there is more than one file information corresponding to the offset value of the file information. If the file information in the front does not match, it needs to be collided through a hash The offset of one sub-file information is positioned to the next file information. If there are many colliding files, reading will be slow. In this embodiment, information such as file priority or access frequency of sub-files with the same remainder result is obtained, and they are further sorted so that important or frequently accessed sub-files are arranged at a higher position. When the file is read, When fetching, it can be read first, which improves reading and writing efficiency.
进一步地,对各个子文件进行求余后,判断是否发生碰撞。若碰撞数量满足预设条件时,则改变文件名索引存储区的结构,采用余数-文件信息偏移值映射的关系,如此,仅需要为文件名索引存储区预留一下存储空间即可(未碰撞子文件个数+碰撞子文件不同余数的个数)×6字节,其中,6字节中有4字节用于存放文件信息的偏移值,2字节用于存放子文件的个数。所述预设条件可以为碰撞数量达到一定的比例,例如50%等,满足该预设条件,则说明子文件信息求余后,大部分会发生碰撞。示例性的,当需要存储的子文件集中有100个子文件,对其文件名哈希并且与文件个数求余之后,可能的结果为30个子文件未发生碰撞,70个子文件发生碰撞,且70个中有40个子文件的余数为2,30个子文件的余数为3。若采用前述实施例中的存储结构,则需要为文件名索引存储区预留(100)×4字节存储空间,使用本实施方式的方法仅需要为文件名索引存储区预留(30+2)×6字节存储空间即可。通过对碰撞情况进行分析,结合分析结果确定文件名索引存储区的具体结构,以预留相应的存储空间,使得文件名索引存储区所占用的存储空间最小。Further, after calculating the remainder of each sub-file, it is determined whether a collision occurs. If the number of collisions meets the preset conditions, the structure of the file name index storage area is changed and the remainder-file information offset value mapping relationship is adopted. In this way, it is only necessary to reserve some storage space for the file name index storage area (not yet The number of collision sub-files + the number of different remainders of collision sub-files) × 6 bytes, of which 4 bytes are used to store the offset value of the file information, and 2 bytes are used to store the individual sub-files. number. The preset condition can be that the number of collisions reaches a certain proportion, such as 50%, etc. If the preset condition is met, it means that most of the sub-file information will collide. For example, when there are 100 sub-files in the set of sub-files that need to be stored, after hashing the file names and finding the remainder from the number of files, the possible results are 30 sub-files without collision, 70 sub-files with collision, and 70 Among them, 40 sub-files have a remainder of 2, and 30 sub-files have a remainder of 3. If the storage structure in the previous embodiment is adopted, (100) × 4 bytes of storage space need to be reserved for the file name index storage area. Using the method of this embodiment only requires (30+2) to be reserved for the file name index storage area. )×6 bytes of storage space. By analyzing the collision situation and combining the analysis results, the specific structure of the file name index storage area is determined to reserve corresponding storage space, so that the storage space occupied by the file name index storage area is minimized.
进一步地,由于本实施方式中对子文件集中的各个子文件进行了预先分析。因此,可以针对分析结果进一步优化目标文件的文件结构。对于未发生碰撞的子文件,可将其文件信息存储区中的哈希碰撞下一个子文件信息的偏移信息删除,以节省更多资源。Furthermore, in this embodiment, each sub-file in the sub-file set is analyzed in advance. Therefore, the file structure of the target file can be further optimized based on the analysis results. For sub-files that have not collided, the offset information of the next sub-file information in the hash collision in the file information storage area can be deleted to save more resources.
在一个实施例中,参见图2,本申请还提供一种基于freeRTOS文件信息读取方法,所述方法包括:In one embodiment, referring to Figure 2, this application also provides a method for reading file information based on freeRTOS. The method includes:
获取要读取的子文件名;Get the subfile name to be read;
根据所述子文件名计算哈希值,得到该文件在文件名索引区的位置;Calculate the hash value according to the sub-file name to obtain the position of the file in the file name index area;
根据所述文件名索引区的位置得到对应文件信息区的存储偏移量;Obtain the storage offset of the corresponding file information area according to the position of the file name index area;
根据文件信息区的偏移量获取当前子文件的正文偏移量和文件大小;Get the text offset and file size of the current subfile based on the offset of the file information area;
根据当前子文件的正文偏移量和文件大小,定位到子文件正文位置并按所述文件大小读取所述子文件正文信息。According to the text offset and file size of the current sub-file, locate the sub-file text position and read the sub-file text information according to the file size.
可以看出,由于目标文件采用了上述结构,在文件读取时,可以通过子文件名的哈希值与目标文件内的文件个数进行求余以对子文件进行一个粗定位,在获取到该子文件在文件名索引区的位置后,通过对应文件信息区的存储偏移量,获取到当前子文件的正文偏移量和文件大小,进而定位到子文件正文位置并按所述文件大小读取所述子文件正文信息。如此,可以做到当需要读取子文件时,可以快速且精准的找到所需的子文件数据。It can be seen that since the target file adopts the above structure, when reading the file, the sub-file can be roughly positioned by taking the remainder of the hash value of the sub-file name and the number of files in the target file. After obtaining After the subfile is located in the file name index area, the text offset and file size of the current subfile are obtained through the storage offset of the corresponding file information area, and then the text position of the subfile is located and the file size is determined. Read the sub-file text information. In this way, when a subfile needs to be read, the required subfile data can be found quickly and accurately.
进一步地,由于可能存在碰撞的情况,因此所述根据文件信息区的存储偏移量获取当前子文件的正文偏移量和文件大小,包括:Further, since there may be a collision, the method of obtaining the text offset and file size of the current subfile based on the storage offset of the file information area includes:
根据文件信息区的存储偏移量定位;Positioning based on the storage offset of the file information area;
判断文件名是否相同,若相同,则读取正文偏移量信息和文件大小信息;Determine whether the file names are the same. If they are the same, read the text offset information and file size information;
若不相同,则获取当前文件信息区得到哈希碰撞下一个子文件信息的偏移量信息,循环寻找下一个文件信息区信息,直到找到当前子文件正文偏移量和文件大小为止。目标文件使用文件名的哈希值计算位置,可以快速定位到子文件的具体位置,减少查找的次数,加快访问子文件的速度。If they are not the same, obtain the current file information area to obtain the offset information of the next sub-file information of the hash collision, and loop to find the next file information area information until the current sub-file text offset and file size are found. The target file uses the hash value of the file name to calculate the location, which can quickly locate the specific location of the sub-file, reduce the number of searches, and speed up access to the sub-file.
相应地,本申请一个示例性实施例还提供了一种计算机设备,包括:处理器、存储器、网络接口、输入设备以及输出设备;处理器与存储器、网络接口相连,其中,网络接口用于提供网络通信功能,存储器用于存储程序代码,输入设备用于接收输入的指令,以产生与计算机设备的设置以及功能控制有关的信号输入,所述输出设备用于输出数据信息,处理器用于调用程序代码,以执行本申请实施例中的数据存储或读取方法。Correspondingly, an exemplary embodiment of the present application also provides a computer device, including: a processor, a memory, a network interface, an input device, and an output device; the processor is connected to the memory and the network interface, where the network interface is used to provide Network communication function, the memory is used to store program code, the input device is used to receive input instructions to generate signal input related to the setting and function control of the computer device, the output device is used to output data information, and the processor is used to call the program Code to execute the data storage or reading methods in the embodiments of this application.
此外,还应指出,本申请一个示例性实施例还提供了一种计算机可读存储介质,该存储介质中存储了计算机程序,该计算机程序包括程序指令,当一个或多个处理器加载并执行该程序指令,可以实现实施例中对前述数据存储或读取方法的描述,这里不再赘述,对采用相同方法的有益效果描述,也在此不再赘述。可以理解的是,程序指令可以被部署在一个或能够互相通信的多个计算机设备上执行。In addition, it should also be noted that an exemplary embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When one or more processors load and execute The program instructions can realize the description of the foregoing data storage or reading method in the embodiment, which will not be described again here. The description of the beneficial effects of using the same method will not be described again here. It will be appreciated that the program instructions may be deployed and executed on one or multiple computer devices capable of communicating with each other.
上述计算机可读存储介质可以是前述任一实施例提供的数据存储或读取方法的内部存储单元,例如计算机设备的硬盘或内存。该计算机可读存储介质也可以是该计算机设备的外部存储设备,例如该计算机设备上配备的插接式硬盘,智能存储卡(smart mediacard,SMC),安全数字(secure digital,SD)卡,闪存卡(flash card)等。进一步地,该计算机可读存储介质还可以既包括该计算机设备的内部存储单元也包括外部存储设备。该计算机可读存储介质用于存储该计算机程序以及该计算机设备所需的其他程序和数据。该计算机可读存储介质还可以用于暂时地存储已经输出或者将要输出的数据。The above-mentioned computer-readable storage medium may be an internal storage unit of the data storage or reading method provided in any of the foregoing embodiments, such as a hard disk or memory of a computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, or a flash memory equipped on the computer device. flash card, etc. Further, the computer-readable storage medium may also include both an internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.
本申请实施例方法中的步骤可以根据实际需要进行顺序调整、合并和删减。The steps in the methods of the embodiments of this application can be sequence adjusted, combined, and deleted according to actual needs.
本申请实施例装置中的模块可以根据实际需要进行合并、划分和删减。Modules in the device of the embodiment of the present application can be merged, divided, and deleted according to actual needs.
以上所揭露的仅为本申请的部分实施例而已,当然不能以此来限定本申请之权利范围,本领域普通技术人员可以理解实现上述实施例的全部或部分流程,并依本申请权利要求所作的等同变化,仍属于发明所涵盖的范围。What is disclosed above is only some of the embodiments of the present application. Of course, it cannot be used to limit the scope of rights of the present application. Those of ordinary skill in the art can understand all or part of the processes for implementing the above embodiments and make decisions according to the claims of the present application. Equivalent changes still fall within the scope of the invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311554890.6A CN117271440B (en) | 2023-11-21 | 2023-11-21 | File information storage method, reading method and related equipment based on freeRTOS |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311554890.6A CN117271440B (en) | 2023-11-21 | 2023-11-21 | File information storage method, reading method and related equipment based on freeRTOS |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117271440A true CN117271440A (en) | 2023-12-22 |
| CN117271440B CN117271440B (en) | 2024-02-06 |
Family
ID=89212799
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311554890.6A Active CN117271440B (en) | 2023-11-21 | 2023-11-21 | File information storage method, reading method and related equipment based on freeRTOS |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117271440B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118035246A (en) * | 2024-02-28 | 2024-05-14 | 中山大学 | A method and device for intelligent indexing based on query perception |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5754844A (en) * | 1995-12-14 | 1998-05-19 | Sun Microsystems, Inc. | Method and system for accessing chunks of data using matching of an access tab and hashing code to generate a suggested storage location |
| US20030233381A1 (en) * | 2002-06-13 | 2003-12-18 | Hitachi, Ltd. | File management method |
| US20080228843A1 (en) * | 2006-08-25 | 2008-09-18 | Dan Dodge | Filesystem having a filename cache |
| US20140095458A1 (en) * | 2012-09-28 | 2014-04-03 | Samsung Electronics Co., Ltd. | Storage device for storing directory entries, directory entry lookup apparatus and method, and storage medium storing directory entry lookup program |
| CN104978355A (en) * | 2014-04-10 | 2015-10-14 | 炬芯(珠海)科技有限公司 | Operation method of CPIO file and operation device of CPIO file |
| CN105069048A (en) * | 2015-07-23 | 2015-11-18 | 东方网力科技股份有限公司 | Small file storage method, query method and device |
| US20150331619A1 (en) * | 2012-12-14 | 2015-11-19 | Tencent Technology (Shenzhen) Company Limited | Data storage method and apparatus |
| WO2016199230A1 (en) * | 2015-06-10 | 2016-12-15 | 株式会社日立製作所 | Storage system |
| CN107679182A (en) * | 2017-09-29 | 2018-02-09 | 华为技术有限公司 | A kind of catalogue collocation method and device |
| WO2018121430A1 (en) * | 2016-12-26 | 2018-07-05 | 贵州白山云科技有限公司 | File storage and indexing method, apparatus, media, device and method for reading files |
| CN111258499A (en) * | 2020-01-08 | 2020-06-09 | 杭州电子科技大学 | A small file system with variable sectors |
| CN111984591A (en) * | 2019-05-21 | 2020-11-24 | 北京京东尚科信息技术有限公司 | File storage method, file reading method, file storage device, file reading device, equipment and computer readable storage medium |
| CN113220644A (en) * | 2021-05-28 | 2021-08-06 | 北京微纳星空科技有限公司 | File processing method, device, equipment and storage medium |
| CN115017098A (en) * | 2022-06-09 | 2022-09-06 | 重庆川仪速达机电有限公司 | A file system management method and device |
| CN115525605A (en) * | 2022-08-11 | 2022-12-27 | 上海睿赛德电子科技有限公司 | Transmission system and method of file system |
-
2023
- 2023-11-21 CN CN202311554890.6A patent/CN117271440B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5754844A (en) * | 1995-12-14 | 1998-05-19 | Sun Microsystems, Inc. | Method and system for accessing chunks of data using matching of an access tab and hashing code to generate a suggested storage location |
| US20030233381A1 (en) * | 2002-06-13 | 2003-12-18 | Hitachi, Ltd. | File management method |
| US20080228843A1 (en) * | 2006-08-25 | 2008-09-18 | Dan Dodge | Filesystem having a filename cache |
| US20140095458A1 (en) * | 2012-09-28 | 2014-04-03 | Samsung Electronics Co., Ltd. | Storage device for storing directory entries, directory entry lookup apparatus and method, and storage medium storing directory entry lookup program |
| US20150331619A1 (en) * | 2012-12-14 | 2015-11-19 | Tencent Technology (Shenzhen) Company Limited | Data storage method and apparatus |
| CN104978355A (en) * | 2014-04-10 | 2015-10-14 | 炬芯(珠海)科技有限公司 | Operation method of CPIO file and operation device of CPIO file |
| WO2016199230A1 (en) * | 2015-06-10 | 2016-12-15 | 株式会社日立製作所 | Storage system |
| CN105069048A (en) * | 2015-07-23 | 2015-11-18 | 东方网力科技股份有限公司 | Small file storage method, query method and device |
| WO2018121430A1 (en) * | 2016-12-26 | 2018-07-05 | 贵州白山云科技有限公司 | File storage and indexing method, apparatus, media, device and method for reading files |
| CN107679182A (en) * | 2017-09-29 | 2018-02-09 | 华为技术有限公司 | A kind of catalogue collocation method and device |
| CN111984591A (en) * | 2019-05-21 | 2020-11-24 | 北京京东尚科信息技术有限公司 | File storage method, file reading method, file storage device, file reading device, equipment and computer readable storage medium |
| CN111258499A (en) * | 2020-01-08 | 2020-06-09 | 杭州电子科技大学 | A small file system with variable sectors |
| CN113220644A (en) * | 2021-05-28 | 2021-08-06 | 北京微纳星空科技有限公司 | File processing method, device, equipment and storage medium |
| CN115017098A (en) * | 2022-06-09 | 2022-09-06 | 重庆川仪速达机电有限公司 | A file system management method and device |
| CN115525605A (en) * | 2022-08-11 | 2022-12-27 | 上海睿赛德电子科技有限公司 | Transmission system and method of file system |
Non-Patent Citations (1)
| Title |
|---|
| 曹海涛;余永红;: "基于multimap映射的动态内存分配算法探究", 电脑知识与技术, no. 30 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118035246A (en) * | 2024-02-28 | 2024-05-14 | 中山大学 | A method and device for intelligent indexing based on query perception |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117271440B (en) | 2024-02-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9377959B2 (en) | Data storage method and apparatus | |
| US10628200B2 (en) | Base state for thin-provisioned volumes | |
| CN107506447A (en) | A kind of small documents reading/writing method and system based on local file system | |
| CN113625973B (en) | Data writing method, device, electronic equipment and computer readable storage medium | |
| CN110908707A (en) | Resource packaging method, device, server and storage medium | |
| US20240061599A1 (en) | Method and system for processing file read-write service, device, and medium | |
| CN114217738A (en) | Dynamic queue type cyclic storage method, device, equipment and medium | |
| CN113836184A (en) | A business persistence method and device | |
| CN118796272A (en) | A memory access method, processor, electronic device and readable storage medium | |
| US20240264931A1 (en) | Storage optimization method and apparatus for distributed storage system | |
| CN107846327A (en) | A kind of processing method and processing device of network management performance data | |
| KR102425596B1 (en) | Systems and methods for low latency hardware memory management | |
| CN111694992A (en) | Data processing method and device | |
| CN117271440B (en) | File information storage method, reading method and related equipment based on freeRTOS | |
| CN111459474B (en) | Templated data processing method and device | |
| CN118885413A (en) | Data reading method and device | |
| CN117591018A (en) | A request processing method and device in a distributed object storage system | |
| WO2024119930A1 (en) | Scheduling method and apparatus, and computer device and storage medium | |
| CN115840654B (en) | Message processing method, system, computing device and readable storage medium | |
| CN115952177A (en) | Method and device for checking database consistency | |
| CN112835967B (en) | Data processing method, device, device and medium based on distributed storage system | |
| CN115309554A (en) | Debugging method, debugging system and computing device for application running based on compatible layer | |
| CN114490693A (en) | Data modification method, device, electronic device and storage medium | |
| CN113535714A (en) | Data storage method, reading method and computer equipment | |
| CN113326004A (en) | Efficient log centralization method and device in cloud computing environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 3401 Haofang Tianji Plaza Office Building, 11008 Beihuan Avenue, Nanlian Community, Nantou Street, Nanshan District, Shenzhen City, Guangdong Province 518000 Patentee after: SHENZHEN YUNXIGU TECHNOLOGY Co.,Ltd. Country or region after: China Address before: 518000 floor 2, building 16, Zhiheng Industrial Park, No. 15, Guankou Second Road, Anle community, Nantou street, Nanshan District, Shenzhen, Guangdong Patentee before: SHENZHEN YUNXIGU TECHNOLOGY Co.,Ltd. Country or region before: China |
|
| CP03 | Change of name, title or address |