CN118264256A - Data coding method, device, hardware acceleration card, program product and medium - Google Patents

Abstract

The invention discloses a data coding method, a device, a hardware acceleration card, a program product and a medium, relates to the field of data processing, and is used for solving the problem that an application scene is limited when the hardware acceleration card realizes a compression algorithm. The scheme obtains channel number configuration information determined based on state parameters of a hardware accelerator card and/or demand instructions of a user, and configures the number of channels to be the number of target channels; after the computing module determines the matching information according to the data to be encoded and the encoded data, the matching information and the data to be encoded are output to the encoding module through the channel so as to encode the data to be encoded according to the matching information. In the invention, when the hardware accelerator card realizes the compression algorithm, the number of channels can be dynamically adjusted according to different scenes and requirements, so that the flexibility and applicability are improved, and the application scene is prevented from being limited; by the configuration and matching of the number of channels, the number of channels can be utilized more effectively, the efficiency and performance of a compression algorithm are improved, and the load of a hardware acceleration card is reduced.

Description

A data encoding method, device, hardware acceleration card, program product and medium

Technical Field

The present invention relates to the field of data processing, and in particular to a data encoding method, device, hardware acceleration card, program product and medium.

Background technique

Data compression is currently mainly implemented using software, but it faces problems such as low compression efficiency, heavy processor burden and low security. Therefore, hardware accelerator cards are becoming increasingly popular as coprocessors to complete data compression. Hardware accelerator cards are mainly used to execute the Gzip compression algorithm, which uses a fixed-size sliding window to process repeated strings in the data to be encoded and obtains the best matching information through dictionary matching. The matching information is then output to the encoding module through the channel provided by the encoding module, so that the encoding module encodes the data to be encoded according to the matching information.

However, when the hardware acceleration card implements the Gzip algorithm, after the computing module calculates the matching information, the output matching information will vary according to the number of channels provided by the back-end encoding module. Currently, the number of channels provided by the back-end encoding module is usually fixed, so the scenario in which the hardware acceleration card implements the compression algorithm is very limited.

Summary of the invention

The purpose of the present invention is to provide a data encoding method, device, hardware acceleration card, program product and medium. When the hardware acceleration card implements the compression algorithm, the number of channels can be dynamically adjusted according to different scenarios and requirements, thereby improving flexibility and applicability and avoiding limitations on application scenarios. By configuring and matching the number of channels, the number of channels can be more effectively utilized, the efficiency and performance of the compression algorithm can be improved, and the burden on the hardware acceleration card can be reduced.

In a first aspect, the present invention provides a data encoding method, which is applied to a processor in a hardware acceleration card, wherein the hardware acceleration card further includes a computing module and an encoding module, and the data encoding method includes:

Obtain channel quantity configuration information determined based on status parameters of the hardware acceleration card and/or user demand instructions;

Configuring the number of channels of the encoding module to be the target number of channels according to the channel number configuration information;

After the calculation module determines matching information according to the data to be encoded and the encoded data, the matching information and the data to be encoded are output to the encoding module through the channels of the target number of channels, triggering the encoding module to encode the data to be encoded according to the matching information;

The matching information is the matching distance and matching length of the preset character string in the encoded data when the preset character string in the data to be encoded is the same as the preset character string in the encoded data.

Wherein, obtaining the channel quantity configuration information determined based on the state parameters of the hardware acceleration card and/or the user's demand instruction includes:

Determine the number of test channels according to the user's demand instructions, and configure the number of channels of the encoding module to the number of test channels;

estimating a first computing resource occupancy of the hardware acceleration card under the number of test channels;

The number of test channels is adjusted according to the first computing resource occupancy to obtain a target number of channels.

The step of adjusting the number of test channels according to the first computing resource occupancy to obtain a target number of channels includes:

Comparing the first computing resource occupancy with a first preset threshold;

If the first computing resource usage is greater than the first preset threshold, reducing the number of test channels to obtain the target number of channels;

If the first computing resource usage is less than the first preset threshold, increasing the number of test channels to obtain the target number of channels;

If the first computing resource occupancy is equal to the first preset threshold, the number of test channels is determined to be the target number of channels.

After comparing the first computing resource usage with a first preset threshold, the method further includes:

Calculating a difference between the first computing resource usage and the first preset threshold, and calculating a ratio of the difference to the first preset threshold;

Determine the change in the number of channels according to the ratio and a preset ratio-change correspondence;

If the first computing resource usage is greater than the first preset threshold, reducing the number of test channels to obtain the target number of channels includes:

If the first computing resource usage is greater than the first preset threshold, subtracting the channel quantity change amount from the test channel quantity to obtain the target channel quantity;

If the first computing resource usage is less than the first preset threshold, increasing the number of test channels to obtain the target number of channels includes:

If the first computing resource occupancy is less than the first preset threshold, the target channel number is obtained by adding the test channel number to the channel number change.

When it is determined that the first computing resource usage is greater than the first preset threshold, the method further includes:

Comparing the first computing resource usage with a second preset threshold, where the second preset threshold is greater than the first preset threshold;

If the first computing resource occupancy is greater than the second preset threshold, a reconfiguration signal is fed back to the user so that the user redetermines the number of test channels based on the reconfiguration signal and re-enters the step of configuring the number of channels of the encoding module to the number of test channels.

Before calculating the difference between the first computing resource usage and the first preset threshold, the method further includes:

Determine an application platform of the hardware acceleration card, and determine an amount of second computing resources occupied by the hardware acceleration card for executing a preset task on the application platform;

Determine the remaining amount of computing resources of the hardware acceleration card according to the second computing resource occupation, wherein the sum of the second computing resource occupation and the remaining amount of computing resources is not greater than all computing resources of the hardware acceleration card;

The first preset threshold is determined according to the remaining amount of computing resources.

Wherein, obtaining the channel quantity configuration information determined based on the state parameters of the hardware acceleration card and/or the user's demand instruction includes:

Determine an application platform of the hardware acceleration card, and determine an amount of second computing resources occupied by the hardware acceleration card for executing a preset task on the application platform;

Determine the remaining amount of computing resources of the hardware acceleration card according to the second computing resource occupation, wherein the sum of the second computing resource occupation and the remaining amount of computing resources is not greater than all computing resources of the hardware acceleration card;

The target number of channels is determined according to the remaining amount of computing resources.

Wherein, obtaining the channel quantity configuration information determined based on the state parameters of the hardware acceleration card and/or the user's demand instruction includes:

Acquire, through the first interface, channel quantity configuration information determined based on the state parameters of the hardware acceleration card and/or the user's demand instruction;

Using an interface conversion device to convert the channel quantity configuration information acquired by the first interface into the channel quantity configuration information corresponding to the second interface;

Refresh the channel quantity configuration information corresponding to the second interface into the channel quantity configuration register;

Configuring the number of channels of the encoding module to be the target number of channels according to the channel number configuration information includes:

According to the channel number configuration information in the channel number configuration register, the channel number of the encoding module is configured as the target channel number.

Wherein, all bytes in the data to be encoded are divided into N groups, N is an integer greater than 1, and the number of bytes in each group is equal to the number of target channels; and matching information is determined according to the data to be encoded and the encoded data, including:

Taking each byte in each group of the bytes as a starting byte, N groups of the bytes simultaneously search the encoded data for a character string that matches each byte in the group as the starting byte, wherein the length of the character string is at least 2;

Recording the matching information of the character string that matches each byte as the starting byte, the matching information includes the matching length of the matched character string and the matching distance of the first byte of the character string in the encoded data;

Determine the best matching information according to the matching information corresponding to each byte in the N groups in the data to be encoded;

Outputting the matching information and the data to be encoded to the encoding module through the target number of channels includes:

The data result set corresponding to the optimal matching information is output to the encoding module through the channels of the target number of channels, and the data result set includes the optimal matching information and the data to be encoded.

Wherein, determining the optimal matching information according to the matching information corresponding to each byte in the N groups in the to-be-encoded data includes:

Polling starts from the first group, and determining the address of the current byte in the next comparison process according to the first matching length of the current byte in the current comparison process and the second matching length of the next byte;

The matching information of the current byte in the comparison process that meets the preset end condition is determined as the optimal matching information.

Before determining the matching information of the current byte in the comparison process that meets the preset end condition as the best matching information, the method further includes:

Determine whether the address offset of the current byte in the next comparison process is greater than the number of target channels;

If it is greater than the target channel number, determining that the current byte in the current comparison process meets the preset end condition;

If it is less than the target channel quantity, it is determined that the current byte of the current comparison process does not meet the preset end condition.

Wherein, determining the address of the current byte in the next comparison process according to the first matching length of the current byte in the current comparison process and the second matching length of the next byte includes:

When the first matching length is smaller than the second matching length, the current byte in the current comparison process is output, and the address of the current byte in the current comparison process is increased by one as the address of the current byte in the next comparison process.

Wherein, determining the address of the current byte in the next comparison process according to the first matching length of the current byte in the current comparison process and the second matching length of the next byte includes:

When the first match length is not less than the second match length, the first match length is output, and the address of the current byte in the current comparison process plus the address after the first match length is used as the address of the current byte in the next comparison process.

Among them, it also includes:

If the address of the current byte is equal to the maximum channel address, cache the matching information of the current byte;

When entering the comparison process of the next group of bytes, constructing a virtual 0 address, in which the matching information of the bytes of the maximum channel address is stored;

The matching information of the byte of the maximum channel address stored in the virtual 0 address is used as the matching information of the current byte of the current comparison process, and the step of determining the address of the current byte in the next comparison process according to the first matching length of the current byte in the current comparison process and the second matching length of the next byte is entered.

The step of outputting the data result set corresponding to the optimal matching information to the encoding module through the target number of channels includes:

Writing the data result set corresponding to the optimal matching information into a ping-pong register group with a depth of m, where m is a positive integer multiple of N;

When the N channel flags of the ping-pong register group are valid, the encoding module is triggered to read the data result set through the channels of the target number of channels.

There are multiple ping-pong register groups, and writing the data result set corresponding to the optimal matching information into a ping-pong register group with a depth of m includes:

When the encoding module reads the data result set stored in one of the ping-pong register groups through the channels of the target number of channels, the data result set corresponding to the best matching information is written into other ping-pong register groups that have not been read.

In a second aspect, the present invention provides a data encoding device, which is applied to a hardware acceleration card, wherein the hardware acceleration card further includes a computing module and an encoding module, and the data encoding device includes:

Memory for storing computer programs;

A processor is used to implement the steps of the above-mentioned data encoding method when storing a computer program.

In a third aspect, the present invention provides a hardware acceleration card, comprising the data encoding device described above, and further comprising a computing module and an encoding module;

The calculation module is used to determine matching information according to the data to be encoded and the encoded data, and output the matching information and the data to be encoded to the encoding module through channels of the target number of channels;

The encoding module is used to encode the data to be encoded according to the matching information;

The matching information is the matching distance and matching length of the preset character string in the encoded data when the preset character string in the data to be encoded is the same as the preset character string in the encoded data.

In a fourth aspect, the present invention provides a computer program product, comprising a computer program/instructions, which implement the steps of the above-mentioned data encoding method when executed by a processor.

In a fifth aspect, the present invention provides a non-volatile storage medium having a computer program stored thereon, and the computer program, when executed by a processor, implements the steps of the above-mentioned data encoding method.

The present invention provides a data encoding method, device, hardware acceleration card, program product and medium, which relate to the field of data processing and are used to solve the problem of limited application scenarios when hardware acceleration cards implement compression algorithms. The solution obtains channel number configuration information determined based on the state parameters of the hardware acceleration card and/or the user's demand instructions, and configures the channel number as the target channel number; after the calculation module determines the matching information according to the data to be encoded and the encoded data, the matching information and the data to be encoded are output to the encoding module through the channel to encode the data to be encoded according to the matching information. In the present invention, when the hardware acceleration card implements the compression algorithm, the number of channels can be dynamically adjusted according to different scenarios and requirements, which improves flexibility and applicability and avoids limited application scenarios; through channel number configuration and matching, the number of channels can be more effectively utilized, the efficiency and performance of the compression algorithm can be improved, and the burden on the hardware acceleration card can be reduced.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the prior art and the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

FIG1 is a flow chart of a data encoding method provided by the present invention;

FIG2 is a hardware architecture diagram of searching for optimal matching information provided by the present invention;

FIG3 is a flow chart of searching for optimal matching information provided by the present invention;

FIG4 is a schematic diagram of a specific embodiment of searching for optimal matching information provided by the present invention;

FIG5 is a schematic diagram of a data encoding device provided by the present invention;

FIG6 is a schematic diagram of a hardware acceleration card provided by the present invention;

FIG. 7 is a schematic diagram of a non-volatile storage medium provided by the present invention.

Detailed ways

The core of the present invention is to provide a data encoding method, device, hardware acceleration card, program product and medium. When the hardware acceleration card implements the compression algorithm, the number of channels can be dynamically adjusted according to different scenarios and requirements, thereby improving flexibility and applicability and avoiding limitations on application scenarios. By configuring and matching the number of channels, the number of channels can be more effectively utilized, thereby improving the efficiency and performance of the compression algorithm and reducing the burden on the hardware acceleration card.

In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

First, lazy matching is explained. The process of calculating matching information by the calculation module is as follows: using a "sliding window" of a constant size to process the strings in the data to be encoded that are repeated with the strings in the encoded data, one or more matching information (including matching length and matching distance) can be used to represent this segment of matching string after dictionary matching. Finally, after chain point screening, a matching information with the longest matching length is selected. If there are multiple matching information with the same longest matching length, a matching information with the shortest matching distance is selected. Lazy matching is an improvement on the above-mentioned matching information calculation algorithm. After finding the matching information with the longest matching length, it is not used immediately, but compared with the matching information of the next byte to determine whether it is longer than the matching length of the current string. If so, the matching information of the next byte is selected. If not, it continues to be compared with the matching length of the next byte, and so on. The cycle continues until a matching information longer than the matching length of the current byte is found.

In a first aspect, as shown in FIG. 1 , the present invention provides a data encoding method, which is applied to a processor in a hardware acceleration card. The hardware acceleration card further includes a computing module and an encoding module. The data encoding method includes:

S11: Obtain channel quantity configuration information determined based on status parameters of the hardware acceleration card and/or user demand instructions.

Specifically, in the process of hardware accelerator cards implementing compression algorithms such as Gzip, the number of channels (or "parallelism") is a key parameter that determines the degree of parallelism when data is processed inside the hardware accelerator card. The choice of the number of channels directly affects compression efficiency, throughput, and resource utilization. Therefore, dynamically determining the channel number configuration information based on the status parameters of the hardware accelerator card or the user's demand instructions is crucial to optimizing the performance of the hardware accelerator card.

When determining the channel number configuration information based on the status parameters of the hardware accelerator card, the status parameters of the hardware accelerator card may include a variety of information, such as: Hardware resource utilization: If the utilization of certain resources of the hardware accelerator card (such as memory, computing unit, etc.) is low, the number of channels can be increased to improve the degree of parallelization, thereby improving compression efficiency; Power consumption and heat dissipation: In some cases, an excessively high number of channels may increase the power consumption of the hardware accelerator card and even cause overheating problems. Therefore, it is reasonable to dynamically adjust the number of channels based on the power consumption and heat dissipation status; Task queue length: If the task queue of the hardware accelerator card is long, it means that there is more data waiting to be processed. At this time, increasing the number of channels can speed up the processing speed and reduce the waiting time. By monitoring and evaluating these status parameters in real time, the appropriate channel number configuration information can be dynamically determined to adapt to the current hardware status and workload.

When determining the channel number configuration information based on the user's demand instructions, in some scenarios, the user may need to specify the channel number based on specific application requirements. For example, some applications may have strict requirements on compression speed, while other applications may pay more attention to compression ratio. By providing a user interface or API (Application Programming Interface), users can send channel number configuration instructions to the hardware accelerator card according to their needs. After receiving these instructions, the hardware accelerator card will adjust the number of channels of the encoding module accordingly.

It can be seen that this step dynamically determines the appropriate channel quantity configuration information to optimize the performance of the hardware accelerator card by real-time monitoring of the hardware accelerator card's status parameters or receiving user demand instructions. This dynamic configuration method can enable the hardware accelerator card to maintain high efficiency and performance under different workloads and hardware conditions.

S12: configuring the number of channels of the encoding module to be the target number of channels according to the channel number configuration information.

In a hardware accelerator card, the number of channels of the encoding module is usually fixed, but the state parameters of the hardware accelerator card and user demand instructions may cause the number of channels to be dynamically adjusted. Therefore, the purpose of this step is to configure the number of channels of the encoding module to the target number of channels that best matches the current demand based on this information.

In general, the core principle of this step is to dynamically adjust the channel number configuration of the encoding module according to the status parameters of the hardware acceleration card and user demand instructions to adapt to different compression scenarios and requirements. This can improve the flexibility and applicability of the hardware acceleration card, so that it can play the best compression performance in various application scenarios.

S13: After the calculation module determines the matching information according to the data to be encoded and the encoded data, the matching information and the data to be encoded are output to the encoding module through the channels of the target number of channels, triggering the encoding module to encode the data to be encoded according to the matching information.

The matching information is the matching distance and matching length of the preset character string in the encoded data when the preset character string in the data to be encoded is the same as the preset character string in the encoded data.

Specifically, this step includes the following key operations: Determination of matching information: In the calculation module, a matching operation is performed on the data to be encoded and the encoded data to determine the matching information. This matching information usually includes the matching distance and matching length of the preset string in the encoded data. For example, in the Gzip algorithm, by searching for matching strings in the encoded data, it is determined whether there are similar substrings in the data to be encoded, as well as their matching distances and lengths.

Channel output: Once the matching information is determined, the computing module outputs the matching information and the data to be encoded to the encoding module through the target number of channels. The target number of channels here is determined according to step S12 to adapt to the current hardware acceleration card status and user needs.

Coding trigger: After the coding module receives the matching information and the data to be coded output by the calculation module, it triggers the coding operation based on this information. Specifically, the coding module will perform corresponding coding processing on the data to be coded based on the matching information, such as using a compression algorithm to compress the matching substring, thereby achieving data compression and coding.

In general, this step realizes the process of data encoding in the hardware acceleration card by outputting the matching information and the data to be encoded to the encoding module, which can fully utilize the advantages of hardware acceleration and improve the efficiency and performance of data encoding.

In summary, the data encoding method provided by the present invention obtains the channel number configuration information determined based on the state parameters of the hardware acceleration card and/or the user's demand instructions, and configures the channel number as the target channel number; after the calculation module determines the matching information according to the data to be encoded and the encoded data, the matching information and the data to be encoded are output to the encoding module through the channel to encode the data to be encoded according to the matching information. In the present invention, when the hardware acceleration card implements the compression algorithm, the number of channels can be dynamically adjusted according to different scenarios and requirements, which improves flexibility and applicability and avoids limitation of application scenarios; through the configuration and matching of the number of channels, the number of channels can be more effectively utilized, the efficiency and performance of the compression algorithm can be improved, and the burden on the hardware acceleration card can be reduced.

In one embodiment, obtaining channel number configuration information determined based on status parameters of a hardware acceleration card and/or a user's demand instructions includes: determining the number of test channels according to the user's demand instructions, and configuring the number of channels of an encoding module as the number of test channels; estimating a first computing resource occupancy of the hardware acceleration card under the number of test channels; and adjusting the number of test channels according to the first computing resource occupancy to obtain a target number of channels.

This embodiment describes how to determine the target channel number configuration information according to the state parameters of the hardware acceleration card and/or the user's demand instructions, aiming to ensure efficient and stable encoding performance on the hardware acceleration card. First, receive the user's demand instruction on the number of channels. This demand may be based on the user's specific requirements for encoding speed, compression rate or hardware resource utilization, or it may be the number of test channels set directly. According to this demand instruction, an initial "number of test channels" is set. Based on the number of test channels parsed by the user's demand instruction, the number of channels of the encoding module in the hardware acceleration card is configured to this value. This is to simulate or test the hardware acceleration card before actual operation. When the encoding module runs with the number of test channels, monitor and estimate the computing resource usage of the hardware acceleration card under these numbers of channels. This estimate may include the usage of resources such as CPU (Central Processing Unit), memory, bandwidth, etc. Through this step, the resource utilization efficiency and possible bottlenecks of the hardware acceleration card under the current configuration can be understood. According to the estimated first computing resource usage, evaluate whether the current configuration meets the user's needs and whether it will lead to excessive use or waste of hardware resources. If the estimated result shows that the resource usage is too high, the number of test channels may be reduced to reduce resource consumption; if the resource usage is low, the number of test channels may be increased to improve encoding efficiency. After a series of adjustments and evaluations, a target number of channels is finally determined. This number not only meets the needs of users, but also ensures the stable and efficient operation of the hardware accelerator card.

In summary, this embodiment allows the system to dynamically adjust the number of channels of the encoding module according to the real-time status of the hardware acceleration card and the specific needs of the user. This can not only improve the encoding efficiency, but also avoid the waste and overload of hardware resources, thereby extending the service life of the hardware acceleration card. In addition, through the direct participation of user needs, the system can also provide more personalized and customized encoding services.

In one embodiment, the number of test channels is adjusted according to the first computing resource occupancy to obtain the target channel number, including: comparing the first computing resource occupancy with a first preset threshold; if the first computing resource occupancy is greater than the first preset threshold, reducing the number of test channels to obtain the target channel number; if the first computing resource occupancy is less than the first preset threshold, increasing the number of test channels to obtain the target channel number; if the first computing resource occupancy is equal to the first preset threshold, determining the number of test channels to be the target channel number.

This embodiment describes how to estimate the computing resource usage of the hardware acceleration card under a specific number of test channels, and dynamically adjust the number of test channels based on the comparison result between the estimated value and the target threshold, so as to obtain the optimal target number of channels. The core idea of this embodiment is to gradually approach the optimal channel number configuration by simulating or estimating the computing resource usage of the hardware acceleration card under different numbers of channels, and comparing the estimated result with the target threshold.

First, an initial number of test channels is determined according to the user's demand instructions, and the number of channels of the encoding module is configured to this number of test channels. Then, the computing resource occupancy of the hardware acceleration card under this number of test channels (i.e., the first computing resource occupancy) is estimated by simulation or actual operation. Next, the estimated first computing resource occupancy is compared with the first preset threshold. This first preset threshold is usually set according to the performance, power consumption, temperature and other parameters of the hardware acceleration card, and represents a safe or ideal upper limit of computing resource occupancy. If the estimated first computing resource occupancy is greater than the first preset threshold, it means that the number of test channels currently configured is too large, resulting in excessive computing resource occupancy of the hardware acceleration card, which may cause performance degradation, increased power consumption, overheating and other problems. Therefore, it is necessary to reduce the number of test channels and re-estimate the computing resource occupancy until a suitable channel number configuration that is less than or equal to the first preset threshold is found. If the estimated first computing resource occupancy is less than the first preset threshold, it means that the number of test channels currently configured still has room for improvement, and the number of channels can be further increased to improve encoding efficiency. Therefore, the number of test channels can be increased, and the computing resource occupancy can be estimated again to find a more optimal channel number configuration. After many iterations and adjustments, we will eventually find a channel number configuration that meets both user needs and the performance requirements of the hardware accelerator card. This configuration is the target channel number.

In a preferred embodiment, the first preset threshold value can be a fixed value or a range. If the first preset threshold value is a range, such as a preset range, the first computing resource occupancy is compared with the maximum value and the minimum value in the preset range. At this time, if it is greater than the maximum value, the step of reducing the number of test channels to obtain the target number of channels is entered; if it is less than the minimum value, the step of increasing the number of test channels to obtain the target number of channels is entered; if it is not greater than the maximum value and not less than the minimum value, the number of test channels is determined as the target number of channels.

In summary, this embodiment can dynamically adjust the number of channels according to the real-time status of the hardware accelerator card and user needs, achieving more efficient resource utilization and better encoding performance. By setting a reasonable preset threshold, problems such as hardware accelerator card overload and overheating caused by too many channels can be avoided, ensuring the stability and security of the system. It can adapt to changes in different application scenarios and user needs, and provide flexible channel number configuration options.

In one embodiment, after comparing the first computing resource occupancy with the first preset threshold, it also includes: calculating the difference between the first computing resource occupancy and the first preset threshold, and calculating the ratio of the difference to the first preset threshold; determining the change in the number of channels according to the ratio and the preset ratio-change correspondence; if the first computing resource occupancy is greater than the first preset threshold, reducing the number of test channels to obtain the target number of channels, including: if the first computing resource occupancy is greater than the first preset threshold, subtracting the change in the number of channels from the number of test channels to obtain the target number of channels; if the first computing resource occupancy is less than the first preset threshold, increasing the number of test channels to obtain the target number of channels, including: if the first computing resource occupancy is less than the first preset threshold, adding the change in the number of channels to the number of test channels to obtain the target number of channels.

This embodiment describes how to dynamically determine the optimal number of encoding module channels based on the status parameters of the hardware accelerator card and user demand instructions. The core principle is to optimize performance, avoid resource overload, and maximize hardware utilization by estimating and adjusting the computing resource usage of the encoding module under different numbers of channels.

Specifically, it has been explained in the above embodiment that if the first computing resource occupancy is greater than the first preset threshold, it means that the number of channels currently configured is too large and needs to be reduced; conversely, if the first computing resource occupancy is less than the first preset threshold, it means that the number of channels currently configured may be too small and can be considered to be increased. In order to adjust the number of channels more accurately, this embodiment also needs to calculate the difference between the first computing resource occupancy and the first preset threshold, and calculate the ratio of this difference to the first preset threshold. This ratio reflects the degree to which the current computing resource occupancy exceeds or falls below the preset threshold. Based on the ratio calculated above, combined with a preset ratio-variation correspondence (this correspondence may be a lookup table, a function or other form of mapping relationship), the change in the number of channels that should be increased or decreased is determined. This change is usually set according to the performance and stability requirements of the hardware accelerator card to ensure that the adjusted number of channels is neither overloaded nor too small. Finally, according to the determined change, the initial number of test channels is adjusted to obtain the target number of channels. If the first computing resource usage is greater than the first preset threshold, the number of test channels is subtracted from the channel number change; if the first computing resource usage is less than the first preset threshold, the number of test channels is added to the channel number change. In this way, a target channel number that meets performance requirements and avoids resource overload is obtained.

Similarly, if the first preset threshold is a range (such as a preset range), if the first computing resource occupancy is greater than the maximum value, then the difference calculated in this embodiment is the difference between the first computing resource occupancy and the maximum value; if the first computing resource occupancy is less than the minimum value, then the difference in this embodiment is the difference between the first computing resource occupancy and the minimum value.

In summary, this embodiment can flexibly configure the number of channels according to the actual status of the hardware acceleration card and user requirements, thereby achieving better performance and resource utilization.

In one embodiment, when it is determined that the first computing resource occupancy is greater than a first preset threshold, it also includes: comparing the first computing resource occupancy with a second preset threshold, the second preset threshold is greater than the first preset threshold; if the first computing resource occupancy is greater than the second preset threshold, then feeding back a reconfiguration signal to the user, so that the user re-determines the number of test channels based on the reconfiguration signal, and re-enters the step of configuring the number of channels of the encoding module to the number of test channels.

In this embodiment, the first preset threshold is a critical value for determining whether the hardware acceleration card may face performance degradation or unstable operation due to excessive resource usage. When the estimated first computing resource usage exceeds this threshold, it means that the current configuration of the number of test channels may be too aggressive and needs to be adjusted to avoid potential performance problems. However, directly reducing the number of test channels is not always the best solution, especially when the resource usage is only slightly higher than the first preset threshold, and there is still potential to improve performance by further optimizing the algorithm or adjusting other parameters. Therefore, this embodiment introduces an additional judgment step, namely, comparison with the second preset threshold.

The second preset threshold is a higher threshold, which usually represents the limit of severe performance degradation or unstable operation that the hardware accelerator may face. When the first computing resource usage exceeds this threshold, it means that the current configuration of the number of test channels is too conservative and may even severely limit the performance of the hardware accelerator. In this case, simply reducing the number of test channels may not be the best choice, but requires the user to reconsider and configure a more suitable number of channels.

Therefore, when the first computing resource usage is greater than the first preset threshold but less than the second preset threshold, an attempt is made to reduce the resource usage by reducing the number of test channels. However, when the first computing resource usage exceeds the second preset threshold, a reconfiguration signal is fed back to the user, prompting the user that there may be a problem with the current channel number configuration, and suggesting that the user re-determine the number of test channels based on this signal. In this way, the user can reconfigure a more appropriate number of channels based on the system feedback and their own needs to ensure that the hardware accelerator card can run efficiently and stably.

In one embodiment, before calculating the difference between the first computing resource occupancy and the first preset threshold, the method further includes: determining an application platform of the hardware acceleration card, and determining a second computing resource occupancy required for the hardware acceleration card to execute a preset task on the application platform; determining a remaining computing resource of the hardware acceleration card according to the second computing resource occupancy, wherein the sum of the second computing resource occupancy and the remaining computing resource is not greater than all computing resources of the hardware acceleration card; and determining the first preset threshold according to the remaining computing resource.

This embodiment describes how to determine the first preset threshold value based on the application platform of the hardware acceleration card and the computing resources required to perform the preset task. This step is to more accurately evaluate and adjust the number of channels of the encoding module to ensure that the encoding requirements can be met within the limited computing resources of the hardware acceleration card, while ensuring that other preset tasks can run normally.

First, it is necessary to clarify the specific platform or environment in which the hardware accelerator card is used. Different platforms may have different requirements and usage scenarios for hardware accelerator cards. For example, platforms such as servers, data centers, or edge devices may have different performance and functional requirements for hardware accelerator cards.

After determining the application platform, it is necessary to identify the preset tasks that the hardware accelerator card needs to perform on the platform and estimate the computing resource usage required for these tasks. These preset tasks may include encoding, decoding, data processing, etc. By quantifying the computing resource requirements for these tasks, a reference can be provided for subsequent computing resource allocation.

After knowing the computing resources required for the preset task, the remaining computing resources of the hardware acceleration card are further determined. The remaining computing resources refer to the remaining computing resources of the hardware acceleration card after removing the resources required to perform the preset task. This remaining amount will be used to dynamically configure the number of channels of the encoding module to meet the encoding requirements.

Based on the remaining amount of computing resources, a first preset threshold can be determined. This threshold is used to determine whether the current configuration will cause hardware acceleration card overload or resource waste when configuring the number of encoding module channels. If the estimated first computing resource occupancy exceeds this threshold, it means that the number of channels currently configured is too large, which may cause the performance of the hardware acceleration card to degrade or other preset tasks to fail to run normally; if the estimated occupancy is lower than the threshold, you can consider increasing the number of channels to improve encoding efficiency.

The method provided in this embodiment for determining and adjusting the number of encoding module channels based on the hardware acceleration card application platform and preset task requirements ensures that the hardware acceleration card can operate efficiently and stably under limited resources.

In one embodiment, obtaining channel quantity configuration information determined based on a state parameter of the hardware acceleration card and/or a user's demand instruction includes: determining an application platform of the hardware acceleration card, and determining a second computing resource occupancy amount required for the hardware acceleration card to execute a preset task on the application platform; determining a remaining computing resource amount of the hardware acceleration card according to the second computing resource occupancy amount, wherein a sum of the second computing resource occupancy amount and the remaining computing resource amount is not greater than all computing resources of the hardware acceleration card; and determining a target channel quantity according to the remaining computing resource amount.

In another embodiment, the target channel parameters can also be determined directly based on the state parameters of the hardware accelerator card (here specifically the application platform and the computing resources required for the preset task) and possible user needs. Specifically, the application platform refers to the environment where the hardware accelerator card will be deployed and run, such as a server, workstation, data center, etc. Different application platforms may have different workloads and performance requirements. Determining the application platform is an important step in understanding the operating environment of the hardware accelerator card.

Preset tasks refer to a series of operations or computing tasks that the hardware accelerator card needs to perform on the application platform (tasks that are fixed to be performed by the hardware accelerator card itself or tasks that other modules connected to the hardware accelerator card on the application platform need to perform using the hardware accelerator card). In order to determine the target number of channels, it is necessary to know how much computing resources these preset tasks require. This is usually estimated through testing, simulation or historical data.

Once the computing resource usage required for the preset task is known (called the second computing resource usage), this value can be subtracted from the total computing resources of the hardware accelerator card to obtain the remaining computing resources. This remaining amount indicates how much computing resources are left for other tasks (such as encoding tasks) when the hardware accelerator card does not perform the preset task.

After obtaining the remaining amount of computing resources, the target number of channels for the encoding module can be determined based on this value. The determination of the target number of channels needs to take into account the working efficiency and performance requirements of the encoding module itself. Too many channels may exceed the carrying capacity of the remaining computing resources, resulting in performance degradation; while too few channels may not be able to fully utilize the remaining computing resources, resulting in resource waste. Therefore, it is necessary to find a balance point (that is, the target number of channels determined based on the remaining computing resources) so that the target number of channels can meet the encoding performance requirements without exceeding the computing resource limitations of the hardware accelerator card.

In one embodiment, obtaining channel quantity configuration information determined based on the state parameters of the hardware acceleration card and/or the user's demand instructions includes: obtaining channel quantity configuration information determined based on the state parameters of the hardware acceleration card and/or the user's demand instructions through a first interface; using an interface conversion device to convert the channel quantity configuration information obtained by the first interface into channel quantity configuration information corresponding to a second interface; refreshing the channel quantity configuration information corresponding to the second interface into a channel quantity configuration register; and configuring the channel quantity of the encoding module to a target channel quantity according to the channel quantity configuration information, including: configuring the channel quantity of the encoding module to the target channel quantity according to the channel quantity configuration information in the channel quantity configuration register.

In this embodiment, the first interface: the hardware acceleration card may have multiple interfaces to receive external signals or instructions, wherein the first interface is used to receive state parameters based on the hardware acceleration card or user demand instructions; these parameters or instructions may come from a system monitor, a user interface, a control software, etc. Interface conversion device: Since different interfaces may use different data formats or communication protocols, an interface conversion device is required to convert the channel quantity configuration information received by the first interface into a data format corresponding to the second interface (usually an interface that directly communicates with the encoding module); this step ensures the accuracy and compatibility of the information. Channel quantity configuration register: The hardware acceleration card usually contains multiple registers to store and manage various configuration information; wherein the channel quantity configuration register is specifically used to store the channel quantity configuration information of the encoding module; by refreshing this register, it can be ensured that the encoding module can work according to the latest configuration information. Refresh operation: write the channel quantity configuration information corresponding to the second interface into the channel quantity configuration register to complete the register refresh; this operation is usually performed when the system is initialized, or dynamically performed when a new configuration instruction is received. Configuration process: After refreshing the channel number configuration register, the hardware accelerator card will configure the number of channels of the encoding module according to the configuration information in the register; specifically, it may send a series of control signals to the encoding module to instruct it to turn on or off a certain number of channels.

This embodiment realizes the dynamic configuration of the number of channels of the encoding module according to the state parameters of the hardware acceleration card and/or user demand instructions through a series of steps such as interface communication, information conversion and register refresh. This dynamic configuration method can improve the flexibility and adaptability of the hardware acceleration card, so that it can better meet the needs of various application scenarios.

In one embodiment, all bytes in the data to be encoded are divided into N groups, N is an integer greater than 1, and the number of bytes in each group is equal to the target number of channels; matching information is determined based on the data to be encoded and the encoded data, including: taking each byte in each group of bytes as a starting byte, and simultaneously searching the N groups of bytes in the encoded data to see whether there is a character string that matches each byte in the group as a starting byte; recording matching information of the character string that matches each byte as the starting byte, the matching information includes the matching length of the matched character string and the matching distance of the first byte of the character string in the encoded data; determining optimal matching information based on the matching information corresponding to each byte in the N groups in the data to be encoded; outputting the matching information and the data to be encoded to the encoding module through the channels of the target number of channels, including: outputting a data result set corresponding to the optimal matching information to the encoding module through the channels of the target number of channels, the data result set including the optimal matching information and the data to be encoded.

This embodiment describes a specific way of implementing a data encoding method in a hardware accelerator card, which focuses on optimizing the search and output process of matching information and improving the data processing speed through parallel processing. Specifically, in the hardware accelerator card, the data to be encoded is divided into N groups, and the number of bytes in each group is equal to the number of target channels (determined by step S12). This grouping strategy is to achieve parallel processing, that is, N groups of data can be processed in the computing module of the hardware accelerator card at the same time to find matching strings with the encoded data. For each group of data, each byte in the group is used as the starting byte, and the encoded data is searched for whether there are strings matching these starting bytes. This search is parallel, which means that N groups of data are searched at the same time, thereby significantly improving the search efficiency. For each starting byte, if a matching string is found, the matching length of the string and the matching distance of the first byte in the encoded data will be recorded. These matching information will be used in the subsequent encoding process. The matching length of the matched string is usually a positive integer. If no string is matched, the matching length is counted as 0. In all N groups of data, each byte corresponds to a matching information (if there are multiple matching strings). In order to select the best matching information, the best match for each byte is usually determined based on a certain standard (such as the longest matching length or the shortest matching distance). Then, the data result set corresponding to all these best matching information is used in the subsequent encoding process. Finally, according to the number of target channels, the data result set corresponding to the best matching information is output to the encoding module. This data result set includes not only the best matching information, but also the original data to be encoded. Due to the previous data grouping and parallel processing, this output process is also efficient, and multiple channels of the hardware accelerator card can be used for parallel transmission.

It can be seen that this embodiment significantly improves the efficiency of searching and outputting matching information through data grouping and parallel processing, thereby speeding up the overall data encoding process. This method fully utilizes the parallel computing capability of the hardware acceleration card, making data encoding more efficient and faster.

In one embodiment, the optimal matching information is determined based on the matching information corresponding to each byte in N groups of the data to be encoded, including: starting polling from the first group, determining the address of the current byte in the next comparison process based on the first matching length of the current byte in the current comparison process and the second matching length of the next byte; and determining the matching information of the current byte in the comparison process that meets the preset end condition as the optimal matching information.

This embodiment describes a method for determining optimal matching information based on matching information of multiple groups of bytes when processing data to be encoded. This method is mainly used to improve encoding efficiency and accuracy when implementing data encoding in a hardware acceleration card.

First, polling starts from the first group of data to be encoded, which means that the data to be encoded is compared group by group from the beginning; this ensures that each group of data to be encoded will be processed and each group has the opportunity to find the best matching information. During the comparison process, it is necessary to determine the address of the current byte in the next comparison process in order to continue the comparison; by calculating the first matching length of the current byte and the second matching length of the next byte, the address of the current byte in the next comparison process can be determined; this step ensures the continuity of the comparison process, so that matching searches can be smoothly performed in the data to be encoded. During the comparison process, it is necessary to determine when to end the comparison and determine the best matching information; once the preset end condition is met, the matching information of the current byte in the current comparison process can be determined as the best matching information; the preset end condition may involve matching length, matching distance or other indicators, and the specific conditions should be determined according to the specific implementation.

Through these steps, the data to be encoded can be compared group by group, the best matching information that meets the preset conditions can be found, and the best matching information can be determined in each group. This helps to improve encoding efficiency and accuracy so that matching information can be used more effectively in the encoding process.

In one embodiment, before determining the matching information of the current byte in the comparison process that meets the preset end condition as the optimal matching information, it also includes: judging whether the address offset of the current byte in the next comparison process is greater than the target channel number; if it is greater than the target channel number, judging that the current byte in the current comparison process meets the preset end condition; if it is less than the target channel number, judging that the current byte in the current comparison process does not meet the preset end condition.

Specifically, determine whether the address offset of the current byte in the next comparison process is greater than the target channel number: the purpose of this step is to check whether the distance between the byte to be compared in the next comparison process and the N groups of bytes in the data to be encoded has exceeded the target channel number. The address offset refers to the offset of the byte relative to the starting position of the first byte of the group of bytes. The target channel number is the channel number configuration information determined according to the state parameters of the hardware accelerator card and/or the user's demand instructions. If the address offset of the current byte in the next comparison process is greater than the target channel number, it means that this byte has exceeded the target channel range and the comparison needs to be stopped. That is, if this byte has exceeded the range of the target channel number, it can be determined that the comparison process has been completed and the current byte in the current comparison process meets the preset end condition; this means that the optimal matching information has been found and no further comparison is required. If the address offset of the current byte to be compared in the next comparison process is still less than the target channel number, it means that the comparison process has not been completed and it is necessary to continue to compare subsequent bytes to determine the optimal matching information; at this time, it cannot be determined that the current byte meets the preset end condition and it is necessary to continue to compare until the end condition is met.

The purpose of this embodiment is to timely determine whether the optimal matching information has been found during the comparison process, so as to improve the efficiency and accuracy of data encoding.

In one embodiment, the address of the current byte in the next comparison process is determined based on the first matching length of the current byte in the current comparison process and the second matching length of the next byte, including: when the first matching length is less than the second matching length, outputting the current byte in the current comparison process, and adding one to the address of the current byte in the current comparison process as the address of the current byte in the next comparison process.

In one embodiment, the address of the current byte in the next comparison process is determined based on the first match length of the current byte in the current comparison process and the second match length of the next byte, including: when the first match length is not less than the second match length, the first match length is output, and the address of the current byte in the current comparison process plus the address after the first match length is added is used as the address of the current byte in the next comparison process.

This embodiment describes how to determine the address of the current byte in the next comparison process according to the matching length between the current byte and the next byte when processing the data to be encoded. This embodiment helps to find the optimal matching information in the data encoding process, thereby improving the encoding efficiency and compression ratio.

Step 1: Compare the matching lengths of the current byte and the next byte. In this step, the first matching length (recorded as length A) of the current byte (recorded as byte A) and the second matching length (recorded as length B) of the next byte (recorded as byte B) are compared. These two matching lengths represent the lengths of the longest matching strings that can be found in the encoded data starting from byte A and byte B, respectively.

Step 2: Determine the length of the match. Compare length A and length B to determine which one is longer. This step is to determine the search strategy, that is, whether to continue searching for a match starting with byte A or jump to searching starting with byte B.

Step 3: Determine the output and address update based on the size of the matching length.

The first case: the first match length is less than the second match length. Output the current byte: This means that the match starting from the current byte A is not optimal, so the system may choose not to output the match information of the current byte A, but continue to search for a better match. However, in some implementations, because the match of the current byte A is not optimal, the current byte is output. Address plus one: Add one to the address of the current byte A, use byte B as the new current byte, and continue the next round of comparison. This is because the match length of byte B is longer, which may bring better compression effect.

The second case: the first match length is not less than the second match length. Output the first match length: Since length A is not less than length B, it is considered that the match starting from byte A is more likely to be optimal (or at least in the current comparison round). Therefore, choose to output the matching information related to byte A, including the first match length. Update the address: Next, add the address of the current byte A plus the address after the first match length as the starting address for the next round of comparison. This is because, since a longer matching string can be found starting from byte A, continuing the search at a certain position after byte A (that is, the position after the address plus the first match length) may find another potential matching point.

In this embodiment, by comparing the matching lengths of the current byte and the next byte, the starting point and direction of the search can be selected more intelligently, avoiding unnecessary searches at locations where it is unlikely to find a longer matching string. By selecting bytes that are more likely to produce long matching strings as the search starting point, the system can find the optimal matching information more quickly, thereby improving encoding efficiency and compression rate. Different data may have different characteristics, for example, some data segments may be more likely to produce long matching strings. By dynamically adjusting the search process, the characteristics of different data can be adapted to achieve more efficient encoding.

In one embodiment, it also includes: if the address of the current byte is equal to the maximum channel address, caching the matching information of the current byte; when entering the comparison process of the next group of bytes, constructing a virtual 0 address, storing the matching information of the byte of the maximum channel address in the virtual 0 address; using the matching information of the byte of the maximum channel address stored in the virtual 0 address as the matching information of the current byte in the current comparison process, and entering the step of determining the address of the current byte in the next comparison process according to the first matching length of the current byte in the current comparison process and the second matching length of the next byte.

This embodiment describes a method for processing multi-channel matching information and determining optimal matching information in a specific case during data encoding of a hardware acceleration card.

Step 1: If the address of the current byte is equal to the maximum channel address, cache the matching information of the current byte. When searching for matching strings in multiple groups of bytes in parallel, each group has a starting byte (current byte) and a corresponding address. If the address of the current byte (i.e. the position of the currently compared byte in the group) reaches the maximum channel address (i.e. the last byte in each group), it indicates that the matching search for the group of bytes has reached the end. At this time, cache the matching information of the current byte for subsequent processing.

Step 2: When entering the comparison process of the next group of bytes, construct a virtual 0 address. After completing the matching search of a group of bytes and caching the matching information of the last byte, it is necessary to enter the search process of the next group of bytes. In order to maintain the continuity and consistency of the search process, a virtual 0 address is constructed. This virtual 0 address is not the actual data address, but is used to store the matching information of the maximum channel address (the last byte) in the previous group, so as to facilitate comparison with the data of the first address in the next group.

Step 3: The matching information of the byte of the maximum channel address stored in the virtual 0 address is used as the matching information of the current byte of the current comparison process. When entering a new comparison process (i.e., searching for the next group of bytes), the matching information stored in the virtual 0 address is used as the matching information of the current byte. This is because, although a new byte group is physically entered, logically, it is hoped to continue the matching search process of the previous group of bytes, especially when it is found that a match may span two byte groups.

Step 4: Enter the step of determining the address of the current byte in the next comparison process according to the first matching length of the current byte in the current comparison process and the second matching length of the next byte. After determining the matching information of the current byte (whether real or obtained from the virtual 0 address), continue to determine the address of the current byte in the next comparison process according to the defined strategy (as described in the above embodiment). This process will guide the system on how to continue to search for matching strings in multiple groups of bytes and finally determine the optimal matching information.

It can be seen that this embodiment provides a strategy for crossing byte group boundaries when processing multi-channel matching information. By caching the matching information of the last group of bytes, constructing a virtual 0 address, and using the information in the virtual 0 address as the starting matching information of a new group of bytes, the search process can be seamlessly continued between different groups, thereby finding the optimal matching information.

Please refer to Figure 2, which describes a hardware-implemented architecture. Specifically, two types of storage queues are provided, namely, a data storage queue for storing each byte in the data to be encoded, and a matching information storage queue for storing matching information corresponding to each byte. It should be noted that due to the parallel processing capability of the hardware accelerator card, the byte data in each data storage queue comes at the same time, so when any fifo (First Input First Output) is detected to be non-empty, the data of 8 fifos can be read out at the same time. Each matching information comes randomly (related to the speed at which the front-end executes the matching algorithm of each byte), so when it is detected that 8 match fifos (matching information storage queues) are not empty at the same time, the 8 matching information are read out together. All output byte matching information is cached in the corresponding register array through tapping, and the register array will display the data valid flag. When the data valid flag is detected, the control module reads the byte matching information cached in the register data. As the main management and scheduling module, the control module mainly includes two parts: a module for pipeline parallel comparison and a matching module for obtaining the data result set corresponding to the optimal matching information. The module for pipeline parallel comparison is mainly used to execute the process in Figure 3, specifically: compare the first matching length of the current byte in the current comparison process with the second matching length of the next byte; whether the first matching length of the current byte is less than the second matching length of the next byte; if so, output the current byte in the current comparison process, and add one to the address of the current byte in the current comparison process as the address of the current byte in the next comparison process; if not, output the first matching length, and add the address of the current byte in the current comparison process plus the address after the first matching length as the address of the current byte in the next comparison process; then, whether the address offset of the current byte in the next comparison process is greater than the number of target channels; if greater than the number of target channels, end; otherwise, re-enter the process of comparing the first matching length of the current byte in the current comparison process with the second matching length of the next byte. This method maintains the consistency of the comparison of matching information to a certain extent.

As shown in FIG4 , in a specific embodiment, <1>, <2>, <3>, and <4> are four byte groups, each of which includes 8 bytes, such as byte group <1> includes a1-h1, byte group <2> includes a2-h2, etc. The number under each byte is the matching length.

The matching length of each byte in each byte group is calculated in parallel in groups to improve the calculation efficiency. After the matching length of each byte is obtained, the best matching information is determined by comparison and polling starts from the first group until the best matching information is matched.

Specifically, the process of determining the best matching information is as follows:

Start comparing from the first group (offset=1):

If a1 is used as the starting byte, the optimal comparison begins and the output is a1, 4, 7; the address offset of the next comparison address in the next group is offset1=7-2=5.

If b1 is used as the starting byte, the optimal comparison begins and the output is 4, 7; the address offset of the next comparison address in the next group is offset2=7-2=5.

If c1 is used as the starting byte, the optimal comparison begins and the output is 4, g1, 10; the address offset of the next comparison address in the next group is offset3=10-0=10.

If d1 is used as the starting byte, the optimal comparison begins and the output is 3, g1, 10; the address offset of the next comparison address in the next group is offset4=10-0=10.

If e1 is used as the starting byte, the optimal comparison begins and the output is e1, 7; the address offset of the next comparison address in the next group is offset5=7-2=5.

If f1 is used as the starting byte, the optimal comparison begins and the output is 7; the address offset of the next comparison address in the next group is offset6=7-2=5.

If g1 is used as the starting byte, the optimal comparison begins and the output is g1, 10; the address offset of the next comparison address in the next group is offset7=10-0=10.

If h1 is used as the starting byte, the optimal comparison begins and the output is 10; the address offset of the next comparison address in the next group is offset8=10-0=10.

The default value of the first comparison offset is 1, so the first optimal result matching data set is selected, the output result is a1, 4, 7, and the offset is updated to 5.

The second set of comparisons (offset=5):

The optimal comparison starts from a2, the output is 3, d2, 3, and the address offset of the next comparison address in the next group is offset1=0.

The optimal comparison starts from b2, and the output is b2, 4, 7. The address offset of the next comparison address in the next group is offset2=7-2=5.

The optimal comparison starts from c2, the output is 4, 7, and the address offset of the next comparison address in the next group is offset3=7-2=5.

The optimal comparison starts from d2, the output is 3, 7, and the address offset of the next comparison address in the next group is offset4=7-2=5.

The optimal comparison starts from e2, the output is 3, and the address offset of the next comparison address in the next group is offset5=0.

The optimal comparison starts from f2, the output is f2,7, and the address offset of the next comparison address in the next group is offset6=7-2=5.

The optimal comparison starts from g2, the output is 7, and the address offset of the next comparison address in the next group is offset7=7-2=5.

The optimal comparison starts from h2. Since the address of h2 is equal to the number of channels 8, it is not output and the matching information of h2 is cached. The address offset of the next comparison address in the next group is offset8=0.

The second set of comparison offsets starts at 5, so the fifth best result matching data set is directly selected, the output result is 3, and the offset is updated to 0.

The third group of comparisons (offset=0):

The optimal comparison starts from h2, and the output is 6, 11; the address offset of the next comparison address in the next group is offset0=11-2=9.

The optimal comparison starts from a3, and the output is h2, a3, 20; the address offset of the next comparison address in the next group is offset1 = 20-6 = 14.

The offset starting value of the third comparison is 0, so the last matching information of the previous comparison is involved in this comparison. This group only needs to compare the size of h2 and a3 to determine the best matching data set of this group. If h2>a3, the result set starting with h2 is selected. If h2＜a3, the result set starting with a3 is selected. Therefore, after this comparison, the offset is updated to 9.

The fourth group of comparisons (offset=9):

For the last set of data, the offset is greater than the maximum number of channels, so it is not output this time.

In one embodiment, the data result set corresponding to the optimal matching information is output to the encoding module through the channels of the target number of channels, including: writing the data result set corresponding to the optimal matching information into a ping-pong register group with a depth of m, where m is a positive integer multiple of N; when the N channel flags of the ping-pong register group are valid, triggering the encoding module to read the data result set through the channels of the target number of channels.

This embodiment describes a specific implementation method of how to effectively output the data result set corresponding to the best matching information to the encoding module through the channels of the target number of channels during the data encoding process.

Step 1: Write the data result set corresponding to the optimal matching information into a ping-pong register group with a depth of m. Using a ping-pong register group with a depth of m (usually including two or more registers used alternately) as a buffer, the data result set corresponding to the optimal matching information can be temporarily stored. In this way, even if the processing speed of the encoding module is slow, the computing module can continue to process the next set of data, improving the throughput and processing efficiency of the entire system. The depth m of the ping-pong register group is a positive integer multiple of N, which means that it can store data result sets of multiple channels at the same time. This allows the system to process multiple sets of data in parallel, further improving the parallel processing capability of the system.

Step 2: When the N channel flags of the ping-pong register group are valid, the encoding module is triggered to read the data result set through the channels of the target number of channels. The channel flags in the ping-pong register group are used to indicate whether the data result set of a certain channel is ready for the encoding module to read. When all N channel flags are valid (that is, the data result sets of N channels are ready), the encoding module is triggered to start reading data. This mechanism ensures data consistency and synchronization.

In this embodiment, since the depth and number of channels of the ping-pong register group are configurable (based on the settings of N and m), the system can be flexibly adjusted according to different application requirements. For example, if the application requires higher throughput, the depth or number of channels of the ping-pong register group can be increased; conversely, if resources are limited, these parameters can be reduced to save hardware resources. Buffering and synchronization through the ping-pong register group can reduce the risk of data loss caused by the mismatch of the processing speed of the encoding module and the computing module. Even if the encoding module is temporarily unable to process a new data result set, the computing module can continue to write the result to the ping-pong register group and wait for the encoding module to be ready before reading and processing. In summary, this embodiment introduces the ping-pong register group as a buffer, and combines the synchronization mechanism of the channel flag to achieve the effective output of the data result set corresponding to the optimal matching information and the synchronous processing of the encoding module, thereby improving the efficiency and reliability of the entire data encoding system.

In one embodiment, there are multiple ping-pong register groups, and the data result set corresponding to the optimal matching information is written into the ping-pong register group with a depth of m, including: when the encoding module reads the data result set stored in one of the ping-pong register groups through the channels of the target number of channels, the data result set corresponding to the optimal matching information is written into other ping-pong register groups that have not been read.

This embodiment describes the operation steps when using multiple ping-pong register groups for data buffering. Ping-pong register groups are usually used in high-speed data processing to achieve seamless buffering and transmission of continuous data streams. Multiple ping-pong register groups are set to achieve continuous processing and transmission of data. When a ping-pong register group is being read by the encoding module, another ping-pong register group can be written to a new data result set at the same time, thereby achieving parallelism and continuity of data processing. When the encoding module needs to process data, it will read the data result set corresponding to the optimal matching information from a ping-pong register group through the channels of the target number of channels. This process is a key step in data processing because it ensures that the encoding module can continuously receive the data to be processed. While the encoding module reads data from a ping-pong register group, the computing module writes the data result set corresponding to the new optimal matching information into another ping-pong register group that has not been read. This step ensures the continuity and efficiency of data processing because it does not need to wait for the encoding module to complete the processing of the current data result set before starting to write new data.

In summary, this embodiment realizes parallel processing and continuous transmission of data by introducing multiple ping-pong register groups, thereby improving the overall efficiency and performance of the data encoding method in the hardware acceleration card. This design enables the processor and the encoding module to work together more efficiently, reduces the waiting time in the data processing process, and improves the throughput.

In the above embodiments, the calculation module may be specifically used to execute the LZ77 algorithm (Lemper-Ziff 77 algorithm), and the encoding module may be a Huffman encoding module.

In a second aspect, as shown in FIG. 5 , the present invention provides a data encoding device, which is applied to a hardware acceleration card. The hardware acceleration card also includes a computing module and an encoding module. The data encoding device includes: a memory 51 for storing a computer program; a processor 52 for implementing the steps of the above-mentioned data encoding method when storing the computer program.

For the introduction of the data encoding device, please refer to the above embodiment, and the present invention will not be described in detail here.

In a third aspect, as shown in FIG6 , the present invention provides a hardware acceleration card, comprising the above-mentioned data encoding device, and further comprising a computing module and an encoding module;

The calculation module is used to determine matching information according to the data to be encoded and the encoded data, and output the matching information and the data to be encoded to the encoding module through channels of the target number of channels;

The encoding module is used for encoding the data to be encoded according to the matching information;

The matching information is the matching distance and matching length of the preset character string in the encoded data when the preset character string in the data to be encoded is the same as the preset character string in the encoded data.

The hardware acceleration card may be, but is not limited to, an FPGA (Field-Programmable Gate Array). For other introductions to the hardware acceleration card, please refer to the above embodiments, which will not be described in detail in the present invention.

In a fourth aspect, the present invention provides a computer program product, comprising a computer program/instructions, which implement the steps of the above-mentioned data encoding method when the computer program/instructions are executed by a processor.

For the introduction of the computer program product, please refer to the above embodiments, and the present invention will not be described in detail here.

In a fifth aspect, as shown in FIG. 7 , the present invention provides a non-volatile storage medium 61 on which a computer program 62 is stored. When the computer program 62 is executed by a processor, the steps of the above-mentioned data encoding method are implemented.

For the introduction of the non-volatile storage medium 61 , please refer to the above embodiment, and the present invention will not go into details here.

It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims (20)

1. A data encoding method, applied to a processor in a hardware accelerator card, the hardware accelerator card further comprising a computing module and an encoding module, the data encoding method comprising:

acquiring channel quantity configuration information determined based on state parameters of a hardware accelerator card and/or demand instructions of a user;

configuring the channel number of the coding module as a target channel number according to the channel number configuration information;

After the computing module determines matching information according to the data to be encoded and the encoded data, outputting the matching information and the data to be encoded to the encoding module through the channels with the target channel number, and triggering the encoding module to encode the data to be encoded according to the matching information;

and the matching information is the matching distance and the matching length of the preset character string in the coded data when the preset character string in the data to be coded is the same as the preset character string in the coded data.

2. The data encoding method according to claim 1, wherein acquiring channel number configuration information determined based on a state parameter of a hardware accelerator card and/or a demand instruction of a user, comprises:

Determining the number of test channels according to a demand instruction of a user, and configuring the number of channels of the coding module as the number of test channels;

estimating the first computing resource occupation amount of the hardware accelerator card under the number of the test channels;

and adjusting the number of the test channels according to the first computing resource occupation amount to obtain the number of target channels.

3. The data encoding method of claim 2, wherein adjusting the number of test channels according to the first computing resource occupancy to obtain the target number of channels comprises:

comparing the first computing resource occupation amount with a first preset threshold value;

if the first computing resource occupation amount is larger than the first preset threshold value, reducing the number of the test channels to obtain the number of the target channels;

If the first computing resource occupation amount is smaller than the first preset threshold value, increasing the number of the test channels to obtain the number of the target channels;

And if the first computing resource occupation amount is equal to the first preset threshold value, determining that the number of the test channels is the target channel number.

4. The data encoding method of claim 3, wherein after comparing the first computing resource occupancy with a first preset threshold, further comprising:

calculating a difference value between the first calculation resource occupation amount and the first preset threshold value, and calculating a ratio of the difference value to the first preset threshold value;

determining the channel quantity variation according to the ratio and the preset ratio-variation correspondence;

if the first computing resource occupation amount is greater than the first preset threshold, reducing the number of the test channels to obtain the number of the target channels, including:

If the first computing resource occupation amount is larger than the first preset threshold value, subtracting the channel number variation amount from the test channel number to obtain the target channel number;

If the first computing resource occupation amount is smaller than the first preset threshold value, increasing the number of the test channels to obtain the number of the target channels, including:

And if the first computing resource occupation amount is smaller than the first preset threshold value, adding the channel number variation amount to the test channel number to obtain the target channel number.

5. The data encoding method of claim 3, wherein upon determining that the first computing resource occupancy is greater than the first preset threshold, further comprising:

Comparing the first computing resource occupation amount with a second preset threshold value, wherein the second preset threshold value is larger than the first preset threshold value;

And if the first computing resource occupation amount is larger than the second preset threshold value, feeding back a reconfiguration signal to a user so that the user can redetermine the number of the test channels based on the reconfiguration signal, and re-entering the step of configuring the number of the channels of the coding module into the number of the test channels.

6. The data encoding method of claim 4, wherein prior to calculating the difference between the first computing resource occupancy and the first preset threshold, further comprising:

determining an application platform of the hardware accelerator card, and determining a second computing resource occupation amount required by the hardware accelerator card to execute a preset task on the application platform;

determining the computing resource residual quantity of the hardware acceleration card according to the second computing resource occupation quantity, wherein the sum value of the second computing resource occupation quantity and the computing resource residual quantity is not more than all computing resources of the hardware acceleration card;

and determining the first preset threshold according to the computing resource remaining quantity.

7. The data encoding method according to claim 1, wherein acquiring channel number configuration information determined based on a state parameter of a hardware accelerator card and/or a demand instruction of a user, comprises:

determining an application platform of the hardware accelerator card, and determining a second computing resource occupation amount required by the hardware accelerator card to execute a preset task on the application platform;

determining the computing resource residual quantity of the hardware acceleration card according to the second computing resource occupation quantity, wherein the sum value of the second computing resource occupation quantity and the computing resource residual quantity is not more than all computing resources of the hardware acceleration card;

And determining the number of the target channels according to the residual quantity of the computing resources.

8. The data encoding method according to claim 1, wherein acquiring channel number configuration information determined based on a state parameter of a hardware accelerator card and/or a demand instruction of a user, comprises:

Acquiring channel quantity configuration information determined based on state parameters of a hardware accelerator card and/or demand instructions of a user through a first interface;

converting the channel number configuration information acquired by the first interface into channel number configuration information corresponding to the second interface by using an interface conversion device;

Refreshing the channel number configuration information corresponding to the second interface into a channel number configuration register;

configuring the channel number of the coding module to be the target channel number according to the channel number configuration information, including:

And configuring the channel number of the coding module as a target channel number according to the channel number configuration information in the channel number configuration register.

9. The data encoding method according to any one of claims 1 to 8, wherein the data to be encoded is divided into N groups, N being an integer greater than 1, the number of bytes in each group being equal to the target channel number; determining matching information according to the data to be encoded and the encoded data, including:

taking each byte in each group of bytes as a starting byte, and simultaneously searching whether a character string matched with each byte in the group as the starting byte exists in the encoded data or not by N groups of bytes, wherein the length of the character string is at least 2;

Recording the matching information of the character string with each byte as the initial byte, wherein the matching information comprises the matching length of the matched character string and the matching distance of the initial byte of the character string in the encoded data;

Determining optimal matching information according to the matching information corresponding to each byte in N groups of data to be coded;

Outputting the matching information and the data to be encoded to the encoding module through the channels of the target channel number, wherein the method comprises the following steps:

Outputting a data result set corresponding to the optimal matching information to the coding module through the channels with the target channel number, wherein the data result set comprises the optimal matching information and the data to be coded.

10. The data encoding method of claim 9, wherein determining optimal matching information according to matching information corresponding to each of the bytes in the N groups of data to be encoded, comprises:

polling is started from the first group, and the address of the current byte in the next comparison process is determined according to the first matching length of the current byte in the current comparison process and the second matching length of the next byte;

and determining the matching information of the current byte in the comparison process meeting the preset ending condition as the optimal matching information.

11. The data encoding method of claim 10, wherein before determining the matching information of the current byte in the comparison process satisfying the preset end condition as the optimal matching information, further comprising:

judging whether the address offset of the current byte in the next comparison process is larger than the number of the target channels;

if the number of the current bytes is larger than the number of the target channels, judging that the current bytes in the current comparison process meet the preset ending condition;

if the number of the target channels is smaller than the number of the target channels, judging that the current byte in the current comparison process does not meet the preset ending condition.

12. The data encoding method of claim 10, wherein determining the address of the current byte in the next comparison process based on the first matching length of the current byte and the second matching length of the next byte in the current comparison process comprises:

And when the first matching length is smaller than the second matching length, outputting the current byte in the current comparison process, and adding one address of the current byte in the current comparison process as the address of the current byte in the next comparison process.

13. The data encoding method of claim 10, wherein determining the address of the current byte in the next comparison process based on the first matching length of the current byte and the second matching length of the next byte in the current comparison process comprises:

and outputting the first matching length when the first matching length is not smaller than the second matching length, and taking the address of the current byte in the current comparison process plus the address after the first matching length as the address of the current byte in the next comparison process.

14. The data encoding method of claim 10, further comprising:

if the address of the current byte is equal to the maximum channel address, caching matching information of the current byte;

When the next byte comparison process is entered, a virtual 0 address is constructed, and matching information of the bytes of the maximum channel address is stored in the virtual 0 address;

And taking the matching information of the byte of the maximum channel address stored by the virtual 0 address as the matching information of the current byte of the current comparison process, and entering a step of determining the address of the current byte in the next comparison process according to the first matching length of the current byte and the second matching length of the next byte in the current comparison process.

15. The data encoding method of claim 10, wherein outputting the data result set corresponding to the optimal matching information to the encoding module through the channel of the target channel number comprises:

Writing the data result set corresponding to the optimal matching information into a ping-pong register set with the depth of m, wherein m is positive integer multiple of N;

and triggering the coding module to read the data result set through the channels of the target channel number when N channel marks of the ping-pong register set are valid.

16. The data encoding method of claim 15, wherein the number of ping-pong register sets is plural, and writing the data result set corresponding to the optimal matching information into the ping-pong register set with depth m comprises:

and when the encoding module reads the data result set stored in one of the ping-pong register sets through the channels with the target channel number, writing the data result set corresponding to the optimal matching information into other ping-pong register sets which are not read.

17. A data encoding device, applied to a hardware accelerator card, the hardware accelerator card further comprising a computing module and an encoding module, the data encoding device comprising:

A memory for storing a computer program;

a processor for implementing the steps of the data encoding method according to any one of claims 1-16 when storing a computer program.

18. A hardware accelerator card, comprising the data encoding device of claim 17, further comprising a computing module and an encoding module;

The computing module is used for determining matching information according to the data to be encoded and the encoded data, and outputting the matching information and the data to be encoded to the encoding module through channels with the number of target channels;

the encoding module is used for encoding the data to be encoded according to the matching information;

and the matching information is the matching distance and the matching length of the preset character string in the coded data when the preset character string in the data to be coded is the same as the preset character string in the coded data.

19. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the data encoding method of any of claims 1-16.

20. A non-volatile storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the data encoding method according to any of claims 1-16.