Detailed Description
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
In carrying out the invention, the inventors found that:
Redis (Remote Dictionary Server)) is a high-performance, memory-storage open source database that stores data in key-value pairs and supports a variety of data structures. Redis is commonly used for cache, session management, message queuing, and other named applications. In Redis, lua scripts are typically used to perform a series of operations, such as reading and writing data, as well as complex logic operations. The Redis lua script is then an instruction that parses the lua script into executable instructions so that the Redis server can execute the scripts.
Since Redis is a single-threaded-multiplexing IO model, executing lua blocks all cached information in the entire instance of Redis, i.e., while executing lua, it cannot respond to other instruction operations. When the blocking is performed for too long, the performance and resource reuse of Redis will be negatively affected.
Particularly in a high concurrency scenario, if the application end is improperly set up lua, during the execution blocking period, the Redis refuses other requests, which can cause chain reaction, and cause the client end to continuously reconnect and the application to be interrupted.
If the cache high availability module detects that the ping operation cannot be responded normally during the blocking execution, the high availability module misdeems that the redis service node is abnormal, and the master-slave switching operation is triggered, so that the service application is negatively affected.
The embodiment of the invention provides an optimization method, an optimization device, electronic equipment and a readable medium for blocking an lua script, wherein the method loads the lua script into a cache, and each time the lua script is executed, the method comprises the steps of obtaining average execution time of the lua script, setting a zone bit if the average execution time is greater than a preset threshold value, executing a read instruction in the lua script to obtain an instruction execution result, and canceling the zone bit if the execution result of the lua script is determined to be finished according to the instruction execution result. When the lua script is executed each time, if the average execution time of the lua script is larger than the preset threshold value, the method enables the lua script to only execute the read instruction through the zone bit to obtain the instruction execution result, and in a scene with overlong execution time of the lua script, the open sensitivity of the instruction is adjusted through the preset threshold value, so that the problems of continuous reconnection and application of the terminal are avoided.
Referring to fig. 1, a step flow chart of an optimizing method of lua script blocking provided in an embodiment of the present invention may specifically include the following steps:
101, loading the lua script into a cache.
After the lua script is loaded into the cache, a SHA-1 hash value is generated for the lua script as a key, and the content of the lua script is used as a value. And in the subsequent execution, the corresponding value can be found through the key, and then the lua script in the value is executed.
102, Each time the lua script is executed, the following steps 201 to 204 are performed:
the average execution time of the lua script is obtained 201.
The average execution time of the lua script is updated after each execution of the lua script, and the current average execution time is directly acquired in this step.
202, If the average execution time is greater than a preset threshold, setting a flag bit.
When the average time consumption of the script is smaller than a preset threshold, the execution time is controllable, the whole service is not blocked, and intervention processing is not needed.
When the average time consumption exceeds a threshold value, the risk that the overall availability is reduced due to the existence of a blocking surface in the execution of the script is indicated, and a read request forwarding flow is triggered to improve the availability.
The threshold value can be configured through a configuration item, such as parameter 1, parameter 2 and parameter 3, which are newly added in the configuration item.
When the lua script is executed, the parameter 1 indicates whether to use the optimization method of lua script blocking provided by the embodiment to process, the parameter 2 indicates whether to set a flag bit, and the parameter 3 indicates a threshold value.
For example, before executing the lua script blocking optimization method provided in this embodiment, if the newly added content in the configuration item is script-analysis-disable no NULL, where parameter 1 is script-analysis-disable, parameter 2 is no, and parameter 3 is NULL (i.e., NULL). The optimization method indicating that the lua script is not blocked by the lua script provided by the present embodiment is executed.
For example, before executing the lua script blocking optimization method provided in this embodiment, if the newly added content in the configuration item is script-analysis-enable no 10, where parameter 1 is script-analysis-enable, parameter 2 is no, and parameter 3 is 10. The lua script is executed by adopting the optimization method for blocking the lua script provided by the embodiment, and the threshold is 10 milliseconds, and the flag bit is not set at this time.
If in step 202 it is determined that the average execution time is greater than 10 milliseconds, then parameter 2 of the configuration item is modified to yes, and the flag bit is set. The newly added content in the configuration item after modification is changed into script-analysis-enable yes 10, wherein parameter 1 is script-analysis-enable, parameter 2 is no, and parameter 3 is 10. Wherein, parameter 1 is script-analysis-enable, parameter 2 is yes, and parameter 3 is 10. The lua script is executed by adopting the optimization method for blocking the lua script provided by the embodiment, and a flag bit is set, wherein the threshold value is 10 milliseconds.
Since the flag bit is set when step 102 is performed, it indicates that the lua script is being executed as long as the flag bit is set.
203, Executing the read instruction in the lua script to obtain an instruction execution result. And terminating the execution of the non-read instruction in the lua script to obtain an instruction execution result of script execution blocking, wherein the blocking is described as read-only.
The read instructions in the lua script are processed in step 203 and are not executed for other instructions, so the read instructions in the lua script are determined before step 203 is executed.
The procedure for determining the read instruction in the lua script is as follows:
1. the type of each instruction in the lua script is determined, wherein the type is read or unread.
2. An instruction of the type read is determined to be a read instruction.
If instructions in the lua script are classified, read instructions are classified as one type, and non-read instructions (e.g., write instructions) are classified as one type.
In step 203, only the read instruction is executed, and the non-read instruction is not executed.
The execution of the read instruction in step 203 is:
1.a slave node is acquired.
2. If the slave node acquires the read instruction successfully, forwarding the read instruction in the lua script to the slave node so as to enable the slave node to execute the read instruction, and determining the execution result of the slave node as the instruction execution result.
3. If the acquisition from the node fails, the execution of the read instruction in the lua script is terminated, an instruction execution result of script execution blocking is obtained, and the blocking is described as busy.
For example, the instruction execution results in busy script execution blocking.
That is, if all the instructions in the lua script are read instructions when executing step 203, all the read instructions are executed by the slave node, and the execution result of the read instructions is taken as the instruction execution result obtained by executing the read instructions in the lua script in step 203.
If all the instructions in the lua script are non-read instructions, terminating executing all the non-read instructions to obtain an instruction execution result of script execution blocking, and the blocking is described as read-only.
For example, the instruction execution results in read-only, script execution blocking.
If part of the instructions in the lua script are read instructions and part of the instructions are non-read instructions, as shown in fig. 2, the read instructions in the lua script are executed by the slave nodes to obtain an instruction execution result, the non-read instructions in the lua script are terminated to obtain an instruction execution result of script execution blocking, and the blocking instruction is read only.
And the slave node executes the read instruction in the lua script to acquire the slave node, and forwards the read instruction in the lua script to the slave node so as to enable the slave node to execute the read instruction, and the execution result of the slave node is determined to be the instruction execution result.
In a specific implementation, in order to ensure that the execution of step 203 is in the execution blocking period, before executing step 203, it may be further confirmed whether the lua script blocking occurs again, that is, that the average execution time is greater than the preset threshold again, and the flag bit is set.
If the average execution time is greater than the preset threshold and the flag bit is set, it is confirmed that the lua script is blocked, step 203 is executed, otherwise, it is considered that no lua script is blocked, step 203 is not executed any more, and the execution scheme of the existing lua script (i.e. the local execution instruction) is converted, as shown in fig. 3.
In step 203, only read instructions are allowed to execute, achieving a response read-only request effect by forwarding to the slave node. For write instructions, a hint error is quickly returned. Because the write request is still refused/blocked, the read-only request theory has no problem of consistency of master-slave data except the data related to the script, and the data accuracy can be trusted.
204, If the lua script is determined to be executed according to the instruction execution result, the flag bit is canceled.
And if the parameter 2 of the configuration item is modified to be no, the cancellation of the flag bit is realized. The newly added content in the configuration item after modification is changed into a script-analysis-enable no 10, wherein the parameter 1 is the script-analysis-enable, the parameter 2 is no, and the parameter 3 is 10. The lua script is executed by adopting the optimization method for blocking the lua script provided by the embodiment, and the threshold is 10 milliseconds, and the flag bit is not set (i.e. the flag bit is cancelled).
The cancellation flag bit is the lua script is executed at the present time.
In addition, after determining that the execution of the lua script is finished according to the instruction execution result, the average execution time of the lua script is updated, and the updating process is as follows:
1. and determining the current execution time of the lua script.
If the current execution is the ith execution of the lua script, the ith execution time T i of the lua script is determined.
2. The average execution time of the updated lua script is (average execution time of the lua script x the historical execution times of the lua script + the present execution time)/(the historical execution times of the lua script + 1).
The historical execution times of the lua script do not contain the current execution time after the current execution is finished, namely the historical execution times of the lua script are i-1.
That is, the average execution time of all i-1 times of execution and the i-th execution time of the lua script after i-1 times of execution before the lua script are obtained, and the average execution time of all i times of execution after i times of execution before the lua script is the updated average execution time of the lua script.
If the average execution time of the lua script isThe update procedure is:
In addition, if the script execution block is determined according to the instruction execution result, the application side can perform mechanisms such as retry, etc., so as to avoid overtime caused by long-time waiting of the application, and further trigger disconnection reconnection operation.
The method for optimizing the lua script blocking can be used for solving the problems of overlong execution time of a complex lua script, releasing a service detection request, releasing a read-through instruction, improving the overall service availability, simultaneously counting and recording the average execution time of the lua script, adjusting the instruction opening sensitivity according to a preset threshold, improving the service availability in the blocking stage, avoiding various problems caused by service blocking, such as high availability judgment errors caused by service blocking, and applying overtime reconnection.
The optimization method for lua script blocking provided by the embodiment can realize more accurate judgment of the high-availability module and avoid error master-slave switching. In the existing lua script execution scheme, if the lua script execution time is long, the ping request is blocked and not responded in the time, so that a high-availability module can consider that the service is not available, and then the master-slave switching logic is triggered. The optimization method for lua script blocking provided by the embodiment can ensure that ping can still respond normally in the execution process of the lua script, and avoid judgment errors.
According to the lua script blocking optimization method provided by the embodiment, a function configuration switch is additionally arranged, the average script execution time is counted, and the instruction opening sensitivity is adjusted according to the configuration threshold, so that the adaptation is more flexible.
The optimizing method for lua script blocking provided in this embodiment releases a read-only instruction during the execution of the lua script blocking, and transfers the read-only instruction to the slave node for execution. And the busy error can be returned quickly through the instruction execution result, so that the high-availability logic is prevented from being triggered due to long-time waiting of the application end.
The optimization method for the lua script blocking provided by the embodiment can realize compatibility with the existing lua script execution scheme. As shown in fig. 4.
After execution of the lua script, the lua script is loaded into the cache, via step 101.
Then, it is determined whether the parameter 1 is script-analysis-enable.
If parameter 1 is not script-analysis-enable (e.g., is script-analysis-disable), the lua script is executed using an existing lua script execution scheme.
If parameter 1 is script-analysis-enable, then the lua script is executed (i.e., executed using the optimization method of lua script blocking provided in this embodiment) via step 102.
The average execution time of the lua script is obtained as per step 201.
Through step 202, when the average execution time is greater than a preset threshold, a flag bit is set.
The read instruction in the lua script is executed, resulting in an instruction execution result, via step 203. And terminating the execution of the non-read instruction in the lua script to obtain an instruction execution result of script execution blocking, wherein the blocking is described as read-only.
Through step 204, after determining that the lua script is executed this time according to the instruction execution result, the flag bit is cancelled.
Regardless of the manner in which the lua script is executed, the average execution time of the lua script is updated after each execution of the lua script is completed.
It should be noted that, for simplicity of description, the method embodiments are shown as a series of acts, but it should be understood by those skilled in the art that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.
The embodiment provides an optimization method for blocking a lua script, which loads the lua script into a cache, and executes the steps of obtaining average execution time of the lua script each time the lua script is executed, setting a flag bit if the average execution time is greater than a preset threshold value, executing a read instruction in the lua script to obtain an instruction execution result, and canceling the flag bit after the execution of the lua script is confirmed according to the instruction execution result.
When the lua script is executed each time, if the average execution time of the lua script is greater than a preset threshold value, the lua script only executes the read instruction through the flag bit to obtain an instruction execution result, and in a scene with overlong execution time of the lua script, the open sensitivity of the instruction is adjusted through the preset threshold value, so that the problems of continuous reconnection and application of a terminal are avoided.
Referring to fig. 5, a block diagram of an optimizing device for lua script blocking according to an embodiment of the present invention is shown, which may specifically include the following modules:
a loading module 501, configured to load the lua script into the cache.
The execution module 502 is configured to execute the following steps each time the lua script is executed:
The average execution time of the lua script is obtained.
If the average execution time is greater than the preset threshold value, setting a flag bit.
And executing the read instruction in the lua script to obtain an instruction execution result.
And if the lua script is determined to be executed according to the instruction execution result, the flag bit is canceled.
The method comprises the steps of executing a read instruction in the lua script, and determining the type of each instruction in the lua script before obtaining an instruction execution result, wherein the type is read or unread.
An instruction of the type read is determined to be a read instruction.
Executing the read instruction in the lua script to obtain an instruction execution result, wherein the method comprises the following steps:
A slave node is acquired.
If the slave node acquires the read instruction successfully, forwarding the read instruction in the lua script to the slave node so as to enable the slave node to execute the read instruction, and determining the execution result of the slave node as the instruction execution result.
Wherein after obtaining the slave node, further comprising:
If the acquisition from the node fails, the execution of the read instruction in the lua script is terminated, an instruction execution result of script execution blocking is obtained, and the blocking is described as busy.
Wherein, after determining the type of each instruction in the lua script, the method further comprises:
and terminating the execution of the non-read instruction in the lua script to obtain an instruction execution result of script execution blocking, wherein the blocking is described as read-only.
After determining that the lua script is executed this time according to the instruction execution result, the method further comprises:
And determining the current execution time of the lua script.
The average execution time of the updated lua script is (average execution time of the lua script x the historical execution times of the lua script + the present execution time)/(the historical execution times of the lua script + 1).
And if the script execution block is determined according to the instruction execution result, triggering the disconnection reconnection operation.
For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.
According to the device provided by the embodiment, when the lua script is executed each time, if the average execution time of the lua script is greater than the preset threshold, the lua script only executes the read instruction through the flag bit to obtain the instruction execution result, and in a scene with overlong execution time of the lua script, the open sensitivity of the instruction is adjusted through the preset threshold, so that the problems of continuous reconnection and application terminals are avoided.
In addition, the embodiment of the invention also provides an electronic device, as shown in fig. 6, which comprises a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602 and the memory 603 complete communication with each other through the communication bus 604,
A memory 603 for storing a computer program;
the processor 601 is configured to execute the program stored in the memory 603, and implement the following steps:
the lua script is loaded into the cache.
Each time the lua script is executed, the following steps are performed:
The average execution time of the lua script is obtained.
If the average execution time is greater than the preset threshold value, setting a flag bit.
And executing the read instruction in the lua script to obtain an instruction execution result.
And if the lua script is determined to be executed according to the instruction execution result, the flag bit is canceled.
Optionally, executing the read instruction in the lua script, before obtaining the instruction execution result, further includes:
the type of each instruction in the lua script is determined, wherein the type is read or unread.
An instruction of the type read is determined to be a read instruction.
Optionally, executing the read instruction in the lua script to obtain an instruction execution result, including:
A slave node is acquired.
If the slave node acquires the read instruction successfully, forwarding the read instruction in the lua script to the slave node so as to enable the slave node to execute the read instruction, and determining the execution result of the slave node as the instruction execution result.
Optionally, after the slave node is acquired, the method further includes:
If the acquisition from the node fails, the execution of the read instruction in the lua script is terminated, an instruction execution result of script execution blocking is obtained, and the blocking is described as busy.
Optionally, after determining the type of each instruction in the lua script, the method further includes:
and terminating the execution of the non-read instruction in the lua script to obtain an instruction execution result of script execution blocking, wherein the blocking is described as read-only.
Optionally, after determining that the lua script is executed this time according to the instruction execution result, the method further includes:
And determining the current execution time of the lua script.
The average execution time of the updated lua script is (average execution time of the lua script x the historical execution times of the lua script + the present execution time)/(the historical execution times of the lua script + 1).
Optionally, if the script execution block is determined according to the instruction execution result, the disconnection reconnection operation is triggered.
The communication bus mentioned by the above terminal may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the terminal and other devices.
The memory may include random access memory (Random Access Memory, RAM) or may include non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central Processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), a digital signal processor (DIGITAL SIGNAL Processing, DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable gate array (FPGA) or other Programmable logic device, discrete gate or transistor logic device, or discrete hardware components.
According to the electronic device provided by the embodiment, when the lua script is executed each time, if the average execution time of the lua script is greater than the preset threshold, the lua script only executes the read instruction through the flag bit to obtain the instruction execution result, and in a scene with overlong execution time of the lua script, the open sensitivity of the instruction is adjusted through the preset threshold, so that the problems of continuous reconnection and application terminals are avoided.
In yet another embodiment provided by the present invention, as shown in fig. 7, there is further provided a computer readable storage medium 701 having instructions stored therein, which when run on a computer, cause the computer to perform the method of optimizing lua script blocking described in the above embodiment.
In a further embodiment of the present invention, a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of optimizing lua script blocking described in the above embodiments is also provided.
Specifically, the method comprises the following steps:
the lua script is loaded into the cache.
Each time the lua script is executed, the following steps are performed:
The average execution time of the lua script is obtained.
If the average execution time is greater than the preset threshold value, setting a flag bit.
And executing the read instruction in the lua script to obtain an instruction execution result.
And if the lua script is determined to be executed according to the instruction execution result, the flag bit is canceled.
Optionally, executing the read instruction in the lua script, before obtaining the instruction execution result, further includes:
the type of each instruction in the lua script is determined, wherein the type is read or unread.
An instruction of the type read is determined to be a read instruction.
Optionally, executing the read instruction in the lua script to obtain an instruction execution result, including:
A slave node is acquired.
If the slave node acquires the read instruction successfully, forwarding the read instruction in the lua script to the slave node so as to enable the slave node to execute the read instruction, and determining the execution result of the slave node as the instruction execution result.
Optionally, after the slave node is acquired, the method further includes:
If the acquisition from the node fails, the execution of the read instruction in the lua script is terminated, an instruction execution result of script execution blocking is obtained, and the blocking is described as busy.
Optionally, after determining the type of each instruction in the lua script, the method further includes:
and terminating the execution of the non-read instruction in the lua script to obtain an instruction execution result of script execution blocking, wherein the blocking is described as read-only.
Optionally, after determining that the lua script is executed this time according to the instruction execution result, the method further includes:
And determining the current execution time of the lua script.
The average execution time of the updated lua script is (average execution time of the lua script x the historical execution times of the lua script + the present execution time)/(the historical execution times of the lua script + 1).
Optionally, if the script execution block is determined according to the instruction execution result, the disconnection reconnection operation is triggered.
When the lua script is executed each time, if the average execution time of the lua script is greater than the preset threshold, the lua script only executes the read instruction through the flag bit to obtain an instruction execution result, and in a scene with overlong execution time of the lua script, the open sensitivity of the instruction is adjusted through the preset threshold, so that the problems of continuous reconnection and application of the terminal are avoided.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk Solid STATE DISK (SSD)), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.