CN119311556B - Performance evaluation method, system and storage medium for software system - Google Patents
Performance evaluation method, system and storage medium for software system Download PDFInfo
- Publication number
- CN119311556B CN119311556B CN202411859635.7A CN202411859635A CN119311556B CN 119311556 B CN119311556 B CN 119311556B CN 202411859635 A CN202411859635 A CN 202411859635A CN 119311556 B CN119311556 B CN 119311556B
- Authority
- CN
- China
- Prior art keywords
- evaluation
- execution
- functional modules
- functional module
- program
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Prevention of errors by analysis, debugging or testing of software
- G06F11/3604—Analysis of software for verifying properties of programs
- G06F11/3612—Analysis of software for verifying properties of programs by runtime analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
- Stored Programmes (AREA)
Abstract
本申请提供一种用于软件系统的性能评估方法、系统及存储介质，涉及数据处理技术领域，该方法包括基于功能模块创建评估用例，基于评估用例创建对应的评估程序，执行评估程序模拟用户操作获取第一评估结果，预设第二执行结果，对第一执行结果和第二执行结果不同的评估程序对应的功能模块进行修改，对修改后的功能模块，重复执行评估程序获取第二评估结果，训练第一模型预测执行时长，基于预测执行时长获取需要改进的功能模块并进行优化，基于各功能模块之间的相关关系获取相关功能模块，将优化后的功能模块和相关功能模块统称为第一功能模块，对第一功能模块执行对应的评估程序获取第三评估结果，减少了评估范围，提高了软件系统的评估效率。
The present application provides a performance evaluation method, system and storage medium for a software system, and relates to the field of data processing technology. The method includes creating an evaluation case based on a functional module, creating a corresponding evaluation program based on the evaluation case, executing the evaluation program to simulate user operation to obtain a first evaluation result, presetting a second execution result, modifying the functional module corresponding to the evaluation program that has different first and second execution results, repeatedly executing the evaluation program for the modified functional module to obtain a second evaluation result, training a first model to predict execution time, obtaining and optimizing functional modules that need to be improved based on the predicted execution time, obtaining related functional modules based on the correlation between the functional modules, collectively referring to the optimized functional modules and the related functional modules as first functional modules, executing the corresponding evaluation program on the first functional module to obtain a third evaluation result, thereby reducing the evaluation scope and improving the evaluation efficiency of the software system.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a performance evaluation method, system and storage medium for a software system.
Background
With the rapid development of information technology, software systems are becoming more and more complex. Performance assessment is a key step in ensuring that a software system meets user requirements and expected performance. The existing performance evaluation method often depends on manual testing and experience judgment, lacks automation and accuracy, and cannot comprehensively evaluate the performance of the software system in real time.
The invention provides a software system state evaluation method based on parameter correlation, which is disclosed in the China patent application No. CN104679655A and comprises the following steps of 1, determining state parameters of a software system, 2, determining the type of the state parameters, wherein the state parameters of the software system, the performance of which increases with the increase of the state parameter value, are positive correlation parameters, the state parameters of which the performance of the software system decreases with the increase of the state parameter value are negative correlation parameters, 3, collecting the values of all the state parameters in the current software system to obtain measured values of each state parameter, 4, scoring each state parameter according to the type, normal value and measured values of the state parameter, and 5, calculating the total score of all the state parameters to judge the system state of the current software system. In addition, the Chinese invention application with publication number of CN114780359A discloses a software performance quality assessment method, a device and a storage medium, wherein the method comprises the steps of data acquisition, namely acquiring historical data reflecting the software performance quality to form a sample set, wherein each sample comprises a plurality of performance quality index data; extracting principal component analysis based on the sample set to extract a plurality of principal components, determining a principal component expression, constructing a principal component comprehensive model for evaluating the performance quality of the software based on the extracted principal components, and evaluating the performance quality of the software to be evaluated according to the principal component comprehensive model. However, none of the above-mentioned patent applications considers the problem of how to improve the evaluation efficiency, nor how to improve the performance of the software system based on the evaluation result, and based on this, a new technical solution is needed.
Disclosure of Invention
In view of this, the present embodiments provide a performance evaluation method, system, and storage medium for a software system.
The embodiment of the specification provides the following technical scheme:
Step S1, creating a corresponding evaluation case based on a functional module of a software system, creating a corresponding evaluation program based on the evaluation case, executing the evaluation program to simulate user operation, and recording a first evaluation result of executing each evaluation program, wherein the first evaluation result comprises execution time, execution duration, a first execution result and resource utilization rate when executing the functional module;
Step S2, presetting a corresponding second execution result for each evaluation program, acquiring the evaluation programs with different first execution results and second execution results, acquiring the corresponding functional modules based on the evaluation programs, modifying the functional modules, and repeatedly executing the evaluation programs to acquire a second evaluation result in a preset time period;
Step S3, training a first model based on the second evaluation result, wherein the first model can predict the execution duration of each evaluation program under different resource utilization rates, acquiring a predicted execution duration based on the first model, acquiring a functional module to be improved based on the predicted execution duration, and optimizing the functional module to be improved;
And S4, creating a correlation between the function modules, acquiring a correlation function module related to the optimized function module based on the correlation, and collectively called a first function module, and executing a corresponding evaluation program for the first function module to acquire a third evaluation result in a preset time period.
Preferably, the method for creating the correlation between the functional modules comprises the following steps:
based on a software dependency analysis tool, analyzing a reference relation among the function modules, regarding other function modules referring to the function modules as the related function modules of the function modules, counting the function modules with the corresponding first execution results in the second evaluation results of each function module being abnormal at the same time, regarding the function modules with the abnormalities at the same time, counting the first frequency of the first execution results being abnormal at the same time, and regarding the function modules with the first frequency being larger than a preset first threshold value as the related function modules.
Preferably, the function module to be improved is obtained based on the predicted execution duration, and the method comprises the following steps:
Step S31, acquiring a distribution range of the execution duration based on the second execution result, determining a plurality of duration intervals based on the distribution range, and distributing the execution result to the corresponding duration interval based on the execution duration of the second execution result;
Step S32, counting the first number of the second evaluation results contained in each time interval, obtaining the time interval in which the first maximum number is located, and obtaining all execution time lengths of the first time interval, wherein the time interval is called a first time interval;
step S33, obtaining a first mode of all the execution durations, obtaining a first resource utilization rate corresponding to the first mode, obtaining a first predicted execution duration under the first resource utilization rate, comparing whether the first mode is smaller than or equal to the first predicted execution duration, if yes, judging that the corresponding function module does not need to be improved, judging that whether the resource configuration of the software system needs to be optimized, and if not, judging that the corresponding function module needs to be improved.
Preferably, the determining whether the resource configuration of the software system needs to be optimized includes the following steps:
step 331, judging whether the first time interval is a maximum time interval, wherein the maximum time interval is the time interval in which the maximum execution time is located, and if yes, ending the step;
Step S332, if not, obtaining all the execution durations of the maximum duration interval, obtaining a second mode of the execution durations, further obtaining a second resource utilization rate corresponding to the execution duration, obtaining a second predicted execution duration under the second resource utilization rate, calculating a first change rate of the first mode and the second mode, calculating a second change rate of the first predicted duration and the second predicted duration, and if the second change rate is smaller than the first change rate, judging that the resource configuration of the software system needs to be optimized.
Preferably, the method for creating the corresponding evaluation case based on the functional module of the software system comprises the following steps:
step S11, acquiring a requirement document, a program code and a modification record of the program code of the software system, associating each functional module and the corresponding program code with each other based on the requirement document, and determining evaluation angles based on the program code and the functional module, wherein the number of the evaluation angles is more than or equal to one;
step S12, collecting relevant data for the evaluation angles, carrying out quantification on the evaluation angles based on the relevant data to determine corresponding first evaluation values, obtaining the corresponding first evaluation values for each evaluation angle, and calculating an evaluation value of each functional module based on the first evaluation values, wherein the evaluation value is used for evaluating the risk degree of each functional module;
step S13, setting a plurality of different evaluation cases for the functional module with the evaluation value larger than a preset evaluation threshold, wherein the evaluation cases comprise input data, an execution step and an expected result.
Preferably, the step of acquiring the corresponding first evaluation value for each of the evaluation angles includes the steps of:
The related data is to correspond to a first modification number of the program code in a preset first time period in the past, and a result value obtained by dividing the first modification number by the total number of times the program code is submitted is used as a first evaluation value of the evaluation angle.
Preferably, calculating the evaluation value of each of the functional modules includes the steps of:
step S121, setting a corresponding weight value for each evaluation angle of valence degrees, multiplying the weight value corresponding to each evaluation angle of valence degrees of each functional module by a corresponding quantization value, and then adding to obtain a total evaluation value;
Step S122, obtaining first abnormal times of the program codes corresponding to the functional modules, wherein the first abnormal times are found abnormal in the examined process, and dividing the first abnormal times by the total abnormal times corresponding to the functional modules to obtain second evaluation values;
And step S123, calculating an average value of the total evaluation value and the second evaluation value, and taking the average value as the evaluation value of the functional module.
The invention also provides a performance evaluation system for a software system, comprising the following modules:
The first evaluation unit is used for creating a corresponding evaluation case based on a functional module of the software system, creating a corresponding evaluation program based on the evaluation case, executing the evaluation program to simulate user operation, and recording a first evaluation result of executing each evaluation program, wherein the first evaluation result comprises execution time, execution duration, a first execution result and resource utilization rate when the functional module is executed;
The second evaluation unit is used for presetting a corresponding second execution result for each evaluation program, acquiring the evaluation programs with different first execution results and second execution results, acquiring the corresponding functional modules based on the evaluation programs, modifying the functional modules, and repeatedly executing the evaluation programs to acquire a second evaluation result in a preset time period;
The function optimization unit is used for training a first model based on the second evaluation result, the first model can predict the execution duration of each evaluation program under different resource utilization rates, acquiring the predicted execution duration based on the first model, acquiring a function module needing improvement based on the predicted execution duration, and optimizing the function module needing improvement;
and the third evaluation unit is used for creating a correlation between the functional modules, acquiring a correlation function module related to the optimized functional module based on the correlation, and collectively called a first functional module, and executing a corresponding evaluation program for the first functional module to acquire a third evaluation result in a preset time period.
The present invention also provides a storage medium storing program instructions, wherein the program instructions, when executed, control a device in which the storage medium is located to perform the performance evaluation method for a software system according to any one of the above
Compared with the prior art, the beneficial effects that above-mentioned at least one technical scheme that this description embodiment adopted can reach include at least:
The invention creates a comprehensive evaluation case based on a plurality of evaluation angles, ensures that a software system can meet expected quality standards in all aspects, obtains functional modules which are inconsistent with expected execution results through a first execution result, modifies the functional modules, obtains a second execution result through repeatedly executing an evaluation program in a budget period, creates a first model based on the second execution result, predicts execution time, and judges which functional modules need to be improved based on the second execution result and the predicted execution time
The method comprises the steps of carrying out corresponding optimization on the functional modules to be improved based on the judging result, pertinently improving the performance of program codes corresponding to the functional modules, further improving the performance of the whole software system, and carrying out evaluation on the functional modules related to the functional modules obtained and optimized based on the correlation, thereby improving the pertinence of evaluation, reducing the evaluation range and improving the evaluation efficiency of the software system.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of steps of a method for performance evaluation of a software system in an embodiment of the application;
FIG. 2 is a block diagram of the components of a performance evaluation system for a software system in an embodiment of the application.
Detailed Description
Embodiments of the present application will be described in detail below with reference to the accompanying drawings.
Other advantages and effects of the present application will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present application with reference to specific examples. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. The application may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present application. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
It is noted that various aspects of the embodiments are described below within the scope of the following claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is merely illustrative. Based on the present disclosure, one skilled in the art will appreciate that one aspect described herein may be implemented independently of any other aspect, and that two or more of these aspects may be combined in various ways. For example, apparatus may be implemented and/or methods practiced using any number and aspects set forth herein. In addition, such apparatus may be implemented and/or such methods practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.
It should also be noted that the illustrations provided in the following embodiments merely illustrate the basic concept of the present application by way of illustration, and only the components related to the present application are shown in the drawings and are not drawn according to the number, shape and size of the components in actual implementation, and the form, number and proportion of the components in actual implementation may be arbitrarily changed, and the layout of the components may be more complicated.
In addition, in the following description, specific details are provided in order to provide a thorough understanding of the examples. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details.
For easy understanding, the following describes a performance evaluation method for a software system in an embodiment of the present application, please refer to fig. 1, and the performance evaluation method for a software system in the embodiment of the present application is implemented by the following steps:
Step S1, a corresponding evaluation case is created based on a functional module of a software system, a corresponding evaluation program is created based on the evaluation case, the evaluation program is executed to simulate user operation, and a first evaluation result of executing each evaluation program is recorded, wherein the first evaluation result comprises execution time, execution duration, a first execution result and resource utilization rate when the functional module is executed.
Specifically, in order to perform performance evaluation on a software system, an evaluation case is created based on each functional module of the software system, the creation of a conventional evaluation case is mostly based on knowledge and experience of an evaluator, which may result in insufficient or incorrect evaluation, so that the comprehensiveness and effectiveness of the evaluation need to be improved, a specific process of creating the evaluation case is explained in detail later, an evaluation program is created based on the created evaluation case, that is, a corresponding evaluation code is created based on the evaluation case, the evaluation program is executed to simulate various operations that a user may perform during the use of the software system, and a first evaluation result of the evaluation program is recorded, wherein the execution time refers to an execution time point when the evaluation program is executed, the execution time refers to a total time for executing the evaluation program, the first execution result refers to an execution result output when the evaluation program is executed, the software program is managed by a person, the execution result is success or failure of the addition, and the resource utilization rate when the functional module is executed refers to the p utilization rate, the memory utilization rate, and the like when the evaluation program is executed.
Step S2, presetting a corresponding second execution result for each evaluation program, acquiring the evaluation programs with different first execution results and second execution results, acquiring corresponding functional modules based on the evaluation programs, modifying the functional modules, and repeatedly executing the evaluation programs to acquire the second evaluation results in a preset time period for the modified functional modules.
Specifically, the second execution result is an execution result that each evaluation program should output in case that the function of the function module is not wrong, but if the function module is problematic, different execution results may occur and be expected when the evaluation program is used to evaluate the function module, therefore, by comparing the first execution result and the second execution result to obtain the evaluation program different from the expected execution result, when the function module corresponding to the evaluation program is obtained, the functional module with the problem can be quickly obtained by the above method, and the function modules can be modified, and the modified function module can evaluate the repeated execution evaluation program of the function module in a preset time period, such as in one day, and then the software performance of the software program in different time periods can be judged by the obtained second evaluation result.
And step S3, training a first model based on a second evaluation result, wherein the first model can predict the execution time of each evaluation program under different resource utilization rates, acquiring the predicted execution time based on the first model, acquiring a functional module needing improvement based on the predicted execution time, and optimizing the functional module needing improvement.
Specifically, in general, a software system may have different access amounts in different time periods in a day, when the access amount is large, performance of the software system may be affected, in order to scientifically optimize the software system, based on the obtained second evaluation result, a first model is trained, based on the first model, whether the execution duration of a program needs to be increased for the software system is predicted under different resource utilization rates, or when the resource utilization rate is lower, it is found that the execution duration is longer, program codes of the corresponding function modules may be described to be optimized, the function modules needing to be improved may be accurately obtained based on the predicted execution duration, a specific method for judging which function modules need to be improved may be explained in detail later may be correspondingly optimized based on the judgment result.
And S4, creating a correlation between the functional modules, acquiring a correlation function module related to the optimized functional module based on the correlation, and collectively called a first functional module, and executing a corresponding evaluation program for the first functional module to acquire a third evaluation result in a preset time period.
Specifically, the software system may be iterated and optimized continuously over time, but each function module of the software system may be optimized by only one function module, if each iteration and optimization of the software system perform evaluation on all the function modules once, the evaluation efficiency may be reduced, the evaluation duration may also be reduced with increase, so as to improve the evaluation efficiency and effectively identify the evaluation content during the evaluation, reduce the evaluation range, shorten the evaluation time, after optimizing the software system, obtain the relevant function module related to the optimized function module by creating the correlation between the function modules, and collectively refer to the optimized function module and the relevant function module as the first function module, and then execute the corresponding evaluation program for the first function module in a preset time period to obtain the third evaluation result, wherein the evaluation program is newly created based on the optimized function module, and after obtaining the third evaluation result, the relevant staff can evaluate the software system based on the third evaluation result.
Further, the method for creating the correlation between the functional modules comprises the following steps:
Based on the software dependency analysis tool, analyzing the reference relation among the functional modules, regarding other functional modules of the reference functional modules as related functional modules of the functional modules, counting the functional modules with the corresponding first execution results in the second evaluation results of each functional module and simultaneously abnormal, regarding the functional modules with the simultaneously abnormal, counting the first frequency of the simultaneous abnormal occurrence of the first execution results, and regarding the functional modules with the first frequency larger than the preset first threshold as related functional modules.
Specifically, in order to quickly acquire the functional modules related to the optimized functional modules, firstly, the software dependency analysis tool is used for analyzing the reference relation between the functional modules, in the process of editing the program code, if one functional module references other functional modules, reference information is usually added in front of an entry file, the software dependency analysis tool can acquire the referenced other functional modules based on the reference information, can acquire the reference relation between the functional modules based on the reference information, can acquire the related functional modules of each functional module, and can acquire the functional module of each functional module based on the second execution result for further evaluation, wherein the evaluation abnormality refers to the functional module with the evaluation abnormality occurring simultaneously, the evaluation abnormality refers to the first evaluation result being different from the expected second evaluation result, and if the two functional modules always occur simultaneously, the two functional modules are considered to have correlation, and the first frequency refers to the number of times that the evaluation abnormality occurs simultaneously, and the functional module with the evaluation abnormality occurring simultaneously is also used as the related functional module through the method, so that the occurrence of insufficient evaluation in the subsequent evaluation is prevented.
Further, the method for acquiring the functional module needing improvement based on the predicted execution duration comprises the following steps:
Step S31, acquiring a distribution range of execution time based on a second execution result, determining a plurality of time intervals based on the distribution range, and distributing the execution result to the corresponding time intervals based on the execution time of the second execution result;
step S32, counting the first number of second evaluation results contained in each time interval, acquiring a time interval in which the first number is located, and acquiring all execution time lengths of the first time interval, wherein the time interval is called a first time interval;
Step S33, obtaining a first mode of all execution time periods, obtaining a first resource utilization rate corresponding to the first mode, obtaining a first prediction execution time period under the first resource utilization rate, comparing whether the first mode is smaller than or equal to the first prediction execution time period, if yes, judging that the corresponding functional module does not need to be improved, judging whether the resource configuration of the software system needs to be optimized, and if not, judging that the corresponding functional module needs to be improved.
Specifically, in order to determine which functional modules need improvement, since multiple evaluations are performed within a predetermined period of time, multiple second execution results are obtained, each second execution result includes one execution time length, based on the obtained distribution range of all the execution time lengths, for example, the shortest execution time length is 50ms, the longest execution time length is 3000ms, the distribution range of the execution time lengths is 50ms-3000ms, assuming that the distribution range of the execution time lengths is divided into three time length sections, the three time length sections are divided according to historic experience, for example, the execution time length is distributed in the section of 500ms-1800ms in most cases, the execution time length is divided into three time length sections of 50ms-500ms,500 ms-180ms, 180 ms-3000ms, each second evaluation result is respectively allocated in the corresponding time length section based on the execution time length, the first number of the second evaluation results included in each time length section is obtained, obtaining a first time interval corresponding to the maximum first quantity, obtaining all execution time lengths corresponding to the first time interval, obtaining a first mode of all execution time lengths, wherein the first mode is the execution time length with the maximum occurrence number in the first time interval, taking the first mode as the execution time length of an evaluation program under normal conditions, obtaining a first resource utilization rate corresponding to the first mode based on a second evaluation result, predicting a first prediction execution time length corresponding to the first resource utilization rate by using a first model, wherein the first prediction execution time length refers to the execution time length which should be used for executing the evaluation program under the condition of the first resource utilization rate, and indicating that the execution time length of a corresponding function module is relatively smaller under the condition that the first mode is smaller than or equal to the first prediction time length, and simultaneously indicating that the performance of the function module is better, therefore, it is judged that the corresponding function module does not need to be improved, but only represents the situation that the execution duration is in the first execution interval, and there is a situation that the execution duration is longer, for example, the execution duration of 1800ms-3000ms is possibly the situation that the performance is reduced due to too many users accessing simultaneously, so that further judgment is needed, and in particular, a detailed explanation will be made later, if not, it is explained that the execution duration in the duration interval represented by the first mode is relatively longer, and the performance of the corresponding function module is possibly not good, so that the corresponding function module needs to be optimized, for example, the logic of the corresponding program code is optimized, and the performance of the corresponding function module is improved.
Further, determining whether optimization of the resource configuration of the software system is required includes the following steps:
Step S331, determining whether the first time interval is a maximum time interval, where the maximum time interval is a time interval where the maximum execution time is located, and if yes, ending the step.
Specifically, under the condition of the first resource utilization rate corresponding to the first time interval, the corresponding execution time length is smaller, the maximum time length interval is the time length interval where the maximum execution time length is located, the longest execution time length is relatively shorter, and the performance of the software system is described, so that the function module does not need to be optimized, the resource utilization rate corresponding to the longest execution time length is generally larger, the execution time length is also shorter under the condition of larger resource utilization rate, the performance of the software system is described as being good enough, and the resource configuration is suitable, so that the optimization of the resource configuration of the software system is not required.
Step S332, under the condition of no, acquiring all execution time periods of the maximum time period, acquiring a second mode of the execution time period, further acquiring a second resource utilization rate corresponding to the execution time period, acquiring a second predicted execution time period under the second resource utilization rate, calculating a first change rate of the first mode and the second mode, calculating a second change rate of the first predicted time period and the second predicted time period, and judging that the resource configuration of the software system needs to be optimized if the second change rate is smaller than the first change rate.
Specifically, if no, it is indicated that there is a longer execution duration, where the longer execution duration may correspond to a larger resource utilization rate, so that all execution durations corresponding to the maximum duration interval are acquired, a second mode is acquired therein, then a corresponding second resource utilization rate is acquired, and a first model is used to predict a second predicted execution duration under the second resource utilization rate, and a first change rate and a second change rate are calculated, where the first change rate is a change rate of an actual execution duration under the condition that the resource utilization rate increases, and the second change rate is a change rate of an execution duration under the condition that the resource utilization rate increases, and if the second change rate is smaller than the first change rate, it is indicated that the execution duration is too fast under the condition that the resource utilization rate increases, and an expected effect is not met, and therefore, it is determined that the resource configuration of the software system needs to be optimized under the condition.
Further, creating a corresponding evaluation case based on a functional module of the software system, including the following steps:
Step S11, acquiring a demand document, program codes and a modification record of the program codes of the software system, associating each functional module with the corresponding program codes based on the demand document, and determining evaluation angles based on the program codes and the functional modules, wherein the number of the evaluation angles is more than or equal to one;
Step S12, collecting relevant data for the evaluation angles, quantifying the evaluation angles based on the relevant data to determine corresponding first evaluation values, obtaining corresponding first evaluation values for each evaluation angle, and calculating evaluation values of each functional module based on the first evaluation values, wherein the evaluation values are used for evaluating the risk degree of each functional module;
Step S13, setting a plurality of different evaluation cases for the functional module with the evaluation value larger than the preset evaluation threshold, wherein the evaluation cases comprise input data, execution steps and expected results.
Specifically, the requirement document is a document for recording a usage instruction of a software system and functional details of each functional module, the functional modules and the corresponding program codes can be associated based on the requirement document and the program codes, then different evaluation angles can be determined based on the functional details of each functional module and the corresponding program codes, the evaluation angles include an integrity angle, a reliability angle, a compatibility angle and the like, the functional integrity angle refers to whether the functional module considers all small functional points under the functional module, the reliability angle refers to whether the functional module is stable under the condition of continuous operation for a long time, the compatibility angle refers to whether the functional module can be executed under the condition of different types of equipment, and the like, relevant data is collected for each evaluation angle, the evaluation angles are quantized based on the relevant data to obtain corresponding first evaluation values, and the evaluation values of the functional modules are calculated based on the first evaluation values.
Further, the method for acquiring the corresponding first evaluation value for each evaluation angle includes the following steps:
The related data refers to a first modification number of the corresponding program code in a preset first time period in the past, and a result value obtained by dividing the first modification number by the total number of times the program code is submitted is used as a first evaluation value of the evaluation angle.
Specifically, taking the evaluation angle of the functional integrity as an example for quantifying, the first modification number refers to the modification number of the program code for the missing function in a preset time period in the past, the preset first time period in the past refers to a self-checking stage of a developer, that is, a self-checking time reserved for the developer in the time period from the completion of the development of the functional module to the submission of the evaluation, if the modification number is more, it may indicate that the corresponding functional module has more holes, and the possibility of subsequent errors is relatively high, so that the evaluation can be performed based on the modification number, the modification number is assumed to be 2 times, the total number refers to the total number of times of code submitted in the development process of the functional module, and the value obtained by 2/8 is assumed to be 8 times, as the first evaluation value.
Further, calculating an evaluation value of each functional module includes the steps of:
step S121, setting a corresponding weight value for each evaluation angle, multiplying the corresponding weight value of each evaluation angle of each functional module by a corresponding quantized value, and then adding to obtain a total evaluation value;
Step S122, obtaining first abnormal times of the program codes corresponding to the functional modules when the program codes find abnormality in the examined process, and dividing the first abnormal times by the total abnormal times corresponding to the functional modules to obtain second evaluation values;
Step S123, calculating an average value of the total evaluation value and the second evaluation value, and taking the average value as the evaluation value of the functional module.
Specifically, in different software systems, different evaluation angles may be different, based on the actual situation of the software system, for example, the software system is a specific software system for a small number of people, and there is no situation that a plurality of people use, the weight value of the compatibility angle may be set to be smaller, the total evaluation value is obtained through the step S121, a certain specification, such as a necessary naming specification, a submitting specification, etc., is set for the program code in the process of developing the program software, if the specification is not met during the process of submitting the program code, abnormality is prompted and the program code cannot be submitted successfully, the more the number of times that the corresponding program code is found abnormal in the process of developing indicates that the developer who develops the function module may not be careful, so the possibility that the corresponding function module has a problem is higher, therefore, the second evaluation value is calculated through the step S122, and finally, the total evaluation value and the average value of the second evaluation value are taken as the evaluation value, so that the evaluation value of the function module is more scientific can be obtained.
According to another aspect of the embodiment of the present invention, referring to fig. 2, there is further provided a performance evaluation system for a software system, including a first evaluation unit, a second evaluation unit, a function optimization unit, and a third evaluation unit, for implementing a performance evaluation method for a software system as described above, where specific functions of each module are as follows:
The first evaluation unit is used for creating a corresponding evaluation case based on a functional module of the software system, creating a corresponding evaluation program based on the evaluation case, executing the evaluation program to simulate user operation, and recording a first evaluation result of executing each evaluation program, wherein the first evaluation result comprises execution time, execution duration, a first execution result and resource utilization rate when the functional module is executed;
The second evaluation unit is used for presetting a corresponding second execution result for each evaluation program, acquiring the evaluation programs with different first execution results and second execution results, acquiring corresponding functional modules based on the evaluation programs, modifying the functional modules, and repeatedly executing the evaluation programs to acquire the second evaluation results in a preset time period;
The function optimization unit is used for training a first model based on the second evaluation result, the first model can predict the execution time of each evaluation program under different resource utilization rates, acquiring the predicted execution time based on the first model, acquiring a function module needing improvement based on the predicted execution time, and optimizing the function module needing improvement;
The third evaluation unit is used for creating a correlation between the functional modules, acquiring a correlation function module related to the optimized functional module based on the correlation, and collectively called a first functional module, and executing a corresponding evaluation program for the first functional module in a preset time period to acquire a third evaluation result.
According to another aspect of the embodiment of the present invention, there is further provided a storage medium storing program instructions, where the program instructions, when executed, control a device in which the storage medium is located to perform a performance evaluation method for a software system according to any one of the above.
In summary, the method includes creating an evaluation case based on a function module, creating a corresponding evaluation program based on the evaluation case, executing the evaluation program to simulate user operation to obtain a first evaluation result, presetting a second execution result, modifying a function module corresponding to the evaluation program with different first and second execution results, repeatedly executing the evaluation program to obtain the second evaluation result, training a first model to predict execution duration, obtaining a function module to be improved based on the predicted execution duration, optimizing the function module, obtaining a related function module based on a related relation between the function modules, collectively called a first function module, and executing the corresponding evaluation program to the first function module to obtain a third evaluation result. The application can improve the performance evaluation efficiency of the software system.
In this specification, identical and similar parts of the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the description is relatively simple for the embodiments described later, and reference is made to the description of the foregoing embodiments for relevant points.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any changes or substitutions easily contemplated by those skilled in the art within the scope of the present application should be included in the present application. Therefore, the protection scope of the application is subject to the protection scope of the claims.
Claims (7)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411859635.7A CN119311556B (en) | 2024-12-17 | 2024-12-17 | Performance evaluation method, system and storage medium for software system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411859635.7A CN119311556B (en) | 2024-12-17 | 2024-12-17 | Performance evaluation method, system and storage medium for software system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119311556A CN119311556A (en) | 2025-01-14 |
| CN119311556B true CN119311556B (en) | 2025-04-04 |
Family
ID=94189800
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411859635.7A Active CN119311556B (en) | 2024-12-17 | 2024-12-17 | Performance evaluation method, system and storage medium for software system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119311556B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN120066472A (en) * | 2025-04-28 | 2025-05-30 | 深圳有为通讯科技有限公司 | Smartphone function module customizing method and system based on multi-industry application scene |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114490349A (en) * | 2021-12-30 | 2022-05-13 | 北京百度网讯科技有限公司 | Software quality testing method, device, electronic equipment and storage medium |
| CN117076315A (en) * | 2023-08-22 | 2023-11-17 | 中国工商银行股份有限公司 | Quality evaluation method and device for test cases and electronic equipment |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11481311B2 (en) * | 2020-06-10 | 2022-10-25 | Sap Se | Automatic evaluation of test code quality |
| US12061903B2 (en) * | 2022-09-16 | 2024-08-13 | Microsoft Technology Licensing, Llc | Software development quality assessment |
| CN117093477A (en) * | 2023-08-11 | 2023-11-21 | 中国平安财产保险股份有限公司 | Software quality assessment method and device, computer equipment and storage medium |
-
2024
- 2024-12-17 CN CN202411859635.7A patent/CN119311556B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114490349A (en) * | 2021-12-30 | 2022-05-13 | 北京百度网讯科技有限公司 | Software quality testing method, device, electronic equipment and storage medium |
| CN117076315A (en) * | 2023-08-22 | 2023-11-17 | 中国工商银行股份有限公司 | Quality evaluation method and device for test cases and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119311556A (en) | 2025-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8087001B2 (en) | Computer-implemented systems and methods for software application testing | |
| US9280436B2 (en) | Modeling a computing entity | |
| CN119311556B (en) | Performance evaluation method, system and storage medium for software system | |
| CN107992410B (en) | Software quality monitoring method and device, computer equipment and storage medium | |
| US8170894B2 (en) | Method of identifying innovations possessing business disrupting properties | |
| CN109242363B (en) | Full life cycle test management platform based on multiple quality control models | |
| CN112632179A (en) | Model construction method and device, storage medium and equipment | |
| CN119829469B (en) | Firmware testing method, electronic device, storage medium and program product | |
| US20050278301A1 (en) | System and method for determining an optimized process configuration | |
| CN116737554A (en) | An intelligent analysis and processing system and method based on big data | |
| KR102195629B1 (en) | Method for selecting workers based on capability of work in crowdsourcing based projects for artificial intelligence training data generation | |
| JP3883449B2 (en) | Software system test plan creation support method and test plan creation support program | |
| CN116954624B (en) | Compilation method, software development system and server based on software development kit | |
| US20110078102A1 (en) | System and method for detecting system relationships by correlating system workload activity levels | |
| Lee et al. | Software reliability prediction for open source software adoption systems based on early lifecycle measurements | |
| US20220114519A1 (en) | Method for Device Monitoring | |
| CN116795710A (en) | Performance evaluation method and device for real-time system | |
| CN115222149A (en) | Resource conversion parameter management method, device, equipment and storage medium | |
| CN106020982A (en) | Method for simulating resource consumption of software component | |
| CN106855840B (en) | System CPU analysis method and device | |
| Jain et al. | Markovian software reliability model for two types of failures with imperfect debugging rate and generation of errors | |
| US20250061038A1 (en) | System and method for managing software application performance | |
| CN119201652B (en) | A method for testing server hardware performance | |
| US12381942B2 (en) | Correlations between workload characteristics and elapsed times | |
| CN113010269B (en) | Virtual machine scheduling method and device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |