CN111857793B

CN111857793B - Training method, device, equipment and storage medium of network model

Info

Publication number: CN111857793B
Application number: CN201910360267.4A
Authority: CN
Inventors: 王翔
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2019-04-30
Filing date: 2019-04-30
Publication date: 2024-06-07
Anticipated expiration: 2039-04-30
Also published as: CN111857793A

Abstract

The application discloses a training method, device and equipment of a network model and a storage medium, and belongs to the technical field of data processing. The method comprises the following steps: displaying a first configuration interface, wherein the first configuration interface comprises a plurality of reference data enhancement modes; receiving a first configuration instruction based on a first configuration interface, wherein the first configuration instruction is triggered after a user configures at least one reference data enhancement mode in a plurality of reference data enhancement modes; according to the first configuration instruction, target configuration content is acquired, the target configuration content is used for acquiring an enhanced training sample, and the enhanced training sample is obtained by carrying out data enhancement on an original training sample according to a data enhancement mode responded by the target configuration content. Therefore, after the network model to be trained is trained based on the enhanced training sample, the network model obtained through training can be ensured to be better suitable for the application scene.

Description

Training method, device, equipment and storage medium of network model

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for training a network model.

Background

Currently, network models such as object detection, classification, identification, etc. are widely used. Such network models typically require training prior to use. Currently, a deep learning algorithm is generally adopted, and features are automatically learned from a large number of training samples in an iterative training mode, so that training of a model is completed.

In the deep learning algorithm, a larger number of training samples are generally required, and for this purpose, data enhancement processing may be generally performed on the existing training samples in a data enhancement manner, so as to expand the existing training samples, thereby obtaining a large number of training samples, and training the network model based on the obtained large number of training samples.

However, currently, a given data enhancement method is used to perform data enhancement processing on the training samples, but the given data enhancement method cannot be applied to various application scenarios, so that the training effect of the training samples subjected to data enhancement processing on the network model in some application scenarios is poor.

Disclosure of Invention

The embodiment of the application provides a training method, device, equipment and storage medium of a network model, which can solve the problem that a given data enhancement mode cannot be applied to various different application scenes in the related technology. The technical scheme is as follows:

In one aspect, a method for training a network model is provided, the method comprising:

displaying a first configuration interface, wherein the first configuration interface comprises a plurality of reference data enhancement modes;

Receiving a first configuration instruction based on the first configuration interface, wherein the first configuration instruction is triggered after a user configures at least one reference data enhancement mode in the multiple reference data enhancement modes;

According to the first configuration instruction, target configuration content is obtained, the target configuration content is used for obtaining an enhanced training sample, and the enhanced training sample is obtained by carrying out data enhancement on the training sample according to a data enhancement mode responded by the target configuration content.

In another aspect, there is provided a training apparatus for a network model, the apparatus comprising:

the display module is used for displaying a first configuration interface, and the first configuration interface comprises a plurality of reference data enhancement modes;

The receiving module is used for receiving a first configuration instruction based on the first configuration interface, wherein the first configuration instruction is triggered after a user configures at least one reference data enhancement mode in the multiple reference data enhancement modes;

The acquisition module is used for acquiring target configuration content according to the first configuration instruction, wherein the target configuration content is used for acquiring an enhanced training sample, and the enhanced training sample is obtained by carrying out data enhancement on the training sample according to a data enhancement mode responded by the target configuration content.

In another aspect, there is provided an electronic device comprising:

A processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to implement the training method of any of the network models described above.

In another aspect, a computer readable storage medium having instructions stored thereon that when executed by a processor implement a training method for any of the above network models is provided.

In another aspect, a computer program product is provided comprising instructions that, when executed on a computer, cause the computer to perform the training method of any of the network models described above.

The technical scheme provided by the embodiment of the application has the beneficial effects that:

And displaying a first configuration interface comprising a plurality of reference data enhancement modes, wherein a user can configure at least one data enhancement mode in the first configuration interface, correspondingly, a first configuration instruction triggered after user configuration is received, target configuration content is obtained according to the first configuration instruction, and data enhancement processing is carried out on an original training sample according to the data enhancement mode responded by the target configuration content so as to obtain an enhanced training sample. Because the target configuration content is configured by a user according to the actual requirements of the application scene, the obtained enhanced training sample can meet the training requirements of the network model in the application scene, and the network model obtained by training can be ensured to be better suitable for the application scene.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flowchart of a training method for a network model, according to an exemplary embodiment;

FIG. 2 is a flowchart of a training method for a network model, according to another exemplary embodiment;

FIG. 3 is a schematic display of a configuration interface shown in accordance with an exemplary embodiment;

FIG. 4 is a schematic display of a configuration interface shown in accordance with an exemplary embodiment;

FIG. 5 is a flowchart of a training method for a network model, according to another exemplary embodiment;

FIG. 6 is a schematic diagram of a training apparatus of a network model, according to an example embodiment;

FIG. 7 is a schematic diagram of a training apparatus of a network model according to another exemplary embodiment;

FIG. 8 is a schematic diagram of a training apparatus of a network model according to another exemplary embodiment;

fig. 9 shows a block diagram of an electronic device 900 provided by an exemplary embodiment of the application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.

Before describing the training method of the network model provided by the embodiment of the application in detail, the application scene and the implementation environment related to the embodiment of the application are briefly described.

First, an application scenario according to an embodiment of the present application is briefly described.

In the technical fields of target detection, recognition, classification and the like, most algorithms based on traditional pattern recognition and machine learning adopt manually designed features for training, and the problems of low precision and poor generalization capability often exist. Therefore, the deep learning algorithm is provided for training the network model, and the principle is that the characteristics are automatically learned from a large number of training samples through iterative training, so that the training of the network model is realized, and compared with the traditional algorithm, the deep learning algorithm has better precision and generalization capability, and is widely applied to a plurality of fields in recent years. Deep learning algorithms typically require a huge number of training samples, such as thousands or even tens of thousands of training samples, which are difficult to obtain in some application scenarios, such as defect detection. Based on this, a common processing method is to process the existing training samples by adopting a data enhancement method to expand the existing training samples, so as to obtain a large number of training samples. However, when the fixed data enhancement mode is used in different application scenarios for network training, the fixed data enhancement mode may be difficult to be applied to some application scenarios, for example, a certain application scenario is a high brightness scenario, and the given data enhancement mode includes randomly adjusting the brightness of the sample from low to high in a wide range, so that the adaptability to the application scenario is limited. Therefore, the embodiment of the application provides a training method for a network model, which can support a user to configure a data enhancement mode on line according to the actual requirement of an application scene, so that a training sample processed by the configured data enhancement mode can meet the training requirement of the network model in the application scene, and the network model obtained by training can be ensured to have stronger adaptability in the application scene. For specific implementation, please refer to the following examples.

Next, an implementation environment related to the embodiment of the present application will be briefly described.

The training method of the network model provided by the embodiment of the application can be executed by the electronic equipment, and the electronic equipment can be provided with the display screen so as to realize man-machine interaction through the display screen. The electronic device may be a computer device or an embedded device, and in some embodiments, the computer device may be a tablet computer, a notebook computer, a desktop computer, a portable computer, or the like, which is not limited in this embodiment of the present application.

After describing application scenarios and implementation environments related to the embodiments of the present application, a detailed description will be given next to a training method of a network model provided by the embodiments of the present application with reference to the accompanying drawings.

FIG. 1 is a flowchart illustrating a method of training a network model, which may include the following steps, according to an exemplary embodiment:

Step 101: displaying a first configuration interface, wherein the first configuration interface comprises a plurality of reference data enhancement modes;

Step 102: and receiving a first configuration instruction based on the first configuration interface, wherein the first configuration instruction is triggered after a user configures at least one reference data enhancement mode in the plurality of reference data enhancement modes.

Step 103: and acquiring target configuration content according to the first configuration instruction, wherein the target configuration content is used for acquiring an enhanced training sample, and the enhanced training sample is obtained by carrying out data enhancement on an original training sample according to a data enhancement mode responded by the target configuration content.

In the embodiment of the application, a first configuration interface comprising a plurality of reference data enhancement modes is displayed, a user can configure at least one data enhancement mode in the first configuration interface, correspondingly, a first configuration instruction triggered after user configuration is received, target configuration content is obtained according to the first configuration instruction, and data enhancement processing is carried out on an original training sample according to the data enhancement mode responded by the target configuration content so as to obtain an enhanced training sample. Because the target configuration content is configured by a user according to the actual requirements of the application scene, the obtained enhanced training sample can meet the training requirements of the network model in the application scene, and the network model obtained by training can be ensured to be better suitable for the application scene.

In one possible implementation manner of the present application, according to the first configuration instruction, obtaining the target configuration content includes:

Determining a reference data enhancement mode selected by a user from the plurality of reference data enhancement modes according to the first configuration instruction;

Displaying a second configuration interface based on the determined reference data enhancement mode;

receiving a second configuration instruction in the second configuration interface, wherein the second configuration instruction is triggered after the user performs parameter configuration on the determined parameters of the reference data enhancement mode;

and determining the target configuration content based on the reference data enhancement mode after parameter configuration operation according to the second configuration instruction.

determining two or more reference data enhancement modes selected by a user from the plurality of reference data enhancement modes according to the first configuration instruction;

receiving a third configuration instruction in the second configuration interface, wherein the third configuration instruction is triggered after the user performs parameter configuration on the determined parameters of each reference data enhancement mode;

and according to the third configuration instruction, determining the target configuration content based on the reference data enhancement mode after parameter configuration operation.

In one possible implementation manner of the present application, after obtaining the target configuration content according to the first configuration instruction, the method further includes:

Performing data enhancement processing on the original training sample based on a data enhancement mode responded by the target configuration content to obtain the enhanced training sample;

and training the network model to be trained based on the enhanced training sample.

In one possible implementation manner of the present application, before training the network model to be trained based on the enhanced training sample, the method further includes:

Obtaining a test sample;

The training of the network model to be trained based on the enhanced training sample comprises the following steps:

performing iterative training on the network model to be trained based on the enhanced training sample;

when the iterative training times reach a training time threshold value, evaluating a network model obtained by current training through the test sample;

when the evaluation result does not reach the reference performance, adjusting the target configuration content;

And carrying out data enhancement processing on the original training sample based on a data enhancement mode responded by the adjusted target configuration content, re-determining the obtained training sample as an enhanced training sample, and returning to the operation of carrying out iterative training on the network model to be trained based on the enhanced training sample until the evaluation result reaches the reference performance or the total number of iterative training reaches the reference training number.

In one possible implementation of the present application, adjusting the target configuration content includes:

Displaying a third configuration interface, receiving an adjustment instruction for the target configuration content based on the third configuration interface, and adjusting the target configuration content according to the adjustment instruction;

Or alternatively

And respectively acquiring characteristic information of various characteristics of the test sample and the enhanced training sample, correspondingly acquiring a first characteristic information set and a second characteristic information set, and adjusting the target configuration content according to the first characteristic information set and the second characteristic information set, wherein various characteristics correspond to a reference data enhancement mode.

In one possible implementation manner of the present application, adjusting the target configuration content according to the first feature information set and the second feature information set includes:

for any one of the features, acquiring feature information of any one feature from the first feature information set and the second feature information set respectively;

according to the obtained characteristic information, counting a first characteristic range and a second characteristic range corresponding to any one of the characteristics, wherein the first characteristic range is a characteristic range corresponding to the test sample, and the second characteristic range is a characteristic range corresponding to the enhanced training sample;

When the first feature range is different from the second feature range, if the target configuration content does not include the reference data enhancement mode corresponding to any feature, configuring the reference data enhancement mode corresponding to any feature in the target configuration content to obtain adjusted target configuration content; and if the target configuration content comprises the reference data enhancement mode corresponding to any feature, adjusting parameters of the reference data enhancement mode corresponding to any feature according to the adjustment step length threshold.

In one possible implementation of the present application, the plurality of reference data enhancements include at least two of rotation, mirroring, scaling, noise addition, blurring, brightness adjustment, and cropping.

All the above optional technical solutions may be combined according to any choice to form an optional embodiment of the present application, and the embodiments of the present application will not be described in detail.

Fig. 2 is a flowchart of a training method of a network model according to another exemplary embodiment, where the training method of the network model may be applied to an electronic device, and the training method of the network model may include the following steps:

Step 201: a first configuration interface is presented, the first configuration interface including a plurality of reference data enhancements.

In one possible implementation manner, the electronic device may display the first configuration interface when receiving an enhanced manner configuration instruction, where the enhanced manner configuration instruction may be triggered by a user, and the user may trigger a reference operation, where the reference operation may include a clicking operation, a sliding operation, and so on, and the embodiment of the present application is not limited in this regard.

For example, the electronic device may be provided with an enhancement mode configuration option, when the user wants to configure the data enhancement mode, the enhancement mode configuration option may be clicked to trigger an enhancement mode configuration instruction, and accordingly, after the electronic device receives the enhancement mode configuration instruction, the electronic device indicates that the user wants to configure the data enhancement mode, and the first configuration interface is displayed, where the first configuration interface displays a plurality of preset reference data enhancement modes.

That is, in this embodiment, the electronic device may provide a first configuration interface, and display multiple reference data enhancement modes in the first configuration interface, so that a user may configure the data enhancement modes based on the multiple reference data enhancement modes according to actual requirements of different application scenarios, thereby avoiding using fixed data enhancement modes in different application scenarios.

The plurality of reference data enhancement modes may include at least two of rotation, mirroring, scaling, noise addition, blurring, brightness adjustment, and cropping, among others.

Of course, it should be noted that, the various reference data enhancement modes include rotation, mirroring, scaling, noise adding, blurring, brightness adjustment and clipping, and in another embodiment, the various reference data enhancement modes may also include other data enhancement modes, which are not limited in this embodiment of the present application.

Step 202: and receiving a first configuration instruction based on the first configuration interface, wherein the first configuration instruction is triggered after a user configures at least one reference data enhancement mode in the plurality of reference data enhancement modes.

After the electronic device displays the multiple reference data enhancement modes through the first configuration interface, a user can select which one or more reference data enhancement modes are needed to be adopted in the first configuration interface so as to trigger the first configuration instruction.

For example, each of the plurality of reference data enhancements may correspond to an option for selection, and when the user wants to select a certain reference data enhancement, the option corresponding to the certain reference data enhancement may be checked. Further, the first configuration interface may further provide a confirmation selection option, and after the user selects the reference data enhancement mode to be adopted, the confirmation selection option may be clicked to trigger the first configuration instruction.

Step 203: according to the first configuration instruction, target configuration content is acquired, the target configuration content is used for acquiring an enhanced training sample, and the enhanced training sample is obtained by carrying out data enhancement on an original training sample according to a data enhancement mode responded by the target configuration content.

According to the different quantity of the reference data enhancement modes selected by the user, the content carried by the first configuration instruction is different, and according to the different content carried by the first configuration instruction, the specific implementation mode for acquiring the target configuration content is also different. As an example, the implementation of obtaining the target configuration content according to the first configuration instruction may include the following several possible implementations:

The first implementation mode: determining a reference data enhancement mode selected by a user from the plurality of reference data enhancement modes according to the first configuration instruction, displaying a second configuration interface based on the determined reference data enhancement mode, receiving a second configuration instruction in the second configuration interface, wherein the second configuration instruction is triggered after the user performs parameter configuration on the parameters of the determined reference data enhancement mode, and determining target configuration content based on the reference data enhancement mode after the reference configuration operation according to the second configuration instruction.

In such an implementation, the user selects only one reference data enhancement, wherein the second configuration interface may be used for the user to parameter configure the selected reference data enhancement. In some embodiments, the electronic device may display the reference data enhancement mode in the second configuration interface, and display the reference configuration item in an area corresponding to the reference data enhancement mode, for example, as shown in fig. 3.

As an example, the parameters configured for such reference data enhancement may include an enhancement parameter range and a probability indicating a proportion of samples in the training samples to be processed using the corresponding reference data enhancement, e.g., if a probability of 60% for a certain reference data enhancement, 60% of the training samples are processed using such reference data enhancement at a later time.

The user may perform reference configuration on the reference data enhancement mode displayed on the second configuration interface according to actual needs, so as to trigger a second configuration instruction, where the second configuration instruction may include, as an example, the configured parameter value, and further may further include indication information of the selected reference data enhancement mode. For example, if the application scene is that an object with a fixed size needs to be detected, and the selected reference data enhancement mode is a zoom enhancement mode, the user may configure a zoom parameter of the zoom enhancement mode within a smaller range around 1 time, and set the probability to 100%, where the second configuration instruction includes the zoom parameter around 1 time and the probability to 100%, and further includes indication information of the zoom enhancement mode; if the application scenario is that defects with large differences in size need to be detected, the scaling parameters can be configured in a relatively large range (can be configured according to actual requirements) on a reasonable premise, and the probability is set to 80%, and at this time, the second configuration instruction includes a range in which the scaling parameters are large (i.e., values configured according to actual requirements), and the probability is 80%.

It should be noted that, the foregoing description only uses two examples of the parameters corresponding to the reference data enhancement mode as an example, and in another embodiment, the reference data enhancement mode may also include other parameters, which is not limited in the embodiment of the present application.

And the electronic equipment determines target configuration content according to the second configuration instruction, wherein the target configuration content comprises the reference data enhancement mode selected by the user and the enhancement parameter range and probability of the reference data enhancement mode.

The second implementation mode: determining two or more than two reference data enhancement modes selected by a user from the plurality of reference data enhancement modes according to the first configuration instruction, displaying a second configuration interface based on the determined reference data enhancement modes, receiving a third configuration instruction in the second configuration interface, wherein the third configuration instruction is triggered after the user performs parameter configuration on the determined parameters of each reference data enhancement mode, and determining the target configuration content based on the reference data enhancement modes after parameter configuration operation according to the third configuration instruction.

In such an implementation, the user selects two or more reference data enhancement modes, wherein the second configuration interface may be used for the user to configure parameters for all of the selected reference data enhancement modes. In some embodiments, the electronic device may display each selected reference data enhancement mode in the second configuration interface, and display the reference configuration item in a region corresponding to each reference data enhancement mode, for example, as shown in fig. 4.

As an example, the parameters that each reference data enhancement mode needs to be configured may include the enhancement parameter ranges and probabilities described previously. And the user can perform reference configuration on each reference data enhancement mode displayed on the second configuration interface according to actual requirements, so as to trigger a second configuration instruction. As an example, the second configuration instruction may include the configured parameter value, and further, may further include indication information of each selected reference data enhancement mode. For example, if the application scenario is that an object with a fixed size needs to be detected, and the selected reference data enhancement mode includes a zoom enhancement mode, the user may configure a zoom parameter of the zoom enhancement mode within a smaller range around 1 time, and set a probability to 100%, where the second configuration instruction includes the zoom parameter around 1 time and the probability to 100%, and further includes indication information of the zoom enhancement mode; if the application scenario is that defects with large differences in size need to be detected, the scaling parameters can be configured in a relatively large range (can be configured according to actual requirements) on a reasonable premise, and the probability is set to 80%, and at this time, the second configuration instruction includes a range in which the scaling parameters are large (i.e., values configured according to actual requirements), and the probability is 80%.

In addition, when the user selects two or more reference data enhancement modes, the electronic device can also perform parameter configuration on other reference data enhancement modes according to the conditions of brightness, blurring, noise and the like of the acquired image in the application scene.

It should be noted that, the foregoing only describes, as an example, that at least one reference data enhancement mode is selected from a plurality of reference data enhancement modes, and then parameter configuration is performed on the selected at least one reference data enhancement mode. In another embodiment, one or some of the plurality of reference data enhancement modes may be turned off according to actual requirements, so as to obtain the at least one parameter data enhancement mode, and then parameter configuration is performed on the obtained at least one reference data enhancement mode.

The method provided by the embodiment of the present application has been implemented so far, and in another embodiment, in order to enable the network model to be further adapted to an application scenario, the embodiment of the present application may further include the following implementation steps.

Step 204: and carrying out data enhancement processing on the original training sample based on the data enhancement mode responded by the configuration content to obtain an enhanced training sample.

For example, when the at least one reference data enhancement mode in the configuration content includes a scaling and noise adding mode, the electronic device performs scaling and noise adding processing on the original training sample, thereby obtaining an enhanced training sample after expansion. Wherein, the original training sample can be obtained through collection.

Step 205: and training the network model to be trained based on the enhanced training sample.

After the enhanced training sample is obtained, the enhanced training sample can be used for training the network model to be trained by adopting a deep learning algorithm.

Further, in the deep learning training process, the performance of the network model on some types of enhanced training samples in the enhanced training samples may be weaker, and for this case, the electronic device may further obtain a test sample, so as to evaluate the network model according to the test sample, and modify the data enhancement mode according to the evaluation result.

Specifically, the electronic device obtains a test sample, and at this time, the specific implementation of training the network model to be trained based on the enhanced training sample may include the following implementation steps:

2051: and carrying out iterative training on the network model to be trained based on the enhanced training sample.

Referring to fig. 5, the electronic device performs a certain number of iterative training on the network model to be trained based on the enhanced training sample.

2052: And when the iterative training times reach a training time threshold, evaluating the network model obtained by the current training through the test sample.

And after the electronic equipment iteratively trains the network model to be trained for a certain number of times based on the enhanced training sample, performing performance evaluation on the network model obtained by the current training by using the test sample. The test sample can be obtained by sample collection in advance.

The training frequency threshold may be set by a user in a user-defined manner according to actual requirements, or may be set by default by the electronic device, which is not limited in the embodiment of the present application.

2053: And when the evaluation result does not reach the reference performance, adjusting the target configuration content.

The reference performance can be preset according to actual requirements. When the evaluation result does not reach the reference performance, the current trained network model is not in accordance with the actual requirement, and the target configuration content can be adjusted at the moment. In one possible implementation, adjusting the target configuration content may include two implementations:

In a first possible implementation manner, a third configuration interface is displayed, an adjustment instruction for the target configuration content is received based on the third configuration interface, and the target configuration content is adjusted according to the adjustment instruction.

When the electronic device determines that the evaluation result does not reach the reference performance, the third configuration interface may be displayed, so that the user may reconfigure the target configuration content based on the third configuration interface to trigger the adjustment instruction. For example, parameters of a certain reference data enhancement mode can be modified in the target configuration content, the reference data enhancement mode can be deleted or added in the target configuration content, and further, when the reference data enhancement mode is added, the parameter configuration operation of a user on the added reference data enhancement mode can be received.

As an example, the third configuration interface may be the same as the first configuration interface, that is, when it is determined that the evaluation result does not reach the reference performance, the electronic device may re-display the first configuration interface, so that the user may reconfigure the reference data enhancement mode based on the first configuration interface. As another example, the third configuration interface may also be different from the first configuration interface, e.g., the third configuration interface may present the currently configured target configuration content and provide a reconfiguration option that the user may click to trigger an adjustment instruction to readjust the target configuration content.

In a second possible implementation manner, feature information of various features of the test sample and the enhanced training sample is obtained respectively, a first feature information set and a second feature information set are obtained correspondingly, and the target configuration content is adjusted according to the first feature information set and the second feature information set, wherein the various features correspond to a reference data enhancement mode.

The target configuration content may also be automatically adjusted when the electronic device determines that the evaluation result does not reach the reference performance. In an implementation, feature information of various features may be obtained from the test sample to obtain a first feature information set, and feature information of the various features may be obtained from the enhanced training sample to obtain a second feature information set. The various features may include, but are not limited to, color, size, blur, brightness, noise, among others.

And then, the electronic equipment adjusts the target configuration content according to the first characteristic information set and the second characteristic information set. In one possible implementation, the process may include: and for any one of the features, acquiring the feature information of the any one feature from the first feature information set and the second feature information set, and counting a first feature range and a second feature range corresponding to the any one feature according to the acquired feature information, wherein the first feature range is a feature range corresponding to a test sample, and the second feature range is a feature range corresponding to an enhanced training sample. When the first characteristic range is different from the second characteristic range, if the target configuration content does not comprise the reference data enhancement mode corresponding to any one of the characteristics, configuring the reference data enhancement mode corresponding to any one of the characteristics in the target configuration content to obtain adjusted target configuration content; and if the target configuration content comprises the reference data enhancement mode corresponding to any feature, adjusting parameters of the reference data enhancement mode corresponding to any feature according to the adjustment step length threshold.

As an example, according to the obtained feature information, the implementation of counting the first feature range and the second feature range corresponding to any feature may be: and counting the feature information corresponding to any feature by a certain proportion of pixel points in the test sample, taking the minimum value in the counted feature information as the lower limit of the first feature range, and taking the maximum value in the counted feature information as the upper limit of the first feature range, so that the first feature range is obtained. Similarly, the pixel points with the same proportion in the statistical enhancement training sample are used for corresponding feature information of any feature, the minimum value in the statistical feature information is used as the lower limit of a second feature range, and the maximum value in the statistical feature information is used as the upper limit of the second feature range, so that the second feature range is obtained.

For any feature, when the feature range corresponding to the feature information in the first feature information set is different from the feature range corresponding to the feature information in the second feature information set, the difference between the test sample and the enhanced training sample for the feature is larger, and at this time, the reference data enhancement mode corresponding to any feature can be reconfigured to adjust the target configuration content.

For example, for the feature of brightness, when the feature range corresponding to the test sample is different from the feature range corresponding to the enhanced training sample, for example, the feature range corresponding to the test sample is [128,255], and the feature range corresponding to the enhanced training sample is [128,200], which indicates that the brightness difference between the enhanced training sample and the test sample is large, at this time, the reference data enhancement mode of brightness adjustment corresponding to brightness can be reconfigured to increase the specific gravity of the training sample in the enhanced training sample.

As an example, whether the configured target configuration content includes the reference data enhancement mode corresponding to the any feature may be detected, and if the configured target configuration content does not include the reference data enhancement mode corresponding to the any feature, the reference data enhancement mode corresponding to the feature may be added to the target configuration content, and the corresponding parameter may be automatically configured. If the configured target configuration content comprises the reference data enhancement mode corresponding to any feature, the current configuration parameters are unreasonable, so that the corresponding parameters can be adjusted according to the step length adjustment threshold.

The step length adjustment threshold may be set by a user in a user-defined manner according to actual needs, or may be set by default by the electronic device, which is not limited in the embodiment of the present application. And, the adjustment step thresholds corresponding to different parameters may be the same or different.

For example, after a certain number of iterative training, the performance of the current network model can be evaluated by using the test sample, and when the network model is determined to have weaker processing capability on the test sample with larger noise in the application scene, the training sample needs to be re-expanded by modifying the reference data enhancement mode, so that the specific gravity of the noise training sample in the obtained enhanced training sample is increased, and the robustness of the network model on the noise sample is improved.

It should be noted that, for the test sample and the enhanced training sample, when there are different feature ranges corresponding to multiple features, the configuration of the reference data enhancement mode corresponding to one feature in the multiple features may be modified each time, and then the reference data enhancement modes corresponding to other features may be adjusted after the next training.

2054: And carrying out data enhancement processing on the original training sample based on the adjusted target configuration content, re-determining the obtained training sample as an enhanced training sample, and returning to the operation of carrying out iterative training on the network model to be trained based on the enhanced training sample until the evaluation result reaches the reference performance or the total number of iterative training reaches the reference training number.

That is, after the electronic device adjusts the target configuration content, the data enhancement processing may be performed on the original training sample again according to the adjusted target configuration content, so as to re-expand the original training sample to obtain an enhanced training sample. The electronic device then continues to iteratively train the network model to be trained using the retrieved enhanced training samples, i.e., returns to step 2051 described above. If the evaluation result of evaluating the network model obtained by training by using the test sample does not reach the reference performance after the network model to be trained is trained for a certain number of times based on the redetermined enhanced training sample, continuing to readjust the target configuration content, performing data enhancement processing on the original training sample based on the adjusted target configuration content, redetermining the obtained training sample as the enhanced training sample, and returning to step 2051. And repeating the training process until the evaluation result reaches the reference performance, ending the training, and outputting the currently obtained network model as a final network model.

Or when the total number of iterative training reaches the reference training number, the performance of the network model still does not reach the reference performance, and the electronic equipment terminates training, and the network model with the best trained performance can be used as a final network model. The reference training times can be preset according to actual requirements.

It is worth mentioning that the training samples are re-expanded by online modification data enhancement mode to increase the specific gravity of certain types of training samples until the network model can reach the reference performance, so that the deep learning model with strong adaptability to the application scene is finally obtained, and the model performance is improved. In addition, the method provided by the application is especially suitable for the scene operated by the user instead of the developer in the training process, so that the deep learning product can adapt to different application scenes, and the cost of secondary development of the product is reduced.

Fig. 6 is a schematic diagram of a training apparatus of a network model, which may be implemented in software, hardware, or a combination of both, according to an exemplary embodiment. The training apparatus of the network model may include:

the display module 610 is configured to display a first configuration interface, where the first configuration interface includes multiple reference data enhancement modes;

A receiving module 620, configured to receive a first configuration instruction based on the first configuration interface, where the first configuration instruction is triggered after a user configures at least one reference data enhancement mode of the multiple reference data enhancement modes;

The obtaining module 630 is configured to obtain, according to the first configuration instruction, target configuration content, where the target configuration content is used to obtain an enhanced training sample, and the enhanced training sample is obtained by performing data enhancement on a training sample according to a data enhancement mode responded by the target configuration content.

In one possible implementation of the present application, the obtaining module 630 is configured to:

receiving a second configuration instruction in the second configuration interface, wherein the second configuration instruction is triggered after a user performs parameter configuration on the determined parameters of the reference data enhancement mode;

In one possible implementation manner of the present application, referring to fig. 7, the apparatus further includes:

The processing module 640 is configured to perform data enhancement processing on the original training sample in a data enhancement manner responded by the target configuration content, so as to obtain the enhanced training sample;

And the training module 650 is configured to train the network model to be trained based on the enhanced training samples.

In one possible implementation manner of the present application, referring to fig. 8, the apparatus further includes:

A sample acquisition module 660 for acquiring a test sample;

The training module 650 is configured to:

When the iterative training times reach a training time threshold, evaluating a network model obtained by current training through the test sample;

In one possible implementation of the present application, the training module 650 is configured to:

Or alternatively

According to the obtained characteristic information, counting a first characteristic range and a second characteristic range corresponding to any characteristic, wherein the first characteristic range is a characteristic range corresponding to the test sample, and the second characteristic range is a characteristic range corresponding to the enhanced training sample;

When the first feature range is different from the second feature range, if the target configuration content does not include the reference data enhancement mode corresponding to any feature, configuring the reference data enhancement mode corresponding to any feature in the target configuration content to obtain an adjusted target configuration content;

And if the target configuration content comprises the reference data enhancement mode corresponding to any feature, adjusting parameters of the reference data enhancement mode corresponding to any feature according to an adjustment step threshold.

In one possible implementation of the present application, the plurality of reference data enhancement modes includes at least two of rotation, mirroring, scaling, noise adding, blurring, brightness adjustment and cropping.

It should be noted that: in the training device for a network model provided in the above embodiment, when implementing the training method for a network model, only the division of the above functional modules is used for illustration, in practical application, the above functional allocation may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the training device of the network model provided in the above embodiment and the training method embodiment of the network model belong to the same concept, and the specific implementation process is detailed in the method embodiment, which is not repeated here.

Fig. 9 shows a block diagram of an electronic device 900 provided by an exemplary embodiment of the application. The electronic device 900 may be: tablet, notebook or desktop. Electronic device 900 may also be referred to by other names of user devices, portable terminals, laptop terminals, desktop terminals, and the like.

Generally, the electronic device 900 includes: a processor 901 and a memory 902.

Processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 901 may be implemented in at least one hardware form of DSP (DIGITAL SIGNAL Processing), FPGA (Field-Programmable gate array), PLA (Programmable Logic Array ). Processor 901 may also include a main processor, which is a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 901 may integrate a GPU (Graphics Processing Unit, image processor) for rendering and drawing of content required to be displayed by the display screen. In some embodiments, the processor 901 may also include an AI (ARTIFICIAL INTELLIGENCE ) processor for processing computing operations related to machine learning.

The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 902 is used to store at least one instruction for execution by processor 901 to implement a method of training a network model provided by an embodiment of a method in the present application.

In some embodiments, the electronic device 900 may further optionally include: a peripheral interface 903, and at least one peripheral. The processor 901, memory 902, and peripheral interface 903 may be connected by a bus or signal line. The individual peripheral devices may be connected to the peripheral device interface 903 via buses, signal lines, or circuit boards. Specifically, the peripheral device includes: at least one of radio frequency circuitry 904, a display 905, a camera assembly 906, audio circuitry 907, a positioning assembly 908, and a power source 909.

The peripheral interface 903 may be used to connect at least one peripheral device associated with an I/O (Input/Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, memory 902, and peripheral interface 903 are integrated on the same chip or circuit board; in some other embodiments, either or both of the processor 901, the memory 902, and the peripheral interface 903 may be implemented on separate chips or circuit boards, which is not limited in this embodiment.

The Radio Frequency circuit 904 is configured to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 904 communicates with a communication network and other communication devices via electromagnetic signals. The radio frequency circuit 904 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 904 includes: antenna systems, RF transceivers, one or more amplifiers, tuners, oscillators, digital signal processors, codec chipsets, subscriber identity module cards, and so forth. The radio frequency circuitry 904 may communicate with other devices via at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the world wide web, metropolitan area networks, intranets, generation mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or WiFi (WIRELESS FIDELITY ) networks. In some embodiments, the radio frequency circuit 904 may further include NFC (NEAR FIELD Communication) related circuits, which is not limited by the present application.

The display 905 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display 905 is a touch display, the display 905 also has the ability to capture touch signals at or above the surface of the display 905. The touch signal may be input as a control signal to the processor 901 for processing. At this time, the display 905 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 905 may be one, providing a front panel of the electronic device 900; in other embodiments, the display 905 may be at least two, respectively disposed on different surfaces of the electronic device 900 or in a folded design; in still other embodiments, the display 905 may be a flexible display disposed on a curved surface or a folded surface of the electronic device 900. Even more, the display 905 may be arranged in an irregular pattern other than rectangular, i.e., a shaped screen. The display 905 may be made of LCD (Liquid CRYSTAL DISPLAY), OLED (Organic Light-Emitting Diode) or other materials.

The camera assembly 906 is used to capture images or video. Optionally, the camera assembly 906 includes a front camera and a rear camera. In general, a front camera is disposed on a front panel of an electronic device, and a rear camera is disposed on a rear surface of the electronic device. In some embodiments, the at least two rear cameras are any one of a main camera, a depth camera, a wide-angle camera and a tele camera, so as to realize that the main camera and the depth camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize a panoramic shooting and Virtual Reality (VR) shooting function or other fusion shooting functions. In some embodiments, camera assembly 906 may also include a flash. The flash lamp can be a single-color temperature flash lamp or a double-color temperature flash lamp. The dual-color temperature flash lamp refers to a combination of a warm light flash lamp and a cold light flash lamp, and can be used for light compensation under different color temperatures.

The audio circuit 907 may include a microphone and a speaker. The microphone is used for collecting sound waves of users and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 901 for processing, or inputting the electric signals to the radio frequency circuit 904 for voice communication. For purposes of stereo acquisition or noise reduction, the microphone may be multiple and separately disposed at different locations of the electronic device 900. The microphone may also be an array microphone or an omni-directional pickup microphone. The speaker is used to convert electrical signals from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker may be a conventional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only the electric signal can be converted into a sound wave audible to humans, but also the electric signal can be converted into a sound wave inaudible to humans for ranging and other purposes. In some embodiments, the audio circuit 907 may also include a headphone jack.

The location component 908 is used to locate the current geographic location of the electronic device 900 to enable navigation or LBS (Location Based Service, location-based services). The positioning component 908 may be a positioning component based on the United states GPS (Global Positioning System ), the Beidou system of China, or the Galileo system of Russia.

The power supply 909 is used to power the various components in the electronic device 900. The power supply 909 may be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power source 909 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, the electronic device 900 also includes one or more sensors 910. The one or more sensors 910 include, but are not limited to: acceleration sensor 911, gyroscope sensor 912, pressure sensor 913, fingerprint sensor 914, optical sensor 915, and proximity sensor 916.

The acceleration sensor 911 can detect the magnitudes of accelerations on three coordinate axes of the coordinate system established with the electronic device 900. For example, the acceleration sensor 911 may be used to detect components of gravitational acceleration in three coordinate axes. The processor 901 may control the display 905 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal acquired by the acceleration sensor 911. The acceleration sensor 911 may also be used for the acquisition of motion data of a game or a user.

The gyro sensor 912 may detect a body direction and a rotation angle of the electronic device 900, and the gyro sensor 912 may collect a 3D motion of the user on the electronic device 900 in cooperation with the acceleration sensor 911. The processor 901 may implement the following functions according to the data collected by the gyro sensor 912: motion sensing (e.g., changing UI according to a tilting operation by a user), image stabilization at shooting, game control, and inertial navigation.

The pressure sensor 913 may be disposed on a side frame of the electronic device 900 and/or on an underside of the display 905. When the pressure sensor 913 is disposed on the side frame of the electronic device 900, a holding signal of the electronic device 900 by the user may be detected, and the processor 901 performs left-right hand recognition or quick operation according to the holding signal collected by the pressure sensor 913. When the pressure sensor 913 is provided at the lower layer of the display 905, the processor 901 performs control of the operability control on the UI interface according to the pressure operation of the user on the display 905. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

The fingerprint sensor 914 is used for collecting the fingerprint of the user, and the processor 901 identifies the identity of the user according to the fingerprint collected by the fingerprint sensor 914 or the fingerprint sensor 914 identifies the identity of the user according to the collected fingerprint. Upon recognizing that the user's identity is a trusted identity, the processor 901 authorizes the user to perform relevant sensitive operations including unlocking the screen, viewing encrypted information, downloading software, paying for and changing settings, etc. The fingerprint sensor 914 may be provided on the front, back, or side of the electronic device 900. When a physical key or vendor Logo is provided on the electronic device 900, the fingerprint sensor 914 may be integrated with the physical key or vendor Logo.

The optical sensor 915 is used to collect the intensity of ambient light. In one embodiment, the processor 901 may control the display brightness of the display panel 905 based on the intensity of ambient light collected by the optical sensor 915. Specifically, when the ambient light intensity is high, the display luminance of the display screen 905 is turned up; when the ambient light intensity is low, the display luminance of the display panel 905 is turned down. In another embodiment, the processor 901 may also dynamically adjust the shooting parameters of the camera assembly 906 based on the ambient light intensity collected by the optical sensor 915.

A proximity sensor 916, also referred to as a distance sensor, is typically provided on the front panel of the electronic device 900. The proximity sensor 916 is used to capture the distance between the user and the front of the electronic device 900. In one embodiment, when the proximity sensor 916 detects that the distance between the user and the front of the electronic device 900 gradually decreases, the processor 901 controls the display 905 to switch from the bright screen state to the off screen state; when the proximity sensor 916 detects that the distance between the user and the front surface of the electronic device 900 gradually increases, the processor 901 controls the display 905 to switch from the off-screen state to the on-screen state.

Those skilled in the art will appreciate that the structure shown in fig. 9 is not limiting of the electronic device 900 and may include more or fewer components than shown, or may combine certain components, or may employ a different arrangement of components.

The embodiments of the present application also provide a non-transitory computer readable storage medium, which when executed by a processor of an electronic device, enables the electronic device to perform the training method of the network model provided in the foregoing embodiments.

The embodiments of the present application also provide a computer program product containing instructions which, when run on a computer, cause the computer to perform the method of training a network model provided in the above embodiments.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The foregoing description of the preferred embodiments of the application is not intended to limit the application to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the application are intended to be included within the scope of the application.

Claims

1. A method of training a network model, the method for target detection, identification and classification, the method comprising:

Displaying a first configuration interface, wherein the first configuration interface comprises a plurality of reference data enhancement modes, and the plurality of reference data enhancement modes comprise at least two of rotation, mirroring, scaling, noise adding, blurring, brightness adjustment and clipping;

Acquiring target configuration content according to the first configuration instruction;

Performing data enhancement processing on the original training sample based on a data enhancement mode responded by the target configuration content to obtain an enhanced training sample;

When the iterative training times reach a training time threshold, evaluating a network model obtained by current training through a test sample;

when the evaluation result does not reach the reference performance, respectively acquiring the characteristic information of various characteristics of the test sample and the enhanced training sample, correspondingly acquiring a first characteristic information set and a second characteristic information set, and respectively acquiring the characteristic information of any one of the various characteristics from the first characteristic information set and the second characteristic information set for any one of the various characteristics;

When the first feature range is different from the second feature range, if the target configuration content does not include the reference data enhancement mode corresponding to any feature, configuring the reference data enhancement mode corresponding to any feature in the target configuration content to obtain an adjusted target configuration content; if the target configuration content comprises a reference data enhancement mode corresponding to any feature, adjusting parameters of the reference data enhancement mode corresponding to any feature according to an adjustment step threshold, wherein various features correspond to one reference data enhancement mode;

2. The method of claim 1, wherein the obtaining the target configuration content according to the first configuration instruction comprises:

Determining at least one reference data enhancement mode selected by a user from the plurality of reference data enhancement modes according to the first configuration instruction;

3. The method of claim 1, wherein the obtaining the target configuration content according to the first configuration instruction comprises:

determining two or more reference data enhancement modes selected by a user from the multiple reference data enhancement modes according to the first configuration instruction;

4. The method of claim 1, wherein prior to iteratively training the network model to be trained based on the enhanced training samples, further comprising:

and obtaining the test sample.

5. A training apparatus for network models, the apparatus for object detection, identification and classification comprising:

the display module is used for displaying a first configuration interface, wherein the first configuration interface comprises a plurality of reference data enhancement modes, and the plurality of reference data enhancement modes comprise at least two of rotation, mirroring, scaling, noise adding, blurring, brightness adjustment and clipping;

The acquisition module is used for acquiring target configuration content according to the first configuration instruction;

The processing module is used for carrying out data enhancement processing on the original training sample in a data enhancement mode responded by the target configuration content to obtain an enhanced training sample;

The training module is used for carrying out iterative training on the network model to be trained based on the enhanced training sample; when the iterative training times reach a training time threshold, evaluating a network model obtained by current training through a test sample; when the evaluation result does not reach the reference performance, respectively acquiring the characteristic information of various characteristics of the test sample and the enhanced training sample, correspondingly acquiring a first characteristic information set and a second characteristic information set, and respectively acquiring the characteristic information of any one of the various characteristics from the first characteristic information set and the second characteristic information set for any one of the various characteristics; according to the obtained characteristic information, counting a first characteristic range and a second characteristic range corresponding to any one of the characteristics, wherein the first characteristic range is a characteristic range corresponding to the test sample, and the second characteristic range is a characteristic range corresponding to the enhanced training sample; when the first feature range is different from the second feature range, if the target configuration content does not include the reference data enhancement mode corresponding to any feature, configuring the reference data enhancement mode corresponding to any feature in the target configuration content to obtain an adjusted target configuration content; if the target configuration content comprises a reference data enhancement mode corresponding to any feature, adjusting parameters of the reference data enhancement mode corresponding to any feature according to an adjustment step threshold, wherein various features correspond to one reference data enhancement mode; and carrying out data enhancement processing on the original training sample based on a data enhancement mode responded by the adjusted target configuration content, re-determining the obtained training sample as an enhanced training sample, and returning to the operation of carrying out iterative training on the network model to be trained based on the enhanced training sample until the evaluation result reaches the reference performance or the total number of iterative training reaches the reference training number.

6. An electronic device, comprising:

A processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to implement the steps of any of the methods of claims 1-4.

7. A computer readable storage medium having instructions stored thereon, which when executed by a processor, implement the steps of the method of any of claims 1-4.