HK1247423B - Image-based vehicle damage estimation method and device and electronic equipment - Google Patents

Description

Image-based vehicle damage assessment method, device, and electronic equipment

Technical Field

The present application belongs to the field of computer image data processing technology, and in particular relates to an image-based vehicle damage assessment method, device and electronic equipment.

Background Art

After a traffic accident, it's often necessary to wait for the insurance company's claims adjuster to arrive and process the claim, taking photos and other procedures to obtain the required evidence. With the recent increase in motor vehicle ownership, the number of traffic accidents has remained high annually. However, vehicle claims assessment often relies on on-site manual processing by professional insurance personnel, which is costly, time-consuming, and inefficient.

Currently, some approaches in the industry use traffic accident scene images for automatic analysis to categorize pre-set vehicle damage locations. For example, application publication number "CN105678622A," entitled "Auto Insurance Claim Photo Analysis Method and System," discloses an algorithm that uses a conventional convolutional neural network (CNN) to analyze claim photos uploaded by mobile terminals, identify damage location categories, and generate reminder information based on the analysis results. However, this approach simply identifies vehicle damage location categories, such as front, side, and rear, without identifying specific damage types. The identified damage location reminder information is primarily used by insurance company staff for manual comparison with manual damage assessment results, serving as reference information to assist insurance company staff in damage assessment calculations. Furthermore, this algorithm only utilizes a general CNN object recognition algorithm, leaving the final vehicle damage assessment result to rely on manual verification, which is labor-intensive and time-consuming. Furthermore, different insurance companies have inconsistent standards for damage assessment, and coupled with human subjective factors, vehicle damage assessment results vary widely, resulting in low reliability.

Summary of the Invention

The purpose of this application is to provide an image-based vehicle damage assessment method, device and electronic equipment that can quickly, accurately and reliably detect specific information such as the location and degree of damage to vehicle components, and the damage assessment results are more accurate and reliable, providing users with repair plan information, quickly and efficiently performing vehicle damage assessment processing, and greatly improving the user service experience.

The present application provides an image-based vehicle damage assessment method, device, and electronic device that are implemented as follows:

A vehicle damage assessment method based on an image, the method comprising:

Obtaining images to be processed for vehicle damage assessment;

Detecting the image to be processed using the constructed component recognition model, identifying the vehicle component in the image to be processed, and confirming the component region of the vehicle component in the image to be processed;

Detecting the image to be processed using the constructed damage recognition model to identify the damage location and damage type in the image to be processed;

Determining a damaged component in the image to be processed according to the damaged portion and component region, and determining a damaged portion and a damage type of the damaged component;

A maintenance plan is generated based on the information including the damaged component, damaged location, and damage type.

A vehicle damage assessment device based on an image, comprising:

An image acquisition module is used to acquire images to be processed for vehicle damage assessment;

a first recognition module, configured to detect the image to be processed using the constructed component recognition model, identify the vehicle component in the image to be processed, and confirm the component region of the vehicle component in the image to be processed;

A second recognition module is used to detect the image to be processed using the constructed damage recognition model, and identify the damage location and damage type in the image to be processed;

a damage calculation module, configured to determine a damaged component in the image to be processed, and determine a damage location and damage type of the damaged component based on the processing results of the first recognition module 102 and the second recognition module 103;

The damage assessment processing module is used to generate a maintenance plan based on information including the damaged component, damaged location, and damage type.

An image-based vehicle damage assessment device includes a processor and a memory for storing processor-executable instructions, wherein when the processor executes the instructions, the following are achieved:

Obtaining images to be processed for vehicle damage assessment;

Detecting the image to be processed using the constructed component recognition algorithm, identifying the vehicle component in the image to be processed, and confirming the component area of the vehicle component in the image to be processed;

Detecting the image to be processed using the constructed damage recognition algorithm to identify the damage location and damage type in the image to be processed;

Determining a damaged component in the image to be processed according to the damaged portion and component region, and determining a damaged portion and a damage type of the damaged component;

A maintenance plan is generated based on the information including the damaged component, damaged location, and damage type.

A computer-readable storage medium having computer instructions stored thereon, which, when executed, implement the following steps:

Obtaining images to be processed for vehicle damage assessment;

Detecting the image to be processed using the constructed component recognition algorithm, identifying the vehicle component in the image to be processed, and confirming the component area of the vehicle component in the image to be processed;

Detecting the image to be processed using the constructed damage recognition algorithm to identify the damage location and damage type in the image to be processed;

Determining a damaged component in the image to be processed according to the damaged portion and component region, and determining a damaged portion and a damage type of the damaged component;

A maintenance plan is generated based on the information including the damaged component, damaged location, and damage type.

An electronic device includes a processor and a memory for storing processor-executable instructions, wherein when the processor executes the instructions, the following is achieved:

Obtaining images to be processed for vehicle damage assessment;

Detecting the image to be processed using the constructed component recognition algorithm, identifying the vehicle component in the image to be processed, and confirming the component area of the vehicle component in the image to be processed;

Detecting the image to be processed using the constructed damage recognition algorithm to identify the damage location and damage type in the image to be processed;

Determining a damaged component in the image to be processed according to the damaged portion and component region, and determining a damaged portion and a damage type of the damaged component;

A maintenance plan is generated based on the information including the damaged component, damaged location, and damage type.

The present application provides an image-based vehicle damage assessment method, device and electronic device that can identify damaged parts contained in the image to be processed, and then detect the damaged parts of the damaged parts and the damage type corresponding to each damaged part based on the constructed damage recognition model, thereby obtaining accurate, comprehensive and reliable vehicle damage assessment information for the vehicle parts. Furthermore, the implementation scheme of the present application generates a vehicle repair plan based on these damaged parts and the information on the damaged parts, the damage type and the repair strategy, providing insurance personnel and car owners with more accurate, reliable and practical damage assessment information. The implementation scheme of the present application can identify one or more damaged parts in one or more images, and one or more damaged parts and the degree of damage in the damaged parts, etc., quickly obtain more comprehensive and accurate damage assessment information, and then automatically generate a repair plan, which can meet the needs of insurance companies or car owners for fast, comprehensive, accurate and reliable vehicle damage assessment processing, improve the accuracy and reliability of vehicle damage assessment processing results, and improve user service experience.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

FIG1 is a schematic diagram of a method flow of an embodiment of an image-based vehicle damage assessment method described in the present application;

FIG2 is a schematic diagram of the network architecture of a damage identification model according to an embodiment of the present application;

FIG3 is a schematic diagram of a network architecture of a component identification model according to an embodiment of the present application;

FIG4 is a schematic diagram of a method flow of another embodiment of an image-based vehicle damage assessment method of the present application;

FIG5 is a schematic diagram of an implementation process of determining a damaged component and the damaged location and damage type of the damaged component according to the present application;

FIG6 is a schematic diagram of the module structure of an embodiment of an image-based vehicle damage assessment device provided by the present application;

FIG7 is a schematic structural diagram of an embodiment of an electronic device provided by the present application;

FIG8 is a schematic diagram of a processing scenario for vehicle damage assessment using the implementation scheme of the present application.

DETAILED DESCRIPTION

In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

FIG1 is a method flow chart of an embodiment of an image-based vehicle damage assessment method described in the present application. Although the present application provides method operation steps or device structures as shown in the following embodiments or drawings, the method or device may include more or fewer operation steps or module units after partial merger based on routine or no creative labor. In the steps or structures where there is no necessary causal relationship logically, the execution order of these steps or the module structure of the device is not limited to the execution order or module structure shown in the embodiments or drawings of the present application. When the method or module structure is applied to an actual device, server or terminal product, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiments or drawings (for example, a parallel processor or multi-threaded processing environment, or even a distributed processing, server cluster implementation environment).

In existing actual traffic accident handling, such as scratch accidents, it is often necessary to wait for the insurance company's claims adjuster to arrive at the scene to take photos before leaving the scene, which often leads to traffic jams, wastes a lot of time, and takes a long time to obtain information on the damage assessment results. However, when a traffic accident occurs, the owner of the vehicle involved often wants to know the loss or claim status of his or the other party's vehicle. At this time, you can take pictures of the traffic accident scene yourself. On the one hand, it can be used as on-site evidence. On the other hand, you can use the pictures to estimate the loss and claim status of the vehicle through the terminal APP, so as to meet the needs of the owner of the vehicle involved in the accident to obtain vehicle damage quickly, comprehensively, accurately and reliably.

For the sake of clarity, the following embodiment will be described using a specific application scenario in which a car owner uses a mobile terminal APP (application) to request a vehicle damage assessment service. In this embodiment, the car owner can use a mobile terminal (such as a mobile phone) to take photos of the damaged area of the vehicle and the entire vehicle at the scene of the traffic accident. In some cases, the vehicle ID, user ID, etc. can also be photographed, and then the photos (images) taken can be uploaded through the terminal application. After the cloud server obtains the image to be processed for vehicle damage assessment, it can first identify the damaged parts and one or more damaged parts of these damaged parts and the corresponding damage types. Then, a rule engine can be designed to call different price libraries based on maintenance strategy information such as vehicle model, location, and repair shop, and ultimately generate at least one maintenance plan. This maintenance plan can be returned to the car owner, who can then quickly obtain the vehicle damage assessment results. Of course, if the user is an insurance company operator, the results of the maintenance plan can be returned to the insurance company or directly displayed. However, those skilled in the art will appreciate that the essence of this solution can be applied to other implementation scenarios of vehicle damage assessment, such as automatic vehicle damage assessment by insurance companies or repair shops, or self-service vehicle damage assessment services provided by 4S stores or other servers.

A specific embodiment is shown in FIG1 . In one embodiment of an image-based vehicle damage assessment method provided by the present application, the method may include:

S1: Obtain the image to be processed for vehicle damage assessment.

The server can obtain the vehicle's image to be processed from the client or a third-party server (such as an insurance company's server). The image to be processed typically includes user-captured images of the vehicle's location. It can also include user-uploaded images of the vehicle's ID, user identification, or surrounding environment (such as traffic lights or landmarks). The image to be processed in this embodiment generally refers to various graphics and images, typically visually appealing, and can include images on paper, film or photographs, or on televisions, projectors, or computer screens.

In an optional embodiment, it is also possible to determine whether the image quality of the image to be processed meets the set processing requirements. If the image quality is poor, such as a blurry photo that cannot be recognized, the component image can be discarded, and feedback is provided to the mobile terminal app to remind the user to pay attention to focus when taking the image, lighting, and other factors that affect clarity. Image quality can be determined by using methods such as blur thresholds and information entropy values.

S2: Detecting the image to be processed using the constructed component recognition model, identifying the vehicle component in the image to be processed, and confirming the component region of the vehicle component in the image to be processed.

In this embodiment, after the cloud server acquires the image to be processed, it can use a pre-built component recognition model to detect the image to be processed and locate the vehicle components contained in the image to be processed. If one or more vehicle components are detected in the image to be processed, information about the location area (hereinafter referred to as the component area) of these vehicle components in the image to be processed is simultaneously calculated and confirmed. The vehicle components described in this embodiment generally refer to components on the vehicle, such as the front bumper, left front door, rear taillight, etc.

In this embodiment, a designed machine learning algorithm can be used in advance to construct a component recognition model for identifying vehicle components in an image. After being trained with sample images, the component recognition model can identify which vehicle components are contained in the component image. In this embodiment, the component recognition model can be constructed and generated using a network model of a deep neural network or a variant of the network model after training with sample images. In another embodiment of the method provided in the present application, the component recognition model can be constructed and generated based on a convolutional neural network (CNN) and a region proposal network (RPN), combined with input model-trained damaged sample images, fully connected layers, and the like. Therefore, in another embodiment of the method described in the present application, the component recognition model includes:

S201: A network model based on convolutional layers and region proposal layers is trained with sample data to build a generated deep neural network.

A convolutional neural network generally refers to a neural network with a convolutional layer (CNN) as the main structure and combined with other components such as activation layers, and is mainly used for image recognition. The deep neural network described in this embodiment may include convolutional layers and other important layers (such as damaged sample images for input model training, several normalization layers, activation layers, etc.), and is jointly constructed and generated in combination with a regional network. A convolutional neural network usually combines two-dimensional discrete convolution operations in image processing with artificial neural networks. This convolution operation can be used to automatically extract features. The region proposal network (RPN) can take features extracted from an image (of any size) as input (two-dimensional features extracted using a convolutional neural network) and output a set of rectangular target proposal boxes, each box having an object score. To avoid confusion, the convolutional neural network (CNN) used in this embodiment can be referred to as a convolutional layer (CNN) and the region proposal network (RPN) can be referred to as a region proposal layer (RPN). In other embodiments of the present application, the component recognition model may also include a deep convolutional neural network constructed and generated after training with sample data based on an improved variant network model of the convolutional neural network or an improved variant network model of the region proposal network.

The models and algorithms used in the above embodiments can be selected from the same type of models or algorithms. Specifically, for example, in the component recognition model, a variety of models and variants based on convolutional neural networks and region proposal networks can be used, such as Faster R-CNN, YOLO, Mask-FCN, etc. The convolutional neural network (CNN) can use any CNN model, such as ResNet, Inception, VGG, etc. and their variants. Usually, the convolutional network (CNN) part of the neural network can use a mature network structure that has achieved good results in object recognition, such as Inception, ResNet, etc., such as the ResNet network, which inputs an image and outputs multiple component regions, and corresponding component classifications and confidence levels (the confidence level here is a parameter indicating the degree of authenticity of the identified vehicle components). Faster R-CNN, YOLO, Mask-FCN, etc. are all deep neural networks containing convolutional layers that can be used in this embodiment. The deep neural network used in this embodiment, combined with the region proposal layer and the CNN layer, can detect the vehicle components in the image to be processed and confirm the component regions of the vehicle components in the image to be processed.

It should be noted that in one embodiment of the present application, a separate algorithm server can be used to implement a component recognition model to detect the image to be processed and identify the vehicle components within it. For example, a service server can be provided to obtain user-uploaded images to be processed and output repair plans. A separate algorithm server can also be provided to store the constructed component recognition model and detect and identify the images to be processed from the service server to determine the vehicle components within them. Of course, the aforementioned processing can also be performed by the same server.

S3: Detect the image to be processed using the constructed damage recognition model to identify the damage location and damage type in the image to be processed.

After the cloud server obtains the image to be processed, it can use the pre-built damage recognition model to detect the component image and identify the damaged part and damage type in the image to be processed. The damaged part described in this embodiment generally refers to a damaged part on the vehicle. A damaged vehicle component may contain multiple damaged parts, and each damaged part corresponds to a damage type (such as severe scratches, mild deformation, etc.). This embodiment can detect the location area of the damaged part in the image to be processed (which can be called a damaged area here, and can be understood as a damaged part corresponding to a specific damaged area image area data, or the damaged area expresses the entity data information of the damaged part), and can detect the damaged area to identify the damage type. The damage types described in this embodiment may include mild scratches, severe scratches, mild deformation, moderate deformation, severe deformation, breakage, and need for disassembly inspection.

In this embodiment, a designed machine learning algorithm can be used in advance to construct a damage recognition model for identifying the damage sites and damage types contained in the image. After sample training, the damage recognition model can identify one or more damage sites and the corresponding damage types in the image to be processed. In this embodiment, the damage recognition model can be constructed and generated using a network model of a deep neural network or a variant thereof after sample training. In another embodiment of the method provided in the present application, the damage recognition model can be constructed based on a convolutional neural network (CNN) and a region proposal network (RPN), combined with an input model-trained damage sample image, a fully connected layer, and the like. Therefore, in another embodiment of the method described in the present application, the damage recognition model includes:

301: A network model based on convolutional layers and region proposal layers, which is trained with sample data to construct a generated deep neural network.

A convolutional neural network generally refers to a neural network with a convolutional layer (CNN) as its main structure and combined with other components such as activation layers, and is mainly used for image recognition. The deep neural network described in this embodiment may include convolutional layers and other important layers (such as the damage sample image for input model training, several normalization layers, activation layers, etc.), and is jointly constructed and generated in combination with a region proposal network (RPN). A convolutional neural network generally combines two-dimensional discrete convolution operations in image processing with artificial neural networks. This convolution operation can be used to automatically extract features. The region proposal network (RPN) can take features extracted from an image (of any size) as input (two-dimensional features extracted using a convolutional neural network) and output a set of rectangular target proposal boxes, each box having an object score. As described above, to avoid confusion, in this embodiment, the convolutional neural network (CNN) used can be referred to as a convolutional layer (CNN) and the region proposal network (RPN) can be referred to as a region proposal layer (RPN). In other embodiments of the present application, the damage recognition model may also include a deep convolutional neural network constructed and generated after training with sample data based on an improved convolutional neural network or a variant network model of an improved region proposal network.

The above-mentioned implementation method can identify multiple damaged parts on a single damaged sample image during model training. Specifically, during sample training, the input is a picture, and the output is multiple picture areas and corresponding damage classifications. The parameters of the selected neural network can be obtained by using the labeled data for mini-batch gradient descent training. For example, when mini-batch = 32, 32 training pictures are used as input for training at the same time. The labeled data marks the areas and corresponding types of pictures, and can be obtained by manually labeling real vehicle damage pictures. The input of this neural network is a picture, and the output area is related to the number of damaged parts contained in the picture. Specifically, for example, if there is one damaged part, one picture area is output; if there are k damaged parts, k picture areas can be output; if there is no damaged part, 0 picture areas are output.

The models and algorithms used in the above embodiments can be selected from the same type of models or algorithms. Specifically, for example, in the component recognition model, various models and variants based on convolutional neural networks and region proposal networks can be used, such as Faster R-CNN, YOLO, Mask-FCN, etc. The convolutional neural network (CNN) can be any CNN model, such as ResNet, Inception, VGG, etc. and their variants. Generally, the convolutional network (CNN) portion of the neural network can use a mature network structure that has achieved good results in object recognition, such as Inception, ResNet, etc. For example, the ResNet network takes an image as input and outputs multiple image regions containing damage sites, along with corresponding damage classifications (damage classifications are used to determine damage types) and confidence levels (here, confidence levels are parameters indicating the degree of authenticity of damage types). Faster R-CNN, YOLO, Mask-FCN, etc. are all deep neural networks containing convolutional layers that can be used in this embodiment. The deep neural network used in this embodiment, combined with the region proposal layer and CNN layer, can detect damage sites, damage types, and the location of the damage sites in the component image.

It should be noted that in one embodiment of the present application, a separate algorithm server can be used to detect the image to be processed and identify the damage location and damage type in the image to be processed. For example, a business server is set up to obtain the image to be processed uploaded by the user and output the repair plan. At the same time, an algorithm server can also be set up to store the constructed damage recognition model to detect and identify the image to be processed from the business server and determine the damage location, damage type, and damage area contained in the image to be processed. Of course, the above-mentioned acquisition of the image to be processed and identification of the damage location, damage type, and damage area can also be performed by the same server.

The component recognition model and damage recognition model described above may use a variety of training data. For example, in one embodiment, the component recognition model is configured to be trained using component sample images containing marking data, wherein the component sample images include at least one vehicle component.

The damage recognition model is configured to take as input a sample damage image for model training and output at least one damage location and the damage type corresponding to the damage location. Furthermore, when the damage recognition model is used to examine an image to be processed, it also outputs data indicating the degree of confidence in the authenticity of the damage type. While the damage recognition model may not output a confidence level during training, it will output a confidence level when the model is used.

It should be noted that the above-mentioned processing of detecting vehicle components using the component recognition model in S2 and the processing of detecting damage locations, damage types, and damage regions using the damage recognition model in S3 can be performed in parallel, that is, the same algorithm server or corresponding algorithm servers can be used to process the image to be processed and perform the image processing calculations of S2 and S3. Of course, the application does not exclude the implementation method of performing the processing of identifying vehicle components or the processing of identifying damage locations first in S2. As shown in Figures 2 and 3, Figure 2 is a schematic diagram of the network architecture structure of the damage recognition model in one embodiment, and Figure 3 is a schematic diagram of the network architecture of the component recognition model in one embodiment of the present application. In the actual terminal APP implementation process, the network model architecture of the component recognition model and the damage recognition model are basically the same. In the component recognition model, the damage region in the damage recognition model becomes the component region, and the damage type becomes the component type. Based on the network model structure of the present application shown in Figures 2 and 3, other improved, deformed, or transformed network models can also be included. However, in other embodiments of the present application, at least one of the component recognition model and the damage recognition model is a network model based on a convolutional layer and a region proposal layer. The implementation method of constructing a deep neural network generated after sample data training should fall within the scope of the implementation of the present application.

S4: Determine the damaged component in the image to be processed according to the damaged location and component area, and determine the damaged location and damage type of the damaged component.

After obtaining the information of the vehicle parts contained in the image to be processed, as well as the information of the damaged parts, damage types, and damaged areas in the image to be processed, it is possible to further search for the damaged parts in the detected vehicle parts. One implementation method of this embodiment can locate the damaged parts by analyzing the component area and the damaged area calculated in the above-mentioned recognition process. Specifically, it can be confirmed based on the position area of the damaged area and the component area in the image to be processed. For example, in a picture P1, if the damaged area detected in P1 is contained in the component area detected in P1 (usually the area of the identified component area is larger than the area of the damaged area), then it can be considered that the vehicle part corresponding to the component area in P1 is a damaged part. Alternatively, in picture P2, if the damaged area detected in P2 has an area overlap with the component area detected in P2, then it can also be considered that the vehicle part corresponding to the component area in P2 is also a damaged part. Therefore, in a specific implementation method provided by another embodiment of the present application, the determination of the damaged parts in the image to be processed based on the damaged parts and component areas can include:

S401: Query whether there is a damaged area of the damaged part within the component area range; if so, determine that the vehicle component corresponding to the component area is a damaged component.

In a specific example, for example, in an image P, when S2 is processing, it is detected that the vehicle parts are the left front door and the left front fender. The component areas where these two vehicle parts are located in the image P are (r1, r2), and the corresponding confidence levels are (p1, p2). S3 detects that there is a slight scratch (a type of damage) in the image P. The slight scratch is located in the damaged area r3 in the image P, with a confidence level of p3. After processing the image position area correspondence, it is found that the slight scratch area r3 is in the component area r1 where the left front door is located, thus obtaining the damaged component left front door, and the damaged part of the damaged component is r3. The damage type of the damaged component in this single image P is a slight scratch, and the confidence level is p1*p3.

Of course, if damage is detected on the left front fender at the same time, then according to the above example, it can be determined that the damaged component in image P also includes the left front fender, and its damage location and damage type can also be calculated.

During damage assessment, the image to be processed is input into a convolutional neural network. If multiple damage locations exist, multiple image regions containing the damage locations are detected, the image regions are examined, the damage types of the image regions are determined, and the damage location and damage type corresponding to each image region are output. Furthermore, in this embodiment, the damage location corresponding to the damage type with the highest degree of damage among the damage types can be selected as the damage location of the damaged component. Accordingly, the damage type with the highest degree of damage is determined as the damage type of the damaged component.

S5: Generate a maintenance plan based on the information including the damaged component, damaged location, and damage type.

After identifying vehicle components in the image to be processed, confirming damaged components, and identifying the location and type of damage, information for vehicle damage assessment in this embodiment is obtained. A repair plan can then be generated based on this information. This repair plan can be a damage assessment result for each damaged component, or a single damage assessment result for multiple damaged components across the entire vehicle.

In this embodiment, each damage type can be set to correspond to a maintenance plan, such as severe deformation corresponds to replacement parts, mild deformation requires sheet metal, and minor scratches require painting. For users, the final output of a damaged component can be a maintenance plan. When a damaged component has multiple damages, the maintenance method of the most severely damaged part can be used as the final treatment method for the entire component. Usually, a component on a vehicle is a whole. If there are multiple damages, it is more reasonable to treat it with the most severe damage. In this embodiment, a maintenance plan can be selected to solve all damages on the damaged component. For example, in a damaged component, the damage type of one damaged part is severe damage and requires replacement parts, and the damage type of another damaged part is moderate deformation and requires sheet metal. In this case, you can choose to replace the part and there is no need to perform sheet metal processing.

It should be understood that damage assessment typically includes both damage assessment and price assessment information. In the embodiments of this application, if the output repair plan does not include repair cost information, it can be classified as the damage assessment portion. If it includes repair cost information, it can be considered that both the damage assessment and price assessment have been calculated. Therefore, the repair plans described in this embodiment are all one type of repair plan processing result of vehicle damage assessment.

In a specific example, after the algorithm server identifies the damaged component in the image to be processed, as well as the damage location and type of the damaged component, it can generate a repair plan for the vehicle component based on this information and preset processing rules. For example, the left front fender of the 2016 B1 model car from manufacturer A1 is slightly deformed and requires sheet metal repair; the left front door of the 2010 B2 model car from manufacturer A2 is severely scratched and severely deformed and requires replacement; the front bumper of the 2013 B3 model car from manufacturer A3 is slightly scratched and requires painting; the left headlight requires disassembly and inspection, etc.

In another embodiment of the method described herein, to meet the user's need for information on cost and price in vehicle damage assessment, the repair plan may also include information on estimated revised prices for vehicle component repairs. This allows the user to be informed of the revised cost information and select a more appropriate repair method, thereby meeting the user's needs and improving the user experience. Therefore, in another embodiment of the method described herein, the method may further include:

S500: Acquire information on the maintenance strategy of the vehicle component;

Correspondingly, the maintenance plan may also include an estimated maintenance price corresponding to the maintenance strategy; the estimated maintenance price of the vehicle component is calculated based on information including the damaged location, damage type, and maintenance strategy of the vehicle component, and after querying the price data of the products and/or maintenance services corresponding to the vehicle component in the maintenance strategy.

Figure 4 is a flow chart of another embodiment of the method described in the present application. In terms of specific implementation, a calculation rule can be designed to call different price libraries based on the vehicle model to which the vehicle component belongs, the selected vehicle component maintenance location, the repair shop (4S point or ordinary comprehensive repair shop), and other maintenance strategy information to generate a maintenance plan for the vehicle component that includes a pre-repair processing method and a corresponding estimated maintenance price. The maintenance strategy information can be determined by user selection. For example, the user can select the maintenance location (such as city-level or district-level division), whether it is a 4S store or a comprehensive repair shop, and input the car brand and model. Then, the algorithm server can obtain a maintenance plan similar to the following based on the maintenance strategy information of the vehicle component and the identified damage location and damage type:

The front bumper of the A3 manufacturer's 2013 B3 model has slight scratches and needs to be painted; the estimated repair cost at the local 4S shop is: 600 yuan.

Of course, other implementations could also leverage traditional auto insurance companies' claims experience to compile information such as damaged parts, damage type, and extent at the accident scene, combined with repair time and costs at the 4S dealership to create an engine module. Once the actual processing application identifies the damaged part and type, the engine module can be invoked to output the damage assessment results.

The maintenance strategy information described above can be modified and replaced. For example, if the user chooses to have the vehicle repaired at a 4S dealership, a maintenance strategy will be generated, along with a corresponding maintenance plan. If the user chooses to have the vehicle repaired at a comprehensive repair shop, a different maintenance strategy will be generated, along with a different maintenance plan.

This application also provides a specific implementation of a damage identification model sample training process. In another specific embodiment of the method, as shown in FIG5 , the damaged component and the damaged location and damage type of the damaged component can be determined in the following manner:

S10: obtaining a set of images to be processed containing the damaged part;

S20: extracting feature vectors of the images to be processed in the set using a convolutional neural network, performing clustering processing on images of the same vehicle parts based on the feature vectors, and determining damaged parts;

S30: merging the damaged parts belonging to the same damaged component to obtain damage clustering feature data of the damaged parts;

S40: Determine the damaged parts included in the damaged component and the damage types corresponding to the damaged parts according to the damage clustering feature data.

In a specific example, for any detected damaged component p, it corresponds to one or more damaged locations (including damage type, location, and confidence) detected in one or more images. These images are clustered, and the image distance is calculated using the feature vectors extracted from the images through a convolutional network, such as the output vector of the damaged sample image trained using the last input model of the convolutional network in Ns. The damaged locations within images belonging to the same cluster t are merged (selecting the top-K by confidence, where k can be 15) as the feature Ft. The features (Ft1, Ft2, ...) of the top-C clusters (C can be 5, sorted by the weighted number of damaged locations within the cluster, with the weight being the confidence of the damaged location) are selected as the feature input to a multi-class gradient boosting decision tree (GBDT). A multi-class gradient boosting decision tree (GBDT) model is used to ultimately determine the damage type and severity. This GBDT model can be trained using gradient descent on labeled data.

It is understood that the damaged image can be a sample image used during model training. In actual user use, it can be the image to be processed. The image clustering described above primarily clusters images containing identical components. The purpose of clustering is to find images captured of approximately the same portion of the damaged component. The damaged components, damaged locations, and damaged types obtained in S2 and S3 are then confirmed using the above-described embodiment to obtain the damaged components, corresponding damaged locations, and damaged types in the image to be processed.

Furthermore, in another embodiment, merging the damaged parts belonging to the same damaged component may include:

From the images to be processed that belong to the same damaged component in the image clustering, the damaged parts of K images to be processed are selected in descending order of confidence and merged, where K ≥ 2.

After merging, TOPK confidence levels are selected for processing, which can improve the recognition processing speed, especially when training a large number of sample images. In the implementation scenario of model training in this embodiment, K can be set to 10 to 15.

In another embodiment, obtaining the damage clustering feature data of the damage site may include:

From the merged image clusters, select C damage cluster feature data of the images to be processed in descending order of the weighted values of the damage locations, where C ≥ 2, and the weight factor of the weighted value is the confidence level of the damage location. In the implementation scenario of the model training of this embodiment, C can be 3 to 5.

In some other embodiments, determining the damage locations included in the damaged component and the damage types corresponding to the damage locations based on the damage clustering feature data includes:

The damage clustering feature data is used as input data of a multi-classification gradient boosting decision tree model to identify the damage location and damage type.

It is understood that the aforementioned processing of the to-be-processed images can be sample images used during model training. For example, the set of training sample images containing damaged parts obtained in S10 above, or the merging of damaged parts from training sample images belonging to the same damaged part in image clustering, selected in descending order of confidence, etc. The implementation process of model training refers to the description of the to-be-processed images above and will not be repeated here.

The implementation scheme described above can improve the reliability and accuracy of damage assessment results while further accelerating the processing speed.

Optionally, a realistic rendering is performed using the vehicle 3D model under multiple angles and lighting models to generate images, and the positions of various vehicle components in the images are simultaneously determined. The rendered images are added to the training data and trained together with the labeled data. Therefore, in another embodiment, the training sample images used in at least one of the component recognition model and the damage recognition model may include:

Image information generated by computer simulation of damaged vehicle parts.

The present application provides an image-based vehicle damage assessment method that can identify damaged parts contained in an image to be processed, and then detect multiple damaged parts of the damaged parts and the damage type corresponding to each damaged part based on a constructed damage recognition model, thereby obtaining accurate, comprehensive, and reliable vehicle damage assessment information for the vehicle parts. Furthermore, the implementation scheme of the present application generates a vehicle repair plan based on these damaged parts and the information on the damaged parts, the damaged parts type, and the repair strategy, providing insurance personnel and car owners with more accurate, reliable, and practical damage assessment information. The implementation scheme of the present application can identify one or more damaged parts in one or more images, and one or more damaged parts and the degree of damage in the damaged parts, etc., quickly obtain more comprehensive and accurate damage assessment information, and then automatically generate a repair plan, which can meet the insurance company or car owner's needs for fast, comprehensive, accurate, and reliable vehicle damage assessment processing, improve the accuracy and reliability of vehicle damage assessment processing results, and improve user service experience.

Based on the above-mentioned image-based vehicle damage assessment method, the present application also provides an image-based vehicle damage assessment device. The device may include a system (including a distributed system), software (application), modules, components, servers, clients, etc. using the method described in the present application and a device combined with necessary implementation hardware. Based on the same innovative concept, the device in an embodiment provided by the present application is as described in the following embodiment. Since the implementation scheme of the device to solve the problem is similar to the method, the implementation of the specific device of the present application can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. As used below, the term "unit" or "module" can implement a combination of software and/or hardware for predetermined functions. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived. Specifically, Figure 6 is a schematic diagram of the module structure of an embodiment of an image-based vehicle damage assessment device provided by the present application. As shown in Figure 6, the device may include:

The image acquisition module 101 can be used to acquire the image to be processed for vehicle damage assessment;

The first recognition module 102 may be configured to detect the image to be processed using the constructed component recognition model, identify the vehicle component in the image to be processed, and confirm the component region of the vehicle component in the image to be processed;

The second recognition module 103 may be used to detect the image to be processed using the constructed damage recognition model, and identify the damage location and damage type in the image to be processed;

The damage calculation module 104 may be configured to determine a damaged component in the image to be processed, and determine a damage location and damage type of the damaged component based on the processing results of the first recognition module 102 and the second recognition module 103;

The damage assessment processing module 105 can be used to generate a maintenance plan based on information including the damaged component, damaged location, and damage type.

With reference to the aforementioned method, the device may also include other implementations. For example, the damage identification model may be a network model based on a convolutional layer and a region proposal layer, and a deep neural network may be constructed and generated after training with sample data. Alternatively, the device may also include a maintenance strategy acquisition module or directly obtain maintenance strategy information for the vehicle components through the damage assessment processing module 105, and generate a maintenance plan including an estimated maintenance price. For details, please refer to the relevant description of the aforementioned method embodiment, and a detailed description will not be given here.

The method or apparatus described above in this application can be implemented through a computer program combined with necessary hardware and can be installed in an application of a device to quickly and reliably output image-based vehicle damage assessment results. Therefore, this application also provides an image-based vehicle damage assessment device that can be used on a server and can include a processor and a memory for storing processor-executable instructions. When the processor executes the instructions, it achieves the following:

Obtaining images to be processed for vehicle damage assessment;

Detecting the image to be processed using the constructed component recognition algorithm, identifying the vehicle component in the image to be processed, and confirming the component area of the vehicle component in the image to be processed;

Detecting the image to be processed using the constructed damage recognition algorithm to identify the damage location and damage type in the image to be processed;

Determining a damaged component in the image to be processed according to the damaged portion and component region, and determining a damaged portion and a damage type of the damaged component;

A maintenance plan is generated based on the information including the damaged component, damaged location, and damage type.

The specific actual processing of the above-mentioned device may also include other processing hardware, such as a GPU (Graphics Processing Unit). As described in the above method, in another embodiment of the device, when the processor executes the instruction, it can also achieve:

Obtaining information on a repair strategy for the damaged component;

Correspondingly, the maintenance plan also includes an estimated maintenance price corresponding to the maintenance strategy; the estimated maintenance price is calculated based on information including the damaged component, damaged location, damage type, and maintenance strategy, and after querying the price data of the product and/or maintenance service corresponding to the damaged component in the maintenance strategy.

In another embodiment of the device, the instructions for constructing the damage identification algorithm may include algorithm processing instructions for constructing a deep neural network based on a network model of a convolutional layer and a region proposal layer after training with sample data.

In another embodiment of the device, when the processor executes the instruction, the damaged component and the damaged location and damage type of the damaged component are determined in the following manner:

Acquire a set of images to be processed containing the injury site;

Extracting feature vectors of the images to be processed in the set using a convolutional neural network, performing clustering processing on images of the same vehicle parts based on the feature vectors, and determining damaged parts;

Merge the damaged parts belonging to the same damaged component to obtain damage clustering feature data of the damaged parts;

The damaged parts included in the damaged component and the damage types corresponding to the damaged parts are determined according to the damage clustering feature data.

By using an image-based vehicle damage assessment device provided by an embodiment of the present application, it is possible to identify damaged parts contained in an image to be processed, and then detect multiple damaged parts of the damaged parts and the damage type corresponding to each damaged part based on the constructed damage recognition model, thereby obtaining accurate, comprehensive and reliable vehicle damage assessment information for the vehicle parts. Furthermore, the implementation scheme of the present application generates a vehicle repair plan based on these damaged parts and the information on the damaged parts, the damage type and the repair strategy, providing insurance personnel and car owners with more accurate, reliable and practical damage assessment information. The implementation scheme of the present application can identify one or more damaged parts in one or more images, and one or more damaged parts and the degree of damage in the damaged parts, etc., quickly obtain more comprehensive and accurate damage assessment information, and then automatically generate a repair plan, which can meet the needs of insurance companies or car owners for fast, comprehensive, accurate and reliable vehicle damage assessment processing, improve the accuracy and reliability of vehicle damage assessment processing results, and improve user service experience.

The methods or devices described in the above embodiments of the present application can implement business logic through computer programs and record them on a storage medium, which can be read and executed by a computer to achieve the effects of the solutions described in the embodiments of the present application. Therefore, the present application also provides a computer-readable storage medium on which computer instructions are stored, which, when executed, can implement the following steps:

Obtaining images to be processed for vehicle damage assessment;

Detecting the image to be processed using the constructed component recognition algorithm, identifying the vehicle component in the image to be processed, and confirming the component area of the vehicle component in the image to be processed;

Detecting the image to be processed using the constructed damage recognition algorithm to identify the damage location and damage type in the image to be processed;

Determining a damaged component in the image to be processed according to the damaged portion and component region, and determining a damaged portion and a damage type of the damaged component;

A maintenance plan is generated based on the information including the damaged component, damaged location, and damage type.

The computer-readable storage medium may include a physical device for storing information, typically digitizing the information and then storing it in a medium utilizing electrical, magnetic, or optical means. The computer-readable storage medium described in this embodiment may include: devices that store information using electrical energy, such as various types of memory, such as RAM and ROM; devices that store information using magnetic energy, such as hard disks, floppy disks, magnetic tapes, magnetic core memories, bubble memories, and USB flash drives; and devices that store information optically, such as CDs or DVDs. Of course, there are other types of computer-readable storage media, such as quantum memories, graphene memories, and so on.

The device or method described above can be used in electronic devices for image processing to achieve rapid processing of vehicle damage assessment based on images. The electronic device can be a separate server, a system cluster composed of multiple application servers, or a server in a distributed system. Figure 7 is a structural diagram of an embodiment of an electronic device provided by the present application. In one embodiment, the electronic device may include a processor and a memory for storing processor-executable instructions, and when the processor executes the instructions, it implements:

Obtaining images to be processed for vehicle damage assessment;

Detecting the image to be processed using the constructed component recognition algorithm, identifying the vehicle component in the image to be processed, and confirming the component area of the vehicle component in the image to be processed;

Detecting the image to be processed using the constructed damage recognition algorithm to identify the damage location and damage type in the image to be processed;

Determining a damaged component in the image to be processed according to the damaged portion and component region, and determining a damaged portion and a damage type of the damaged component;

A maintenance plan is generated based on the information including the damaged component, damaged location, and damage type.

Figure 8 is a schematic diagram of a processing scenario for vehicle damage assessment using the implementation scheme of the present application. The client in Figure 8 is the user's mobile terminal, and it can also be a PC or other terminal device in other implementation scenarios. The present application provides an image-based vehicle damage assessment method, device and electronic device, which uses deep learning technology to detect damaged parts and damage types, and uses image matching methods to accurately locate damaged parts, and can use multi-image detection results to improve the accuracy of damage assessment. Image detection technology is combined with a vehicle parts price library and maintenance rules to automatically generate repair plans and estimates. Some implementation schemes of the present application can automatically generate repair plans and estimated repair costs based on more specific vehicle damage information, vehicle parts price libraries, and repair processing methods, which can meet the needs of insurance companies or car owners for fast, comprehensive, accurate and reliable vehicle damage assessment processing, improve the accuracy and reliability of vehicle damage assessment processing results, and improve user service experience.

It should be noted that although the above embodiments provide descriptions of embodiments of some devices, electronic devices, and computer-readable storage media, based on the descriptions of the aforementioned related methods or device embodiments, the devices, electronic devices, and computer-readable storage media may also include other implementation methods. For details, please refer to the descriptions of the related methods or device embodiments, and no further examples will be given here.

Although the content of this application mentions image quality processing, convolutional neural networks and region proposal networks and deep neural networks generated by their combination, calculation methods for estimating repair prices, and descriptions of data model construction, data acquisition, interaction, calculation, judgment, etc. using the GBDT model to obtain damage location and type processing methods, etc., this application is not limited to situations that must comply with industry communication standards, standard data models, computer processing and storage rules, or the embodiments of this application. Certain industry standards or slightly modified implementation plans based on the implementation described in the custom methods or embodiments can also achieve the same, equivalent or similar, or predictable implementation effects as the above-mentioned embodiments. Examples obtained by applying these modified or deformed data acquisition, storage, judgment, processing methods, etc. can still fall within the scope of the optional implementation plans of this application.

In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, an in-vehicle human-computer interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

Although the application provides the method operation steps as described in the embodiment or flow chart, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiment is only a way in the order of many step executions and does not represent a unique execution order. When the device or terminal product in practice is executed, it can be performed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (such as a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "comprise", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or equipment including a series of elements not only include those elements, but also include other elements not clearly listed, or also include elements inherent to such process, method, product or equipment. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or equipment including the elements.

For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and/or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

Those skilled in the art will also appreciate that, in addition to implementing the controller in pure computer-readable program code, it is entirely possible to implement the same functionality by logically programming the method steps in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, and the like. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered structures within the hardware component. Alternatively, the devices for implementing various functions can be considered both software modules implementing the method and structures within the hardware component.

The present invention is described with reference to the flowcharts and/or block diagrams of the methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and/or box in the flowchart and/or block diagram, as well as the combination of processes and/or boxes in the flowchart and/or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and/or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

The various embodiments in this specification are described in a progressive manner. Similar or identical parts between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from other embodiments. In particular, since the system embodiments are generally similar to the method embodiments, the description is relatively simple. For relevant parts, reference can be made to the partial description of the method embodiments. In the description of this specification, reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as features of different embodiments or examples, without conflict.

The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims (15)

1. A vehicle damage assessment method based on an image, characterized in that the method comprises: Obtaining images to be processed for vehicle damage assessment; Detecting the image to be processed using the constructed component recognition model, identifying the vehicle component in the image to be processed, and confirming the component region of the vehicle component in the image to be processed; Using the constructed damage recognition model to detect the image to be processed, and identifying the damage location and damage type in the image to be processed; Determining a damaged component in the image to be processed according to the damaged portion and component region, and determining a damaged portion and a damage type of the damaged component; generating a maintenance plan based on information including the damaged component, damaged location, and damage type; The damaged component and the damaged location and damage type of the damaged component are determined in the following manner: Acquire a set of images to be processed containing the injury site; Extracting feature vectors of the images to be processed in the set using a convolutional neural network, performing clustering processing on images of the same vehicle parts based on the feature vectors, and determining damaged parts; Merge the damaged parts belonging to the same damaged component to obtain damage clustering feature data of the damaged parts; The damaged parts included in the damaged component and the damage types corresponding to the damaged parts are determined according to the damage clustering feature data. 2. The image-based vehicle damage assessment method according to claim 1, further comprising: Obtaining information on a repair strategy for the damaged component; Correspondingly, the maintenance plan also includes an estimated maintenance price corresponding to the maintenance strategy; the estimated maintenance price is calculated based on information including the damaged component, damaged location, damage type, and maintenance strategy, and after querying the price data of the product and/or maintenance service corresponding to the damaged component in the maintenance strategy. 3. The image-based vehicle damage assessment method according to claim 1, wherein at least one of the component identification model and the damage identification model is: A network model based on convolutional layers and region proposal layers is used to build a deep neural network after training with sample data. 4. The image-based vehicle damage assessment method according to claim 3, wherein the component recognition model is configured to be trained using component sample images containing labeled data, wherein the component sample images include at least one vehicle component; The damage recognition model is configured to input a damage sample image for model training, and output at least one damage site and a damage type corresponding to the damage site; and when using the damage recognition model to detect the image to be processed, also output data information indicating the degree of confidence in the authenticity of the damage type. 5. The image-based vehicle damage assessment method according to claim 4, wherein determining the damaged component in the image to be processed based on the damaged location and component area comprises: In the component area range, it is checked whether there is a damaged area of the damaged part; if so, the vehicle component corresponding to the component area is determined to be a damaged component. 6. The image-based vehicle damage assessment method according to claim 1, wherein merging damaged parts belonging to the same damaged component comprises: From the images to be processed that belong to the same damaged sample component in the image clustering, the damaged parts of K images to be processed are selected in descending order of confidence and merged, where K ≥ 2. 7. The image-based vehicle damage assessment method according to claim 1, wherein obtaining damage clustering feature data of the damaged parts comprises: From the merged image clusters, damage cluster feature data of C images to be processed are selected in descending order according to the weighted values of the damage parts, where C≥2, and the weight factor of the weighted value is the confidence of the damage part. 8. The image-based vehicle damage assessment method according to any one of claims 6 or 7, wherein identifying the damage location and damage type in the damage sample image based on the damage cluster feature data comprises: The damage clustering feature data is used as input data of a multi-classification gradient boosting decision tree model to identify the damage location and damage type. 9. The image-based vehicle damage assessment method according to claim 1, wherein the training sample images used by at least one of the component recognition model or the damage recognition model include: Image information generated by computer simulation of damaged vehicle parts. 10. An image-based vehicle damage assessment device, characterized in that the device comprises: An image acquisition module is used to acquire images to be processed for vehicle damage assessment; a first recognition module, configured to detect the image to be processed using the constructed component recognition model, identify the vehicle component in the image to be processed, and confirm the component region of the vehicle component in the image to be processed; A second recognition module is used to detect the image to be processed using the constructed damage recognition model, and identify the damage location and damage type in the image to be processed; a damage calculation module, configured to determine a damaged component in the image to be processed, and determine a damage location and damage type of the damaged component based on processing results of the first recognition module and the second recognition module; A damage assessment processing module, configured to generate a repair plan based on information including the damaged component, damaged location, and damage type; The second identification module determines the damaged component and the damaged location and damage type of the damaged component in the following manner: Acquire a set of images to be processed containing the injury site; Extracting feature vectors of the images to be processed in the set using a convolutional neural network, performing clustering processing on images of the same vehicle parts based on the feature vectors, and determining damaged parts; Merge the damaged parts belonging to the same damaged component to obtain damage clustering feature data of the damaged parts; The damaged parts included in the damaged component and the damage types corresponding to the damaged parts are determined according to the damage clustering feature data. 11. An image-based vehicle damage assessment device, comprising a processor and a memory for storing processor-executable instructions, wherein when the processor executes the instructions: Obtaining images to be processed for vehicle damage assessment; Detecting the image to be processed using the constructed component recognition algorithm, identifying the vehicle component in the image to be processed, and confirming the component area of the vehicle component in the image to be processed; Detecting the image to be processed using the constructed damage recognition algorithm to identify the damage location and damage type in the image to be processed; Determining a damaged component in the image to be processed according to the damaged portion and component region, and determining a damaged portion and a damage type of the damaged component; generating a maintenance plan based on information including the damaged component, damaged location, and damage type; The damaged component and the damaged location and damage type of the damaged component are determined in the following manner: Acquire a set of images to be processed containing the injury site; Extracting feature vectors of the images to be processed in the set using a convolutional neural network, performing clustering processing on images of the same vehicle parts based on the feature vectors, and determining damaged parts; Merge the damaged parts belonging to the same damaged component to obtain damage clustering feature data of the damaged parts; The damaged parts included in the damaged component and the damage types corresponding to the damaged parts are determined according to the damage clustering feature data. 12. The image-based vehicle damage assessment device according to claim 11, wherein when the processor executes the instruction, it further implements: Obtaining information on a repair strategy for the damaged component; Correspondingly, the maintenance plan also includes an estimated maintenance price corresponding to the maintenance strategy; the estimated maintenance price is calculated based on information including the damaged component, damaged location, damage type, and maintenance strategy, and after querying the price data of the product and/or maintenance service corresponding to the damaged component in the maintenance strategy. 13. The image-based vehicle damage assessment device according to claim 11, characterized in that: The instructions for constructing the damage identification algorithm include a network model based on a convolutional layer and a region proposal layer, and algorithm processing instructions for constructing a deep neural network generated after training with sample data. 14. A computer-readable storage medium having computer instructions stored thereon, wherein when the instructions are executed, the following steps are implemented: Obtaining images to be processed for vehicle damage assessment; Detecting the image to be processed using the constructed component recognition algorithm, identifying the vehicle component in the image to be processed, and confirming the component area of the vehicle component in the image to be processed; Detecting the image to be processed using the constructed damage recognition algorithm to identify the damage location and damage type in the image to be processed; Determining a damaged component in the image to be processed according to the damaged portion and component region, and determining a damaged portion and a damage type of the damaged component; generating a maintenance plan based on information including the damaged component, damaged location, and damage type; The damaged component and the damaged location and damage type of the damaged component are determined in the following manner: Acquire a set of images to be processed containing the injury site; Extracting feature vectors of the images to be processed in the set using a convolutional neural network, performing clustering processing on images of the same vehicle parts based on the feature vectors, and determining damaged parts; Merge the damaged parts belonging to the same damaged component to obtain damage clustering feature data of the damaged parts; The damaged parts included in the damaged component and the damage types corresponding to the damaged parts are determined according to the damage clustering feature data. 15. An electronic device, comprising a processor and a memory for storing processor-executable instructions, wherein when the processor executes the instructions: Obtaining images to be processed for vehicle damage assessment; Detecting the image to be processed using the constructed component recognition algorithm, identifying the vehicle component in the image to be processed, and confirming the component area of the vehicle component in the image to be processed; Detecting the image to be processed using the constructed damage recognition algorithm to identify the damage location and damage type in the image to be processed; Determining a damaged component in the image to be processed according to the damaged portion and component region, and determining a damaged portion and a damage type of the damaged component; generating a maintenance plan based on information including the damaged component, damaged location, and damage type; The damaged component and the damaged location and damage type of the damaged component are determined in the following manner: Acquire a set of images to be processed containing the injury site; Extracting feature vectors of the images to be processed in the set using a convolutional neural network, performing clustering processing on images of the same vehicle parts based on the feature vectors, and determining damaged parts; Merge the damaged parts belonging to the same damaged component to obtain damage clustering feature data of the damaged parts; The damaged parts included in the damaged component and the damage types corresponding to the damaged parts are determined according to the damage clustering feature data.