CN113342396B - Method for pre-selecting targets in Android system image recognition - Google Patents

Abstract

The invention discloses a method for pre-selecting targets in Android system image recognition, and belongs to the technical field of digital image recognition. The method comprises the following specific processes: the method comprises the steps that a user selects a system patch package on a cloud server according to an image to be identified to download, if the cloud server does not have a corresponding system patch package, the user-defined patch package is uploaded to the cloud server, then the downloaded patch package is decompressed, a dex file, a so link library file and a model file are decompressed, the image identification function is reconstructed in an Android system, and the pre-selection of the image to be identified is achieved; the system patch package and the custom patch package both comprise: model files, so link library files, and dex files. According to the method, dynamic switching can be realized for different types of target identification in the Android system image identification, an App package does not need to be frequently issued, and meanwhile, the volume of the APP package is not additionally increased.

Description

A Method of Target Preselection in Image Recognition of Android System

technical field

The invention belongs to the technical field of digital image recognition, and in particular relates to a method for preselection of targets in image recognition of an Android system.

Background technique

With the rapid development of artificial intelligence technology, digital image recognition processing technology has been applied in many fields. At present, the mainstream Android system image recognition application equipment is mainly customized image processing equipment, and customized image processing equipment has real-time intelligence of video stream frames. Processing function. Since Android is a Linux kernel, and the performance of java at the application layer is not as good as that of the underlying C/C++, image recognition on the Android system needs to take into account both real-time and performance requirements, and it is necessary to call the underlying framework through JNI (Java Native Interface). Algorithm, first use the Android framework to write the java-side program code, write the local C/C++ program code through JNI and OpenCV or other interfaces, use the Android NDK (Native Development Kit) to compile it to generate a shared library that the Java code can call, and pass The data set model of the trained custom image is loaded and recognized. Therefore, in actual operation, the integrated device of the Android system or the Android mobile phone is connected to the image acquisition device, and the pictures collected by the mobile phone or the device pass the YUV data to the App processing layer through the interface. , and pass the parameters through the dynamic link library technology of the JNI layer to complete the image preprocessing by the so (shared object) library algorithm, and then complete the identification of the main target through the dataset model trained by the image dataset, and return it through the JNI interface callback of the NDK Go to the Android APP data processing layer to complete the marking and early warning of the target and the drawing of the related interface.

The existing model files are collected and classified in the early stage, pre-processed, converted to format, and batch processed to generate a model file in a specified format. The final trained model file is stored in the resource package assets of the Android program and compiled together with the main program. Generate an app installation package, so it does not have the ability to recognize the diversity and scalability of targets. For example, if the intelligent recognition device realizes the recognition of targets of the type of flame, the system cannot recognize other target objects other than flames, so there are limitations performance and poor scalability. When it is necessary to identify and expand other targets, the existing technology is developed through incremental programs, that is, retraining to generate model files, and developing new so link library files through NDK, integrating multiple so library files in the APP package and Store multiple image data set model files in the resource directory, so as to complete the recognition function of multiple targets on the APP, which will cause an expansion of the APP package size. The more types of target recognition supported, the larger the package size. . At the same time, the existing technology does not support user-defined requirements. When the user needs to identify the class of objects he is interested in, and the system does not have the ability to dynamically generate image dataset model files, the image recognition function for user-defined objects cannot be completed.

Contents of the invention

Aiming at the problems existing in the prior art, the present invention provides a method for pre-selecting targets in image recognition of the Android system. This method can dynamically switch between different types of target recognition in Android system image recognition, and at the same time satisfy the upload of pre-target image datasets according to user-defined targets, and the cloud server automation script completes the dataset training and distribution. In this way, the recognition of custom targets can be completed, and the function of recognizing multi-type targets and custom targets on the Android system can be realized. Type expansion is supported, and there is no need to frequently release App packages, and at the same time, it does not increase the size of App packages.

In order to achieve the above object, the present invention adopts the following technical solutions: a method for pre-selecting targets in Android system image recognition, the specific process is: the user selects the system patch package on the cloud server according to the image to be recognized to download, such as the cloud server If there is no corresponding system patch package on the Internet, upload the user-defined patch package to the cloud server, then download it, decompress the downloaded patch package, and decompress the dex file, so link library file and model file. In the Android system The image recognition function is reconstructed to realize the pre-selection of the image to be recognized; both the system patch package and the custom patch package include: model files, so link library files and dex files.

Further, the generation process of the system patch package is specifically: the program developer collects the images of the model files of the system patch package, classifies them according to the image features, uses pytorch to train the images of the model files, and optimizes the model files of the images to be recognized Structural parameters, provide the trained model files to NDK development; program developers develop API processing class java code programs to form . The dx tool converts the .jar file into a dex file recognizable by the Android system; the program developer completes image preprocessing, algorithm processing, heterogeneous scheduling, and NPU acceleration in the NDK through NDK C/C++ program development, and outputs the so link Library files; finally compress the dex file, so link library file and model file into a system patch package.

Further, the API handles the interface line parameters and method names of java-like code programs to be consistent.

Further, the system patch package is mapped and configured according to different image types, and stored in the cloud server.

Further, the generation process of the custom patch package is as follows: the user selects an image set, and the images in the image set are screened, cropped, and uploaded to the cloud server to generate a custom image data set, and the automation on the cloud server The script uses Jenkins and pytorch's automated training to complete the processing and training of the image dataset, generate model files, and then compress the so link library files, dex files, and generated model files developed for the corresponding image sets into a custom patch package.

Further, the reorganization process of the image recognition function of the decompressed dex file is specifically: pass the decompressed dex file through two class loaders, DexClassLoader and PathClassLoader, modify PathClassLoader.pathList.dexElements during runtime, and construct a DexClassLoader object, Obtain the system default PathClassLoader.pathList.dexElements through reflection technology; then merge the dex file with the default Elements array in the original system, set the merged array back to PathClassLoader.pathList.dexElements, and then call the element.dexFile object The loadClassBinaryName method on to complete the class loading.

Further, the dynamic loading process of the decompressed so link library file is specifically: copy the decompressed so link library file to the private directory /data/packagename/, and before calling the API of the so link library file, pass the System. The load() method dynamically loads the so link library file under the private directory path and then calls the API.

Compared with the prior art, the present invention has the following beneficial effects: the preselection method of the present invention solves the defect of single function in the image recognition system, and the present invention stores the system patch package and the custom patch on the cloud server package, download and decompress the patch package, extract the system executable program files in the patch package, the program files of different patch packages can complete the identification function for different targets, and use the dynamic loading of the patch package after disassembly to make the Android system The image recognition ability has been restructured, and the personalized image recognition function in the Android system has been enhanced. The object of the present invention can be pre-selected to support type expansion, without frequent release of App packages, and does not increase the size of the APP package.

Description of drawings

Fig. 1 is the flow chart of the present invention based on the method for the pre-selection of the target in the image recognition of the Android system;

Fig. 2 is a flow chart of establishing a recognition module in a method for pre-selecting a target in image recognition based on an Android system according to the present invention.

Detailed ways

The technical solution of the present invention will be further explained below in conjunction with the accompanying drawings.

Referring to Figures 1-2, the present invention provides a method for pre-selecting targets in image recognition in the Android system. Users can replace patch packages according to different images to be recognized to achieve pre-selection of targets in image recognition. For the purpose, the patch package involved in the present invention includes: a system patch package and a custom patch package, and the patch package includes: model files, so link library files and dex files. Therefore, the specific process of the method for pre-selecting the object of the present invention is as follows: the user selects the system patch package on the cloud server according to the image to be recognized to download, if there is no corresponding system patch package on the cloud server, upload the user-defined patch package To the cloud server, download again, decompress the downloaded patch package, decompress the dex file, so link library file and model file, and rebuild the image recognition function in the Android system to realize the pre-selection of the image to be recognized .

Specifically, the system patch package is a patch package in which developers complete the development of different types of subjects with characteristic objects. The image data sets in the system patch package are relatively standardized, complete and highly recognizable, allowing users to select scenes on the App . The generation process of the system patch package is specifically as follows: the program developer collects the image of the model file of the system patch package, first classifies it according to the image features, uses pytorch to train the image of the model file, optimizes the structural parameters of the model file of the image to be recognized, Provide the trained model file to the NDK development call, the model file is loaded through the association of the storage path in the program call, and use the model file to complete the identification of the target and effectively extract the main parameter information; the program developer performs API processing like java code The program develops into a .java file, compiles and generates a .jar file through the jdk tool, and then converts the .jar file into a dex file recognizable by the Android system by the dx tool in the platform-tools directory of the Android SDK, and completes the conversion of the patch java code program class file Become an executable file recognizable by the Android system, and the interface line parameter and method name of the API processing class java code program in the present invention are consistent; the program developer outputs the so link library file in the NDK through the C/C++ program development of the NDK, and the so library Complete the image preprocessing, algorithm processing, heterogeneous scheduling and NPU acceleration to obtain the best performance, and ensure the real-time requirements of the system recognition ability through the algorithm performance; finally compress the dex file, so link library file and model file into the system Patch package, the system patch package is mapped and configured according to different image types, and stored in the cloud server. The App downloads the corresponding system patch package according to the image to be recognized and completes the loading.

The generation process of the self-defined patch package in the present invention is specifically: the user selects the image set, and the images in the image set are screened, cut and uploaded to the cloud server to generate a custom image data set, and the automated script on the cloud server Through the automatic training of Jenkins and pytorch, the processing and training of the image data set is completed, the model file is generated, and the so link library file, dex file and generated model file developed for the corresponding image set are compressed into a custom patch package. Since the uncertain image recognition scene changes from time to time, resulting in the diversity of user needs, it is necessary to use the ability provided by the invention to realize the extension of the recognition subject type, and to achieve the goal of covering the recognition ability with the change of the scene target.

The decompressed dex file in the present invention carries out the reconstruction process of the image recognition function specifically: the dex file that is decompressed is passed through two class loaders of DexClassLoader and PathClassLoader, and PathClassLoader.pathList.dexElements is modified during operation, because the class loading order adopts It is an array traversal method, so the order in which dex appears in the dexElements array is very important. Construct a DexClassLoader object, which can load a dex file from the storage space, and obtain the system default PathClassLoader.pathList.dexElements through reflection technology; then combine the dex file with The default Elements array merge in the original system can ensure that the dex in the patch package is before the system default Elements array; set the merged array back to PathClassLoader.pathList.dexElements. Since the class loading is first obtained from the dexElements array in the DexPathList object, the processing method implementation class for loading the patch package can be realized, thereby replacing the default method body, and the default class is completed before the dex file loading order of the system default class Replacement to achieve coverage effect.

The dynamic loading process of the decompressed so link library file in the present invention is specifically: the so link library file that decompresses is copied in the private directory/data/packagename/, because in the C/C++ code environment of Native layer, when calling so link Before the API of the library file, use the System.load() method to dynamically load the so link library file under the private directory path and then call the API. There may be no direct dependence between the caller and the so link library file, and the loading is dynamically completed, thereby completely completing the decoupling, and realizing the loading of the corresponding so link library file according to different identification types of targets.

The method of using the model file in the present invention is as follows: store the decompressed model file in the designated directory of the private directory /data/packagename/, set the absolute path of the model file through the relevant initialization interface of JNI, and complete the model in the process of calling the algorithm File loading.

The package volume supporting multiple target types in the image recognition of the Android system of the present invention is compared with the package volume in the image recognition of the Android system based on traditional incremental development, as shown in Table 1: in the package volume of a single target type, using this The package volume of the inventive method is equal to the package volume of the incremental development method, but in the Android system package volume supporting two types of image recognition, the method of the present invention is adopted, and its package volume is equal to that of a single type of image recognition. And it is obviously smaller than the package volume of traditional incremental development. This is because in the traditional incremental development method, in the development of supporting two types of target type recognition functions, two so libraries and model files are put into arm64-v8a and assets respectively Under the directory, the size of the program package is significantly larger than that of a single target, but in the method of the present invention, the size of the APP package is not significantly increased in terms of ensuring the realization of the original functions by downloading and loading the patch package server.

Table 1: The package volume of the target of the present invention can be pre-selected method and the package volume comparison of the traditional incremental development method

Package volume for a single target type Package volume for two target types Traditional Incremental Development Approach 98.3M 143.6M The method of the invention 98.3M 98.3M

The above are only preferred implementations of the present invention, and the protection scope of the present invention is not limited to the above-mentioned embodiments, and all technical solutions under the idea of the present invention belong to the protection scope of the present invention. It should be pointed out that for those skilled in the art, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims (5)

1. a method for preselection of target in Android system image recognition, characterized in that, the specific process is: the user selects the system patch package on the cloud server according to the image to be identified to download, if there is no corresponding system patch on the cloud server package, upload the user-defined patch package to the cloud server, and then download it, decompress the downloaded patch package, decompress the dex file, so link library file and model file, and re-install the image recognition function in the Android system Forming to realize the pre-selection of the image to be recognized; both the system patch package and the custom patch package include: model files, so link library files and dex files; The generation process of the system patch package is specifically: the program developer collects the image of the model file of the system patch package, classifies according to the image features, uses pytorch to train the image of the model file, and optimizes the structural parameters of the model file of the image to be recognized , provide the trained model file to the NDK development; the program developer develops the API processing class java code program to form a . The .jar file is converted into a dex file recognizable by the Android system; the program developer completes the image preprocessing, algorithm processing, heterogeneous scheduling and NPU acceleration in the NDK through the C/C++ program development of the NDK, and outputs the so link library file; Finally, compress the dex file, so link library file and model file into a system patch package; The system patch package is mapped and configured according to different image types, and stored in the cloud server. 2. according to the method that target can be pre-selected in the described Android system image recognition of claim 1, it is characterized in that, described API handles the interface row parameter and method name of class java code program to keep consistent. 3. according to the method that target can be pre-selected in the described Android system image recognition of claim 1, it is characterized in that, the generation process of described self-defined patch package is specifically: select image collection by the user, and carry out to the image in image collection After screening, cropping and uploading to the cloud server, a custom image data set is generated. The automated script on the cloud server uses Jenkins and pytorch to complete the processing and training of the image data set, generate a model file, and then convert the corresponding image The developed so link library file, dex file and generated model file are compressed into a custom patch package. 4. according to the method for the pre-selection of the target in the Android system image recognition of claim 1, it is characterized in that, the reorganization process of the image recognition function carried out by the decompressed dex file is specifically: the decompressed dex file is passed through Two class loaders, DexClassLoader and PathClassLoader, modify PathClassLoader.pathList.dexElements at runtime, construct a DexClassLoader object, obtain the system default PathClassLoader.pathList.dexElements through reflection technology; then merge the dex file with the default Elements array in the original system, Set the merged array back to PathClassLoader.pathList.dexElements, and then complete class loading by calling the loadClassBinaryName method on the element.dexFile object. 5. according to the method that target can be pre-selected in the described Android system image recognition of claim 4, it is characterized in that, the dynamic loading process of the so link library file of described decompression is specifically: the so link library file that decompresses is copied to In the private directory /data/packagename/, before calling the API of the so link library file, dynamically load the so link library file under the private directory path through the System.load() method and then call the API.

Images