CN115480703B

CN115480703B - Method and system for reading USB data to disk under KMDF framework

Info

Publication number: CN115480703B
Application number: CN202211126848.XA
Authority: CN
Inventors: 赖成宾; 刘浩元; 张少校; 龚锐; 王鹏强
Original assignee: Zhongfu Safety Technology Co Ltd
Current assignee: Zhongfu Safety Technology Co Ltd
Priority date: 2022-09-16
Filing date: 2022-09-16
Publication date: 2025-09-16
Anticipated expiration: 2042-09-16
Also published as: CN115480703A

Abstract

The present invention relates to the technical field of data reading, and provides a method and system for reading USB data to a disk under a KMDF framework. The method comprises reading data packets from a USB pipe, marking each data packet with a serial number, and then pushing the data packets into a WDFWORKITEM queue; in a WDFWORKITEM queue callback, pushing the data packets in the WDFWORKITEM queue in the order in which thread processing is completed; adjusting the order of the data packets based on the transmission protocol and the serial number, completing the data packet splicing, and storing the data packets in a non-paged cache; determining whether storage conditions are met, and if so, writing the cached data to the disk and clearing the cache; otherwise, returning to continue reading the data packets; determining whether the transmission is complete, and if so, terminating the process; otherwise, returning to continue reading the data packets. This method not only improves the flexibility of processing data packets after reading USB device data, but also eliminates the need to consider interrupt priority restrictions; and simultaneously reduces I/O operations and the frequency of I/O interrupt resource occupation.

Description

Method and system for reading USB data to disk under KMDF framework

Technical Field

The invention belongs to the technical field of data reading, and particularly relates to a method and a system for reading USB data to a disk under a KMDF framework.

Background

The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.

The method for reading the USB data of the computer and storing the USB data on the disk is always an important mode for information exchange between the computer and USB equipment, and the USB is characterized by hot plug and play. From the advent of USB1.0 in 1996, USB3.2 has been developed so far, and the read/write speed has increased at a remarkable rate. At present, the application field of the USB interface is very wide, and a plurality of USB devices are derived, so that each device manufacturer is capable of improving the speed of reading data.

The device is driven to read data by a driver under the Windows system, and when the USB device is inserted, the system automatically identifies the USB device through interrupt response, so that plug and play is realized. The conventional method for reading USB data in the windows driving layer comprises the steps of establishing an IRP (I/O Request Package) in a WDM (Windows Driver Model) kernel frame to send a request to a USB pipeline, completing a routine to obtain data, and sending a USB high-capacity transmission request and using a continuous reader to read the data from the USB pipeline in a KMDF (Kernel Mode Driver Framework) kernel driving frame in a WDF (Windows Driver Framework) driving frame, and completing the routine to obtain the data. And sending the data packet read by the kernel layer to the user layer, and analyzing the data packet and storing the data packet by an application program.

For the reading of the WDM framework, if the USB pipeline data is required to be continuously read, the IRP request is required to be continuously sent to the bus, and the method consumes more system resources and has higher requirement on the fault tolerance of the program.

The analysis and storage of the data packet are carried out at the user layer, which relates to the data interaction between the kernel and the user layer, the interaction can increase the data copying operation to reduce the speed of writing into the disk, and the requirement on the transmission reliability is extremely high, so that the data cannot be lost.

If the read request completion routine of the WDM and WDF frames is directly stored in the read completion routine, the interrupt Level is a dispatch_level, and the function of the passive_level interrupt Level is not allowed to run in the range of the dispatch_level interrupt Level in the kernel, because the low interrupt Level cannot preempt the working thread of the high interrupt Level, once the working thread can cause a machine blue screen during use, and meanwhile, under the condition of multi-thread concurrency, the waiting lock is often used to complete data synchronization, the waiting time of the waiting lock needs to be set to be infinitely long, and the data can be guaranteed to have enough time to be correctly synchronized each time, but the interrupt Level of the waiting lock is a passive_level and cannot be used in the read completion routine.

In general, the above schemes either affect speed or consider the level of disruption of kernel functions, with the risk of blue-screening the machine with inappropriate functions.

Disclosure of Invention

In order to solve the technical problems in the background art, the invention provides a method and a system for reading USB data to a disk under a KMDF framework, which improve the flexibility of processing data packets after reading USB equipment data by using WDFWORKITEM queues without considering the limitation of interrupt priority, and simultaneously, buffer batch data by using non-paging continuous memory fragments, write the batch data into the disk once, reduce I/O operation and reduce the frequency of occupying I/O interrupt resources.

In order to achieve the above purpose, the present invention adopts the following technical scheme:

a first aspect of the present invention provides a method for reading USB data to a disk under a KMDF framework, including:

reading data packets from the USB pipeline, marking the serial numbers of the data packets, and pushing the data packets into a WDFWORKITEM queue;

in WDFWORKITEM queue callback, pushing out data packets in WDFWORKITEM queue according to the sequence of thread processing completion;

According to the transmission protocol and the serial number, the sequence of the data packets is adjusted, the splicing of the data packets is completed, and the data packets are stored in a non-paging cache;

judging whether the storage condition is met, if so, writing the cache data into a disk, and clearing the cache, otherwise, returning to continue reading the data packet;

and judging whether the transmission is finished, if so, ending the flow, and otherwise, returning to continue reading the data packet.

Further, the WDFWORKITEM queue may run in an environment of a workitem thread owned by the operating system, where the priority of the interruption of the workitem thread is passive_level, and the function with the interruption LEVEL of dispatch_level may preempt resources in the workitem thread, improve the operation of the interruption LEVEL, and reduce the interruption LEVEL after the operation is completed.

Further, a continuous reader is used to read data packets from the USB pipeline, specifically:

Enumerating an interrupt endpoint pipeline handle of the USB device, and storing the interrupt endpoint pipeline handle in a device context structure of the driver;

Creating a continuous reader object, configuring parameters, and enabling a driver to establish a USB pipeline;

starting the continuous reader, wherein a read request is arranged on the bus at any moment;

The data packets are all sequence numbered in the continuous reader read completion routine.

Further, the storage condition is specifically that the buffer is full or the file is completely transferred.

Further, the judging method of whether the transmission is finished is that if the received data packet band end identification is received, the transmission is finished.

A second aspect of the present invention provides a system for reading USB data to disk under KMDF framework, comprising:

the data reading module is configured to read data packets from the USB pipeline, mark the data packets with serial numbers, and push the data packets into a WDFWORKITEM queue;

A queue callback module configured to push out data packets in the WDFWORKITEM queue according to the sequence of thread processing completion in WDFWORKITEM queue callbacks;

the buffer module is configured to adjust the sequence of the data packets according to a transmission protocol and a serial number, finish the splicing of the data packets and store the data packets in the non-paging cache;

the storage module is configured to judge whether the storage condition is met, if yes, write the cache data into the disk and empty the cache, otherwise, return to continue reading the data packet;

and the transmission completion judging module is configured to judge whether the transmission is completed or not, if so, the process is ended, and if not, the data packet is returned to be read continuously.

A third aspect of the present invention provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of a method of reading USB data to disk under a KMDF framework as described above.

A fourth aspect of the invention provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, said processor implementing the steps in a method of reading USB data to disk under a KMDF frame as described above when said program is executed.

Compared with the prior art, the invention has the beneficial effects that:

The invention provides a method for reading USB data to a disk under a KMDF framework, which improves the flexibility of processing data packets after reading USB device data by using WDFWORKITEM queues, and the callback interrupt priority Level of the WDFWORKITEM queues can run a function smaller than or equal to a dispatch_level without considering the limitation of the interrupt priority Level.

The invention provides a method for reading USB data to a disk under a KMDF framework, which uses non-paging continuous memory fragments to buffer batch data, and the batch data is written into the disk once, so that I/O operation is reduced, and the frequency of occupying I/O interrupt resources is reduced.

The invention provides a method for reading USB data to a disk under a KMDF framework, which completes the work of writing the disk in a drive layer and reduces unnecessary memory copying.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.

FIG. 1 is a diagram of the overall architecture of a method for reading USB data to disk under the KMDF framework according to the first embodiment of the present invention;

FIG. 2 is a flowchart of a method for reading USB data to disk under a KMDF framework according to a first embodiment of the present invention;

Fig. 3 is a schematic view of 32 interrupt priority levels specified by Windows.

Detailed Description

The invention will be further described with reference to the drawings and examples.

It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Term interpretation:

windows driver Windows drives some important data structures including driver objects, device extensions.

The DRIVER OBJECT (DRIVER OBJECT) has a unique DRIVER OBJECT corresponding to it, which is created by the kernel's OBJECT manager at the time of DRIVER loading.

DEVICE OBJECTs (DEVICE OBJECT) each driver creates one or more DEVICE OBJECTs that may point to the DEVICE OBJECTs to form a DEVICE chain.

Device extensions, also called device contexts, device objects record information of "generic" devices, and device extensions can be used by programmers to define their own specific information, such as some objects, data. Its parent object is a device object.

Interrupt resource-for Windows drive, interrupt is a precious resource, and interrupt mode is adopted for hardware operation. If the peripheral wants to inform the CPU to read the data, the CPU applies for interruption, and then responds to the peripheral through an interruption processing program. The Windows has interrupt priority system, windows prescribes 32 interrupt priority levels, 0-2 is software interrupt, 3-31 is hardware interrupt, as shown in figure 3,0 Level corresponds to the passive_level,1 Level corresponds to the APC_level,2 Level corresponds to the dispatch_level, the. The processing principle is that 1) high priority interrupts can be prioritized, 2) if one interrupt request has been responded, other interrupt requests of the same level will be prohibited.

WDFWORKITEM objects-work item objects in windows drivers. When creating the work item, the event callback function address needs to be set.

Example 1

The embodiment provides a method for reading USB data to a disk under a KMDF framework, which is suitable for Windows kernel driving of the KMDF framework, as shown in figure 1, and adopts a mode of continuous reader, WDFWORKITEM first-level buffer memory and pre-allocation continuous non-paging second-level buffer memory of the KMDF framework to rapidly transmit the data to the disk at a driving layer by maintaining connection between a machine and USB equipment through an interrupt pipeline at a hardware layer.

The method for reading USB data to a disk under the KMDF frame provided in this embodiment, as shown in fig. 2, includes the following steps:

The method comprises the following specific steps of adopting a continuous reader to read data packets from a USB pipeline:

and 0, enumerating an interrupt terminal point pipeline handle of the USB device, and storing the interrupt terminal point pipeline handle in a device context structure of the driver.

Step 1, creating a continuous reader object, and setting configuration parameters (including a frame pipe object and USB pipe configuration information), letting a driver create a USB pipe, setting the transfer length (the maximum length (in bytes) of data that can be received from a device) of the reader to 2 ⁿ times the size of a transmission packet of the USB device, and setting n to 2 in this embodiment according to practical situations.

Step 2, starting the continuous reader, wherein a read request is arranged on the bus at any moment.

And 3, if the continuous reader is used for reading the data packets from the USB pipeline, marking the serial numbers on the data packets in a continuous reader reading completion routine, then pushing the data packets into a WDFWORKITEM object queue (WDFWORKITEM queue) according to the serial numbers, running the queue in an environment of a work item thread owned by an operating system, wherein the interrupt priority LEVEL of the thread is a passive_level, and a function with the interrupt LEVEL of the interrupt_level can preempt resources in the thread, improve the interrupt LEVEL operation and reduce the interrupt LEVEL after the operation is finished.

And 4, in WDFWORKITEM queue callback, pushing out the data packets in the WDFWORKITEM queue according to the sequence of thread processing completion.

And step 5, adjusting the sequence of the data packets according to the transmission protocol and the serial number, completing the splicing of the data packets, and storing the data packets into a non-paging cache. For example, 5 packets are received, and the sequence numbers are 1001, 1002, 1004, 1003, 1005, respectively. Since 1003 completes the process later than 1004, the order of pushing out follows 1004. Here, the order of the packets needs to be adjusted to 1001, 1002, 1003, 1004, 1005 according to the sequence number.

And 6, judging whether the storage condition is met, if so, entering the step 7, and if not, entering the step 3, and continuing to read the data packet.

And 7, writing the cache data into a disk, and emptying the cache.

And 8, judging whether the transmission is finished, if not, entering the step 3, continuing to read the data packet, and if the data packet with the end mark is received, entering the step 9.

And 9, ending the flow.

The method for reading USB data to the disk under the KMDF framework provided by the embodiment is characterized in that the work of writing the disk is completed in a driving layer, unnecessary memory copying is reduced, a WDFWORKITEM-to-data packet storage mechanism is used in a read completion routine, a WDFWORKITEM-queue-push data packet is stored in a high-speed cache mechanism, flexibility of processing the data packet after reading USB equipment data is improved by using a WDFWORKITEM queue, a function with a callback interrupt priority Level of WDFWORKITEM can be operated to be smaller than or equal to a dispatch_level without considering the limit of the interrupt priority Level, batch data is cached by using non-paging continuous memory fragments, batch data is written into the disk once, I/O operation is reduced, and the frequency of occupying I/O interrupt resources is reduced.

Example two

The embodiment provides a system for reading USB data to a disk under a KMDF framework, which is characterized in that a machine is connected with USB equipment through an interrupt pipeline in a hardware layer, and the data is quickly transmitted to the disk in a drive layer by adopting a continuous reader, WDFWORKITEM first-level cache and a preassigned continuous non-paging second-level cache of the KMDF framework. The device specifically comprises the following modules:

The data reading module is configured to read data packets from the USB pipeline, mark serial numbers on the data packets and push the data packets into the WDFWORKITEM queue, wherein the WDFWORKITEM queue operates in an environment of one workitem thread owned by an operating system, the interrupt priority LEVEL of the workitem thread is a passive_level, and a function with the interrupt LEVEL of the interrupt_level can preempt resources in the workitem thread, improve the interrupt LEVEL operation and reduce the interrupt LEVEL after the operation is finished;

The data packets are read from the USB pipe using a continuous reader.

The continuous reader is used for reading data packets from the USB pipeline, specifically:

The storage condition is specifically that the buffer is full or one file is transmitted completely.

The judging method of whether the transmission is finished or not is that if the received data packet band end identification is received, the transmission is finished.

The system for reading USB data to a disk under a KMDF framework provided by the embodiment is used for completing the work of writing the disk in a driving layer, reducing unnecessary memory copying, aiming at a storage mechanism of WDFWORKITEM pairs of data packets in a read completion routine, a mechanism of storing the data packets pushed out by a WDFWORKITEM queue into a cache, improving the flexibility of processing the data packets after reading USB equipment data by using a WDFWORKITEM queue, enabling a callback interrupt priority Level of the WDFWORKITEM queue to run a function smaller than or equal to a dispatch_level without considering the limit of the interrupt priority Level, caching batch data by using non-paging continuous memory fragments, writing the batch data into the disk once, reducing I/O operation and reducing the frequency of occupying I/O interrupt resources.

It should be noted that, each module in the embodiment corresponds to each step in the first embodiment one to one, and the implementation process is the same, which is not described here.

Example III

The present embodiment provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of a method of reading USB data to a disk under a KMDF framework as described in the above embodiment.

Example IV

The present embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the steps in a method for reading USB data to a disk under a KMDF frame according to the foregoing embodiment when executing the program.

It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disc, a Read-Only Memory (ROM), a Random access Memory (Random AccessMemory, RAM), or the like.

The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for reading USB data to disk under KMDF framework, comprising:

2. The method of claim 1, wherein the WDFWORKITEM queue runs in a workitem thread environment owned by an operating system, the workitem thread has an interrupt priority of a passive_ LEVEL, WDFWORKITEM queue callback interrupt priority that can run a function equal to or less than a dispatch_level, a 0 Level corresponds to the passive_level, a1 Level corresponds to apc_level, a2 Level corresponds to the dispatch_level, high priority interrupts are prioritized, the function with the interrupt priority of the dispatch_level preempting resources in the workitem thread, the interrupt priority is increased, and the interrupt priority is decreased after the operation is completed.

3. The method for reading USB data to disk under KMDF framework of claim 1, wherein the continuous reader is used to read data packets from USB pipe, specifically:

4. The method for reading USB data to disk under KMDF framework according to claim 1, wherein the storage condition is buffer fullness or a file transfer completion.

5. The method for reading USB data to a disk under a KMDF framework according to claim 1, wherein the method for determining whether transmission is completed is that if the received packet end identifier is received, the transmission is completed.

6. A system for reading USB data to disk under KMDF framework, comprising:

7. The system for reading USB data to disk under KMDF framework of claim 6, wherein the WDFWORKITEM queue runs in an environment of a workitem thread owned by an operating system, the workitem thread has an interrupt priority LEVEL of passive_level, and a function with an interrupt LEVEL of interrupt_level can preempt resources in the workitem thread, increase interrupt LEVEL, and run with the interrupt LEVEL reduced after running.

8. The system for reading USB data to disk under KMDF framework of claim 6, wherein the continuous reader is used to read packets from USB pipes, in particular:

9. A computer readable storage medium having stored thereon a computer program, which when executed by a processor performs the steps of a method of reading USB data to disk under a KMDF framework according to any of claims 1 to 5.

10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor, when executing the program, performs the steps of a method of reading USB data to disk under the KMDF framework of any of claims 1 to 5.