CN114900699A - Video codec card virtualization method, device, storage medium and terminal - Google Patents

Abstract

The invention provides a virtualization method, a virtualization device, a storage medium and a virtualization terminal for a video coding and decoding card, wherein the method comprises the following steps: receiving a video coding and decoding instruction, wherein the video coding and decoding instruction comprises priority information; creating a virtual video coding and decoding card, wherein the virtual video coding and decoding card generates a video coding and decoding request based on the video coding and decoding instruction package; and sending the video coding and decoding request to a physical video coding and decoding card so that the physical video coding and decoding card can execute video coding and decoding operation based on the priority information. The response granularity of the video coding and decoding request is smaller than the switching granularity of the virtual equipment; the whole register is not required to be refreshed, so that the bandwidth of a bus interface is saved; the scheduling mechanism of the virtual video coding and decoding card is not needed to be realized, and the design of a host side driving program is simplified.

Description

Video codec card virtualization method, device, storage medium and terminal

technical field

The present invention relates to the field of video coding and decoding, in particular to a video coding and decoding card virtualization method, device, storage medium and terminal.

Background technique

With the rapid development of the Internet, the virtualization of various peripherals has an increasing demand in the server field. The virtualization of peripherals is of great significance to improve the utilization of hardware resources and meet the growing market demand. Among them, the field of video codec is an important application field of virtualization technology.

The existing video codec card virtualization technology mainly shares hardware devices through time-division multiplexing, and creates a corresponding virtual video codec card for each virtual machine. register. When the virtual machine sends commands and data to the physical video codec card, it first writes the address of the command and data to the virtual register of the virtual video codec card, and then the virtual video codec card scheduler selects a virtual video codec card. , and refresh all the virtual registers of the virtual video codec card to the real hardware device, that is, the physical video codec card, and the physical video codec card performs corresponding video codec operations.

The existing video codec card virtualization technology has the following defects: First, the scheduling algorithm needs to store multiple copies of virtual devices on the host side, and when there are many registers (VPU (Video Processing Unit, video processing unit) registers) As many as thousands), it takes up a lot of memory, and the refresh efficiency of the register is low; secondly, the design and implementation cost of the scheduling mechanism of the virtual device is relatively high.

SUMMARY OF THE INVENTION

In view of the shortcomings of the above-mentioned prior art, the object of the present invention is to provide a video codec card virtualization method, device, storage medium and terminal, which are used to solve the problem that the video codec card virtualization takes up a lot of memory and registers in the prior art. Refresh is inefficient and costly.

In order to achieve the above object and other related objects, a first aspect of the present invention provides a method for virtualizing a video codec card, which includes: receiving a video codec instruction, where the video codec instruction includes priority information; creating a virtual video codec; A decoding card, where the virtual video encoding and decoding card packages and generates a video encoding and decoding request based on the video encoding and decoding instructions; and sends the video encoding and decoding request to the physical video encoding and decoding card, so that the physical video encoding and decoding card can request the physical video encoding and decoding card based on the priority level information to perform video codec operations.

In some embodiments of the first aspect of the present invention, the physical video codec card includes a video codec unit, a microprocessing unit, a memory unit and a bus interface unit, and the video codec card virtualization method includes: a physical video The codec card receives the video codec request through the bus interface unit, and stores it in the memory unit; the firmware on the microprocessing unit parses the video codec request; the video codec unit is based on The parsed video codec request performs video codec operations.

In some embodiments of the first aspect of the present invention, the generation method of the video codec request includes: the host creates a virtual video codec card corresponding to the virtual machine based on the device virtualization infrastructure, and the virtual video codec The card does not retain all registers of the physical video codec card; the device virtualization infrastructure drives the virtual video codec card to generate the video codec request based on the received video codec instruction.

In some embodiments of the first aspect of the present invention, the method for virtualizing a video codec includes: using a VFIO-MDEV (Virtual Function I/O Mediated devices) device virtualization infrastructure to create and virtualize a device through a driver layer of a host. The virtual video codec card corresponding to the machine; after receiving the video codec instruction sent by the virtual machine, the virtual video codec card is driven by the VFIO-MDEV device virtualization infrastructure to generate the video codec request.

In some embodiments of the first aspect of the present invention, the method for virtualizing a video codec includes: designing a quality of service policy based on the priority information to provide different user services.

In order to achieve the above object and other related objects, a second aspect of the present invention provides a video codec card virtualization device, which includes: an instruction receiving module for receiving a video codec instruction, wherein the video codec instruction includes a priority information; a codec request generation module, used to create a virtual video codec card, the virtual video codec card based on the video codec instruction packaging generates a video codec request; codec module, used for the video codec A request is sent to the physical video codec card for the physical video codec card to perform video codec operations based on the priority information.

In order to achieve the above object and other related objects, a third aspect of the present invention provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the video codec card virtualization method is realized. .

In order to achieve the above object and other related objects, a fourth aspect of the present invention provides an electronic terminal, comprising: a processor and a memory; the memory is used for storing a computer program, and the processor is used for executing the computer program stored in the memory , so that the terminal executes the video codec card virtualization method.

As described above, the present invention proposes a method, device, storage medium and terminal for virtualizing a video codec card, which have the following beneficial effects: overcoming the defects of the traditional video codec virtualization method of time-division multiplexing and sharing hardware devices, and proposes A new idea that can utilize hardware resources more efficiently. It generates a request containing priority information by packaging video codec instructions, that is, optimizing the time-sharing method to a priority request queue method, and realizes the virtualization of the video codec card. . In the present invention, the response granularity of the video codec request (v-request) is smaller than the switching granularity of the virtual device; there is no need to refresh all registers, which saves the bus interface bandwidth; there is no need to implement the scheduling mechanism of the virtual video codec card, which simplifies the host side. Driver design; further, design a quality of service policy (QoS, Quality of Service) to provide different user services and improve user experience; analyze the video codec request through the firmware of the physical video codec card, can Better protect the security of enterprise information (such as VPU IP information); make full use of the microprocessor resources on the physical video codec card.

Description of drawings

FIG. 1 is a schematic flowchart of a method for virtualizing a video codec card according to an embodiment of the present invention.

FIG. 2 is a structural block diagram of a video codec card according to an embodiment of the present invention.

FIG. 3 is a schematic diagram illustrating a working principle of a method for virtualizing a video codec card based on time division multiplexing according to an embodiment of the present invention.

FIG. 4 is a schematic diagram illustrating a working principle of a method for virtualizing a video codec card based on a priority request queue according to an embodiment of the present invention.

FIG. 5 is a schematic structural diagram of a device for virtualizing a video codec card according to an embodiment of the present invention.

FIG. 6 is a schematic structural diagram of an electronic terminal according to an embodiment of the present invention.

Detailed ways

The embodiments of the present invention are described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other under the condition of no conflict.

It should be noted that, in the following description, reference is made to the accompanying drawings, which describe several embodiments of the present invention. It is to be understood that other embodiments may be utilized and mechanical, structural, electrical, as well as operational changes may be made without departing from the spirit and scope of the present invention. The following detailed description should not be considered limiting, and the scope of embodiments of the present invention is limited only by the claims of the issued patent. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the present invention. Spatially related terms, such as "upper," "lower," "left," "right," "below," "below," "lower," "above," "upper," etc., may be used in the text for ease of description The relationship of one element or feature shown in the figures to another element or feature.

In the present invention, unless otherwise expressly specified and limited, terms such as "installation", "connection", "connection", "fixing", "retaining" and the like should be understood in a broad sense, for example, it may be a fixed connection or a It is a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal communication between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

Also, as used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context dictates otherwise. It should be further understood that the terms "comprising", "including" indicate the presence of a stated feature, operation, element, component, item, kind, and/or group, but do not exclude one or more other features, operations, elements, components, The existence, appearance or addition of items, categories, and/or groups. The terms "or" and "and/or" as used herein are to be construed to be inclusive or to mean any one or any combination. Thus, "A, B or C" or "A, B and/or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C" . Exceptions to this definition arise only when combinations of elements, functions, or operations are inherently mutually exclusive in some way.

The purpose of the present invention is to provide a video codec card virtualization method, device, storage medium and terminal, which are used to solve the problems in the prior art that the video codec card virtualization occupies a lot of memory, the register refresh efficiency is low, and the cost is high. question.

In order to make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention are further described in detail through the following embodiments and in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. The following embodiments mainly take the Linux system and the device virtualization infrastructure VFIO-MDEV as an example to describe the present invention, but the method is not limited to the Linux system or the device virtualization infrastructure VFIO-MDEV.

Example 1

As shown in FIG. 1 , an embodiment of the present invention provides a schematic flowchart of a method for virtualizing a video codec card. The method specifically includes the following steps:

Step S11. Receive a video codec instruction, where the video codec instruction includes priority information. The video encoding and decoding instructions can be sent by the virtual machine client, and the data to be encoded and decoded or the data address information can be sent to the server at the same time. The priority information is information assigned to the client based on a preset priority rule.

Step S12. Create a virtual video codec card, and the virtual video codec card packages and generates a video codec request based on the video codec instruction. There is a mapping relationship between the virtual video codec card and the physical video codec card (real hardware device), and the virtual machine uses the corresponding hardware device by operating the virtual video codec card. And the video codec request generated by packaging includes priority information.

In a preferred implementation of this embodiment, the method of generating the video codec request includes: the host creates a virtual video codec card corresponding to the virtual machine based on the device virtualization infrastructure, and the virtual video codec card does not reserve All registers of the physical video codec card; the device virtualization infrastructure drives the virtual video codec card to generate the video codec request based on the received video codec instruction.

Taking the Linux system as an example, it includes a VFIO-MDEV device virtualization infrastructure; the video codec card virtualization method includes: the VFIO-MDEV device virtualization infrastructure creates a virtual video codec corresponding to the virtual machine through the driver layer of the host. A decoding card; after receiving the video encoding and decoding instruction sent by the virtual machine, the virtual video encoding and decoding card is driven by the VFIO-MDEV device virtualization infrastructure to generate the video encoding and decoding request.

Step S13. Send the video codec request to a physical video codec card, so that the physical video codec card can perform a video codec operation based on the priority information. The physical video codec generates a request queue based on the priority information in the request, that is, the priority request queue.

In a preferred implementation of this embodiment, the physical video codec card includes a video codec unit, a microprocessing unit, a memory unit and a bus interface unit, and the video codec card virtualization method includes: a physical video codec card The video codec request is received through the bus interface unit and stored in the memory unit; the firmware on the micro-processing unit parses the video codec request; the video codec unit is based on the parsed A video codec request performs a video codec operation.

As shown in FIG. 2 , as an optional implementation manner, an embodiment of the present invention provides a structural block diagram of a video codec card, where the video codec card includes a video codec unit 21 (preferably a VPU), a memory unit 22 (preferably a VPU) DDR (Double Rate Synchronous Dynamic Random Access Memory)), a microprocessing unit 23 and a bus interface unit 24 (preferably a PCIE bus (peripheral component interconnect express high-speed serial computer expansion bus)).

The video codec card is communicated and connected to the host through the PCIE slot of the server host, and the server host runs a driver of the video codec card to drive the video codec card to work. Its working principle is described as follows: the server host sends the commands and data to be encoded and decoded to the register and memory unit 22 of the video encoding and decoding unit 21 through the bus interface unit 24, and the video encoding and decoding unit 21 performs the encoding and decoding operations to obtain processed data. Retrieve the processed data from the video codec. The micro-processing unit 23 on the video codec card is mainly used to detect the working state of the card.

In a preferred implementation of this embodiment, the method for virtualizing a video codec includes: designing a quality of service (QoS, Quality of Service) policy based on the priority information to provide different user services, that is, for different users Provide corresponding services. QoS is an agreement on the quality of information transmission and sharing between the network and users and between users who communicate with each other on the network. When the network is overloaded or congested, QoS can ensure that important traffic is not delayed or discarded, while ensuring the high efficiency of the network. run.

This embodiment respectively provides a schematic diagram of the working principle of a method for virtualizing a video codec card based on time-division multiplexing ( FIG. 3 ) and a schematic diagram of the working principle of a method for virtualizing a video codec card based on a priority request queue ( FIG. 3 ) ( Figure 4).

As shown in FIG. 3, in the video codec card virtualization method based on time division multiplexing, the virtual machine (VM) corresponds to the virtual video codec card (Virtual Video Card) in the physical host (Host) one-to-one, and the virtual The video codec card includes virtual registers (V-registers, which simulate the registers of the physical video codec card). The workflow is as follows: VFIO-MDEV creates a virtual video codec card in the driver layer of the host, and the virtual machine writes the video codec instruction and the address of the data to be processed to the virtual register of the virtual video codec card; the virtual codec card The scheduler selects a virtual codec card, and refreshes all the virtual registers of the selected virtual codec card to the registers (registers) of the physical video codec card (HW Video Card), so that the physical video codec card executes the corresponding video Codec operation.

As shown in FIG. 4 , in the video codec card virtualization method based on the priority request queue, the virtual machine (VM) corresponds to the virtual video codec card (Virtual Video Card) in the physical host (Host) one-to-one, and the virtual Video codec cards do not include virtual registers. When the virtual machine operates the virtual video codec card, the VFIO-MDEV driver on the host generates a video codec request (v-request) and sends it to the physical video codec card; the physical video codec card generates a video codec request queue (Request -queues), the firmware on it responds to the v-request according to the priority and operates the video codec unit (VPU) to perform the corresponding video codec operation.

In some embodiments, the video codec card virtualization method can be applied to a controller, and the electronic control unit is, for example, an ARM (Advanced RISC Machines) controller, an FPGA (Field Programmable Gate Array) controller, a SoC (System on Chip) controller, DSP (Digital Signal Processing) controller, or MCU (Microcontroller Unit) controller, etc. In some embodiments, the video codec virtualization method is also applicable to include memory, memory controller, one or more processing units (CPU), peripheral interface, RF circuit, audio circuit, speaker, microphone, Computers with components such as input/output (I/O) subsystems, displays, other output or control devices, and external ports; such computers include, but are not limited to, such as desktop computers, laptops, tablets, smartphones, smart TVs , Personal Digital Assistant (Personal Digital Assistant, PDA) and other personal computers. In other embodiments, the video codec card virtualization method can also be applied to a server, and the server can be arranged on one or more physical servers according to various factors such as function and load, or can be distributed or Centralized server cluster composition.

Embodiment 2

As shown in FIG. 5, an embodiment of the present invention provides a schematic structural diagram of a video codec card virtualization device, which includes: an instruction receiving module 51, configured to receive a video codec instruction, where the video codec instruction includes priority information The codec request generation module 52 is used to create a virtual video codec card, and the virtual video codec card is packaged and generated based on the video codec instruction to generate a video codec request; the codec module 53 is used to encode and decode the video. The decoding request is sent to the physical video codec card for the physical video codec card to perform video codec operations based on the priority information.

It should be noted that the modules provided in this embodiment are similar to the methods and implementation manners provided above, and thus are not described again. In addition, it should be noted that it should be understood that the division of each module of the above apparatus is only a division of logical functions, and may be fully or partially integrated into a physical entity in actual implementation, or may be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also all be implemented in hardware; some modules can also be implemented in the form of calling software through processing elements, and some modules can be implemented in hardware. For example, the codec module 53 can be a separately established processing element, or can be integrated into a certain chip of the above-mentioned device to realize, in addition, it can also be stored in the memory of the above-mentioned device in the form of a program code, and it can be generated by one of the above-mentioned devices. The processing element calls and executes the functions of the above codec module 53 . The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, and can also be implemented independently. The processing element described here may be an integrated circuit with signal processing capability. In the implementation process, each step of the above-mentioned method or each of the above-mentioned modules can be completed by an integrated logic circuit of hardware in the processor element or an instruction in the form of software.

For example, the above modules may be one or more integrated circuits configured to implement the above method, such as: one or more specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), or one or more digital signal processors (digital signal processor, DSP for short), or one or more Field Programmable Gate Array (Field Programmable Gate Array, FPGA for short), etc. For another example, when one of the above modules is implemented in the form of processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (Central Processing Unit, CPU for short) or other processors that can call program codes. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC for short).

Embodiment 3

As shown in FIG. 6 , an embodiment of the present invention provides a schematic structural diagram of an electronic terminal. The electronic terminal provided in this embodiment includes: a processor 61, a memory 62, and a communicator 63; the memory 62 is connected to the processor 61 and the communicator 63 through a system bus and communicates with each other, and the memory 62 is used to store computer programs, The communicator 63 is used to communicate with other devices, and the processor 61 is used to run a computer program, so that the electronic terminal executes each step of the above video codec card virtualization method.

The system bus mentioned above may be a Peripheral Component Interconnect (PCI for short) bus or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, EISA for short) bus or the like. The system bus can be divided into address bus, data bus, control bus and so on. For ease of presentation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize the communication between the database access device and other devices (eg client, read-write library and read-only library). The memory may include random access memory (Random Access Memory, RAM for short), and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

The above-mentioned processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, referred to as CPU), a network processor (Network Processor, referred to as NP), etc.; may also be a digital signal processor (Digital Signal Processing, referred to as DSP) , Application Specific Integrated Circuit (ASIC for short), Field-Programmable Gate Array (FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

Embodiment 4

An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the foregoing method for virtualizing a video codec card.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by hardware related to computer programs. The aforementioned computer program may be stored in a computer-readable storage medium. When the program is executed, the steps including the above method embodiments are executed; and the foregoing storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

In summary, the present invention provides a method, device, storage medium and terminal for virtualizing a video codec card, by packaging video codec instructions to generate a request including priority information, that is, optimizing the time-sharing method into a priority request Queue mode, realize the virtualization of video codec card. The response granularity of the video codec request in the present invention is smaller than the switching granularity of the virtual device; it is not necessary to simulate all the registers, and it is not necessary to refresh all the registers, which saves the bus interface bandwidth and host storage space; there is no need to implement the scheduling mechanism of the virtual video codec card, which simplifies The design of the host-side driver saves manpower; further, the service quality policy is designed to provide different services for different users and improve the user experience; the firmware of the physical video codec card parses the video codec request, which can better The security of enterprise information is fully protected; the microprocessor resources on the physical video codec card are fully utilized. Therefore, the present invention effectively overcomes the problems in the prior art that the virtualization of the video codec card occupies a lot of memory, the register refresh efficiency is low, and the cost is high, and it has high industrial utilization value.

The above-mentioned embodiments merely illustrate the principles and effects of the present invention, but are not intended to limit the present invention. Anyone skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed in the present invention should still be covered by the claims of the present invention.

Claims (8)

1. A virtualization method for a video codec card is characterized by comprising the following steps:

receiving a video coding and decoding instruction, wherein the video coding and decoding instruction comprises priority information;

creating a virtual video coding and decoding card, wherein the virtual video coding and decoding card generates a video coding and decoding request based on the video coding and decoding instruction package;

and sending the video coding and decoding request to a physical video coding and decoding card so that the physical video coding and decoding card can execute video coding and decoding operation based on the priority information.

2. The method of claim 1, wherein the physical video codec card comprises a video codec unit, a microprocessor unit, a memory unit, and a bus interface unit, and the method comprises:

the physical video coding and decoding card receives the video coding and decoding request through the bus interface unit and stores the video coding and decoding request to the memory unit;

the firmware on the micro-processing unit analyzes the video coding and decoding request;

and the video coding and decoding unit executes video coding and decoding operation based on the analyzed video coding and decoding request.

3. The method of claim 1, wherein the video codec card virtualization method is characterized in that the video codec request generation mode comprises:

the method comprises the steps that a host computer creates a virtual video coding and decoding card corresponding to a virtual machine based on an equipment virtualization infrastructure, wherein the virtual video coding and decoding card does not reserve all registers of a physical video coding and decoding card;

the device virtualization infrastructure drives the virtual video codec card to generate the video codec request based on the received video codec instruction.

4. The video codec card virtualization method according to claim 1, comprising:

creating a virtual video coding and decoding card corresponding to the virtual machine through a driving layer of the host by using a VFIO-MDEV equipment virtualization infrastructure;

and after the virtual video coding and decoding card receives the video coding and decoding command sent by the virtual machine, the video coding and decoding request is generated by the driving of the VFIO-MDEV equipment virtualization infrastructure.

5. The video codec card virtualization method according to claim 1, comprising:

and designing a service quality strategy based on the priority information to provide different user services.

6. A video codec card virtualization apparatus, comprising:

the video coding and decoding device comprises an instruction receiving module, a video coding and decoding module and a video coding and decoding module, wherein the instruction receiving module is used for receiving a video coding and decoding instruction which comprises priority information;

the encoding and decoding request generating module is used for creating a virtual video encoding and decoding card, and the virtual video encoding and decoding card generates a video encoding and decoding request based on the video encoding and decoding instruction package;

and the coding and decoding module is used for sending the video coding and decoding request to a physical video coding and decoding card so that the physical video coding and decoding card can execute video coding and decoding operation based on the priority information.

7. A computer-readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, implementing the video codec card virtualization method according to any one of claims 1 to 5.

8. An electronic terminal, comprising: a processor and a memory;

the memory is used for storing a computer program, and the processor is used for executing the computer program stored by the memory to enable the terminal to execute the video codec card virtualization method according to any one of claims 1 to 5.

Images