[go: up one dir, main page]

WO2018176303A1 - Procédé, système et dispositif d'émission et de réception de vidéo, et véhicule aérien sans pilote - Google Patents

Procédé, système et dispositif d'émission et de réception de vidéo, et véhicule aérien sans pilote Download PDF

Info

Publication number
WO2018176303A1
WO2018176303A1 PCT/CN2017/078728 CN2017078728W WO2018176303A1 WO 2018176303 A1 WO2018176303 A1 WO 2018176303A1 CN 2017078728 W CN2017078728 W CN 2017078728W WO 2018176303 A1 WO2018176303 A1 WO 2018176303A1
Authority
WO
WIPO (PCT)
Prior art keywords
sub
video data
image
data units
encoded
Prior art date
Application number
PCT/CN2017/078728
Other languages
English (en)
Chinese (zh)
Inventor
朱磊
崔浩
龚明
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2017/078728 priority Critical patent/WO2018176303A1/fr
Priority to CN201780005017.8A priority patent/CN108496369A/zh
Publication of WO2018176303A1 publication Critical patent/WO2018176303A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/164—Feedback from the receiver or from the transmission channel
    • H04N19/166—Feedback from the receiver or from the transmission channel concerning the amount of transmission errors, e.g. bit error rate [BER]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2385—Channel allocation; Bandwidth allocation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display

Definitions

  • Embodiments of the present invention relate to the field of image processing, and in particular, to a video transmission and reception method, system, device, and unmanned aerial vehicle.
  • the unmanned aerial vehicle is equipped with a photographing device, and the photographing device can perform aerial photography, and the aerial video is transmitted to the ground receiving device such as a user terminal, a remote controller, and the like through a communication system of the unmanned aerial vehicle.
  • the code stream data size corresponding to each frame of image data will change in real time (ie, the source will change in real time); in addition, it will be subject to unmanned aerial vehicles and receiving equipment.
  • the distance between the distance, the relative position, the presence of occlusion, and the presence of electromagnetic interference, etc., the channel bandwidth between the UAV and the receiving device will also change in real time (ie, the channel will change in real time), and the source channel changes to each other.
  • the transmission and reception modes of frame-level image data are relatively fixed, which is difficult to adapt to real-time changing source channels, and lacks effective image transmission and image receiving methods, which may result in image transmission delay due to real-time source and channel mismatch problems. shake.
  • Embodiments of the present invention provide a video transmission and reception method, system, device, and an unmanned aerial vehicle to effectively reduce transmission delay jitter of video data.
  • An aspect of the embodiments of the present invention provides a video transmission method, including:
  • each sub-video data unit includes one or more sub-images
  • One or more encoded sub-video data units are selected and transmitted based on one or more characteristics of the channel, and one or more characteristics of the sub-video data unit.
  • Another aspect of the embodiments of the present invention provides a video receiving method, including:
  • the video data includes one or more image frames
  • the sub-video data unit includes a plurality of sub-decompositions obtained by decomposing each of the image frames At least one sub-image in the image.
  • One or more imaging devices configured to acquire video data
  • One or more processors on the movable object working alone or in concert, the processor being configured to:
  • each sub-video data unit includes one or more sub-images
  • One or more encoded sub-video data units are selected and transmitted based on one or more characteristics of the channel, and one or more characteristics of the sub-video data unit.
  • a receiving device including: a communication interface, one or more processors, working alone or in cooperation, and the communication interface is in communication with the processor;
  • the communication interface is configured to receive a plurality of encoded sub-video data units
  • the one or more processors are configured to: control a decoder to decode the plurality of encoded sub-video data units; reconstruct the video data according to the decoded sub-video data unit, wherein the video data includes one Or a plurality of image frames, the sub-video data unit including at least one of the plurality of sub-images obtained by decomposing each of the image frames.
  • a power system mounted to the fuselage for providing flight power
  • the video transmission and receiving method, the system, the device and the unmanned aerial vehicle provided by the embodiment are respectively decomposed into a plurality of sub-video data units by the video data, and respectively, and the plurality of sub-video data units are respectively Encoding, selecting one or more encoded sub-video data units and said selected sub-video data units according to channel characteristics and characteristics of sub-video data units, such that one or more encoded sub-elements are selected
  • the video data unit conforms to the channel characteristics, and when the selected one or more encoded sub-video data units are transmitted on the channel matched with the same, the problem of mismatch between the source and the channel can be effectively solved, and the video data can be effectively reduced. Transmission delay jitter caused by source channel mismatch problem.
  • 1 is a schematic diagram of transmission delay jitter according to an embodiment of the present invention.
  • FIG. 2 is a flowchart of a video transmission method according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a sub video data unit according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a sub video data unit according to another embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a sub video data unit according to another embodiment of the present invention.
  • FIG. 6 is a flowchart of a video transmission method according to another embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of a frame image according to an embodiment of the present disclosure.
  • FIG. 8 is a coefficient image of a frame image after Hadamard transform according to an embodiment of the present invention.
  • FIG. 9 is a schematic diagram of spatial transformation decomposition according to an embodiment of the present invention.
  • FIG. 10 is a schematic diagram of spatial downsampling decomposition according to an embodiment of the present invention.
  • FIG. 11 is a flowchart of a video receiving method according to an embodiment of the present invention.
  • FIG. 12 is a schematic diagram of a decoded sub-image according to an embodiment of the present invention.
  • FIG. 13 is a schematic diagram of a decoded sub-image according to another embodiment of the present invention.
  • FIG. 14 is a schematic diagram of reconstructing an original image according to an embodiment of the present invention.
  • FIG. 15 is a schematic diagram of reconstructing an original image according to another embodiment of the present invention.
  • FIG. 16 is a structural diagram of a video transmission system according to an embodiment of the present invention.
  • FIG. 17 is a structural diagram of a receiving device according to an embodiment of the present invention.
  • FIG. 18 is a structural diagram of an unmanned aerial vehicle according to an embodiment of the present invention.
  • a component when referred to as being "fixed” to another component, it can be directly on the other component or the component can be present. When a component is considered to "connect” another component, it can be directly connected to another component or possibly a central component.
  • the stability of the transmission delay of image data is an important indicator to measure the performance of the image transmission system.
  • the image data transmission delay is the basic condition for ensuring the smooth display of the video image at the receiving end.
  • the source and channel are used.
  • the real-time change will cause the jitter of the transmission delay between frames and frames, and reduce the performance of the image transmission system.
  • the source data changes and channel changes are taken as examples to describe the frame-level image data transmission delay in detail. Jitter problem.
  • Figure 1 includes scenario 1 and scenario 2.
  • scenario 1 the bandwidth of the channel between the sender and the receiver remains stable.
  • the camera at the transmitting end suddenly moves, or the object within the camera shooting range suddenly moves rapidly, for example, at a certain moment, the camera's subject is a blue sky at the next moment.
  • the camera suddenly turns to shoot a colorful hot air balloon flying in the sky, causing the corresponding stream data size after frame 4 encoding to increase to twice the size of the corresponding stream data after the frame 3 encoding, that is, the sudden change of the source
  • the transmission delay of frame 4 will become twice the transmission delay of frame 3.
  • the code stream data corresponding to each frame image is basically stable, that is, the source remains stable.
  • the channel bandwidth corresponding to frame 4 suddenly drops to half of the channel bandwidth corresponding to frame 3.
  • the UAV suddenly approaches the nearby wireless communication base station.
  • the wireless communication base station will affect the transmission channel of the UAV, that is, the channel changes, and the bandwidth of the channel decreases to half of the original bandwidth.
  • the transmission delay of frame 4 will also become twice the transmission delay of frame 3.
  • the method for video transmission provided by the embodiment of the present invention is described in detail below with reference to FIG. 1 to effectively reduce the transmission delay jitter of the video data.
  • FIG. 2 is the basis A flowchart of a video transmission method provided by an embodiment of the invention. As shown in FIG. 2, the method in this embodiment may include:
  • Step S201 the video data is decomposed into a plurality of sub-video data units, wherein each sub-video data unit includes one or more sub-images.
  • the execution body of the embodiment may be a processor, a controller, or a general-purpose processor having an image processing function, and is not specifically limited herein.
  • an image processor is taken as an example to introduce the principle of a video transmission method.
  • the image processor acquires video data captured by a shooting device mounted on an unmanned aerial vehicle in real time, and the video data may include one frame image or continuous multi-frame.
  • the image processor may decompose the video data into a plurality of sub-video data units.
  • the embodiment does not limit the number of sub-video data units obtained by decomposing the video data, and each sub-video data unit includes one or more sub-images.
  • a feasible implementation manner of decomposing the video data into a plurality of sub-video data units is: decomposing each image frame included in the video data, that is, each frame image into a plurality of sub-images, and selecting at least one sub-image of each image frame.
  • the selected sub-image constitutes a sub-video data unit, that is, the sub-video data unit includes at least one of a plurality of sub-images obtained by decomposing each image frame in the video data.
  • This embodiment does not limit the number of image frames included in one video data.
  • the video data includes 6 image frames, that is, 6 frames, and further, in other embodiments, the video The number of image frames included in the data may also be other values.
  • the video data includes six image frames, which are frame 1, frame 2, frame 3, frame 4, frame 5, frame 6, and frame 1, frame 2, frame 3, frame 4, and frame 5.
  • the frame 6 is separately decomposed.
  • the number of sub-images into which each image frame is decomposed is not limited.
  • each image frame is decomposed into four sub-images, which is only schematically illustrated herein.
  • the number of sub-images obtained after each image frame is decomposed may also be other values.
  • Each of the sub-video data units includes at least one of the four sub-images corresponding to each of the image frames after each of the six image frames is decomposed.
  • each sub-video data unit includes one of the four sub-images corresponding to each of the six image frames.
  • the sub-video data unit 310 includes one sub-image 11 of the frame 1.
  • the sub-video data unit 410 includes two sub-images 11, 12 of the frame 1, and two sub-images 21, 22 of the frame 2.
  • sub-video data unit 420 includes one sub-image 13 of frame 1, frame 2
  • two sub-images 52, 53 of frame 5 two sub-images 62, 63 of frame 6
  • sub-video data unit 430 includes One sub-image of each of the six image frames.
  • each sub-video data unit do not overlap.
  • the manner in which at least one of the plurality of sub-images corresponding to each of the plurality of image frames is combined to form a sub-video data unit may also have other combinations, which are not enumerated here.
  • the video data may also include only one image frame, that is, one frame image.
  • 50 indicates an image frame included in the video data, and the image frame 50 is decomposed. This embodiment does not limit one image frame.
  • the number of sub-images obtained after the decomposition is selected, and the image frame 50 is optionally decomposed into four sub-images, such as the sub-image 11, the sub-image 12, the sub-image 13, and the sub-image 14 as shown in FIG.
  • Sub-picture 11, sub-image 12, sub-image 13, and sub-image 14 can be divided into the following achievable ways:
  • each sub-video data unit includes one sub-image, such as sub-video data unit 510, sub-video data unit 520, sub-video data unit 530, and sub-video data unit 540 as shown in FIG.
  • each sub-video data unit includes two sub-images. This embodiment does not limit the combination of two sub-images included in one sub-video data unit.
  • each sub-video data unit includes a different number of sub-images, such as sub-video data unit 570 and sub-video data unit 580 as shown in FIG. 5, wherein sub-video data unit 570 includes three sub-pictures.
  • Combination, optional, sub-video data unit 570 includes sub-image 11, sub-image 12, sub-picture
  • the sub-video data unit 580 includes a sub-image 14.
  • Step S202 encoding the plurality of sub-video data units separately.
  • the image processor separately encodes each of the plurality of sub-video data units by using each sub-video data unit as a coding unit, and obtains a plurality of code stream data after encoding, and optionally encodes one sub-video data unit. Then, a code stream data is obtained, where the coding includes source coding and/or channel coding, and the manner of source coding may include H.263, H.264, H.265, MPEG4, etc., and the channel coding method may include correcting
  • the error coding type may include an RS code, that is, a Reed-Solomon code, a convolutional code, a Turbo code, a Polar code, an interleaving code, a pseudo random sequence scrambling code, and the like.
  • Step S203 Select one or more encoded sub-video data units based on one or more characteristics of the channel and one or more characteristics of the sub-video data unit.
  • one or more characteristics of the channel include at least a bandwidth.
  • one or more characteristics of the channel include at least one of: noise, interference, signal to noise ratio, bit error rate, fading rate, bandwidth.
  • the one or more characteristics of the sub-video data unit include: a code stream data size encoded by the sub-video data unit, or an energy concentration of the sub-video data unit.
  • the image processor selects one or more of the plurality of encoded sub-video data units for transmission over the wireless channel based on one or more characteristics of the current wireless channel and one or more characteristics of the sub-video data unit, eg, transmitting
  • the receiving device can be a remote control smartphone, a tablet, a ground control station, a laptop, a watch, a wristband, and the like, and combinations thereof.
  • selecting one or more encoded sub-video data units can be implemented in the following feasible ways:
  • a first feasible way is to select one or more encoded sub-video data units such that the total code stream data size of the one or more encoded sub-video data units matches the channel bandwidth.
  • the image processor decomposes the video data to obtain four sub-video data units 310-340 as shown in FIG. 3, and encodes the four sub-video data units separately to obtain the size of the code stream data as S0, S1, S2, and S3.
  • the bandwidth of the current wireless channel is T
  • the image processor may select one or more code stream data from the four code stream data according to the bandwidth T of the wireless channel, and the selection may be based on: one or more selected codes.
  • the total size of the stream is as close as possible to the bandwidth T of the wireless channel.
  • one or more of the plurality of sub-video data units are selected for combination, so that the total code stream data size of the combined sub-video data unit is as close as possible to the bandwidth of the wireless channel, and the wireless channel can transmit the code stream with the current bandwidth.
  • the child video data unit is as large as possible.
  • the sum of S0, S1, S2, and S3 is less than or equal to the current bandwidth of the channel, and the encoded sub-video data unit 310, the sub-video data unit 320, the sub-video data unit 330, and the sub-video data unit 340 may be selected for transmission.
  • the three largest code stream data may be selected from S0, S1, S2, and S3, and the three largest code stream data are S0, S1, and S2, if S0, If the sum of S1 and S2 is less than T, the image processor may select the encoded sub-video data unit 310, the sub-video data unit 320, and the sub-video data unit 330 for transmission, and so on, when the sum of S0, S1, S2 is greater than T, other combinations of the code stream data that can be transmitted with the bandwidth of the current wireless channel and the largest sub-video data unit can be selected.
  • the second feasible way is that a plurality of sub-video data units are prioritized according to the energy concentration.
  • the one or more encoded sub-video data units are selected based on a priority of the sub-video data unit and a channel bandwidth.
  • the energy concentration of each of the plurality of sub-video data units may be the same or similar, it may be different. Therefore, in other embodiments, if the energy concentration of each sub-video data unit is different, it may be based on each The energy concentration of the sub-video data units prioritizes the plurality of sub-video data units. Alternatively, the greater the energy concentration, the higher the priority. For example, the image processor decomposes the video data to obtain four sub-video data units as shown in FIG. 3, 4 or 5, and the four sub-video data units are respectively recorded as sub-video data unit A, sub-video data unit B, and sub-video.
  • the data unit C, the sub-video data unit D, and the priority of the four sub-video data units are successively decremented, the code stream data size encoded by the sub-video data unit A is S0, and the code stream data encoded by the sub-video data unit B is sequentially obtained.
  • the size of the code stream data encoded by the sub video data unit C is S2
  • the size of the code stream data encoded by the sub video data unit D is S3.
  • the image processor needs to select and transmit the sub-video data unit according to the code stream data size and channel bandwidth of the encoded sub-video data unit, and in some cases, according to the priority and channel of the sub-video data unit.
  • the bandwidth selects the sub-video data unit and transmits it.
  • one or more encoded sub-video data units selected from the above four encoded sub-video data units may be selected according to one or more selected sub-views selected.
  • the total code stream data size of the frequency data unit is smaller than the channel bandwidth, and the higher priority sub video data unit is preferentially selected, thereby ensuring that the high priority sub video data unit is preferentially transmitted.
  • the encoded sub-video data unit A, sub-video data unit B, sub-video data unit C, and sub-video data unit D may be selected for transmission.
  • the image processor transmits the encoded sub-video data unit A, the encoded sub-video data unit B, and the encoded sub-video data unit C; If the sum of S0, S1, and S2 is greater than T, the first two sub-video data units A and sub-video data units B having higher priority are determined, and so on, to ensure that at least the sub-video data unit A having the highest priority is transmitted. .
  • the video data is decomposed into a plurality of sub-video data units, and the plurality of sub-video data units are separately encoded, and one or more encoded sub-video data units are selected according to channel characteristics and characteristics of the sub-video data units.
  • the selected one or more encoded sub-video data units conform to the channel characteristics, which can effectively solve the problem of mismatch between the source and the channel, and can effectively reduce the transmission delay of the video data due to the source channel mismatch problem. shake.
  • FIG. 6 is a flowchart of a video transmission method according to another embodiment of the present invention.
  • the video data includes one or more image frames.
  • the method in this embodiment may include:
  • Step S601 Decompose each of the one or more image frames in the video data into a plurality of sub-images.
  • the video data may include one frame image, and may also include consecutive multi-frame images.
  • This embodiment does not limit the number of pixels included in one frame image, and does not limit the pixels of each pixel. value.
  • the image processor decomposes the video data, specifically, each frame image of one or more frames in the video data may be decomposed into a plurality of sub-images, and the sub-video data unit includes each of the image frames. At least one of the plurality of sub-images obtained after the decomposition is performed.
  • each of the one or more image frames in the video data is decomposed into a plurality of sub-images.
  • a frame image included in the video data is taken as an example to introduce the frame image.
  • the process of spatial decomposition can be achieved in several possible ways:
  • the Fourier correlation transform or orthogonal transform is selected from a Hadamard transform, a discrete cosine transform, a discrete Fourier transform, a Walsh-Hadamard transform, a Haar transform, or an oblique transform.
  • This embodiment specifically takes the Hadamard transform as an example to introduce a process of spatially decomposing the frame image.
  • other spatial transforms may also be used to spatially decompose the frame image.
  • a schematic diagram of a frame image is used.
  • This embodiment does not limit the number of pixels included in a frame image, and a frame image includes 16 pixels.
  • P1-P16 represents 16 pixels.
  • the pixel value is spatially transformed and decomposed into pixel values of every four adjacent pixel points of the 16 pixel points, and is decomposed into four sub-images.
  • the following is schematically illustrated by Hadamard transform, and the specific spatial transformation decomposition process includes The following steps:
  • Step 1 Perform a Hadamard transform by using four adjacent pixels of the 16 pixels as a unit.
  • the conversion coefficients obtained by the Hadamard transform of P1, P2, P3, and P4 are H1, H2, and H3. H4, wherein the relationship between P1, P2, P3, P4 and H1, H2, H3, H4 satisfies the formulas (1), (2), (3), (4):
  • H1 contains the average energy of 4 pixels
  • H2 contains the average gradient of 4 pixels in the vertical direction
  • H3 contains 4 pixels.
  • H4 contains a cross gradient of 4 pixels, ie texture information. It can be seen that the energy concentration of H1 is the highest, the energy concentration of H2 and H3 is second, and the energy concentration of H4 is the smallest.
  • P1, P2, P3, and P4 with H1, H2, H3, and H4, H1 is essential. H1 is the most important, followed by H2 and H3, and finally H4 is important.
  • H1 is the most important
  • H2 and H3 are the second most important
  • H4 is the least important, that is, H1, H2, H3, and H4 are important.
  • Sex is decremented in turn.
  • the same Hadamard transform is performed on P5-P8 to obtain H5-H8, and the energy concentration of H5, H6, H7 and H8 is successively decreased, and the importance of H5, H6, H7 and H8 is successively decreased, and P9-P12 is successively performed.
  • the same Hadamard transform gets H9-H12, H9, The energy concentration of H10, H11 and H12 decreases in turn, and the importance of H9, H10, H11 and H12 decreases successively.
  • Step 2 Decompose the conversion coefficients obtained by the Hadamard transform into different sub-images.
  • This embodiment does not limit the number of sub-images obtained by spatially transforming each frame of the image, for example, the sub-images obtained after the decomposition.
  • the number is four, which is only a schematic description. In other embodiments, the number of sub-images obtained by spatially transforming and decomposing each frame of image may be other values.
  • H1 is assigned to the first sub-image
  • H2 is assigned to the second sub-image
  • H3 is assigned to the third sub-image
  • H4 is assigned to the fourth sub-image.
  • H5-H8 is decomposed into 4 sub-images
  • H9-H12 is decomposed into 4 sub-images
  • H13-H16 is decomposed into 4 sub-images
  • the decomposition result shown in FIG. 9 is obtained, wherein the sub-image 1 Concentrates the conversion coefficient with the highest energy concentration
  • sub-image 2 and sub-image 3 concentrate the conversion coefficient of energy concentration second
  • sub-image 4 concentrates the conversion coefficient with the smallest energy concentration, so sub-image 1 is the most important, sub-image 2.
  • the importance of sub-image 3 is second, sub-image 4 is of the least importance.
  • the resolution of each of the four sub-images after the spatial transformation decomposition is one quarter of the original image before the decomposition.
  • This embodiment does not limit the number of sub-images obtained by down-sampling the image space of each frame.
  • the number of sub-images obtained after decomposition is four, which is only schematically illustrated herein.
  • the number of sub-images obtained by down-sampling the image space of each frame may also be other values.
  • spatially down-sampling the pixel values of every four adjacent pixel points of the 16 pixel points is performed.
  • the specific spatial down-sampling decomposition process is: decomposing four pixel points in one unit into different sub-images by using four adjacent pixel points of 16 pixels as one unit.
  • P1 is decomposed into the first sub-image
  • P2 is decomposed into the second sub-image
  • P3 is decomposed into the third sub-image
  • P4 is decomposed into the fourth sub-image
  • P5-P8 is decomposed into 4 sub-images.
  • P9-P12 is decomposed into 4 sub-images
  • P13-P16 is decomposed into 4 sub-images, and the decomposition result as shown in FIG. 10 is obtained.
  • the resolution of each of the four sub-images after spatial downsampling is one quarter of the original image before the decomposition. Without loss of generality, it is assumed that the size of the original image before decomposition is W*H. If the original image is decomposed into four sub-images, the row number or column number of the pixel matrix corresponding to the original image or sub-image is counted from 0, then the first The sub-images may include pixels in the original image with coordinates (2i, 2j), where The second sub-image may include a pixel with coordinates (2i+1, 2j) in the original image, and the third sub-image may include a pixel with coordinates (2i, 2j+1) in the original image, and the fourth sub-image. The pixel in the original image with coordinates (2i+1, 2j+1) may be included.
  • Each of the one or more image frames in the video data can be decomposed into a plurality of sub-images according to any of the above-described decomposition modes, that is, spatial transformation or spatial downsampling.
  • one image frame is one frame image
  • a plurality of image frames are multi-frame images.
  • the video data includes one or more image frames, the sub-video data unit including at least one of a plurality of sub-images obtained by decomposing each of the image frames.
  • the image frame is decomposed in the manner shown in FIG. 9 or FIG. 10, and the sub-video data unit may include at least one of the plurality of sub-images obtained by decomposing the image frame, for example, one.
  • the sub-video data unit includes a sub-image, and each sub-image obtained after the decomposition is compressed and encoded to obtain encoded code stream data.
  • each image frame that is, each frame image is decomposed as shown in FIG. 9 or FIG. 10, for example, the video data includes 24 image frames, and if each image frame is decomposed For 4 sub-images, 24 consecutive image frames are decomposed to obtain 24*4 sub-images, and each sub-video data unit may include multiple sub-images of 24*4 sub-images, specifically, the sub-video data unit At least one of the plurality of sub-images corresponding to each image frame obtained by decomposing each of the image frames, the sub-video data unit may be four, wherein the sub-video data unit Each of them includes a sub-image of each of the 24 image frames.
  • the encoded code stream data is obtained by compressing and encoding each sub-video data unit.
  • each sub-image includes a portion of an image frame.
  • each sub-image includes one or more pixels of an image frame.
  • each sub-image includes one or more conversion coefficients of the image frame.
  • the receiving device reconstructs a frame of image as described in FIG. 7, in FIG. 10, the importance of the sub-image 1, the sub-image 2, the sub-image 3, and the sub-image 4 are the same.
  • the sub-image 1 is the most important, and the sub-image 2 and the sub-image 3 are the second most important, and the sub-image 4 has the lowest importance.
  • the plurality of sub-video data units are prioritized according to the energy concentration. Sort by level.
  • sub-image 1 concentrates the conversion coefficient with the highest energy concentration
  • sub-picture The image 2 and the sub-image 3 concentrate the conversion coefficient of the energy concentration second
  • sub-image 4 concentrates the conversion coefficient with the smallest energy concentration. If a sub-video data unit includes a sub-image, the sub-video data unit including the sub-image 1
  • the priority of the sub-video data unit including sub-image 2 or sub-image 3 is the highest, and the sub-video data unit including sub-image 4 has the lowest priority.
  • Step S602 encoding the plurality of sub-video data units separately.
  • a first possible way multiple sub-video data units are encoded by a plurality of separate encoders.
  • the plurality of sub-video data units are encoded in parallel by using the plurality of separate encoders; or the plurality of sub-video data units are encoded by using different video encoding rules; or, the same video encoding rule is adopted.
  • the plurality of sub-video data units are encoded.
  • a second possible way two or more of the plurality of sub-video data units are encoded by the same encoder.
  • a third possible way encoding at least one of the plurality of sub-video data units based on a motion compensated video compression standard.
  • a fourth feasible way compressing the plurality of sub-video data units according to different compression ratios.
  • the compression ratio is determined based on one or more characteristics of the sub-video data unit.
  • Step S603 Select one or more encoded sub-video data units based on one or more characteristics of the channel and one or more characteristics of the sub-video data unit.
  • one sub-video data unit includes one sub-image, as shown in FIG. 9, the sub-image 1 is the most important, the sub-image 2, the sub-image 3 are of the second most important, and the sub-image 4 has the lowest importance. Then, the sub-video data unit including the sub-image 1 has the highest priority, and the sub-video data unit including the sub-image 2 or the sub-image 3 has the lower priority, and the sub-video data unit including the sub-image 4 has the lowest priority.
  • the code stream data size encoded by the sub video data unit A including the sub image 1 is S0
  • the code stream data size encoded by the sub video data unit B including the sub image 2 is S1
  • the sub video including the sub image 3 is used.
  • the code stream data size encoded by the data unit C is S2
  • the code stream data size encoded by the sub video data unit D including the sub image 4 is S3.
  • the image processor can receive and transmit error status according to historical data, priority of the current sub-video data unit,
  • the size of the code stream data encoded by the current sub-video data unit, and the estimated value of the real-time channel, such as the channel bandwidth, are combined and transmitted to the four sub-video data units to achieve real-time matching between the source and the channel. For example, when the sum of S0, S1, S2, and S3 is less than or equal to the current bandwidth of the channel, the encoded sub-video data unit A, sub-video data unit B, sub-video data unit C, and sub-video data unit D may be selected for transmission.
  • the image processor transmits the encoded sub-video data unit A, the encoded sub-video data unit B, and the encoded sub-video data unit C; If the sum of S0, S1, S2 is greater than T, then the first two higher priority sub-video data units A, sub-video data units B are determined, and so on, ensuring that at least the highest priority sub-video data unit A is transmitted.
  • sub-video data unit includes one sub-image, as shown in FIG. 10, sub-image 1, sub-image 2, sub-image 3, sub-image 4 have the same importance, or in some cases The importance of the image 1, the sub-image 2, the sub-image 3, and the sub-image 4 cannot be determined, that is, the sub-video data unit A including the sub-image 1, the sub-video data unit B including the sub-image 2, and the sub-video including the sub-image 3.
  • the data unit C, the sub-video data unit D including the sub-image 4 have the same priority or the priority cannot be determined, and the code stream data size encoded by the sub-video data unit A is S0, and the code encoded by the sub-video data unit B
  • the stream data size is S1
  • the code stream data size encoded by the sub video data unit C is S2
  • the code stream data size encoded by the sub video data unit D is S3.
  • the image processor can jointly transmit the four sub-video data units according to the transmission and error status of the historical data, the current stream data size of the sub-video data unit, and the estimated value of the real-time channel, such as the channel bandwidth, to achieve real-time source and channel. match.
  • the sum of S0, S1, S2, and S3 is less than or equal to the current bandwidth of the channel, and the encoded sub-video data unit A, the sub-video data unit B, the sub-video data unit C, and the sub-video data unit D may be selected for transmission.
  • the three largest code stream data can be selected from S0, S1, S2, and S3, and the three largest code stream data are S0, S1, S2, if S0, S1 If the sum of S2 is less than T, the image processor may select the encoded sub-video data unit A, the sub-video data unit B, and the sub-video data unit C to transmit, and so on, when the sum of S0, S1, S2 is greater than T. It is possible to select other combinations of the code stream data that can be transmitted with the bandwidth of the current wireless channel and the largest sub-picture data unit.
  • the combined code stream data size may fall within the range of [T-D, T+D].
  • the specific spatial decomposition includes spatial transformation decomposition and spatial down sampling decomposition, so that the sub-image obtained after the decomposition includes the image frame.
  • One or more pixels, or one or more conversion coefficients including an image frame the sub-images obtained by the decomposition are combined into a sub-video data unit, and when the encoded sub-video data unit is transmitted, the sub-video data unit is encoded
  • the code stream data size matches the characteristics of the channel (such as bandwidth), or the priority of the sub-video data unit is matched to the characteristics of the channel (such as bandwidth), which enables source and channel matching and reduces the frame level.
  • the transmission delay jitter caused by the video channel mismatch problem is a sub-video data unit.
  • FIG. 11 is a flowchart of a video receiving method according to an embodiment of the present invention. As shown in FIG. 11, the method in this embodiment may include:
  • Step S1101 receiving a plurality of encoded sub-video data units.
  • the video data includes one or more image frames
  • the sub-video data unit includes at least one of a plurality of sub-images obtained by decomposing each of the image frames.
  • the decomposition of the video data may be as shown in FIG. 3, FIG. 4 or FIG. 5, and the sub-video data unit is obtained; wherein each frame of the video data is performed.
  • the decomposition may be as shown in FIG. 9 or FIG. 10, and the specific decomposition process is consistent with the foregoing embodiment, and details are not described herein again.
  • the receiving device receives a plurality of encoded sub-video data units transmitted by the communication system of the unmanned aerial vehicle, wherein the video data may include one or more image frames, and the unmanned aerial vehicle transmits the video data before transmitting the video data.
  • Each image frame included in the data is decomposed into a plurality of sub-images.
  • the sub-video data unit includes at least one of a plurality of sub-images corresponding to each image frame decomposed by each image frame.
  • each sub-video data unit includes at least one sub-image of each image frame to reduce the correlation between the sub-images in each sub-video data unit, to avoid loss or distortion due to the sub-video data unit during transmission.
  • the resulting sub-images with high correlation are lost or distorted. Because the correlation between the sub-images obtained after the same frame image is decomposed is higher, and the sub-images with higher correlation are lost. Or when it is distorted, it will be difficult to recover an image frame composed of a highly correlated sub-image.
  • the receiving device receives 4 sub-video data units, and each sub-video data unit includes a sub-image obtained as shown in FIG. 9 or FIG.
  • Step S1102 Decode the plurality of encoded sub-video data units.
  • the plurality of encoded sub-video data units are separately decoded.
  • the receiving device separately decodes the four encoded sub-video data units, that is, separately performs code stream data corresponding to the four sub-images. Decode to obtain the decoded sub-video data unit.
  • the sub-video data unit is transmitted in the wireless channel, the sub-video data unit obtained by the receiving device and the encoded system actually transmitted by the communication system of the UAV may be caused due to noise interference, multipath effect, fading, and the like.
  • the video data units are different, causing the receiving device to receive an error.
  • the four sub-images obtained by the receiving device after decoding the code stream data are as shown in FIG. 12, wherein if the sub-image 1 is transmitted correctly, H1 Same as h1, H2 and h2 are the same, H3 and h3 are the same, H4 and h4 are the same, and if sub-image 1 is transmitted incorrectly, at least one of H1 and h1, H2 and h2, H3 and h3, H4 and h4 are different. Similarly, other sub-pictures are transmitted correctly or transmitted incorrectly, and the conversion coefficients before transmission and the conversion coefficients after transmission also have the same relationship.
  • the four sub-images obtained by the receiving device after decoding the code stream data are as shown in FIG. 13, wherein if the sub-image 1 is transmitted correctly, P1 The same as p1, P2 and p2 are the same, P3 and p3 are the same, and P4 and p4 are the same. If sub-image 1 is transmitted incorrectly, at least one of P1 and p1, P2 and p2, P3 and p3, P4 and p4 are different. Similarly, other sub-images are transmitted correctly or transmitted incorrectly, and the pixels before transmission and the pixels after transmission also have the same relationship.
  • Step S1103 Reconstruct the video data according to the decoded sub video data unit.
  • a transmission error of one or more sub-images of the sub-video data unit is detected, and the video data is reconstructed according to receiving the correct sub-image.
  • the receiving device decodes the code stream data to obtain 4 sub-images, it detects that each sub-image is transmitted correctly or transmits an error, and reconstructs the original image according to receiving the correct sub-image.
  • the sub-image transmitted by the communication system is the sub-image shown in FIG. 9, and the sub-image received by the receiving device is as shown in FIG. 12.
  • FIG. 12 it is assumed that the sub-image 2 receives an error, and the sub-image 1, The sub-image 3 and the sub-image 4 are all received correctly, and the receiving device reconstructs the original image according to the sub-image 1, the sub-image 3, and the sub-image 4 shown in FIG. 12, and reconstructs the original image, reconstructs the video data by using an inverse transform, and gives
  • the sub-picture in which the error is transmitted in the sub-picture data unit is assigned a value.
  • a possible implementation is to assign a value of 0 to the sub-picture that transmits the error in the sub-video data unit.
  • h2, h6, h10, and h14 of the sub-image 2 are both set to 0.
  • H1, H2, H3, and H4 are obtained based on P1, P2, P3, and P4.
  • spatial transformation is used to decompose the image frame, other spatial transformations are used.
  • the receiving device reconstructs the original image, the corresponding spatial transformation is adopted.
  • H1 and h1 are the same, H3 and h3 are the same, and H4 and h4 are the same. Therefore, p1, p2, p3, p4 obtained by inverse Hadamard transform and pixel values P1, P2, P3 in the original image, P4 may be different, but reconstructing the original image based on receiving the correct sub-image ensures that the reconstructed image is close to the original image.
  • h5, h6, h7, h8 are inversely transformed by Hadamard to obtain p5, p6, p7, p8, and h9, h10, h11, h12 are inversely transformed by Hadamama to obtain p9, p10, p11, p12, h13, H14, h15, h16 perform inverse Hadamard transform to obtain p13, p14, p15, p16, wherein h6, h10, h14 are all 0, and reconstruct the original image according to p1-p16 obtained by inverse Hadamard transform, as shown in Fig. 14. .
  • the sub-image transmitted by the communication system is the sub-image shown in FIG. 10, and the sub-image received by the receiving device is as shown in FIG. 13, and in FIG. 13, it is assumed that the sub-image 3 receives the error, and the sub-image 1, The sub-image 2 and the sub-image 4 are all received correctly, and the receiving device reconstructs the original image according to the sub-image 1, the sub-image 2, the sub-image 3, and the sub-image 4 shown in FIG. 13, and when the original image is reconstructed, the sub-video data unit is given.
  • the sub-image of the transmission error is assigned a value.
  • the value assigned to the sub-picture transmitted in the sub-video data unit is determined by interpolation, specifically, for transmission in the sub-video data unit.
  • the value of the wrong sub-image assignment is based on the transmission
  • the correct sub-image is determined, and the sub-image transmitted incorrectly and the sub-image transmitted correctly are from the same image frame.
  • the sub-image 3 receives an error, and the sub-image 1, the sub-image 2, and the sub-image 4 are all received correctly, and the sub-image 3 does not participate in the reconstruction process, that is, the receiving device only according to the sub-image 1, the sub-image 2,
  • the sub-image 4 reconstructs the original image.
  • the specific process is as follows: since the original image includes 16 pixels, the sub-image 1, the sub-image 2, and the sub-image 4 have a total of 12 pixels. According to FIG. 10, each of the 16 pixels in the original image is known. Four adjacent pixels are decomposed into four different sub-images. Therefore, when the original image is reconstructed from the sub-image 1, sub-image 2, and sub-image 4, the first pixel p1 of the sub-image 1 and the sub-image 2 The first pixel p2 and the first pixel p4 of the sub-image 4 are respectively three pixels P1, P2, and P4 of the first four adjacent pixels of the original image. Similarly, p5, p6, and p8 are original.
  • the three pixels P5, P6, P8, p9, p10, and p12 in the image P5-P8 are the three pixels P9, P10, and P12 in the original image P9-P12, respectively, and the p13, p14, and p16 are the original images P13-P16, respectively.
  • the three pixels P 13 , P 14 , P 16 in accordance with p1, p2, p4, p5, p6, p8, p9, p10, p12, p13, p14 p16 obtained image A shown in FIG.
  • the pixel image A spare reception error that is included in the sub-image pixel 3 due to reception error sub-image 3, and therefore, the sub-image 3 can not participate in the process of reconstruction of the original image. Since p1, p2, p4, p5, p6, p8, p9, p10, p12, p13, p14, and p16 are all correctly received, ie, p1, p2, p4, p5, p6, p8, p9, p10, p12, p13 , p14, p16 are the same as the pixel points at the same position of the original image. In this embodiment, the pixel value of the image A can be determined according to the interpolation method.
  • a feasible interpolation method is: p3 is equal to the arithmetic average of p1, p2, and p4 The value, p7 is equal to the arithmetic mean of p5, p6, p8, p11 is equal to the arithmetic mean of p9, p10, p12, and p15 is equal to the arithmetic mean of p13, p14, p16, resulting in the reconstructed original image B.
  • p3 is equal to the arithmetic average of p1, p2, and p4
  • the value, p7 is equal to the arithmetic mean of p5
  • p6, p8 p11 is equal to the arithmetic mean of p9, p10, p12
  • p15 is equal to the arithmetic mean of p13, p14, p16, resulting in the reconstructed original image B.
  • the receiving device separately decodes the plurality of sub-video data units to obtain the decoded sub-video data unit, and reconstructs the original image according to the decoded sub-video data unit, and specifically, receives the correct sub-image reconstruction according to the decoding.
  • the original image can make the reconstructed image be close to the original image to the greatest extent when the partial sub-image is received incorrectly, which improves the fault tolerance of the receiving device in reconstructing the image and enhances the robustness of the system.
  • FIG. 16 is a structural diagram of a video transmission system according to an embodiment of the present invention. As shown in FIG. 16, the video transmission system 1600 includes one or more imaging devices 1601, and one or more processors 1602 on a movable object.
  • the plurality of imaging devices 1601 are configured to acquire video data; the one or more processors 1602 work alone or in concert, configured to: decompose the video data into a plurality of sub-video data units, wherein each sub-video data unit includes one or Multiple sub-images; encoding the plurality of sub-video data units separately; and selecting one or more encoded sub-videos based on one or more characteristics of the channel and one or more characteristics of the sub-video data unit Data unit and transfer.
  • one or more characteristics of the channel include at least a bandwidth.
  • the one or more characteristics of the channel include at least one of: noise, interference, signal to noise ratio, bit error rate, fading rate, bandwidth.
  • the one or more characteristics of the sub video data unit include: a code stream data size encoded by the sub video data unit, or an energy concentration of the sub video data unit.
  • the total code stream data size of the one or more encoded sub-video data units matches the channel bandwidth.
  • the plurality of sub-video data units are prioritized according to the energy concentration.
  • the processor 1602, when selecting one or more encoded sub-video data units is configured to select the one or more encoded sub-video data units based on a priority and a channel bandwidth of the sub-video data units.
  • the video data is decomposed into a plurality of sub-video data units, and the plurality of sub-video data units are separately encoded, and one or more encoded sub-video data units are selected according to channel characteristics and characteristics of the sub-video data units.
  • the selected one or more encoded sub-video data units are made to conform to channel characteristics, and when the selected one or more encoded sub-video data units are transmitted on the channel matched thereto, the source and the source can be effectively solved.
  • the channel mismatch problem can effectively reduce the transmission delay jitter of video data due to the source channel mismatch problem.
  • Embodiments of the present invention provide a video transmission system.
  • the video data includes one or more image frames.
  • Processor 1602 will When the video data is decomposed into a plurality of sub-video data units, configured to: decompose each of the one or more image frames in the video data into a plurality of sub-images, wherein each of the sub-video data units At least one of the plurality of sub-images obtained by each of the image frames is included.
  • Each sub-image includes a portion of the image frame. Specifically, each sub-image includes one or more pixels of the image frame. Alternatively, each sub-image includes one or more conversion coefficients of the image frame.
  • the processor 1602 when the processor 1602 decomposes each of the one or more image frames in the video data into a plurality of sub-images, the processor 1602 is configured to: each of the one or more image frames in the video data A space is decomposed into multiple sub-images.
  • the processor 1602 When the processor 1602 decomposes each of the one or more image frames in the video data into a plurality of sub-images, the processor 1602 is configured to: use one of the video data by Fourier correlation transform or orthogonal transform Or each of the plurality of image frames is decomposed into a plurality of sub-images.
  • the Fourier correlation transform or orthogonal transform is selected from a Hadamard transform, a discrete cosine transform, a discrete Fourier transform, a Walsh-Hadamard transform, a Haar transform or a skew transform.
  • the processor 1602 when the processor 1602 decomposes each of the one or more image frames in the video data into a plurality of sub-images, the processor 1602 is configured to: spatially downsample one or more images in the video data Each space in the frame is decomposed into multiple sub-images.
  • processor 1602 separately encodes the plurality of sub video data units
  • the following may be implemented in several feasible manners:
  • the processor 1602 controls a plurality of encoders to encode the plurality of sub-video data units. Specifically, the processor 1602 controls the plurality of encoders to perform parallel encoding on the plurality of sub video data units; or the processor 1602 controls the plurality of encoders to respectively use the different video encoding rules to the plurality of sub video data. The unit performs encoding; or alternatively, the processor 1602 controls the plurality of encoders to encode the plurality of sub-video data units using the same video encoding rule.
  • the processor 1602 controls the encoder to encode two or more of the plurality of sub-video data units.
  • the processor 1602 controls the encoder to encode at least one of the plurality of sub-video data units based on a motion compensated video compression standard.
  • the processor 1602 is configured to compress the plurality of sub-video data units according to different compression ratios. Wherein the compression ratio is determined according to one or more characteristics of the sub-video data unit.
  • the encoder may be a hardware entity that is independent of the processor 1602 and is electrically connected to the processor 1602, or may be software for implementing the encoding function in the processor 1602.
  • the movable object is an unmanned aerial vehicle.
  • One or more imaging devices 1601 are coupled to the movable object by a carrier, which may be a multi-axis universal joint.
  • the specific spatial decomposition includes spatial transformation decomposition and spatial down sampling decomposition, so that the sub-image obtained after the decomposition includes the image frame.
  • One or more pixels, or one or more conversion coefficients including an image frame, for a frame image a plurality of sub-images obtained by the decomposition thereof may be combined and transmitted, so that the combined code stream data size and channel
  • the bandwidth matching, or the combined code stream data further includes code stream data corresponding to the sub-images with higher importance, so that each frame image can be matched on the wireless channel to achieve source and channel matching, and the frame is reduced.
  • FIG. 17 is a structural diagram of a receiving device according to an embodiment of the present invention. As shown in FIG. 17, the receiving device 1700 includes a communication interface 1701, one or more processors 1702, and one or more processors 1702 work alone or in cooperation.
  • the interface 1701 is communicatively coupled to the processor 1702; the communication interface 1701 is configured to receive a plurality of encoded sub-video data units, wherein the video data includes one or more image frames, and the sub-video data unit includes the At least one of the plurality of sub-images obtained after each of the image frames is decomposed; the one or more processors 1702 are configured to: control the decoder to decode the plurality of encoded sub-video data units; The subsequent sub-video data unit reconstructs the video data.
  • the processor 1702 when the processor 1702 controls the decoder to decode the plurality of encoded sub-video data units, the processor 1702 is specifically configured to: control the decoder to separately decode the plurality of encoded sub-video data units.
  • the decoder may be independent of the processor 1702 and with the processor
  • a hardware entity that is electrically connected to 1702 may also be software for implementing a decoding function in the processor 1702.
  • the processor 1702 is further configured to: detect a transmission error of one or more sub-images of the sub-video data unit; when the processor 1702 reconstructs the video data according to the decoded sub-video data unit, specifically, according to the receiving The correct sub-image reconstructs the video data.
  • the processor 1702 is further configured to assign a value to the sub-image in which the error is transmitted in the sub-video data unit.
  • the value assigned to the sub-image in which the error is transmitted in the sub-video data unit is 0.
  • the value assigned to the sub-picture in which the error is transmitted in the sub-video data unit is determined by interpolation.
  • the value assigned to the sub-picture in which the error is transmitted in the sub-video data unit is determined based on the transmission of the correct sub-image, and the sub-image of the transmission error and the sub-image of the transmission are from the same image frame.
  • the processor 1702 When the processor 1702 reconstructs the video data according to the decoded sub video data unit, the processor 1702 is specifically configured to: reconstruct the video data by using an inverse transform.
  • the receiving device may be a remote controller, a smart phone, a tablet computer, a ground control station, a laptop computer, a watch, a wristband, and the like, and combinations thereof.
  • the receiving device separately decodes the plurality of sub-video data units to obtain the decoded sub-video data unit, and reconstructs the original image according to the decoded sub-video data unit, and specifically, receives the correct sub-image reconstruction according to the decoding.
  • the original image can make the reconstructed image be close to the original image to the greatest extent when the partial sub-image is received incorrectly, which improves the fault tolerance of the receiving device in reconstructing the image and enhances the robustness of the system.
  • An embodiment of the present invention provides a control terminal, where the control terminal includes the receiving device described in the foregoing embodiment.
  • the control terminal can be a remote control, a smart phone, a tablet, a ground control station, a laptop, a watch, a wristband, etc., and combinations thereof, and can also control the unmanned aerial vehicle on the ground.
  • FIG. 18 is a schematic diagram of an embodiment of the present invention
  • the unmanned aerial vehicle 1800 includes: a fuselage, a power system, and a video transmission system, the power system including at least one of the following: a motor 1801, a propeller 1802, and an electronic governor 1803.
  • a power system is mounted to the airframe for providing flight power;
  • a flight controller 1804 is in communication with the power system for controlling the unmanned flight 1806, wherein the flight controller 1804 includes an inertial measurement unit And gyroscopes.
  • the inertial measurement unit and the gyroscope are configured to detect an acceleration, a pitch angle, a roll angle, a yaw angle, and the like of the unmanned aerial vehicle.
  • the video transmission system includes one or more imaging devices 1805, and one or more processors 1806 disposed on the movable object.
  • the imaging device 1805 is coupled to the body through the support device 1807, and the processor 1806 and the imaging device 1805 are communicatively coupled.
  • the supporting device 1807 may specifically be a pan/tilt. The principle and implementation of the video transmission system are similar to the above embodiments, and are not described herein again.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
  • the above-described integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium.
  • the above software functional unit is stored in a storage medium and includes a plurality of instructions for causing a computer device (which may be a personal computer, a server, or A network device or the like) or a processor performs part of the steps of the method described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Selon certains modes de réalisation, la présente invention concerne un procédé, un système et un dispositif d'émission et de réception de vidéo, et un véhicule aérien sans pilote. Le procédé consiste : à décomposer des données vidéo en une pluralité d'unités de données de sous-vidéo ; à coder séparément la pluralité d'unités de données de sous-vidéo ; et à sélectionner une ou plusieurs unités de données de sous-vidéo codées sur la base d'une ou de plusieurs caractéristiques d'un canal et d'une ou de plusieurs caractéristiques des unités de données de sous-vidéo, et à transmettre l'unité ou les unités de données de sous-vidéo codées. Selon certains modes de réalisation de la présente invention, des données vidéo sont décomposées en une pluralité d'unités de données de sous-vidéo, et la pluralité d'unités de données de sous-vidéo sont codées séparément, et une ou plusieurs unités de données de sous-vidéo codées sont sélectionnées en fonction d'une caractéristique de canal et d'une caractéristique des unités de données de sous-vidéo, de telle sorte que l'unité ou les unités de données de sous-vidéo codées sélectionnées correspondent à la caractéristique de canal. De cette manière, le problème de discordance entre une source de signal et un canal peut être efficacement résolu, et le retard de transmission et l'instabilité de données vidéo provoqués par une discordance entre une source de signal et un canal peuvent être efficacement réduits.
PCT/CN2017/078728 2017-03-30 2017-03-30 Procédé, système et dispositif d'émission et de réception de vidéo, et véhicule aérien sans pilote WO2018176303A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2017/078728 WO2018176303A1 (fr) 2017-03-30 2017-03-30 Procédé, système et dispositif d'émission et de réception de vidéo, et véhicule aérien sans pilote
CN201780005017.8A CN108496369A (zh) 2017-03-30 2017-03-30 视频传输、接收方法、系统、设备及无人飞行器

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/078728 WO2018176303A1 (fr) 2017-03-30 2017-03-30 Procédé, système et dispositif d'émission et de réception de vidéo, et véhicule aérien sans pilote

Publications (1)

Publication Number Publication Date
WO2018176303A1 true WO2018176303A1 (fr) 2018-10-04

Family

ID=63344689

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/078728 WO2018176303A1 (fr) 2017-03-30 2017-03-30 Procédé, système et dispositif d'émission et de réception de vidéo, et véhicule aérien sans pilote

Country Status (2)

Country Link
CN (1) CN108496369A (fr)
WO (1) WO2018176303A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021179304A1 (fr) * 2020-03-13 2021-09-16 深圳市大疆创新科技有限公司 Procédé d'ajustement automatique de débit binaire de diffusion vidéo en direct, dispositif de transmission vidéo et serveur

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111656785A (zh) * 2019-06-28 2020-09-11 深圳市大疆创新科技有限公司 可移动平台的图像处理方法、装置、可移动平台及介质
WO2021056575A1 (fr) * 2019-09-29 2021-04-01 华为技术有限公司 Procédé de codage source-canal combiné à faible retard, et dispositif associé
CN111565291B (zh) * 2020-07-14 2020-11-10 深圳市慧明捷科技有限公司 一种无人机多基站图像传输方法
CN114040226B (zh) * 2022-01-10 2022-03-11 北京小鸟科技股份有限公司 低带宽高分辨率视频传输的数据传输方法、系统及设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102907096A (zh) * 2010-05-10 2013-01-30 三星电子株式会社 用于发送和接收分层编码视频的方法和设备
WO2013128010A2 (fr) * 2012-03-02 2013-09-06 Canon Kabushiki Kaisha Procédé et dispositifs pour coder une séquence d'images en un flux binaire vidéo hiérarchique, et décoder un flux binaire vidéo hiérarchique correspondant
CN103945165A (zh) * 2014-05-05 2014-07-23 北京领通科技有限公司 终端设备远程视频传输的处理方法和装置
CN104349142A (zh) * 2014-11-03 2015-02-11 南京航空航天大学 一种基于分层表达的无人机视频自适应传输方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7480252B2 (en) * 2002-10-04 2009-01-20 Koniklijke Philips Electronics N.V. Method and system for improving transmission efficiency using multiple-description layered encoding
JP4187746B2 (ja) * 2005-01-26 2008-11-26 三洋電機株式会社 動画像データ伝送装置
US8189621B2 (en) * 2006-05-12 2012-05-29 Microsoft Corporation Stack signaling to application with lack of requested bandwidth
US20100114581A1 (en) * 2006-10-06 2010-05-06 Te Li Method for encoding, method for decoding, encoder, decoder and computer program products
CN101848499B (zh) * 2009-03-25 2013-05-08 上海贝尔股份有限公司 改进无线系统中的分级业务传输的方法、网络单元及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102907096A (zh) * 2010-05-10 2013-01-30 三星电子株式会社 用于发送和接收分层编码视频的方法和设备
WO2013128010A2 (fr) * 2012-03-02 2013-09-06 Canon Kabushiki Kaisha Procédé et dispositifs pour coder une séquence d'images en un flux binaire vidéo hiérarchique, et décoder un flux binaire vidéo hiérarchique correspondant
CN103945165A (zh) * 2014-05-05 2014-07-23 北京领通科技有限公司 终端设备远程视频传输的处理方法和装置
CN104349142A (zh) * 2014-11-03 2015-02-11 南京航空航天大学 一种基于分层表达的无人机视频自适应传输方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021179304A1 (fr) * 2020-03-13 2021-09-16 深圳市大疆创新科技有限公司 Procédé d'ajustement automatique de débit binaire de diffusion vidéo en direct, dispositif de transmission vidéo et serveur

Also Published As

Publication number Publication date
CN108496369A (zh) 2018-09-04

Similar Documents

Publication Publication Date Title
US12003770B2 (en) Face discontinuity filtering for 360-degree video coding
US10931851B2 (en) Image stitching with electronic rolling shutter correction
US11388438B2 (en) 360-degree video coding using geometry projection
WO2018176303A1 (fr) Procédé, système et dispositif d'émission et de réception de vidéo, et véhicule aérien sans pilote
US11064110B2 (en) Warp processing for image capture
US20230412839A1 (en) Geometry Conversion for 360-degree Video Coding
US11057646B2 (en) Image processor and image processing method
US11212496B2 (en) Geometry conversion and frame packing associated with 360-degree videos
WO2018237146A1 (fr) Quantification adaptative pour un codage vidéo à 360 degrés
US20190355098A1 (en) Multiscale denoising of raw images with noise estimation
WO2018176340A1 (fr) Procédé de transmission vidéo, procédé de réception, système, et véhicule aérien sans pilote
WO2017205597A1 (fr) Indices de codage sur la base d'un traitement de signal d'images pour l'estimation de mouvement
JP2025528684A (ja) Ａｉベースの点群圧縮の深層分布認識点特徴抽出器
CN116033155A (zh) 双目图像的压缩方法、设备及可读存储介质
WO2018017599A1 (fr) Système et procédé d'évaluation de qualité pour vidéo à 360 degrés
WO2018176341A1 (fr) Procédé de transmission vidéo, procédé de réception, système et véhicule aérien sans pilote
EP3091742A1 (fr) Dispositif et procédé pour coder une première image d'une scène à l'aide d'une seconde image présentant une résolution inférieure et capturée au même instant
WO2025131487A1 (fr) Stabilisation de fenêtre d'affichage

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17904344

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17904344

Country of ref document: EP

Kind code of ref document: A1