[go: up one dir, main page]

CN107835434B - Event wide-view live broadcasting equipment and associated live broadcasting system and method - Google Patents

Event wide-view live broadcasting equipment and associated live broadcasting system and method Download PDF

Info

Publication number
CN107835434B
CN107835434B CN201710431777.7A CN201710431777A CN107835434B CN 107835434 B CN107835434 B CN 107835434B CN 201710431777 A CN201710431777 A CN 201710431777A CN 107835434 B CN107835434 B CN 107835434B
Authority
CN
China
Prior art keywords
video stream
wide
view
audio
live
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710431777.7A
Other languages
Chinese (zh)
Other versions
CN107835434A (en
Inventor
支小牧
刘善红
王旭
倪建聪
段硕
李宝玉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Heyu Lanqiao Technology Management Center (Limited Partnership)
Original Assignee
Fblife Beijing Media Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fblife Beijing Media Technology Co ltd filed Critical Fblife Beijing Media Technology Co ltd
Priority to CN201710431777.7A priority Critical patent/CN107835434B/en
Publication of CN107835434A publication Critical patent/CN107835434A/en
Application granted granted Critical
Publication of CN107835434B publication Critical patent/CN107835434B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/2187Live feed
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/233Processing of audio elementary streams
    • H04N21/2335Processing of audio elementary streams involving reformatting operations of audio signals, e.g. by converting from one coding standard to another
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41Structure of client; Structure of client peripherals
    • H04N21/4104Peripherals receiving signals from specially adapted client devices
    • H04N21/4122Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643Communication protocols
    • H04N21/6437Real-time Transport Protocol [RTP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65Transmission of management data between client and server
    • H04N21/658Transmission by the client directed to the server

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The application relates to a competition wide-view-angle live broadcast device, which comprises a wide-view-angle camera unit, a wide-view-angle image acquisition unit and a live broadcast unit, wherein the wide-view-angle camera unit is used for acquiring a wide-view-angle image; a sequence of executable computer instructions stored in a memory; to realize that: obtaining an internal parameter matrix and a distortion coefficient matrix of the camera unit; preprocessing the wide-view image; expanding the pre-processed image based on the intrinsic parameter matrix and the distortion coefficient matrix to generate an expanded wide-view image; generating an expanded wide-view video stream based on the expanded wide-view image; and transmitting the expanded wide-view video stream to a streaming server. The application also relates to related live systems and methods.

Description

Event wide-view live broadcasting equipment and associated live broadcasting system and method
Technical Field
The present application relates to live devices and methods, and more particularly, to live event devices, systems, and methods.
Background
Live events generally refer to the process of transmitting images and sounds of the event scene to, for example, a television, a radio, a computer, a mobile terminal, etc. through a cable television network, a broadcast network, or the internet, etc. in real time. With the development of internet technology, live broadcast technology based on internet has been developed, for example, streaming media technology appears, which can compress media data in sections and then send the data through a network, and transmit video and audio on the network in real time for a viewer to use specific playing software to play and view, and the viewer can use the playing software to enjoy live broadcast pictures of a certain shooting angle of a video capture device in real time.
When the sports racing car events such as the rally are rebroadcast, a relatively wide view angle is usually required to embody a large scene, and long-time playing of videos with wide view angles can lead viewers to tired emotion, thereby losing interest in live broadcasting. Conventional live broadcasting techniques usually employ time-sequential scene switching to realize live broadcasting from different angles, such as playing a first local close-up section at a first time, playing a panoramic section at a second time, and the like. Although pip technology is available, it occupies extra resources in the digital transmission mode, and thus has been rarely applied in digital tv systems.
Accordingly, there is a need for an apparatus, system, and associated method that enables wide-view live broadcasting of events.
Disclosure of Invention
Some embodiments of the present application relate to a live broadcast device with a wide viewing angle for an event, which includes a wide viewing angle camera unit for acquiring a wide viewing angle image; a memory for storing a sequence of computer instructions; a processor that executes the sequence of computer instructions to implement: obtaining an internal parameter matrix and a distortion coefficient matrix of the camera unit; preprocessing the wide-view image; expanding the pre-processed image based on the intrinsic parameter matrix and the distortion coefficient matrix to generate an expanded wide-view image; generating an expanded wide-view video stream based on the expanded wide-view image; and transmitting the expanded wide-view video stream to a streaming media server.
Other embodiments of the present application relate to a live event system comprising at least one wide-view live device for generating a video stream and/or an audio-video stream; the streaming media server is used for receiving the video stream or the audio and video stream transmitted by the wide-view-angle live broadcast equipment; the live broadcast server is used for distributing the video stream or the audio and video stream received by the streaming media server to the playing equipment through a live broadcast network; the wide-view-angle live broadcast equipment comprises a wide-view-angle camera unit and a wide-view-angle display unit, wherein the wide-view-angle camera unit is used for acquiring a wide-view-angle image; a memory for storing a sequence of computer instructions;
a processor that executes the sequence of computer instructions to implement: obtaining an internal parameter matrix and a distortion coefficient matrix of the camera unit; preprocessing the wide-view image; expanding the pre-processed image based on the intrinsic parameter matrix and the distortion coefficient matrix to generate an expanded wide-view image; forming an expanded wide-view video stream based on the expanded wide-view image; and transmitting the video stream to a streaming media server.
Still other embodiments of the present application provide a live event broadcasting method, including: generating an expanded wide-view video stream or audio/video stream by at least one event wide-view live broadcast device; transmitting the expanded wide-view video stream or audio/video stream to the streaming media server; generating a common visual angle video stream or an audio/video stream by at least one common live broadcast device; transmitting the common visual angle video stream or the audio and video stream to the streaming media server; the streaming media server transmits the wide-view video stream or the audio/video stream and the common-view video stream or the audio/video stream to a live broadcast server in parallel; and a step in which the live broadcast server provides the wide view video stream or audio/video stream and the normal view video stream or audio/video stream to a playback device synchronously or asynchronously.
Another embodiment of the present application provides a live event broadcasting method, including generating an expanded wide-view video stream or audio/video stream by at least one live event wide-view device; transmitting the expanded wide-view video stream or audio/video stream to the streaming media server; generating a common visual angle video stream or an audio/video stream by at least one common live broadcast device; transmitting the common visual angle video stream or the audio and video stream to the streaming media server; the streaming media server transmits the wide-view video stream or the audio/video stream and the common-view video stream or the audio/video stream to a live broadcast server in parallel; and the live broadcast server provides the wide visual angle video stream or audio/video stream and the common visual angle video stream or audio/video stream to the playing equipment according to the selected operation of the playing equipment.
The live broadcast system and the live broadcast method with the wide viewing angles are particularly suitable for scenes such as live broadcast of events, suitable solutions are provided for live broadcast of off-road events and the like, and particularly, application of live broadcast equipment with the wide viewing angles of the events to a streaming media live broadcast platform is provided.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is a schematic diagram of a scenario according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a live device according to an embodiment of the present application;
fig. 3 is a flow diagram of image processing of a live device in accordance with an embodiment of the present application;
fig. 4 is a schematic diagram of a live system according to an embodiment of the present application; and
fig. 5 is a schematic diagram of a playback interface of a playback device according to an embodiment of the present application.
Detailed Description
Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel, concurrently, or simultaneously. In addition, the order of the operations may be re-arranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.
The term "server" in this context refers to an intelligent electronic device that can perform predetermined processes such as numerical calculation and/or logic calculation by executing predetermined programs or instructions, and may include a processor and a memory, wherein the predetermined processes are performed by the processor executing program instructions prestored in the memory, or the predetermined processes are performed by hardware such as ASIC, FPGA, DSP, or a combination thereof.
Wherein the user equipment includes but is not limited to smart phones, PDAs, PCs, notebook computers, etc.; the network device includes, but is not limited to, a single network server, a server group consisting of a plurality of network servers, or a Cloud Computing (Cloud Computing) based Cloud consisting of a large number of computers or network servers, wherein Cloud Computing is one of distributed Computing, a super virtual computer consisting of a collection of loosely coupled computers. The computer equipment can be independently operated to realize the application, and can also be accessed into a network to realize the application through the interactive operation with other computer equipment in the network. The network in which the computer device is located includes, but is not limited to, the internet, a wide area network, a metropolitan area network, a local area network, a VPN network, and the like.
It should be noted that the user equipment, the network device, the network, etc. are only examples, and other existing or future computer devices or networks may also be included in the scope of the present application, if applicable, and are included by reference.
The methodologies discussed hereinafter, some of which are illustrated by flow diagrams, may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to perform the necessary tasks may be stored in a machine or computer readable medium such as a storage medium. The processor(s) may perform the necessary tasks.
Specific structural and functional details disclosed herein are merely representative and are provided for purposes of describing example embodiments of the present application. This application may, however, be embodied in many alternate forms and should not be construed as limited to only the embodiments set forth herein.
It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element may be termed a second element, and, similarly, a second element may be termed a first element, without departing from the scope of example embodiments. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be noted that, in some alternative implementations, the functions/acts noted may occur out of the order noted in the figures. For example, two figures shown in succession may, in fact, be executed substantially concurrently, or the figures may sometimes be executed in the reverse order, depending upon the functionality/acts involved.
The live event system of one embodiment of the present application may be adapted for use in, for example, live events such as vehicle races, which may include, for example, field racing races such as slot and cross-country racing races such as rallies. FIG. 1 illustrates a schematic view of a live system scenario for a vehicle athletic event. As shown in FIG. 1, a typical vehicle athletic event includes at least two vehicles, at least one of which has a live device mounted thereon. The field of the event may have live broadcast devices oriented at different shooting angles, the live broadcast devices 10 may generate video streams, audio streams, or audio/video streams, the audio streams, or the audio/video streams may be transmitted to a streaming media server, and the streaming media server 20 may transmit the video streams, the audio streams, or the audio/video streams to a playing device of a user equipment, such as a smart phone, through a live broadcast platform for playing. The live platform 30 may be a live server that may have a director system that controls the video, audio, or audio-video streams transmitted to the user devices and selects the video, audio, or audio-video streams received by the live platform 30. The live platform 30 may store the received video stream, audio stream, or audio/video stream in different storage locations, and the user device 40 may view the video stream, audio stream, or audio/video stream by accessing an address specified by the live platform.
Among the live devices shown in fig. 1, some of the live devices may be live devices 10 having wide-angle camera units. A wide viewing angle is an acquisition device with a viewing angle greater than 170 degrees. The wide-angle camera unit can be, for example, an image with a fisheye lens, a video capture module, such as a fisheye camera. These devices with wide-angle lenses may be deployed, for example, inside a vehicle, such as an instrument desk or a roof, or anywhere on a racing track.
In the illustration shown in fig. 1, a part of the vehicles is equipped with a live broadcast device 10 having a fisheye camera, which may also be referred to as a fisheye live broadcast device, and the fisheye live broadcast devices 10A, 10B, 10C, and 10D may be mounted on the vehicles, for example, or may be mounted in a stadium, as shown by live broadcast devices 10E and 10F.
Fig. 2 shows a schematic diagram of a live device 10 with a fisheye camera in an embodiment. Referring to fig. 2, the live broadcasting device includes a fisheye camera 11, a memory 12, a wired or wireless output unit 13, and a processor 14. The processor may communicate with the fisheye camera, memory, wired or wireless output unit via bus 16. The image acquired by the fisheye camera 11 is an image with a viewing angle substantially larger than 180 degrees, and a program may be stored in the memory 12, and when the processor runs the program, the image acquired by the fisheye camera is firstly acquired or received S101 and the camera intrinsic parameters and distortion parameters S102, and then the acquired or received image data is subjected to pre-processing S200.
The preprocessing may include one or more of vignetting processing and image position correction of the image.
After the pre-processing, the processor runs the program to perform unfolding processing on the wide-view-angle image based on the measured internal parameters and distortion parameters of the lens S300.
The image expansion process may use, for example, an equal arc length mapping technique of a cylindrical model, i.e., a cylindrical model with an equal radius to a spherical model of a fisheye lens, wherein the spherical model of the fisheye lens is based on CCS (camera coordinate system) and the cylindrical model is based on WCS (world coordinate system). The coordinate system of the cylindrical model and the coordinate system of the spherical model need to be corresponded, and the corresponding relation between the point on the cylindrical model in the WCS coordinate system and the point on the spherical model in the CCS is established. For example, the height and width of the cylinder can be first determined from the maximum range of the horizontal viewing angle of the cylinder model; and then, corresponding a certain point of the image plane to a point of a spherical surface with the radius of R according to an equidistant imaging model, mapping the point to a cylindrical surface in an equal arc length manner, and establishing the relation between the inclination angle and the horizontal azimuth angle as well as the height and width positions of the expanded graph. Therefore, the one-to-one mapping relation between the image plane points and the cylindrical plane points can be established, so that the corresponding relation between the fisheye image and the cylindrical image is completed, and the color value of each point of the fisheye image is obtained according to a bilinear interpolation method to give a more accurate gray value to the point on the cylindrical surface.
By the above procedure, an expanded wide view image can be generated at the live broadcast device 10, and an expanded wide view video stream is generated from the expanded wide view image according to the generation timing of the expanded view or the acquisition timing of the original image, and the video stream is compressed by encoding and transmitted to the streaming media server 20 based on, for example, an RTMP (real time messaging protocol) or RTSP (real time streaming protocol) protocol.
The streaming media server 20 may be geographically located remotely from the live platform 30. For example, the streaming server 20 may be located in the center of the track, or on any vehicle.
The streaming media server 20 may store the received video stream segment in the storage location 31 of the live platform 30 for the user equipment to retrieve, or may also directly transmit the video stream segment to the user equipment through the live platform based on the RTMP, RTSP, HLS protocol or HTTP FLV protocol.
The live device 10 may be provided with a sound collection unit 15, the sound collection unit 15 collecting sound field signals, and the sound field signals may be digitized into audio data by a program run by a processor, and the audio data may be pre-processed, e.g., de-noised, at the live device. The audio data and the image after the pre-processing are synchronized, and are encapsulated into audio and video streams based on the RTMP or RTSP protocol after being encoded and compressed, and then the audio and video streams are transmitted to the streaming media server 20.
Or, a request for transmitting the video stream or the audio/video stream may be sent to the live device 10 by the streaming media server 20 at regular time, and the live device 10 may compress and transmit the generated video stream or the audio/video stream to the streaming media server 20 in response to the request.
The streaming server 20 stores and simultaneously plays the obtained stream live to the user device 40 of the viewer through the live platform 30.
The live system 30 may also obtain video streams or audio-video streams from a common viewing angle live device 50. One or more of the normal-view live devices 50 may be arranged. May be arranged on the vehicle or at any position of the playing field. The normal view live broadcast device may have a similar structure to the wide view live broadcast device. For example, an image acquisition unit, a processor, a memory, a bus, a wired or wireless output unit, etc. may be included as well. The normal view live broadcasting device 50 can generate a video stream only by correcting the acquired image, and push the video stream to the streaming media server.
Similarly, the normal-view live broadcasting device 50 may also include a sound collection unit for acquiring sound field signals and digitally generating audio data. The audio data can be encoded to form an audio stream and pushed to the streaming media server or encapsulated into an audio/video stream together with the video stream and pushed to the streaming media server.
The delivery of the expanded wide view video stream or audio-visual stream and the normal view expanded video stream or audio-visual stream may be controlled via the director system 32 of the live platform.
The video stream, the audio stream, or the audio/video stream generated by the normal viewing angle live broadcasting device 50 may be pushed to the streaming media server 20 in parallel with the expanded wide viewing angle video stream or the expanded audio/video stream generated by the wide viewing angle live broadcasting device. The streaming media server 20 may synchronize the expanded wide view video stream or audio/video stream with the normal view video stream or audio/video stream and then transmit the synchronized stream to the live broadcast platform for storage, or transmit the synchronized stream to a playing device such as a smart phone through the live broadcast platform.
The expanded wide-view video stream or audio-video stream and the common-view video stream or audio-video stream can also be asynchronously transmitted to the playing device.
A multi-angle play interface may be provided at the play device, for example, a play interface as shown in fig. 5 may be provided, which includes a play screen in the middle portion in conformity with the aspect ratio of the expanded wide view image, for example, an expanded wide view screen 1, an expanded wide view screen 2, and an expanded wide view screen 3. And normal-viewing-angle screens 1 to 4 having a normal viewing angle aspect ratio on both sides of the unfolded wide-viewing-angle screen. The video source played by the common visual angle screen can be a video stream pushed by the common visual angle live broadcasting equipment arranged on the competition field or the common visual angle live broadcasting equipment arranged on the racing car.
For example, a video stream or an audio/video stream may be played on one of the expanded wide view screen and the expanded normal view screen, and a segment of the video stream in a time cycle is repeatedly played on the other screens, and when the segment played in the cycle is selected by a user through, for example, a click operation or a touch operation, the video stream or the audio/video stream stored by the corresponding live broadcast equipment is pulled from the memory of the live broadcast platform, or the video stream or the audio/video stream just obtained by the streaming media server from the live broadcast equipment is pulled from the streaming media server in real time.
The playing interface can also judge whether the screen needs to be switched from the expanded wide view angle screen to the common view angle screen or vice versa when receiving the operation of a user on the screen corresponding to a certain direct playing device, and if the switching is judged to be needed, the playing interface is replaced.
It should be understood by those skilled in the art that the above-mentioned function options and their corresponding live event applications are only examples for the purpose of illustrating the present application, and should not be construed as any limitation to the present application, and other existing or future function options and their corresponding live event applications, as applicable to the present application, should be included in the scope of the present application.
It should be noted that the processing portions of the present application may be implemented in software and/or a combination of software and hardware, for example, using Application Specific Integrated Circuits (ASICs), general purpose computers or any other similar hardware devices. In one embodiment, the software programs of the present application may be executed by a processor to implement the steps or functions described above. Likewise, the software programs (including associated data structures) of the present application may be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Additionally, some of the steps or functions of the present application may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present application may be implemented as a computer program product, such as computer program instructions, which when executed by a computer, may invoke or provide methods and/or techniques in accordance with the present application through the operation of the computer. Program instructions which invoke the methods of the present application may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the present application comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or a solution according to the aforementioned embodiments of the present application.
Some specific examples of the present application are listed below:
1. a wide-viewing-angle live broadcasting equipment for events comprises
The wide-view-angle shooting unit is used for acquiring a wide-view-angle image;
a memory for storing a sequence of computer instructions;
a processor that executes the sequence of computer instructions to implement:
obtaining an internal parameter matrix and a distortion coefficient matrix of the camera unit;
preprocessing the wide-view image;
expanding the pre-processed image based on the intrinsic parameter matrix and the distortion coefficient matrix to generate an expanded wide-view image;
generating an expanded wide-view video stream based on the expanded wide-view image;
and transmitting the expanded wide-view video stream to a streaming media server.
2. The wide-view live event equipment according to example 1, characterized in that: the wide view live broadcasting device pushes the expanded wide view video stream to the streaming media server based on an RTMP or RTSP protocol, or the wide view live broadcasting device transmits the expanded wide view video stream to the streaming media server based on an RTMP/RTSP/HTTP request of the streaming media server.
3. The wide-view live event equipment according to example 1, characterized in that: the sports racing car sound field signal acquisition device further comprises a sound acquisition unit used for acquiring sound field signals of the live broadcasting equipment of the sports racing cars.
4. A wide-view live event device according to example 3, characterized in that: the memory also stores sequences of computer instructions that are executed by the processor to implement
Digitizing the sound field signals into audio data;
encoding the audio data and the expanded wide-view image to be packaged into an expanded wide-view audio and video stream; and
and transmitting the expanded wide-view audio and video stream to the streaming media server.
5. The wide-view live event equipment according to example 1, characterized in that: the events include vehicle athletic events, and the event wide-view live broadcast device is arranged on a vehicle.
6. The wide-view live event equipment according to example 1, characterized in that: the events comprise vehicle athletic events, and the event wide-view live broadcast equipment is arranged on the event field.
7. A live event system, comprising: comprises that
At least one wide-view live device for generating a video stream and/or an audio-video stream;
the streaming media server is used for receiving the video stream or the audio and video stream transmitted by the wide-view-angle live broadcast equipment; and
the live broadcast server is used for distributing the video stream or the audio and video stream received by the streaming media server to the playing equipment through a live broadcast network;
wherein the wide-view live broadcasting equipment comprises
The wide-view-angle shooting unit is used for acquiring a wide-view-angle image;
a memory for storing a sequence of computer instructions;
a processor that executes the sequence of computer instructions to implement:
obtaining an internal parameter matrix and a distortion coefficient matrix of the camera unit;
preprocessing the wide-view image;
expanding the pre-processed image based on the intrinsic parameter matrix and the distortion coefficient matrix to generate an expanded wide-view image;
forming an expanded wide-view video stream based on the expanded wide-view image;
and transmitting the video stream to a streaming media server.
8. A live event system according to example 5, characterized by: the wide view live broadcasting device pushes the expanded wide view video stream to the streaming media server based on an RTMP or RTSP protocol, or the wide view live broadcasting device transmits the expanded wide view video stream to the streaming media server based on an RTMP/RTSP/HTTP request of the streaming media server.
9. A live event system according to example 5, characterized by: the sports racing car sound field signal acquisition device further comprises a sound acquisition unit used for acquiring sound field signals of the live broadcasting equipment of the sports racing cars.
10. A live event system according to example 7, characterized by: the memory also stores sequences of computer instructions that are executed by the processor to implement
Digitizing the sound field signals into audio data;
encoding the audio data and the expanded wide-view image to be packaged into an expanded wide-view audio and video stream; and
and transmitting the expanded wide-view audio and video stream to the streaming media server.
11. A live event system according to example 7, characterized by: the system also comprises at least one common visual angle live broadcast device, wherein the common visual angle live broadcast device generates a common visual angle video stream and transmits the common visual angle video stream to the streaming media server.
12. A live event system according to example 11, wherein: and the streaming media system transmits the common visual angle video stream received from the common visual angle live broadcast equipment and the expanded wide visual angle video stream received from the wide visual angle live broadcast equipment to a live broadcast server in parallel.
13. A live event broadcasting method, comprising:
generating an expanded wide-view video stream or audio/video stream by at least one event wide-view live broadcast device;
transmitting the expanded wide-view video stream or audio/video stream to the streaming media server;
generating a common visual angle video stream or an audio/video stream by at least one common live broadcast device;
transmitting the common visual angle video stream or the audio and video stream to the streaming media server;
the streaming media server transmits the wide-view video stream or the audio/video stream and the common-view video stream or the audio/video stream to a live broadcast server in parallel; and
and the live broadcast server synchronously or asynchronously provides the wide visual angle video stream or the audio and video stream and the common visual angle video stream or the audio and video stream to a playing device.
14. A live event broadcasting method, comprising:
generating an expanded wide-view video stream or audio/video stream by at least one event wide-view live broadcast device;
transmitting the expanded wide-view video stream or audio/video stream to the streaming media server;
generating a common visual angle video stream or an audio/video stream by at least one common live broadcast device;
transmitting the common visual angle video stream or the audio and video stream to the streaming media server;
the streaming media server transmits the wide-view video stream or the audio/video stream and the common-view video stream or the audio/video stream to a live broadcast server in parallel; and
and the live broadcast server provides the wide visual angle video stream or the audio and video stream and the common visual angle video stream or the audio and video stream to a playing device according to the selected operation of the playing device.
15. A computer readable storage medium having stored thereon sequences of computer instructions which, when executed, implement the method of claim 13 or 14.

Claims (10)

1.一种赛事直播系统,其特征在于:包括1. a live event system, characterized in that: comprising 至少一个宽视角直播设备,用于产生视频流和/或音视频流;At least one wide-angle live broadcast device for generating video streams and/or audio and video streams; 流媒体服务器,用于接收所述宽视角直播设备传送的视频流或音视频流;A streaming media server, configured to receive a video stream or an audio and video stream transmitted by the wide-view live broadcast device; 至少一个普通视角直播设备,所述普通视角直播设备产生普通视角视频流,并将所述普通视角视频流传送至所述流媒体服务器;以及at least one normal-view live broadcast device that generates a normal-view video stream and transmits the normal-view video stream to the streaming media server; and 直播服务器,用于通过直播网络将流媒体服务器接收的视频流或音视频流分配至播放设备;The live server is used to distribute the video stream or audio and video stream received by the streaming media server to the playback device through the live network; 其中,所述宽视角直播设备包括Wherein, the wide viewing angle live broadcast device includes 宽视角摄像单元,用于获取宽视角图像;A wide-angle camera unit for acquiring wide-angle images; 存储器,用于存储计算机指令序列;memory for storing sequences of computer instructions; 处理器,执行所述计算机指令序列以实现:a processor executing the sequence of computer instructions to achieve: 获得所述摄像单元的内参数矩阵和畸变系数矩阵;obtaining an internal parameter matrix and a distortion coefficient matrix of the camera unit; 对宽视角图像前置处理;Preprocessing of wide-angle images; 基于所述内参数矩阵和畸变系数矩阵对经前置处理的图像进行展开处理生成展开的宽视角图像;Expanding the pre-processed image based on the internal parameter matrix and the distortion coefficient matrix to generate an expanded wide-view image; 基于所述展开的宽视角图像形成展开的宽视角视频流;forming an expanded wide-view video stream based on the expanded wide-view image; 将视频流传送至流媒体服务器;Stream the video to a streaming server; 其中,所述流媒体服务器将自所述普通视角直播设备接收的所述普通视角视频流和自所述宽视角直播设备接收的所述展开的宽视角视频流并行传送至直播服务器;Wherein, the streaming media server transmits the normal-view video stream received from the normal-view live device and the expanded wide-view video stream received from the wide-view live device to the live server in parallel; 还包括播放设备,所述播放设备提供多角度的播放界面,所述播放界面包括用于播放所述展开的宽视角视频流的部分以及用于播放所述普通视角视频流的部分。A playback device is also included, the playback device provides a multi-angle playback interface, and the playback interface includes a portion for playing the unfolded wide-view video stream and a portion for playing the normal-view video stream. 2.根据权利要求1的赛事直播系统,其特征在于:所述宽视角直播设备将所述展开的宽视角视频流基于RTMP或RTSP协议推送至所述流媒体服务器,或者所述宽视角直播设备基于所述流媒体服务器的请求将所述展开的宽视角视频流基于RTMP/RTSP/HTTP传送至所述流媒体服务器。2. the event live system according to claim 1, is characterized in that: described wide-angle live broadcast equipment pushes described unfolded wide-angle video stream to described streaming media server based on RTMP or RTSP protocol, or described wide-angle live broadcast equipment The expanded wide-view video stream is transmitted to the streaming media server based on RTMP/RTSP/HTTP based on the request of the streaming media server. 3.根据权利要求1的赛事直播系统,其特征在于:还包括声音采集单元,用于获取所述宽视角直播设备的声场信号。3 . The event live broadcast system according to claim 1 , further comprising a sound acquisition unit for acquiring a sound field signal of the wide viewing angle live broadcast device. 4 . 4.根据权利要求3的赛事直播系统,其特征在于:所述存储器还存储有计算机指令序列,所述处理器执行所述计算机指令序列以实现4. The event live broadcast system according to claim 3, wherein the memory also stores a sequence of computer instructions, and the processor executes the sequence of computer instructions to achieve 将所述声场信号数字化成音频数据;digitizing the sound field signal into audio data; 将所述音频数据与所述展开的宽视角图像编码以封装成展开的宽视角音视频流;以及encoding the audio data and the expanded wide-view image to encapsulate into an expanded wide-view audio and video stream; and 将所述展开的宽视角音视频流传送至所述流媒体服务器。The expanded wide-view audio and video stream is transmitted to the streaming media server. 5.根据权利要求1的赛事直播系统,其特征在于:所述宽视角直播设备布置于车辆上。5 . The event live broadcast system according to claim 1 , wherein the wide viewing angle live broadcast device is arranged on a vehicle. 6 . 6.根据权利要求1的赛事直播系统,其特征在于:所述宽视角直播设备布置于赛事场地上。6 . The event live broadcast system according to claim 1 , wherein the wide viewing angle live broadcast device is arranged on the event venue. 7 . 7.根据权利要求6的赛事直播系统,其特征在于:还可以在展开的宽视角屏幕与普通视角屏幕中的一个播放视频流或音视频流,而在其他屏幕重复播放视频流一个时间循环中的片段,并在用户通过点击或触摸操作选择该循环播放的片段时,从直播平台的存储器拉取对应直播设备已经存储的视频流或音视频流或者从流媒体服务器实时地拉取流媒体服务器刚刚从直播设备刚刚获得的视频流或音视频流。7. according to the event live broadcast system of claim 6, it is characterized in that: can also play video stream or audio-video stream in the wide viewing angle screen of unfolding and common viewing angle screen, and repeat playing video stream in other screen in a time loop and when the user selects the looped segment through a click or touch operation, pulls the video stream or audio and video stream that has been stored by the corresponding live broadcast device from the storage of the live broadcast platform or pulls the streaming media server in real time from the streaming media server. The video stream or audio/video stream just obtained from the live device. 8.一种赛事直播方法,其特征在于包括:8. A method for live broadcasting of events, comprising: 由至少一个赛事宽视角直播设备生成展开的宽视角视频流或音视频流;An unfolded wide-view video stream or audio and video stream generated by at least one event wide-view live broadcast device; 将所述展开的宽视角视频流或音视频流传送至流媒体服务器;Sending the expanded wide-view video stream or audio and video stream to a streaming media server; 由至少一个普通直播设备生成普通视角视频流或音视频流;A normal viewing angle video stream or audio and video stream is generated by at least one normal live broadcast device; 将所述普通视角视频流或音视频流传送至所述流媒体服务器;transmitting the normal-view video stream or audio-video stream to the streaming media server; 所述流媒体服务器将所述宽视角视频流或音视频流以及所述普通视角视频流或音视频流并行传送至直播服务器;The streaming media server transmits the wide-view video stream or audio-video stream and the normal-view video stream or audio-video stream to the live server in parallel; 所述直播服务器将所述宽视角视频流或音视频流以及所述普通视角视频流或音视频流同步地或异步地提供至播放设备;以及The live server provides the wide-view video stream or audio-video stream and the normal-view video stream or audio-video stream to the playback device synchronously or asynchronously; and 在所述播放设备提供的包括用于播放所述展开的宽视角视频流的部分以及用于播放所述普通视角视频流的部分的播放界面分别播放所述展开的宽视角视频流以及所述普通视角视频流的步骤。The expanded wide-view video stream and the normal Steps for perspective video streaming. 9.一种赛事直播方法,其特征在于包括:9. A method for live broadcasting of events, comprising: 由至少一个赛事宽视角直播设备生成展开的宽视角视频流或音视频流;An unfolded wide-view video stream or audio and video stream generated by at least one event wide-view live broadcast device; 将所述展开的宽视角视频流或音视频流传送至流媒体服务器;Sending the expanded wide-view video stream or audio and video stream to a streaming media server; 由至少一个普通直播设备生成普通视角视频流或音视频流;A normal viewing angle video stream or audio and video stream is generated by at least one normal live broadcast device; 将所述普通视角视频流或音视频流传送至所述流媒体服务器;transmitting the normal-view video stream or audio-video stream to the streaming media server; 所述流媒体服务器将所述宽视角视频流或音视频流以及所述普通视角视频流或音视频流并行传送至直播服务器;The streaming media server transmits the wide-view video stream or audio-video stream and the normal-view video stream or audio-video stream to the live server in parallel; 所述直播服务器将所述宽视角视频流或音视频流以及所述普通视角视频流或音视频流根据播放设备的选定操作提供至播放设备;以及The live server provides the wide-view video stream or audio-video stream and the normal-view video stream or audio-video stream to the playback device according to the selected operation of the playback device; and 在所述播放设备提供的包括用于播放所述展开的宽视角视频流的部分以及用于播放所述普通视角视频流的部分的播放界面分别播放所述展开的宽视角视频流以及所述普通视角视频流的步骤。The expanded wide-view video stream and the normal Steps for perspective video streaming. 10.一种计算机可读存储介质,其上存储有计算机指令序列,当执行所述计算机指令序列时,实现所述权利要求8或9中的方法。10. A computer-readable storage medium having a sequence of computer instructions stored thereon which, when executed, implements the method of claim 8 or 9.
CN201710431777.7A 2017-06-09 2017-06-09 Event wide-view live broadcasting equipment and associated live broadcasting system and method Active CN107835434B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710431777.7A CN107835434B (en) 2017-06-09 2017-06-09 Event wide-view live broadcasting equipment and associated live broadcasting system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710431777.7A CN107835434B (en) 2017-06-09 2017-06-09 Event wide-view live broadcasting equipment and associated live broadcasting system and method

Publications (2)

Publication Number Publication Date
CN107835434A CN107835434A (en) 2018-03-23
CN107835434B true CN107835434B (en) 2021-09-14

Family

ID=61642980

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710431777.7A Active CN107835434B (en) 2017-06-09 2017-06-09 Event wide-view live broadcasting equipment and associated live broadcasting system and method

Country Status (1)

Country Link
CN (1) CN107835434B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103369192A (en) * 2012-03-31 2013-10-23 深圳市振华微电子有限公司 Method and device for Full-hardware splicing of multichannel video images
CN104602129A (en) * 2015-01-27 2015-05-06 三星电子(中国)研发中心 Playing method and system of interactive multi-view video
CN105376530A (en) * 2015-10-27 2016-03-02 南京泓众电子科技有限公司 Panorama video image generation device based on fixed position
CN105744137A (en) * 2016-04-22 2016-07-06 王俊懿 System for real-time correction and splicing of image formed by fish-eye lens through using hardware circuit
CN106101503A (en) * 2016-07-18 2016-11-09 优势拓展(北京)科技有限公司 Real time panoramic Living Network video camera and system and method
CN106570938A (en) * 2016-10-21 2017-04-19 哈尔滨工业大学深圳研究生院 OPENGL based panoramic monitoring method and system

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2682877C (en) * 2007-06-12 2012-03-13 In Extenso Holdings, Inc. Distributed synchronized video viewing and editing
US9462301B2 (en) * 2013-03-15 2016-10-04 Google Inc. Generating videos with multiple viewpoints
GB2517730A (en) * 2013-08-29 2015-03-04 Mediaproduccion S L A method and system for producing a video production
CN104539896B (en) * 2014-12-25 2018-07-10 桂林远望智能通信科技有限公司 The intelligent monitor system and method for a kind of overall view monitoring and hot spot feature
CN105872570A (en) * 2015-12-11 2016-08-17 乐视网信息技术(北京)股份有限公司 Method and apparatus for implementing multi-camera video synchronous playing
CN106341711B (en) * 2016-09-27 2019-09-24 成都西可科技有限公司 A kind of multimachine digital video direct playing and playback method and system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103369192A (en) * 2012-03-31 2013-10-23 深圳市振华微电子有限公司 Method and device for Full-hardware splicing of multichannel video images
CN104602129A (en) * 2015-01-27 2015-05-06 三星电子(中国)研发中心 Playing method and system of interactive multi-view video
CN105376530A (en) * 2015-10-27 2016-03-02 南京泓众电子科技有限公司 Panorama video image generation device based on fixed position
CN105744137A (en) * 2016-04-22 2016-07-06 王俊懿 System for real-time correction and splicing of image formed by fish-eye lens through using hardware circuit
CN106101503A (en) * 2016-07-18 2016-11-09 优势拓展(北京)科技有限公司 Real time panoramic Living Network video camera and system and method
CN106570938A (en) * 2016-10-21 2017-04-19 哈尔滨工业大学深圳研究生院 OPENGL based panoramic monitoring method and system

Also Published As

Publication number Publication date
CN107835434A (en) 2018-03-23

Similar Documents

Publication Publication Date Title
CN107615338B (en) Methods and apparatus for generating and using reduced resolution images and/or delivering such images to playback or content distribution devices
JP7045856B2 (en) Video transmission based on independent coded background update
JP5777185B1 (en) All-round video distribution system, all-round video distribution method, communication terminal device, and control method and control program thereof
WO2015174501A1 (en) 360-degree video-distributing system, 360-degree video distribution method, image-processing device, and communications terminal device, as well as control method therefor and control program therefor
CN107835435B (en) Event wide-view live broadcasting equipment and associated live broadcasting system and method
US11282169B2 (en) Method and apparatus for processing and distributing live virtual reality content
KR20190008901A (en) Method, device, and computer program product for improving streaming of virtual reality media content
CN112235585B (en) Live broadcasting method, device and system for virtual scene
US11706375B2 (en) Apparatus and system for virtual camera configuration and selection
CN106231317A (en) Video processing, coding/decoding method and device, VR terminal, audio/video player system
JP6002191B2 (en) All-round video distribution system, all-round video distribution method, communication terminal device, and control method and control program thereof
CN110149542A (en) transmission control method
JP2015173424A (en) Video distribution system and video display device
WO2007061068A1 (en) Receiver and line video distributing device
US12211140B2 (en) Information processing apparatus, information processing method, and storage medium
CN111669567A (en) Multi-angle free visual angle video data generation method and device, medium and server
CN111669561A (en) Multi-angle free visual angle image data processing method and device, medium and equipment
JP5520146B2 (en) Video receiving apparatus and control method thereof
CN109756683B (en) Panoramic audio and video recording method and device, storage medium and computer equipment
CN107835433B (en) Event wide-view live broadcasting system, associated equipment and live broadcasting method
CN111669570A (en) Multi-angle free visual angle video data processing method and device, medium and equipment
CN111669568A (en) Multi-angle free visual angle interaction method and device, medium, terminal and equipment
CN107835434B (en) Event wide-view live broadcasting equipment and associated live broadcasting system and method
CN109862385B (en) Live broadcast method and device, computer readable storage medium and terminal equipment
JP2012004991A (en) Broadcast receiving apparatus and control method for the same

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20211130

Address after: Room 501, gate 9, 9 / F, zhixueyuan, No. 29, Xierqi Road West, Haidian District, Beijing 100085

Patentee after: Zhi Xiaomu

Address before: Room 1513, building 3, No. 3, Xijing Road, Badachu high tech park, Shijingshan District, Beijing 100041

Patentee before: FBLIFE (BEIJING) MEDIA TECHNOLOGY CO.,LTD.

TR01 Transfer of patent right

Effective date of registration: 20250925

Address after: 100027 Beijing City Chaoyang District Shuaguang Xili Jia 1 No. 15 Floor B-1806

Patentee after: Beijing Heyu Lanqiao Technology Management Center (Limited Partnership)

Country or region after: China

Address before: Room 501, gate 9, 9 / F, zhixueyuan, No. 29, Xierqi Road West, Haidian District, Beijing 100085

Patentee before: Zhi Xiaomu

Country or region before: China